summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorWolfgang Walther <walther@technowledgy.de>2025-06-29 19:53:09 +0000
committerGitHub <noreply@github.com>2025-06-29 19:53:09 +0000
commitdc92673fc043e800b76d12a80bfc0c27cfb7685c (patch)
tree38476fe5a68760f335845c836dc04058bff99657
parent[Backport release-24.11] nexus: mark as insecure and remove inactive maintain... (diff)
parent.github/workflows/README.md: one sentence per line (diff)
downloadnixpkgs-dc92673fc043e800b76d12a80bfc0c27cfb7685c.tar.gz
[Backport release-24.11] ci/README.md: one sentence per line (#421097)
-rw-r--r--.github/workflows/README.md47
-rw-r--r--ci/README.md9
-rw-r--r--ci/eval/README.md13
3 files changed, 48 insertions, 21 deletions
diff --git a/.github/workflows/README.md b/.github/workflows/README.md
index e3be33ae6456..c2a16827ebd4 100644
--- a/.github/workflows/README.md
+++ b/.github/workflows/README.md
@@ -2,21 +2,35 @@
Some architectural notes about key decisions and concepts in our workflows:
-- Instead of `pull_request` we use [`pull_request_target`](https://docs.github.com/actions/writing-workflows/choosing-when-your-workflow-runs/events-that-trigger-workflows#pull_request_target) for all PR-related workflows. This has the advantage that those workflows will run without prior approval for external contributors.
+- Instead of `pull_request` we use [`pull_request_target`](https://docs.github.com/actions/writing-workflows/choosing-when-your-workflow-runs/events-that-trigger-workflows#pull_request_target) for all PR-related workflows.
+ This has the advantage that those workflows will run without prior approval for external contributors.
-- Running on `pull_request_target` also optionally provides us with a GH_TOKEN with elevated privileges (write access), which we need to do things like adding labels, requesting reviewers or pushing branches. **Note about security:** We need to be careful to limit the scope of elevated privileges as much as possible. Thus they should be lowered to the minimum with `permissions: {}` in every workflow by default.
+- Running on `pull_request_target` also optionally provides us with a GH_TOKEN with elevated privileges (write access), which we need to do things like adding labels, requesting reviewers or pushing branches.
+ **Note about security:** We need to be careful to limit the scope of elevated privileges as much as possible.
+ Thus they should be lowered to the minimum with `permissions: {}` in every workflow by default.
-- By definition `pull_request_target` runs in the context of the **base** of the pull request. This means, that the workflow files to run will be taken from the base branch, not the PR, and actions/checkout will not checkout the PR, but the base branch, by default. To protect our secrets, we need to make sure to **never execute code** from the pull request and always evaluate or build nix code from the pull request with the **sandbox enabled**.
+- By definition `pull_request_target` runs in the context of the **base** of the pull request.
+ This means, that the workflow files to run will be taken from the base branch, not the PR, and actions/checkout will not checkout the PR, but the base branch, by default.
+ To protect our secrets, we need to make sure to **never execute code** from the pull request and always evaluate or build nix code from the pull request with the **sandbox enabled**.
-- To test the pull request's contents, we checkout the "test merge commit". This is a temporary commit that GitHub creates automatically as "what would happen, if this PR was merged into the base branch now?". The checkout could be done via the virtual branch `refs/pull/<pr-number>/merge`, but doing so would cause failures when this virtual branch doesn't exist (anymore). This can happen when the PR has conflicts, in which case the virtual branch is not created, or when the PR is getting merged while workflows are still running, in which case the branch won't exist anymore at the time of checkout. Thus, we use the `get-merge-commit.yml` workflow to check whether the PR is mergeable and the test merge commit exists and only then run the relevant jobs.
+- To test the pull request's contents, we checkout the "test merge commit".
+ This is a temporary commit that GitHub creates automatically as "what would happen, if this PR was merged into the base branch now?".
+ The checkout could be done via the virtual branch `refs/pull/<pr-number>/merge`, but doing so would cause failures when this virtual branch doesn't exist (anymore).
+ This can happen when the PR has conflicts, in which case the virtual branch is not created, or when the PR is getting merged while workflows are still running, in which case the branch won't exist anymore at the time of checkout.
+ Thus, we use the `get-merge-commit.yml` workflow to check whether the PR is mergeable and the test merge commit exists and only then run the relevant jobs.
-- Various workflows need to make comparisons against the base branch. In this case, we checkout the parent of the "test merge commit" for best results. Note, that this is not necessarily the same as the default commit that actions/checkout would use, which is also a commit from the base branch (see above), but might be older.
+- Various workflows need to make comparisons against the base branch.
+ In this case, we checkout the parent of the "test merge commit" for best results.
+ Note, that this is not necessarily the same as the default commit that actions/checkout would use, which is also a commit from the base branch (see above), but might be older.
## Terminology
-- **base commit**: The pull_request_target event's context commit, i.e. the base commit given by GitHub Actions. Same as `github.event.pull_request.base.sha`.
-- **head commit**: The HEAD commit in the pull request's branch. Same as `github.event.pull_request.head.sha`.
-- **merge commit**: The temporary "test merge commit" that GitHub Actions creates and updates for the pull request. Same as `refs/pull/${{ github.event.pull_request.number }}/merge`.
+- **base commit**: The pull_request_target event's context commit, i.e. the base commit given by GitHub Actions.
+ Same as `github.event.pull_request.base.sha`.
+- **head commit**: The HEAD commit in the pull request's branch.
+ Same as `github.event.pull_request.head.sha`.
+- **merge commit**: The temporary "test merge commit" that GitHub Actions creates and updates for the pull request.
+ Same as `refs/pull/${{ github.event.pull_request.number }}/merge`.
- **target commit**: The base branch's parent of the "test merge commit" to compare against.
## Concurrency Groups
@@ -25,15 +39,22 @@ We use [GitHub's Concurrency Groups](https://docs.github.com/en/actions/writing-
When two workflows are in the same group, a newer workflow cancels an older workflow.
Thus, it is important how to construct the group keys:
-- Because we want to run jobs for different events at same time, we add `github.event_name` to the key. This is the case for the `pull_request` which runs on changes to the workflow files to test the new files and the same workflow from the base branch run via `pull_request_event`.
+- Because we want to run jobs for different events at same time, we add `github.event_name` to the key.
+ This is the case for the `pull_request` which runs on changes to the workflow files to test the new files and the same workflow from the base branch run via `pull_request_event`.
-- We don't want workflows of different Pull Requests to cancel each other, so we include `github.event.pull_request.number`. The [GitHub docs](https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/control-the-concurrency-of-workflows-and-jobs#example-using-a-fallback-value) show using `github.head_ref` for this purpose, but this doesn't work well with forks: Different users could have the same head branch name in their forks and run CI for their PRs at the same time.
+- We don't want workflows of different Pull Requests to cancel each other, so we include `github.event.pull_request.number`.
+ The [GitHub docs](https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/control-the-concurrency-of-workflows-and-jobs#example-using-a-fallback-value) show using `github.head_ref` for this purpose, but this doesn't work well with forks: Different users could have the same head branch name in their forks and run CI for their PRs at the same time.
-- Sometimes, there is no `pull_request.number`. To ensure non-PR runs are never cancelled, we add a fallback of `github.run_id`. This is a unique value for each workflow run.
+- Sometimes, there is no `pull_request.number`.
+ To ensure non-PR runs are never cancelled, we add a fallback of `github.run_id`.
+ This is a unique value for each workflow run.
-- Of course, we run multiple workflows at the same time, so we add `github.workflow` to the key. Otherwise workflows would cancel each other.
+- Of course, we run multiple workflows at the same time, so we add `github.workflow` to the key.
+ Otherwise workflows would cancel each other.
-- There is a special case for reusable workflows called via `workflow_call` - they will have `github.workflow` set to their parent workflow's name. Thus, they would cancel each other. That's why we additionally hardcode the name of the workflow as well.
+- There is a special case for reusable workflows called via `workflow_call` - they will have `github.workflow` set to their parent workflow's name.
+ Thus, they would cancel each other.
+ That's why we additionally hardcode the name of the workflow as well.
This results in a key with the following semantics:
diff --git a/ci/README.md b/ci/README.md
index 7df3b1533e59..c55d0ca62d07 100644
--- a/ci/README.md
+++ b/ci/README.md
@@ -6,20 +6,21 @@ This is in contrast with [`maintainers/scripts`](../maintainers/scripts) which i
## Pinned Nixpkgs
CI may need certain packages from Nixpkgs.
-In order to ensure that the needed packages are generally available without building,
-[`pinned.json`](./pinned.json) contains a pinned Nixpkgs version tested by Hydra.
+In order to ensure that the needed packages are generally available without building, [`pinned.json`](./pinned.json) contains a pinned Nixpkgs version tested by Hydra.
Run [`update-pinned.sh`](./update-pinned.sh) to update it.
## `ci/nixpkgs-vet.sh BASE_BRANCH [REPOSITORY]`
-Runs the [`nixpkgs-vet` tool](https://github.com/NixOS/nixpkgs-vet) on the HEAD commit, closely matching what CI does. This can't do exactly the same as CI, because CI needs to rely on GitHub's server-side Git history to compute the mergeability of PRs before the check can be started.
+Runs the [`nixpkgs-vet` tool](https://github.com/NixOS/nixpkgs-vet) on the HEAD commit, closely matching what CI does.
+This can't do exactly the same as CI, because CI needs to rely on GitHub's server-side Git history to compute the mergeability of PRs before the check can be started.
In turn, when contributors are running this tool locally, we don't want to have to push commits to test them, and we can also rely on the local Git history to do the mergeability check.
Arguments:
- `BASE_BRANCH`: The base branch to use, e.g. master or release-24.05
-- `REPOSITORY`: The repository from which to fetch the base branch. Defaults to <https://github.com/NixOS/nixpkgs.git>.
+- `REPOSITORY`: The repository from which to fetch the base branch.
+ Defaults to <https://github.com/NixOS/nixpkgs.git>.
# Branch classification
diff --git a/ci/eval/README.md b/ci/eval/README.md
index 011f3dd74ed0..ec7429b7bc78 100644
--- a/ci/eval/README.md
+++ b/ci/eval/README.md
@@ -11,10 +11,15 @@ nix-build ci -A eval.full \
--arg evalSystems '["x86_64-linux" "aarch64-darwin"]'
```
-- `--max-jobs`: The maximum number of derivations to run at the same time. Only each [supported system](../supportedSystems.json) gets a separate derivation, so it doesn't make sense to set this higher than that number.
-- `--cores`: The number of cores to use for each job. Recommended to set this to the amount of cores on your system divided by `--max-jobs`.
-- `chunkSize`: The number of attributes that are evaluated simultaneously on a single core. Lowering this decreases memory usage at the cost of increased evaluation time. If this is too high, there won't be enough chunks to process them in parallel, and will also increase evaluation time.
-- `evalSystems`: The set of systems for which `nixpkgs` should be evaluated. Defaults to the four official platforms (`x86_64-linux`, `aarch64-linux`, `x86_64-darwin` and `aarch64-darwin`).
+- `--max-jobs`: The maximum number of derivations to run at the same time.
+ Only each [supported system](../supportedSystems.json) gets a separate derivation, so it doesn't make sense to set this higher than that number.
+- `--cores`: The number of cores to use for each job.
+ Recommended to set this to the amount of cores on your system divided by `--max-jobs`.
+- `chunkSize`: The number of attributes that are evaluated simultaneously on a single core.
+ Lowering this decreases memory usage at the cost of increased evaluation time.
+ If this is too high, there won't be enough chunks to process them in parallel, and will also increase evaluation time.
+- `evalSystems`: The set of systems for which `nixpkgs` should be evaluated.
+ Defaults to the four official platforms (`x86_64-linux`, `aarch64-linux`, `x86_64-darwin` and `aarch64-darwin`).
A good default is to set `chunkSize` to 10000, which leads to about 3.6GB max memory usage per core, so suitable for fully utilising machines with 4 cores and 16GB memory, 8 cores and 32GB memory or 16 cores and 64GB memory.