| Commit message (Expand) | Author | Age | Files | Lines |
|---|
| * | ci/eval: use sane defaults•••Using these defaults makes it possible to just run `nix-build ci -A
eval.singleSystem` without passing additional arguments and get a sane
result back. Especially helpful when testing or debugging.
A `chunkSize` of 5000 is conservative to be able to run on systems with
less memory as well. Run-time is not impacted by that, as recent
benchmarks show.
| Wolfgang Walther | 2025-08-09 | 1 | -2/+2 |
| * | ci/eval: reduce closure size•••`procps` pulls in 180 MB of systemd, but busybox also provides `kill`.
`busybox` also ships `time`, so no need for that extra dependency.
Using `nativeBuildInputs` pulls in all the -dev outputs of the listed
packages - which we don't need. We only need to run these tools, thus
map to their bin outputs.
Brings down the closure size from 500+ MB to 193 MB for the Eval job.
This probably saves ~10 seconds for the job.
| Wolfgang Walther | 2025-08-09 | 2 | -11/+13 |
| * | ci/eval: return min memory in megabyte•••No need to return bytes in these files. Also busybox has problems to
render `free -b` with sizes > 100 GB properly in the next commit,
leading to extraction errors with awk.
| Wolfgang Walther | 2025-08-07 | 1 | -3/+3 |
| * | ci/eval: fix min-free-swap report•••This was checking the wrong condition, likely from a copy&pasto.
| Wolfgang Walther | 2025-08-07 | 1 | -1/+1 |
| * | ci/eval/compare: don't treat renames as rebuilds•••When a package's attrpath is renamed it is currently treated as a
rebuild, even though the outpath already exists and is already cached.
This also happens when adding new names for packagesets that already
exist, for example when starting to eval `perlPackages` in CI, which is
just the same as `perl540Packages` currently. It would also happen when
`perlPackages` is switched from `perl540Packages` to `perl999Packages`.
Assuming that `perl999Packages` had already been built before, this
doesn't really cause any rebuilds.
| Wolfgang Walther | 2025-08-06 | 3 | -6/+23 |
| * | top-level/release-attrpaths-superset: remove __attrsFailEvaluation (#428015) | Wolfgang Walther | 2025-07-24 | 1 | -1/+1 |
| |\ |
|
| | * | top-level/release-outpaths-parallel: rename from top-level/release-attrpaths-...•••This file takes attrpaths as input and returns chunks of derivations,
aka a subset of release-outpaths - while release-attrpaths-superset
returns only an attrpath, not any derivations.
Thus, as a specialization of release-outpaths, the name
release-outpaths-parallel matches much better.
| Wolfgang Walther | 2025-07-23 | 1 | -1/+1 |
| * | | ci/eval: fail on asserts when generating attrpaths•••This doesn't fail on *all* asserts, yet, because nix-env still ignores
these in the main eval step. But it already gives some converage during
the attrpath generation.
| Wolfgang Walther | 2025-07-24 | 1 | -2/+1 |
| |/ |
|
| * | ci/eval/README.md: one sentence per line | Wolfgang Walther | 2025-06-29 | 1 | -4/+9 |
| * | ci/eval: fix comparing to null in getLabels•••Check if `to` is null _before_ comparing it to `rebuildCount`.
| Matt Sturgeon | 2025-06-22 | 1 | -1/+1 |
| * | workflows/labels: manage labels with a single API call•••Instead of deleting each label separately and then making another call
to add new labels, this replaces all labels at once, thus saving API
calls in some cases. Also, the labels are now managed in object-style
compared to the array-style before. This allows putting all the
knowledge about each label into a single place instead of in multiple
places. For example, the rebuild labels had to be special cased in the
workflow before - and the nix code to compare had to match that. Also,
the approval labels had to be considered in the `before` and `after`
phases.
The next commit shows how easy it is to add a new label now.
| Wolfgang Walther | 2025-06-22 | 2 | -59/+44 |
| * | ci/eval/compare: fix rebuild-stdenv labels•••Those have not been working since before the migration from OfBorg.
Those `rebuildsByKernel` are an attrset of lists coming from
`groupByKernel` (also see lengthy comment at the top of the file) - thus
we need `lib.elem` instead.
| Wolfgang Walther | 2025-06-21 | 1 | -1/+1 |
| * | ci/eval: accept `nix` directly•••Previously we were taking nixVersions and this made external use from
the Lix repo's CI annoying.
We should probably also test other nix versions than stable (i.e. also
latest and Lix), but this involves writing GitHub Actions about it and
maybe not running it on every single PR. Future work.
| Jade Lovelace | 2025-06-10 | 1 | -5/+11 |
| * | workflows/eval: add header for packages in summary•••After the stats were added with a header, the list of packages now
appeared directly below it, without any separation.
| Wolfgang Walther | 2025-05-31 | 1 | -1/+6 |
| * | ci/eval: don't evaluate packages marked as broken (#409867) | Wolfgang Walther | 2025-05-28 | 1 | -1/+3 |
| |\ |
|
| | * | ci/eval: don't evaluate packages marked as broken•••We really can't expect packages that are marked as broken to evaluate,
and *especially* not on unsupported platforms.
For context, we were attempting to eval them *past* the broken throw
previously, which caused fun side effects like [0].
When we set `includeBroken = true` before, this also included unfree
packages. Those would now be excluded, which is not what we want. Thus,
we explicitly enable them separately.
Commit by winterqt, message slightly reworded by wolfgangwalther.
[0]:
https://github.com/NixOS/nixpkgs/issues/355847#issuecomment-2878873137
| Winter | 2025-05-22 | 1 | -1/+3 |
| * | | workflows/eval: diff outpaths immediately•••This moves the diff of outpaths into the outpaths job, mainly as a
preparation to allow future improvements. For example, this will allow
running the purity release checks only on changed outpaths instead of
the whole eval.
This also removes the inefficiency introduced in the last commit about
uploading the intermediate paths twice. Now, only the diff is passed on.
Also, technically, the diff is now run in parallel across 4 jobs. This
should be *slightly* faster than before, where outpaths from all systems
were combined first and then diffed. It's probably only a few seconds,
though.
| Wolfgang Walther | 2025-05-25 | 4 | -54/+96 |
| * | | workflows/eval: consistently avoid "result" in arguments•••Everything is a result, especially when nix-build uses "result" as its
default output. This becomes confusing, when re-wiring the different
parts later.
Thus, consistently name those things after some of their properties and
avoid the term result.
| Wolfgang Walther | 2025-05-25 | 2 | -14/+14 |
| * | | ci/eval.combine: avoid parsing of plain text file into JSON•••Instead of parsing a plain text file with jq, we can make nix-env output
JSON directly, which is significantly faster.
This saves about 8 out of 10 seconds for the combine step.
| Wolfgang Walther | 2025-05-24 | 1 | -18/+4 |
| * | | ci/eval.compare: use lib from pinned nixpkgs•••compare/maintainers.nix needs to access the current checkout to check
attrpaths, but makes the mistake of using lib from that checkout as
well. All other code in ci/ uses the pinned nixpkgs instance, so
maintainers.nix should do so as well.
| Wolfgang Walther | 2025-05-24 | 3 | -13/+7 |
| * | | ci/eval: improve api when calling in steps•••Previously, `eval.full` organized the results for the supported systems
in a specific layout, i.e. with a folder with one subfolder per system.
Then, `eval.combine` relied on that.
When using `eval.singleSystem` and `eval.combine` directly, the caller
was responsible to recreate the same layout. This is annoying and
error-prone to do, when downloading artifacts from CI to recreate some
steps locally.
With this change, all the artifacts can be downloaded and extracted into
the same folder - because the result from `eval.singleSystem` already
contains the <system-name>/ subfolder.
| Wolfgang Walther | 2025-05-24 | 1 | -16/+17 |
| * | | ci/eval: remove left-over stats.json•••This seems to be a left-over from before the performance comparison was
changed to a difference-per-chunk analysis.
| Wolfgang Walther | 2025-05-18 | 1 | -38/+0 |
| * | | ci/eval/nix: 2.24 -> latest (#404589) | Wolfgang Walther | 2025-05-17 | 1 | -1/+1 |
| |\ \ |
|
| | * | | ci/nix: 2.24 -> 2.28 | Robert Hensing | 2025-05-17 | 1 | -1/+1 |
| | |/ |
|
| * | | ci/eval/compare: manage the "by: package-maintainer" label (#407465) | Wolfgang Walther | 2025-05-17 | 1 | -1/+10 |
| |\ \
| |/
|/| |
|
| | * | ci/eval/compare: manage the "by: package-maintainer" label•••While OfBorg is still adding these, it takes a much longer time to do so
compared to the eval action. Since we're adding rebuild labels, I think
it'd be nice to just do it within the eval action.
| Winter | 2025-05-17 | 1 | -1/+10 |
| * | | ci/eval: allow configuration of the system to eval attrpaths on•••Right now, there are some paths that don't even get exposed to certain
systems (notably Darwin, but some outliers exist for Linux such as the
Darwin-specific Hackage overlay) for one reason or another, usually
because of assertions like `stdenv.isLinux`. To catch these scenarios,
this change implements a way to specify the system to evaluate attrpaths
on, and makes it default to the system that we're evaluating outpaths
for.
| Winter | 2025-05-16 | 1 | -1/+5 |
| |/ |
|
| * | workflows/eval: remove attrs step•••Previously, the attrs step consisted of:
- 7s queue time
- 1m 15s run time
Only 25s of this were spent preparing the attr paths. A bit more than a
minute was just spent for queuing, checking out the repo, downloading
nix, downloading dependencies, uploading the artifacts - and then
downloading them again in the next step. All of that can be avoided if
we collect the attrs as part of the outpaths job.
By running the attrs step as part of each outpaths step the attrpaths
will be collected 4x, but:
- We save a minute for each eval run to complete.
- We save a full job, giving us more free runners and *possibly* less
queue times for other jobs in the repo.
- We reduce complexity in the workflow file.
| Wolfgang Walther | 2025-05-11 | 1 | -2/+1 |
| * | workflows/eval: load supportedSystems from JSON file•••This is a refactor to prepare the next commit. It doesn't do much on its
own, but is separated for ease of review.
| Wolfgang Walther | 2025-05-11 | 2 | -6/+3 |
| * | check-meta: wrap maintainers attribute to include team members (#402991) | Winter | 2025-05-07 | 1 | -4/+3 |
| |\ |
|
| | * | ci/eval: use correct maintainer and team positions•••We need to pass through the maintainers and teams positions from the
original meta so pings work correctly, since check-meta clobbers the
original attribute positions in them.
Tested with `maintainers/scripts/get-maintainer-pings-between.sh` on a
handful of major packages maintained by both individuals and teams.
| Morgan Jones | 2025-05-05 | 1 | -4/+3 |
| * | | ci/eval-stats: sort output table by metric name | Johannes Kirschbauer | 2025-05-07 | 1 | -0/+1 |
| * | | ci/compare: nix stats comparison•••Displays stats table in the step-summary if there are no added/removed packages
| Johannes Kirschbauer | 2025-05-06 | 3 | -4/+201 |
| |/ |
|
| * | Revert "ci/compare: Bring back nix stats comparison" | Johannes Kirschbauer | 2025-05-01 | 3 | -189/+4 |
| * | ci/compare: nix stats comparison•••Displays stats table in the step-summary if there are no added/removed packages
| Johannes Kirschbauer | 2025-05-01 | 3 | -4/+189 |
| * | ci/eval: output per chunk stats | Johannes Kirschbauer | 2025-05-01 | 1 | -0/+8 |
| * | ci/eval/compare: support optional byName argument•••Sometimes it is quite useful to output names instead of GitHub IDs, e.g.
for maintainer scripts that show you who you would ping. Add this as an
option, but keep the existing default.
| Morgan Jones | 2025-04-17 | 2 | -2/+9 |
| * | ci/eval/compare: add meta.teams | Tristan Ross | 2025-04-17 | 1 | -1/+12 |
| * | ci/eval/compare: fix reading store paths from json file•••This suddenly appeared after updating Nix to v26, which then complained:
… while calling the 'fromJSON' builtin
at
/home/runner/work/nixpkgs/nixpkgs/target/ci/eval/compare/default.nix:74:19:
73|
74| getAttrs = dir: builtins.fromJSON (builtins.readFile
"${dir}/outpaths.json");
| ^
75| beforeAttrs = getAttrs beforeResultDir;
… while evaluating the first argument passed to builtins.fromJSON
error: the string '{
"AMB-plugins.aarch64-linux": {
"out":
"/nix/store/faw59ba5p6h4b177n8q2ilb3hlm7xlc2-AMB-plugins-0.8.1"
},
....
"zzuf.aarch64-linux": {
"out": "/nix/store/bqvm1h7jfd8smgnjc1v1gpmbwdgvwy5g-zzuf-0.15"
},
"zzuf.x86_64-linux": {
"out": "/nix/store/6qs4lnmzn1qlr3smqqxnmhnrcdcfiv6a-zzuf-0.15"
}
}
' is not allowed to refer to a store path (such as
'134m2q047vsr9miwh5l227j7sh9jb130-jq-1.7.1-bin')
By discard the unsafe string context, we explicitly allow loading those
store paths. It's unclear why this blew up now, especially because I was
not possible to consistently replicate this locally, so far.
| Wolfgang Walther | 2025-03-19 | 1 | -1/+9 |
| * | ci/eval: fail on non-empty stderr (#381266) | Wolfgang Walther | 2025-02-14 | 1 | -3/+10 |
| |\ |
|
| | * | ci/eval: Fail on non-empty stderr•••Just like the channel eval requires.
| Silvan Mosberger | 2025-02-11 | 1 | -0/+3 |
| | * | ci/eval: Refactor to cleanly separate stderr•••Previously stderr was mixed with the time stats
This allows checking stderr in the next commit
| Silvan Mosberger | 2025-02-11 | 1 | -3/+7 |
| * | | ci/eval: make eval for non-native platforms less incorrect (#378922) | Wolfgang Walther | 2025-02-14 | 1 | -0/+1 |
| |\ \ |
|
| | * | | ci/eval: make eval for non-native platforms less incorrect•••We commonly use platform-dependent conditional patterns like
`lib.meta.availableOn stdenv.hostPlatform` and `stdenv.hostPlatform.isLinux`
to enable different features in a given derivation or to evaluate
completely different derivations based on the platform.
For example, source builds of a given derivation may only be available
on linux but not on darwin. The use of such conditionals allow us to
fall back to patched binaries on darwin instead.
In `chromedriver` (pkgs/development/tools/selenium/chromedriver/default.nix), we use
~~~nix
if lib.meta.availableOn stdenv.hostPlatform chromium then
callPackage ./source.nix { }
else
callPackage ./binary.nix { }
~~~
To provide some context, `chromedriver` source builds are based on `chromium.mkDerivation`
and `chromium` is limited to `lib.platforms.linux`.
Based on the same `chromium.mkDerivation`, we also do source builds for
`electron` (pkgs/top-level/all-packages.nix):
~~~nix
electron_33 = if lib.meta.availableOn stdenv.hostPlatform electron-source.electron_33 then electron-source.electron_33 else electron_33-bin;
electron_34 = electron_34-bin;
electron = electron_34;
~~~
And finally, the top-level `jdk` (Java) attribute has a lot of
indirection, but eventually also boils down to `stdenv.hostPlatform.isLinux`
for source builds and binaries for x86_64-darwin and aarch64-darwin.
A surprising amount of electron and jdk consumers use variations of
`meta.platforms = electron.meta.platforms` in their own meta block.
Due to internal implementation details, the conditionals in those
top-level attributes like `chromedriver`, `electron` and `jdk` are
evaluated based on the value from `builtins.currentSystem` and not the
system passed to `import <nixpkgs> { }`.
This then causes `chromedriver`, `electron`, `jdk` and all dependents
that inherit those `meta.platforms` to appear only available on linux
despite also being available on darwin. Hydra is affected similarly, but
it's a lot more nuanced and in practice not actually *that* bad.
The addition of `--eval-system` ensures that `builtins.currentSystem`
matches the requested platform.
As a bonus, this also fixes the store paths of an impure test that
should probably be made pure:
~~~diff
@@ -885069,13 +886119,13 @@
"out": "/nix/store/lb2500hc69czy4sfga9mbh2k679cr1rp-test-compressDrv"
},
"tests.config.allowPkgsInPermittedInsecurePackages.aarch64-darwin": {
- "out": "/nix/store/0l5h8svrpzwymq35mnpvx82gyc7nf8s4-hello-2.12.1"
+ "out": "/nix/store/v1zjb688mp4y2132b6chii43d5kkxnpa-hello-2.12.1"
},
"tests.config.allowPkgsInPermittedInsecurePackages.aarch64-linux": {
- "out": "/nix/store/0l5h8svrpzwymq35mnpvx82gyc7nf8s4-hello-2.12.1"
+ "out": "/nix/store/hb21z2zdk03dwygsw5lvpa8zc3fbr500-hello-2.12.1"
},
"tests.config.allowPkgsInPermittedInsecurePackages.x86_64-darwin": {
- "out": "/nix/store/0l5h8svrpzwymq35mnpvx82gyc7nf8s4-hello-2.12.1"
+ "out": "/nix/store/gljdqsf0mxv1j8zb04phx9ws09pp7z3l-hello-2.12.1"
},
"tests.config.allowPkgsInPermittedInsecurePackages.x86_64-linux": {
"out": "/nix/store/0l5h8svrpzwymq35mnpvx82gyc7nf8s4-hello-2.12.1"
~~~
Diff stats between two full evals based on 75c8548d8118c31509b89ffd7df6c322b94017dd
with and without this fix on x86_64-linux:
~~~bash
# git diff --no-index --stat /nix/store/659l3xp78255wx7abbahggsnrlj3a1la-combined-result/outpaths.json /nix/store/4fhlq4g5qa65cxbibskq9pma40zigrx7-combined-result/outpaths.json
/nix/store/{659l3xp78255wx7abbahggsnrlj3a1la-combined-result => 4fhlq4g5qa65cxbibskq9pma40zigrx7-combined-result}/outpaths.json | 1416 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-
1 file changed, 1405 insertions(+), 11 deletions(-)
~~~
The full diff is available as a gist at <https://gist.github.com/emilylange/d40c50031fc332bbcca133ad56d224f6>.
When we added `electron_34` only as binary instead of the usual source
on linux with binary fallback in cfed9a19cbc98d12c0167f69238673f4ed89f798
and made the unversioned `electron` top-level point to the newly added
`electron_34` instead of `electron_33`, the GitHub workflow suddenly
reported 20 new packages. Of those 20 reported packages, 17 where
false-positives caused by dropping the wrongly evaluated conditional.
| emilylange | 2025-02-02 | 1 | -0/+1 |
| | |/ |
|
| * / | ci/eval/compare: Ignore null packages•••CI can fail to evaluate if a package is null:
https://github.com/NixOS/nixpkgs/actions/runs/13209876145/job/36881335314?pr=380228
| Silvan Mosberger | 2025-02-11 | 1 | -2/+5 |
| |/ |
|
| * | ci/eval/compare: Improve performance and avoid large stacks•••Various improvements such as:
1. Avoiding deduplications when there can't be any duplicates
2. Avoiding O(n^2) deduplications
3. Using builtins.any to avoid list allocations
4. Using builtins.concatMap instead of lib.flatten when it's known that there's only one level of nesting
5. Using builtins.groupBy instead of folding with an accumulator
In particular 5. should fix CI exceeding the stack size on staging: https://github.com/NixOS/nixpkgs/actions/runs/12989244871/job/36240781244?pr=377253
While 2. in particular should make CI a lot faster.
| Silvan Mosberger | 2025-01-28 | 2 | -32/+16 |
| * | maintainers.nix: Remove unused code | Silvan Mosberger | 2025-01-27 | 1 | -14/+0 |
| * | ci/eval: restore `501+` label | zowoq | 2025-01-22 | 1 | -4/+16 |
| * | ci/eval: support "10.rebuild-${kernel}: 1" labels•••This should restore the old behavior of ofborg
| Peder Bergebakken Sundt | 2025-01-20 | 1 | -25/+34 |
| * | workflows/eval: Request reviews from changed package maintainers (#366046) | Masum Reza | 2025-01-02 | 3 | -22/+163 |
| |\ |
|
