| Commit message (Expand) | Author | Age | Files | Lines |
|---|
| * | ci/github-script/labels: set maintainer labels from latest maintainer map (#4... | Wolfgang Walther | 2025-11-01 | 1 | -4/+1 |
| |\ |
|
| | * | ci/github-script/labels: set maintainer labels from latest maintainer map•••Instead of setting the maintainer-related labels based entirely on Eval
results, this uses the new maintainer map from the target branch. This
allows labeling PRs correctly, that had been created *before* a
contributor became a maintainer of the respective package.
| Wolfgang Walther | 2025-11-01 | 1 | -4/+1 |
| * | | ci/eval: add extraNixpkgsConfig argument | Gaetan Lepage | 2025-11-01 | 1 | -0/+4 |
| * | | ci/eval: avoid passing args around•••There is no point in passing these arguments around between all the
different parts of the eval chain - these global settings should surely
not be modified between different steps.
This also makes it simpler to add new arguments like extra nixpkgs
configuration.
| Wolfgang Walther | 2025-10-31 | 1 | -15/+11 |
| |/ |
|
| * | workflows/eval: upload maintainer list of all packages as artifact•••This creates a simple mapping of all packages to github ids of all their
maintainers. This is uploaded as an artifact and is then available for
download on the latest commit of each branch with a merge queue. This
will allow scheduled jobs to use this information for setting
maintainer-related labels, to request reviewers and to implement the
merge-bot.
The advantage over querying this information directly via Nix in each
case: The scheduled job does not need to install Nix and does not need
to checkout the target branch.
Compared to obtaining the maintainer information just for a single PR
during Eval, this will allow retroactively changing maintainers for a
package: For example, it allows to request a new maintainer as reviewer
for a PR that was created before they became maintainer, but is still
open - and similarly for maintainer labels and merge-bot rights.
None of these extensions are implemented by this PR, yet.
| Wolfgang Walther | 2025-10-31 | 1 | -0/+10 |
| * | ci/eval: disable GC warning•••This came up more often recently, and breaks Eval because it prints on
stderr.
| Wolfgang Walther | 2025-10-18 | 1 | -0/+4 |
| * | ci: have `eval.full` return the report as displayed in CI•••Update the README.md to document what gets returned.
We might in the future split these up into other attrsets but I don't see a usecase for that at the moment.
| Philip Taron | 2025-09-08 | 1 | -3/+14 |
| * | ci/eval: remove unused checkMeta argument•••This should always be set anyways.
| Wolfgang Walther | 2025-08-31 | 1 | -2/+0 |
| * | top-level/release-outpaths: move to ci/eval•••These files are tightly coupled with the code in ci/eval and not used
anywhere else. They are subject to the same backporting requirements as
the remaining CI code. They are better placed next here.
| Wolfgang Walther | 2025-08-31 | 1 | -4/+7 |
| * | ci/eval.full: allow local comparison with rebuilds•••This allows running a full comparison between two commits locally.
What was previously `eval.full` is now called `eval.all`. The new
`eval.full` takes a `baseline` argument for the comparison.
| Wolfgang Walther | 2025-08-28 | 1 | -10/+26 |
| * | ci/eval: clarify README with current defaults and memory requirements•••We had set a default of 5000 for local evaluation earlier for
`singleSystem`, it makes sense to also use that for `full`.
The README is also a bit outdated, because Nix 2.30 significantly
changed the memory requirements. Rewriting the README to also show the
ability to directly evaluate the current system only.
| Wolfgang Walther | 2025-08-28 | 1 | -1/+1 |
| * | workflows/eval: pass outpaths via cachix instead of artifacts•••Instead of uploading the outpaths as artifact, this uploads them via
cachix. Most of all, this makes CI less brittle, because Eval in PRs
will still be able to succeed, even if no workflow run for the push
event could be found on the target branch. It will just take longer.
This also makes moving Eval into the Merge Queue easier to do: When
downloading artifacts from a different run, these would always have to
match on the right event, too. By pulling from cachix, the same workflow
can support target branches with merge queue and without merge queue at
the same time. The latter would still use the push event, while the
former could use the merge_group event.
Last but not least, this should fix Eval on PRs targeting `wip-`
branches and any other branches that the push event doesn't trigger on.
These would never find an Eval result from the target branch and could
never show rebuilds accurately. Now these PRs should work at a slightly
higher runtime cost.
| Wolfgang Walther | 2025-08-22 | 1 | -0/+2 |
| * | Factor out the NixOS `meta.maintainers` module (#431450) | Robert Hensing | 2025-08-21 | 1 | -0/+1 |
| |\ |
|
| | * | nixos/documentation: Allow the inclusion of a nixpkgs/modules directory | Robert Hensing | 2025-08-06 | 1 | -0/+1 |
| * | | ci/eval: use sane defaults•••Using these defaults makes it possible to just run `nix-build ci -A
eval.singleSystem` without passing additional arguments and get a sane
result back. Especially helpful when testing or debugging.
A `chunkSize` of 5000 is conservative to be able to run on systems with
less memory as well. Run-time is not impacted by that, as recent
benchmarks show.
| Wolfgang Walther | 2025-08-09 | 1 | -2/+2 |
| * | | ci/eval: reduce closure size•••`procps` pulls in 180 MB of systemd, but busybox also provides `kill`.
`busybox` also ships `time`, so no need for that extra dependency.
Using `nativeBuildInputs` pulls in all the -dev outputs of the listed
packages - which we don't need. We only need to run these tools, thus
map to their bin outputs.
Brings down the closure size from 500+ MB to 193 MB for the Eval job.
This probably saves ~10 seconds for the job.
| Wolfgang Walther | 2025-08-09 | 1 | -10/+11 |
| * | | ci/eval: return min memory in megabyte•••No need to return bytes in these files. Also busybox has problems to
render `free -b` with sizes > 100 GB properly in the next commit,
leading to extraction errors with awk.
| Wolfgang Walther | 2025-08-07 | 1 | -3/+3 |
| * | | ci/eval: fix min-free-swap report•••This was checking the wrong condition, likely from a copy&pasto.
| Wolfgang Walther | 2025-08-07 | 1 | -1/+1 |
| * | | ci/eval/compare: don't treat renames as rebuilds•••When a package's attrpath is renamed it is currently treated as a
rebuild, even though the outpath already exists and is already cached.
This also happens when adding new names for packagesets that already
exist, for example when starting to eval `perlPackages` in CI, which is
just the same as `perl540Packages` currently. It would also happen when
`perlPackages` is switched from `perl540Packages` to `perl999Packages`.
Assuming that `perl999Packages` had already been built before, this
doesn't really cause any rebuilds.
| Wolfgang Walther | 2025-08-06 | 1 | -1/+2 |
| |/ |
|
| * | top-level/release-attrpaths-superset: remove __attrsFailEvaluation (#428015) | Wolfgang Walther | 2025-07-24 | 1 | -1/+1 |
| |\ |
|
| | * | top-level/release-outpaths-parallel: rename from top-level/release-attrpaths-...•••This file takes attrpaths as input and returns chunks of derivations,
aka a subset of release-outpaths - while release-attrpaths-superset
returns only an attrpath, not any derivations.
Thus, as a specialization of release-outpaths, the name
release-outpaths-parallel matches much better.
| Wolfgang Walther | 2025-07-23 | 1 | -1/+1 |
| * | | ci/eval: fail on asserts when generating attrpaths•••This doesn't fail on *all* asserts, yet, because nix-env still ignores
these in the main eval step. But it already gives some converage during
the attrpath generation.
| Wolfgang Walther | 2025-07-24 | 1 | -2/+1 |
| |/ |
|
| * | ci/eval: accept `nix` directly•••Previously we were taking nixVersions and this made external use from
the Lix repo's CI annoying.
We should probably also test other nix versions than stable (i.e. also
latest and Lix), but this involves writing GitHub Actions about it and
maybe not running it on every single PR. Future work.
| Jade Lovelace | 2025-06-10 | 1 | -5/+11 |
| * | ci/eval: don't evaluate packages marked as broken (#409867) | Wolfgang Walther | 2025-05-28 | 1 | -1/+3 |
| |\ |
|
| | * | ci/eval: don't evaluate packages marked as broken•••We really can't expect packages that are marked as broken to evaluate,
and *especially* not on unsupported platforms.
For context, we were attempting to eval them *past* the broken throw
previously, which caused fun side effects like [0].
When we set `includeBroken = true` before, this also included unfree
packages. Those would now be excluded, which is not what we want. Thus,
we explicitly enable them separately.
Commit by winterqt, message slightly reworded by wolfgangwalther.
[0]:
https://github.com/NixOS/nixpkgs/issues/355847#issuecomment-2878873137
| Winter | 2025-05-22 | 1 | -1/+3 |
| * | | workflows/eval: diff outpaths immediately•••This moves the diff of outpaths into the outpaths job, mainly as a
preparation to allow future improvements. For example, this will allow
running the purity release checks only on changed outpaths instead of
the whole eval.
This also removes the inefficiency introduced in the last commit about
uploading the intermediate paths twice. Now, only the diff is passed on.
Also, technically, the diff is now run in parallel across 4 jobs. This
should be *slightly* faster than before, where outpaths from all systems
were combined first and then diffed. It's probably only a few seconds,
though.
| Wolfgang Walther | 2025-05-25 | 1 | -10/+31 |
| * | | workflows/eval: consistently avoid "result" in arguments•••Everything is a result, especially when nix-build uses "result" as its
default output. This becomes confusing, when re-wiring the different
parts later.
Thus, consistently name those things after some of their properties and
avoid the term result.
| Wolfgang Walther | 2025-05-25 | 1 | -7/+7 |
| * | | ci/eval.combine: avoid parsing of plain text file into JSON•••Instead of parsing a plain text file with jq, we can make nix-env output
JSON directly, which is significantly faster.
This saves about 8 out of 10 seconds for the combine step.
| Wolfgang Walther | 2025-05-24 | 1 | -18/+4 |
| * | | ci/eval.compare: use lib from pinned nixpkgs•••compare/maintainers.nix needs to access the current checkout to check
attrpaths, but makes the mistake of using lib from that checkout as
well. All other code in ci/ uses the pinned nixpkgs instance, so
maintainers.nix should do so as well.
| Wolfgang Walther | 2025-05-24 | 1 | -10/+2 |
| * | | ci/eval: improve api when calling in steps•••Previously, `eval.full` organized the results for the supported systems
in a specific layout, i.e. with a folder with one subfolder per system.
Then, `eval.combine` relied on that.
When using `eval.singleSystem` and `eval.combine` directly, the caller
was responsible to recreate the same layout. This is annoying and
error-prone to do, when downloading artifacts from CI to recreate some
steps locally.
With this change, all the artifacts can be downloaded and extracted into
the same folder - because the result from `eval.singleSystem` already
contains the <system-name>/ subfolder.
| Wolfgang Walther | 2025-05-24 | 1 | -16/+17 |
| * | | ci/eval: remove left-over stats.json•••This seems to be a left-over from before the performance comparison was
changed to a difference-per-chunk analysis.
| Wolfgang Walther | 2025-05-18 | 1 | -38/+0 |
| * | | ci/nix: 2.24 -> 2.28 | Robert Hensing | 2025-05-17 | 1 | -1/+1 |
| |/ |
|
| * | ci/eval: allow configuration of the system to eval attrpaths on•••Right now, there are some paths that don't even get exposed to certain
systems (notably Darwin, but some outliers exist for Linux such as the
Darwin-specific Hackage overlay) for one reason or another, usually
because of assertions like `stdenv.isLinux`. To catch these scenarios,
this change implements a way to specify the system to evaluate attrpaths
on, and makes it default to the system that we're evaluating outpaths
for.
| Winter | 2025-05-16 | 1 | -1/+5 |
| * | workflows/eval: remove attrs step•••Previously, the attrs step consisted of:
- 7s queue time
- 1m 15s run time
Only 25s of this were spent preparing the attr paths. A bit more than a
minute was just spent for queuing, checking out the repo, downloading
nix, downloading dependencies, uploading the artifacts - and then
downloading them again in the next step. All of that can be avoided if
we collect the attrs as part of the outpaths job.
By running the attrs step as part of each outpaths step the attrpaths
will be collected 4x, but:
- We save a minute for each eval run to complete.
- We save a full job, giving us more free runners and *possibly* less
queue times for other jobs in the repo.
- We reduce complexity in the workflow file.
| Wolfgang Walther | 2025-05-11 | 1 | -2/+1 |
| * | workflows/eval: load supportedSystems from JSON file•••This is a refactor to prepare the next commit. It doesn't do much on its
own, but is separated for ease of review.
| Wolfgang Walther | 2025-05-11 | 1 | -5/+2 |
| * | ci/compare: nix stats comparison•••Displays stats table in the step-summary if there are no added/removed packages
| Johannes Kirschbauer | 2025-05-06 | 1 | -0/+2 |
| * | Revert "ci/compare: Bring back nix stats comparison" | Johannes Kirschbauer | 2025-05-01 | 1 | -2/+0 |
| * | ci/compare: nix stats comparison•••Displays stats table in the step-summary if there are no added/removed packages
| Johannes Kirschbauer | 2025-05-01 | 1 | -0/+2 |
| * | ci/eval: output per chunk stats | Johannes Kirschbauer | 2025-05-01 | 1 | -0/+8 |
| * | ci/eval: fail on non-empty stderr (#381266) | Wolfgang Walther | 2025-02-14 | 1 | -3/+10 |
| |\ |
|
| | * | ci/eval: Fail on non-empty stderr•••Just like the channel eval requires.
| Silvan Mosberger | 2025-02-11 | 1 | -0/+3 |
| | * | ci/eval: Refactor to cleanly separate stderr•••Previously stderr was mixed with the time stats
This allows checking stderr in the next commit
| Silvan Mosberger | 2025-02-11 | 1 | -3/+7 |
| * | | ci/eval: make eval for non-native platforms less incorrect•••We commonly use platform-dependent conditional patterns like
`lib.meta.availableOn stdenv.hostPlatform` and `stdenv.hostPlatform.isLinux`
to enable different features in a given derivation or to evaluate
completely different derivations based on the platform.
For example, source builds of a given derivation may only be available
on linux but not on darwin. The use of such conditionals allow us to
fall back to patched binaries on darwin instead.
In `chromedriver` (pkgs/development/tools/selenium/chromedriver/default.nix), we use
~~~nix
if lib.meta.availableOn stdenv.hostPlatform chromium then
callPackage ./source.nix { }
else
callPackage ./binary.nix { }
~~~
To provide some context, `chromedriver` source builds are based on `chromium.mkDerivation`
and `chromium` is limited to `lib.platforms.linux`.
Based on the same `chromium.mkDerivation`, we also do source builds for
`electron` (pkgs/top-level/all-packages.nix):
~~~nix
electron_33 = if lib.meta.availableOn stdenv.hostPlatform electron-source.electron_33 then electron-source.electron_33 else electron_33-bin;
electron_34 = electron_34-bin;
electron = electron_34;
~~~
And finally, the top-level `jdk` (Java) attribute has a lot of
indirection, but eventually also boils down to `stdenv.hostPlatform.isLinux`
for source builds and binaries for x86_64-darwin and aarch64-darwin.
A surprising amount of electron and jdk consumers use variations of
`meta.platforms = electron.meta.platforms` in their own meta block.
Due to internal implementation details, the conditionals in those
top-level attributes like `chromedriver`, `electron` and `jdk` are
evaluated based on the value from `builtins.currentSystem` and not the
system passed to `import <nixpkgs> { }`.
This then causes `chromedriver`, `electron`, `jdk` and all dependents
that inherit those `meta.platforms` to appear only available on linux
despite also being available on darwin. Hydra is affected similarly, but
it's a lot more nuanced and in practice not actually *that* bad.
The addition of `--eval-system` ensures that `builtins.currentSystem`
matches the requested platform.
As a bonus, this also fixes the store paths of an impure test that
should probably be made pure:
~~~diff
@@ -885069,13 +886119,13 @@
"out": "/nix/store/lb2500hc69czy4sfga9mbh2k679cr1rp-test-compressDrv"
},
"tests.config.allowPkgsInPermittedInsecurePackages.aarch64-darwin": {
- "out": "/nix/store/0l5h8svrpzwymq35mnpvx82gyc7nf8s4-hello-2.12.1"
+ "out": "/nix/store/v1zjb688mp4y2132b6chii43d5kkxnpa-hello-2.12.1"
},
"tests.config.allowPkgsInPermittedInsecurePackages.aarch64-linux": {
- "out": "/nix/store/0l5h8svrpzwymq35mnpvx82gyc7nf8s4-hello-2.12.1"
+ "out": "/nix/store/hb21z2zdk03dwygsw5lvpa8zc3fbr500-hello-2.12.1"
},
"tests.config.allowPkgsInPermittedInsecurePackages.x86_64-darwin": {
- "out": "/nix/store/0l5h8svrpzwymq35mnpvx82gyc7nf8s4-hello-2.12.1"
+ "out": "/nix/store/gljdqsf0mxv1j8zb04phx9ws09pp7z3l-hello-2.12.1"
},
"tests.config.allowPkgsInPermittedInsecurePackages.x86_64-linux": {
"out": "/nix/store/0l5h8svrpzwymq35mnpvx82gyc7nf8s4-hello-2.12.1"
~~~
Diff stats between two full evals based on 75c8548d8118c31509b89ffd7df6c322b94017dd
with and without this fix on x86_64-linux:
~~~bash
# git diff --no-index --stat /nix/store/659l3xp78255wx7abbahggsnrlj3a1la-combined-result/outpaths.json /nix/store/4fhlq4g5qa65cxbibskq9pma40zigrx7-combined-result/outpaths.json
/nix/store/{659l3xp78255wx7abbahggsnrlj3a1la-combined-result => 4fhlq4g5qa65cxbibskq9pma40zigrx7-combined-result}/outpaths.json | 1416 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-
1 file changed, 1405 insertions(+), 11 deletions(-)
~~~
The full diff is available as a gist at <https://gist.github.com/emilylange/d40c50031fc332bbcca133ad56d224f6>.
When we added `electron_34` only as binary instead of the usual source
on linux with binary fallback in cfed9a19cbc98d12c0167f69238673f4ed89f798
and made the unversioned `electron` top-level point to the newly added
`electron_34` instead of `electron_33`, the GitHub workflow suddenly
reported 20 new packages. Of those 20 reported packages, 17 where
false-positives caused by dropping the wrongly evaluated conditional.
| emilylange | 2025-02-02 | 1 | -0/+1 |
| |/ |
|
| * | ci/eval: allow precisely choosing which systems to evaluate for (evalSystem -... | Silvan Mosberger | 2024-12-15 | 1 | -4/+3 |
| |\ |
|
| | * | ci/eval: allow precisely choosing which systems to evaluate for (evalSystem -... | Gaetan Lepage | 2024-12-14 | 1 | -4/+3 |
| * | | ci/eval: Avoid noise for failing attribute evals•••It's currently annoying to see the actual failure in the attrs step,
because `time -v` displays like 20 lines, which get repeated, therefore
requiring you to scroll up most of the time:
https://github.com/NixOS/nixpkgs/actions/runs/12290298121/job/34297218345#step:5:794
This commit fixes that by only displaying the most important stats, the
same ones as the chunked system-specific evals.
| Silvan Mosberger | 2024-12-12 | 1 | -1/+1 |
| |/ |
|
| * | ci/eval: re-implement compare in nix | Gaetan Lepage | 2024-12-08 | 1 | -18/+10 |
| * | github/workflows/eval: limit number of packages in markdown | Jörg Thalheim | 2024-12-01 | 1 | -5/+1 |
| * | github/workflows/eval: add markdown of added, removed and changed | Noa Aarts | 2024-11-30 | 1 | -0/+5 |
| * | Use GHA eval to assign rebuild labels (#359704) | Jörg Thalheim | 2024-11-29 | 1 | -0/+19 |
| |\ |
|
