summaryrefslogtreecommitdiff
Commit message (Expand)AuthorAgeFilesLines
...
* Use the 2.2 'setvbuf' API style.•••* bin/cuirass.in (main): Use the 2.2 'setvbuf' API style. * examples/guix-track-git.scm (current-error-port): Likewise. Ludovic Courtès2018-04-092-4/+4
* base: 'spawn-builds' really builds by chunks.•••Fixes a regression introduced in 074b9d02f1ca01007f39adbc019763027a51d9bd whereby we'd attempt to build all of DRV at once. * src/cuirass/base.scm (spawn-builds): Pass BATCH, not DRV, to 'build-derivations&' and to 'update-build-statuses!'. Ludovic Courtès2018-04-091-2/+2
* evaluate: Change '%load-path' once and for all.•••* bin/evaluate.in (main): Do not restore the original '%load-path'. Ludovic Courtès2018-04-091-8/+5
* evaluate: Leave GUILE_LOAD_PATH unchanged.•••Previously we'd systematically add the first argument to the search path. When that first argument was the empty string (as with the 'guix-modular' jobset), we'd thus add the current directory to the search path, even if the intent was to leave the load path unchanged. Furthermore, a3a7c09b06027bd30a96ae4607fa40bd790af840 changed load-path handling to be explicit in Scheme. * bin/evaluate.in: Remove GUILE_LOAD_PATH assignment. Ludovic Courtès2018-04-091-2/+0
* evaluate: Change directory to SOURCE.•••* bin/evaluate.in (with-directory-excursion): Remove. (main): chdir to SOURCE. Ludovic Courtès2018-04-091-20/+13
* evaluate: Do not load Guix/Cuirass modules upfront.•••This avoids a situation whereby, when evaluating from a Guix checkout, we'd have already loaded slightly different and incompatible (guix …) modules. Hydra's 'hydra-eval-guile-jobs' implemented the same solution as in this patch already. * bin/evaluate.in: Remove use of (cuirass …) and (guix …) modules. (ref): New procedure. (with-directory-excursion): New macro. (main): Use 'ref'. Remove uses of Guix or Cuirass modules. Ludovic Courtès2018-04-091-37/+62
* base: Pass the correct load path to the 'evaluate' command.•••The previous load path was potentially incorrect since commit 2fe7ff87e23b18d49bd33cffc4766b7eaa382054. * src/cuirass/base.scm (evaluate)[tokenize, load-path]: New variables. Assume #:load-path is colon-separated. Pass LOAD-PATH as the second argument to 'evaluate'. * doc/cuirass.texi (Database): Adjust documentation. Ludovic Courtès2018-04-072-10/+27
* evaluate: Honor the given load path.•••* bin/evaluate.in (main): Prepend LOAD-PATH to '%load-path' for the dynamic extend of the 'primitive-load' call. Ludovic Courtès2018-04-071-2/+14
* evaluate: Really support 'build-derivations' RPCs.•••Fixes a thinko introduced in 1872dd95253b4805a00bfe5dee8d1a0ed90af149. * bin/evaluate.in (main): Make sure 'real-build-things' is bound. Ludovic Courtès2018-04-071-5/+6
* evaluate: Really pass arguments like Hydra.•••* bin/evaluate.in (main): Use SPEC's #:name as a key in ARGS. * examples/random-jobs.scm (make-random-jobs): Adjust accordingly. Ludovic Courtès2018-04-062-7/+11
* base: Let sqlite handle deduplication of the list of pending derivations.•••Previously we would make a SQL query that would return many build jobs, and then call 'delete-duplicates' on that. This was extremely wasteful because the list of returned by the query was huge leading to a heap of several tens of GiB on a big database, and 'delete-duplicates' would lead to more GC and it would take ages. Furthermore, since 'delete-duplicates' is written in C as of Guile 2.2.3, it is uninterruptible from Fiber's viewpoint. Consequently, the kernel thread running the 'restart-builds' fiber would never schedule other fibers, which could lead to deadlocks--e.g., since fibers are scheduled on a circular shuffled list of kernel threads, once every N times, a web server fiber would be sent to that kernel thread and not be serviced. * src/cuirass/base.scm (shuffle-jobs): Remove. (shuffle-derivations): New procedure. (spawn-builds): Take a list of derivations instead of a list of jobs. (restart-builds): Remove 'builds' parameter. Remove 'delete-duplicates' call. Remove done/remaining partitioning. (build-packages): Adjust to pass 'spawn-builds' a list of derivations. * bin/cuirass.in (main): Remove computation of PENDING. Remove second parameter in call to 'restart-builds'. Ludovic Courtès2018-04-052-61/+32
* database: Add 'db-get-pending-derivations'.•••* src/cuirass/database.scm (db-get-pending-derivations): New procedure. * tests/database.scm ("database")["db-get-pending-derivations"]: New test. Ludovic Courtès2018-04-052-0/+40
* evaluate: Pass the file name and revision to the user procedure.•••* bin/evaluate.in (main): Always pass an alist as the arguments to PROC, containing at least 'file-name' and 'revision'. * examples/random-jobs.scm (make-random-jobs): Display 'file-name' and 'revision' from ARGUMENTS. Ludovic Courtès2018-04-052-1/+10
* utils: Add critical sections.•••* src/cuirass/utils.scm (make-critical-section) (call-with-critical-section): New procedures. (with-critical-section): New macro. * src/cuirass/http.scm (with-database-access): Remove. (handle-build-request, handle-builds-request, url-handler): Use 'with-critical-section' instead of 'with-database-access'. (run-cuirass-server): Remove 'spawn-fiber' call. Use 'make-critical-section' instead. Ludovic Courtès2018-04-023-31/+46
* base: Do not resort to Coreutils' "chmod".•••* src/cuirass/base.scm (make-writable-copy)[chmod+w]: New procedure. Replace 'system*' call with 'file-system-fold' call. Ludovic Courtès2018-04-011-1/+15
* base: Make a writable copy of the checkout only when #:no-compile? is false.•••This avoids copying things back and forth. * src/cuirass/base.scm (fetch-repository): Add #:writable-copy? parameter. Call 'make-writable-copy' when it's true. (copy-repository-cache): Remove. (make-writable-copy): New procedure. (evaluate): Add 'source' parameter and pass it to the 'evaluate' program. (process-specs): Define 'compile?'. Pass #:writable-copy? to 'fetch-repository'. Remove call to 'copy-repository-cache'. Remove computation of the checkout directory name. Pass CHECKOUT to 'evaluate'. * bin/evaluate.in (main): Replace 'cachedir' with 'source'. Remove computation of the checkout directory name. Ludovic Courtès2018-04-012-31/+41
* database: 'db-get-builds' honors 'status+submission-time' ordering again.•••Fixes a regression introduced in 1bab5c4e56eb1849edc2cf0b23d433aeb2cac421 whereby the 'status+submission-time' order would no longer be honored. As a result, /api/queue would return the queue ordered by build IDs, making it largely useless. * src/cuirass/database.scm (db-get-builds): Remove 'order' and rename 'order-column-name' to 'order'. Add case for 'status+submission-time'. * tests/database.scm ("database")["db-get-builds"]: Move below "db-update-build-status!" test. Add case for the 'status+submission-time' order. Ludovic Courtès2018-03-292-46/+52
* http: Process client connections really concurrently, again.•••This reinstates c47dfdf82b4be62501a7932eaec4c124566a1829 and fixes the issues that led to the revert in b71f0cdca5aeb82e5eb24f54b32e3f09fee22bad. Before that, 'run-server' would force sequential processing of client requests one after another. * src/cuirass/http.scm (run-cuirass-server): Rewrite to use its own loop instead of 'run-server'. Spawn a database fiber. (with-database-access): New macro. (handle-build-request): Expect 'db-channel' and use 'with-database-access'. (handle-builds-request): Likewise. (url-handler): Likewise. Ludovic Courtès2018-03-281-32/+79
* base: Add 'cancel-old-builds'.•••* src/cuirass/base.scm (cancel-old-builds): New procedure. Ludovic Courtès2018-03-281-0/+9
* evaluate: Tolerate calls to 'build-things' during evaluations.•••* bin/evaluate.in (main): 'build-things' replacement no longer calls 'exit'. Ludovic Courtès2018-03-281-5/+7
* database: Set a 'busy_timeout' to handle concurrent accesses.•••Fixes a bug whereby some fibers would get a SQLITE_BUSY exception while accessing the database: see <https://debbugs.gnu.org/cgi/bugreport.cgi?bug=30644#26>. Suggested by Danny Milosavljevic <dannym@scratchpost.org>. * src/cuirass/database.scm (wal-mode): Rename to... (set-db-options): ... this. Add call to 'sqlite-exec' for 'busy_timeout'. Ludovic Courtès2018-03-251-5/+15
* cuirass: Line-buffer stdout and stderr.•••* bin/cuirass.in (main): Add 'setvbuf' calls. Ludovic Courtès2018-03-251-0/+5
* 'with-store' and 'with-database' and written in terms of 'unwind-protect'.•••* src/cuirass/base.scm (with-store): Rewrite using 'unwind-protect'. * src/cuirass/database.scm (with-database): Likewise. Ludovic Courtès2018-03-192-17/+14
* utils: Add 'unwind-protect'.•••* src/cuirass/utils.scm (unwind-protect): New macro. Ludovic Courtès2018-03-191-0/+20
* http: Correctly set #:timestamp for non-finished builds.•••Fixes a regression introduced in f083282fd3bf813fda0b54ed33278d2d5325dfa1, whereby we'd return 0 as the timestamp for everything in /api/queue. * src/cuirass/http.scm (build->hydra-build): Make 'finished?' a Boolean. Move 'bool->int' call in #:finished definition. Ludovic Courtès2018-03-181-5/+4
* base: Catch errors in the 'process-build-log' handler.•••Previously, when an exception was raised from 'handle-build-event' (e.g., a "database is locked" error), we'd throw, thereby leaving PORT open and we'd never read from it again. Thus, the corresponding 'guix-daemon' process would eventually get stuck in a 'write' call to that socket, and its build processes would stall. * src/cuirass/base.scm (exception-reporter): New procedure. (spawn-builds): Use it. Ludovic Courtès2018-03-181-1/+18
* base: Move database update from 'evaluate' process to the main process.•••Fixes <https://bugs.gnu.org/30618>. Reported by Andreas Enge <andreas@enge.fr>. * bin/evaluate.in (fill-job): Remove. (main): Remove 'database' command-line argument. Remove DB and its uses. Write an (evaluation EVAL JOBS) sexp. * src/cuirass/base.scm (evaluate)[augment-job]: New procedure. Use it. Adjust to read (evaluation EVAL JOBS) sexp. Call 'db-add-evaluation' and 'db-add-derivation'. Ludovic Courtès2018-03-012-39/+39
* base: 'spawn-builds' correctly keeps track of remaining builds.•••Fixes <https://bugs.gnu.org/30645>. Reported by Andreas Enge <andreas@enge.fr>. * src/cuirass/base.scm (spawn-builds): Fix TOTAL vs. COUNT mismatches. Ludovic Courtès2018-03-011-3/+3
* http: Return build completion time as #:timestamp when completed.•••* src/cuirass/http.scm (build->hydra-build): Set #:timestamp to #:stoptime when BUILD is finished. Ludovic Courtès2018-03-011-5/+14
* database: Indent 'db-get-builds'.•••* src/cuirass/database.scm (db-get-builds): Reindent. Ludovic Courtès2018-03-011-49/+49
* database: Adjust style of 'assqx-ref'.•••* src/cuirass/database.scm (db-get-builds)[assqx-ref]: Rewrite with 'match'. Ludovic Courtès2018-03-011-6/+7
* http: 'request-parameters' always returns a list.•••* src/cuirass/http.scm (request-parameters): Return the empty list when QUERY is #f. Ludovic Courtès2018-03-011-11/+12
* http: Fix interpretation of non-numerical parameters.•••Fixes a regression introduced in 593cb7be108ed97bca5371aad2e53fa8ce4817ba. * src/cuirass/http.scm (request-parameters): Fix fallback case in 'match' form. Previously it would return a procedure in this case, as returned by (const param), leading to a failure down the road in 'sqlite-bind-arguments' as could be seen by running tests/http.scm. Ludovic Courtès2018-03-011-4/+5
* logging: "Defensive programming" for 'log-monitoring-stats'.•••I've seen 'scandir' report #f once, even though that's theoretically impossible. * src/cuirass/logging.scm (log-monitoring-stats): Return '() if 'scandir' returns #f. Ludovic Courtès2018-02-271-3/+6
* database: Fix grouping in db-get-builds.•••* src/cuirass/database.scm (db-get-builds): Fix grouping. Danny Milosavljevic2018-02-191-12/+5
* http: Interpret id and nr request-parameters as numbers.•••* src/cuirass/http.scm (request-parameters): Interpret id and nr parameters as numbers. Danny Milosavljevic2018-02-191-1/+5
* database: db-get-builds: Remove debugging output.•••* src/cuirass/database.scm (db-get-builds): Remove debugging output. (db-get-build): Remove debugging output. Danny Milosavljevic2018-02-191-9/+0
* http: Convert build-id URL part to number.•••* src/cuirass/http.scm (url-handler): Convert build-id URL part to number. Danny Milosavljevic2018-02-191-2/+2
* database: db-get-builds: Inline output selection.•••* src/cuirass/database.scm (db-get-builds): Inline output selection. Danny Milosavljevic2018-02-191-4/+93
* database: Simplify 'db-get-builds'.•••* src/cuirass/database.scm (db-get-builds): Modify. (db-get-build): Modify. Danny Milosavljevic2018-02-191-109/+54
* build: Clean cuirass.go.•••* Makefile.am (CLEANFILES): Add $(nodist_guilesite_DATA). Ludovic Courtès2018-02-151-0/+1
* build: Install (web server fiberized) in its right place.•••Previously it would be installed under cuirass/. * Makefile.am (webmoduledir, webobjectdir): New variables. (dist_pkgmodule_DATA): Remove src/web/server/fiberized.scm. (dist_webmodule_DATA, nodist_webobject_DATA): New variables. (CLEANFILES): Add $(nodist_webobject_DATA). Ludovic Courtès2018-02-151-5/+13
* cuirass: Clear the build queue when starting.•••* src/cuirass/base.scm (clear-build-queue): New procedure. * bin/cuirass.in (main): Call it. Ludovic Courtès2018-02-142-0/+8
* examples: random: Make store file names more distinguishable.•••* examples/random-jobs.scm (random-derivation): Add 'suffix' parameter and honor it. (make-random-jobs): Pass a non-empty suffix to 'random-derivation'. Ludovic Courtès2018-02-141-4/+5
* sql: Add indices to speed up common queries.•••* src/schema.sql: Add indices. Ludovic Courtès2018-02-141-0/+6
* Revert "http: Process client connections really concurrently."•••This reverts commit c47dfdf82b4be62501a7932eaec4c124566a1829. Processing connections concurrently would require having separate database handles. See <https://lists.gnu.org/archive/html/guix-devel/2018-02/msg00206.html>. Ludovic Courtès2018-02-141-25/+8
* database: Make 'db-add-derivation' idempotent.•••* src/cuirass/database.scm (db-add-derivation): Catch 'sqlite-error and handle SQLITE_CONSTRAINT_PRIMARYKEY. (SQLITE_CONSTRAINT_UNIQUE): New variable. * tests/database.scm ("database")["db-add-derivation"]: Add extra call to 'db-add-derivation'. Ludovic Courtès2018-02-142-7/+20
* http: Process client connections really concurrently.•••Before that, 'run-server' would force sequential processing of client requests one after another. * src/cuirass/http.scm (run-cuirass-server): Rewrite to use its own loop instead of 'run-server'. Ludovic Courtès2018-02-101-5/+26
* utils: 'non-blocking' forwards exceptions to the calling fiber.•••* src/cuirass/utils.scm (%non-blocking): Forward exceptions to the calling fiber. Ludovic Courtès2018-02-081-4/+13
* database: Use argument binding in 'db-get-builds' queries.•••That makes it safe from SQL injection. * src/cuirass/database.scm (db-get-builds): Rewrite to use question marks in SQL queries and binding through '%sqlite-exec'. * tests/database.scm ("database")["db-get-builds"]: Exercise 'WHERE' clauses. Ludovic Courtès2018-02-082-44/+70
generated by cgit v1.2.3 (git 2.52.0) at 2026-01-17 01:11:08 +0000