diff mbox series

[bug#43494,v2,4/4] services: guix: Add guix-build-coordinator-queue-builds-service-type.

Message ID 20200919101059.8930-4-mail@cbaines.net
State Accepted
Headers show
Series [bug#43494,v2,1/4] gnu: Add guix-build-coordinator. | expand

Checks

Context Check Description
cbaines/comparison success View comparision
cbaines/git branch success View Git branch
cbaines/applying patch success View Laminar job

Commit Message

Christopher Baines Sept. 19, 2020, 10:10 a.m. UTC
* gnu/services/guix.scm (<guix-build-coordinator-queue-builds-configuration>):
New record type.
(guix-build-coordinator-queue-builds-configuration,
guix-build-coordinator-queue-builds-configuration?,
guix-build-coordinator-queue-builds-configuration-package,
guix-build-coordinator-queue-builds-configuration-user,
guix-build-coordinator-queue-builds-coordinator,
guix-build-coordinator-queue-builds-configuration-systems,
guix-build-coordinator-queue-builds-configuration-system-and-targets,
guix-build-coordinator-queue-builds-configuration-guix-data-service,
guix-build-coordinator-queue-builds-configuration-processed-commits-file,
guix-build-coordinator-queue-builds-shepherd-services,
guix-build-coordinator-queue-builds-activation,
guix-build-coordinator-queue-builds-account): New procedures.
(guix-build-coordinator-queue-builds-service-type): New variable.
---
 doc/guix.texi         |  38 +++++++++++++++
 gnu/services/guix.scm | 108 ++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 146 insertions(+)

Comments

Ludovic Courtès Oct. 5, 2020, 8 a.m. UTC | #1
Hi Chris,

Sorry for the delay.  Patches 1–3 LGTM!  One minor comment on patch 4:

Christopher Baines <mail@cbaines.net> skribis:

> * gnu/services/guix.scm (<guix-build-coordinator-queue-builds-configuration>):
> New record type.
> (guix-build-coordinator-queue-builds-configuration,
> guix-build-coordinator-queue-builds-configuration?,
> guix-build-coordinator-queue-builds-configuration-package,
> guix-build-coordinator-queue-builds-configuration-user,
> guix-build-coordinator-queue-builds-coordinator,
> guix-build-coordinator-queue-builds-configuration-systems,
> guix-build-coordinator-queue-builds-configuration-system-and-targets,
> guix-build-coordinator-queue-builds-configuration-guix-data-service,
> guix-build-coordinator-queue-builds-configuration-processed-commits-file,
> guix-build-coordinator-queue-builds-shepherd-services,
> guix-build-coordinator-queue-builds-activation,
> guix-build-coordinator-queue-builds-account): New procedures.
> (guix-build-coordinator-queue-builds-service-type): New variable.

[...]

>  
> +@defvar {Scheme Variable} guix-build-coordinator-queue-builds-service-type
> +Service type for the
> +guix-build-coordinator-queue-builds-from-guix-data-service script.  Its
> +value must be a @code{guix-build-coordinator-queue-builds-configuration}
> +object.
> +@end defvar

Could you add a paragraph right above this explaining what this service
is about?

> +(define guix-build-coordinator-queue-builds-service-type
> +  (service-type
> +   (name 'guix-build-coordinator-queue-builds)
> +   (extensions
> +    (list
> +     (service-extension shepherd-root-service-type
> +                        guix-build-coordinator-queue-builds-shepherd-services)
> +     (service-extension activation-service-type
> +                        guix-build-coordinator-queue-builds-activation)
> +     (service-extension account-service-type
> +                        guix-build-coordinator-queue-builds-account)))
> +   (description
> +    "Run the Guix Build Coordinator queue builds script.")))

Likewise it’d be great if you could add a few works here.

OK to push with these changes, thank you!

Ludo’.
Christopher Baines Oct. 5, 2020, 5:16 p.m. UTC | #2
Ludovic Courtès <ludo@gnu.org> writes:

> Hi Chris,
>
> Sorry for the delay.  Patches 1–3 LGTM!  One minor comment on patch 4:
>
> Christopher Baines <mail@cbaines.net> skribis:
>
>> * gnu/services/guix.scm (<guix-build-coordinator-queue-builds-configuration>):
>> New record type.
>> (guix-build-coordinator-queue-builds-configuration,
>> guix-build-coordinator-queue-builds-configuration?,
>> guix-build-coordinator-queue-builds-configuration-package,
>> guix-build-coordinator-queue-builds-configuration-user,
>> guix-build-coordinator-queue-builds-coordinator,
>> guix-build-coordinator-queue-builds-configuration-systems,
>> guix-build-coordinator-queue-builds-configuration-system-and-targets,
>> guix-build-coordinator-queue-builds-configuration-guix-data-service,
>> guix-build-coordinator-queue-builds-configuration-processed-commits-file,
>> guix-build-coordinator-queue-builds-shepherd-services,
>> guix-build-coordinator-queue-builds-activation,
>> guix-build-coordinator-queue-builds-account): New procedures.
>> (guix-build-coordinator-queue-builds-service-type): New variable.
>
> [...]
>
>>  
>> +@defvar {Scheme Variable} guix-build-coordinator-queue-builds-service-type
>> +Service type for the
>> +guix-build-coordinator-queue-builds-from-guix-data-service script.  Its
>> +value must be a @code{guix-build-coordinator-queue-builds-configuration}
>> +object.
>> +@end defvar
>
> Could you add a paragraph right above this explaining what this service
> is about?
>
>> +(define guix-build-coordinator-queue-builds-service-type
>> +  (service-type
>> +   (name 'guix-build-coordinator-queue-builds)
>> +   (extensions
>> +    (list
>> +     (service-extension shepherd-root-service-type
>> +                        guix-build-coordinator-queue-builds-shepherd-services)
>> +     (service-extension activation-service-type
>> +                        guix-build-coordinator-queue-builds-activation)
>> +     (service-extension account-service-type
>> +                        guix-build-coordinator-queue-builds-account)))
>> +   (description
>> +    "Run the Guix Build Coordinator queue builds script.")))
>
> Likewise it’d be great if you could add a few works here.
>
> OK to push with these changes, thank you!

I've tried to add some clarity in the places you mention. I'm still very
tempted to make a separate repository/package for the queue builds from
Guix Data Service script... I'm not quite decided yet though.

Anyway, thanks for taking another look. I've gone ahead and pushed these
patches as c14714cbbfe9239410aec2ed73282192a64fbbdc. I did tweak the
package definition a bit though to move closer to being able to
cross-compile it (you can now if you tweak the inputs).

Thanks again,

Chris
diff mbox series

Patch

diff --git a/doc/guix.texi b/doc/guix.texi
index b2dde30ddb..767c99c5cb 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -27603,6 +27603,44 @@  input store items aren't already available.
 @end table
 @end deftp
 
+@defvar {Scheme Variable} guix-build-coordinator-queue-builds-service-type
+Service type for the
+guix-build-coordinator-queue-builds-from-guix-data-service script.  Its
+value must be a @code{guix-build-coordinator-queue-builds-configuration}
+object.
+@end defvar
+
+@deftp {Data Type} guix-build-coordinator-queue-builds-configuration
+Data type representing the options to the queue builds from guix data
+service script.
+
+@table @asis
+@item @code{package} (default: @code{guix-build-coordinator})
+The Guix Build Coordinator package to use.
+
+@item @code{user} (default: @code{"guix-build-coordinator-queue-builds"})
+The system user to run the service as.
+
+@item @code{coordinator} (default: @code{"http://localhost:8745"})
+The URI to use when connecting to the coordinator.
+
+@item @code{systems} (default: @code{#f})
+The systems for which to fetch derivations to build.
+
+@item @code{systems-and-targets} (default: @code{#f})
+An association list of system and target pairs for which to fetch
+derivations to build.
+
+@item @code{guix-data-service} (default: @code{"https://data.guix.gnu.org"})
+The Guix Data Service instance from which to query to find out about
+derivations to build.
+
+@item @code{processed-commits-file} (default: @code{"/var/cache/guix-build-coordinator-queue-builds/processed-commits"})
+A file to record which commits have been processed, to avoid needlessly
+processing them again if the service is restarted.
+
+@end table
+@end deftp
 
 @subsubheading Guix Data Service
 The @uref{http://data.guix.gnu.org,Guix Data Service} processes, stores
diff --git a/gnu/services/guix.scm b/gnu/services/guix.scm
index e4f5cf57b9..1493a24be5 100644
--- a/gnu/services/guix.scm
+++ b/gnu/services/guix.scm
@@ -65,6 +65,18 @@ 
 
             guix-build-coordinator-agent-service-type
 
+            guix-build-coordinator-queue-builds-configuration
+            guix-build-coordinator-queue-builds-configuration?
+            guix-build-coordinator-queue-builds-configuration-package
+            guix-build-coordinator-queue-builds-configuration-user
+            guix-build-coordinator-queue-builds-coordinator
+            guix-build-coordinator-queue-builds-configuration-systems
+            guix-build-coordinator-queue-builds-configuration-system-and-targets
+            guix-build-coordinator-queue-builds-configuration-guix-data-service
+            guix-build-coordinator-queue-builds-configuration-processed-commits-file
+
+            guix-build-coordinator-queue-builds-service-type
+
             <guix-data-service-configuration>
             guix-data-service-configuration
             guix-data-service-configuration?
@@ -137,6 +149,28 @@ 
    guix-build-coordinator-agent-configuration-non-derivation-substitute-urls
    (default #f)))
 
+(define-record-type* <guix-build-coordinator-queue-builds-configuration>
+  guix-build-coordinator-queue-builds-configuration
+  make-guix-build-coordinator-queue-builds-configuration
+  guix-build-coordinator-queue-builds-configuration?
+  (package              guix-build-coordinator-queue-builds-configuration-package
+                        (default guix-build-coordinator))
+  (user                 guix-build-coordinator-queue-builds-configuration-user
+                        (default "guix-build-coordinator-queue-builds"))
+  (coordinator          guix-build-coordinator-queue-builds-coordinator
+                        (default "http://localhost:8745"))
+  (systems              guix-build-coordinator-queue-builds-configuration-systems
+                        (default #f))
+  (systems-and-targets
+   guix-build-coordinator-queue-builds-configuration-system-and-targets
+   (default #f))
+  (guix-data-service
+   guix-build-coordinator-queue-builds-configuration-guix-data-service
+   (default "https://data.guix.gnu.org"))
+  (processed-commits-file
+   guix-build-coordinator-queue-builds-configuration-processed-commits-file
+   (default "/var/cache/guix-build-coordinator-queue-builds/processed-commits")))
+
 (define* (make-guix-build-coordinator-start-script database-uri-string
                                                    allocation-strategy
                                                    pid-file
@@ -360,6 +394,80 @@ 
    (description
     "Run a Guix Build Coordinator agent.")))
 
+(define (guix-build-coordinator-queue-builds-shepherd-services config)
+  (match-record config <guix-build-coordinator-queue-builds-configuration>
+    (package user coordinator systems systems-and-targets
+             guix-data-service processed-commits-file)
+    (list
+     (shepherd-service
+      (documentation "Guix Build Coordinator queue builds from Guix Data Service")
+      (provision '(guix-build-coordinator-queue-builds))
+      (requirement '(networking))
+      (start
+       #~(make-forkexec-constructor
+          (list
+           #$(file-append
+              package
+              "/bin/guix-build-coordinator-queue-builds-from-guix-data-service")
+           #$(string-append "--coordinator=" coordinator)
+           #$@(map (lambda (system)
+                     (string-append "--system=" system))
+                   (or systems '()))
+           #$@(map (match-lambda
+                     ((system . target)
+                      (string-append "--system-and-target=" system "=" target)))
+                   (or systems-and-targets '()))
+           #$@(if guix-data-service
+                  #~(#$(string-append "--guix-data-service=" guix-data-service))
+                  #~())
+           #$@(if processed-commits-file
+                  #~(#$(string-append "--processed-commits-file="
+                                      processed-commits-file))
+                  #~()))
+          #:user #$user
+          #:pid-file "/var/run/guix-build-coordinator-queue-builds/pid"
+          #:environment-variables
+          `(,(string-append
+              "GUIX_LOCPATH=" #$glibc-utf8-locales "/lib/locale")
+            "LC_ALL=en_US.utf8")
+          #:log-file "/var/log/guix-build-coordinator/queue-builds.log"))
+      (stop #~(make-kill-destructor))))))
+
+(define (guix-build-coordinator-queue-builds-activation config)
+  #~(begin
+      (use-modules (guix build utils))
+
+      (mkdir-p "/var/log/guix-build-coordinator")
+
+      ;; Allow writing the PID file
+      (mkdir-p "/var/run/guix-build-coordinator-queue-builds")
+      (chown "/var/run/guix-build-coordinator-queue-builds"
+             (passwd:uid %user)
+             (passwd:gid %user))))
+
+(define (guix-build-coordinator-queue-builds-account config)
+  (list (user-account
+         (name (guix-build-coordinator-queue-builds-configuration-user config))
+         (group "nogroup")
+         (system? #t)
+         (comment "Guix Build Coordinator queue-builds user")
+         (home-directory "/var/empty")
+         (shell (file-append shadow "/sbin/nologin")))))
+
+(define guix-build-coordinator-queue-builds-service-type
+  (service-type
+   (name 'guix-build-coordinator-queue-builds)
+   (extensions
+    (list
+     (service-extension shepherd-root-service-type
+                        guix-build-coordinator-queue-builds-shepherd-services)
+     (service-extension activation-service-type
+                        guix-build-coordinator-queue-builds-activation)
+     (service-extension account-service-type
+                        guix-build-coordinator-queue-builds-account)))
+   (description
+    "Run the Guix Build Coordinator queue builds script.")))
+
 
 ;;;
 ;;; Guix Data Service