diff mbox series

[bug#43494,3/4] services: guix: Add guix-build-coordinator-agent-service-type.

Message ID 20200918184042.22660-3-mail@cbaines.net
State Accepted
Headers show
Series Add package and services for the Guix Build Coordinator | expand

Checks

Context Check Description
cbaines/comparison success View comparision
cbaines/git branch success View Git branch
cbaines/applying patch success View Laminar job

Commit Message

Christopher Baines Sept. 18, 2020, 6:40 p.m. UTC
* gnu/services/guix.scm (<guix-build-coordinator-agent-configuration>): New
record type.
(guix-build-coordinator-agent-configuration,
guix-build-coordinator-agent-configuration?,
guix-build-coordinator-agent-configuration-package,
guix-build-coordinator-agent-configuration-user,
guix-build-coordinator-agent-configuration-coordinator,
guix-build-coordinator-agent-configuration-uuid),
guix-build-coordinator-agent-configuration-password,
guix-build-coordinator-agent-configuration-password-file,
guix-build-coordinator-agent-configuration-systems,
guix-build-coordinator-agent-configuration-max-parallel-builds,
guix-build-coordinator-agent-configuration-derivation-substitute-urls,
guix-build-coordinator-agent-configuration-non-derivation-substitute-urls,
guix-build-coordinator-agent-shepherd-services,
guix-build-coordinator-agent-activation,
guix-build-coordinator-agent-account): New procedures.
(guix-build-coordinator-agent-service-type): New variable.
* doc/guix.texi (Guix Services): Document it.
---
 doc/guix.texi         |  50 +++++++++++++++++
 gnu/services/guix.scm | 121 ++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 171 insertions(+)

Comments

Ludovic Courtès Sept. 18, 2020, 8:25 p.m. UTC | #1
Christopher Baines <mail@cbaines.net> skribis:

> * gnu/services/guix.scm (<guix-build-coordinator-agent-configuration>): New
> record type.
> (guix-build-coordinator-agent-configuration,
> guix-build-coordinator-agent-configuration?,
> guix-build-coordinator-agent-configuration-package,
> guix-build-coordinator-agent-configuration-user,
> guix-build-coordinator-agent-configuration-coordinator,
> guix-build-coordinator-agent-configuration-uuid),
> guix-build-coordinator-agent-configuration-password,
> guix-build-coordinator-agent-configuration-password-file,
> guix-build-coordinator-agent-configuration-systems,
> guix-build-coordinator-agent-configuration-max-parallel-builds,
> guix-build-coordinator-agent-configuration-derivation-substitute-urls,
> guix-build-coordinator-agent-configuration-non-derivation-substitute-urls,
> guix-build-coordinator-agent-shepherd-services,
> guix-build-coordinator-agent-activation,
> guix-build-coordinator-agent-account): New procedures.
> (guix-build-coordinator-agent-service-type): New variable.
> * doc/guix.texi (Guix Services): Document it.

[...]

> +@defvar {Scheme Variable} guix-build-coordinator-agent-service-type
> +Service type for a Guix Build Coordinator agent.  Its value must be a

Perhaps “coordinator” and “agent” should be defined in a few sentences
above to clarify what this is about.

> +@item @code{derivation-substitute-urls} (default: @code{1})
> +URLs from which to attempt to fetch substitutes for derivations, if the
> +derivations aren't already available.
> +
> +@item @code{non-derivation-substitute-urls} (default: @code{1})
> +URLs from which to attempt to fetch substitutes for build inputs, if the
> +input store items aren't already available.

This is interesting, I wonder how you can distinguish between the two in
code.  You have to open different sessions, right?  Or to call
‘set-build-options’ again?

> +   (description
> +    "Run an instance of the Guix Build Coordinator.")))

+ “agent”

Otherwise LGTM!

Ludo’.
Christopher Baines Sept. 19, 2020, 9:49 a.m. UTC | #2
Ludovic Courtès <ludo@gnu.org> writes:

> Christopher Baines <mail@cbaines.net> skribis:
>
>> * gnu/services/guix.scm (<guix-build-coordinator-agent-configuration>): New
>> record type.
>> (guix-build-coordinator-agent-configuration,
>> guix-build-coordinator-agent-configuration?,
>> guix-build-coordinator-agent-configuration-package,
>> guix-build-coordinator-agent-configuration-user,
>> guix-build-coordinator-agent-configuration-coordinator,
>> guix-build-coordinator-agent-configuration-uuid),
>> guix-build-coordinator-agent-configuration-password,
>> guix-build-coordinator-agent-configuration-password-file,
>> guix-build-coordinator-agent-configuration-systems,
>> guix-build-coordinator-agent-configuration-max-parallel-builds,
>> guix-build-coordinator-agent-configuration-derivation-substitute-urls,
>> guix-build-coordinator-agent-configuration-non-derivation-substitute-urls,
>> guix-build-coordinator-agent-shepherd-services,
>> guix-build-coordinator-agent-activation,
>> guix-build-coordinator-agent-account): New procedures.
>> (guix-build-coordinator-agent-service-type): New variable.
>> * doc/guix.texi (Guix Services): Document it.
>
> [...]
>
>> +@defvar {Scheme Variable} guix-build-coordinator-agent-service-type
>> +Service type for a Guix Build Coordinator agent.  Its value must be a
>
> Perhaps “coordinator” and “agent” should be defined in a few sentences
> above to clarify what this is about.

I've added a bit more information, I want to at some point actually give
a workable minimal example configuration, but I need to do some more
thinking about that.

>> +@item @code{derivation-substitute-urls} (default: @code{1})
>> +URLs from which to attempt to fetch substitutes for derivations, if the
>> +derivations aren't already available.
>> +
>> +@item @code{non-derivation-substitute-urls} (default: @code{1})
>> +URLs from which to attempt to fetch substitutes for build inputs, if the
>> +input store items aren't already available.
>
> This is interesting, I wonder how you can distinguish between the two in
> code.  You have to open different sessions, right?  Or to call
> ‘set-build-options’ again?

Yeah, it's not strictly necessary to have some separation. In fact, I
can't remember exactly, but I think it should be possible to deploy the
coordinator and agents without explicit configuration of the substitute
URLs, but I need to check that.

These options were added mostly because it useful if you have two
seperate sources of substitutes, one for just derivations (say an
instance of the Guix Data Service), and one just for build outputs (say
populated by one of the hooks) [1].

1: https://git.cbaines.net/guix/build-coordinator/commit/?id=bda7d58853ed4fba976cac92a70c4dc68db263aa

>> +   (description
>> +    "Run an instance of the Guix Build Coordinator.")))
>
> + “agent”
>
> Otherwise LGTM!

Thanks!
diff mbox series

Patch

diff --git a/doc/guix.texi b/doc/guix.texi
index e83008c177..db91831197 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -27541,6 +27541,56 @@  The Guile package with which to run the Guix Build Coordinator.
 @end table
 @end deftp
 
+@defvar {Scheme Variable} guix-build-coordinator-agent-service-type
+Service type for a Guix Build Coordinator agent.  Its value must be a
+@code{guix-build-coordinator-agent-configuration} object.
+@end defvar
+
+@deftp {Data Type} guix-build-coordinator-agent-configuration
+Data type representing the configuration a Guix Build Coordinator agent.
+
+@table @asis
+@item @code{package} (default: @code{guix-build-coordinator})
+The Guix Build Coordinator package to use.
+
+@item @code{user} (default: @code{"guix-build-coordinator-agent"})
+The system user to run the service as.
+
+@item @code{coordinator} (default: @code{"http://localhost:8745"})
+The URI to use when connecting to the coordinator.
+
+@item @code{uuid}
+The UUID of the agent.  This should be generated by the coordinator
+process, stored in the coordinator database, and used by the intended
+agent.
+
+@item @code{password} (default: @code{#f})
+The password to use when connecting to the coordinator.  A file to read
+the password from can also be specified, and this is more secure.
+
+@item @code{password-file} (default: @code{#f})
+A file containing the password to use when connecting to the
+coordinator.
+
+@item @code{systems} (default: @var{#f})
+The systems for which this agent should fetch builds.  The agent process
+will use the current system it's running on as the default.
+
+@item @code{max-parallel-builds} (default: @code{1})
+The number of builds to perform in parallel.
+
+@item @code{derivation-substitute-urls} (default: @code{1})
+URLs from which to attempt to fetch substitutes for derivations, if the
+derivations aren't already available.
+
+@item @code{non-derivation-substitute-urls} (default: @code{1})
+URLs from which to attempt to fetch substitutes for build inputs, if the
+input store items aren't already available.
+
+@end table
+@end deftp
+
+
 @subsubheading Guix Data Service
 The @uref{http://data.guix.gnu.org,Guix Data Service} processes, stores
 and provides data about GNU Guix.  This includes information about
diff --git a/gnu/services/guix.scm b/gnu/services/guix.scm
index de14fcadb2..71e58237da 100644
--- a/gnu/services/guix.scm
+++ b/gnu/services/guix.scm
@@ -50,6 +50,21 @@ 
 
             guix-build-coordinator-service-type
 
+            guix-build-coordinator-agent-configuration
+            guix-build-coordinator-agent-configuration?
+            guix-build-coordinator-agent-configuration-package
+            guix-build-coordinator-agent-configuration-user
+            guix-build-coordinator-agent-configuration-coordinator
+            guix-build-coordinator-agent-configuration-uuid)
+            guix-build-coordinator-agent-configuration-password
+            guix-build-coordinator-agent-configuration-password-file
+            guix-build-coordinator-agent-configuration-systems
+            guix-build-coordinator-agent-configuration-max-parallel-builds
+            guix-build-coordinator-agent-configuration-derivation-substitute-urls
+            guix-build-coordinator-agent-configuration-non-derivation-substitute-urls
+
+            guix-build-coordinator-agent-service-type
+
             <guix-data-service-configuration>
             guix-data-service-configuration
             guix-data-service-configuration?
@@ -95,6 +110,33 @@ 
   (guile                           guix-build-coordinator-configuration-guile
                                    (default guile-3.0-latest)))
 
+(define-record-type* <guix-build-coordinator-agent-configuration>
+  guix-build-coordinator-agent-configuration
+  make-guix-build-coordinator-agent-configuration
+  guix-build-coordinator-agent-configuration?
+  (package             guix-build-coordinator-agent-configuration-package
+                       (default guix-build-coordinator))
+  (user                guix-build-coordinator-agent-configuration-user
+                       (default "guix-build-coordinator-agent"))
+  (coordinator         guix-build-coordinator-agent-configuration-coordinator
+                       (default "http://localhost:8745"))
+  (uuid                guix-build-coordinator-agent-configuration-uuid)
+  (password            guix-build-coordinator-agent-configuration-password
+                       (default #f))
+  (password-file       guix-build-coordinator-agent-configuration-password-file
+                       (default #f))
+  (systems             guix-build-coordinator-agent-configuration-systems
+                       (default #f))
+  (max-parallel-builds
+   guix-build-coordinator-agent-configuration-max-parallel-builds
+   (default 1))
+  (derivation-substitute-urls
+   guix-build-coordinator-agent-configuration-derivation-substitute-urls
+   (default #f))
+  (non-derivation-substitute-urls
+   guix-build-coordinator-agent-configuration-non-derivation-substitute-urls
+   (default #f)))
+
 (define* (make-guix-build-coordinator-start-script database-uri-string
                                                    allocation-strategy
                                                    pid-file
@@ -243,6 +285,85 @@ 
    (description
     "Run an instance of the Guix Build Coordinator.")))
 
+(define (guix-build-coordinator-agent-shepherd-services config)
+  (match-record config <guix-build-coordinator-agent-configuration>
+    (package user coordinator uuid password password-file max-parallel-builds
+             derivation-substitute-urls non-derivation-substitute-urls
+             systems)
+    (list
+     (shepherd-service
+      (documentation "Guix Build Coordinator Agent")
+      (provision '(guix-build-coordinator-agent))
+      (requirement '(networking))
+      (start #~(make-forkexec-constructor
+                (list #$(file-append package "/bin/guix-build-coordinator-agent")
+                      #$(string-append "--coordinator=" coordinator)
+                      #$(string-append "--uuid=" uuid)
+                      #$@(if password
+                             #~(#$(string-append "--password=" password))
+                             #~())
+                      #$@(if password-file
+                             #~(#$(string-append "--password-file=" password-file))
+                             #~())
+                      #$(simple-format #f "--max-parallel-builds=~A"
+                                       max-parallel-builds)
+                      #$@(if derivation-substitute-urls
+                             #~(#$(string-append
+                                   "--derivation-substitute-urls="
+                                 (string-join derivation-substitute-urls " ")))
+                             #~())
+                      #$@(if non-derivation-substitute-urls
+                             #~(#$(string-append
+                                   "--non-derivation-substitute-urls="
+                                   (string-join derivation-substitute-urls " ")))
+                             #~())
+                      #$@(map (lambda (system)
+                                (string-append "--system=" system))
+                              (or systems '())))
+                #:user #$user
+                #:pid-file "/var/run/guix-build-coordinator-agent/pid"
+                #:environment-variables
+                `(,(string-append
+                    "GUIX_LOCPATH=" #$glibc-utf8-locales "/lib/locale")
+                  "LC_ALL=en_US.utf8")
+                #:log-file "/var/log/guix-build-coordinator/agent.log"))
+      (stop #~(make-kill-destructor))))))
+
+(define (guix-build-coordinator-agent-activation config)
+  #~(begin
+      (use-modules (guix build utils))
+
+      (mkdir-p "/var/log/guix-build-coordinator")
+
+      ;; Allow writing the PID file
+      (mkdir-p "/var/run/guix-build-coordinator-agent")
+      (chown "/var/run/guix-build-coordinator-agent"
+             (passwd:uid %user)
+             (passwd:gid %user))))
+
+(define (guix-build-coordinator-agent-account config)
+  (list (user-account
+         (name (guix-build-coordinator-agent-configuration-user config))
+         (group "nogroup")
+         (system? #t)
+         (comment "Guix Build Coordinator agent user")
+         (home-directory "/var/empty")
+         (shell (file-append shadow "/sbin/nologin")))))
+
+(define guix-build-coordinator-agent-service-type
+  (service-type
+   (name 'guix-build-coordinator-agent)
+   (extensions
+    (list
+     (service-extension shepherd-root-service-type
+                        guix-build-coordinator-agent-shepherd-services)
+     (service-extension activation-service-type
+                        guix-build-coordinator-agent-activation)
+     (service-extension account-service-type
+                        guix-build-coordinator-agent-account)))
+   (description
+    "Run an instance of the Guix Build Coordinator.")))
+
 
 ;;;
 ;;; Guix Data Service