From patchwork Sat Oct 14 21:36:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Giacomo Leidi X-Patchwork-Id: 54830 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id BA8D927BBE2; Sat, 14 Oct 2023 22:37:55 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,SPF_HELO_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 5084627BBE9 for ; Sat, 14 Oct 2023 22:37:52 +0100 (BST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qrmKJ-0003x4-RG; Sat, 14 Oct 2023 17:37:39 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qrmKI-0003wu-Rl for guix-patches@gnu.org; Sat, 14 Oct 2023 17:37:38 -0400 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qrmKI-0005oo-JW for guix-patches@gnu.org; Sat, 14 Oct 2023 17:37:38 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1qrmKg-0005wZ-Be for guix-patches@gnu.org; Sat, 14 Oct 2023 17:38:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#66160] [PATCH] gnu: Add oci-container-service-type. References: <650a02e2-9425-a7fd-2014-7cf6e17ee65b@autistici.org> In-Reply-To: <650a02e2-9425-a7fd-2014-7cf6e17ee65b@autistici.org> Resent-From: Giacomo Leidi Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sat, 14 Oct 2023 21:38:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 66160 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: "66160@debbugs.gnu.org"@debbugs.gnu.org Cc: Giacomo Leidi Received: via spool by 66160-submit@debbugs.gnu.org id=B66160.169731943822779 (code B ref 66160); Sat, 14 Oct 2023 21:38:02 +0000 Received: (at 66160) by debbugs.gnu.org; 14 Oct 2023 21:37:18 +0000 Received: from localhost ([127.0.0.1]:50760 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qrmJx-0005vK-AB for submit@debbugs.gnu.org; Sat, 14 Oct 2023 17:37:18 -0400 Received: from confino.investici.org ([93.190.126.19]:23381) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qrmJu-0005vB-Ie for 66160@debbugs.gnu.org; Sat, 14 Oct 2023 17:37:16 -0400 Received: from mx1.investici.org (unknown [127.0.0.1]) by confino.investici.org (Postfix) with ESMTP id 4S7Gss1R6xz112P; Sat, 14 Oct 2023 21:36:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=autistici.org; s=stigmate; t=1697319409; bh=SNhrLOp4rcsZ23W9XxBGRJjHILYPtJxp0yxV4pSQVsg=; h=From:To:Cc:Subject:Date:From; b=PFuhxJrgdygRmhHkynjHq/Hfmuh50gJ0ImcNgSph03Ge2IT4Kf02KoavC5Y6YolpO JhlFxfdjyuOlZKWTWg1AqqYXZP1G0b0U/KTbbFc0Nu5NlqQX5Z6MHUkLSdsOplt+GM uTmH0Kn2pNotZc7r5K12ppNI+cib2uOH9NdHHimk= Received: from [93.190.126.19] (mx1.investici.org [93.190.126.19]) (Authenticated sender: goodoldpaul@autistici.org) by localhost (Postfix) with ESMTPSA id 4S7Gss0q91z10yD; Sat, 14 Oct 2023 21:36:48 +0000 (UTC) Date: Sat, 14 Oct 2023 23:36:12 +0200 Message-ID: X-Mailer: git-send-email 2.41.0 MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-to: Giacomo Leidi X-ACL-Warn: , Giacomo Leidi via Guix-patches X-Patchwork-Original-From: Giacomo Leidi via Guix-patches via From: Giacomo Leidi Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org X-getmail-retrieved-from-mailbox: Patches * gnu/services/docker.scm (oci-container-configuration): New variable; (oci-container-shepherd-service): new variable; (oci-container-service-type): new variable. * doc/guix.texi: Document it. --- doc/guix.texi | 123 +++++++++++++++++++++ gnu/services/docker.scm | 237 +++++++++++++++++++++++++++++++++++++++- 2 files changed, 359 insertions(+), 1 deletion(-) base-commit: 8aad7210ea06992ee3f36ca7f57678240949e063 diff --git a/doc/guix.texi b/doc/guix.texi index 083504dcb8..54d5074bd7 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -39534,6 +39534,129 @@ Miscellaneous Services @command{singularity run} and similar commands. @end defvar +@cindex OCI-backed, Shepherd services +@subsubheading OCI backed services + +Should you wish to manage your Docker containers with the same consistent +interface you use for your other Shepherd services, +@var{oci-container-service-type} is the tool to use: given an +@acronym{Open Container Initiative, OCI} container image, it will run it in a +Shepherd service. One example where this is useful: it lets you run services +that are available as Docker/OCI images but not yet packaged for Guix. + +@defvar oci-container-service-type + +This is a thin wrapper around Docker's CLI that executes OCI images backed +processes as Shepherd Services. + +@lisp +(service oci-container-service-type + (list + (oci-container-configuration + (image "prom/prometheus") + (network "host") + (ports + '(("9000" . "9000") + ("9090" . "9090")))))) + +(service oci-container-service-type + (list + (oci-container-configuration + (image "grafana/grafana:10.0.1") + (network "host") + (ports + '(("3000" . "3000"))) + (volumes + '("/var/lib/grafana:/var/lib/grafana")))))) +@end lisp + +In this example two different Shepherd services are going be added to the +system. Each @code{oci-container-configuration} record translates to a +@code{docker run} invocation and its fields directly map to options. You can +refer to the +@url{https://docs.docker.com/engine/reference/commandline/run,upstream}, +documentation for the semantics of each value. If the images are not found they +will be +@url{https://docs.docker.com/engine/reference/commandline/pull/,pulled}. The +spawned services are going to be attached to the host network and are supposed +to behave like other processes. + +@end defvar + +@c %start of fragment + +@deftp {Data Type} oci-container-configuration +Available @code{oci-container-configuration} fields are: + +@table @asis +@item @code{user} (default: @code{"oci-container"}) (type: string) +The user under whose authority docker commands will be run. + +@item @code{group} (default: @code{"docker"}) (type: string) +The group under whose authority docker commands will be run. + +@item @code{command} (default: @code{()}) (type: list-of-strings) +Overwrite the default command (@code{CMD}) of the image. + +@item @code{entrypoint} (default: @code{""}) (type: string) +Overwrite the default entrypoint (@code{ENTRYPOINT}) of the image. + +@item @code{environment} (default: @code{()}) (type: list) +Set environment variables. This can be a list of pairs or strings, even mixed: + +@lisp +(list '("LANGUAGE" . "eo:ca:eu") + "JAVA_HOME=/opt/java") +@end lisp + +String are passed directly to the Docker CLI. You can refer to the +@uref{https://docs.docker.com/engine/reference/commandline/run/#env,upstream} +documentation for semantics. + +@item @code{image} (type: string) +The image used to build the container. Images are resolved by the +Docker Engine, and follow the usual format +@code{myregistry.local:5000/testing/test-image:tag}. + +@item @code{name} (default: @code{""}) (type: string) +Set a name for the spawned container. + +@item @code{network} (default: @code{""}) (type: string) +Set a Docker network for the spawned container. + +@item @code{ports} (default: @code{()}) (type: list) +Set the port or port ranges to expose from the spawned container. This can be a +list of pairs or strings, even mixed: + +@lisp +(list '("8080" . "80") + "10443:443") +@end lisp + +String are passed directly to the Docker CLI. You can refer to the +@uref{https://docs.docker.com/engine/reference/commandline/run/#publish,upstream} +documentation for semantics. + +@item @code{volumes} (default: @code{()}) (type: list) +Set volume mappings for the spawned container. This can be a +list of pairs or strings, even mixed: + +@lisp +(list '("/root/data/grafana" . "/var/lib/grafana") + "/gnu/store:/gnu/store") +@end lisp + +String are passed directly to the Docker CLI. You can refer to the +@uref{https://docs.docker.com/engine/reference/commandline/run/#volume,upstream} +documentation for semantics. + +@end table + +@end deftp + + +@c %end of fragment + @cindex Audit @subsubheading Auditd Service diff --git a/gnu/services/docker.scm b/gnu/services/docker.scm index c2023d618c..2d709bf2ce 100644 --- a/gnu/services/docker.scm +++ b/gnu/services/docker.scm @@ -5,6 +5,7 @@ ;;; Copyright © 2020 Efraim Flashner ;;; Copyright © 2020 Jesse Dowell ;;; Copyright © 2021 Brice Waegeneire +;;; Copyright © 2023 Giacomo Leidi ;;; ;;; This file is part of GNU Guix. ;;; @@ -29,15 +30,34 @@ (define-module (gnu services docker) #:use-module (gnu services shepherd) #:use-module (gnu system setuid) #:use-module (gnu system shadow) + #:use-module (gnu packages admin) ;shadow #:use-module (gnu packages docker) #:use-module (gnu packages linux) ;singularity #:use-module (guix records) + #:use-module (guix diagnostics) #:use-module (guix gexp) + #:use-module (guix i18n) #:use-module (guix packages) + #:use-module (srfi srfi-1) + #:use-module (ice-9 format) + #:use-module (ice-9 match) #:export (docker-configuration docker-service-type - singularity-service-type)) + singularity-service-type + oci-container-configuration + oci-container-configuration? + oci-container-configuration-fields + oci-container-configuration-command + oci-container-configuration-entrypoint + oci-container-configuration-environment + oci-container-configuration-image + oci-container-configuration-name + oci-container-configuration-network + oci-container-configuration-ports + oci-container-configuration-volumes + oci-container-service-type + oci-container-shepherd-service)) (define-configuration docker-configuration (docker @@ -216,3 +236,218 @@ (define singularity-service-type (service-extension activation-service-type (const %singularity-activation)))) (default-value singularity))) + + +;;; +;;; OCI container. +;;; + +(define (oci-sanitize-pair pair delimiter) + (define (valid? member) + (or (string? member) + (gexp? member) + (file-like? member))) + (match pair + (((? valid? key) . (? valid? value)) + #~(string-append #$key #$delimiter #$value)) + (_ + (raise + (formatted-message + (G_ "pair members must contain only strings, gexps or file-like objects +but ~a was found") + pair))))) + +(define (oci-sanitize-mixed-list name value delimiter) + (map + (lambda (el) + (cond ((string? el) el) + ((pair? el) (oci-sanitize-pair el delimiter)) + (else + (raise + (formatted-message + (G_ "~a members must be either a string or a pair but ~a was +found!") + name el))))) + value)) + +(define (oci-sanitize-environment value) + ;; Expected spec format: + ;; '(("HOME" . "/home/nobody") "JAVA_HOME=/java") + (oci-sanitize-mixed-list "environment" value "=")) + +(define (oci-sanitize-ports value) + ;; Expected spec format: + ;; '(("8088" . "80") "2022:22") + (oci-sanitize-mixed-list "ports" value ":")) + +(define (oci-sanitize-volumes value) + ;; Expected spec format: + ;; '(("/mnt/dir" . "/dir") "/run/current-system/profile:/java") + (oci-sanitize-mixed-list "volumes" value ":")) + +(define-maybe/no-serialization string) + +(define-configuration/no-serialization oci-container-configuration + (user + (string "oci-container") + "The user under whose authority docker commands will be run.") + (group + (string "docker") + "The group under whose authority docker commands will be run.") + (command + (list-of-strings '()) + "Overwrite the default command (@code{CMD}) of the image.") + (entrypoint + (maybe-string) + "Overwrite the default entrypoint (@code{ENTRYPOINT}) of the image.") + (environment + (list '()) + "Set environment variables. This can be a list of pairs or strings, even +mixed: + +@lisp +(list '(\"LANGUAGE\" . \"eo:ca:eu\") + \"JAVA_HOME=/opt/java\") +@end lisp + +String are passed directly to the Docker CLI. You can refer to the +@url{https://docs.docker.com/engine/reference/commandline/run/#env,upstream} +documentation for semantics." + (sanitizer oci-sanitize-environment)) + (image + (string) + "The image used to build the container. Images are resolved by the Docker +Engine, and follow the usual format +@code{myregistry.local:5000/testing/test-image:tag}.") + (name + (maybe-string) + "Set a name for the spawned container.") + (network + (maybe-string) + "Set a Docker network for the spawned container.") + (ports + (list '()) + "Set the port or port ranges to expose from the spawned container. This can +be a list of pairs or strings, even mixed: + +@lisp +(list '(\"8080\" . \"80\") + \"10443:443\") +@end lisp + +String are passed directly to the Docker CLI. You can refer to the +@url{https://docs.docker.com/engine/reference/commandline/run/#publish,upstream} +documentation for semantics." + (sanitizer oci-sanitize-ports)) + (volumes + (list '()) + "Set volume mappings for the spawned container. This can be a +list of pairs or strings, even mixed: + +@lisp +(list '(\"/root/data/grafana\" . \"/var/lib/grafana\") + \"/gnu/store:/gnu/store\") +@end lisp + +String are passed directly to the Docker CLI. You can refer to the +@url{https://docs.docker.com/engine/reference/commandline/run/#volume,upstream} +documentation for semantics." + (sanitizer oci-sanitize-volumes))) + +(define oci-container-configuration->options + (lambda (config) + (let ((entrypoint + (oci-container-configuration-entrypoint config)) + (network + (oci-container-configuration-network config))) + (apply append + (filter (compose not unspecified?) + `(,(if (maybe-value-set? entrypoint) + `("--entrypoint" ,entrypoint) + '()) + ,(append-map + (lambda (spec) + (list "--env" spec)) + (oci-container-configuration-environment config)) + ,(if (maybe-value-set? network) + `("--network" ,network) + '()) + ,(append-map + (lambda (spec) + (list "-p" spec)) + (oci-container-configuration-ports config)) + ,(append-map + (lambda (spec) + (list "-v" spec)) + (oci-container-configuration-volumes config)))))))) + +(define (oci-container-shepherd-service config) + (define (guess-name name image) + (if (maybe-value-set? name) + name + (string-append "docker-" + (basename (car (string-split image #\:)))))) + + (let* ((docker-command (file-append docker-cli "/bin/docker")) + (user (oci-container-configuration-user config)) + (group (oci-container-configuration-group config)) + (command (oci-container-configuration-command config)) + (config-name (oci-container-configuration-name config)) + (image (oci-container-configuration-image config)) + (options (oci-container-configuration->options config)) + (name (guess-name config-name image))) + + (shepherd-service (provision `(,(string->symbol name))) + (requirement '(dockerd user-processes)) + (respawn? #f) + (documentation + (string-append + "Docker backed Shepherd service for image: " image)) + (start + #~(make-forkexec-constructor + ;; docker run [OPTIONS] IMAGE [COMMAND] [ARG...] + (list #$docker-command "run" "--rm" + "--name" #$name + #$@options #$image #$@command) + #:user #$user + #:group #$group)) + (stop + #~(lambda _ + (invoke #$docker-command "rm" "-f" #$name))) + (actions + (list + (shepherd-action + (name 'pull) + (documentation + (format #f "Pull ~a's image (~a)." + name image)) + (procedure + #~(lambda _ + (invoke #$docker-command "pull" #$image))))))))) + +(define %oci-container-accounts + (list (user-account + (name "oci-container") + (comment "OCI services account") + (group "docker") + (system? #t) + (home-directory "/var/empty") + (shell (file-append shadow "/sbin/nologin"))))) + +(define (configs->shepherd-services configs) + (map oci-container-shepherd-service configs)) + +(define oci-container-service-type + (service-type (name 'oci-container) + (extensions (list (service-extension profile-service-type + (lambda _ (list docker-cli))) + (service-extension account-service-type + (const %oci-container-accounts)) + (service-extension shepherd-root-service-type + configs->shepherd-services))) + (default-value '()) + (extend append) + (compose concatenate) + (description + "This service allows the management of Docker and OCI +containers as Shepherd services.")))