From patchwork Thu Jan 11 20:39:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Giacomo Leidi X-Patchwork-Id: 58803 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 387E727BBE9; Thu, 11 Jan 2024 20:41:41 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-3.7 required=5.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H5,RCVD_IN_MSPIKE_WL, SPF_HELO_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id BECF227BBE2 for ; Thu, 11 Jan 2024 20:41:38 +0000 (GMT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rO1re-0004tf-F3; Thu, 11 Jan 2024 15:41:22 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rO1rO-0004qF-TQ for guix-patches@gnu.org; Thu, 11 Jan 2024 15:41:08 -0500 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rO1rO-0006Cu-Ij for guix-patches@gnu.org; Thu, 11 Jan 2024 15:41:06 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1rO1rM-0002m2-4M for guix-patches@gnu.org; Thu, 11 Jan 2024 15:41:04 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#67613] [PATCH v2 5/5] gnu: Add tests and documentation for oci-container-service-type. Resent-From: Giacomo Leidi Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Thu, 11 Jan 2024 20:41:04 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 67613 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: To: 67613@debbugs.gnu.org Cc: Giacomo Leidi Received: via spool by 67613-submit@debbugs.gnu.org id=B67613.170500562210550 (code B ref 67613); Thu, 11 Jan 2024 20:41:04 +0000 Received: (at 67613) by debbugs.gnu.org; 11 Jan 2024 20:40:22 +0000 Received: from localhost ([127.0.0.1]:34048 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rO1qf-0002k4-KO for submit@debbugs.gnu.org; Thu, 11 Jan 2024 15:40:22 -0500 Received: from confino.investici.org ([2a11:7980:1::2:0]:42745) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rO1qY-0002j8-7o for 67613@debbugs.gnu.org; Thu, 11 Jan 2024 15:40:15 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=autistici.org; s=stigmate; t=1705005616; bh=6boZOwO4I0ZIE1UTL8wcix2bNKAFfYQcrb52wh6MITQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=e99uyPf8aVlccV9GJ4FpZWqzFTk9z6L9SNVhDUznE7e7URQ98i2YlyptKi92L83Hm k8Fr2zJNHbonfq1o0WKZZWvUp7NjBu++pO+W9lma9YEwS2924IQ9OuI68Vac9fgFcD F1PSIsH6cr7jf/3q1bm+YwOtfsenY8XnPtMuenkk= Received: from mx1.investici.org (unknown [127.0.0.1]) by confino.investici.org (Postfix) with ESMTP id 4T9xPX04msz1135; Thu, 11 Jan 2024 20:40:16 +0000 (UTC) Received: from [93.190.126.19] (mx1.investici.org [93.190.126.19]) (Authenticated sender: goodoldpaul@autistici.org) by localhost (Postfix) with ESMTPSA id 4T9xPW3n5Dz10w5; Thu, 11 Jan 2024 20:40:15 +0000 (UTC) Date: Thu, 11 Jan 2024 21:39:53 +0100 Message-ID: <20240111203954.29335-5-goodoldpaul@autistici.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20240111203954.29335-1-goodoldpaul@autistici.org> References: <20240111203954.29335-1-goodoldpaul@autistici.org> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-to: Giacomo Leidi X-ACL-Warn: , Giacomo Leidi via Guix-patches X-Patchwork-Original-From: Giacomo Leidi via Guix-patches via From: Giacomo Leidi Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org X-getmail-retrieved-from-mailbox: Patches * doc/guix.texi: Add documentation for the oci-image record and update the oci-container-configuration documentation. * gnu/tests/docker.scm (run-oci-container-test): New variable; (%test-oci-container): new variable. Change-Id: Id8f4f5454aa3b88d8aa3fa47de823e921acece05 --- doc/guix.texi | 91 +++++++++++++++++++++++++++- gnu/services/docker.scm | 6 +- gnu/tests/docker.scm | 131 +++++++++++++++++++++++++++++++++++++++- 3 files changed, 221 insertions(+), 7 deletions(-) diff --git a/doc/guix.texi b/doc/guix.texi index ce239c603d..1916a00412 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -39790,6 +39790,17 @@ processes as Shepherd Services. @lisp (service oci-container-service-type (list + (oci-container-configuration + (image + (oci-image + (repository "guile") + (tag "3") + (value (specifications->manifest '("guile"))) + (pack-options '(#:symlinks (("/bin/guile" -> "bin/guile")) + #:max-layers 2)))) + (entrypoint "/bin/guile") + (command + '("-c" "(display \"hello!\n\")"))) (oci-container-configuration (image "prom/prometheus") (network "host") @@ -39836,6 +39847,23 @@ Overwrite the default command (@code{CMD}) of the image. @item @code{entrypoint} (default: @code{""}) (type: string) Overwrite the default entrypoint (@code{ENTRYPOINT}) of the image. +@item @code{host-environment} (default: @code{()}) (type: list) +Set environment variables in the host environment where @command{docker +run} is invoked. This is especially useful to pass secrets from the +host to the container without having them on the @command{docker run}'s +command line: by setting the @code{MYSQL_PASSWORD} on the host and by passing +@code{--env MYSQL_PASSWORD} through the @code{extra-arguments} field, it is +possible to securely set values in the container environment. This field's +value can be a list of pairs or strings, even mixed: + +@lisp +(list '(\"LANGUAGE\" . \"eo:ca:eu\") + \"JAVA_HOME=/opt/java\") +@end lisp + +Pair members can be strings, gexps or file-like objects. Strings are passed +directly to @code{make-forkexec-constructor}. + @item @code{environment} (default: @code{()}) (type: list) Set environment variables. This can be a list of pairs or strings, even mixed: @@ -39849,14 +39877,19 @@ Strings are passed directly to the Docker CLI. You can refer to the @uref{https://docs.docker.com/engine/reference/commandline/run/#env,upstream} documentation for semantics. -@item @code{image} (type: string) -The image used to build the container. Images are resolved by the -Docker Engine, and follow the usual format +@item @code{image} (type: string-or-oci-image) +The image used to build the container. It can be a string or an +@code{oci-image} record. Strings are resolved by the Docker Engine, and +follow the usual format @code{myregistry.local:5000/testing/test-image:tag}. @item @code{provision} (default: @code{""}) (type: string) Set the name of the provisioned Shepherd service. +@item @code{requirement} (default: @code{()}) (type: list-of-symbols) +Set additional Shepherd services dependencies to the provisioned +Shepherd service. + @item @code{network} (default: @code{""}) (type: string) Set a Docker network for the spawned container. @@ -39908,6 +39941,58 @@ passed to the @command{docker run} invokation. @end deftp +@c %end of fragment + +@c %start of fragment + +@deftp {Data Type} oci-image +Available @code{oci-image} fields are: + +@table @asis +@item @code{repository} (type: string) +A string like @code{myregistry.local:5000/testing/test-image} that names +the OCI image. + +@item @code{tag} (default: @code{"latest"}) (type: string) +A string representing the OCI image tag. Defaults to @code{latest}. + +@item @code{value} (type: oci-lowerable-image) +A @code{manifest} or @code{operating-system} record that will be lowered +into an OCI compatible tarball. Otherwise this field's value can be a +gexp or a file-like object that evaluates to an OCI compatible tarball. + +@item @code{pack-options} (default: @code{()}) (type: list) +An optional set of keyword arguments that will be passed to the +@code{docker-image} procedure from @code{guix scripts pack}. They can +be used to replicate @command{guix pack} behavior: + +@lisp +(oci-image + (repository "guile") + (tag "3") + (value + (specifications->manifest '("guile"))) + (pack-options '(#:symlinks (("/bin/guile" -> "bin/guile")) + #:max-layers 2))) +@end lisp + +If the @code{value} field is an @code{operating-system} record, this field's +value will be ignored. + +@item @code{system} (default: @code{""}) (type: string) +Attempt to build for a given system, e.g. "i686-linux" + +@item @code{target} (default: @code{""}) (type: string) +Attempt to cross-build for a given triple, e.g. "aarch64-linux-gnu" + +@item @code{grafts?} (default: @code{#f}) (type: boolean) +Whether to allow grafting or not in the pack build. + +@end table + +@end deftp + + @c %end of fragment @cindex Audit diff --git a/gnu/services/docker.scm b/gnu/services/docker.scm index 58a725737c..7aff8dcc5f 100644 --- a/gnu/services/docker.scm +++ b/gnu/services/docker.scm @@ -420,7 +420,7 @@ (define-configuration/no-serialization oci-container-configuration "Set environment variables in the host environment where @command{docker run} is invoked. This is especially useful to pass secrets from the host to the container without having them on the @command{docker run}'s command line: by -setting the @{MYSQL_PASSWORD} on the host and by passing +setting the @code{MYSQL_PASSWORD} on the host and by passing @code{--env MYSQL_PASSWORD} through the @code{extra-arguments} field, it is possible to securely set values in the container environment. This field's value can be a list of pairs or strings, even mixed: @@ -435,8 +435,8 @@ (define-configuration/no-serialization oci-container-configuration (sanitizer oci-sanitize-host-environment)) (environment (list '()) - "Set environment variables. This can be a list of pairs or strings, even -mixed: + "Set environment variables inside the container. This can be a list of pairs +or strings, even mixed: @lisp (list '(\"LANGUAGE\" . \"eo:ca:eu\") diff --git a/gnu/tests/docker.scm b/gnu/tests/docker.scm index 9e9d2e2d07..d550136b4a 100644 --- a/gnu/tests/docker.scm +++ b/gnu/tests/docker.scm @@ -1,6 +1,7 @@ ;;; GNU Guix --- Functional package management for GNU ;;; Copyright © 2019 Danny Milosavljevic ;;; Copyright © 2019-2023 Ludovic Courtès +;;; Copyright © 2024 Giacomo Leidi ;;; ;;; This file is part of GNU Guix. ;;; @@ -29,6 +30,7 @@ (define-module (gnu tests docker) #:use-module (gnu services networking) #:use-module (gnu services docker) #:use-module (gnu services desktop) + #:use-module (gnu packages) #:use-module ((gnu packages base) #:select (glibc)) #:use-module (gnu packages guile) #:use-module (gnu packages docker) @@ -43,7 +45,8 @@ (define-module (gnu tests docker) #:use-module (guix build-system trivial) #:use-module ((guix licenses) #:prefix license:) #:export (%test-docker - %test-docker-system)) + %test-docker-system + %test-oci-container)) (define %docker-os (simple-operating-system @@ -316,3 +319,129 @@ (define %test-docker-system (locale-libcs (list glibc))) #:type docker-image-type))) run-docker-system-test))))) + + +(define %oci-os + (simple-operating-system + (service dhcp-client-service-type) + (service dbus-root-service-type) + (service polkit-service-type) + (service elogind-service-type) + (service docker-service-type) + (extra-special-file "/shared.txt" + (plain-file "shared.txt" "hello")) + (service oci-container-service-type + (list + (oci-container-configuration + (image + (oci-image + (repository "guile") + (value + (specifications->manifest '("guile"))) + (pack-options + '(#:symlinks (("/bin" -> "bin")))))) + (entrypoint + "/bin/guile") + (command + '("-c" "(let l ((c 300))(display c)(sleep 1)(when(positive? c)(l (- c 1))))")) + (host-environment + '(("VARIABLE" . "value"))) + (volumes + '(("/shared.txt" . "/shared.txt:ro"))) + (extra-arguments + '("--env" "VARIABLE"))))))) + +(define (run-oci-container-test) + "Run IMAGE as an OCI backed Shepherd service, inside OS." + + (define os + (marionette-operating-system + (operating-system-with-gc-roots + %oci-os + (list)) + #:imported-modules '((gnu services herd) + (guix combinators)))) + + (define vm + (virtual-machine + (operating-system os) + (volatile? #f) + (memory-size 1024) + (disk-image-size (* 3000 (expt 2 20))) + (port-forwardings '()))) + + (define test + (with-imported-modules '((gnu build marionette)) + #~(begin + (use-modules (srfi srfi-11) (srfi srfi-64) + (gnu build marionette)) + + (define marionette + ;; Relax timeout to accommodate older systems and + ;; allow for pulling the image. + (make-marionette (list #$vm) #:timeout 60)) + + (test-runner-current (system-test-runner #$output)) + (test-begin "oci-container") + + (test-assert "dockerd running" + (marionette-eval + '(begin + (use-modules (gnu services herd)) + (match (start-service 'dockerd) + (#f #f) + (('service response-parts ...) + (match (assq-ref response-parts 'running) + ((pid) (number? pid)))))) + marionette)) + + (sleep 10) ; let service start + + (test-assert "docker-guile running" + (marionette-eval + '(begin + (use-modules (gnu services herd)) + (match (start-service 'docker-guile) + (#f #f) + (('service response-parts ...) + (match (assq-ref response-parts 'running) + ((pid) (number? pid)))))) + marionette)) + + (test-equal "passing host environment variables and volumes" + '("value" "hello") + (marionette-eval + `(begin + (use-modules (ice-9 popen) + (ice-9 rdelim)) + + (define slurp + (lambda args + (let* ((port (apply open-pipe* OPEN_READ args)) + (output (let ((line (read-line port))) + (if (eof-object? line) + "" + line))) + (status (close-pipe port))) + output))) + (let* ((response1 (slurp + ,(string-append #$docker-cli "/bin/docker") + "exec" "docker-guile" + "/bin/guile" "-c" "(display (getenv \"VARIABLE\"))")) + (response2 (slurp + ,(string-append #$docker-cli "/bin/docker") + "exec" "docker-guile" + "/bin/guile" "-c" "(begin (use-modules (ice-9 popen) (ice-9 rdelim)) +(display (call-with-input-file \"/shared.txt\" read-line)))"))) + (list response1 response2))) + marionette)) + + (test-end)))) + + (gexp->derivation "oci-container-test" test)) + +(define %test-oci-container + (system-test + (name "oci-container") + (description "Test OCI backed Shepherd service.") + (value (run-oci-container-test))))