From patchwork Sun May 12 10:38:02 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Ludovic_Court=C3=A8s?= X-Patchwork-Id: 13959 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id AF70616F30; Sun, 12 May 2019 11:39:13 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTP id 01D8516F30 for ; Sun, 12 May 2019 11:39:13 +0100 (BST) Received: from localhost ([127.0.0.1]:41517 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hPlsu-00076R-DA for patchwork@mira.cbaines.net; Sun, 12 May 2019 06:39:12 -0400 Received: from eggs.gnu.org ([209.51.188.92]:53196) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hPlsp-00071q-D1 for guix-patches@gnu.org; Sun, 12 May 2019 06:39:08 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hPlsn-0007XU-L3 for guix-patches@gnu.org; Sun, 12 May 2019 06:39:07 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:56640) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1hPlsn-0007Ws-GG for guix-patches@gnu.org; Sun, 12 May 2019 06:39:05 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1hPlsn-0002u0-CQ for guix-patches@gnu.org; Sun, 12 May 2019 06:39:05 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#35697] [PATCH 8/8] vm: 'system-docker-image' provides an entry point. Resent-From: Ludovic =?utf-8?q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sun, 12 May 2019 10:39:05 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 35697 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 35697@debbugs.gnu.org Received: via spool by 35697-submit@debbugs.gnu.org id=B35697.155765750911065 (code B ref 35697); Sun, 12 May 2019 10:39:05 +0000 Received: (at 35697) by debbugs.gnu.org; 12 May 2019 10:38:29 +0000 Received: from localhost ([127.0.0.1]:41944 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hPlsC-0002sK-LN for submit@debbugs.gnu.org; Sun, 12 May 2019 06:38:29 -0400 Received: from eggs.gnu.org ([209.51.188.92]:38687) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hPlsA-0002r6-Aj for 35697@debbugs.gnu.org; Sun, 12 May 2019 06:38:26 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:54356) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hPls5-0007K6-3m; Sun, 12 May 2019 06:38:21 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=41590 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1hPls4-00052w-HM; Sun, 12 May 2019 06:38:20 -0400 From: Ludovic =?utf-8?q?Court=C3=A8s?= Date: Sun, 12 May 2019 12:38:02 +0200 Message-Id: <20190512103802.17032-8-ludo@gnu.org> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190512103802.17032-1-ludo@gnu.org> References: <20190512103802.17032-1-ludo@gnu.org> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.51.188.43 X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Chris Marusich Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches This simplifies use of images created with 'guix system docker-image'. * gnu/system/vm.scm (system-docker-image)[boot-program]: New variable. [os]: Add it to the GC roots. [build]: Pass #:entry-point to 'build-docker-image'. * gnu/tests/docker.scm (run-docker-system-test): New procedure. (%test-docker-system): New variable. * doc/guix.texi (Invoking guix system): Remove GUIX_NEW_SYSTEM hack and '--entrypoint' from the example. Mention 'docker create', 'docker start', and 'docker exec'. --- doc/guix.texi | 18 ++++--- gnu/system/vm.scm | 18 ++++++- gnu/tests/docker.scm | 118 ++++++++++++++++++++++++++++++++++++++++++- 3 files changed, 145 insertions(+), 9 deletions(-) diff --git a/doc/guix.texi b/doc/guix.texi index df7208229c..da65fd8a4e 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -24497,20 +24497,26 @@ system configuration file. You can then load the image and launch a Docker container using commands like the following: @example -image_id="$(docker load < guix-system-docker-image.tar.gz)" -docker run -e GUIX_NEW_SYSTEM=/var/guix/profiles/system \\ - --entrypoint /var/guix/profiles/system/profile/bin/guile \\ - $image_id /var/guix/profiles/system/boot +image_id="`docker load < guix-system-docker-image.tar.gz`" +container_id="`docker create $image_id`" +docker start $container_id @end example This command starts a new Docker container from the specified image. It will boot the Guix system in the usual manner, which means it will start any services you have defined in the operating system -configuration. Depending on what you run in the Docker container, it +configuration. You can get an interactive shell running in the container +using @command{docker exec}: + +@example +docker exec -ti $container_id /run/current-system/profile/bin/bash --login +@end example + +Depending on what you run in the Docker container, it may be necessary to give the container additional permissions. For example, if you intend to build software using Guix inside of the Docker container, you may need to pass the @option{--privileged} option to -@code{docker run}. +@code{docker create}. @item container Return a script to run the operating system declared in @var{file} diff --git a/gnu/system/vm.scm b/gnu/system/vm.scm index 124abd0fc9..f3027cd4ca 100644 --- a/gnu/system/vm.scm +++ b/gnu/system/vm.scm @@ -473,7 +473,7 @@ the image." (define* (system-docker-image os #:key - (name "guixsd-docker-image") + (name "guix-docker-image") register-closures?) "Build a docker image. OS is the desired . NAME is the base name to use for the output file. When REGISTER-CLOSURES? is not #f, @@ -487,7 +487,19 @@ should set REGISTER-CLOSURES? to #f." (local-file (search-path %load-path "guix/store/schema.sql")))) - (let ((os (containerized-operating-system os '())) + (define boot-program + ;; Program that runs the boot script of OS, which in turn starts shepherd. + (program-file "boot-program" + #~(let ((system (cadr (command-line)))) + (setenv "GUIX_NEW_SYSTEM" system) + (execl #$(file-append guile-2.2 "/bin/guile") + "guile" "--no-auto-compile" + (string-append system "/boot"))))) + + + (let ((os (operating-system-with-gc-roots + (containerized-operating-system os '()) + (list boot-program))) (name (string-append name ".tar.gz")) (graph "system-graph")) (define build @@ -538,9 +550,11 @@ should set REGISTER-CLOSURES? to #f." (string-append "/xchg/" #$graph) read-reference-graph))) #$os + #:entry-point '(#$boot-program #$os) #:compressor '(#+(file-append gzip "/bin/gzip") "-9n") #:creation-time (make-time time-utc 0 1) #:transformations `((,root-directory -> "")))))))) + (expression->derivation-in-linux-vm name build #:make-disk-image? #f diff --git a/gnu/tests/docker.scm b/gnu/tests/docker.scm index 25e172efae..3cd3a27884 100644 --- a/gnu/tests/docker.scm +++ b/gnu/tests/docker.scm @@ -1,5 +1,6 @@ ;;; GNU Guix --- Functional package management for GNU ;;; Copyright © 2019 Danny Milosavljevic +;;; Copyright © 2019 Ludovic Courtès ;;; ;;; This file is part of GNU Guix. ;;; @@ -28,6 +29,7 @@ #:use-module (gnu services desktop) #:use-module (gnu packages bootstrap) ; %bootstrap-guile #:use-module (gnu packages docker) + #:use-module (gnu packages guile) #:use-module (guix gexp) #:use-module (guix grafts) #:use-module (guix monads) @@ -38,7 +40,8 @@ #:use-module (guix tests) #:use-module (guix build-system trivial) #:use-module ((guix licenses) #:prefix license:) - #:export (%test-docker)) + #:export (%test-docker + %test-docker-system)) (define %docker-os (simple-operating-system @@ -166,3 +169,116 @@ standard output device and then enters a new line.") (name "docker") (description "Test Docker container of Guix.") (value (build-tarball&run-docker-test)))) + + +(define (run-docker-system-test tarball) + "Load DOCKER-TARBALL as Docker image and run it in a Docker container, +inside %DOCKER-OS." + (define os + (marionette-operating-system + %docker-os + #:imported-modules '((gnu services herd) + (guix combinators)))) + + (define vm + (virtual-machine + (operating-system os) + ;; FIXME: Because we're using the volatile-root setup where the root file + ;; system is a tmpfs overlaid over a small root file system, 'docker + ;; load' must be able to store the whole image into memory, hence the + ;; huge memory requirements. We should avoid the volatile-root setup + ;; instead. + (memory-size 3000) + (port-forwardings '()))) + + (define test + (with-imported-modules '((gnu build marionette) + (guix build utils)) + #~(begin + (use-modules (srfi srfi-11) (srfi srfi-64) + (gnu build marionette) + (guix build utils)) + + (define marionette + (make-marionette (list #$vm))) + + (mkdir #$output) + (chdir #$output) + + (test-begin "docker") + + (test-assert "service running" + (marionette-eval + '(begin + (use-modules (gnu services herd)) + (match (start-service 'dockerd) + (#f #f) + (('service response-parts ...) + (match (assq-ref response-parts 'running) + ((pid) (number? pid)))))) + marionette)) + + (test-assert "load system image and run it" + (marionette-eval + `(begin + (define (slurp command . args) + ;; Return the output from COMMAND. + (let* ((port (apply open-pipe* OPEN_READ command args)) + (output (read-line port)) + (status (close-pipe port))) + output)) + + (define (docker-cli command . args) + ;; Run the given Docker COMMAND. + (apply invoke #$(file-append docker-cli "/bin/docker") + command args)) + + (define (wait-for-container-file container file) + ;; Wait for FILE to show up in CONTAINER. + (docker-cli "exec" container + #$(file-append guile-2.2 "/bin/guile") + "-c" + (object->string + `(let loop ((n 15)) + (when (zero? n) + (error "file didn't show up" ,file)) + (unless (file-exists? ,file) + (sleep 1) + (loop (- n 1))))))) + + (let* ((line (slurp #$(file-append docker-cli "/bin/docker") + "load" "-i" #$tarball)) + (repository&tag (string-drop line + (string-length + "Loaded image: "))) + (container (slurp + #$(file-append docker-cli "/bin/docker") + "create" repository&tag))) + (docker-cli "start" container) + + ;; Wait for shepherd to be ready. + (wait-for-container-file container + "/var/run/shepherd/socket") + + (docker-cli "exec" container + "/run/current-system/profile/bin/herd" + "status") + (slurp #$(file-append docker-cli "/bin/docker") + "exec" container + "/run/current-system/profile/bin/herd" + "status" "guix-daemon"))) + marionette)) + + (test-end) + (exit (= (test-runner-fail-count (test-runner-current)) 0))))) + + (gexp->derivation "docker-system-test" test)) + +(define %test-docker-system + (system-test + (name "docker-system") + (description "Run a system image as produced by @command{guix system +docker-image} inside Docker.") + (value (with-monad %store-monad + (>>= (system-docker-image (simple-operating-system)) + run-docker-system-test)))))