From patchwork Thu Dec 16 13:06:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Mathieu Othacehe X-Patchwork-Id: 35276 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 8B7BF27BBEA; Thu, 16 Dec 2021 13:17:21 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-3.7 required=5.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS, URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 43A1827BBE9 for ; Thu, 16 Dec 2021 13:17:20 +0000 (GMT) Received: from localhost ([::1]:58794 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mxqdL-0003cC-AV for patchwork@mira.cbaines.net; Thu, 16 Dec 2021 08:17:19 -0500 Received: from eggs.gnu.org ([209.51.188.92]:59950) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mxqUO-0005lr-8S for guix-patches@gnu.org; Thu, 16 Dec 2021 08:08:05 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:51689) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mxqUN-0004aj-UZ for guix-patches@gnu.org; Thu, 16 Dec 2021 08:08:03 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1mxqUM-0006fk-R6 for guix-patches@gnu.org; Thu, 16 Dec 2021 08:08:02 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#52550] [PATCH 04/10] system: image: Add docker support. Resent-From: Mathieu Othacehe Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Thu, 16 Dec 2021 13:08:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 52550 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 52550@debbugs.gnu.org Cc: Mathieu Othacehe Received: via spool by 52550-submit@debbugs.gnu.org id=B52550.163966002925481 (code B ref 52550); Thu, 16 Dec 2021 13:08:02 +0000 Received: (at 52550) by debbugs.gnu.org; 16 Dec 2021 13:07:09 +0000 Received: from localhost ([127.0.0.1]:34973 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mxqTU-0006ci-8R for submit@debbugs.gnu.org; Thu, 16 Dec 2021 08:07:09 -0500 Received: from eggs.gnu.org ([209.51.188.92]:58726) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mxqTS-0006bk-79 for 52550@debbugs.gnu.org; Thu, 16 Dec 2021 08:07:06 -0500 Received: from [2001:470:142:3::e] (port=33786 helo=fencepost.gnu.org) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mxqTM-0004OY-5v for 52550@debbugs.gnu.org; Thu, 16 Dec 2021 08:07:01 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:References:In-Reply-To:Date:Subject:To: From; bh=lP52trcD4uW/RqAL8mREiI0HFJol4baDRCCb50wGyaY=; b=XaVlg7O6yzqpyE/ITXac 6aQr13XMs4J+CYePMdNVCgfCIkZm2ijy15sA85gL/3nDHMMBpxC7pUkWbG5NySL39uJC7IsduA0lm wn5dgRUGN2V7dyXAUehQy0FBjvS0uwgRC+X5D1oNiqdjKeCz6Sq24RT/TixBSyDHOD8OeiLsgH156 ImxGxXo0pRfaYlO9AoByJXIU5S1SE17wonDhsDuVO6ThITiP498w6w1dmBeDX6AkK8njyxBNG0AUF tuf0gJ9QZTNJrvP0W5+KVusTdjdDrnGy7/mWBpn6fveR5uZpXV7VivAmQhaXgAVPmDDK8Vk2NituR 42l61WhDVC1hfg==; Received: from [2a01:e0a:19b:d9a0:2f3b:16f2:b776:3ef9] (port=57550 helo=localhost.localdomain) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mxqTK-0003iQ-6e; Thu, 16 Dec 2021 08:07:00 -0500 From: Mathieu Othacehe Date: Thu, 16 Dec 2021 14:06:43 +0100 Message-Id: <20211216130649.30285-4-othacehe@gnu.org> X-Mailer: git-send-email 2.34.0 In-Reply-To: <20211216130649.30285-1-othacehe@gnu.org> References: <20211216130649.30285-1-othacehe@gnu.org> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches * gnu/system/image.scm (docker-image, docker-image-type): New variables. (system-docker-image): New procedure. (image->root-file-system): Add docker image support. (system-image): Ditto. --- gnu/system/image.scm | 125 +++++++++++++++++++++++++++++++++++++++---- 1 file changed, 116 insertions(+), 9 deletions(-) diff --git a/gnu/system/image.scm b/gnu/system/image.scm index 4b6aaf2e32..42e215f614 100644 --- a/gnu/system/image.scm +++ b/gnu/system/image.scm @@ -1,5 +1,5 @@ ;;; GNU Guix --- Functional package management for GNU -;;; Copyright © 2020 Mathieu Othacehe +;;; Copyright © 2020, 2021 Mathieu Othacehe ;;; Copyright © 2020 Jan (janneke) Nieuwenhuizen ;;; ;;; This file is part of GNU Guix. @@ -36,12 +36,14 @@ (define-module (gnu system image) #:use-module (gnu services base) #:use-module (gnu system) #:use-module (gnu system file-systems) + #:use-module (gnu system linux-container) #:use-module (gnu system uuid) #:use-module (gnu system vm) #:use-module (guix packages) #:use-module (gnu packages base) #:use-module (gnu packages bootloaders) #:use-module (gnu packages cdrom) + #:use-module (gnu packages compression) #:use-module (gnu packages disk) #:use-module (gnu packages gawk) #:use-module (gnu packages genimage) @@ -67,6 +69,7 @@ (define-module (gnu system image) efi-disk-image iso9660-image + docker-image raw-with-offset-disk-image image-with-os @@ -74,6 +77,7 @@ (define-module (gnu system image) qcow2-image-type iso-image-type uncompressed-iso-image-type + docker-image-type raw-with-offset-image-type image-with-label @@ -127,6 +131,10 @@ (define iso9660-image (label "GUIX_IMAGE") (flags '(boot))))))) +(define docker-image + (image + (format 'docker))) + (define* (raw-with-offset-disk-image #:optional (offset root-offset)) (image (format 'disk-image) @@ -179,6 +187,11 @@ (define uncompressed-iso-image-type (compression? #f)) <>)))) +(define docker-image-type + (image-type + (name 'docker) + (constructor (cut image-with-os docker-image <>)))) + (define raw-with-offset-image-type (image-type (name 'raw-with-offset) @@ -220,8 +233,7 @@ (define gcrypt-sqlite3&co (define-syntax-rule (with-imported-modules* gexp* ...) (with-extensions gcrypt-sqlite3&co (with-imported-modules `(,@(source-module-closure - '((gnu build vm) - (gnu build image) + '((gnu build image) (gnu build bootloader) (gnu build hurd-boot) (gnu build linux-boot) @@ -229,8 +241,7 @@ (define-syntax-rule (with-imported-modules* gexp* ...) #:select? not-config?) ((guix config) => ,(make-config.scm))) #~(begin - (use-modules (gnu build vm) - (gnu build image) + (use-modules (gnu build image) (gnu build bootloader) (gnu build hurd-boot) (gnu build linux-boot) @@ -337,6 +348,8 @@ (define (partition-image partition) (initializer image-root #:references-graphs '#$graph #:deduplicate? #f + #:copy-closures? (not + #$(image-shared-store? image)) #:system-directory #$os #:grub-efi #+grub-efi #:bootloader-package @@ -527,6 +540,97 @@ (define (image-with-label base-image label) (label label)) others)))))) + +;; +;; Docker image. +;; + +(define* (system-docker-image image + #:key + (name "docker-image")) + "Build a docker image for IMAGE. NAME is the base name to use for the +output file." + (define boot-program + ;; Program that runs the boot script of OS, which in turn starts shepherd. + (program-file "boot-program" + #~(let ((system (cadr (command-line)))) + (setenv "GUIX_NEW_SYSTEM" system) + (execl #$(file-append guile-3.0 "/bin/guile") + "guile" "--no-auto-compile" + (string-append system "/boot"))))) + + (define shared-network? + (image-shared-network? image)) + + (let* ((os (operating-system-with-gc-roots + (containerized-operating-system + (image-operating-system image) '() + #:shared-network? + shared-network?) + (list boot-program))) + (substitutable? (image-substitutable? image)) + (register-closures? (has-guix-service-type? os)) + (schema (and register-closures? + (local-file (search-path %load-path + "guix/store/schema.sql")))) + (name (string-append name ".tar.gz")) + (graph "system-graph")) + (define builder + (with-extensions (cons guile-json-3 ;for (guix docker) + gcrypt-sqlite3&co) ;for (guix store database) + (with-imported-modules `(,@(source-module-closure + '((guix docker) + (guix store database) + (guix build utils) + (guix build store-copy) + (gnu build image)) + #:select? not-config?) + ((guix config) => ,(make-config.scm))) + #~(begin + (use-modules (guix docker) + (guix build utils) + (gnu build image) + (srfi srfi-19) + (guix build store-copy) + (guix store database)) + + ;; Set the SQL schema location. + (sql-schema #$schema) + + ;; Allow non-ASCII file names--e.g., 'nss-certs'--to be decoded. + (setenv "GUIX_LOCPATH" + #+(file-append glibc-utf8-locales "/lib/locale")) + (setlocale LC_ALL "en_US.utf8") + + (set-path-environment-variable "PATH" '("bin" "sbin") '(#+tar)) + + (let ((image-root (string-append (getcwd) "/tmp-root"))) + (mkdir-p image-root) + (initialize-root-partition image-root + #:references-graphs '(#$graph) + #:copy-closures? #f + #:register-closures? #$register-closures? + #:deduplicate? #f + #:system-directory #$os) + (build-docker-image + #$output + (cons* image-root + (map store-info-item + (call-with-input-file #$graph + read-reference-graph))) + #$os + #:entry-point '(#$boot-program #$os) + #:compressor '(#+(file-append gzip "/bin/gzip") "-9n") + #:creation-time (make-time time-utc 0 1) + #:transformations `((,image-root -> "")))))))) + + (computed-file name builder + ;; Allow offloading so that this I/O-intensive process + ;; doesn't run on the build farm's head node. + #:local-build? #f + #:options `(#:references-graphs ((,graph ,os)) + #:substitutable? ,substitutable?)))) + ;; ;; Image creation. @@ -534,10 +638,11 @@ (define (image-with-label base-image label) (define (image->root-file-system image) "Return the IMAGE root partition file-system type." - (let ((format (image-format image))) - (if (eq? format 'iso9660) - "iso9660" - (partition-file-system (find-root-partition image))))) + (case (image-format image) + ((iso9660) "iso9660") + ((docker) "dummy") + (else + (partition-file-system (find-root-partition image))))) (define (root-size image) "Return the root partition size of IMAGE." @@ -671,6 +776,8 @@ (define target (cond #:register-closures? register-closures? #:inputs `(("system" ,os) ("bootcfg" ,bootcfg)))) + ((memq image-format '(docker)) + (system-docker-image image*)) ((memq image-format '(iso9660)) (system-iso9660-image image*