From patchwork Tue Dec 26 02:18:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Oleg Pykhalov X-Patchwork-Id: 58074 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 5C69127BBEA; Tue, 26 Dec 2023 02:21:15 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FROM,MAILING_LIST_MULTI, SPF_HELO_PASS autolearn=unavailable autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 2922E27BBE2 for ; Tue, 26 Dec 2023 02:21:14 +0000 (GMT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rHx3z-0006z6-N2; Mon, 25 Dec 2023 21:20:59 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rHx3x-0006yi-Pr for guix-patches@gnu.org; Mon, 25 Dec 2023 21:20:57 -0500 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rHx3x-0007hi-HI; Mon, 25 Dec 2023 21:20:57 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1rHx42-0000mh-MK; Mon, 25 Dec 2023 21:21:02 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#62153] [PATCH 1/5] guix: pack: Add '--entry-point-argument' option. Resent-From: Oleg Pykhalov Original-Sender: "Debbugs-submit" Resent-CC: guix@cbaines.net, dev@jpoiret.xyz, ludo@gnu.org, othacehe@gnu.org, rekado@elephly.net, zimon.toutoune@gmail.com, me@tobias.gr, guix-patches@gnu.org Resent-Date: Tue, 26 Dec 2023 02:21:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 62153 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 62153@debbugs.gnu.org Cc: Oleg Pykhalov , Graham James Addis , Christopher Baines , Josselin Poiret , Ludovic =?utf-8?q?Court=C3=A8s?= , Mathieu Othacehe , Ricardo Wurmus , Simon Tournier , Tobias Geerinckx-Rice X-Debbugs-Original-Xcc: Christopher Baines , Josselin Poiret , Ludovic =?utf-8?q?Court=C3=A8s?= , Mathieu Othacehe , Ricardo Wurmus , Simon Tournier , Tobias Geerinckx-Rice Received: via spool by 62153-submit@debbugs.gnu.org id=B62153.17035572022867 (code B ref 62153); Tue, 26 Dec 2023 02:21:02 +0000 Received: (at 62153) by debbugs.gnu.org; 26 Dec 2023 02:20:02 +0000 Received: from localhost ([127.0.0.1]:55609 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rHx33-0000jz-88 for submit@debbugs.gnu.org; Mon, 25 Dec 2023 21:20:02 -0500 Received: from mail-lj1-x231.google.com ([2a00:1450:4864:20::231]:39939) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rHx30-0000jh-JW for 62153@debbugs.gnu.org; Mon, 25 Dec 2023 21:20:00 -0500 Received: by mail-lj1-x231.google.com with SMTP id 38308e7fff4ca-2ccb6ddf651so5292681fa.0 for <62153@debbugs.gnu.org>; Mon, 25 Dec 2023 18:19:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1703557185; x=1704161985; darn=debbugs.gnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Hv0rEVoBjazJDcZYBpF+gk3yoNxlURtQzepwVar2VuU=; b=mG4n0XE7QK49MGBrg+B/EzRH9VkUYzn6+Fnh6ZV9vD7/m6luUhT6l3X0OKAi1IFBxZ oYlaywrCtLlFWvRlhyBAQ++mXlXlRZ16icongN9XmGAUHzJp9GPy5f24bmgE5s59PCUx JUz9mqb6zV3prvaiofY0rpGT18E9eTpE/Op7Ad7B1kyoJQjh0eCU1qCHnhxXGuDWsEoA LjlHHz6xsB0Rt0oz52BbnSxifPu75q5fe8TKfSAAGrCBrnxjq0sY5GjthyRyxJGjrLmw PpQMMAb9xBOzgUJu87r6nbbeFXjrmFZOu7taFcWJKD88dk35ywmx/FqctOCEE9xvVSkT TeQg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1703557185; x=1704161985; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Hv0rEVoBjazJDcZYBpF+gk3yoNxlURtQzepwVar2VuU=; b=CeN3ve6uxjl9RvkZP3kRHKZYyFEnCRubzlAbdhYplpWWjsQoxLbwNKxCdcGrvipM7Y gO3fOzre+U45VhGhoEuuWH0y9752zauo7xa52/7Z2NrdgRfKZZOxk6ag2/UjQVgIaWSA dCqrHlUrwPq+5cAtzFP6TtelYpy/UsnIm2VvMs1jazw8U4CwYsllKKbVqJwNN2uaLoY7 X2lpb1IYEOjPucTnIU/unzz9lzTjXYiaq81t56MZDX37dkx0c8M2hLXCxteEZkKgWJE1 KJoR+5H2my5kslh3tWHUVwU3E/TkfJw4mZiQZaQkwQl4ugIK1/oHUKmqdE3J1lWtQckh GG2Q== X-Gm-Message-State: AOJu0YzeUZaRlbDIE8wB/QTDYx6lrcgPLsThAFC4ZLRS//LbyLUDCd27 xAiTzp/XgGNfhD2nHr3Sm5uFHBadI0o= X-Google-Smtp-Source: AGHT+IHFFetgLYy48aec+x5VrKfcF3LG25sJhYmRI+fZj2xsKHhXYI9eM+a26dsCpmSmezfECQQhIg== X-Received: by 2002:a05:651c:1187:b0:2cc:6ce9:b927 with SMTP id w7-20020a05651c118700b002cc6ce9b927mr4134558ljo.1.1703557184654; Mon, 25 Dec 2023 18:19:44 -0800 (PST) Received: from guixsd.wugi.info ([88.201.161.72]) by smtp.gmail.com with ESMTPSA id e11-20020a05651c038b00b002cca609ef8bsm1633436ljp.111.2023.12.25.18.19.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 25 Dec 2023 18:19:44 -0800 (PST) From: Oleg Pykhalov Date: Tue, 26 Dec 2023 05:18:53 +0300 Message-ID: <9e2ecba897127f31c8c6fedd9ce20b6cdc0be4a2.1703556298.git.go.wigust@gmail.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: References: MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org X-getmail-retrieved-from-mailbox: Patches From: Graham James Addis * guix/scripts/pack.scm: (entry-point-argument-spec-option-parser): New procedure. (docker-image, %default-options, %docker-format-options, show-docker-format-options/detailed, %options, show-docker-format-options, guix-pack): Handle '--entry-point-argument' option. * doc/guix.texi: (Invoking guix pack): Document this Signed-off-by: Oleg Pykhalov Change-Id: I1124feff6af39dcc63c85fd6cc7ad50f398489dc --- doc/guix.texi | 14 +++++++++++- guix/scripts/pack.scm | 50 +++++++++++++++++++++++++++++++++++-------- 2 files changed, 54 insertions(+), 10 deletions(-) diff --git a/doc/guix.texi b/doc/guix.texi index 76b4eae67f..cca250dc31 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -122,6 +122,7 @@ Copyright @copyright{} 2023 Foundation Devices, Inc.@* Copyright @copyright{} 2023 Thomas Ieong@* Copyright @copyright{} 2023 Saku Laesvuori@* +Copyright @copyright{} 2023 Graham James Addis@* Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or @@ -7406,7 +7407,7 @@ Invoking guix pack @env{GUIX_EXECUTION_ENGINE} environment variable accordingly. @end quotation -@cindex entry point, for Docker images +@cindex entry point, for Docker and Singularity images @item --entry-point=@var{command} Use @var{command} as the @dfn{entry point} of the resulting pack, if the pack format supports it---currently @code{docker} and @code{squashfs} (Singularity) @@ -7429,6 +7430,17 @@ Invoking guix pack docker run @var{image-id} @end example +@cindex entry point arguments, for docker images +@item --entry-point-argument=@var{command} +@itemx -A @var{command} +Use @var{command} as an argument to @dfn{entry point} of the resulting pack. +This option is only valid in conjunction with @code{--entry-point} and can +appear multiple times on the command line. + +@example +guix pack -f docker --entry-point=bin/guile --entry-point-argument="--help" guile +@end example + @item --expression=@var{expr} @itemx -e @var{expr} Consider the package @var{expr} evaluates to. diff --git a/guix/scripts/pack.scm b/guix/scripts/pack.scm index 8071840de1..4c0a602eb1 100644 --- a/guix/scripts/pack.scm +++ b/guix/scripts/pack.scm @@ -8,6 +8,7 @@ ;;; Copyright © 2020, 2021, 2022, 2023 Maxim Cournoyer ;;; Copyright © 2020 Eric Bavier ;;; Copyright © 2022 Alex Griffin +;;; Copyright © 2023 Graham James Addis ;;; ;;; This file is part of GNU Guix. ;;; @@ -202,6 +203,16 @@ (define (symlink-spec-option-parser opt name arg result) (leave (G_ "~a: invalid symlink specification~%") arg)))) +(define (entry-point-argument-spec-option-parser opt name arg result) + "A SRFI-37 opion parser for the --entry-point-argument option. The spec +takes multiple occurances. The entries are used in the exec form for the +docker entry-point. The values are used as parameters in conjunction with +the --entry-point option which is used as the first value in the exec form." + (let ((entry-point-argument (assoc-ref result 'entry-point-argument))) + (alist-cons 'entry-point-argument + (append entry-point-argument (list arg)) + (alist-delete 'entry-point-argument result eq?)))) + (define (set-utf8-locale profile) "Configure the environment to use the \"en_US.utf8\" locale provided by the GLIBC-UT8-LOCALES package." @@ -562,10 +573,22 @@ (define* (docker-image name profile `((directory "/tmp" ,(getuid) ,(getgid) #o1777) ,@(append-map symlink->directives '#$symlinks))) + (define (form-entry-point prefix entry-point entry-point-argument) + ;; Construct entry-point parameter for build-docker-image. The + ;; first entry is constructed by prefixing the entry-point with + ;; the supplied index subsequent entries are taken from the + ;; --entry-point-argument options. + (and=> entry-point + (lambda (entry-point) + (cons* (string-append prefix "/" entry-point) + entry-point-argument)))) + (setenv "PATH" #+(file-append archiver "/bin")) (let-keywords '#$extra-options #f - ((image-tag #f)) + ((image-tag #f) + (entry-point-argument #f)) + (build-docker-image #$output (map store-info-item (call-with-input-file "profile" @@ -578,11 +601,10 @@ (define* (docker-image name profile #:database #+database #:system (or #$target %host-type) #:environment environment - #:entry-point - #$(and entry-point - #~(list - (string-append #$profile "/" - #$entry-point))) + #:entry-point (form-entry-point + #$profile + #$entry-point + entry-point-argument) #:extra-files directives #:compressor #+(compressor-command compressor) @@ -1264,6 +1286,7 @@ (define %default-options (debug . 0) (verbosity . 1) (symlinks . ()) + (entry-point-argument . ()) (compressor . ,(first %compressors)))) (define %formats @@ -1299,7 +1322,9 @@ (define (required-option symbol) rest)))) (define %docker-format-options - (list (required-option 'image-tag))) + (list (required-option 'image-tag) + (option '(#\A "entry-point-argument") #t #f + entry-point-argument-spec-option-parser))) (define (show-docker-format-options) (display (G_ " @@ -1308,7 +1333,12 @@ (define (show-docker-format-options) (define (show-docker-format-options/detailed) (display (G_ " --image-tag=NAME - Use the given NAME for the Docker image repository")) + Use the given NAME for the Docker image repository + + -A, --entry-point-argument=COMMAND/PARAMETER + Value(s) to use for the Docker EntryPoint arguments. + Multiple instances are accepted. This is only valid + in conjunction with the --entry-point option")) (newline) (exit 0)) @@ -1619,7 +1649,9 @@ (define-command (guix-pack . args) (extra-options (match pack-format ('docker (list #:image-tag - (assoc-ref opts 'image-tag))) + (assoc-ref opts 'image-tag) + #:entry-point-argument + (assoc-ref opts 'entry-point-argument))) ('deb (list #:control-file (process-file-arg opts 'control-file) From patchwork Tue Dec 26 02:18:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oleg Pykhalov X-Patchwork-Id: 58076 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 5145B27BBEA; Tue, 26 Dec 2023 02:21:28 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FROM,MAILING_LIST_MULTI, SPF_HELO_PASS autolearn=unavailable autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 40D4D27BBE2 for ; Tue, 26 Dec 2023 02:21:24 +0000 (GMT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rHx3y-0006yq-Fy; Mon, 25 Dec 2023 21:20:58 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rHx3w-0006yY-QM for guix-patches@gnu.org; Mon, 25 Dec 2023 21:20:56 -0500 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rHx3w-0007hc-IT for guix-patches@gnu.org; Mon, 25 Dec 2023 21:20:56 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1rHx43-0000mn-4v for guix-patches@gnu.org; Mon, 25 Dec 2023 21:21:03 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#62153] [PATCH 2/5] tests: docker-system: Increase image size. Resent-From: Oleg Pykhalov Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Tue, 26 Dec 2023 02:21:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 62153 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 62153@debbugs.gnu.org Cc: Oleg Pykhalov Received: via spool by 62153-submit@debbugs.gnu.org id=B62153.17035572082917 (code B ref 62153); Tue, 26 Dec 2023 02:21:03 +0000 Received: (at 62153) by debbugs.gnu.org; 26 Dec 2023 02:20:08 +0000 Received: from localhost ([127.0.0.1]:55614 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rHx3A-0000ky-90 for submit@debbugs.gnu.org; Mon, 25 Dec 2023 21:20:08 -0500 Received: from mail-lj1-x233.google.com ([2a00:1450:4864:20::233]:41104) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rHx31-0000jk-Qv for 62153@debbugs.gnu.org; Mon, 25 Dec 2023 21:20:03 -0500 Received: by mail-lj1-x233.google.com with SMTP id 38308e7fff4ca-2ccae571ac4so8870631fa.0 for <62153@debbugs.gnu.org>; Mon, 25 Dec 2023 18:19:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1703557186; x=1704161986; darn=debbugs.gnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=qO1LqsbW/+uFtxIVMwh5hKow66K9/hH0YccsRzB92H0=; b=CksmI1jpD2Z0vSrMymiBr2jrn6V5jbRyBnFjbxksRzPHj0s3hgbn1ckClzDK+ixwa3 O2IGFFoiyFyRX+u1cX1PwYSDE6Zh7trJF4zS4HacbzwH6IY7KnMD83lsMYsJEUG/n5NV jhxShqhJZQQSSQSDwiwRQFXd/vjmBpkvt8V+LKX00GLsYWZCd6/LgyNvW/AdCetyTJLp m+/xCUdqQcSddR/XrhWW3/hi9tpymEPxY6HH6U/G6Km/4DvK6FiwXL8cfXOrFQ4uQnZr rWebPuCN47EXsukxutHiKtuFHB8WSd5UzzYUWYgD0Vp//6RbOn73kzFMjlT008Rt6LZ/ 5Hpg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1703557186; x=1704161986; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=qO1LqsbW/+uFtxIVMwh5hKow66K9/hH0YccsRzB92H0=; b=UbwgFIQRenltG+o0HQ8Dzh1ggBS9n8w3bRaFEHNS2bNLNlnr3QfCHU9wOLv7w3pURh ncsOfw7q4bnbOlMuPJ5Cd7RMuyy/VLXJgJ7deSSFe0aAqH9fZqIOB8P1lah26Oh8kUdO ux9HIxNo7UMZ7oUw6e+fRvAXJT++wMZr3fExRb2NGvpJJ5Mt8mLjubew91lJuiHq/NBl TxbRBU9UGWlBoAb+0P2tsOFfubOZhrb0FKChyy3++cAOnNU0lwWjP1OtzKzlcr5vJ5M/ SVyDTtfkOwmdbn7Z0eIu3swx6f9ivK4MHjij4EVeqS1EMSQN4ILJtIt6v28RskDSZOqi XCVQ== X-Gm-Message-State: AOJu0YyfHvJ8Zk75Kx/QvKapbd5UdWwhDTm6ub7x0Kzp1bBuqmYwHuAj xvINiUqs7mXPDQb4/N8Zv2EIsyolK5M= X-Google-Smtp-Source: AGHT+IFuLXSGL8/unhJZAUoAdDptCTHPwz9anwYutnU9teGNada/S+8GYZJRMLJjdz9MNGADekM7BA== X-Received: by 2002:a05:651c:1688:b0:2cc:6ddf:1504 with SMTP id bd8-20020a05651c168800b002cc6ddf1504mr4906372ljb.1.1703557186115; Mon, 25 Dec 2023 18:19:46 -0800 (PST) Received: from guixsd.wugi.info ([88.201.161.72]) by smtp.gmail.com with ESMTPSA id e11-20020a05651c038b00b002cca609ef8bsm1633436ljp.111.2023.12.25.18.19.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 25 Dec 2023 18:19:45 -0800 (PST) From: Oleg Pykhalov Date: Tue, 26 Dec 2023 05:18:54 +0300 Message-ID: <6baac8dbf5f43a2e8458b592723463b205ba57f3.1703556298.git.go.wigust@gmail.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: References: MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org X-getmail-retrieved-from-mailbox: Patches * gnu/tests/docker.scm (run-docker-system-test)[vm]: Increase 'disk-image-size'. Change-Id: If88588d8981efdfdc539460900f1cbb9a663f9cb --- gnu/tests/docker.scm | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/gnu/tests/docker.scm b/gnu/tests/docker.scm index edc9804414..9e9d2e2d07 100644 --- a/gnu/tests/docker.scm +++ b/gnu/tests/docker.scm @@ -212,7 +212,7 @@ (define (run-docker-system-test tarball) (virtual-machine (operating-system os) (volatile? #f) - (disk-image-size (* 5500 (expt 2 20))) + (disk-image-size (* 6000 (expt 2 20))) (memory-size 2048) (port-forwardings '()))) From patchwork Tue Dec 26 02:18:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Oleg Pykhalov X-Patchwork-Id: 58077 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id EE98327BBEA; Tue, 26 Dec 2023 02:21:38 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FROM,MAILING_LIST_MULTI, SPF_HELO_PASS autolearn=unavailable autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 4806027BBE2 for ; Tue, 26 Dec 2023 02:21:37 +0000 (GMT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rHx40-0006za-7P; Mon, 25 Dec 2023 21:21:00 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rHx3z-0006z0-GZ for guix-patches@gnu.org; Mon, 25 Dec 2023 21:20:59 -0500 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rHx3z-0007i0-7m; Mon, 25 Dec 2023 21:20:59 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1rHx43-0000mw-Jz; Mon, 25 Dec 2023 21:21:03 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#62153] [PATCH 3/5] guix: docker: Build layered images. Resent-From: Oleg Pykhalov Original-Sender: "Debbugs-submit" Resent-CC: guix@cbaines.net, dev@jpoiret.xyz, ludo@gnu.org, othacehe@gnu.org, rekado@elephly.net, zimon.toutoune@gmail.com, me@tobias.gr, guix-patches@gnu.org Resent-Date: Tue, 26 Dec 2023 02:21:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 62153 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 62153@debbugs.gnu.org Cc: Oleg Pykhalov , Christopher Baines , Josselin Poiret , Ludovic =?utf-8?q?Court=C3=A8s?= , Mathieu Othacehe , Ricardo Wurmus , Simon Tournier , Tobias Geerinckx-Rice X-Debbugs-Original-Xcc: Christopher Baines , Josselin Poiret , Ludovic =?utf-8?q?Court=C3=A8s?= , Mathieu Othacehe , Ricardo Wurmus , Simon Tournier , Tobias Geerinckx-Rice Received: via spool by 62153-submit@debbugs.gnu.org id=B62153.17035572092925 (code B ref 62153); Tue, 26 Dec 2023 02:21:03 +0000 Received: (at 62153) by debbugs.gnu.org; 26 Dec 2023 02:20:09 +0000 Received: from localhost ([127.0.0.1]:55616 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rHx3A-0000l0-JC for submit@debbugs.gnu.org; Mon, 25 Dec 2023 21:20:09 -0500 Received: from mail-lf1-x130.google.com ([2a00:1450:4864:20::130]:38233) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rHx33-0000jm-7P for 62153@debbugs.gnu.org; Mon, 25 Dec 2023 21:20:04 -0500 Received: by mail-lf1-x130.google.com with SMTP id 2adb3069b0e04-50e79299da9so148826e87.1 for <62153@debbugs.gnu.org>; Mon, 25 Dec 2023 18:19:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1703557188; x=1704161988; darn=debbugs.gnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=B4CW8S5mwrxE9wU3Tigfd4N1Fl8MimAsHBmgn6Zu100=; b=YWO1WIjaxNOwYz/0PB1myKwtlkcq4rjfgKUQuCy7jbuXSyZE/C6UFC5gtGwwVQFVjR KVJwbW3rl1/8hR1gPm5kIinkZzI856WNWDgU8ED792RQlqnYjc83sKyoD3P8LSG5QmCr Wm6NdAjcxUa+q/djP6qLqyQVzEFPEBZ4IWRevumhkM9FFjmFLC46dgVyV3Qmg9oUS6Bz 2jqIezLM3J8Y/o86cgERmmWuCwfRP9vncWOqz3ek8FzGinrQ7NkJL2A/u4UOYIJGkycX rqz4HF1YMmw8Q9CPO5s1MH5EzTbKI4lWIxXBqb2MvzmbXs6R9VwbRQWbj6b1jofwZoJk Isxw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1703557188; x=1704161988; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=B4CW8S5mwrxE9wU3Tigfd4N1Fl8MimAsHBmgn6Zu100=; b=uo320T4sX4zO+VKPyx0/Ai14oo92cO9XaqQPlY64mXWDnGqDxQq0F7fGgj3vPzB1nP CBWsF2dElBiYh7Zl6z/c7ykq27tQUoq47Kdf8owWlNd+JshDgl1Sm/l1EBKVXB9gYbXq 2QJU6R8J4RTzGp0T45UKl2MPzhEYrZrKOBqJ4MRTRPIEBW93cPiKTcj4i++FFEYehTti caUeKqfX6JxLVGpsxQ1zWhkehCdI/ryzKo9+QeO27vki2QaVDIvDsIEIt1j5mCPRky0h SbDZ4+cwHVdmATTI1hCnXn1IASKwFqrp47hv2NNAQ8i1NgLZQ7DezaezUyEaFfX7O1T3 OgKA== X-Gm-Message-State: AOJu0YyhtaPwtqsQHmf+/2wRWKw3viDFI8a61MwckZP0KRBJLC2xHg9g fKUBn8RhXqBblF6owgCC49B4deZFmhk= X-Google-Smtp-Source: AGHT+IEmtJ9jElPwdlzxbbQk89wdTmtszxClF2esPBA6hju6UpZDj9fvWN+r2Gmem9IEnA/2kFIw3Q== X-Received: by 2002:a2e:9dd2:0:b0:2ca:143:a52f with SMTP id x18-20020a2e9dd2000000b002ca0143a52fmr5071742ljj.2.1703557187638; Mon, 25 Dec 2023 18:19:47 -0800 (PST) Received: from guixsd.wugi.info ([88.201.161.72]) by smtp.gmail.com with ESMTPSA id e11-20020a05651c038b00b002cca609ef8bsm1633436ljp.111.2023.12.25.18.19.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 25 Dec 2023 18:19:46 -0800 (PST) From: Oleg Pykhalov Date: Tue, 26 Dec 2023 05:18:55 +0300 Message-ID: <49f8906ba06af461e17d9badcbbf3967f1a8be3b.1703556298.git.go.wigust@gmail.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: References: MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org X-getmail-retrieved-from-mailbox: Patches * guix/docker.scm (%docker-image-max-layers): New variable. (size-sorted-store-items, create-empty-tar): New procedures. (config, manifest, build-docker-image): Build layered images. Change-Id: I4c8846bff0a3ceccb77e6bdf95d4942e5c3efe41 --- guix/docker.scm | 212 +++++++++++++++++++++++++++++++++++++----------- 1 file changed, 166 insertions(+), 46 deletions(-) diff --git a/guix/docker.scm b/guix/docker.scm index 5e6460f43f..5deca2afdb 100644 --- a/guix/docker.scm +++ b/guix/docker.scm @@ -3,6 +3,7 @@ ;;; Copyright © 2017, 2018, 2019, 2021 Ludovic Courtès ;;; Copyright © 2018 Chris Marusich ;;; Copyright © 2021 Maxim Cournoyer +;;; Copyright © 2023 Oleg Pykhalov ;;; ;;; This file is part of GNU Guix. ;;; @@ -29,16 +30,27 @@ (define-module (guix docker) with-directory-excursion invoke)) #:use-module (gnu build install) + #:use-module ((guix build store-copy) + #:select (file-size)) #:use-module (json) ;guile-json #:use-module (srfi srfi-1) #:use-module (srfi srfi-19) #:use-module (srfi srfi-26) + #:use-module (srfi srfi-71) #:use-module ((texinfo string-utils) #:select (escape-special-chars)) #:use-module (rnrs bytevectors) #:use-module (ice-9 ftw) #:use-module (ice-9 match) - #:export (build-docker-image)) + #:export (%docker-image-max-layers + build-docker-image)) + +;; The maximum number of layers allowed in a Docker image is typically around +;; 128, although it may vary depending on the Docker daemon. However, we +;; recommend setting the limit to 100 to ensure sufficient room for future +;; extensions. +(define %docker-image-max-layers + #f) ;; Generate a 256-bit identifier in hexadecimal encoding for the Docker image. (define docker-id @@ -92,12 +104,12 @@ (define (canonicalize-repository-name name) (make-string (- min-length l) padding-character))) (_ normalized-name)))) -(define* (manifest path id #:optional (tag "guix")) +(define* (manifest path layers #:optional (tag "guix")) "Generate a simple image manifest." (let ((tag (canonicalize-repository-name tag))) `#(((Config . "config.json") (RepoTags . #(,(string-append tag ":latest"))) - (Layers . #(,(string-append id "/layer.tar"))))))) + (Layers . ,(list->vector layers)))))) ;; According to the specifications this is required for backwards ;; compatibility. It duplicates information provided by the manifest. @@ -106,8 +118,8 @@ (define* (repositories path id #:optional (tag "guix")) `((,(canonicalize-repository-name tag) . ((latest . ,id))))) ;; See https://github.com/opencontainers/image-spec/blob/master/config.md -(define* (config layer time arch #:key entry-point (environment '())) - "Generate a minimal image configuration for the given LAYER file." +(define* (config layers-diff-ids time arch #:key entry-point (environment '())) + "Generate a minimal image configuration for the given LAYERS files." ;; "architecture" must be values matching "platform.arch" in the ;; runtime-spec at ;; https://github.com/opencontainers/runtime-spec/blob/v1.0.0-rc2/config.md#platform @@ -125,7 +137,7 @@ (define* (config layer time arch #:key entry-point (environment '())) (container_config . #nil) (os . "linux") (rootfs . ((type . "layers") - (diff_ids . #(,(layer-diff-id layer))))))) + (diff_ids . ,(list->vector layers-diff-ids)))))) (define directive-file ;; Return the file or directory created by a 'evaluate-populate-directive' @@ -136,6 +148,26 @@ (define directive-file (('directory name _ ...) (string-trim name #\/)))) +(define (size-sorted-store-items items max-layers) + "Split list of ITEMS at %MAX-LAYERS and sort by disk usage." + (let* ((items-length (length items)) + (head tail + (split-at + (map (match-lambda ((size . item) item)) + (sort (map (lambda (item) + (cons (file-size item) item)) + items) + (lambda (item1 item2) + (< (match item2 ((size . _) size)) + (match item1 ((size . _) size)))))) + (if (>= items-length max-layers) + (- max-layers 2) + (1- items-length))))) + (list head tail))) + +(define (create-empty-tar file) + (invoke "tar" "-cf" file "--files-from" "/dev/null")) + (define* (build-docker-image image paths prefix #:key (repository "guix") @@ -146,11 +178,13 @@ (define* (build-docker-image image paths prefix entry-point (environment '()) compressor - (creation-time (current-time time-utc))) - "Write to IMAGE a Docker image archive containing the given PATHS. PREFIX -must be a store path that is a prefix of any store paths in PATHS. REPOSITORY -is a descriptive name that will show up in \"REPOSITORY\" column of the output -of \"docker images\". + (creation-time (current-time time-utc)) + max-layers + root-system) + "Write to IMAGE a layerer Docker image archive containing the given PATHS. +PREFIX must be a store path that is a prefix of any store paths in PATHS. +REPOSITORY is a descriptive name that will show up in \"REPOSITORY\" column of +the output of \"docker images\". When DATABASE is true, copy it to /var/guix/db in the image and create /var/guix/gcroots and friends. @@ -172,7 +206,14 @@ (define* (build-docker-image image paths prefix SYSTEM is a GNU triplet (or prefix thereof) of the system the binaries in PATHS are for; it is used to produce metadata in the image. Use COMPRESSOR, a command such as '(\"gzip\" \"-9n\"), to compress IMAGE. Use CREATION-TIME, a -SRFI-19 time-utc object, as the creation time in metadata." +SRFI-19 time-utc object, as the creation time in metadata. + +When MAX-LAYERS is not false build layered image, providing a Docker +image with many of the store paths being on their own layer to improve sharing +between images. + +ROOT-SYSTEM is a directory with a provisioned root file system, which will be +added to image as a layer." (define (sanitize path-fragment) (escape-special-chars ;; GNU tar strips the leading slash off of absolute paths before applying @@ -203,6 +244,59 @@ (define* (build-docker-image image paths prefix (if (eq? '() transformations) '() `("--transform" ,(transformations->expression transformations)))) + (define (seal-layer) + ;; Add 'layer.tar' to 'image.tar' under the right name. Return its hash. + (let* ((file-hash (layer-diff-id "layer.tar")) + (file-name (string-append file-hash "/layer.tar"))) + (mkdir file-hash) + (rename-file "layer.tar" file-name) + (invoke "tar" "-rf" "image.tar" file-name) + (delete-file file-name) + file-hash)) + (define layers-hashes + ;; Generate a tarball that includes container image layers as tarballs, + ;; along with a manifest.json file describing the layer and config file + ;; locations. + (match-lambda + (((head ...) (tail ...) id) + (create-empty-tar "image.tar") + (let* ((head-layers + (map + (lambda (file) + (invoke "tar" "cf" "layer.tar" file) + (seal-layer)) + head)) + (tail-layer + (begin + (create-empty-tar "layer.tar") + (for-each (lambda (file) + (invoke "tar" "-rf" "layer.tar" file)) + tail) + (let* ((file-hash (layer-diff-id "layer.tar")) + (file-name (string-append file-hash "/layer.tar"))) + (mkdir file-hash) + (rename-file "layer.tar" file-name) + (invoke "tar" "-rf" "image.tar" file-name) + (delete-file file-name) + file-hash))) + (customization-layer + (let* ((file-id (string-append id "/layer.tar")) + (file-hash (layer-diff-id file-id)) + (file-name (string-append file-hash "/layer.tar"))) + (mkdir file-hash) + (rename-file file-id file-name) + (invoke "tar" "-rf" "image.tar" file-name) + file-hash)) + (all-layers + (append head-layers (list tail-layer customization-layer)))) + (with-output-to-file "manifest.json" + (lambda () + (scm->json (manifest prefix + (map (cut string-append <> "/layer.tar") + all-layers) + repository)))) + (invoke "tar" "-rf" "image.tar" "manifest.json") + all-layers)))) (let* ((directory "/tmp/docker-image") ;temporary working directory (id (docker-id prefix)) (time (date->string (time-utc->date creation-time) "~4")) @@ -229,26 +323,39 @@ (define* (build-docker-image image paths prefix (with-output-to-file "json" (lambda () (scm->json (image-description id time)))) - ;; Create a directory for the non-store files that need to go into the - ;; archive. - (mkdir "extra") + (if root-system + (let ((directory (getcwd))) + (with-directory-excursion root-system + (apply invoke "tar" + "-cf" (string-append directory "/layer.tar") + `(,@transformation-options + ,@(tar-base-options) + ,@(scandir "." + (lambda (file) + (not (member file '("." ".."))))))))) + (begin + ;; Create a directory for the non-store files that need to go + ;; into the archive. + (mkdir "extra") - (with-directory-excursion "extra" - ;; Create non-store files. - (for-each (cut evaluate-populate-directive <> "./") - extra-files) + (with-directory-excursion "extra" + ;; Create non-store files. + (for-each (cut evaluate-populate-directive <> "./") + extra-files) - (when database - ;; Initialize /var/guix, assuming PREFIX points to a profile. - (install-database-and-gc-roots "." database prefix)) + (when database + ;; Initialize /var/guix, assuming PREFIX points to a + ;; profile. + (install-database-and-gc-roots "." database prefix)) - (apply invoke "tar" "-cf" "../layer.tar" - `(,@transformation-options - ,@(tar-base-options) - ,@paths - ,@(scandir "." - (lambda (file) - (not (member file '("." "..")))))))) + (apply invoke "tar" "-cf" "../layer.tar" + `(,@transformation-options + ,@(tar-base-options) + ,@(if max-layers '() paths) + ,@(scandir "." + (lambda (file) + (not (member file '("." "..")))))))) + (delete-file-recursively "extra"))) ;; It is possible for "/" to show up in the archive, especially when ;; applying transformations. For example, the transformation @@ -261,24 +368,37 @@ (define* (build-docker-image image paths prefix ;; error messages. (with-error-to-port (%make-void-port "w") (lambda () - (system* "tar" "--delete" "/" "-f" "layer.tar"))) - - (delete-file-recursively "extra")) + (system* "tar" "--delete" "/" "-f" "layer.tar")))) (with-output-to-file "config.json" (lambda () - (scm->json (config (string-append id "/layer.tar") - time arch - #:environment environment - #:entry-point entry-point)))) - (with-output-to-file "manifest.json" - (lambda () - (scm->json (manifest prefix id repository)))) - (with-output-to-file "repositories" - (lambda () - (scm->json (repositories prefix id repository))))) - - (apply invoke "tar" "-cf" image "-C" directory - `(,@(tar-base-options #:compressor compressor) - ".")) + (scm->json + (config (if max-layers + (layers-hashes + (append (size-sorted-store-items paths max-layers) + (list id))) + (list (layer-diff-id (string-append id "/layer.tar")))) + time arch + #:environment environment + #:entry-point entry-point)))) + (if max-layers + (begin + (invoke "tar" "-rf" "image.tar" "config.json") + (if compressor + (begin + (apply invoke `(,@compressor "image.tar")) + (copy-file "image.tar.gz" image)) + (copy-file "image.tar" image))) + (begin + (with-output-to-file "manifest.json" + (lambda () + (scm->json (manifest prefix + (list (string-append id "/layer.tar")) + repository)))) + (with-output-to-file "repositories" + (lambda () + (scm->json (repositories prefix id repository)))) + (apply invoke "tar" "-cf" image + `(,@(tar-base-options #:compressor compressor) + "."))))) (delete-file-recursively directory))) From patchwork Tue Dec 26 02:18:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Oleg Pykhalov X-Patchwork-Id: 58075 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 551D227BBEB; Tue, 26 Dec 2023 02:21:27 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FROM,MAILING_LIST_MULTI, SPF_HELO_PASS autolearn=unavailable autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 8B5DA27BBE9 for ; Tue, 26 Dec 2023 02:21:26 +0000 (GMT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rHx41-00070k-Ru; Mon, 25 Dec 2023 21:21:01 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rHx3z-0006zF-Rf for guix-patches@gnu.org; Mon, 25 Dec 2023 21:20:59 -0500 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rHx3z-0007i8-JE; Mon, 25 Dec 2023 21:20:59 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1rHx44-0000n6-3L; Mon, 25 Dec 2023 21:21:04 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#62153] [PATCH 4/5] guix: pack: Build layered images. Resent-From: Oleg Pykhalov Original-Sender: "Debbugs-submit" Resent-CC: guix@cbaines.net, dev@jpoiret.xyz, ludo@gnu.org, othacehe@gnu.org, rekado@elephly.net, zimon.toutoune@gmail.com, me@tobias.gr, guix-patches@gnu.org Resent-Date: Tue, 26 Dec 2023 02:21:04 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 62153 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 62153@debbugs.gnu.org Cc: Oleg Pykhalov , Christopher Baines , Josselin Poiret , Ludovic =?utf-8?q?Court=C3=A8s?= , Mathieu Othacehe , Ricardo Wurmus , Simon Tournier , Tobias Geerinckx-Rice X-Debbugs-Original-Xcc: Christopher Baines , Josselin Poiret , Ludovic =?utf-8?q?Court=C3=A8s?= , Mathieu Othacehe , Ricardo Wurmus , Simon Tournier , Tobias Geerinckx-Rice Received: via spool by 62153-submit@debbugs.gnu.org id=B62153.17035572102932 (code B ref 62153); Tue, 26 Dec 2023 02:21:04 +0000 Received: (at 62153) by debbugs.gnu.org; 26 Dec 2023 02:20:10 +0000 Received: from localhost ([127.0.0.1]:55618 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rHx3B-0000l8-FY for submit@debbugs.gnu.org; Mon, 25 Dec 2023 21:20:10 -0500 Received: from mail-lf1-x136.google.com ([2a00:1450:4864:20::136]:38077) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rHx34-0000jp-Nf for 62153@debbugs.gnu.org; Mon, 25 Dec 2023 21:20:05 -0500 Received: by mail-lf1-x136.google.com with SMTP id 2adb3069b0e04-50e7a0d07a0so122988e87.0 for <62153@debbugs.gnu.org>; Mon, 25 Dec 2023 18:19:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1703557189; x=1704161989; darn=debbugs.gnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=BhVNkDMX7JP5NGW3OEcnZMSwtVpydjEjv3znzIK2QTY=; b=IAD3+8Wopa8r9AmKhjSvQWV92KOV2fGIw0PoJrm4Jgk7d16QMI84WtOjTHL48hJBVw uGPJReZzFGb8TggXZbFZrwv/SO7sc1SjY2erhEOwDePrasSSFffWngVtb8Xh4s9C5ydW v4LPMv52x3ocvwtDrs169+BukvJvK4jcM2V5lA9QmbDHciOpjO7bLp4sHDR1/0gJWTMr eFakZpRC65hakwv+dBls3N0WaELOJLQviVY3HO9U+RlJKpS7B74kf1FJPOJFVAGc+bZ9 MmnS+OtETtesZ3RM8+cvW3Xjs2vjV6cnJwzHtLWFkC1Jnr97usFoA67N3Ih7u4GRdi4T NT9Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1703557189; x=1704161989; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=BhVNkDMX7JP5NGW3OEcnZMSwtVpydjEjv3znzIK2QTY=; b=tmar6f+NSnk7ZENlbReUxUDLrmtUY5vc5zo/KbuTfLbpWZAQnvTLo3wnhQU8p8Q5UF ZcvNiYRifcoG0Wy88NLKVvIuFiOAsnD6xSOO3qJiCQw4seoNKOqOWiSNKgpxaFbaFMtH rTONWBQTLGKKy9h0diEYla4Iu3MxGCvssBbZviSyFteYG1plMobG7S48VqydxHHOPWCp /c+dZwAevjO7YMd0U97laXDDoQ02cjEv1O+tXXKwma1f7FBeAC6d64nTYRnIDopaiVsR y6+mra7q0DSbFSoGViV3lzvDJI+317rc+G4w6cq2E+kPyB440dzy0ZdSXJSNWYBiEnav Nmcg== X-Gm-Message-State: AOJu0YzOOQ0YeTWFfYN+l9VcZaW8tle8W4lxnubrt8l/1VmgmNBF6cvg BRfYnIXzU8RZ8b7Ww/8tnavAqO9rlHk= X-Google-Smtp-Source: AGHT+IH7eZs1bzPjO9KpFR9WnVJAvA8I3VrH9Vm3u7m+R9ydzyw1a1Fa81eaIPjQWV2ELzKWCpRh8A== X-Received: by 2002:a2e:9dd2:0:b0:2ca:143:a52f with SMTP id x18-20020a2e9dd2000000b002ca0143a52fmr5071748ljj.2.1703557189003; Mon, 25 Dec 2023 18:19:49 -0800 (PST) Received: from guixsd.wugi.info ([88.201.161.72]) by smtp.gmail.com with ESMTPSA id e11-20020a05651c038b00b002cca609ef8bsm1633436ljp.111.2023.12.25.18.19.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 25 Dec 2023 18:19:48 -0800 (PST) From: Oleg Pykhalov Date: Tue, 26 Dec 2023 05:18:56 +0300 Message-ID: <77bae2565cb5506342028896e1a45c757f3bad51.1703556298.git.go.wigust@gmail.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: References: MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org X-getmail-retrieved-from-mailbox: Patches * guix/scripts/pack.scm (docker-image, guix-pack, %default-options, %docker-format-options, show-docker-format-options/detailed): Handle '--max-layers' option. * doc/guix.texi (Invoking guix pack): Document this. Change-Id: I90660b2421fcdde891f003469fe2e2edaac7da41 --- doc/guix.texi | 26 ++++++++++++++++++++++- guix/scripts/pack.scm | 38 ++++++++++++++++++++++++++------- tests/pack.scm | 49 +++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 104 insertions(+), 9 deletions(-) diff --git a/doc/guix.texi b/doc/guix.texi index cca250dc31..d21048405a 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -56,7 +56,7 @@ Copyright @copyright{} 2017, 2018, 2019, 2020, 2023 Arun Isaac@* Copyright @copyright{} 2017 nee@* Copyright @copyright{} 2018 Rutger Helling@* -Copyright @copyright{} 2018, 2021 Oleg Pykhalov@* +Copyright @copyright{} 2018, 2021, 2023 Oleg Pykhalov@* Copyright @copyright{} 2018 Mike Gerwitz@* Copyright @copyright{} 2018 Pierre-Antoine Rouby@* Copyright @copyright{} 2018, 2019 Gábor Boskovits@* @@ -7441,6 +7441,30 @@ Invoking guix pack guix pack -f docker --entry-point=bin/guile --entry-point-argument="--help" guile @end example +@cindex maximum layers argument, for docker images +@item --max-layers=@code{n} +Specifies the maximum number of Docker image layers allowed when +building an image. + +@example +guix pack -f docker --max-layers=100 guile +@end example + +This option allows you to limit the number of layers in a Docker image. +Docker images are comprised of multiple layers, and each layer adds to +the overall size and complexity of the image. By setting a maximum +number of layers, you can control the following effects: + +@itemize +@item Disk Usage: +Increasing the number of layers can help optimize the disk space +required to store multiple images built with a similar package graph. + +@item Pulling: +When transferring images between different nodes or systems, having more +layers can reduce the time required to pull the image. +@end itemize + @item --expression=@var{expr} @itemx -e @var{expr} Consider the package @var{expr} evaluates to. diff --git a/guix/scripts/pack.scm b/guix/scripts/pack.scm index 4c0a602eb1..22f0dd6061 100644 --- a/guix/scripts/pack.scm +++ b/guix/scripts/pack.scm @@ -9,6 +9,7 @@ ;;; Copyright © 2020 Eric Bavier ;;; Copyright © 2022 Alex Griffin ;;; Copyright © 2023 Graham James Addis +;;; Copyright © 2023 Oleg Pykhalov ;;; ;;; This file is part of GNU Guix. ;;; @@ -48,6 +49,7 @@ (define-module (guix scripts pack) #:use-module (guix scripts build) #:use-module (guix transformations) #:use-module ((guix self) #:select (make-config.scm)) + #:use-module ((guix docker) #:select (%docker-image-max-layers)) #:use-module (gnu compression) #:use-module (gnu packages) #:use-module (gnu packages bootstrap) @@ -517,12 +519,15 @@ (define* (docker-image name profile localstatedir? (symlinks '()) (archiver tar) - (extra-options '())) + (extra-options '()) + max-layers) "Return a derivation to construct a Docker image of PROFILE. The image is a tarball conforming to the Docker Image Specification, compressed with COMPRESSOR. It can be passed to 'docker load'. If TARGET is true, it must a be a GNU triplet and it is used to derive the architecture metadata in -the image. EXTRA-OPTIONS may contain the IMAGE-TAG keyword argument." +the image. EXTRA-OPTIONS may contain the IMAGE-TAG keyword argument. +If MAX-LAYERS is not false, the image will with many of the store paths being +on their own layer to improve sharing between images." (define database (and localstatedir? (file-append (store-database (list profile)) @@ -583,11 +588,17 @@ (define* (docker-image name profile (cons* (string-append prefix "/" entry-point) entry-point-argument)))) - (setenv "PATH" #+(file-append archiver "/bin")) + (setenv "PATH" + (string-join `(#+(file-append archiver "/bin") + #+@(if max-layers + (list (file-append gzip "/bin")) + '())) + ":")) (let-keywords '#$extra-options #f ((image-tag #f) - (entry-point-argument #f)) + (entry-point-argument #f) + (max-layers #f)) (build-docker-image #$output (map store-info-item @@ -609,7 +620,8 @@ (define* (docker-image name profile #:compressor #+(compressor-command compressor) #:creation-time - (make-time time-utc 0 1))))))) + (make-time time-utc 0 1) + #:max-layers max-layers)))))) (gexp->derivation (string-append name ".tar" (compressor-extension compressor)) @@ -1287,6 +1299,7 @@ (define %default-options (verbosity . 1) (symlinks . ()) (entry-point-argument . ()) + (max-layers . ,%docker-image-max-layers) (compressor . ,(first %compressors)))) (define %formats @@ -1324,7 +1337,11 @@ (define (required-option symbol) (define %docker-format-options (list (required-option 'image-tag) (option '(#\A "entry-point-argument") #t #f - entry-point-argument-spec-option-parser))) + entry-point-argument-spec-option-parser) + (option '("max-layers") #t #f + (lambda (opt name arg result) + (alist-cons 'max-layers (string->number* arg) + result))))) (define (show-docker-format-options) (display (G_ " @@ -1338,7 +1355,10 @@ (define (show-docker-format-options/detailed) -A, --entry-point-argument=COMMAND/PARAMETER Value(s) to use for the Docker EntryPoint arguments. Multiple instances are accepted. This is only valid - in conjunction with the --entry-point option")) + in conjunction with the --entry-point option + + --max-layers=N + Number of image layers")) (newline) (exit 0)) @@ -1651,7 +1671,9 @@ (define-command (guix-pack . args) (list #:image-tag (assoc-ref opts 'image-tag) #:entry-point-argument - (assoc-ref opts 'entry-point-argument))) + (assoc-ref opts 'entry-point-argument) + #:max-layers + (assoc-ref opts 'max-layers))) ('deb (list #:control-file (process-file-arg opts 'control-file) diff --git a/tests/pack.scm b/tests/pack.scm index ac78817a70..fda4dc04c6 100644 --- a/tests/pack.scm +++ b/tests/pack.scm @@ -2,6 +2,7 @@ ;;; Copyright © 2017-2021, 2023 Ludovic Courtès ;;; Copyright © 2018 Ricardo Wurmus ;;; Copyright © 2021, 2023 Maxim Cournoyer +;;; Copyright © 2023 Oleg Pykhalov ;;; ;;; This file is part of GNU Guix. ;;; @@ -29,6 +30,7 @@ (define-module (test-pack) #:use-module (guix gexp) #:use-module (guix modules) #:use-module (guix utils) + #:use-module ((guix build utils) #:select (%store-directory)) #:use-module (gnu packages) #:use-module ((gnu packages base) #:select (libc-utf8-locales-for-target)) #:use-module (gnu packages bootstrap) @@ -250,6 +252,53 @@ (define rpm-for-tests (mkdir #$output))))))) (built-derivations (list check)))) + (unless store (test-skip 1)) + (test-assertm "docker-layered-image + localstatedir" + (mlet* %store-monad + ((guile (set-guile-for-build (default-guile))) + (profile -> (profile + (content (packages->manifest (list %bootstrap-guile))) + (hooks '()) + (locales? #f))) + (tarball (docker-image "docker-pack" profile + #:symlinks '(("/bin/Guile" -> "bin/guile")) + #:localstatedir? #t + #:max-layers 100)) + (check (gexp->derivation "check-tarball" + (with-imported-modules '((guix build utils)) + #~(begin + (use-modules (guix build utils) + (ice-9 match)) + + (define bin + (string-append "." #$profile "/bin")) + + (define store + (string-append "." #$(%store-directory))) + + (setenv "PATH" (string-append #$%tar-bootstrap "/bin")) + (mkdir "base") + (with-directory-excursion "base" + (invoke "tar" "xvf" #$tarball)) + + (match (find-files "base" "layer.tar") + ((layers ...) + (for-each (lambda (layer) + (invoke "tar" "xvf" layer) + (invoke "chmod" "--recursive" "u+w" store)) + layers))) + + (when + (and (file-exists? (string-append bin "/guile")) + (file-exists? "var/guix/db/db.sqlite") + (file-is-directory? "tmp") + (string=? (string-append #$%bootstrap-guile "/bin") + (pk 'binlink (readlink bin))) + (string=? (string-append #$profile "/bin/guile") + (pk 'guilelink (readlink "bin/Guile")))) + (mkdir #$output))))))) + (built-derivations (list check)))) + (unless store (test-skip 1)) (test-assertm "squashfs-image + localstatedir" (mlet* %store-monad From patchwork Tue Dec 26 02:18:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Oleg Pykhalov X-Patchwork-Id: 58073 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id C916327BBEA; Tue, 26 Dec 2023 02:21:04 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FROM,MAILING_LIST_MULTI, SPF_HELO_PASS autolearn=unavailable autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 75DA027BBE2 for ; Tue, 26 Dec 2023 02:21:03 +0000 (GMT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rHx3z-0006zT-Vt; Mon, 25 Dec 2023 21:21:00 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rHx3z-0006yr-2g for guix-patches@gnu.org; Mon, 25 Dec 2023 21:20:59 -0500 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rHx3y-0007hr-Os; Mon, 25 Dec 2023 21:20:58 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1rHx44-0000nE-HL; Mon, 25 Dec 2023 21:21:04 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#62153] [PATCH 5/5] scripts: system: Build layered images. Resent-From: Oleg Pykhalov Original-Sender: "Debbugs-submit" Resent-CC: guix@cbaines.net, dev@jpoiret.xyz, ludo@gnu.org, othacehe@gnu.org, rekado@elephly.net, zimon.toutoune@gmail.com, me@tobias.gr, guix-patches@gnu.org Resent-Date: Tue, 26 Dec 2023 02:21:04 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 62153 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 62153@debbugs.gnu.org Cc: Oleg Pykhalov , Christopher Baines , Josselin Poiret , Ludovic =?utf-8?q?Court=C3=A8s?= , Mathieu Othacehe , Ricardo Wurmus , Simon Tournier , Tobias Geerinckx-Rice X-Debbugs-Original-Xcc: Christopher Baines , Josselin Poiret , Ludovic =?utf-8?q?Court=C3=A8s?= , Mathieu Othacehe , Ricardo Wurmus , Simon Tournier , Tobias Geerinckx-Rice Received: via spool by 62153-submit@debbugs.gnu.org id=B62153.17035572112939 (code B ref 62153); Tue, 26 Dec 2023 02:21:04 +0000 Received: (at 62153) by debbugs.gnu.org; 26 Dec 2023 02:20:11 +0000 Received: from localhost ([127.0.0.1]:55620 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rHx3C-0000lF-AN for submit@debbugs.gnu.org; Mon, 25 Dec 2023 21:20:10 -0500 Received: from mail-lj1-x232.google.com ([2a00:1450:4864:20::232]:39940) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rHx35-0000jr-Kd for 62153@debbugs.gnu.org; Mon, 25 Dec 2023 21:20:06 -0500 Received: by mail-lj1-x232.google.com with SMTP id 38308e7fff4ca-2ccb6ddf651so5292801fa.0 for <62153@debbugs.gnu.org>; Mon, 25 Dec 2023 18:19:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1703557190; x=1704161990; darn=debbugs.gnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=6Ln2kxpz4n9WOq6SK1nelI1pbUl1f4tnE9yQEyDFGgY=; b=YOGYfJItCrRAr3tfB58bHsKHB0WqUJuQxEi8yUN7wRCp21GgrUUXG2pYDNJWXJFcEt IX0mgeQ8bZPx2ybnSs9lRdR+Pmp8bvj7U5geV8ShqEMTGAg+NbIIsukYxmEqL/0GSXr2 EDKWVmhQ1svq1RGxUKFs9k4+NVryYQsBWzyF6y8WaLnsmQ1+xAq98zSLz1WWbgu7yXay qfD/GiYWc5V3gZzX2HDuiFmCZo35ku26fNQgKyvtfWQyfzppI0a80AeAgDcxWP3YiRQQ JjeMduWnu1lya9hp4NL7OQHWZUjghQLnpJlszR7pm3Uz3Ivw1ZIM8PSdMMflG7eMMURa j6fQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1703557190; x=1704161990; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=6Ln2kxpz4n9WOq6SK1nelI1pbUl1f4tnE9yQEyDFGgY=; b=UYnwQeXYNOcyRuziqmwA0cV0lu8dBhu65uI3n4LDdBn9Uoq6oKkRncQ0e9wuXxKA1f eE6ZMo4FPjQxETi6JCOvrCDIGWFT6Nbroqp+EAjQH85iF5qPw6+u3t1qO5XDZzo4iDvm TnuWEcLaUFCgDCh3DchXF1JnFTy2EHaU9chPO8SmZ85eYV3qgi8nQOwqD5kjYvl+4vBM CUgWy6l0GVC55E7iSPQEbs/sGnsm5vSEpDxrXP1IPkeGynCv7jxc2jZNcfqF+Xjidh/K gsTBL8hHNLSGKr1+fVP9QejTVgbbnajwHl7vbgVJ4Bes1RSatJEJwSJXAYxgteXKqDEu FKkw== X-Gm-Message-State: AOJu0YxYxxvok8V1RDuwzFMdJKmRQX9jwLd0tpRSqQ35wbAKFlKwcD9j qUL4WTUHbRrDbUHLg7y6Elmi7F9oCm4= X-Google-Smtp-Source: AGHT+IG/rxMVlkCB9cE/GnGt/Uh6hI6Qx34yP2sHrosJXhsK6g98pVrfRXdpCjFgIhFS0Ophc2iXsw== X-Received: by 2002:a2e:88ce:0:b0:2cc:85a6:25a3 with SMTP id a14-20020a2e88ce000000b002cc85a625a3mr5202517ljk.2.1703557190516; Mon, 25 Dec 2023 18:19:50 -0800 (PST) Received: from guixsd.wugi.info ([88.201.161.72]) by smtp.gmail.com with ESMTPSA id e11-20020a05651c038b00b002cca609ef8bsm1633436ljp.111.2023.12.25.18.19.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 25 Dec 2023 18:19:49 -0800 (PST) From: Oleg Pykhalov Date: Tue, 26 Dec 2023 05:18:57 +0300 Message-ID: <99155dabc366c37acb71f6624aa6e6025b3e571b.1703556298.git.go.wigust@gmail.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: References: MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org X-getmail-retrieved-from-mailbox: Patches * guix/scripts/system.scm (show-help, %docker-format-options, %options, %default-options, show-docker-format-options, show-docker-format-options/detailed, process-action): Handle '--max-layers' option. * gnu/system/image.scm (system-docker-image): Same. * gnu/image.scm ()[max-layers]: New record field. Change-Id: I2726655aefd6688b976057fd5a38e9972ebfc292 --- gnu/image.scm | 4 ++++ gnu/system/image.scm | 41 ++++++++++++++++++++++++++++------------- guix/scripts/system.scm | 28 ++++++++++++++++++++++++++-- 3 files changed, 58 insertions(+), 15 deletions(-) diff --git a/gnu/image.scm b/gnu/image.scm index 523653dd77..7fb06dec10 100644 --- a/gnu/image.scm +++ b/gnu/image.scm @@ -1,5 +1,6 @@ ;;; GNU Guix --- Functional package management for GNU ;;; Copyright © 2020, 2022 Mathieu Othacehe +;;; Copyright © 2023 Oleg Pykhalov ;;; ;;; This file is part of GNU Guix. ;;; @@ -42,6 +43,7 @@ (define-module (gnu image) image-format image-platform image-size + image-max-layers image-operating-system image-partition-table-type image-partitions @@ -170,6 +172,8 @@ (define-record-type* (size image-size ;size in bytes as integer (default 'guess) (sanitize validate-size)) + (max-layers image-max-layers ;number of layers as integer + (default #false)) (operating-system image-operating-system) ; (partition-table-type image-partition-table-type ; 'mbr or 'gpt (default 'mbr) diff --git a/gnu/system/image.scm b/gnu/system/image.scm index b825892232..2cc1012893 100644 --- a/gnu/system/image.scm +++ b/gnu/system/image.scm @@ -5,6 +5,7 @@ ;;; Copyright © 2022 Denis 'GNUtoo' Carikli ;;; Copyright © 2022 Alex Griffin ;;; Copyright © 2023 Efraim Flashner +;;; Copyright © 2023 Oleg Pykhalov ;;; ;;; This file is part of GNU Guix. ;;; @@ -686,7 +687,8 @@ (define (image-with-label base-image label) (define* (system-docker-image image #:key - (name "docker-image")) + (name "docker-image") + (archiver tar)) "Build a docker image for IMAGE. NAME is the base name to use for the output file." (define boot-program @@ -731,6 +733,7 @@ (define* (system-docker-image image (use-modules (guix docker) (guix build utils) (gnu build image) + (srfi srfi-1) (srfi srfi-19) (guix build store-copy) (guix store database)) @@ -754,18 +757,30 @@ (define* (system-docker-image image #:register-closures? #$register-closures? #:deduplicate? #f #:system-directory #$os) - (build-docker-image - #$output - (cons* image-root - (map store-info-item - (call-with-input-file #$graph - read-reference-graph))) - #$os - #:entry-point '(#$boot-program #$os) - #:compressor '(#+(file-append gzip "/bin/gzip") "-9n") - #:creation-time (make-time time-utc 0 1) - #:system #$image-target - #:transformations `((,image-root -> "")))))))) + (when #$(image-max-layers image) + (setenv "PATH" + (string-join (list #+(file-append archiver "/bin") + #+(file-append gzip "/bin")) + ":"))) + (apply build-docker-image + (append (list #$output + (append (if #$(image-max-layers image) + '() + (list image-root)) + (map store-info-item + (call-with-input-file #$graph + read-reference-graph))) + #$os + #:entry-point '(#$boot-program #$os) + #:compressor + '(#+(file-append gzip "/bin/gzip") "-9n") + #:creation-time (make-time time-utc 0 1) + #:system #$image-target + #:transformations `((,image-root -> ""))) + (if #$(image-max-layers image) + (list #:root-system image-root + #:max-layers #$(image-max-layers image)) + '())))))))) (computed-file name builder ;; Allow offloading so that this I/O-intensive process diff --git a/guix/scripts/system.scm b/guix/scripts/system.scm index f85b663d64..a21ecd4d1e 100644 --- a/guix/scripts/system.scm +++ b/guix/scripts/system.scm @@ -58,6 +58,7 @@ (define-module (guix scripts system) #:use-module (guix scripts system reconfigure) #:use-module (guix build utils) #:use-module (guix progress) + #:use-module ((guix docker) #:select (%docker-image-max-layers)) #:use-module (gnu build image) #:use-module (gnu build install) #:autoload (gnu build file-systems) @@ -1053,6 +1054,8 @@ (define (show-help) (newline) (show-native-build-options-help) (newline) + (show-docker-format-options) + (newline) (display (G_ " -h, --help display this help and exit")) (display (G_ " @@ -1060,6 +1063,12 @@ (define (show-help) (newline) (show-bug-report-information)) +(define %docker-format-options + (list (option '("max-layers") #t #f + (lambda (opt name arg result) + (alist-cons 'max-layers (string->number* arg) + result))))) + (define %options ;; Specifications of the command-line options. (cons* (option '(#\h "help") #f #f @@ -1154,7 +1163,8 @@ (define %options (alist-cons 'list-installed (or arg "") result))) (append %standard-build-options %standard-cross-build-options - %standard-native-build-options))) + %standard-native-build-options + %docker-format-options))) (define %default-options ;; Alist of default option values. @@ -1175,7 +1185,8 @@ (define %default-options (label . #f) (volatile-image-root? . #f) (volatile-vm-root? . #t) - (graph-backend . "graphviz"))) + (graph-backend . "graphviz") + (max-layers . ,%docker-image-max-layers))) (define (verbosity-level opts) "Return the verbosity level based on OPTS, the alist of parsed options." @@ -1183,6 +1194,17 @@ (define (verbosity-level opts) (if (eq? (assoc-ref opts 'action) 'build) 3 1))) +(define (show-docker-format-options) + (display (G_ " + --help-docker-format list options specific to the docker image type."))) + +(define (show-docker-format-options/detailed) + (display (G_ " + --max-layers=N + Number of image layers")) + (newline) + (exit 0)) + ;;; ;;; Entry point. @@ -1245,6 +1267,7 @@ (define (process-action action args opts) ((docker-image) docker-image-type) (else image-type))) (image-size (assoc-ref opts 'image-size)) + (image-max-layers (assoc-ref opts 'max-layers)) (volatile? (assoc-ref opts 'volatile-image-root?)) (shared-network? @@ -1258,6 +1281,7 @@ (define (process-action action args opts) (image-with-label base-image label) base-image)) (size image-size) + (max-layers image-max-layers) (volatile-root? volatile?) (shared-network? shared-network?)))) (os (or (image-operating-system image)