From patchwork Sun Nov 4 22:10:32 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Ludovic_Court=C3=A8s?= X-Patchwork-Id: 115 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 76F92167B2; Sun, 4 Nov 2018 22:23:21 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_HI, URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) by mira.cbaines.net (Postfix) with ESMTPS id 0C52B167B0 for ; Sun, 4 Nov 2018 22:23:21 +0000 (GMT) Received: from localhost ([::1]:60396 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gJQng-000376-BE for patchwork@mira.cbaines.net; Sun, 04 Nov 2018 17:23:20 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:35862) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gJQl1-0007qJ-7l for guix-patches@gnu.org; Sun, 04 Nov 2018 17:20:36 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gJQcp-0000zp-HB for guix-patches@gnu.org; Sun, 04 Nov 2018 17:12:09 -0500 Received: from debbugs.gnu.org ([208.118.235.43]:58340) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gJQcm-0000xH-CZ for guix-patches@gnu.org; Sun, 04 Nov 2018 17:12:05 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1gJQcm-0001gi-7B for guix-patches@gnu.org; Sun, 04 Nov 2018 17:12:04 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#33259] [PATCH 4/8] pack: Docker backend now honors '--localstatedir'. Resent-From: Ludovic =?utf-8?q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sun, 04 Nov 2018 22:12:04 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33259 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 33259@debbugs.gnu.org Received: via spool by 33259-submit@debbugs.gnu.org id=B33259.15413694746368 (code B ref 33259); Sun, 04 Nov 2018 22:12:04 +0000 Received: (at 33259) by debbugs.gnu.org; 4 Nov 2018 22:11:14 +0000 Received: from localhost ([127.0.0.1]:34355 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gJQbx-0001eY-Mp for submit@debbugs.gnu.org; Sun, 04 Nov 2018 17:11:14 -0500 Received: from eggs.gnu.org ([208.118.235.92]:53297) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gJQbu-0001dM-Qg for 33259@debbugs.gnu.org; Sun, 04 Nov 2018 17:11:11 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gJQbn-0006Wr-Ru for 33259@debbugs.gnu.org; Sun, 04 Nov 2018 17:11:05 -0500 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:40146) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gJQbl-0006Nv-AW; Sun, 04 Nov 2018 17:11:01 -0500 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=33966 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1gJQbl-0004TQ-1j; Sun, 04 Nov 2018 17:11:01 -0500 From: Ludovic =?utf-8?q?Court=C3=A8s?= Date: Sun, 4 Nov 2018 23:10:32 +0100 Message-Id: <20181104221036.4776-4-ludo@gnu.org> X-Mailer: git-send-email 2.19.1 In-Reply-To: <20181104221036.4776-1-ludo@gnu.org> References: <20181104221036.4776-1-ludo@gnu.org> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 208.118.235.43 X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches * guix/docker.scm (build-docker-image): Add #:database parameter. Create /var/guix/db, /var/guix/profiles, etc. when DATABASE is true. * guix/scripts/pack.scm (docker-image): Export. Remove #:deduplicate? parameter. Define 'database' and pass it to 'docker-image'. * tests/pack.scm (test-assertm): Recompile the derivation of %BOOTSTRAP-GUILE. ("docker-image + localstatedir"): New test. --- guix/docker.scm | 16 ++++++++++++- guix/scripts/pack.scm | 9 +++++++- tests/pack.scm | 53 +++++++++++++++++++++++++++++++++++++++++-- 3 files changed, 74 insertions(+), 4 deletions(-) diff --git a/guix/docker.scm b/guix/docker.scm index 0757d3356f..c19a24d45c 100644 --- a/guix/docker.scm +++ b/guix/docker.scm @@ -26,6 +26,7 @@ delete-file-recursively with-directory-excursion invoke)) + #:use-module (gnu build install) #:use-module (json) ;guile-json #:use-module (srfi srfi-19) #:use-module (srfi srfi-26) @@ -108,11 +109,15 @@ return \"a\"." (symlinks '()) (transformations '()) (system (utsname:machine (uname))) + database compressor (creation-time (current-time time-utc))) "Write to IMAGE a Docker image archive containing the given PATHS. PREFIX must be a store path that is a prefix of any store paths in PATHS. +When DATABASE is true, copy it to /var/guix/db in the image and create +/var/guix/gcroots and friends. + SYMLINKS must be a list of (SOURCE -> TARGET) tuples describing symlinks to be created in the image, where each TARGET is relative to PREFIX. TRANSFORMATIONS must be a list of (OLD -> NEW) tuples describing how to @@ -188,10 +193,15 @@ SRFI-19 time-utc object, as the creation time in metadata." source)))) symlinks) + (when database + ;; Initialize /var/guix, assuming PREFIX points to a profile. + (install-database-and-gc-roots "." database prefix)) + (apply invoke "tar" "-cf" "layer.tar" `(,@transformation-options ,@%tar-determinism-options ,@paths + ,@(if database '("var") '()) ,@(map symlink-source symlinks))) ;; It is possible for "/" to show up in the archive, especially when ;; applying transformations. For example, the transformation @@ -203,7 +213,11 @@ SRFI-19 time-utc object, as the creation time in metadata." (system* "tar" "--delete" "/" "-f" "layer.tar") (for-each delete-file-recursively (map (compose topmost-component symlink-source) - symlinks))) + symlinks)) + + ;; Delete /var/guix. + (when database + (delete-file-recursively "var"))) (with-output-to-file "config.json" (lambda () diff --git a/guix/scripts/pack.scm b/guix/scripts/pack.scm index 3e6430bcce..09fc88988a 100644 --- a/guix/scripts/pack.scm +++ b/guix/scripts/pack.scm @@ -52,6 +52,8 @@ #:export (compressor? lookup-compressor self-contained-tarball + docker-image + guix-pack)) ;; Type of a compression tool. @@ -360,7 +362,6 @@ added to the pack." (define* (docker-image name profile #:key target - deduplicate? (compressor (first %compressors)) localstatedir? (symlinks '()) @@ -370,6 +371,11 @@ image is a tarball conforming to the Docker Image Specification, compressed with COMPRESSOR. It can be passed to 'docker load'. If TARGET is true, it must a be a GNU triplet and it is used to derive the architecture metadata in the image." + (define database + (and localstatedir? + (file-append (store-database (list profile)) + "/db/db.sqlite"))) + (define defmod 'define-module) ;trick Geiser (define build @@ -388,6 +394,7 @@ the image." (call-with-input-file "profile" read-reference-graph)) #$profile + #:database #+database #:system (or #$target (utsname:machine (uname))) #:symlinks '#$symlinks #:compressor '#$(compressor-command compressor) diff --git a/tests/pack.scm b/tests/pack.scm index 6bd18bdee2..e8d4f9f18d 100644 --- a/tests/pack.scm +++ b/tests/pack.scm @@ -22,6 +22,7 @@ #:use-module (guix store) #:use-module (guix derivations) #:use-module (guix profiles) + #:use-module (guix packages) ;XXX: debugging #:use-module (guix monads) #:use-module (guix grafts) #:use-module (guix tests) @@ -37,8 +38,9 @@ (define-syntax-rule (test-assertm name store exp) (test-assert name - (run-with-store store exp - #:guile-for-build (%guile-for-build)))) + (let ((guile (package-derivation store %bootstrap-guile))) + (run-with-store store exp + #:guile-for-build guile)))) (define %gzip-compressor ;; Compressor that uses the bootstrap 'gzip'. @@ -79,6 +81,53 @@ (readlink "bin/Guile")))))))) (built-derivations (list check)))) +;; The following test needs guile-sqlite3, libgcrypt, etc. as a consequence of +;; commit c45477d2a1a651485feede20fe0f3d15aec48b39 and related changes. Thus, +;; run it on the user's store, if it's available, on the grounds that these +;; dependencies may be already there, or we can get substitutes or build them +;; quite inexpensively; see . + +(with-external-store store + (unless store (test-skip 1)) + (test-assertm "docker-image + localstatedir" store + (mlet* %store-monad + ((guile (set-guile-for-build (default-guile))) + (profile (profile-derivation (packages->manifest + (list %bootstrap-guile)) + #:hooks '() + #:locales? #f)) + (tarball (docker-image "docker-pack" profile + #:symlinks '(("/bin/Guile" -> "bin/guile")) + #:localstatedir? #t)) + (check (gexp->derivation + "check-tarball" + (with-imported-modules '((guix build utils)) + #~(begin + (use-modules (guix build utils) + (ice-9 match)) + + (define bin + (string-append "." #$profile "/bin")) + + (setenv "PATH" (string-append #$%tar-bootstrap "/bin")) + (mkdir "base") + (with-directory-excursion "base" + (invoke "tar" "xvf" #$tarball)) + + (match (find-files "base" "layer.tar") + ((layer) + (invoke "tar" "xvf" layer))) + + (when + (and (file-exists? (string-append bin "/guile")) + (file-exists? "var/guix/db/db.sqlite") + (string=? (string-append #$%bootstrap-guile "/bin") + (pk 'binlink (readlink bin))) + (string=? (string-append #$profile "/bin/guile") + (pk 'guilelink (readlink "bin/Guile")))) + (mkdir #$output))))))) + (built-derivations (list check))))) + (test-end) ;; Local Variables: