From patchwork Mon Jun 21 06:11:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Maxim Cournoyer X-Patchwork-Id: 30563 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 137CC27BC81; Mon, 21 Jun 2021 07:13:30 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H2, SPF_HELO_PASS,T_DKIM_INVALID,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 529B827BC78 for ; Mon, 21 Jun 2021 07:13:29 +0100 (BST) Received: from localhost ([::1]:38956 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lvDBY-0008Hg-D3 for patchwork@mira.cbaines.net; Mon, 21 Jun 2021 02:13:28 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47926) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lvDB9-0007uD-5H for guix-patches@gnu.org; Mon, 21 Jun 2021 02:13:03 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:51893) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lvDB8-0001re-Tw for guix-patches@gnu.org; Mon, 21 Jun 2021 02:13:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1lvDB8-0006cQ-Qs for guix-patches@gnu.org; Mon, 21 Jun 2021 02:13:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#49149] [PATCH] tentatively reuse rlib for cargo-build-system Resent-From: Maxim Cournoyer Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Mon, 21 Jun 2021 06:13:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 49149 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 49149@debbugs.gnu.org Cc: Maxim Cournoyer Received: via spool by 49149-submit@debbugs.gnu.org id=B49149.162425596525337 (code B ref 49149); Mon, 21 Jun 2021 06:13:02 +0000 Received: (at 49149) by debbugs.gnu.org; 21 Jun 2021 06:12:45 +0000 Received: from localhost ([127.0.0.1]:35187 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lvDAr-0006aa-9V for submit@debbugs.gnu.org; Mon, 21 Jun 2021 02:12:45 -0400 Received: from mail-qt1-f172.google.com ([209.85.160.172]:42567) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lvDAp-0006aH-Lk for 49149@debbugs.gnu.org; Mon, 21 Jun 2021 02:12:44 -0400 Received: by mail-qt1-f172.google.com with SMTP id x21so3454466qtq.9 for <49149@debbugs.gnu.org>; Sun, 20 Jun 2021 23:12:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Z1X73fhCPAWV4PwnGSeUz2Pz/GvBtru7kZ+p33Emoek=; b=GuVtRoLhhdpOGPTdMwy4gt9GgE4WvLvEzCfMaMuukpbekRumujLdSylI/6F3d8n9pM 79/TgxkIeUhFvSu6QC9lfDd7ahTnA56X9gDbrKlrFeHlvPTKHpZVyIxUxLkEh6eieqy4 fSy9f5BW1PiAuzBl/OV7NrBsLGQ0d1xkqgJ2cq2bshhwbDjohk6MjpF+GyGoW/EsVUbG 3XmYwnEjvIcx/pICaJfNFrdHLbTMkvwqwbtELRfNiUMZypT9xwmodWjzWJOiXBMvuVvk PRP+1bEztvVZ8mvwNGjILazJkfkwxD6YEVruG8NBmbUrVFY4ce0TmMgNYY7anTa6byCY hcXg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Z1X73fhCPAWV4PwnGSeUz2Pz/GvBtru7kZ+p33Emoek=; b=LkhCCkQYItm/AfKyqsK9twi70loI7xqOkTIYV8AcAGQDmRRpO08zaQlax4GhZ/Rblx OQyWh3G+P+vtMuMFU2FHi6kaGbg30rnjGRnnFlOfbeukZ9lhMzsU89YYEfUWlJLlHht8 3qngbNv3BpYx0DtQJpH62rWchp+Fc8y6C+5BFHcnNgLw9hauRLdCvOEc4MuSmQRUzHNY IEcQ2x/+i31pChYlcdTrhKpEcZ0hpacQSipwYtPQBjjtfvm+C2EAe/WisVFJuTwA6gwN GowYbOKWLP/tyuvKLo4Tk1OctbHIw0FuY5ILyAOwQwHvvCCJPX6y3K0E9RteVZN+gX/L FAyg== X-Gm-Message-State: AOAM5301yz6SJbMJa3Vz1Qai6kkbAg6DpUleDIkrO8GX+rqwBcolfChD fGpnZOJZP+9jVObWYwQZlDwROtTp0jNi7A== X-Google-Smtp-Source: ABdhPJy7aedqmMEGhYnzMyYOWC2VuLOhQuuTQ1+xIYxITN/jYtZ4S57u6YSJf1Pzm65/QAwssLXwvA== X-Received: by 2002:ac8:7516:: with SMTP id u22mr22216679qtq.160.1624255958050; Sun, 20 Jun 2021 23:12:38 -0700 (PDT) Received: from localhost.localdomain (dsl-148-219.b2b2c.ca. [66.158.148.219]) by smtp.gmail.com with ESMTPSA id i11sm8478663qke.74.2021.06.20.23.12.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 20 Jun 2021 23:12:37 -0700 (PDT) From: Maxim Cournoyer Date: Mon, 21 Jun 2021 02:11:59 -0400 Message-Id: <20210621061205.31878-2-maxim.cournoyer@gmail.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210621061205.31878-1-maxim.cournoyer@gmail.com> References: <20210621061205.31878-1-maxim.cournoyer@gmail.com> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches --- guix/build-system/cargo.scm | 3 +- guix/build/cargo-build-system.scm | 78 ++++++++++++++++++++++++++----- 2 files changed, 68 insertions(+), 13 deletions(-) diff --git a/guix/build-system/cargo.scm b/guix/build-system/cargo.scm index e53d2a7523..9ef9f6b149 100644 --- a/guix/build-system/cargo.scm +++ b/guix/build-system/cargo.scm @@ -271,7 +271,8 @@ any dependent crates. This can be a benefits: (build-inputs `(("cargo" ,rust "cargo") ("rustc" ,rust) ,@(expand-crate-sources cargo-inputs cargo-development-inputs) - ,@native-inputs)) + ,@native-inputs + ,@(if target '() inputs))) (outputs outputs) (build cargo-build) (arguments (strip-keyword-arguments private-keywords arguments))))) diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm index 0a95672b00..e68f20e463 100644 --- a/guix/build/cargo-build-system.scm +++ b/guix/build/cargo-build-system.scm @@ -5,6 +5,7 @@ ;;; Copyright © 2019, 2020, 2021 Efraim Flashner ;;; Copyright © 2020 Jakub Kądziołka ;;; Copyright © 2020 Marius Bakke +;;; Copyright © 2021 Maxim Cournoyer ;;; ;;; This file is part of GNU Guix. ;;; @@ -24,7 +25,7 @@ (define-module (guix build cargo-build-system) #:use-module ((guix build gnu-build-system) #:prefix gnu:) #:use-module (guix build json) - #:use-module (guix build utils) + #:use-module ((guix build utils) #:hide (delete)) #:use-module (guix build cargo-utils) #:use-module (ice-9 popen) #:use-module (ice-9 rdelim) @@ -34,7 +35,10 @@ #:use-module (srfi srfi-1) #:use-module (srfi srfi-26) #:export (%standard-phases - cargo-build)) + cargo-build + + rust-version + rust-library-prefix)) ;; Commentary: ;; @@ -42,6 +46,25 @@ ;; ;; Code: +(define (rust-version rust) + "Return the version triplet (major.minor.patch) as a string, given RUST, a +store file name." + (let* ((version (last (string-split rust #\-))) + (components (string-split version #\.)) + (major+minor+patch (take components 3))) + (string-join major+minor+patch "."))) + +(define (rust-library-prefix/relative inputs) + "Return the relative versioned Rust library prefix where Rust libraries are +to be installed." + (string-append "lib/rust/" (rust-version (assoc-ref inputs "rustc")))) + +(define (rust-library-prefix inputs outputs) + "Return the absolute versioned Rust library prefix where Rust libraries are +to be installed." + (string-append (assoc-ref outputs "out") "/" + (rust-library-prefix/relative inputs))) + (define (manifest-targets) "Extract all targets from the Cargo.toml manifest" (let* ((port (open-input-pipe "cargo read-manifest")) @@ -73,6 +96,16 @@ Cargo.toml file present at its root." " | cut -d/ -f2" " | grep -q '^Cargo.toml$'"))))) +(define (rlib? file) + "Check if FILE has the .rlib extension." + (string-suffix? ".rlib" file)) + +(define (inputs->directories inputs) + "Extract the directory part from INPUTS." + (match inputs + (((names . directories) ...) + directories))) + (define* (unpack-rust-crates #:key inputs vendor-dir #:allow-other-keys) (define (inputs->rust-inputs inputs) "Filter using the label part from INPUTS." @@ -80,11 +113,6 @@ Cargo.toml file present at its root." (match input ((name . _) (rust-package? name)))) inputs)) - (define (inputs->directories inputs) - "Extract the directory part from INPUTS." - (match inputs - (((names . directories) ...) - directories))) (let ((rust-inputs (inputs->directories (inputs->rust-inputs inputs)))) (unless (null? rust-inputs) @@ -185,6 +213,22 @@ directory = '" port) (generate-all-checksums vendor-dir) #t) +(define* (populate-cargo-cache #:key inputs outputs #:allow-other-keys) + "Populate the 'target/release' directory with any pre-built Rust libraries, +to avoid rebuilding them from sources when possible." + (let* ((rust-lib-prefix (rust-library-prefix/relative inputs)) + (input-dirs (inputs->directories inputs)) + (rust-lib-dirs (filter (lambda (f) + (file-exists? (string-append + f "/" rust-lib-prefix))) + input-dirs)) + (rlibs (delete-duplicates (append-map (cut find-files <> "\\.rlib$") + rust-lib-dirs)))) + (pk 'rust-lib-dirs rust-lib-dirs) + (pk 'rlibs rlibs) + (for-each (cut install-file <> "target/release") rlibs) + (invoke "find" "target"))) + (define* (build #:key skip-build? (features '()) @@ -228,7 +272,9 @@ directory = '" port) "Install a given Cargo package." (let* ((out (assoc-ref outputs "out")) (registry (string-append out "/share/cargo/registry")) - (sources (string-append out "/share/cargo/src"))) + (sources (string-append out "/share/cargo/src")) + (libdir (rust-library-prefix inputs outputs)) + (release-dir "target/release")) (mkdir-p out) ;; Make cargo reuse all the artifacts we just built instead @@ -237,10 +283,17 @@ directory = '" port) ;; Only install crates which include binary targets, ;; otherwise cargo will raise an error. - (or skip-build? - (not (has-executable-target?)) - (invoke "cargo" "install" "--no-track" "--path" "." "--root" out - "--features" (string-join features))) + (unless skip-build? + ;; Install binaries. + (when (has-executable-target?) + (apply invoke "cargo" "install" "--no-track" "--path" "." "--root" out + (if (not (null? features)) + (list "--features" (string-join features)) + '()))) + ;; Install static libraries. + (for-each (lambda (file) + (install-file (string-append release-dir "/" file) libdir)) + (scandir release-dir (cut string-suffix? ".rlib" <>)))) (when install-source? ;; Install crate tarballs and unpacked sources for later use. @@ -260,6 +313,7 @@ directory = '" port) (modify-phases gnu:%standard-phases (delete 'bootstrap) (replace 'configure configure) + (add-before 'build 'populate-cargo-cache populate-cargo-cache) (replace 'build build) (replace 'check check) (replace 'install install) From patchwork Mon Jun 21 06:12:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxim Cournoyer X-Patchwork-Id: 30566 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 2D80527BC81; Mon, 21 Jun 2021 07:13:42 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H2, SPF_HELO_PASS,T_DKIM_INVALID autolearn=unavailable autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 0175327BC78 for ; Mon, 21 Jun 2021 07:13:42 +0100 (BST) Received: from localhost ([::1]:39584 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lvDBl-0000GK-4D for patchwork@mira.cbaines.net; Mon, 21 Jun 2021 02:13:41 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47938) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lvDB9-0007uZ-UI for guix-patches@gnu.org; Mon, 21 Jun 2021 02:13:03 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:51895) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lvDB9-0001tI-Mt for guix-patches@gnu.org; Mon, 21 Jun 2021 02:13:03 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1lvDB9-0006cf-Jd for guix-patches@gnu.org; Mon, 21 Jun 2021 02:13:03 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#49149] [PATCH 3/7] pack: Fix typo. Resent-From: Maxim Cournoyer Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Mon, 21 Jun 2021 06:13:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 49149 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 49149@debbugs.gnu.org Cc: Maxim Cournoyer Received: via spool by 49149-submit@debbugs.gnu.org id=B49149.162425597125384 (code B ref 49149); Mon, 21 Jun 2021 06:13:03 +0000 Received: (at 49149) by debbugs.gnu.org; 21 Jun 2021 06:12:51 +0000 Received: from localhost ([127.0.0.1]:35196 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lvDAx-0006bL-A0 for submit@debbugs.gnu.org; Mon, 21 Jun 2021 02:12:51 -0400 Received: from mail-qk1-f176.google.com ([209.85.222.176]:45759) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lvDAu-0006aQ-9C for 49149@debbugs.gnu.org; Mon, 21 Jun 2021 02:12:48 -0400 Received: by mail-qk1-f176.google.com with SMTP id d196so27550604qkg.12 for <49149@debbugs.gnu.org>; Sun, 20 Jun 2021 23:12:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=bR9tBuVVnovRAAc1CVUMdkJDi9V0lhIZZj6dRJ9yP0o=; b=BWOce8oJDqbC97s8D4c5eFV+il2QIg5dsCR3qEjkjNQf9OgyBlv+EIA5q2KHSX7DLb xkW1cnpjWXfmyW3rjdiqP6ytQE/5mpefkOPPfJ/L1P/U1nqVKL+DPuxsEkXRvr7u/adM 2XloMCslDy5Huifu08vsYQwiMc5uYWtPeZHfK9RcYmd3e6XS6NvgrHy8D14HYRRuJshX mRnSrBMBevvZ3+gAF+nsrLG1uiQRwGhJ25e7TZlU2njiVhiHJl7FnkXpFzRoRRLrl6la 4ca1j6MXmxA500XccL7SSKc9mlIUDALY6n90Bfh1GGXUUxm5SM7F9b47WGY78aJyxVBK 6hYw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=bR9tBuVVnovRAAc1CVUMdkJDi9V0lhIZZj6dRJ9yP0o=; b=dEQx6ra/4DHysfZ6509bLfMTFkLxBEizxZ+ru66EkVjnS0ViyanFaIo96UD6Ish0p3 K+/B1QMb2LJjMXcbxSMoldaS7h0l0coUB4KWU7oYa4yU2BppYFaOZGX4gOW3r4IWZHoO xMfjypdm2y0roakaAMwlMOhQK7sI0w0AMDlGon3+GAD+WwgNV0qKRCDsXiU3VTaRyd83 1qpQaNRmKXQfDvAAYI14u4u0pD6spiZMeeBCM+AfDg8qUB3z9aqEGAWgfLH/umNDTWKi RDkXdotF6zuZ7Zpn33ZxYP7pKtpcSgPmq8cYrJBCeYTazN8k1loMHMQbhVXGZqxqbAjv ZZKQ== X-Gm-Message-State: AOAM530I6dbmgvjBu1aaSxJYY6sIVIbol7wf8Nx4MfddRqavEbg2Gj5U 60X/o1w2RrOvw3JhkaYyq6hClU/yTeZGDw== X-Google-Smtp-Source: ABdhPJytNnR+5KIX9ya27YgyCqL7Wrk/wTk2cJJb7shuSQkvLiEteqEZMOmNTtU5zcslvfjKmdqfAw== X-Received: by 2002:a37:684b:: with SMTP id d72mr21825313qkc.372.1624255962848; Sun, 20 Jun 2021 23:12:42 -0700 (PDT) Received: from localhost.localdomain (dsl-148-219.b2b2c.ca. [66.158.148.219]) by smtp.gmail.com with ESMTPSA id i11sm8478663qke.74.2021.06.20.23.12.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 20 Jun 2021 23:12:42 -0700 (PDT) From: Maxim Cournoyer Date: Mon, 21 Jun 2021 02:12:01 -0400 Message-Id: <20210621061205.31878-4-maxim.cournoyer@gmail.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210621061205.31878-1-maxim.cournoyer@gmail.com> References: <20210621061205.31878-1-maxim.cournoyer@gmail.com> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches * guix/scripts/pack.scm (self-contained-tarball/builder): Fix typo. --- guix/scripts/pack.scm | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/guix/scripts/pack.scm b/guix/scripts/pack.scm index d11f498925..7ea97a4b7a 100644 --- a/guix/scripts/pack.scm +++ b/guix/scripts/pack.scm @@ -229,7 +229,7 @@ dependencies are registered." (let ((target (string-append #$profile "/" target)) (parent (dirname source))) ;; Never add a 'directory' directive for "/" so as to - ;; preserve its ownnership when extracting the archive (see + ;; preserve its ownership when extracting the archive (see ;; below), and also because this would lead to adding the ;; same entries twice in the tarball. `(,@(if (string=? parent "/") From patchwork Mon Jun 21 06:12:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxim Cournoyer X-Patchwork-Id: 30567 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 3EE0027BC81; Mon, 21 Jun 2021 07:13:48 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H2, SPF_HELO_PASS,T_DKIM_INVALID autolearn=unavailable autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id BB53227BC78 for ; Mon, 21 Jun 2021 07:13:47 +0100 (BST) Received: from localhost ([::1]:40006 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lvDBq-0000Z9-RE for patchwork@mira.cbaines.net; Mon, 21 Jun 2021 02:13:46 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47942) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lvDBA-0007vP-CK for guix-patches@gnu.org; Mon, 21 Jun 2021 02:13:04 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:51896) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lvDBA-0001th-3r for guix-patches@gnu.org; Mon, 21 Jun 2021 02:13:04 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1lvDBA-0006cm-0Q for guix-patches@gnu.org; Mon, 21 Jun 2021 02:13:04 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#49149] [PATCH 4/7] pack: Improve naming of the packs store file names. Resent-From: Maxim Cournoyer Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Mon, 21 Jun 2021 06:13:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 49149 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 49149@debbugs.gnu.org Cc: Maxim Cournoyer Received: via spool by 49149-submit@debbugs.gnu.org id=B49149.162425597125391 (code B ref 49149); Mon, 21 Jun 2021 06:13:03 +0000 Received: (at 49149) by debbugs.gnu.org; 21 Jun 2021 06:12:51 +0000 Received: from localhost ([127.0.0.1]:35198 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lvDAx-0006bN-Gw for submit@debbugs.gnu.org; Mon, 21 Jun 2021 02:12:51 -0400 Received: from mail-qk1-f171.google.com ([209.85.222.171]:35488) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lvDAv-0006aX-QZ for 49149@debbugs.gnu.org; Mon, 21 Jun 2021 02:12:50 -0400 Received: by mail-qk1-f171.google.com with SMTP id q190so24939267qkd.2 for <49149@debbugs.gnu.org>; Sun, 20 Jun 2021 23:12:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=v/TsY7TUeHjjKKjuzsJGN8fmXHrFAsNNkILKEI0FWYI=; b=psYIXNjXvH5cb+0EF/NWbr4FsWM9hxGZtBWYR7uOi1B35tkfUaSjAZXFWKLuFzLsel qtD9Oo39SCICdu3OdDUMmjLu0J5VaDwUS1yJwDi1i0VD0u8hNNPYz+aeYPawpiySTmck gpffgNAyuxZXGMsKN3nZlfz9gtuUDDzYY0Q4sz2DsGaAZlz/SGdcB5vvTeVe5ylnmnM9 2mOPtzaoi4jmF3879rEqkIuK9fMufbdCAKSy/I6h9M3JNECmE9M0+Dtek6UUDUpxVbDO 8EWgq0OGnpASysqFKnyr5gvbWcPRr31UP1Pdq7sKtdO0MociXU2iAZlj3oqvUyjZ7Kiy nhRg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=v/TsY7TUeHjjKKjuzsJGN8fmXHrFAsNNkILKEI0FWYI=; b=hA1ZlTMMIGi5WCY0kYsmKFbcOLu91dk+1fskjEDB2aIZ+GmtipeF6UpT8eXi8kAoDW KJOyxdl20gHd3LlmLS2ggoSxwNiPa5ItiwolS/opd+OIWcKjnsaBltCAXAOKzWLNfYo4 zfvytCer1sfcn6h7dpBIs3TENhMQmX4PlFpZMU8brm43UqEG2fE7OXlOtiH83A6u9h71 UeM4ZUSpToAK52WBD0kgAqch/tQkLKCNSKEjWjouC/KUGb51L90V/7Z7/TvImK25iSTR 0O3l9+VQd4o0FBDKXDfFPoMmPsqxCvCuEAwCTJK5Qx6a0NzF+SLWKMcZi010RDedVsd6 v7oA== X-Gm-Message-State: AOAM532VOpkz029g2eZxwWrNBFBWbytR8geNsRohhVQLOuGCNY0vCaqR HDK6QccmmhT0GnEtCXhRejk5J5CEp+B2Jw== X-Google-Smtp-Source: ABdhPJzbtxaphj68wsORpAEvS4vDRtmaaEqjsmZG3TITxky8WVa0EdK3ORNaU1OUejpORE+NrW0/qA== X-Received: by 2002:a37:b881:: with SMTP id i123mr21390929qkf.207.1624255964252; Sun, 20 Jun 2021 23:12:44 -0700 (PDT) Received: from localhost.localdomain (dsl-148-219.b2b2c.ca. [66.158.148.219]) by smtp.gmail.com with ESMTPSA id i11sm8478663qke.74.2021.06.20.23.12.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 20 Jun 2021 23:12:44 -0700 (PDT) From: Maxim Cournoyer Date: Mon, 21 Jun 2021 02:12:02 -0400 Message-Id: <20210621061205.31878-5-maxim.cournoyer@gmail.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210621061205.31878-1-maxim.cournoyer@gmail.com> References: <20210621061205.31878-1-maxim.cournoyer@gmail.com> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches Instead of just naming them by their pack type, add information from the package(s) they contain to make it easier to differentiate them. * guix/scripts/pack.scm (manifest->friendly-name): Extract procedure from ... (docker-image): ... here. Adjust REPOSITORY argument value accordingly. (guix-pack): Derive NAME using MANIFEST->FRIENDLY-NAME. --- guix/scripts/pack.scm | 44 +++++++++++++++++++++++++------------------ 1 file changed, 26 insertions(+), 18 deletions(-) diff --git a/guix/scripts/pack.scm b/guix/scripts/pack.scm index 7ea97a4b7a..9d4bb9f497 100644 --- a/guix/scripts/pack.scm +++ b/guix/scripts/pack.scm @@ -172,6 +172,23 @@ dependencies are registered." (computed-file "store-database" build #:options `(#:references-graphs ,(zip labels items)))) +;;; XXX: The following procedure has to *also* be used in the build side +;;; G-Exp, because PROFILE is passed as a derivation in the tests. +(define define-manifest->friendly-name + '(define (manifest->friendly-name manifest) + "Return a friendly name computed from the entries in MANIFEST, a + object." + (let loop ((names (map manifest-entry-name + (manifest-entries manifest)))) + (define str (string-join names "-")) + (if (< (string-length str) 40) + str + (match names + ((_) str) + ((names ... _) (loop names))))))) ;drop one entry + +(eval define-manifest->friendly-name (current-module)) + ;;; ;;; Tarball format. @@ -540,7 +557,7 @@ the image." (file-append (store-database (list profile)) "/db/db.sqlite"))) - (define defmod 'define-module) ;trick Geiser + (define defmod 'define-module) ;trick Geiser (define build ;; Guile-JSON and Guile-Gcrypt are required by (guix docker). @@ -558,6 +575,8 @@ the image." (srfi srfi-1) (srfi srfi-19) (ice-9 match)) + #$define-manifest->friendly-name + (define environment (map (match-lambda ((spec . value) @@ -581,19 +600,6 @@ the image." `((directory "/tmp" ,(getuid) ,(getgid) #o1777) ,@(append-map symlink->directives '#$symlinks))) - (define tag - ;; Compute a meaningful "repository" name, which will show up in - ;; the output of "docker images". - (let ((manifest (profile-manifest #$profile))) - (let loop ((names (map manifest-entry-name - (manifest-entries manifest)))) - (define str (string-join names "-")) - (if (< (string-length str) 40) - str - (match names - ((_) str) - ((names ... _) (loop names))))))) ;drop one entry - (setenv "PATH" #+(file-append archiver "/bin")) (build-docker-image #$output @@ -601,7 +607,8 @@ the image." (call-with-input-file "profile" read-reference-graph)) #$profile - #:repository tag + #:repository (manifest->friendly-name + (profile-manifest #$profile)) #:database #+database #:system (or #$target %host-type) #:environment environment @@ -1209,8 +1216,6 @@ Create a bundle of PACKAGE.\n")) manifest) manifest))) (pack-format (assoc-ref opts 'format)) - (name (string-append (symbol->string pack-format) - "-pack")) (target (assoc-ref opts 'target)) (bootstrap? (assoc-ref opts 'bootstrap?)) (compressor (if bootstrap? @@ -1244,7 +1249,10 @@ Create a bundle of PACKAGE.\n")) (hooks (if bootstrap? '() %default-profile-hooks)) - (locales? (not bootstrap?))))) + (locales? (not bootstrap?)))) + (name (string-append (manifest->friendly-name manifest) + "-" (symbol->string pack-format) + "-pack"))) (define (lookup-package package) (manifest-lookup manifest (manifest-pattern (name package)))) From patchwork Mon Jun 21 06:12:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxim Cournoyer X-Patchwork-Id: 30564 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id D809927BC81; Mon, 21 Jun 2021 07:13:31 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H2, SPF_HELO_PASS,T_DKIM_INVALID autolearn=unavailable autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id B149527BC78 for ; Mon, 21 Jun 2021 07:13:31 +0100 (BST) Received: from localhost ([::1]:39122 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lvDBa-0008Ok-Ns for patchwork@mira.cbaines.net; Mon, 21 Jun 2021 02:13:30 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47948) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lvDBB-0007x1-6S for guix-patches@gnu.org; Mon, 21 Jun 2021 02:13:05 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:51898) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lvDBA-0001ty-Sq for guix-patches@gnu.org; Mon, 21 Jun 2021 02:13:04 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1lvDBA-0006d3-Pw for guix-patches@gnu.org; Mon, 21 Jun 2021 02:13:04 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#49149] [PATCH 6/7] tests: pack: Fix compressor extension. Resent-From: Maxim Cournoyer Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Mon, 21 Jun 2021 06:13:04 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 49149 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 49149@debbugs.gnu.org Cc: Maxim Cournoyer Received: via spool by 49149-submit@debbugs.gnu.org id=B49149.162425597225406 (code B ref 49149); Mon, 21 Jun 2021 06:13:04 +0000 Received: (at 49149) by debbugs.gnu.org; 21 Jun 2021 06:12:52 +0000 Received: from localhost ([127.0.0.1]:35203 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lvDAy-0006bc-AU for submit@debbugs.gnu.org; Mon, 21 Jun 2021 02:12:52 -0400 Received: from mail-qk1-f176.google.com ([209.85.222.176]:40746) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lvDAw-0006ab-N8 for 49149@debbugs.gnu.org; Mon, 21 Jun 2021 02:12:51 -0400 Received: by mail-qk1-f176.google.com with SMTP id q64so21245717qke.7 for <49149@debbugs.gnu.org>; Sun, 20 Jun 2021 23:12:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=qSAIxVQhLym4ockDJVf5oH4DzRJDWkmIYemB/0gAtOg=; b=X3x07glzHH6igBQEDCe1X0VKzpRoXE2wfiaRLbBQUsgp48SEMTsUsykilkvsv3/E7T jybBF+UQ64vkaWFcCyMir/IViAvo1PD+ArsDaxl1jk594t7wgNbne/wqenh61g0s9Gx1 /3HAXftR2IF3qCPiyKy1bR600KtRP+SEtjCMhY5+DgKGFJOWugHkmHQCXAoZ6ot+WTYH QY+frR1f3yRLnF2OdyRF7JTCTFcsNZJBRR9JFflCl9kx/cafUBZsUE11/qFjOEMp0dIq OOfHmjJ5+PVh3rNR2HE4cKqK3rjoySr2LIwuvai7Dws3fCexp9A9YZPzrmlqPVKSPIId EiOw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=qSAIxVQhLym4ockDJVf5oH4DzRJDWkmIYemB/0gAtOg=; b=qPCyI4FyB0RNLJbJ7F71onsbXWNnLjJOhuA2oPAlP+dX2YLkMFculbh7T2/lu5Yy0Y fxZtBzh+pEMUjURBMGb9Eb0etOyF0y7YHIHtE85UVSHFIzKIIXBVRnvft+ZoBlCMLrE7 lYXO1Dj9y1B0/xS+4OFpaeeaaGgn9nyqA7EgUwMq7hgR2R2z8VcBeSz8GXukq84NHk6s 2ES5gUQc0tmZcdQ2DQ9v1DHQaV9DtFAZNy0RbFLfyUymgCjKwiEgQWjAX0lL4fujf25X qsns+wcRr7Jmuxc5pb4L73BIfF8E2elK/GtgY5lzGfjsv0bTnD9JfXfyTJ5i0VQHo8F5 HqZw== X-Gm-Message-State: AOAM532upURzLa0dbnfYKNh7qoDYnzv/x8jmLmzsZqOB6Ql/gWRwTkXz V9gOh+Rzk71L5oDqD9hcb736bsOPoNuXHA== X-Google-Smtp-Source: ABdhPJx2nFRean3RbgAi3qaimHEW2sROIE8ouTmlwLT/QVo8uupW6mD/oNYcBzx41GD/WVECJuiYwQ== X-Received: by 2002:a05:620a:1241:: with SMTP id a1mr9236193qkl.484.1624255965377; Sun, 20 Jun 2021 23:12:45 -0700 (PDT) Received: from localhost.localdomain (dsl-148-219.b2b2c.ca. [66.158.148.219]) by smtp.gmail.com with ESMTPSA id i11sm8478663qke.74.2021.06.20.23.12.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 20 Jun 2021 23:12:45 -0700 (PDT) From: Maxim Cournoyer Date: Mon, 21 Jun 2021 02:12:04 -0400 Message-Id: <20210621061205.31878-7-maxim.cournoyer@gmail.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210621061205.31878-1-maxim.cournoyer@gmail.com> References: <20210621061205.31878-1-maxim.cournoyer@gmail.com> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches * tests/pack.scm (%gzip-compressor): Add the missing leading period to the gzip compressor file extension. --- tests/pack.scm | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/pack.scm b/tests/pack.scm index e8455b4f37..ae6247a1d5 100644 --- a/tests/pack.scm +++ b/tests/pack.scm @@ -51,7 +51,7 @@ (define %gzip-compressor ;; Compressor that uses the bootstrap 'gzip'. ((@ (guix scripts pack) compressor) "gzip" - "gz" + ".gz" #~(#+(file-append %bootstrap-coreutils&co "/bin/gzip") "-6n"))) (define %tar-bootstrap %bootstrap-coreutils&co) From patchwork Mon Jun 21 06:12:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Maxim Cournoyer X-Patchwork-Id: 30565 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 91C8C27BC81; Mon, 21 Jun 2021 07:13:34 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H2, SPF_HELO_PASS,T_DKIM_INVALID,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id AE01727BC78 for ; Mon, 21 Jun 2021 07:13:33 +0100 (BST) Received: from localhost ([::1]:39290 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lvDBc-0008WA-PA for patchwork@mira.cbaines.net; Mon, 21 Jun 2021 02:13:32 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47952) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lvDBB-0007yF-JG for guix-patches@gnu.org; Mon, 21 Jun 2021 02:13:05 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:51899) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lvDBB-0001uT-AI for guix-patches@gnu.org; Mon, 21 Jun 2021 02:13:05 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1lvDBB-0006dA-6I for guix-patches@gnu.org; Mon, 21 Jun 2021 02:13:05 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#49149] [PATCH 7/7] pack: Add support for the deb format. Resent-From: Maxim Cournoyer Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Mon, 21 Jun 2021 06:13:05 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 49149 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 49149@debbugs.gnu.org Cc: Maxim Cournoyer Received: via spool by 49149-submit@debbugs.gnu.org id=B49149.162425598125427 (code B ref 49149); Mon, 21 Jun 2021 06:13:05 +0000 Received: (at 49149) by debbugs.gnu.org; 21 Jun 2021 06:13:01 +0000 Received: from localhost ([127.0.0.1]:35205 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lvDB6-0006c2-NT for submit@debbugs.gnu.org; Mon, 21 Jun 2021 02:13:01 -0400 Received: from mail-qv1-f46.google.com ([209.85.219.46]:34686) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lvDAx-0006aj-Qh for 49149@debbugs.gnu.org; Mon, 21 Jun 2021 02:12:53 -0400 Received: by mail-qv1-f46.google.com with SMTP id 5so6884565qvf.1 for <49149@debbugs.gnu.org>; Sun, 20 Jun 2021 23:12:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=cP3aVdwN3atijV5JRtQpUyf/pI33KDNEeFpdCFMprWc=; b=SHl1eQKVHKqVuPd/9sH+dgMrG0gkbG6Onx4fQTcAGIb5igKOvLrNdRka+StdZc7E6P Snhp6sioe1DE+rsueqH+infWtBVI8pc3h3l2OrxsDHRIJd/IbWRS7ReFV3coDJcoxp/x CdG/WQWV3NZo4sbeykboIUfETixFX/gHCefzPvtsuow6g/n1cMHOKEJUM6c/RPcb3G1/ A63LJy7lixIwnCyk9M+hlHU2GeIcfMdnkEH9D2woQGDwAx9ohGIysPgQNlnCJ9Ce7W7H txdh8/vlVDh1OVibepWS1gJGlBAsMS+s4ROkW6id/aphkx1XmYxTYA0+FXNNZjpb4iQ1 ++wA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=cP3aVdwN3atijV5JRtQpUyf/pI33KDNEeFpdCFMprWc=; b=d9fuPxi4f5ITPgEfnC7ZG6LJ3TkjaLyuVepNcGAZIle0RpQcYG2CZkhlV/IVwI149W f+w4DBoXax33Fd6mbG9qsYFHQxvA+Ha3NBt+wAsNnRTG8kF++nAkXXlV4ZRWEOWJKrEz 37RFuc7/eK3VBnaHYQJG5fdYKxv7tvW0lhs8ztFOvtrDTusm8bD/B0wheG+vLbj8Czt9 /dQ/ldm/T0v5azkjI/1yN/ohBKQ+75Uum/lNBsHPAclhmq4ypb0tQVxSlU/q0AzLMq66 +OsaC+2eVH777g5f5jG9IbRKEwS2KTx53yDoOXjIhv/Vknj8063olF26zQSFGsNAjDea weUA== X-Gm-Message-State: AOAM532WJgQUGgoukN78BhRtCpa6k4iV3MevupwA6MxTf2Toa31oaYPv nyT6ejx6J5R8afASZHjePCedIvKzTJ8Kjw== X-Google-Smtp-Source: ABdhPJxx/HJsl7t9dofjGf0B6ZlvfzWbdezhMM/bJtiNAriTHBN5cAzbihvCJocC711Rkkgv5aUkdg== X-Received: by 2002:ad4:4e47:: with SMTP id eb7mr18596338qvb.8.1624255966170; Sun, 20 Jun 2021 23:12:46 -0700 (PDT) Received: from localhost.localdomain (dsl-148-219.b2b2c.ca. [66.158.148.219]) by smtp.gmail.com with ESMTPSA id i11sm8478663qke.74.2021.06.20.23.12.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 20 Jun 2021 23:12:45 -0700 (PDT) From: Maxim Cournoyer Date: Mon, 21 Jun 2021 02:12:05 -0400 Message-Id: <20210621061205.31878-8-maxim.cournoyer@gmail.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210621061205.31878-1-maxim.cournoyer@gmail.com> References: <20210621061205.31878-1-maxim.cournoyer@gmail.com> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches * .dir-locals.el (scheme-mode)[gexp->derivation]: Define indentation rule. * guix/scripts/pack.scm (debian-archive): New procedure. (%formats): Register the new deb format. (show-formats): Add it to the usage string. * tests/pack.scm (%ar-bootstrap): New variable. (deb archive with symlinks): New test. * doc/guix.texi (Invoking guix pack): Document it. --- .dir-locals.el | 1 + doc/guix.texi | 5 ++ guix/scripts/pack.scm | 178 +++++++++++++++++++++++++++++++++++++++++- tests/pack.scm | 75 ++++++++++++++++++ 4 files changed, 258 insertions(+), 1 deletion(-) diff --git a/.dir-locals.el b/.dir-locals.el index 8f07a08eb5..a4fcbfe7ca 100644 --- a/.dir-locals.el +++ b/.dir-locals.el @@ -75,6 +75,7 @@ (eval . (put 'origin 'scheme-indent-function 0)) (eval . (put 'build-system 'scheme-indent-function 0)) (eval . (put 'bag 'scheme-indent-function 0)) + (eval . (put 'gexp->derivation 'scheme-indent-function 1)) (eval . (put 'graft 'scheme-indent-function 0)) (eval . (put 'operating-system 'scheme-indent-function 0)) (eval . (put 'file-system 'scheme-indent-function 0)) diff --git a/doc/guix.texi b/doc/guix.texi index 0930a514c7..7fb8d8e9d2 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -6016,6 +6016,11 @@ This produces a SquashFS image containing all the specified binaries and symlinks, as well as empty mount points for virtual file systems like procfs. +@item deb +This produces a Debian archive (a package with the @samp{.deb} file +extension) containing all the specified binaries and symlinks, that can +be installed on top of any dpkg-based GNU/Linux distribution. + @quotation Note Singularity @emph{requires} you to provide @file{/bin/sh} in the image. For that reason, @command{guix pack -f squashfs} always implies @code{-S diff --git a/guix/scripts/pack.scm b/guix/scripts/pack.scm index 8a108b7a1a..18f003dec0 100644 --- a/guix/scripts/pack.scm +++ b/guix/scripts/pack.scm @@ -6,6 +6,7 @@ ;;; Copyright © 2018 Efraim Flashner ;;; Copyright © 2020 Tobias Geerinckx-Rice ;;; Copyright © 2020 Eric Bavier +;;; Copyright © 2021 Maxim Cournoyer ;;; ;;; This file is part of GNU Guix. ;;; @@ -65,6 +66,7 @@ %compressors lookup-compressor self-contained-tarball + debian-archive docker-image squashfs-image @@ -341,6 +343,10 @@ added to the pack." #:target target #:references-graphs `(("profile" ,profile)))) + +;;; +;;; Singularity. +;;; (define (singularity-environment-file profile) "Return a shell script that defines the environment variables corresponding to the search paths of PROFILE." @@ -367,6 +373,10 @@ to the search paths of PROFILE." (computed-file "singularity-environment.sh" build)) + +;;; +;;; SquashFS image format. +;;; (define* (squashfs-image name profile #:key target (profile-name "guix-profile") @@ -541,6 +551,10 @@ added to the pack." #:target target #:references-graphs `(("profile" ,profile)))) + +;;; +;;; Docker image format. +;;; (define* (docker-image name profile #:key target (profile-name "guix-profile") @@ -628,6 +642,165 @@ the image." #:target target #:references-graphs `(("profile" ,profile)))) + +;;; +;;; Debian archive format. +;;; +;;; TODO: When relocatable option is selected, install to a unique prefix. +;;; This would enable installation of multiple deb packs with conflicting +;;; files at the same time. +;;; TODO: Allow passing a custom control file from the CLI. +;;; TODO: Allow providing a postinst script. +(define* (debian-archive name profile + #:key target + (profile-name "guix-profile") + deduplicate? + entry-point + (compressor (first %compressors)) + localstatedir? + (symlinks '()) + (archiver tar)) + "Return a Debian archive (.deb) containing a store initialized with the +closure of PROFILE, a derivation. The archive contains /gnu/store; if +LOCALSTATEDIR? is true, it also contains /var/guix, including /var/guix/db +with a properly initialized store database. The supported compressors are +\"none\", \"gz\" or \"xz\". + +SYMLINKS must be a list of (SOURCE -> TARGET) tuples denoting symlinks to be +added to the pack." + ;; For simplicity, limit the supported compressors to the superset of + ;; compressors able to compress both the control file (gz or xz) and the + ;; data tarball (gz, bz2 or xz). + (define %valid-compressors '("gzip" "xz" "none")) + + (let ((compressor-name (compressor-name compressor))) + (unless (member compressor-name %valid-compressors) + (leave (G_ "~a is not a valid Debian archive compressor. \ +Valid compressors are: ~a~%") compressor-name %valid-compressors))) + + (when entry-point + (warning (G_ "entry point not supported in the '~a' format~%") + 'deb)) + + (define data-tarball + (computed-file (string-append "data.tar" + (compressor-extension compressor)) + (self-contained-tarball/builder + profile + #:profile-name profile-name + #:compressor compressor + #:localstatedir? localstatedir? + #:symlinks symlinks + #:archiver archiver) + #:local-build? #f ;allow offloading + #:options (list #:references-graphs `(("profile" ,profile)) + #:target target))) + + (define build + (with-extensions (list guile-gcrypt) + (with-imported-modules `(((guix config) => ,(make-config.scm)) + ,@(source-module-closure + `((guix build pack) + (guix build utils) + (guix profiles)) + #:select? not-config?)) + #~(begin + (use-modules (guix build pack) + (guix build utils) + (guix profiles) + (ice-9 match) + (srfi srfi-1)) + + (define machine-type + ;; Extract the machine type from the specified target, else from the + ;; current system. + (and=> (or #$target %host-type) (lambda (triplet) + (first (string-split triplet #\-))))) + + (define (gnu-machine-type->debian-machine-type type) + "Translate machine TYPE from the GNU to Debian terminology." + ;; Debian has its own jargon, different from the one used in GNU, for + ;; machine types (see data/cputable in the sources of dpkg). + (match type + ("i686" "i386") + ("x86_64" "amd64") + ("aarch64" "arm64") + ("mipsisa32r6" "mipsr6") + ("mipsisa32r6el" "mipsr6el") + ("mipsisa64r6" "mips64r6") + ("mipsisa64r6el" "mips64r6el") + ("powerpcle" "powerpcel") + ("powerpc64" "ppc64") + ("powerpc64le" "ppc64el") + (machine machine))) + + (define architecture + (gnu-machine-type->debian-machine-type machine-type)) + + #$define-manifest->friendly-name + + (define manifest (profile-manifest #$profile)) + + (define single-entry ;manifest entry + (match (manifest-entries manifest) + ((entry) + entry) + (() #f))) + + (define package-name (or (and=> single-entry manifest-entry-name) + (manifest->friendly-name manifest))) + + (define package-version + (or (and=> single-entry manifest-entry-version) + "0.0.0")) + + (define debian-format-version "2.0") + + ;; Generate the debian-binary file. + (call-with-output-file "debian-binary" + (lambda (port) + (format port "~a~%" debian-format-version))) + + (define data-tarball-file-name (strip-store-file-name + #+data-tarball)) + + (copy-file #+data-tarball data-tarball-file-name) + + (define control-tarball-file-name + (string-append "control.tar" + #$(compressor-extension compressor))) + + ;; Write the compressed control tarball. Only the control file is + ;; mandatory (see: 'man deb' and 'man deb-control'). + (call-with-output-file "control" + (lambda (port) + (format port "\ +Package: ~a +Version: ~a +Description: Debian archive generated by GNU Guix. +Maintainer: GNU Guix +Architecture: ~a +~%" package-name package-version architecture))) + + (define tar (string-append #+archiver "/bin/tar")) + + (apply invoke tar + `(,@(tar-base-options + #:tar tar + #:compressor '#+(and=> compressor compressor-command)) + "-cvf" ,control-tarball-file-name + "control")) + + ;; Create the .deb archive using GNU ar. + (invoke (string-append #+binutils "/bin/ar") "-rv" #$output + "debian-binary" + control-tarball-file-name data-tarball-file-name))))) + + (gexp->derivation (string-append name ".deb") + build + #:target target + #:references-graphs `(("profile" ,profile)))) + ;;; ;;; Compiling C programs. @@ -960,7 +1133,8 @@ last resort for relocation." ;; Supported pack formats. `((tarball . ,self-contained-tarball) (squashfs . ,squashfs-image) - (docker . ,docker-image))) + (docker . ,docker-image) + (deb . ,debian-archive))) (define (show-formats) ;; Print the supported pack formats. @@ -972,6 +1146,8 @@ last resort for relocation." squashfs Squashfs image suitable for Singularity")) (display (G_ " docker Tarball ready for 'docker load'")) + (display (G_ " + deb Debian archive compatible, installable via dpkg/apt")) (newline)) (define %options diff --git a/tests/pack.scm b/tests/pack.scm index ae6247a1d5..ed461c6887 100644 --- a/tests/pack.scm +++ b/tests/pack.scm @@ -1,6 +1,7 @@ ;;; GNU Guix --- Functional package management for GNU ;;; Copyright © 2017, 2018, 2019, 2020 Ludovic Courtès ;;; Copyright © 2018 Ricardo Wurmus +;;; Copyright © 2021 Maxim Cournoyer ;;; ;;; This file is part of GNU Guix. ;;; @@ -32,6 +33,7 @@ #:use-module ((gnu packages base) #:select (glibc-utf8-locales)) #:use-module (gnu packages bootstrap) #:use-module ((gnu packages compression) #:select (squashfs-tools)) + #:use-module ((gnu packages debian) #:select (dpkg)) #:use-module ((gnu packages guile) #:select (guile-sqlite3)) #:use-module ((gnu packages gnupg) #:select (guile-gcrypt)) #:use-module (srfi srfi-64)) @@ -56,6 +58,8 @@ (define %tar-bootstrap %bootstrap-coreutils&co) +(define %ar-bootstrap %bootstrap-binutils) + (test-begin "pack") @@ -270,6 +274,77 @@ 1) (pk 'guilelink (readlink "bin")))) (mkdir #$output)))))))) + (built-derivations (list check)))) + + (unless store (test-skip 1)) + (test-assertm "deb archive with symlinks" store + (mlet* %store-monad + ((guile (set-guile-for-build (default-guile))) + (profile (profile-derivation (packages->manifest + (list %bootstrap-guile)) + #:hooks '() + #:locales? #f)) + (deb (debian-archive "deb-pack" profile + #:compressor %gzip-compressor + #:symlinks '(("/opt/gnu/bin" -> "bin")) + #:archiver %tar-bootstrap)) + (check + (gexp->derivation "check-deb-pack" + (with-imported-modules '((guix build utils)) + #~(begin + (use-modules (guix build utils) + (ice-9 match) + (ice-9 popen) + (ice-9 rdelim) + (ice-9 textual-ports) + (rnrs base)) + + (setenv "PATH" (string-join + (list (string-append #+%tar-bootstrap "/bin") + (string-append #+dpkg "/bin") + (string-append #+%ar-bootstrap "/bin")) + ":")) + + ;; Validate the output of 'dpkg --info'. + (let* ((port (open-pipe* OPEN_READ "dpkg" "--info" #$deb)) + (info (get-string-all port)) + (exit-val (status:exit-val (close-pipe port)))) + (assert (zero? exit-val)) + + (assert (string-contains + info + (string-append "Package: " + #+(package-name %bootstrap-guile)))) + + (assert (string-contains + info + (string-append "Version: " + #+(package-version %bootstrap-guile))))) + + ;; Sanity check .deb contents. + (invoke "ar" "-xv" #$deb) + (assert (file-exists? "debian-binary")) + (assert (file-exists? "data.tar.gz")) + (assert (file-exists? "control.tar.gz")) + + ;; Verify there are no hard links in data.tar.gz, as hard + ;; links would cause dpkg to fail unpacking the archive. + (define hard-links + (let ((port (open-pipe* OPEN_READ "tar" "-tvf" "data.tar.gz"))) + (let loop ((hard-links '())) + (match (pk 'line (read-line port)) + ((? eof-object?) + (assert (zero? (status:exit-val (close-pipe port)))) + hard-links) + (line + (if (string-prefix? "u" line) + (loop (cons line hard-links)) + (loop hard-links))))))) + + (unless (null? hard-links) + (error "hard links found in data.tar.gz" hard-links)) + + (mkdir #$output)))))) (built-derivations (list check))))) (test-end)