From patchwork Sat Oct 2 10:22:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Ludovic_Court=C3=A8s?= X-Patchwork-Id: 33541 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 5631727BBE3; Sat, 2 Oct 2021 11:23:20 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_MSPIKE_H2,SPF_HELO_PASS autolearn=unavailable autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id E5FF727BBE1 for ; Sat, 2 Oct 2021 11:23:19 +0100 (BST) Received: from localhost ([::1]:52052 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mWcAp-00023x-1w for patchwork@mira.cbaines.net; Sat, 02 Oct 2021 06:23:19 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38296) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mWcAZ-0001u2-14 for guix-patches@gnu.org; Sat, 02 Oct 2021 06:23:03 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:46411) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mWcAY-0007ld-Pp for guix-patches@gnu.org; Sat, 02 Oct 2021 06:23:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1mWcAY-0007Vq-L4 for guix-patches@gnu.org; Sat, 02 Oct 2021 06:23:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#50960] [PATCH 01/10] packages: Add 'package-development-inputs'. References: <20211002102116.27726-1-ludo@gnu.org> In-Reply-To: <20211002102116.27726-1-ludo@gnu.org> Resent-From: Ludovic =?utf-8?q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sat, 02 Oct 2021 10:23:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 50960 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 50960@debbugs.gnu.org Cc: Ludovic =?utf-8?q?Court=C3=A8s?= Received: via spool by 50960-submit@debbugs.gnu.org id=B50960.163317017928820 (code B ref 50960); Sat, 02 Oct 2021 10:23:02 +0000 Received: (at 50960) by debbugs.gnu.org; 2 Oct 2021 10:22:59 +0000 Received: from localhost ([127.0.0.1]:57946 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mWcAT-0007UZ-Vf for submit@debbugs.gnu.org; Sat, 02 Oct 2021 06:22:59 -0400 Received: from eggs.gnu.org ([209.51.188.92]:33114) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mWcAS-0007UB-H9 for 50960@debbugs.gnu.org; Sat, 02 Oct 2021 06:22:56 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:49188) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mWcAN-0007b7-Bf; Sat, 02 Oct 2021 06:22:51 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=36460 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mWcAJ-0007gb-7w; Sat, 02 Oct 2021 06:22:51 -0400 From: Ludovic =?utf-8?q?Court=C3=A8s?= Date: Sat, 2 Oct 2021 12:22:31 +0200 Message-Id: <20211002102240.27815-1-ludo@gnu.org> X-Mailer: git-send-email 2.33.0 MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches * guix/packages.scm (package-development-inputs): New procedure. * guix/scripts/environment.scm (package-environment-inputs): Use it. * tests/packages.scm ("package-development-inputs") ("package-development-inputs, cross-compilation"): New tests. * doc/guix.texi (package Reference): Document it. --- doc/guix.texi | 41 ++++++++++++++++++++++++++++++++++++ guix/packages.scm | 10 +++++++++ guix/scripts/environment.scm | 2 +- tests/packages.scm | 14 ++++++++++++ 4 files changed, 66 insertions(+), 1 deletion(-) diff --git a/doc/guix.texi b/doc/guix.texi index a72a726b54..49399e792b 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -6847,6 +6847,47 @@ cross-compiling: It is an error to refer to @code{this-package} outside a package definition. @end deffn +@cindex development inputs, of a package +@cindex implicit inputs, of a package +Sometimes you will want to obtain the list of inputs needed to +@emph{develop} a package---all the inputs that are visible when the +package is compiled. This is what the @code{package-development-inputs} +procedure returns. + +@deffn {Scheme Procedure} package-development-inputs @var{package} @ + [@var{system}] [#:target #f] +Return the list of inputs required by @var{package} for development +purposes on @var{system}. When @var{target} is true, return the inputs +needed to cross-compile @var{package} from @var{system} to +@var{triplet}, where @var{triplet} is a triplet such as +@code{"aarch64-linux-gnu"}. + +Note that the result includes both explicit inputs and implicit +inputs---inputs automatically added by the build system (@pxref{Build +Systems}). Let us take the @code{hello} package to illustrate that: + +@lisp +(use-modules (gnu packages base) (guix packages)) + +hello +@result{} # + +(package-direct-inputs hello) +@result{} () + +(package-development-inputs hello) +@result{} (("source" @dots{}) ("tar" #) @dots{}) +@end lisp + +In this example, @code{package-direct-inputs} returns the empty list, +because @code{hello} has zero explicit dependencies. Conversely, +@code{package-development-inputs} includes inputs implicitly added by +@code{gnu-build-system} that are required to build @code{hello}: tar, +gzip, GCC, libc, Bash, and more. To visualize it, @command{guix graph +hello} would show you explicit inputs, whereas @command{guix graph -t +bag hello} would include implicit inputs (@pxref{Invoking guix graph}). +@end deffn + Because packages are regular Scheme objects that capture a complete dependency graph and associated build procedures, it is often useful to write procedures that take a package and return a modified version diff --git a/guix/packages.scm b/guix/packages.scm index 8c3a0b0b7b..43e0130793 100644 --- a/guix/packages.scm +++ b/guix/packages.scm @@ -153,6 +153,7 @@ bag-transitive-host-inputs bag-transitive-build-inputs bag-transitive-target-inputs + package-development-inputs package-closure default-guile @@ -1070,6 +1071,15 @@ dependencies are known to build on SYSTEM." (%current-system (bag-system bag))) (transitive-inputs (bag-target-inputs bag)))) +(define* (package-development-inputs package + #:optional (system (%current-system)) + #:key target) + "Return the list of inputs required by PACKAGE for development purposes on +SYSTEM. When TARGET is true, return the inputs needed to cross-compile +PACKAGE from SYSTEM to TRIPLET, where TRIPLET is a triplet such as +\"aarch64-linux-gnu\"." + (bag-transitive-inputs (package->bag package system target))) + (define* (package-closure packages #:key (system (%current-system))) "Return the closure of PACKAGES on SYSTEM--i.e., PACKAGES and the list of packages they depend on, recursively." diff --git a/guix/scripts/environment.scm b/guix/scripts/environment.scm index 6958bd6238..d555969b27 100644 --- a/guix/scripts/environment.scm +++ b/guix/scripts/environment.scm @@ -82,7 +82,7 @@ package." packages for PACKAGE." ;; Remove non-package inputs such as origin records. (filter-map input->manifest-entry - (bag-transitive-inputs (package->bag package)))) + (package-development-inputs package system))) (define (show-help) (display (G_ "Usage: guix environment [OPTION]... PACKAGE... [-- COMMAND...] diff --git a/tests/packages.scm b/tests/packages.scm index 3756877270..266b5aeb7a 100644 --- a/tests/packages.scm +++ b/tests/packages.scm @@ -353,6 +353,20 @@ (package-transitive-supported-systems d) (package-transitive-supported-systems e)))) +(test-assert "package-development-inputs" + ;; Note: Due to propagated inputs, 'package-development-inputs' returns a + ;; couple more inputs, such as 'linux-libre-headers'. + (lset<= equal? + `(("source" ,(package-source hello)) ,@(standard-packages)) + (package-development-inputs hello))) + +(test-assert "package-development-inputs, cross-compilation" + (lset<= equal? + `(("source" ,(package-source hello)) + ,@(standard-cross-packages "mips64el-linux-gnu" 'host) + ,@(standard-cross-packages "mips64el-linux-gnu" 'target)) + (package-development-inputs hello #:target "mips64el-linux-gnu"))) + (test-assert "package-closure" (let-syntax ((dummy-package/no-implicit (syntax-rules () From patchwork Sat Oct 2 10:22:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Ludovic_Court=C3=A8s?= X-Patchwork-Id: 33540 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id F1D4027BBE3; Sat, 2 Oct 2021 11:23:14 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 8B0A127BBE1 for ; Sat, 2 Oct 2021 11:23:14 +0100 (BST) Received: from localhost ([::1]:51832 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mWcAj-0001uY-Mq for patchwork@mira.cbaines.net; Sat, 02 Oct 2021 06:23:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38298) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mWcAZ-0001uA-Fd for guix-patches@gnu.org; Sat, 02 Oct 2021 06:23:03 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:46412) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mWcAZ-0007ma-8L for guix-patches@gnu.org; Sat, 02 Oct 2021 06:23:03 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1mWcAZ-0007Vy-4b for guix-patches@gnu.org; Sat, 02 Oct 2021 06:23:03 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#50960] [PATCH 02/10] profiles: Add 'package->development-manifest'. Resent-From: Ludovic =?utf-8?q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sat, 02 Oct 2021 10:23:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 50960 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 50960@debbugs.gnu.org Cc: Ludovic =?utf-8?q?Court=C3=A8s?= Received: via spool by 50960-submit@debbugs.gnu.org id=B50960.163317018028842 (code B ref 50960); Sat, 02 Oct 2021 10:23:03 +0000 Received: (at 50960) by debbugs.gnu.org; 2 Oct 2021 10:23:00 +0000 Received: from localhost ([127.0.0.1]:57951 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mWcAW-0007Uz-5h for submit@debbugs.gnu.org; Sat, 02 Oct 2021 06:23:00 -0400 Received: from eggs.gnu.org ([209.51.188.92]:33116) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mWcAT-0007UF-RC for 50960@debbugs.gnu.org; Sat, 02 Oct 2021 06:22:58 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:49190) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mWcAN-0007c3-TB; Sat, 02 Oct 2021 06:22:51 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=36460 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mWcAN-0007gb-JG; Sat, 02 Oct 2021 06:22:51 -0400 From: Ludovic =?utf-8?q?Court=C3=A8s?= Date: Sat, 2 Oct 2021 12:22:32 +0200 Message-Id: <20211002102240.27815-2-ludo@gnu.org> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20211002102240.27815-1-ludo@gnu.org> References: <20211002102240.27815-1-ludo@gnu.org> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches From: Ludovic Courtès * guix/profiles.scm (package->development-manifest): New procedure. * guix/scripts/environment.scm (input->manifest-entry) (package-environment-inputs): Remove. * guix/scripts/environment.scm (options/resolve-packages): Use 'package->development-manifest' instead of 'package-environment-inputs'. * tests/profiles.scm ("package->development-manifest"): New test. --- doc/guix.texi | 11 +++++++++++ guix/profiles.scm | 19 +++++++++++++++++++ guix/scripts/environment.scm | 27 +++++---------------------- tests/profiles.scm | 7 +++++++ 4 files changed, 42 insertions(+), 22 deletions(-) diff --git a/doc/guix.texi b/doc/guix.texi index 49399e792b..bc3f5a537b 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -3340,6 +3340,17 @@ objects, like this: '("emacs" "guile@@2.2" "guile@@2.2:debug")) @end lisp +@findex package->development-manifest +You might also want to create a manifest for all the dependencies of a +package, rather than the package itself: + +@lisp +(package->development-manifest (specification->package "emacs")) +@end lisp + +The example above gives you all the software required to develop Emacs, +similar to what @command{guix environment emacs} provides. + @xref{export-manifest, @option{--export-manifest}}, to learn how to obtain a manifest file from an existing profile. diff --git a/guix/profiles.scm b/guix/profiles.scm index 2486f91d09..9f30349c69 100644 --- a/guix/profiles.scm +++ b/guix/profiles.scm @@ -124,6 +124,7 @@ profile-manifest package->manifest-entry + package->development-manifest packages->manifest ca-certificate-bundle %default-profile-hooks @@ -400,6 +401,24 @@ file name." (properties properties)))) entry)) +(define* (package->development-manifest package + #:optional + (system (%current-system)) + #:key target) + "Return a manifest for the \"development inputs\" of PACKAGE for SYSTEM, +optionally when cross-compiling to TARGET. Development inputs include both +explicit and implicit inputs of PACKAGE." + (manifest + (filter-map (match-lambda + ((label (? package? package)) + (package->manifest-entry package)) + ((label (? package? package) output) + (package->manifest-entry package output)) + ;; TODO: Support . + (_ + #f)) + (package-development-inputs package system #:target target)))) + (define (packages->manifest packages) "Return a list of manifest entries, one for each item listed in PACKAGES. Elements of PACKAGES can be either package objects or package/string tuples diff --git a/guix/scripts/environment.scm b/guix/scripts/environment.scm index d555969b27..54f48a7482 100644 --- a/guix/scripts/environment.scm +++ b/guix/scripts/environment.scm @@ -66,24 +66,6 @@ do not augment existing environment variables with additional search paths." (newline))) (profile-search-paths profile manifest))) -(define (input->manifest-entry input) - "Return a manifest entry for INPUT, or #f if INPUT does not correspond to a -package." - (match input - ((_ (? package? package)) - (package->manifest-entry package)) - ((_ (? package? package) output) - (package->manifest-entry package output)) - (_ - #f))) - -(define (package-environment-inputs package) - "Return a list of manifest entries corresponding to the transitive input -packages for PACKAGE." - ;; Remove non-package inputs such as origin records. - (filter-map input->manifest-entry - (package-development-inputs package system))) - (define (show-help) (display (G_ "Usage: guix environment [OPTION]... PACKAGE... [-- COMMAND...] Build an environment that includes the dependencies of PACKAGE and execute @@ -297,11 +279,11 @@ for the corresponding packages." ((? package? package) (if (eq? mode 'ad-hoc-package) (list (package->manifest-entry* package)) - (package-environment-inputs package))) + (manifest-entries (package->development-manifest package)))) (((? package? package) (? string? output)) (if (eq? mode 'ad-hoc-package) (list (package->manifest-entry* package output)) - (package-environment-inputs package))) + (manifest-entries (package->development-manifest package)))) ((lst ...) (append-map (cut packages->outputs <> mode) lst)))) @@ -313,8 +295,9 @@ for the corresponding packages." (specification->package+output spec))) (list (package->manifest-entry* package output)))) (('package 'package (? string? spec)) - (package-environment-inputs - (transform (specification->package+output spec)))) + (manifest-entries + (package->development-manifest + (transform (specification->package+output spec))))) (('expression mode str) ;; Add all the outputs of the package STR evaluates to. (packages->outputs (read/eval str) mode)) diff --git a/tests/profiles.scm b/tests/profiles.scm index 06a0387221..cac5b73347 100644 --- a/tests/profiles.scm +++ b/tests/profiles.scm @@ -265,6 +265,13 @@ (manifest-transaction-removal-candidate? guile-2.0.9 t) (null? install) (null? downgrade) (null? upgrade))))) +(test-assert "package->development-manifest" + (let ((manifest (package->development-manifest packages:hello))) + (every (lambda (name) + (manifest-installed? manifest + (manifest-pattern (name name)))) + '("gcc" "binutils" "glibc" "coreutils" "grep" "sed")))) + (test-assertm "profile-derivation" (mlet* %store-monad ((entry -> (package->manifest-entry %bootstrap-guile)) From patchwork Sat Oct 2 10:22:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Ludovic_Court=C3=A8s?= X-Patchwork-Id: 33543 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id B742127BBE1; Sat, 2 Oct 2021 11:24:55 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id C1C8F27BBE3 for ; Sat, 2 Oct 2021 11:24:54 +0100 (BST) Received: from localhost ([::1]:52304 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mWcCL-0002HF-Sc for patchwork@mira.cbaines.net; Sat, 02 Oct 2021 06:24:53 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38416) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mWcBW-0002Fj-8E for guix-patches@gnu.org; Sat, 02 Oct 2021 06:24:02 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:46427) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mWcBW-0000A9-0p for guix-patches@gnu.org; Sat, 02 Oct 2021 06:24:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1mWcBV-0007Ye-U0 for guix-patches@gnu.org; Sat, 02 Oct 2021 06:24:01 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#50960] [PATCH 03/10] DRAFT Add 'guix shell'. Resent-From: Ludovic =?utf-8?q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sat, 02 Oct 2021 10:24:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 50960 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 50960@debbugs.gnu.org Cc: Ludovic =?utf-8?q?Court=C3=A8s?= Received: via spool by 50960-submit@debbugs.gnu.org id=B50960.163317019128934 (code B ref 50960); Sat, 02 Oct 2021 10:24:01 +0000 Received: (at 50960) by debbugs.gnu.org; 2 Oct 2021 10:23:11 +0000 Received: from localhost ([127.0.0.1]:57954 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mWcAW-0007V8-Pu for submit@debbugs.gnu.org; Sat, 02 Oct 2021 06:23:11 -0400 Received: from eggs.gnu.org ([209.51.188.92]:33120) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mWcAT-0007UG-TX for 50960@debbugs.gnu.org; Sat, 02 Oct 2021 06:22:58 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:49192) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mWcAO-0007ci-IF; Sat, 02 Oct 2021 06:22:52 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=36460 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mWcAO-0007gb-8c; Sat, 02 Oct 2021 06:22:52 -0400 From: Ludovic =?utf-8?q?Court=C3=A8s?= Date: Sat, 2 Oct 2021 12:22:33 +0200 Message-Id: <20211002102240.27815-3-ludo@gnu.org> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20211002102240.27815-1-ludo@gnu.org> References: <20211002102240.27815-1-ludo@gnu.org> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches From: Ludovic Courtès DRAFT: Add doc. Print deprecation warning for 'guix environment'? * guix/scripts/shell.scm, tests/guix-shell.sh: New files. * Makefile.am (MODULES): Add 'shell.scm'. (SH_TESTS): Add 'tests/guix-shell.sh'. * guix/scripts/environment.scm (show-environment-options-help): New procedure. (show-help): Use it. (guix-environment*): New procedure. (guix-environment): Use it. * po/guix/POTFILES.in: Add it. --- Makefile.am | 2 + guix/scripts/environment.scm | 52 +++++++++----- guix/scripts/shell.scm | 136 +++++++++++++++++++++++++++++++++++ po/guix/POTFILES.in | 1 + tests/guix-shell.sh | 54 ++++++++++++++ 5 files changed, 228 insertions(+), 17 deletions(-) create mode 100644 guix/scripts/shell.scm create mode 100644 tests/guix-shell.sh diff --git a/Makefile.am b/Makefile.am index b66789fa0b..c28c8799ec 100644 --- a/Makefile.am +++ b/Makefile.am @@ -315,6 +315,7 @@ MODULES = \ guix/scripts/import/stackage.scm \ guix/scripts/import/texlive.scm \ guix/scripts/environment.scm \ + guix/scripts/shell.scm \ guix/scripts/publish.scm \ guix/scripts/edit.scm \ guix/scripts/size.scm \ @@ -550,6 +551,7 @@ SH_TESTS = \ tests/guix-authenticate.sh \ tests/guix-environment.sh \ tests/guix-environment-container.sh \ + tests/guix-shell.sh \ tests/guix-graph.sh \ tests/guix-describe.sh \ tests/guix-repl.sh \ diff --git a/guix/scripts/environment.scm b/guix/scripts/environment.scm index 54f48a7482..77956fc018 100644 --- a/guix/scripts/environment.scm +++ b/guix/scripts/environment.scm @@ -50,7 +50,11 @@ #:use-module (srfi srfi-37) #:use-module (srfi srfi-98) #:export (assert-container-features - guix-environment)) + guix-environment + guix-environment* + show-environment-options-help + (%options . %environment-options) + (%default-options . %environment-default-options))) (define %default-shell (or (getenv "SHELL") "/bin/sh")) @@ -66,23 +70,16 @@ do not augment existing environment variables with additional search paths." (newline))) (profile-search-paths profile manifest))) -(define (show-help) - (display (G_ "Usage: guix environment [OPTION]... PACKAGE... [-- COMMAND...] -Build an environment that includes the dependencies of PACKAGE and execute -COMMAND or an interactive shell in that environment.\n")) +(define (show-environment-options-help) + "Print help about options shared between 'guix environment' and 'guix +shell'." (display (G_ " -e, --expression=EXPR create environment for the package that EXPR evaluates to")) (display (G_ " - -l, --load=FILE create environment for the package that the code within - FILE evaluates to")) - (display (G_ " -m, --manifest=FILE create environment with the manifest from FILE")) (display (G_ " -p, --profile=PATH create environment from profile at PATH")) - (display (G_ " - --ad-hoc include all specified packages in the environment instead - of only their inputs")) (display (G_ " --pure unset existing environment variables")) (display (G_ " @@ -118,7 +115,24 @@ COMMAND or an interactive shell in that environment.\n")) (display (G_ " -v, --verbosity=LEVEL use the given verbosity LEVEL")) (display (G_ " - --bootstrap use bootstrap binaries to build the environment")) + --bootstrap use bootstrap binaries to build the environment"))) + +(define (show-help) + (display (G_ "Usage: guix environment [OPTION]... PACKAGE... [-- COMMAND...] +Build an environment that includes the dependencies of PACKAGE and execute +COMMAND or an interactive shell in that environment.\n")) + (warning (G_ "This command is deprecated in favor of 'guix shell'.\n")) + (newline) + + ;; These two options are left out in 'guix shell'. + (display (G_ " + -l, --load=FILE create environment for the package that the code within + FILE evaluates to")) + (display (G_ " + --ad-hoc include all specified packages in the environment instead + of only their inputs")) + + (show-environment-options-help) (newline) (show-build-options-help) (newline) @@ -649,11 +663,15 @@ message if any test fails." (define-command (guix-environment . args) (category development) - (synopsis "spawn one-off software environments") + (synopsis "spawn one-off software environments (deprecated)") + (guix-environment* (parse-args args))) + +(define (guix-environment* opts) + "Run the 'guix environment' command on OPTS, an alist resulting for +command-line option processing with 'parse-command-line'." (with-error-handling - (let* ((opts (parse-args args)) - (pure? (assoc-ref opts 'pure)) + (let* ((pure? (assoc-ref opts 'pure)) (container? (assoc-ref opts 'container?)) (link-prof? (assoc-ref opts 'link-profile?)) (network? (assoc-ref opts 'network?)) @@ -724,8 +742,8 @@ message if any test fails." (prof-drv (manifest->derivation manifest system bootstrap?)) (profile -> (if profile - (readlink* profile) - (derivation->output-path prof-drv))) + (readlink* profile) + (derivation->output-path prof-drv))) (gc-root -> (assoc-ref opts 'gc-root))) ;; First build the inputs. This is necessary even for diff --git a/guix/scripts/shell.scm b/guix/scripts/shell.scm new file mode 100644 index 0000000000..6a4b7a5092 --- /dev/null +++ b/guix/scripts/shell.scm @@ -0,0 +1,136 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright © 2021 Ludovic Courtès +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see . + +(define-module (guix scripts shell) + #:use-module (guix ui) + #:use-module (guix scripts environment) + #:autoload (guix scripts build) (show-build-options-help) + #:autoload (guix transformations) (show-transformation-options-help) + #:use-module (guix scripts) + #:use-module (srfi srfi-1) + #:use-module (srfi srfi-26) + #:use-module (srfi srfi-37) + #:use-module (srfi srfi-71) + #:use-module (ice-9 match) + #:export (guix-shell)) + +(define (show-help) + (display (G_ "Usage: guix shell [OPTION] PACKAGES... [-- COMMAND...] +Build an environment that includes PACKAGES and execute COMMAND or an +interactive shell in that environment.\n")) + (newline) + + ;; These two options differ from 'guix environment'. + (display (G_ " + -D, --development include the development inputs of the next package")) + (display (G_ " + -f, --install-from-file=FILE + install the package that the code within FILE + evaluates to")) + + (show-environment-options-help) + (newline) + (show-build-options-help) + (newline) + (show-transformation-options-help) + (newline) + (display (G_ " + -h, --help display this help and exit")) + (display (G_ " + -V, --version display version information and exit")) + (newline) + (show-bug-report-information)) + +(define (tag-package-arg opts arg) + "Return a two-element list with the form (TAG ARG) that tags ARG with either +'ad-hoc' in OPTS has the 'ad-hoc?' key set to #t, or 'inputs' otherwise." + (if (assoc-ref opts 'ad-hoc?) + `(ad-hoc-package ,arg) + `(package ,arg))) + +(define (ensure-ad-hoc alist) + (if (assq-ref alist 'ad-hoc?) + alist + `((ad-hoc? . #t) ,@alist))) + +(define (wrapped-option opt) + "Wrap OPT, a SRFI-37 option, such that its processor always adds the +'ad-hoc?' flag to the resulting alist." + (option (option-names opt) + (option-required-arg? opt) + (option-optional-arg? opt) + (compose ensure-ad-hoc (option-processor opt)))) + +(define %options + ;; Specification of the command-line options. + (let ((to-remove '("ad-hoc" "inherit" "load" "help" "version"))) + (append + (list (option '(#\h "help") #f #f + (lambda args + (show-help) + (exit 0))) + (option '(#\V "version") #f #f + (lambda args + (show-version-and-exit "guix shell"))) + + (option '(#\D "development") #f #f + (lambda (opt name arg result) + ;; Temporarily remove the 'ad-hoc?' flag from result. + ;; The next option will put it back thanks to + ;; 'wrapped-option'. + (alist-delete 'ad-hoc? result))) + + ;; For consistency with 'guix package', support '-f' rather than + ;; '-l' like 'guix environment' does. + (option '(#\f "install-from-file") #t #f + (lambda (opt name arg result) + (alist-cons 'load (tag-package-arg result arg) + result)))) + (filter-map (lambda (opt) + (and (not (any (lambda (name) + (member name to-remove)) + (option-names opt))) + (wrapped-option opt))) + %environment-options)))) + +(define %default-options + `((ad-hoc? . #t) ;always true + ,@%environment-default-options)) + +(define (parse-args args) + "Parse the list of command line arguments ARGS." + (define (handle-argument arg result) + (alist-cons 'package (tag-package-arg result arg) + (ensure-ad-hoc result))) + + ;; The '--' token is used to separate the command to run from the rest of + ;; the operands. + (let ((args command (break (cut string=? "--" <>) args))) + (let ((opts (parse-command-line args %options (list %default-options) + #:argument-handler handle-argument))) + (match command + (() opts) + (("--") opts) + (("--" command ...) (alist-cons 'exec command opts)))))) + + +(define-command (guix-shell . args) + (category development) + (synopsis "spawn one-off software environments") + + (guix-environment* (parse-args args))) diff --git a/po/guix/POTFILES.in b/po/guix/POTFILES.in index f5b76bf582..f8abeb2d38 100644 --- a/po/guix/POTFILES.in +++ b/po/guix/POTFILES.in @@ -99,6 +99,7 @@ guix/derivations.scm guix/scripts/archive.scm guix/scripts/build.scm guix/scripts/environment.scm +guix/scripts/shell.scm guix/scripts/time-machine.scm guix/scripts/import/cpan.scm guix/scripts/import/crate.scm diff --git a/tests/guix-shell.sh b/tests/guix-shell.sh new file mode 100644 index 0000000000..f08637f7ff --- /dev/null +++ b/tests/guix-shell.sh @@ -0,0 +1,54 @@ +# GNU Guix --- Functional package management for GNU +# Copyright © 2021 Ludovic Courtès +# +# This file is part of GNU Guix. +# +# GNU Guix is free software; you can redistribute it and/or modify it +# under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 3 of the License, or (at +# your option) any later version. +# +# GNU Guix is distributed in the hope that it will be useful, but +# WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with GNU Guix. If not, see . + +# +# Test the 'guix shell' alias. +# + +guix shell --version + +tmpdir="t-guix-shell-$$" +trap 'rm -r "$tmpdir"' EXIT +mkdir "$tmpdir" + +guix shell --bootstrap --pure guile-bootstrap -- guile --version + +# '--ad-hoc' is a thing of the past. +! guix shell --ad-hoc guile-bootstrap + +if guile -c '(getaddrinfo "www.gnu.org" "80" AI_NUMERICSERV)' 2> /dev/null +then + # Compute the build environment for the initial GNU Make. + guix shell --bootstrap --no-substitutes --search-paths --pure \ + -D -e '(@ (guix tests) gnu-make-for-tests)' > "$tmpdir/a" + + # Make sure bootstrap binaries are in the profile. + profile=`grep "^export PATH" "$tmpdir/a" | sed -r 's|^.*="(.*)/bin"|\1|'` + + # Make sure the bootstrap binaries are all listed where they belong. + grep -E "^export PATH=\"$profile/bin\"" "$tmpdir/a" + grep -E "^export CPATH=\"$profile/include\"" "$tmpdir/a" + grep -E "^export LIBRARY_PATH=\"$profile/lib\"" "$tmpdir/a" + for dep in bootstrap-binaries-0 gcc-bootstrap-0 glibc-bootstrap-0 + do + guix gc --references "$profile" | grep "$dep" + done + + # 'make-boot0' itself must not be listed. + ! guix gc --references "$profile" | grep make-boot0 +fi From patchwork Sat Oct 2 10:22:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Ludovic_Court=C3=A8s?= X-Patchwork-Id: 33542 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id E576927BBE1; Sat, 2 Oct 2021 11:24:54 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 8942727BBE1 for ; Sat, 2 Oct 2021 11:24:54 +0100 (BST) Received: from localhost ([::1]:52306 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mWcCJ-0002HO-Qx for patchwork@mira.cbaines.net; Sat, 02 Oct 2021 06:24:53 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38418) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mWcBW-0002Fv-KQ for guix-patches@gnu.org; Sat, 02 Oct 2021 06:24:02 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:46428) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mWcBW-0000B3-CV for guix-patches@gnu.org; Sat, 02 Oct 2021 06:24:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1mWcBW-0007Ym-9w for guix-patches@gnu.org; Sat, 02 Oct 2021 06:24:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#50960] [PATCH 04/10] DRAFT shell: By default load the local 'guix.scm' or 'manifest.scm' file. Resent-From: Ludovic =?utf-8?q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sat, 02 Oct 2021 10:24:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 50960 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 50960@debbugs.gnu.org Cc: Ludovic =?utf-8?q?Court=C3=A8s?= Received: via spool by 50960-submit@debbugs.gnu.org id=B50960.163317019128942 (code B ref 50960); Sat, 02 Oct 2021 10:24:02 +0000 Received: (at 50960) by debbugs.gnu.org; 2 Oct 2021 10:23:11 +0000 Received: from localhost ([127.0.0.1]:57961 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mWcAh-0007Wd-AF for submit@debbugs.gnu.org; Sat, 02 Oct 2021 06:23:11 -0400 Received: from eggs.gnu.org ([209.51.188.92]:33126) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mWcAU-0007UI-EU for 50960@debbugs.gnu.org; Sat, 02 Oct 2021 06:22:59 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:49194) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mWcAP-0007ct-8y; Sat, 02 Oct 2021 06:22:53 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=36460 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mWcAO-0007gb-Tv; Sat, 02 Oct 2021 06:22:53 -0400 From: Ludovic =?utf-8?q?Court=C3=A8s?= Date: Sat, 2 Oct 2021 12:22:34 +0200 Message-Id: <20211002102240.27815-4-ludo@gnu.org> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20211002102240.27815-1-ludo@gnu.org> References: <20211002102240.27815-1-ludo@gnu.org> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches DRAFT: Add doc. * guix/scripts/shell.scm (parse-args): Add call to 'auto-detect-manifest'. (find-file-in-parent-directories, auto-detect-manifest): New procedures. * tests/guix-shell.sh: Add test. --- guix/scripts/shell.scm | 44 ++++++++++++++++++++++++++++++++++++++++-- tests/guix-shell.sh | 16 +++++++++++++++ 2 files changed, 58 insertions(+), 2 deletions(-) diff --git a/guix/scripts/shell.scm b/guix/scripts/shell.scm index 6a4b7a5092..2f15befbd3 100644 --- a/guix/scripts/shell.scm +++ b/guix/scripts/shell.scm @@ -22,6 +22,8 @@ #:autoload (guix scripts build) (show-build-options-help) #:autoload (guix transformations) (show-transformation-options-help) #:use-module (guix scripts) + #:use-module (guix packages) + #:use-module (guix profiles) #:use-module (srfi srfi-1) #:use-module (srfi srfi-26) #:use-module (srfi srfi-37) @@ -121,13 +123,51 @@ interactive shell in that environment.\n")) ;; The '--' token is used to separate the command to run from the rest of ;; the operands. (let ((args command (break (cut string=? "--" <>) args))) - (let ((opts (parse-command-line args %options (list %default-options) - #:argument-handler handle-argument))) + (let ((opts (auto-detect-manifest + (parse-command-line args %options (list %default-options) + #:argument-handler handle-argument)))) (match command (() opts) (("--") opts) (("--" command ...) (alist-cons 'exec command opts)))))) +(define (find-file-in-parent-directories candidates) + "Find one of CANDIDATES in the current directory or one of its ancestors." + (let loop ((directory (getcwd))) + (and (= (stat:uid (stat directory)) (getuid)) + (or (any (lambda (candidate) + (let ((candidate (string-append directory "/" candidate))) + (and (file-exists? candidate) candidate))) + candidates) + (loop (string-append directory "/..")))))) ;Unix ".." resolution + +(define (auto-detect-manifest opts) + "If OPTS do not specify packages or a manifest, load a \"guix.scm\" or +\"manifest.scm\" file from the current directory or one of its ancestors. +Return the modified OPTS." + (define (options-contain-payload? opts) + (match opts + (() #f) + ((('package . _) . _) #t) + ((('load . _) . _) #t) + ((('manifest . _) . _) #t) + ((('expression . _) . _) #t) + ((_ . rest) (options-contain-payload? rest)))) + + (if (options-contain-payload? opts) + opts + (match (find-file-in-parent-directories '("guix.scm" "manifest.scm")) + (#f + (warning (G_ "no packages specified; creating an empty environment~%")) + opts) + (file + (info (G_ "loading environment from '~a'...~%") file) + (match (basename file) + ("guix.scm" + (alist-cons 'load `(package ,file) opts)) + ("manifest.scm" + (alist-cons 'manifest file opts))))))) + (define-command (guix-shell . args) (category development) diff --git a/tests/guix-shell.sh b/tests/guix-shell.sh index f08637f7ff..498c1c5515 100644 --- a/tests/guix-shell.sh +++ b/tests/guix-shell.sh @@ -31,6 +31,16 @@ guix shell --bootstrap --pure guile-bootstrap -- guile --version # '--ad-hoc' is a thing of the past. ! guix shell --ad-hoc guile-bootstrap +# Honoring the local 'manifest.scm' file. +cat > "$tmpdir/manifest.scm" <manifest '("guile-bootstrap")) +EOF +profile1="$(cd "$tmpdir"; guix shell --bootstrap -- "$SHELL" -c 'echo $GUIX_ENVIRONMENT')" +profile2="$(guix shell --bootstrap guile-bootstrap -- "$SHELL" -c 'echo $GUIX_ENVIRONMENT')" +test -n "$profile1" +test "$profile1" = "$profile2" +rm "$tmpdir/manifest.scm" + if guile -c '(getaddrinfo "www.gnu.org" "80" AI_NUMERICSERV)' 2> /dev/null then # Compute the build environment for the initial GNU Make. @@ -51,4 +61,10 @@ then # 'make-boot0' itself must not be listed. ! guix gc --references "$profile" | grep make-boot0 + + # Honoring the local 'guix.scm' file. + echo '(@ (guix tests) gnu-make-for-tests)' > "$tmpdir/guix.scm" + (cd "$tmpdir"; guix shell --bootstrap --search-paths --pure > "b") + cmp "$tmpdir/a" "$tmpdir/b" + rm "$tmpdir/guix.scm" fi From patchwork Sat Oct 2 10:22:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Ludovic_Court=C3=A8s?= X-Patchwork-Id: 33545 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 26A4627BBE3; Sat, 2 Oct 2021 11:24:59 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 9E79D27BBE1 for ; Sat, 2 Oct 2021 11:24:58 +0100 (BST) Received: from localhost ([::1]:52384 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mWcCP-0002Ki-Nc for patchwork@mira.cbaines.net; Sat, 02 Oct 2021 06:24:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38434) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mWcBZ-0002I6-HO for guix-patches@gnu.org; Sat, 02 Oct 2021 06:24:10 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:46434) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mWcBY-0000DP-P1 for guix-patches@gnu.org; Sat, 02 Oct 2021 06:24:05 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1mWcBY-0007ZU-Ll for guix-patches@gnu.org; Sat, 02 Oct 2021 06:24:04 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#50960] [PATCH 05/10] environment: Add tests for '--profile'. Resent-From: Ludovic =?utf-8?q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sat, 02 Oct 2021 10:24:04 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 50960 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 50960@debbugs.gnu.org Cc: Ludovic =?utf-8?q?Court=C3=A8s?= Received: via spool by 50960-submit@debbugs.gnu.org id=B50960.163317019328992 (code B ref 50960); Sat, 02 Oct 2021 10:24:04 +0000 Received: (at 50960) by debbugs.gnu.org; 2 Oct 2021 10:23:13 +0000 Received: from localhost ([127.0.0.1]:57973 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mWcAj-0007XT-FK for submit@debbugs.gnu.org; Sat, 02 Oct 2021 06:23:13 -0400 Received: from eggs.gnu.org ([209.51.188.92]:33130) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mWcAV-0007UK-JG for 50960@debbugs.gnu.org; Sat, 02 Oct 2021 06:23:12 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:49196) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mWcAQ-0007dr-2w; Sat, 02 Oct 2021 06:22:54 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=36460 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mWcAP-0007gb-KL; Sat, 02 Oct 2021 06:22:54 -0400 From: Ludovic =?utf-8?q?Court=C3=A8s?= Date: Sat, 2 Oct 2021 12:22:35 +0200 Message-Id: <20211002102240.27815-5-ludo@gnu.org> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20211002102240.27815-1-ludo@gnu.org> References: <20211002102240.27815-1-ludo@gnu.org> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches This is a followup to a643deac2de81755a1843a3b41dd53857678bebc. * tests/guix-environment-container.sh, tests/guix-environment.sh: Add tests for '--profile'. --- tests/guix-environment-container.sh | 8 ++++++++ tests/guix-environment.sh | 7 +++++++ 2 files changed, 15 insertions(+) diff --git a/tests/guix-environment-container.sh b/tests/guix-environment-container.sh index f2d15c8d0c..2e238c501d 100644 --- a/tests/guix-environment-container.sh +++ b/tests/guix-environment-container.sh @@ -44,6 +44,14 @@ else test $? = 42 fi +# Try '--root' and '--profile'. +root="$tmpdir/root" +guix environment -C --ad-hoc --bootstrap guile-bootstrap -r "$root" -- guile --version +guix environment -C -p "$root" --bootstrap -- guile --version +path1=$(guix environment -C -p "$root" --bootstrap -- guile -c '(display (getenv "PATH"))') +path2=$(guix environment -C --ad-hoc --bootstrap guile-bootstrap -- guile -c '(display (getenv "PATH"))') +test "$path1" = "$path2" + # Make sure "localhost" resolves. guix environment --container --ad-hoc --bootstrap guile-bootstrap \ -- guile -c '(exit (pair? (getaddrinfo "localhost" "80")))' diff --git a/tests/guix-environment.sh b/tests/guix-environment.sh index afadcbe195..f4fc2e39ed 100644 --- a/tests/guix-environment.sh +++ b/tests/guix-environment.sh @@ -119,6 +119,13 @@ test `readlink "$gcroot"` = "$expected" guix environment --bootstrap -r "$gcroot" --ad-hoc guile-bootstrap \ -- guile -c 1 test `readlink "$gcroot"` = "$expected" + +# Make sure '-p' works as expected. +test $(guix environment -p "$gcroot" -- "$SHELL" -c 'echo $GUIX_ENVIRONMENT') = "$expected" +paths1="$(guix environment -p "$gcroot" --search-paths)" +paths2="$(guix environment --bootstrap --ad-hoc guile-bootstrap --search-paths)" +test "$paths1" = "$paths2" + rm "$gcroot" # Try '-r' with a relative file name. From patchwork Sat Oct 2 10:22:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Ludovic_Court=C3=A8s?= X-Patchwork-Id: 33546 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id D2E5D27BBE3; Sat, 2 Oct 2021 11:25:40 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_MSPIKE_H2,SPF_HELO_PASS autolearn=unavailable autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 8ECD527BBE1 for ; Sat, 2 Oct 2021 11:25:40 +0100 (BST) Received: from localhost ([::1]:53128 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mWcD4-0002s8-Jz for patchwork@mira.cbaines.net; Sat, 02 Oct 2021 06:25:38 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38422) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mWcBX-0002G5-0K for guix-patches@gnu.org; Sat, 02 Oct 2021 06:24:03 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:46429) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mWcBW-0000BW-PM for guix-patches@gnu.org; Sat, 02 Oct 2021 06:24:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1mWcBW-0007Yt-MO for guix-patches@gnu.org; Sat, 02 Oct 2021 06:24:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#50960] [PATCH 06/10] environment: Skip derivation computation when '--profile' is used. Resent-From: Ludovic =?utf-8?q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sat, 02 Oct 2021 10:24:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 50960 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 50960@debbugs.gnu.org Cc: Ludovic =?utf-8?q?Court=C3=A8s?= Received: via spool by 50960-submit@debbugs.gnu.org id=B50960.163317019228950 (code B ref 50960); Sat, 02 Oct 2021 10:24:02 +0000 Received: (at 50960) by debbugs.gnu.org; 2 Oct 2021 10:23:12 +0000 Received: from localhost ([127.0.0.1]:57963 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mWcAh-0007Wl-Os for submit@debbugs.gnu.org; Sat, 02 Oct 2021 06:23:11 -0400 Received: from eggs.gnu.org ([209.51.188.92]:33134) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mWcAW-0007UL-2L for 50960@debbugs.gnu.org; Sat, 02 Oct 2021 06:23:00 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:49198) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mWcAQ-0007eo-Ss; Sat, 02 Oct 2021 06:22:54 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=36460 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mWcAQ-0007gb-E9; Sat, 02 Oct 2021 06:22:54 -0400 From: Ludovic =?utf-8?q?Court=C3=A8s?= Date: Sat, 2 Oct 2021 12:22:36 +0200 Message-Id: <20211002102240.27815-6-ludo@gnu.org> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20211002102240.27815-1-ludo@gnu.org> References: <20211002102240.27815-1-ludo@gnu.org> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches * guix/scripts/environment.scm (guix-environment*): Bypass calls to 'package-derivation' and to 'manifest->derivation' when PROFILE is true. --- guix/scripts/environment.scm | 23 +++++++++++++---------- 1 file changed, 13 insertions(+), 10 deletions(-) diff --git a/guix/scripts/environment.scm b/guix/scripts/environment.scm index 77956fc018..32f376fdd2 100644 --- a/guix/scripts/environment.scm +++ b/guix/scripts/environment.scm @@ -729,18 +729,21 @@ command-line option processing with 'parse-command-line'." ;; Use the bootstrap Guile when requested. (parameterize ((%graft? (assoc-ref opts 'graft?)) (%guile-for-build - (package-derivation - store - (if bootstrap? - %bootstrap-guile - (default-guile))))) + (and (or container? (not profile)) + (package-derivation + store + (if bootstrap? + %bootstrap-guile + (default-guile)))))) (run-with-store store ;; Containers need a Bourne shell at /bin/sh. (mlet* %store-monad ((bash (environment-bash container? bootstrap? system)) - (prof-drv (manifest->derivation - manifest system bootstrap?)) + (prof-drv (if profile + (return #f) + (manifest->derivation + manifest system bootstrap?))) (profile -> (if profile (readlink* profile) (derivation->output-path prof-drv))) @@ -750,9 +753,9 @@ command-line option processing with 'parse-command-line'." ;; --search-paths. Additionally, we might need to build bash for ;; a container. (mbegin %store-monad - (built-derivations (if (derivation? bash) - (list prof-drv bash) - (list prof-drv))) + (built-derivations (append + (if prof-drv (list prof-drv) '()) + (if (derivation? bash) (list bash) '()))) (mwhen gc-root (register-gc-root profile gc-root)) From patchwork Sat Oct 2 10:22:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Ludovic_Court=C3=A8s?= X-Patchwork-Id: 33547 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 69E6627BBE3; Sat, 2 Oct 2021 11:25:50 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_MSPIKE_H2,SPF_HELO_PASS autolearn=unavailable autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 0789E27BBE1 for ; Sat, 2 Oct 2021 11:25:50 +0100 (BST) Received: from localhost ([::1]:53136 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mWcDC-0002sN-1h for patchwork@mira.cbaines.net; Sat, 02 Oct 2021 06:25:46 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38426) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mWcBX-0002GK-Cx for guix-patches@gnu.org; Sat, 02 Oct 2021 06:24:03 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:46430) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mWcBX-0000CK-5e for guix-patches@gnu.org; Sat, 02 Oct 2021 06:24:03 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1mWcBX-0007Z0-2r for guix-patches@gnu.org; Sat, 02 Oct 2021 06:24:03 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#50960] [PATCH 07/10] environment: Do not connect to the daemon when '--profile' is used. Resent-From: Ludovic =?utf-8?q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sat, 02 Oct 2021 10:24:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 50960 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 50960@debbugs.gnu.org Cc: Ludovic =?utf-8?q?Court=C3=A8s?= Received: via spool by 50960-submit@debbugs.gnu.org id=B50960.163317019228959 (code B ref 50960); Sat, 02 Oct 2021 10:24:03 +0000 Received: (at 50960) by debbugs.gnu.org; 2 Oct 2021 10:23:12 +0000 Received: from localhost ([127.0.0.1]:57965 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mWcAi-0007Wt-0p for submit@debbugs.gnu.org; Sat, 02 Oct 2021 06:23:12 -0400 Received: from eggs.gnu.org ([209.51.188.92]:33138) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mWcAW-0007UO-Mu for 50960@debbugs.gnu.org; Sat, 02 Oct 2021 06:23:01 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:49200) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mWcAR-0007fV-Hn; Sat, 02 Oct 2021 06:22:55 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=36460 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mWcAR-0007gb-89; Sat, 02 Oct 2021 06:22:55 -0400 From: Ludovic =?utf-8?q?Court=C3=A8s?= Date: Sat, 2 Oct 2021 12:22:37 +0200 Message-Id: <20211002102240.27815-7-ludo@gnu.org> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20211002102240.27815-1-ludo@gnu.org> References: <20211002102240.27815-1-ludo@gnu.org> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches This further speeds up the 'guix environment -p PROFILE' case. * guix/scripts/environment.scm (guix-environment*)[store-needed?]: New variable. [with-store/maybe]: New macro. Use it instead of 'with-store', and remove 'with-build-handler' form. --- guix/scripts/environment.scm | 169 +++++++++++++++++++---------------- 1 file changed, 93 insertions(+), 76 deletions(-) diff --git a/guix/scripts/environment.scm b/guix/scripts/environment.scm index 32f376fdd2..e23d52df39 100644 --- a/guix/scripts/environment.scm +++ b/guix/scripts/environment.scm @@ -691,6 +691,26 @@ command-line option processing with 'parse-command-line'." (mappings (pick-all opts 'file-system-mapping)) (white-list (pick-all opts 'inherit-regexp))) + (define store-needed? + ;; Whether connecting to the daemon is needed. + (or container? (not profile))) + + (define-syntax-rule (with-store/maybe store exp ...) + ;; Evaluate EXP... with STORE bound to a connection, unless + ;; STORE-NEEDED? is false, in which case STORE is bound to #f. + (let ((proc (lambda (store) exp ...))) + (if store-needed? + (with-store s + (set-build-options-from-command-line s opts) + (with-build-handler (build-notifier #:use-substitutes? + (assoc-ref opts 'substitutes?) + #:verbosity + (assoc-ref opts 'verbosity) + #:dry-run? + (assoc-ref opts 'dry-run?)) + (proc s))) + (proc #f)))) + (when container? (assert-container-features)) (when (and (not container?) link-prof?) @@ -701,88 +721,85 @@ command-line option processing with 'parse-command-line'." (leave (G_ "--no-cwd cannot be used without --container~%"))) - (with-store store - (with-build-handler (build-notifier #:use-substitutes? - (assoc-ref opts 'substitutes?) - #:verbosity - (assoc-ref opts 'verbosity) - #:dry-run? - (assoc-ref opts 'dry-run?)) - (with-status-verbosity (assoc-ref opts 'verbosity) - (define manifest-from-opts - (options/resolve-packages store opts)) + (with-store/maybe store + (with-status-verbosity (assoc-ref opts 'verbosity) + (define manifest-from-opts + (options/resolve-packages store opts)) - (define manifest - (if profile - (profile-manifest profile) - manifest-from-opts)) + (define manifest + (if profile + (profile-manifest profile) + manifest-from-opts)) - (when (and profile - (> (length (manifest-entries manifest-from-opts)) 0)) - (leave (G_ "'--profile' cannot be used with package options~%"))) + (when (and profile + (> (length (manifest-entries manifest-from-opts)) 0)) + (leave (G_ "'--profile' cannot be used with package options~%"))) - (when (null? (manifest-entries manifest)) - (warning (G_ "no packages specified; creating an empty environment~%"))) + (when (null? (manifest-entries manifest)) + (warning (G_ "no packages specified; creating an empty environment~%"))) - (set-build-options-from-command-line store opts) + ;; Use the bootstrap Guile when requested. + (parameterize ((%graft? (assoc-ref opts 'graft?)) + (%guile-for-build + (and store-needed? + (package-derivation + store + (if bootstrap? + %bootstrap-guile + (default-guile)))))) + (run-with-store store + ;; Containers need a Bourne shell at /bin/sh. + (mlet* %store-monad ((bash (environment-bash container? + bootstrap? + system)) + (prof-drv (if profile + (return #f) + (manifest->derivation + manifest system bootstrap?))) + (profile -> (if profile + (readlink* profile) + (derivation->output-path prof-drv))) + (gc-root -> (assoc-ref opts 'gc-root))) - ;; Use the bootstrap Guile when requested. - (parameterize ((%graft? (assoc-ref opts 'graft?)) - (%guile-for-build - (and (or container? (not profile)) - (package-derivation - store - (if bootstrap? - %bootstrap-guile - (default-guile)))))) - (run-with-store store - ;; Containers need a Bourne shell at /bin/sh. - (mlet* %store-monad ((bash (environment-bash container? - bootstrap? - system)) - (prof-drv (if profile - (return #f) - (manifest->derivation - manifest system bootstrap?))) - (profile -> (if profile - (readlink* profile) - (derivation->output-path prof-drv))) - (gc-root -> (assoc-ref opts 'gc-root))) - - ;; First build the inputs. This is necessary even for - ;; --search-paths. Additionally, we might need to build bash for - ;; a container. - (mbegin %store-monad + ;; First build the inputs. This is necessary even for + ;; --search-paths. Additionally, we might need to build bash for + ;; a container. + (mbegin %store-monad + (mwhen store-needed? (built-derivations (append (if prof-drv (list prof-drv) '()) - (if (derivation? bash) (list bash) '()))) - (mwhen gc-root - (register-gc-root profile gc-root)) + (if (derivation? bash) (list bash) '())))) + (mwhen gc-root + (register-gc-root profile gc-root)) - (cond - ((assoc-ref opts 'search-paths) - (show-search-paths profile manifest #:pure? pure?) - (return #t)) - (container? - (let ((bash-binary - (if bootstrap? - (derivation->output-path bash) - (string-append (derivation->output-path bash) - "/bin/sh")))) - (launch-environment/container #:command command - #:bash bash-binary - #:user user - #:user-mappings mappings - #:profile profile - #:manifest manifest - #:white-list white-list - #:link-profile? link-prof? - #:network? network? - #:map-cwd? (not no-cwd?)))) + (cond + ((assoc-ref opts 'search-paths) + (show-search-paths profile manifest #:pure? pure?) + (return #t)) + (container? + (let ((bash-binary + (if bootstrap? + (derivation->output-path bash) + (string-append (derivation->output-path bash) + "/bin/sh")))) + (launch-environment/container #:command command + #:bash bash-binary + #:user user + #:user-mappings mappings + #:profile profile + #:manifest manifest + #:white-list white-list + #:link-profile? link-prof? + #:network? network? + #:map-cwd? (not no-cwd?)))) - (else - (return - (exit/status - (launch-environment/fork command profile manifest - #:white-list white-list - #:pure? pure?))))))))))))))) + (else + (return + (exit/status + (launch-environment/fork command profile manifest + #:white-list white-list + #:pure? pure?)))))))))))))) + +;;; Local Variables: +;;; (put 'with-store/maybe 'scheme-indent-function 1) +;;; End: From patchwork Sat Oct 2 10:22:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Ludovic_Court=C3=A8s?= X-Patchwork-Id: 33544 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 8F37927BBE3; Sat, 2 Oct 2021 11:24:58 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_MSPIKE_H2,SPF_HELO_PASS autolearn=unavailable autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 5DF8327BBE1 for ; Sat, 2 Oct 2021 11:24:58 +0100 (BST) Received: from localhost ([::1]:52362 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mWcCP-0002Jh-GO for patchwork@mira.cbaines.net; Sat, 02 Oct 2021 06:24:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38428) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mWcBX-0002GV-Pr for guix-patches@gnu.org; Sat, 02 Oct 2021 06:24:03 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:46431) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mWcBX-0000CS-ID for guix-patches@gnu.org; Sat, 02 Oct 2021 06:24:03 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1mWcBX-0007Z8-FL for guix-patches@gnu.org; Sat, 02 Oct 2021 06:24:03 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#50960] [PATCH 08/10] environment: Autoload some modules. Resent-From: Ludovic =?utf-8?q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sat, 02 Oct 2021 10:24:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 50960 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 50960@debbugs.gnu.org Cc: Ludovic =?utf-8?q?Court=C3=A8s?= Received: via spool by 50960-submit@debbugs.gnu.org id=B50960.163317019228971 (code B ref 50960); Sat, 02 Oct 2021 10:24:03 +0000 Received: (at 50960) by debbugs.gnu.org; 2 Oct 2021 10:23:12 +0000 Received: from localhost ([127.0.0.1]:57967 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mWcAi-0007X1-FP for submit@debbugs.gnu.org; Sat, 02 Oct 2021 06:23:12 -0400 Received: from eggs.gnu.org ([209.51.188.92]:33142) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mWcAX-0007UP-Bz for 50960@debbugs.gnu.org; Sat, 02 Oct 2021 06:23:02 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:49202) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mWcAS-0007fe-6g; Sat, 02 Oct 2021 06:22:56 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=36460 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mWcAR-0007gb-TF; Sat, 02 Oct 2021 06:22:56 -0400 From: Ludovic =?utf-8?q?Court=C3=A8s?= Date: Sat, 2 Oct 2021 12:22:38 +0200 Message-Id: <20211002102240.27815-8-ludo@gnu.org> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20211002102240.27815-1-ludo@gnu.org> References: <20211002102240.27815-1-ludo@gnu.org> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches This further speeds up the 'guix environment -p PROFILE' case. * guix/scripts/environment.scm: Autoload a bunch of modules. --- guix/scripts/environment.scm | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-) diff --git a/guix/scripts/environment.scm b/guix/scripts/environment.scm index e23d52df39..05a43659da 100644 --- a/guix/scripts/environment.scm +++ b/guix/scripts/environment.scm @@ -34,15 +34,18 @@ #:use-module (guix scripts) #:use-module (guix scripts build) #:use-module (guix transformations) - #:use-module (gnu build linux-container) - #:use-module (gnu build accounts) - #:use-module ((guix build syscalls) #:select (set-network-interface-up)) - #:use-module (gnu system linux-container) + #:autoload (gnu build linux-container) (call-with-container %namespaces + user-namespace-supported? + unprivileged-user-namespace-supported? + setgroups-supported?) + #:autoload (gnu build accounts) (password-entry group-entry + password-entry-name password-entry-directory + write-passwd write-group) + #:autoload (guix build syscalls) (set-network-interface-up) #:use-module (gnu system file-systems) - #:use-module (gnu packages) - #:use-module (gnu packages bash) - #:use-module ((gnu packages bootstrap) - #:select (bootstrap-executable %bootstrap-guile)) + #:autoload (gnu packages) (specification->package+output) + #:autoload (gnu packages bash) (bash) + #:autoload (gnu packages bootstrap) (bootstrap-executable %bootstrap-guile) #:use-module (ice-9 match) #:use-module (srfi srfi-1) #:use-module (srfi srfi-11) From patchwork Sat Oct 2 10:22:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Ludovic_Court=C3=A8s?= X-Patchwork-Id: 33548 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 48B9927BBE3; Sat, 2 Oct 2021 11:26:07 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_MSPIKE_H2,SPF_HELO_PASS autolearn=unavailable autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 1327327BBE1 for ; Sat, 2 Oct 2021 11:26:07 +0100 (BST) Received: from localhost ([::1]:53638 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mWcDW-0003Lm-7L for patchwork@mira.cbaines.net; Sat, 02 Oct 2021 06:26:06 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38430) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mWcBY-0002Gz-81 for guix-patches@gnu.org; Sat, 02 Oct 2021 06:24:04 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:46432) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mWcBY-0000Ca-0H for guix-patches@gnu.org; Sat, 02 Oct 2021 06:24:04 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1mWcBX-0007ZF-TN for guix-patches@gnu.org; Sat, 02 Oct 2021 06:24:03 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#50960] [PATCH 09/10] cache: Gracefully handle non-existent cache. Resent-From: Ludovic =?utf-8?q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sat, 02 Oct 2021 10:24:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 50960 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 50960@debbugs.gnu.org Cc: Ludovic =?utf-8?q?Court=C3=A8s?= Received: via spool by 50960-submit@debbugs.gnu.org id=B50960.163317019328979 (code B ref 50960); Sat, 02 Oct 2021 10:24:03 +0000 Received: (at 50960) by debbugs.gnu.org; 2 Oct 2021 10:23:13 +0000 Received: from localhost ([127.0.0.1]:57969 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mWcAi-0007XE-Rf for submit@debbugs.gnu.org; Sat, 02 Oct 2021 06:23:13 -0400 Received: from eggs.gnu.org ([209.51.188.92]:33146) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mWcAY-0007UW-0t for 50960@debbugs.gnu.org; Sat, 02 Oct 2021 06:23:02 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:49204) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mWcAS-0007gK-RS; Sat, 02 Oct 2021 06:22:56 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=36460 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mWcAS-0007gb-I5; Sat, 02 Oct 2021 06:22:56 -0400 From: Ludovic =?utf-8?q?Court=C3=A8s?= Date: Sat, 2 Oct 2021 12:22:39 +0200 Message-Id: <20211002102240.27815-9-ludo@gnu.org> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20211002102240.27815-1-ludo@gnu.org> References: <20211002102240.27815-1-ludo@gnu.org> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches * guix/cache.scm (maybe-remove-expired-cache-entries): Ignore ENOENT when writing EXPIRY-FILE. --- guix/cache.scm | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/guix/cache.scm b/guix/cache.scm index 0401a9d428..51009809bd 100644 --- a/guix/cache.scm +++ b/guix/cache.scm @@ -101,7 +101,13 @@ CLEANUP-PERIOD denotes the minimum time between two cache cleanups." #:now now #:entry-expiration entry-expiration #:delete-entry delete-entry) - (call-with-output-file expiry-file - (cute write (time-second now) <>)))) + (catch 'system-error + (lambda () + (call-with-output-file expiry-file + (cute write (time-second now) <>))) + (lambda args + ;; ENOENT means CACHE does not exist. + (unless (= ENOENT (system-error-errno args)) + (apply throw args)))))) ;;; cache.scm ends here From patchwork Sat Oct 2 10:22:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Ludovic_Court=C3=A8s?= X-Patchwork-Id: 33549 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 44D3227BBE3; Sat, 2 Oct 2021 11:26:08 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_MSPIKE_H2,SPF_HELO_PASS autolearn=unavailable autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id DCBD027BBE1 for ; Sat, 2 Oct 2021 11:26:07 +0100 (BST) Received: from localhost ([::1]:53648 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mWcDW-0003ML-Vx for patchwork@mira.cbaines.net; Sat, 02 Oct 2021 06:26:07 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38432) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mWcBY-0002HV-KD for guix-patches@gnu.org; Sat, 02 Oct 2021 06:24:04 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:46433) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mWcBY-0000Cy-CX for guix-patches@gnu.org; Sat, 02 Oct 2021 06:24:04 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1mWcBY-0007ZM-9J for guix-patches@gnu.org; Sat, 02 Oct 2021 06:24:04 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#50960] [PATCH 10/10] shell: Maintain a profile cache. Resent-From: Ludovic =?utf-8?q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sat, 02 Oct 2021 10:24:04 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 50960 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 50960@debbugs.gnu.org Cc: Ludovic =?utf-8?q?Court=C3=A8s?= Received: via spool by 50960-submit@debbugs.gnu.org id=B50960.163317019328986 (code B ref 50960); Sat, 02 Oct 2021 10:24:04 +0000 Received: (at 50960) by debbugs.gnu.org; 2 Oct 2021 10:23:13 +0000 Received: from localhost ([127.0.0.1]:57971 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mWcAj-0007XM-3g for submit@debbugs.gnu.org; Sat, 02 Oct 2021 06:23:13 -0400 Received: from eggs.gnu.org ([209.51.188.92]:33150) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mWcAY-0007UY-Li for 50960@debbugs.gnu.org; Sat, 02 Oct 2021 06:23:03 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:49206) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mWcAT-0007hX-G3; Sat, 02 Oct 2021 06:22:57 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=36460 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mWcAT-0007gb-6e; Sat, 02 Oct 2021 06:22:57 -0400 From: Ludovic =?utf-8?q?Court=C3=A8s?= Date: Sat, 2 Oct 2021 12:22:40 +0200 Message-Id: <20211002102240.27815-10-ludo@gnu.org> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20211002102240.27815-1-ludo@gnu.org> References: <20211002102240.27815-1-ludo@gnu.org> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches With this change, running "guix shell" (no arguments) is equivalent to: guix environment -r ~/.cache/guix/profiles/some-root -l guix.scm This is the cache miss. On cache hit, it's equivalent to: guix environment -p ~/.cache/guix/profiles/some-root ... which can run in 0.1s. * guix/scripts/shell.scm (auto-detect-manifest): Looked for a cached GC root to the profile and use it. (%profile-cache-directory): New variable. (profile-cache-key, profile-cached-gc-root): New procedures. (guix-shell)[cache-entries, entry-expiration]: New procedures. Add call to 'maybe-remove-expired-cache-entries'. --- guix/scripts/shell.scm | 90 +++++++++++++++++++++++++++++++++++++++--- 1 file changed, 84 insertions(+), 6 deletions(-) diff --git a/guix/scripts/shell.scm b/guix/scripts/shell.scm index 2f15befbd3..7c116cc770 100644 --- a/guix/scripts/shell.scm +++ b/guix/scripts/shell.scm @@ -29,6 +29,15 @@ #:use-module (srfi srfi-37) #:use-module (srfi srfi-71) #:use-module (ice-9 match) + #:autoload (guix base32) (bytevector->base32-string) + #:autoload (rnrs bytevectors) (string->utf8) + #:autoload (guix utils) (cache-directory) + #:autoload (guix describe) (current-channels) + #:autoload (guix channels) (channel-commit) + #:autoload (gcrypt hash) (sha256) + #:use-module ((guix build utils) #:select (mkdir-p)) + #:use-module (guix cache) + #:use-module ((ice-9 ftw) #:select (scandir)) #:export (guix-shell)) (define (show-help) @@ -161,16 +170,85 @@ Return the modified OPTS." (warning (G_ "no packages specified; creating an empty environment~%")) opts) (file + ;; Load environment from FILE; if possible, use/maintain a GC root to + ;; the corresponding profile in cache. (info (G_ "loading environment from '~a'...~%") file) - (match (basename file) - ("guix.scm" - (alist-cons 'load `(package ,file) opts)) - ("manifest.scm" - (alist-cons 'manifest file opts))))))) + (let* ((root (profile-cached-gc-root file)) + (stat (and root (false-if-exception (lstat root))))) + (if (and stat + (<= (stat:mtime ((@ (guile) stat) file)) + (stat:mtime stat))) + (let ((now (current-time))) + ;; Update the atime on ROOT to reflect usage. + (utime root + now (stat:mtime stat) + 0 (stat:mtimensec stat) + AT_SYMLINK_NOFOLLOW) + (alist-cons 'profile root opts)) ;load right away + (let ((opts (match (basename file) + ("guix.scm" + (alist-cons 'load `(package ,file) opts)) + ("manifest.scm" + (alist-cons 'manifest file opts))))) + (if (and root (not (assq-ref opts 'gc-root))) + (begin + (if stat + (delete-file root) + (mkdir-p (dirname root))) + (alist-cons 'gc-root root opts)) + opts)))))))) + + +;;; +;;; Profile cache. +;;; + +(define %profile-cache-directory + ;; Directory where profiles created by 'guix shell' alone (without extra + ;; options) are cached. + (make-parameter (string-append (cache-directory #:ensure? #f) + "/profiles"))) + +(define (profile-cache-key file) + "Return the cache key for the profile corresponding to FILE, a 'guix.scm' or +'manifest.scm' file, or #f if we lack channel information." + (match (current-channels) + (() #f) + (((= channel-commit commits) ...) + (let ((stat (stat file))) + (bytevector->base32-string + (sha256 (string->utf8 + (string-append (string-join commits) ":" + (basename file) ":" + (number->string (stat:dev stat)) ":" + (number->string (stat:ino stat)))))))))) + +(define (profile-cached-gc-root file) + "Return the cached GC root for FILE, a 'guix.scm' or 'manifest.scm' file, or +#f if we lack information to cache it." + (match (profile-cache-key file) + (#f #f) + (key (string-append (%profile-cache-directory) "/" key)))) (define-command (guix-shell . args) (category development) (synopsis "spawn one-off software environments") - (guix-environment* (parse-args args))) + (define (cache-entries directory) + (filter-map (match-lambda + ((or "." "..") #f) + (file (string-append directory "/" file))) + (or (scandir directory) '()))) + + (define* (entry-expiration file) + ;; Return the time at which FILE, a cached profile, is considered expired. + (match (false-if-exception (lstat file)) + (#f 0) ;FILE may have been deleted in the meantime + (st (+ (stat:atime st) (* 60 60 24 7))))) + + (let ((result (guix-environment* (parse-args args)))) + (maybe-remove-expired-cache-entries (%profile-cache-directory) + cache-entries + #:entry-expiration entry-expiration) + result))