From patchwork Mon Oct 11 21:37:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Ludovic_Court=C3=A8s?= X-Patchwork-Id: 33773 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id A4C7727BBE3; Mon, 11 Oct 2021 22:42:52 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_MSPIKE_H2,SPF_HELO_PASS autolearn=unavailable autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 484BA27BBE1 for ; Mon, 11 Oct 2021 22:42:52 +0100 (BST) Received: from localhost ([::1]:39796 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ma34N-0001Iz-AE for patchwork@mira.cbaines.net; Mon, 11 Oct 2021 17:42:51 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47512) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ma30g-0006aE-Ow for guix-patches@gnu.org; Mon, 11 Oct 2021 17:39:03 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:49607) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1ma30f-0004pd-Lu for guix-patches@gnu.org; Mon, 11 Oct 2021 17:39:01 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1ma30f-0001PE-Jq for guix-patches@gnu.org; Mon, 11 Oct 2021 17:39:01 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#50960] [PATCH v2 01/11] packages: Add 'package-development-inputs'. Resent-From: Ludovic =?utf-8?q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Mon, 11 Oct 2021 21:39:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 50960 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 50960@debbugs.gnu.org Cc: Ludovic =?utf-8?q?Court=C3=A8s?= Received: via spool by 50960-submit@debbugs.gnu.org id=B50960.16339883085231 (code B ref 50960); Mon, 11 Oct 2021 21:39:01 +0000 Received: (at 50960) by debbugs.gnu.org; 11 Oct 2021 21:38:28 +0000 Received: from localhost ([127.0.0.1]:32888 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ma308-0001MJ-5y for submit@debbugs.gnu.org; Mon, 11 Oct 2021 17:38:28 -0400 Received: from eggs.gnu.org ([209.51.188.92]:45444) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ma307-0001Lx-Dy for 50960@debbugs.gnu.org; Mon, 11 Oct 2021 17:38:27 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:40564) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ma301-0004VQ-P9; Mon, 11 Oct 2021 17:38:21 -0400 Received: from 91-160-117-201.subs.proxad.net ([91.160.117.201]:53321 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1ma301-0007Y0-Fk; Mon, 11 Oct 2021 17:38:21 -0400 From: Ludovic =?utf-8?q?Court=C3=A8s?= Date: Mon, 11 Oct 2021 23:37:59 +0200 Message-Id: <20211011213809.17482-2-ludo@gnu.org> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20211011213809.17482-1-ludo@gnu.org> References: <20211002102116.27726-1-ludo@gnu.org> <20211011213809.17482-1-ludo@gnu.org> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches * guix/packages.scm (package-development-inputs): New procedure. * guix/scripts/environment.scm (package-environment-inputs): Use it. * tests/packages.scm ("package-development-inputs") ("package-development-inputs, cross-compilation"): New tests. * doc/guix.texi (package Reference): Document it. --- doc/guix.texi | 41 ++++++++++++++++++++++++++++++++++++ guix/packages.scm | 10 +++++++++ guix/scripts/environment.scm | 2 +- tests/packages.scm | 14 ++++++++++++ 4 files changed, 66 insertions(+), 1 deletion(-) diff --git a/doc/guix.texi b/doc/guix.texi index a72a726b54..49399e792b 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -6847,6 +6847,47 @@ cross-compiling: It is an error to refer to @code{this-package} outside a package definition. @end deffn +@cindex development inputs, of a package +@cindex implicit inputs, of a package +Sometimes you will want to obtain the list of inputs needed to +@emph{develop} a package---all the inputs that are visible when the +package is compiled. This is what the @code{package-development-inputs} +procedure returns. + +@deffn {Scheme Procedure} package-development-inputs @var{package} @ + [@var{system}] [#:target #f] +Return the list of inputs required by @var{package} for development +purposes on @var{system}. When @var{target} is true, return the inputs +needed to cross-compile @var{package} from @var{system} to +@var{triplet}, where @var{triplet} is a triplet such as +@code{"aarch64-linux-gnu"}. + +Note that the result includes both explicit inputs and implicit +inputs---inputs automatically added by the build system (@pxref{Build +Systems}). Let us take the @code{hello} package to illustrate that: + +@lisp +(use-modules (gnu packages base) (guix packages)) + +hello +@result{} # + +(package-direct-inputs hello) +@result{} () + +(package-development-inputs hello) +@result{} (("source" @dots{}) ("tar" #) @dots{}) +@end lisp + +In this example, @code{package-direct-inputs} returns the empty list, +because @code{hello} has zero explicit dependencies. Conversely, +@code{package-development-inputs} includes inputs implicitly added by +@code{gnu-build-system} that are required to build @code{hello}: tar, +gzip, GCC, libc, Bash, and more. To visualize it, @command{guix graph +hello} would show you explicit inputs, whereas @command{guix graph -t +bag hello} would include implicit inputs (@pxref{Invoking guix graph}). +@end deffn + Because packages are regular Scheme objects that capture a complete dependency graph and associated build procedures, it is often useful to write procedures that take a package and return a modified version diff --git a/guix/packages.scm b/guix/packages.scm index 8c3a0b0b7b..43e0130793 100644 --- a/guix/packages.scm +++ b/guix/packages.scm @@ -153,6 +153,7 @@ (define-module (guix packages) bag-transitive-host-inputs bag-transitive-build-inputs bag-transitive-target-inputs + package-development-inputs package-closure default-guile @@ -1070,6 +1071,15 @@ (define (bag-transitive-target-inputs bag) (%current-system (bag-system bag))) (transitive-inputs (bag-target-inputs bag)))) +(define* (package-development-inputs package + #:optional (system (%current-system)) + #:key target) + "Return the list of inputs required by PACKAGE for development purposes on +SYSTEM. When TARGET is true, return the inputs needed to cross-compile +PACKAGE from SYSTEM to TRIPLET, where TRIPLET is a triplet such as +\"aarch64-linux-gnu\"." + (bag-transitive-inputs (package->bag package system target))) + (define* (package-closure packages #:key (system (%current-system))) "Return the closure of PACKAGES on SYSTEM--i.e., PACKAGES and the list of packages they depend on, recursively." diff --git a/guix/scripts/environment.scm b/guix/scripts/environment.scm index 6958bd6238..d555969b27 100644 --- a/guix/scripts/environment.scm +++ b/guix/scripts/environment.scm @@ -82,7 +82,7 @@ (define (package-environment-inputs package) packages for PACKAGE." ;; Remove non-package inputs such as origin records. (filter-map input->manifest-entry - (bag-transitive-inputs (package->bag package)))) + (package-development-inputs package system))) (define (show-help) (display (G_ "Usage: guix environment [OPTION]... PACKAGE... [-- COMMAND...] diff --git a/tests/packages.scm b/tests/packages.scm index 3756877270..266b5aeb7a 100644 --- a/tests/packages.scm +++ b/tests/packages.scm @@ -353,6 +353,20 @@ (define read-at (package-transitive-supported-systems d) (package-transitive-supported-systems e)))) +(test-assert "package-development-inputs" + ;; Note: Due to propagated inputs, 'package-development-inputs' returns a + ;; couple more inputs, such as 'linux-libre-headers'. + (lset<= equal? + `(("source" ,(package-source hello)) ,@(standard-packages)) + (package-development-inputs hello))) + +(test-assert "package-development-inputs, cross-compilation" + (lset<= equal? + `(("source" ,(package-source hello)) + ,@(standard-cross-packages "mips64el-linux-gnu" 'host) + ,@(standard-cross-packages "mips64el-linux-gnu" 'target)) + (package-development-inputs hello #:target "mips64el-linux-gnu"))) + (test-assert "package-closure" (let-syntax ((dummy-package/no-implicit (syntax-rules () From patchwork Mon Oct 11 21:38:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Ludovic_Court=C3=A8s?= X-Patchwork-Id: 33776 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 67A6D27BBE3; Mon, 11 Oct 2021 22:44:22 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id E149527BBE1 for ; Mon, 11 Oct 2021 22:44:21 +0100 (BST) Received: from localhost ([::1]:41204 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ma35p-0002Lv-2X for patchwork@mira.cbaines.net; Mon, 11 Oct 2021 17:44:21 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47516) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ma30g-0006aG-PL for guix-patches@gnu.org; Mon, 11 Oct 2021 17:39:03 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:49609) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1ma30g-0004px-Ef for guix-patches@gnu.org; Mon, 11 Oct 2021 17:39:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1ma30g-0001PS-C2 for guix-patches@gnu.org; Mon, 11 Oct 2021 17:39:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#50960] [PATCH v2 02/11] profiles: Add 'package->development-manifest'. Resent-From: Ludovic =?utf-8?q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Mon, 11 Oct 2021 21:39:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 50960 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 50960@debbugs.gnu.org Cc: Ludovic =?utf-8?q?Court=C3=A8s?= Received: via spool by 50960-submit@debbugs.gnu.org id=B50960.16339883175301 (code B ref 50960); Mon, 11 Oct 2021 21:39:02 +0000 Received: (at 50960) by debbugs.gnu.org; 11 Oct 2021 21:38:37 +0000 Received: from localhost ([127.0.0.1]:32900 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ma30G-0001NG-3y for submit@debbugs.gnu.org; Mon, 11 Oct 2021 17:38:36 -0400 Received: from eggs.gnu.org ([209.51.188.92]:45448) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ma308-0001Lz-Jd for 50960@debbugs.gnu.org; Mon, 11 Oct 2021 17:38:29 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:40566) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ma302-0004VZ-FR; Mon, 11 Oct 2021 17:38:22 -0400 Received: from 91-160-117-201.subs.proxad.net ([91.160.117.201]:53321 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1ma302-0007Y0-4M; Mon, 11 Oct 2021 17:38:22 -0400 From: Ludovic =?utf-8?q?Court=C3=A8s?= Date: Mon, 11 Oct 2021 23:38:00 +0200 Message-Id: <20211011213809.17482-3-ludo@gnu.org> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20211011213809.17482-1-ludo@gnu.org> References: <20211002102116.27726-1-ludo@gnu.org> <20211011213809.17482-1-ludo@gnu.org> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches From: Ludovic Courtès * guix/profiles.scm (package->development-manifest): New procedure. * guix/scripts/environment.scm (input->manifest-entry) (package-environment-inputs): Remove. * guix/scripts/environment.scm (options/resolve-packages): Use 'package->development-manifest' instead of 'package-environment-inputs'. * tests/profiles.scm ("package->development-manifest"): New test. --- doc/guix.texi | 11 +++++++++++ guix/profiles.scm | 19 +++++++++++++++++++ guix/scripts/environment.scm | 27 +++++---------------------- tests/profiles.scm | 7 +++++++ 4 files changed, 42 insertions(+), 22 deletions(-) diff --git a/doc/guix.texi b/doc/guix.texi index 49399e792b..bc3f5a537b 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -3340,6 +3340,17 @@ objects, like this: '("emacs" "guile@@2.2" "guile@@2.2:debug")) @end lisp +@findex package->development-manifest +You might also want to create a manifest for all the dependencies of a +package, rather than the package itself: + +@lisp +(package->development-manifest (specification->package "emacs")) +@end lisp + +The example above gives you all the software required to develop Emacs, +similar to what @command{guix environment emacs} provides. + @xref{export-manifest, @option{--export-manifest}}, to learn how to obtain a manifest file from an existing profile. diff --git a/guix/profiles.scm b/guix/profiles.scm index 2486f91d09..9f30349c69 100644 --- a/guix/profiles.scm +++ b/guix/profiles.scm @@ -124,6 +124,7 @@ (define-module (guix profiles) profile-manifest package->manifest-entry + package->development-manifest packages->manifest ca-certificate-bundle %default-profile-hooks @@ -400,6 +401,24 @@ (define* (package->manifest-entry package #:optional (output "out") (properties properties)))) entry)) +(define* (package->development-manifest package + #:optional + (system (%current-system)) + #:key target) + "Return a manifest for the \"development inputs\" of PACKAGE for SYSTEM, +optionally when cross-compiling to TARGET. Development inputs include both +explicit and implicit inputs of PACKAGE." + (manifest + (filter-map (match-lambda + ((label (? package? package)) + (package->manifest-entry package)) + ((label (? package? package) output) + (package->manifest-entry package output)) + ;; TODO: Support . + (_ + #f)) + (package-development-inputs package system #:target target)))) + (define (packages->manifest packages) "Return a list of manifest entries, one for each item listed in PACKAGES. Elements of PACKAGES can be either package objects or package/string tuples diff --git a/guix/scripts/environment.scm b/guix/scripts/environment.scm index d555969b27..54f48a7482 100644 --- a/guix/scripts/environment.scm +++ b/guix/scripts/environment.scm @@ -66,24 +66,6 @@ (define* (show-search-paths profile manifest #:key pure?) (newline))) (profile-search-paths profile manifest))) -(define (input->manifest-entry input) - "Return a manifest entry for INPUT, or #f if INPUT does not correspond to a -package." - (match input - ((_ (? package? package)) - (package->manifest-entry package)) - ((_ (? package? package) output) - (package->manifest-entry package output)) - (_ - #f))) - -(define (package-environment-inputs package) - "Return a list of manifest entries corresponding to the transitive input -packages for PACKAGE." - ;; Remove non-package inputs such as origin records. - (filter-map input->manifest-entry - (package-development-inputs package system))) - (define (show-help) (display (G_ "Usage: guix environment [OPTION]... PACKAGE... [-- COMMAND...] Build an environment that includes the dependencies of PACKAGE and execute @@ -297,11 +279,11 @@ (define (packages->outputs packages mode) ((? package? package) (if (eq? mode 'ad-hoc-package) (list (package->manifest-entry* package)) - (package-environment-inputs package))) + (manifest-entries (package->development-manifest package)))) (((? package? package) (? string? output)) (if (eq? mode 'ad-hoc-package) (list (package->manifest-entry* package output)) - (package-environment-inputs package))) + (manifest-entries (package->development-manifest package)))) ((lst ...) (append-map (cut packages->outputs <> mode) lst)))) @@ -313,8 +295,9 @@ (define (packages->outputs packages mode) (specification->package+output spec))) (list (package->manifest-entry* package output)))) (('package 'package (? string? spec)) - (package-environment-inputs - (transform (specification->package+output spec)))) + (manifest-entries + (package->development-manifest + (transform (specification->package+output spec))))) (('expression mode str) ;; Add all the outputs of the package STR evaluates to. (packages->outputs (read/eval str) mode)) diff --git a/tests/profiles.scm b/tests/profiles.scm index 06a0387221..cac5b73347 100644 --- a/tests/profiles.scm +++ b/tests/profiles.scm @@ -265,6 +265,13 @@ (define transform1 (manifest-transaction-removal-candidate? guile-2.0.9 t) (null? install) (null? downgrade) (null? upgrade))))) +(test-assert "package->development-manifest" + (let ((manifest (package->development-manifest packages:hello))) + (every (lambda (name) + (manifest-installed? manifest + (manifest-pattern (name name)))) + '("gcc" "binutils" "glibc" "coreutils" "grep" "sed")))) + (test-assertm "profile-derivation" (mlet* %store-monad ((entry -> (package->manifest-entry %bootstrap-guile)) From patchwork Mon Oct 11 21:38:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Ludovic_Court=C3=A8s?= X-Patchwork-Id: 33771 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id B566C27BBE3; Mon, 11 Oct 2021 22:39:46 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 4E91327BBE1 for ; Mon, 11 Oct 2021 22:39:45 +0100 (BST) Received: from localhost ([::1]:36234 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ma31M-0007GD-EF for patchwork@mira.cbaines.net; Mon, 11 Oct 2021 17:39:44 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47526) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ma30j-0006bl-0C for guix-patches@gnu.org; Mon, 11 Oct 2021 17:39:05 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:49614) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1ma30i-0004rC-BL for guix-patches@gnu.org; Mon, 11 Oct 2021 17:39:04 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1ma30i-0001Q4-8O for guix-patches@gnu.org; Mon, 11 Oct 2021 17:39:04 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#50960] [PATCH v2 03/11] Add 'guix shell'. Resent-From: Ludovic =?utf-8?q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Mon, 11 Oct 2021 21:39:04 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 50960 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 50960@debbugs.gnu.org Cc: Ludovic =?utf-8?q?Court=C3=A8s?= Received: via spool by 50960-submit@debbugs.gnu.org id=B50960.16339883325362 (code B ref 50960); Mon, 11 Oct 2021 21:39:04 +0000 Received: (at 50960) by debbugs.gnu.org; 11 Oct 2021 21:38:52 +0000 Received: from localhost ([127.0.0.1]:32912 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ma30R-0001OF-2V for submit@debbugs.gnu.org; Mon, 11 Oct 2021 17:38:52 -0400 Received: from eggs.gnu.org ([209.51.188.92]:45452) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ma308-0001M1-VV for 50960@debbugs.gnu.org; Mon, 11 Oct 2021 17:38:35 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:40568) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ma303-0004Vy-Bx; Mon, 11 Oct 2021 17:38:23 -0400 Received: from 91-160-117-201.subs.proxad.net ([91.160.117.201]:53321 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1ma302-0007Y0-S1; Mon, 11 Oct 2021 17:38:23 -0400 From: Ludovic =?utf-8?q?Court=C3=A8s?= Date: Mon, 11 Oct 2021 23:38:01 +0200 Message-Id: <20211011213809.17482-4-ludo@gnu.org> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20211011213809.17482-1-ludo@gnu.org> References: <20211002102116.27726-1-ludo@gnu.org> <20211011213809.17482-1-ludo@gnu.org> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches From: Ludovic Courtès * guix/scripts/shell.scm, tests/guix-shell.sh: New files. * Makefile.am (MODULES): Add 'shell.scm'. (SH_TESTS): Add 'tests/guix-shell.sh'. * guix/scripts/environment.scm (show-environment-options-help): New procedure. (show-help): Use it. (guix-environment*): New procedure. (guix-environment): Use it. * po/guix/POTFILES.in: Add it. * doc/guix.texi (Features): Refer to "guix shell" (Invoking guix package): Likewise. (Development): Likewise. (Invoking guix shell): New node. (Invoking guix environment): Add deprecation warning. (Debugging Build Failures): Use 'guix shell' in examples. (Invoking guix container): Refer to 'guix shell'. (Invoking guix processes, Virtualization Services): Adjust examples to use 'guix shell'. * doc/contributing.texi (Building from Git): Refer to 'guix shell'. --- Makefile.am | 2 + doc/contributing.texi | 8 +- doc/guix.texi | 366 ++++++++++++++++++++++++++++++++--- guix/scripts/environment.scm | 52 +++-- guix/scripts/shell.scm | 135 +++++++++++++ po/guix/POTFILES.in | 1 + tests/guix-shell.sh | 54 ++++++ 7 files changed, 575 insertions(+), 43 deletions(-) create mode 100644 guix/scripts/shell.scm create mode 100644 tests/guix-shell.sh diff --git a/Makefile.am b/Makefile.am index b66789fa0b..c28c8799ec 100644 --- a/Makefile.am +++ b/Makefile.am @@ -315,6 +315,7 @@ MODULES = \ guix/scripts/import/stackage.scm \ guix/scripts/import/texlive.scm \ guix/scripts/environment.scm \ + guix/scripts/shell.scm \ guix/scripts/publish.scm \ guix/scripts/edit.scm \ guix/scripts/size.scm \ @@ -550,6 +551,7 @@ SH_TESTS = \ tests/guix-authenticate.sh \ tests/guix-environment.sh \ tests/guix-environment-container.sh \ + tests/guix-shell.sh \ tests/guix-graph.sh \ tests/guix-describe.sh \ tests/guix-repl.sh \ diff --git a/doc/contributing.texi b/doc/contributing.texi index fbb3c47c78..363bbe9278 100644 --- a/doc/contributing.texi +++ b/doc/contributing.texi @@ -73,10 +73,10 @@ all the dependencies and appropriate environment variables are set up to hack on Guix: @example -guix environment guix --pure +guix shell -D guix --pure @end example -@xref{Invoking guix environment}, for more information on that command. +@xref{Invoking guix shell}, for more information on that command. If you are unable to use Guix when building Guix from a checkout, the following are the required packages in addition to those mentioned in the @@ -92,10 +92,10 @@ installation instructions (@pxref{Requirements}). @end itemize On Guix, extra dependencies can be added by instead running @command{guix -environment} with @option{--ad-hoc}: +shell}: @example -guix environment guix --pure --ad-hoc help2man git strace +guix shell -D guix help2man git strace --pure @end example Run @command{./bootstrap} to generate the build system infrastructure diff --git a/doc/guix.texi b/doc/guix.texi index bc3f5a537b..b0d745b9e3 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -118,6 +118,7 @@ Documentation License''. @dircategory Software development @direntry +* guix shell: (guix)Invoking guix shell. Creating software environments. * guix environment: (guix)Invoking guix environment. Building development environments with Guix. * guix build: (guix)Invoking guix build. Building packages. * guix pack: (guix)Invoking guix pack. Creating binary bundles. @@ -261,6 +262,7 @@ Channels Development +* Invoking guix shell:: Spawning one-off software environments. * Invoking guix environment:: Setting up development environments. * Invoking guix pack:: Creating software bundles. * The GCC toolchain:: Working with languages supported by GCC. @@ -3066,10 +3068,10 @@ substitutes: they can force a local build and @emph{challenge} providers (@pxref{Invoking guix challenge}). Control over the build environment is a feature that is also useful for -developers. The @command{guix environment} command allows developers of +developers. The @command{guix shell} command allows developers of a package to quickly set up the right development environment for their package, without having to manually install the dependencies of the -package into their profile (@pxref{Invoking guix environment}). +package into their profile (@pxref{Invoking guix shell}). @cindex replication, of software environments @cindex provenance tracking, of software artifacts @@ -3233,7 +3235,7 @@ As an example, @var{file} might contain a definition like this Developers may find it useful to include such a @file{guix.scm} file in the root of their project source tree that can be used to test development snapshots and create reproducible development environments -(@pxref{Invoking guix environment}). +(@pxref{Invoking guix shell}). The @var{file} may also contain a JSON representation of one or more package definitions. Running @code{guix package -f} on @@ -5558,31 +5560,352 @@ If you are a software developer, Guix provides tools that you should find helpful---independently of the language you're developing in. This is what this chapter is about. -The @command{guix environment} command provides a convenient way to set up -@dfn{development environments} containing all the dependencies and tools -necessary to work on the software package of your choice. The @command{guix +The @command{guix shell} command provides a convenient way to set up +one-off software environments, be it for development purposes or to run +a command without installing it in your profile. The @command{guix pack} command allows you to create @dfn{application bundles} that can be easily distributed to users who do not run Guix. @menu +* Invoking guix shell:: Spawning one-off software environments. * Invoking guix environment:: Setting up development environments. * Invoking guix pack:: Creating software bundles. * The GCC toolchain:: Working with languages supported by GCC. * Invoking guix git authenticate:: Authenticating Git repositories. @end menu -@node Invoking guix environment -@section Invoking @command{guix environment} +@node Invoking guix shell +@section Invoking @command{guix shell} @cindex reproducible build environments @cindex development environments @cindex @command{guix environment} @cindex environment, package build environment -The purpose of @command{guix environment} is to assist hackers in -creating reproducible development environments without polluting their -package profile. The @command{guix environment} tool takes one or more -packages, builds all of their inputs, and creates a shell -environment to use them. +The purpose of @command{guix shell} is to make it easy to create one-off +software environments, without changing one's profile. It is typically +used to create development environments; it is also a convenient way to +run applications without ``polluting'' your profile. + +@quotation Note +The @command{guix shell} command was recently introduced to supersede +@command{guix environment} (@pxref{Invoking guix environment}). If you +are familiar with @command{guix environment}, you will notice that it is +similar but also---we hope!---more convenient. +@end quotation + +The general syntax is: + +@example +guix shell [@var{options}] [@var{package}@dots{}] +@end example + +The following example creates a environment containing Python and NumPy, +building and downloading any missing package, and runs the +@command{python3} command in that environment: + +@example +guix shell python python-numpy -- python3 +@end example + +Development environments can be created as in the example below, which +spawns an interactive shell containing all the dependencies and +environment variables needed to work on Inkscape: + +@example +guix shell --development inkscape +@end example + +Exiting the shell places the user back in the original environment +before @command{guix shell} was invoked. The next garbage collection +(@pxref{Invoking guix gc}) may clean up packages that were installed in +the environment and that are no longer used outside of it. + +By default, the shell session or command runs in an @emph{augmented} +environment, where the new packages are added to search path environment +variables such as @code{PATH}. You can, instead, choose to create an +@emph{isolated} environment containing nothing but the packages you +asked for. Passing the @option{--pure} option clears environment +variable definitions found in the parent environment@footnote{Users +sometimes wrongfully augment environment variables such as @env{PATH} in +their @file{~/.bashrc} file. As a consequence, when @command{guix +environment} launches it, Bash may read @file{~/.bashrc}, thereby +introducing ``impurities'' in these environment variables. It is an +error to define such environment variables in @file{.bashrc}; instead, +they should be defined in @file{.bash_profile}, which is sourced only by +log-in shells. @xref{Bash Startup Files,,, bash, The GNU Bash Reference +Manual}, for details on Bash start-up files.}; passing +@option{--container} goes one step further by spawning a @dfn{container} +isolated from the rest of the system: + +@example +guix shell --container emacs gcc-toolchain +@end example + +The command above spawns an interactive shell in a container when +nothing but @code{emacs}, @code{gcc-toolchain}, and their dependencies +is available. The container lacks network access and shares no files +other than the current working directory with the surrounding +environment. This is useful to prevent access to system-wide resources +such as @file{/usr/bin} on foreign distros. + +This @option{--container} option can also prove useful if you wish to +run a security-sensitive application, such as a web browser, in an +isolated environment. For example, the command below launches +Ungoogled-Chromium in an isolated environment, this time sharing network +access with the host and preserving its @code{DISPLAY} environment +variable, but without even sharing the current directory: + +@example +guix shell --container --network --no-cwd ungoogled-chromium \ + --preserve='^DISPLAY$' -- chromium +@end example + +@vindex GUIX_ENVIRONMENT +@command{guix shell} defines the @env{GUIX_ENVIRONMENT} +variable in the shell it spawns; its value is the file name of the +profile of this environment. This allows users to, say, define a +specific prompt for development environments in their @file{.bashrc} +(@pxref{Bash Startup Files,,, bash, The GNU Bash Reference Manual}): + +@example +if [ -n "$GUIX_ENVIRONMENT" ] +then + export PS1="\u@@\h \w [dev]\$ " +fi +@end example + +@noindent +...@: or to browse the profile: + +@example +$ ls "$GUIX_ENVIRONMENT/bin" +@end example + +The available options are summarized below. + +@table @code +@item --development +@itemx -D +Cause @command{guix shell} to include in the environment the +dependencies of the following package rather than the package itself. +This can be combined with other packages. For instance, the command +below starts an interactive shell containing the build-time dependencies +of GNU@tie{}Guile, plus Autoconf, Automake, and Libtool: + +@example +guix shell -D guile autoconf automake libtool +@end example + +@item --expression=@var{expr} +@itemx -e @var{expr} +Create an environment for the package or list of packages that +@var{expr} evaluates to. + +For example, running: + +@example +guix shell -D -e '(@@ (gnu packages maths) petsc-openmpi)' +@end example + +starts a shell with the environment for this specific variant of the +PETSc package. + +Running: + +@example +guix shell -e '(@@ (gnu) %base-packages)' +@end example + +starts a shell with all the base system packages available. + +The above commands only use the default output of the given packages. +To select other outputs, two element tuples can be specified: + +@example +guix shell -e '(list (@@ (gnu packages bash) bash) "include")' +@end example + +@item --file=@var{file} +@itemx -f @var{file} +Create an environment containing the package or list of packages that +the code within @var{file} evaluates to. + +As an example, @var{file} might contain a definition like this +(@pxref{Defining Packages}): + +@lisp +@verbatiminclude environment-gdb.scm +@end lisp + +With the file above, you can enter a development environment for GDB by +running: + +@example +guix shell -D -f gdb-devel.scm +@end example + +@item --manifest=@var{file} +@itemx -m @var{file} +Create an environment for the packages contained in the manifest object +returned by the Scheme code in @var{file}. This option can be repeated +several times, in which case the manifests are concatenated. + +This is similar to the same-named option in @command{guix package} +(@pxref{profile-manifest, @option{--manifest}}) and uses the same +manifest files. + +@item --pure +Unset existing environment variables when building the new environment, except +those specified with @option{--preserve} (see below). This has the effect of +creating an environment in which search paths only contain package inputs. + +@item --preserve=@var{regexp} +@itemx -E @var{regexp} +When used alongside @option{--pure}, preserve the environment variables +matching @var{regexp}---in other words, put them on a ``white list'' of +environment variables that must be preserved. This option can be repeated +several times. + +@example +guix shell --pure --preserve=^SLURM openmpi @dots{} \ + -- mpirun @dots{} +@end example + +This example runs @command{mpirun} in a context where the only environment +variables defined are @env{PATH}, environment variables whose name starts +with @samp{SLURM}, as well as the usual ``precious'' variables (@env{HOME}, +@env{USER}, etc.). + +@item --search-paths +Display the environment variable definitions that make up the +environment. + +@item --system=@var{system} +@itemx -s @var{system} +Attempt to build for @var{system}---e.g., @code{i686-linux}. + +@item --container +@itemx -C +@cindex container +Run @var{command} within an isolated container. The current working +directory outside the container is mapped inside the container. +Additionally, unless overridden with @option{--user}, a dummy home +directory is created that matches the current user's home directory, and +@file{/etc/passwd} is configured accordingly. + +The spawned process runs as the current user outside the container. Inside +the container, it has the same UID and GID as the current user, unless +@option{--user} is passed (see below). + +@item --network +@itemx -N +For containers, share the network namespace with the host system. +Containers created without this flag only have access to the loopback +device. + +@item --link-profile +@itemx -P +For containers, link the environment profile to @file{~/.guix-profile} +within the container and set @code{GUIX_ENVIRONMENT} to that. +This is equivalent to making @file{~/.guix-profile} a symlink to the +actual profile within the container. +Linking will fail and abort the environment if the directory already +exists, which will certainly be the case if @command{guix shell} +was invoked in the user's home directory. + +Certain packages are configured to look in @file{~/.guix-profile} for +configuration files and data;@footnote{For example, the +@code{fontconfig} package inspects @file{~/.guix-profile/share/fonts} +for additional fonts.} @option{--link-profile} allows these programs to +behave as expected within the environment. + +@item --user=@var{user} +@itemx -u @var{user} +For containers, use the username @var{user} in place of the current +user. The generated @file{/etc/passwd} entry within the container will +contain the name @var{user}, the home directory will be +@file{/home/@var{user}}, and no user GECOS data will be copied. Furthermore, +the UID and GID inside the container are 1000. @var{user} +need not exist on the system. + +Additionally, any shared or exposed path (see @option{--share} and +@option{--expose} respectively) whose target is within the current user's +home directory will be remapped relative to @file{/home/USER}; this +includes the automatic mapping of the current working directory. + +@example +# will expose paths as /home/foo/wd, /home/foo/test, and /home/foo/target +cd $HOME/wd +guix shell --container --user=foo \ + --expose=$HOME/test \ + --expose=/tmp/target=$HOME/target +@end example + +While this will limit the leaking of user identity through home paths +and each of the user fields, this is only one useful component of a +broader privacy/anonymity solution---not one in and of itself. + +@item --no-cwd +For containers, the default behavior is to share the current working +directory with the isolated container and immediately change to that +directory within the container. If this is undesirable, +@option{--no-cwd} will cause the current working directory to @emph{not} +be automatically shared and will change to the user's home directory +within the container instead. See also @option{--user}. + +@item --expose=@var{source}[=@var{target}] +@itemx --share=@var{source}[=@var{target}] +For containers, @option{--expose} (resp. @option{--share}) exposes the +file system @var{source} from the host system as the read-only +(resp. writable) file system @var{target} within the container. If +@var{target} is not specified, @var{source} is used as the target mount +point in the container. + +The example below spawns a Guile REPL in a container in which the user's +home directory is accessible read-only via the @file{/exchange} +directory: + +@example +guix shell --container --expose=$HOME=/exchange guile -- guile +@end example + +@item --root=@var{file} +@itemx -r @var{file} +@cindex persistent environment +@cindex garbage collector root, for environments +Make @var{file} a symlink to the profile for this environment, and +register it as a garbage collector root. + +This is useful if you want to protect your environment from garbage +collection, to make it ``persistent''. + +When this option is omitted, the environment is protected from garbage +collection only for the duration of the @command{guix shell} +session. This means that next time you recreate the same environment, +you could have to rebuild or re-download packages. @xref{Invoking guix +gc}, for more on GC roots. +@end table + +@command{guix shell} also supports all of the common build options that +@command{guix build} supports (@pxref{Common Build Options}) as well as +package transformation options (@pxref{Package Transformation Options}). + +@node Invoking guix environment +@section Invoking @command{guix environment} + +The purpose of @command{guix environment} is to assists in creating +development environments. + +@quotation Deprecation warning +The @command{guix environment} command is deprecated in favor of +@command{guix shell}, which performs similar functions but is more +convenient to use. @xref{Invoking guix shell}. + +Being deprecated, @command{guix environment} is slated for eventual +removal, but the Guix project is committed to keeping it until May 1st, +2023. Please get in touch with us at @email{guix-devel@@gnu.org} if you +would like to discuss it. +@end quotation The general syntax is: @@ -11098,14 +11421,14 @@ a container similar to the one the build daemon creates: $ guix build -K foo @dots{} $ cd /tmp/guix-build-foo.drv-0 -$ guix environment --no-grafts -C foo --ad-hoc strace gdb +$ guix shell --no-grafts -C foo strace gdb [env]# source ./environment-variables [env]# cd foo-1.2 @end example Here, @command{guix environment -C} creates a container and spawns a new -shell in it (@pxref{Invoking guix environment}). The @command{--ad-hoc -strace gdb} part adds the @command{strace} and @command{gdb} commands to +shell in it (@pxref{Invoking guix shell}). The @command{strace gdb} +part adds the @command{strace} and @command{gdb} commands to the container, which you may find handy while debugging. The @option{--no-grafts} option makes sure we get the exact same environment, with ungrafted packages (@pxref{Security Updates}, for more @@ -11119,7 +11442,7 @@ remove @file{/bin/sh}: @end example (Don't worry, this is harmless: this is all happening in the throw-away -container created by @command{guix environment}.) +container created by @command{guix shell}.) The @command{strace} command is probably not in the search path, but we can run: @@ -13315,8 +13638,8 @@ is subject to radical change in the future. The purpose of @command{guix container} is to manipulate processes running within an isolated environment, commonly known as a -``container'', typically created by the @command{guix environment} -(@pxref{Invoking guix environment}) and @command{guix system container} +``container'', typically created by the @command{guix shell} +(@pxref{Invoking guix shell}) and @command{guix system container} (@pxref{Invoking guix system}) commands. The general syntax is: @@ -13502,7 +13825,7 @@ listed.}. Here's an example of the information it returns: $ sudo guix processes SessionPID: 19002 ClientPID: 19090 -ClientCommand: guix environment --ad-hoc python +ClientCommand: guix shell python SessionPID: 19402 ClientPID: 19367 @@ -29697,8 +30020,7 @@ When the service is running, you can view its console by connecting to it with a VNC client, for example with: @example -guix environment --ad-hoc tigervnc-client -- \ - vncviewer localhost:5900 +guix shell tigervnc-client -- vncviewer localhost:5900 @end example The default configuration (see @code{hurd-vm-configuration} below) diff --git a/guix/scripts/environment.scm b/guix/scripts/environment.scm index 54f48a7482..77956fc018 100644 --- a/guix/scripts/environment.scm +++ b/guix/scripts/environment.scm @@ -50,7 +50,11 @@ (define-module (guix scripts environment) #:use-module (srfi srfi-37) #:use-module (srfi srfi-98) #:export (assert-container-features - guix-environment)) + guix-environment + guix-environment* + show-environment-options-help + (%options . %environment-options) + (%default-options . %environment-default-options))) (define %default-shell (or (getenv "SHELL") "/bin/sh")) @@ -66,23 +70,16 @@ (define* (show-search-paths profile manifest #:key pure?) (newline))) (profile-search-paths profile manifest))) -(define (show-help) - (display (G_ "Usage: guix environment [OPTION]... PACKAGE... [-- COMMAND...] -Build an environment that includes the dependencies of PACKAGE and execute -COMMAND or an interactive shell in that environment.\n")) +(define (show-environment-options-help) + "Print help about options shared between 'guix environment' and 'guix +shell'." (display (G_ " -e, --expression=EXPR create environment for the package that EXPR evaluates to")) (display (G_ " - -l, --load=FILE create environment for the package that the code within - FILE evaluates to")) - (display (G_ " -m, --manifest=FILE create environment with the manifest from FILE")) (display (G_ " -p, --profile=PATH create environment from profile at PATH")) - (display (G_ " - --ad-hoc include all specified packages in the environment instead - of only their inputs")) (display (G_ " --pure unset existing environment variables")) (display (G_ " @@ -118,7 +115,24 @@ (define (show-help) (display (G_ " -v, --verbosity=LEVEL use the given verbosity LEVEL")) (display (G_ " - --bootstrap use bootstrap binaries to build the environment")) + --bootstrap use bootstrap binaries to build the environment"))) + +(define (show-help) + (display (G_ "Usage: guix environment [OPTION]... PACKAGE... [-- COMMAND...] +Build an environment that includes the dependencies of PACKAGE and execute +COMMAND or an interactive shell in that environment.\n")) + (warning (G_ "This command is deprecated in favor of 'guix shell'.\n")) + (newline) + + ;; These two options are left out in 'guix shell'. + (display (G_ " + -l, --load=FILE create environment for the package that the code within + FILE evaluates to")) + (display (G_ " + --ad-hoc include all specified packages in the environment instead + of only their inputs")) + + (show-environment-options-help) (newline) (show-build-options-help) (newline) @@ -649,11 +663,15 @@ (define (register-gc-root target root) (define-command (guix-environment . args) (category development) - (synopsis "spawn one-off software environments") + (synopsis "spawn one-off software environments (deprecated)") + (guix-environment* (parse-args args))) + +(define (guix-environment* opts) + "Run the 'guix environment' command on OPTS, an alist resulting for +command-line option processing with 'parse-command-line'." (with-error-handling - (let* ((opts (parse-args args)) - (pure? (assoc-ref opts 'pure)) + (let* ((pure? (assoc-ref opts 'pure)) (container? (assoc-ref opts 'container?)) (link-prof? (assoc-ref opts 'link-profile?)) (network? (assoc-ref opts 'network?)) @@ -724,8 +742,8 @@ (define manifest (prof-drv (manifest->derivation manifest system bootstrap?)) (profile -> (if profile - (readlink* profile) - (derivation->output-path prof-drv))) + (readlink* profile) + (derivation->output-path prof-drv))) (gc-root -> (assoc-ref opts 'gc-root))) ;; First build the inputs. This is necessary even for diff --git a/guix/scripts/shell.scm b/guix/scripts/shell.scm new file mode 100644 index 0000000000..190dd8837d --- /dev/null +++ b/guix/scripts/shell.scm @@ -0,0 +1,135 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright © 2021 Ludovic Courtès +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see . + +(define-module (guix scripts shell) + #:use-module (guix ui) + #:use-module (guix scripts environment) + #:autoload (guix scripts build) (show-build-options-help) + #:autoload (guix transformations) (show-transformation-options-help) + #:use-module (guix scripts) + #:use-module (srfi srfi-1) + #:use-module (srfi srfi-26) + #:use-module (srfi srfi-37) + #:use-module (srfi srfi-71) + #:use-module (ice-9 match) + #:export (guix-shell)) + +(define (show-help) + (display (G_ "Usage: guix shell [OPTION] PACKAGES... [-- COMMAND...] +Build an environment that includes PACKAGES and execute COMMAND or an +interactive shell in that environment.\n")) + (newline) + + ;; These two options differ from 'guix environment'. + (display (G_ " + -D, --development include the development inputs of the next package")) + (display (G_ " + -f, --file=FILE create environment for the package that the code within + FILE evaluates to")) + + (show-environment-options-help) + (newline) + (show-build-options-help) + (newline) + (show-transformation-options-help) + (newline) + (display (G_ " + -h, --help display this help and exit")) + (display (G_ " + -V, --version display version information and exit")) + (newline) + (show-bug-report-information)) + +(define (tag-package-arg opts arg) + "Return a two-element list with the form (TAG ARG) that tags ARG with either +'ad-hoc' in OPTS has the 'ad-hoc?' key set to #t, or 'inputs' otherwise." + (if (assoc-ref opts 'ad-hoc?) + `(ad-hoc-package ,arg) + `(package ,arg))) + +(define (ensure-ad-hoc alist) + (if (assq-ref alist 'ad-hoc?) + alist + `((ad-hoc? . #t) ,@alist))) + +(define (wrapped-option opt) + "Wrap OPT, a SRFI-37 option, such that its processor always adds the +'ad-hoc?' flag to the resulting alist." + (option (option-names opt) + (option-required-arg? opt) + (option-optional-arg? opt) + (compose ensure-ad-hoc (option-processor opt)))) + +(define %options + ;; Specification of the command-line options. + (let ((to-remove '("ad-hoc" "inherit" "load" "help" "version"))) + (append + (list (option '(#\h "help") #f #f + (lambda args + (show-help) + (exit 0))) + (option '(#\V "version") #f #f + (lambda args + (show-version-and-exit "guix shell"))) + + (option '(#\D "development") #f #f + (lambda (opt name arg result) + ;; Temporarily remove the 'ad-hoc?' flag from result. + ;; The next option will put it back thanks to + ;; 'wrapped-option'. + (alist-delete 'ad-hoc? result))) + + ;; For consistency with 'guix package', support '-f' rather than + ;; '-l' like 'guix environment' does. + (option '(#\f "file") #t #f + (lambda (opt name arg result) + (alist-cons 'load (tag-package-arg result arg) + result)))) + (filter-map (lambda (opt) + (and (not (any (lambda (name) + (member name to-remove)) + (option-names opt))) + (wrapped-option opt))) + %environment-options)))) + +(define %default-options + `((ad-hoc? . #t) ;always true + ,@%environment-default-options)) + +(define (parse-args args) + "Parse the list of command line arguments ARGS." + (define (handle-argument arg result) + (alist-cons 'package (tag-package-arg result arg) + (ensure-ad-hoc result))) + + ;; The '--' token is used to separate the command to run from the rest of + ;; the operands. + (let ((args command (break (cut string=? "--" <>) args))) + (let ((opts (parse-command-line args %options (list %default-options) + #:argument-handler handle-argument))) + (match command + (() opts) + (("--") opts) + (("--" command ...) (alist-cons 'exec command opts)))))) + + +(define-command (guix-shell . args) + (category development) + (synopsis "spawn one-off software environments") + + (guix-environment* (parse-args args))) diff --git a/po/guix/POTFILES.in b/po/guix/POTFILES.in index f5b76bf582..f8abeb2d38 100644 --- a/po/guix/POTFILES.in +++ b/po/guix/POTFILES.in @@ -99,6 +99,7 @@ guix/derivations.scm guix/scripts/archive.scm guix/scripts/build.scm guix/scripts/environment.scm +guix/scripts/shell.scm guix/scripts/time-machine.scm guix/scripts/import/cpan.scm guix/scripts/import/crate.scm diff --git a/tests/guix-shell.sh b/tests/guix-shell.sh new file mode 100644 index 0000000000..f08637f7ff --- /dev/null +++ b/tests/guix-shell.sh @@ -0,0 +1,54 @@ +# GNU Guix --- Functional package management for GNU +# Copyright © 2021 Ludovic Courtès +# +# This file is part of GNU Guix. +# +# GNU Guix is free software; you can redistribute it and/or modify it +# under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 3 of the License, or (at +# your option) any later version. +# +# GNU Guix is distributed in the hope that it will be useful, but +# WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with GNU Guix. If not, see . + +# +# Test the 'guix shell' alias. +# + +guix shell --version + +tmpdir="t-guix-shell-$$" +trap 'rm -r "$tmpdir"' EXIT +mkdir "$tmpdir" + +guix shell --bootstrap --pure guile-bootstrap -- guile --version + +# '--ad-hoc' is a thing of the past. +! guix shell --ad-hoc guile-bootstrap + +if guile -c '(getaddrinfo "www.gnu.org" "80" AI_NUMERICSERV)' 2> /dev/null +then + # Compute the build environment for the initial GNU Make. + guix shell --bootstrap --no-substitutes --search-paths --pure \ + -D -e '(@ (guix tests) gnu-make-for-tests)' > "$tmpdir/a" + + # Make sure bootstrap binaries are in the profile. + profile=`grep "^export PATH" "$tmpdir/a" | sed -r 's|^.*="(.*)/bin"|\1|'` + + # Make sure the bootstrap binaries are all listed where they belong. + grep -E "^export PATH=\"$profile/bin\"" "$tmpdir/a" + grep -E "^export CPATH=\"$profile/include\"" "$tmpdir/a" + grep -E "^export LIBRARY_PATH=\"$profile/lib\"" "$tmpdir/a" + for dep in bootstrap-binaries-0 gcc-bootstrap-0 glibc-bootstrap-0 + do + guix gc --references "$profile" | grep "$dep" + done + + # 'make-boot0' itself must not be listed. + ! guix gc --references "$profile" | grep make-boot0 +fi From patchwork Mon Oct 11 21:38:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Ludovic_Court=C3=A8s?= X-Patchwork-Id: 33778 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 99BA127BBE3; Mon, 11 Oct 2021 22:44:30 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 1BF2527BBE1 for ; Mon, 11 Oct 2021 22:44:30 +0100 (BST) Received: from localhost ([::1]:41706 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ma35x-0002gZ-5v for patchwork@mira.cbaines.net; Mon, 11 Oct 2021 17:44:29 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47518) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ma30h-0006aT-2t for guix-patches@gnu.org; Mon, 11 Oct 2021 17:39:03 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:49610) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1ma30g-0004qP-R4 for guix-patches@gnu.org; Mon, 11 Oct 2021 17:39:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1ma30g-0001Pb-OB for guix-patches@gnu.org; Mon, 11 Oct 2021 17:39:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#50960] [PATCH v2 04/11] DRAFT shell: By default load the local 'guix.scm' or 'manifest.scm' file. Resent-From: Ludovic =?utf-8?q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Mon, 11 Oct 2021 21:39:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 50960 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 50960@debbugs.gnu.org Cc: Ludovic =?utf-8?q?Court=C3=A8s?= Received: via spool by 50960-submit@debbugs.gnu.org id=B50960.16339883175314 (code B ref 50960); Mon, 11 Oct 2021 21:39:02 +0000 Received: (at 50960) by debbugs.gnu.org; 11 Oct 2021 21:38:37 +0000 Received: from localhost ([127.0.0.1]:32903 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ma30G-0001NS-S3 for submit@debbugs.gnu.org; Mon, 11 Oct 2021 17:38:37 -0400 Received: from eggs.gnu.org ([209.51.188.92]:45458) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ma309-0001M2-5T for 50960@debbugs.gnu.org; Mon, 11 Oct 2021 17:38:30 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:40570) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ma303-0004WO-Ty; Mon, 11 Oct 2021 17:38:23 -0400 Received: from 91-160-117-201.subs.proxad.net ([91.160.117.201]:53321 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1ma303-0007Y0-KX; Mon, 11 Oct 2021 17:38:23 -0400 From: Ludovic =?utf-8?q?Court=C3=A8s?= Date: Mon, 11 Oct 2021 23:38:02 +0200 Message-Id: <20211011213809.17482-5-ludo@gnu.org> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20211011213809.17482-1-ludo@gnu.org> References: <20211002102116.27726-1-ludo@gnu.org> <20211011213809.17482-1-ludo@gnu.org> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches DRAFT: Add doc. * guix/scripts/shell.scm (parse-args): Add call to 'auto-detect-manifest'. (find-file-in-parent-directories, auto-detect-manifest): New procedures. * tests/guix-shell.sh: Add test. --- guix/scripts/shell.scm | 67 ++++++++++++++++++++++++++++++++++++++---- tests/guix-shell.sh | 36 +++++++++++++++++++++++ 2 files changed, 98 insertions(+), 5 deletions(-) diff --git a/guix/scripts/shell.scm b/guix/scripts/shell.scm index 190dd8837d..39d843bde7 100644 --- a/guix/scripts/shell.scm +++ b/guix/scripts/shell.scm @@ -22,6 +22,8 @@ (define-module (guix scripts shell) #:autoload (guix scripts build) (show-build-options-help) #:autoload (guix transformations) (show-transformation-options-help) #:use-module (guix scripts) + #:use-module (guix packages) + #:use-module (guix profiles) #:use-module (srfi srfi-1) #:use-module (srfi srfi-26) #:use-module (srfi srfi-37) @@ -41,6 +43,8 @@ (define (show-help) (display (G_ " -f, --file=FILE create environment for the package that the code within FILE evaluates to")) + (display (G_ " + -q inhibit loading of 'guix.scm' and 'manifest.scm'")) (show-environment-options-help) (newline) @@ -99,7 +103,10 @@ (define %options (option '(#\f "file") #t #f (lambda (opt name arg result) (alist-cons 'load (tag-package-arg result arg) - result)))) + result))) + (option '(#\q) #f #f + (lambda (opt name arg result) + (alist-cons 'explicit-loading? #t result)))) (filter-map (lambda (opt) (and (not (any (lambda (name) (member name to-remove)) @@ -122,10 +129,60 @@ (define (handle-argument arg result) (let ((args command (break (cut string=? "--" <>) args))) (let ((opts (parse-command-line args %options (list %default-options) #:argument-handler handle-argument))) - (match command - (() opts) - (("--") opts) - (("--" command ...) (alist-cons 'exec command opts)))))) + (auto-detect-manifest + (match command + (() opts) + (("--") opts) + (("--" command ...) (alist-cons 'exec command opts))))))) + +(define (find-file-in-parent-directories candidates) + "Find one of CANDIDATES in the current directory or one of its ancestors." + (define start (getcwd)) + (define device (stat:dev (stat start))) + + (let loop ((directory start)) + (let ((stat (stat directory))) + (and (= (stat:uid stat) (getuid)) + (= (stat:dev stat) device) + (or (any (lambda (candidate) + (let ((candidate (string-append directory "/" candidate))) + (and (file-exists? candidate) candidate))) + candidates) + (and (not (string=? directory "/")) + (loop (dirname directory)))))))) ;lexical ".." resolution + +(define (auto-detect-manifest opts) + "If OPTS do not specify packages or a manifest, load a \"guix.scm\" or +\"manifest.scm\" file from the current directory or one of its ancestors. +Return the modified OPTS." + (define (options-contain-payload? opts) + (match opts + (() #f) + ((('package . _) . _) #t) + ((('load . _) . _) #t) + ((('manifest . _) . _) #t) + ((('expression . _) . _) #t) + ((_ . rest) (options-contain-payload? rest)))) + + (define interactive? + (not (assoc-ref opts 'exec))) + + (define disallow-implicit-load? + (assoc-ref opts 'explicit-loading?)) + + (if (or (not interactive?) + disallow-implicit-load? + (options-contain-payload? opts)) + opts + (match (find-file-in-parent-directories '("guix.scm" "manifest.scm")) + (#f + (warning (G_ "no packages specified; creating an empty environment~%")) + opts) + (file + (info (G_ "loading environment from '~a'...~%") file) + (match (basename file) + ("guix.scm" (alist-cons 'load `(package ,file) opts)) + ("manifest.scm" (alist-cons 'manifest file opts))))))) (define-command (guix-shell . args) diff --git a/tests/guix-shell.sh b/tests/guix-shell.sh index f08637f7ff..0988ca0a75 100644 --- a/tests/guix-shell.sh +++ b/tests/guix-shell.sh @@ -31,6 +31,36 @@ guix shell --bootstrap --pure guile-bootstrap -- guile --version # '--ad-hoc' is a thing of the past. ! guix shell --ad-hoc guile-bootstrap +# Ignoring 'manifest.scm' and 'guix.scm' in non-interactive use. +cat > "$tmpdir/guix.scm" < "$tmpdir/manifest.scm" <manifest '("guile-bootstrap")) +EOF +cat > "$tmpdir/fake-shell.sh" < "$tmpdir/manifest.scm" +(cd "$tmpdir"; SHELL="$(realpath fake-shell.sh)" guix shell --bootstrap -q) +rm "$tmpdir/manifest.scm" + if guile -c '(getaddrinfo "www.gnu.org" "80" AI_NUMERICSERV)' 2> /dev/null then # Compute the build environment for the initial GNU Make. @@ -51,4 +81,10 @@ then # 'make-boot0' itself must not be listed. ! guix gc --references "$profile" | grep make-boot0 + + # Honoring the local 'guix.scm' file. + echo '(@ (guix tests) gnu-make-for-tests)' > "$tmpdir/guix.scm" + (cd "$tmpdir"; guix shell --bootstrap --search-paths --pure > "b") + cmp "$tmpdir/a" "$tmpdir/b" + rm "$tmpdir/guix.scm" fi From patchwork Mon Oct 11 21:38:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Ludovic_Court=C3=A8s?= X-Patchwork-Id: 33779 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 0703727BBE3; Mon, 11 Oct 2021 22:44:38 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 9B49227BBE1 for ; Mon, 11 Oct 2021 22:44:37 +0100 (BST) Received: from localhost ([::1]:42104 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ma364-0002xL-N1 for patchwork@mira.cbaines.net; Mon, 11 Oct 2021 17:44:36 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47520) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ma30h-0006b5-Fc for guix-patches@gnu.org; Mon, 11 Oct 2021 17:39:04 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:49611) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1ma30h-0004qq-6I for guix-patches@gnu.org; Mon, 11 Oct 2021 17:39:03 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1ma30h-0001Pi-4B for guix-patches@gnu.org; Mon, 11 Oct 2021 17:39:03 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#50960] [PATCH v2 05/11] DRAFT shell: Honor in ~/.config/guix/shell-authorized-directories. Resent-From: Ludovic =?utf-8?q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Mon, 11 Oct 2021 21:39:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 50960 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 50960@debbugs.gnu.org Cc: Ludovic =?utf-8?q?Court=C3=A8s?= Received: via spool by 50960-submit@debbugs.gnu.org id=B50960.16339883185327 (code B ref 50960); Mon, 11 Oct 2021 21:39:03 +0000 Received: (at 50960) by debbugs.gnu.org; 11 Oct 2021 21:38:38 +0000 Received: from localhost ([127.0.0.1]:32906 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ma30H-0001Nf-LL for submit@debbugs.gnu.org; Mon, 11 Oct 2021 17:38:38 -0400 Received: from eggs.gnu.org ([209.51.188.92]:45468) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ma30A-0001M4-1Q for 50960@debbugs.gnu.org; Mon, 11 Oct 2021 17:38:31 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:40572) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ma304-0004X9-QM; Mon, 11 Oct 2021 17:38:24 -0400 Received: from 91-160-117-201.subs.proxad.net ([91.160.117.201]:53321 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1ma304-0007Y0-9D; Mon, 11 Oct 2021 17:38:24 -0400 From: Ludovic =?utf-8?q?Court=C3=A8s?= Date: Mon, 11 Oct 2021 23:38:03 +0200 Message-Id: <20211011213809.17482-6-ludo@gnu.org> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20211011213809.17482-1-ludo@gnu.org> References: <20211002102116.27726-1-ludo@gnu.org> <20211011213809.17482-1-ludo@gnu.org> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches DRAFT: Squeeze with previous commit, or instead implement "guix shell ." convention? * guix/scripts/shell.scm (authorized-directory-file) (authorized-shell-directory?): New procedure. (auto-detect-manifest): Use it. * doc/guix.texi (Invoking guix shell): Document it. --- doc/guix.texi | 14 ++++++++++ guix/scripts/shell.scm | 60 +++++++++++++++++++++++++++++++++++++++--- tests/guix-shell.sh | 16 ++++++++--- 3 files changed, 83 insertions(+), 7 deletions(-) diff --git a/doc/guix.texi b/doc/guix.texi index b0d745b9e3..b95025a39f 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -5620,6 +5620,20 @@ before @command{guix shell} was invoked. The next garbage collection (@pxref{Invoking guix gc}) may clean up packages that were installed in the environment and that are no longer used outside of it. +As an added convenience, when running from a directory that contains a +@file{guix.scm} or a @file{manifest.scm} file, possibly in a parent +directory, @command{guix shell} automatically loads the file---provided +the directory is listed in +@file{~/.config/guix/shell-authorized-directories}, and only for +interactive use: + +@example +guix shell +@end example + +This provides an easy way to define, share, and enter development +environments. + By default, the shell session or command runs in an @emph{augmented} environment, where the new packages are added to search path environment variables such as @code{PATH}. You can, instead, choose to create an diff --git a/guix/scripts/shell.scm b/guix/scripts/shell.scm index 39d843bde7..45fd536145 100644 --- a/guix/scripts/shell.scm +++ b/guix/scripts/shell.scm @@ -18,6 +18,7 @@ (define-module (guix scripts shell) #:use-module (guix ui) + #:use-module ((guix diagnostics) #:select (location)) #:use-module (guix scripts environment) #:autoload (guix scripts build) (show-build-options-help) #:autoload (guix transformations) (show-transformation-options-help) @@ -29,6 +30,8 @@ (define-module (guix scripts shell) #:use-module (srfi srfi-37) #:use-module (srfi srfi-71) #:use-module (ice-9 match) + #:autoload (ice-9 rdelim) (read-line) + #:autoload (guix utils) (config-directory) #:export (guix-shell)) (define (show-help) @@ -151,6 +154,39 @@ (define device (stat:dev (stat start))) (and (not (string=? directory "/")) (loop (dirname directory)))))))) ;lexical ".." resolution +(define (authorized-directory-file) + "Return the name of the file listing directories for which 'guix shell' may +automatically load 'guix.scm' or 'manifest.scm' files." + (string-append (config-directory) "/shell-authorized-directories")) + +(define (authorized-shell-directory? directory) + "Return true if DIRECTORY is among the authorized directories for automatic +loading. The list of authorized directories is read from +'authorized-directory-file'; each line must be either: an absolute file name, +a hash-prefixed comment, or a blank line." + (catch 'system-error + (lambda () + (call-with-input-file (authorized-directory-file) + (lambda (port) + (let loop () + (match (read-line port) + ((? eof-object?) #f) + ((= string-trim line) + (cond ((string-prefix? "#" line) ;comment + (loop)) + ((string-prefix? "/" line) ;absolute file name + (or (string=? line directory) + (loop))) + ((string-null? (string-trim-right line)) ;blank line + (loop)) + (else ;bogus line + (let ((loc (location (port-filename port) + (port-line port) + (port-column port)))) + (warning loc (G_ "ignoring invalid file name: '~a'~%") + line)))))))))) + (const #f))) + (define (auto-detect-manifest opts) "If OPTS do not specify packages or a manifest, load a \"guix.scm\" or \"manifest.scm\" file from the current directory or one of its ancestors. @@ -179,10 +215,26 @@ (define disallow-implicit-load? (warning (G_ "no packages specified; creating an empty environment~%")) opts) (file - (info (G_ "loading environment from '~a'...~%") file) - (match (basename file) - ("guix.scm" (alist-cons 'load `(package ,file) opts)) - ("manifest.scm" (alist-cons 'manifest file opts))))))) + (if (authorized-shell-directory? (dirname file)) + (begin + (info (G_ "loading environment from '~a'...~%") file) + (match (basename file) + ("guix.scm" (alist-cons 'load `(package ,file) opts)) + ("manifest.scm" (alist-cons 'manifest file opts)))) + (begin + (warning (G_ "not loading '~a' because not authorized to do so~%") + file) + (display-hint (format #f (G_ "To allow automatic loading of +@file{~a} when running @command{guix shell}, you must explicitly authorize its +directory, like so: + +@example +echo ~a >> ~a +@end example\n") + file + (dirname file) + (authorized-directory-file))) + opts)))))) (define-command (guix-shell . args) diff --git a/tests/guix-shell.sh b/tests/guix-shell.sh index 0988ca0a75..95725cba2d 100644 --- a/tests/guix-shell.sh +++ b/tests/guix-shell.sh @@ -22,19 +22,29 @@ guix shell --version +configdir="t-guix-shell-config-$$" tmpdir="t-guix-shell-$$" -trap 'rm -r "$tmpdir"' EXIT -mkdir "$tmpdir" +trap 'rm -r "$tmpdir" "$configdir"' EXIT +mkdir "$tmpdir" "$configdir" "$configdir/guix" + +XDG_CONFIG_HOME="$(realpath $configdir)" +export XDG_CONFIG_HOME guix shell --bootstrap --pure guile-bootstrap -- guile --version # '--ad-hoc' is a thing of the past. ! guix shell --ad-hoc guile-bootstrap -# Ignoring 'manifest.scm' and 'guix.scm' in non-interactive use. +# Ignoring unauthorized files. cat > "$tmpdir/guix.scm" < "$configdir/guix/shell-authorized-directories" + +# Ignoring 'manifest.scm' and 'guix.scm' in non-interactive use. (cd "$tmpdir"; guix shell --bootstrap -- true) mv "$tmpdir/guix.scm" "$tmpdir/manifest.scm" (cd "$tmpdir"; guix shell --bootstrap -- true) From patchwork Mon Oct 11 21:38:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Ludovic_Court=C3=A8s?= X-Patchwork-Id: 33770 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id C546727BBE3; Mon, 11 Oct 2021 22:39:38 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 984ED27BBE1 for ; Mon, 11 Oct 2021 22:39:38 +0100 (BST) Received: from localhost ([::1]:36190 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ma31F-0007Eh-Mp for patchwork@mira.cbaines.net; Mon, 11 Oct 2021 17:39:37 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47522) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ma30h-0006b7-QK for guix-patches@gnu.org; Mon, 11 Oct 2021 17:39:04 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:49612) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1ma30h-0004qz-IZ for guix-patches@gnu.org; Mon, 11 Oct 2021 17:39:03 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1ma30h-0001Pp-G5 for guix-patches@gnu.org; Mon, 11 Oct 2021 17:39:03 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#50960] [PATCH v2 06/11] environment: Add tests for '--profile'. Resent-From: Ludovic =?utf-8?q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Mon, 11 Oct 2021 21:39:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 50960 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 50960@debbugs.gnu.org Cc: Ludovic =?utf-8?q?Court=C3=A8s?= Received: via spool by 50960-submit@debbugs.gnu.org id=B50960.16339883265342 (code B ref 50960); Mon, 11 Oct 2021 21:39:03 +0000 Received: (at 50960) by debbugs.gnu.org; 11 Oct 2021 21:38:46 +0000 Received: from localhost ([127.0.0.1]:32908 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ma30Q-0001O5-EA for submit@debbugs.gnu.org; Mon, 11 Oct 2021 17:38:46 -0400 Received: from eggs.gnu.org ([209.51.188.92]:45486) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ma30E-0001MQ-4X for 50960@debbugs.gnu.org; Mon, 11 Oct 2021 17:38:34 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:40574) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ma308-0004Yg-Rg; Mon, 11 Oct 2021 17:38:28 -0400 Received: from 91-160-117-201.subs.proxad.net ([91.160.117.201]:53321 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1ma305-0007Y0-47; Mon, 11 Oct 2021 17:38:28 -0400 From: Ludovic =?utf-8?q?Court=C3=A8s?= Date: Mon, 11 Oct 2021 23:38:04 +0200 Message-Id: <20211011213809.17482-7-ludo@gnu.org> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20211011213809.17482-1-ludo@gnu.org> References: <20211002102116.27726-1-ludo@gnu.org> <20211011213809.17482-1-ludo@gnu.org> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches This is a followup to a643deac2de81755a1843a3b41dd53857678bebc. * tests/guix-environment-container.sh, tests/guix-environment.sh: Add tests for '--profile'. --- tests/guix-environment-container.sh | 8 ++++++++ tests/guix-environment.sh | 7 +++++++ 2 files changed, 15 insertions(+) diff --git a/tests/guix-environment-container.sh b/tests/guix-environment-container.sh index f2d15c8d0c..2e238c501d 100644 --- a/tests/guix-environment-container.sh +++ b/tests/guix-environment-container.sh @@ -44,6 +44,14 @@ else test $? = 42 fi +# Try '--root' and '--profile'. +root="$tmpdir/root" +guix environment -C --ad-hoc --bootstrap guile-bootstrap -r "$root" -- guile --version +guix environment -C -p "$root" --bootstrap -- guile --version +path1=$(guix environment -C -p "$root" --bootstrap -- guile -c '(display (getenv "PATH"))') +path2=$(guix environment -C --ad-hoc --bootstrap guile-bootstrap -- guile -c '(display (getenv "PATH"))') +test "$path1" = "$path2" + # Make sure "localhost" resolves. guix environment --container --ad-hoc --bootstrap guile-bootstrap \ -- guile -c '(exit (pair? (getaddrinfo "localhost" "80")))' diff --git a/tests/guix-environment.sh b/tests/guix-environment.sh index afadcbe195..f4fc2e39ed 100644 --- a/tests/guix-environment.sh +++ b/tests/guix-environment.sh @@ -119,6 +119,13 @@ test `readlink "$gcroot"` = "$expected" guix environment --bootstrap -r "$gcroot" --ad-hoc guile-bootstrap \ -- guile -c 1 test `readlink "$gcroot"` = "$expected" + +# Make sure '-p' works as expected. +test $(guix environment -p "$gcroot" -- "$SHELL" -c 'echo $GUIX_ENVIRONMENT') = "$expected" +paths1="$(guix environment -p "$gcroot" --search-paths)" +paths2="$(guix environment --bootstrap --ad-hoc guile-bootstrap --search-paths)" +test "$paths1" = "$paths2" + rm "$gcroot" # Try '-r' with a relative file name. From patchwork Mon Oct 11 21:38:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Ludovic_Court=C3=A8s?= X-Patchwork-Id: 33774 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id A9D2327BBE3; Mon, 11 Oct 2021 22:43:34 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_MSPIKE_H2,SPF_HELO_PASS autolearn=unavailable autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 71E8427BBE1 for ; Mon, 11 Oct 2021 22:43:34 +0100 (BST) Received: from localhost ([::1]:40380 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ma353-0001lQ-E6 for patchwork@mira.cbaines.net; Mon, 11 Oct 2021 17:43:33 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47524) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ma30i-0006bB-7h for guix-patches@gnu.org; Mon, 11 Oct 2021 17:39:04 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:49613) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1ma30h-0004r4-VG for guix-patches@gnu.org; Mon, 11 Oct 2021 17:39:03 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1ma30h-0001Px-SL for guix-patches@gnu.org; Mon, 11 Oct 2021 17:39:03 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#50960] [PATCH v2 07/11] environment: Skip derivation computation when '--profile' is used. Resent-From: Ludovic =?utf-8?q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Mon, 11 Oct 2021 21:39:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 50960 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 50960@debbugs.gnu.org Cc: Ludovic =?utf-8?q?Court=C3=A8s?= Received: via spool by 50960-submit@debbugs.gnu.org id=B50960.16339883275350 (code B ref 50960); Mon, 11 Oct 2021 21:39:03 +0000 Received: (at 50960) by debbugs.gnu.org; 11 Oct 2021 21:38:47 +0000 Received: from localhost ([127.0.0.1]:32910 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ma30Q-0001O7-Pe for submit@debbugs.gnu.org; Mon, 11 Oct 2021 17:38:47 -0400 Received: from eggs.gnu.org ([209.51.188.92]:45494) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ma30E-0001MY-Np for 50960@debbugs.gnu.org; Mon, 11 Oct 2021 17:38:34 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:40576) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ma309-0004ZS-GI; Mon, 11 Oct 2021 17:38:29 -0400 Received: from 91-160-117-201.subs.proxad.net ([91.160.117.201]:53321 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1ma309-0007Y0-6p; Mon, 11 Oct 2021 17:38:29 -0400 From: Ludovic =?utf-8?q?Court=C3=A8s?= Date: Mon, 11 Oct 2021 23:38:05 +0200 Message-Id: <20211011213809.17482-8-ludo@gnu.org> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20211011213809.17482-1-ludo@gnu.org> References: <20211002102116.27726-1-ludo@gnu.org> <20211011213809.17482-1-ludo@gnu.org> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches * guix/scripts/environment.scm (guix-environment*): Bypass calls to 'package-derivation' and to 'manifest->derivation' when PROFILE is true. --- guix/scripts/environment.scm | 23 +++++++++++++---------- 1 file changed, 13 insertions(+), 10 deletions(-) diff --git a/guix/scripts/environment.scm b/guix/scripts/environment.scm index 77956fc018..32f376fdd2 100644 --- a/guix/scripts/environment.scm +++ b/guix/scripts/environment.scm @@ -729,18 +729,21 @@ (define manifest ;; Use the bootstrap Guile when requested. (parameterize ((%graft? (assoc-ref opts 'graft?)) (%guile-for-build - (package-derivation - store - (if bootstrap? - %bootstrap-guile - (default-guile))))) + (and (or container? (not profile)) + (package-derivation + store + (if bootstrap? + %bootstrap-guile + (default-guile)))))) (run-with-store store ;; Containers need a Bourne shell at /bin/sh. (mlet* %store-monad ((bash (environment-bash container? bootstrap? system)) - (prof-drv (manifest->derivation - manifest system bootstrap?)) + (prof-drv (if profile + (return #f) + (manifest->derivation + manifest system bootstrap?))) (profile -> (if profile (readlink* profile) (derivation->output-path prof-drv))) @@ -750,9 +753,9 @@ (define manifest ;; --search-paths. Additionally, we might need to build bash for ;; a container. (mbegin %store-monad - (built-derivations (if (derivation? bash) - (list prof-drv bash) - (list prof-drv))) + (built-derivations (append + (if prof-drv (list prof-drv) '()) + (if (derivation? bash) (list bash) '()))) (mwhen gc-root (register-gc-root profile gc-root)) From patchwork Mon Oct 11 21:38:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Ludovic_Court=C3=A8s?= X-Patchwork-Id: 33772 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 7DD1E27BBE3; Mon, 11 Oct 2021 22:40:22 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_MSPIKE_H2,SPF_HELO_PASS autolearn=unavailable autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 1CBA927BBE1 for ; Mon, 11 Oct 2021 22:40:22 +0100 (BST) Received: from localhost ([::1]:37050 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ma31w-0007oV-Tu for patchwork@mira.cbaines.net; Mon, 11 Oct 2021 17:40:21 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47528) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ma30j-0006bm-0e for guix-patches@gnu.org; Mon, 11 Oct 2021 17:39:05 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:49615) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1ma30i-0004rY-NQ for guix-patches@gnu.org; Mon, 11 Oct 2021 17:39:04 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1ma30i-0001QB-Ke for guix-patches@gnu.org; Mon, 11 Oct 2021 17:39:04 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#50960] [PATCH v2 08/11] environment: Do not connect to the daemon when '--profile' is used. Resent-From: Ludovic =?utf-8?q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Mon, 11 Oct 2021 21:39:04 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 50960 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 50960@debbugs.gnu.org Cc: Ludovic =?utf-8?q?Court=C3=A8s?= Received: via spool by 50960-submit@debbugs.gnu.org id=B50960.16339883335370 (code B ref 50960); Mon, 11 Oct 2021 21:39:04 +0000 Received: (at 50960) by debbugs.gnu.org; 11 Oct 2021 21:38:53 +0000 Received: from localhost ([127.0.0.1]:32914 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ma30W-0001OR-Br for submit@debbugs.gnu.org; Mon, 11 Oct 2021 17:38:52 -0400 Received: from eggs.gnu.org ([209.51.188.92]:45504) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ma30F-0001Mf-Dq for 50960@debbugs.gnu.org; Mon, 11 Oct 2021 17:38:35 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:40578) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ma30A-0004Zg-56; Mon, 11 Oct 2021 17:38:30 -0400 Received: from 91-160-117-201.subs.proxad.net ([91.160.117.201]:53321 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1ma309-0007Y0-Rr; Mon, 11 Oct 2021 17:38:30 -0400 From: Ludovic =?utf-8?q?Court=C3=A8s?= Date: Mon, 11 Oct 2021 23:38:06 +0200 Message-Id: <20211011213809.17482-9-ludo@gnu.org> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20211011213809.17482-1-ludo@gnu.org> References: <20211002102116.27726-1-ludo@gnu.org> <20211011213809.17482-1-ludo@gnu.org> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches This further speeds up the 'guix environment -p PROFILE' case. * guix/scripts/environment.scm (guix-environment*)[store-needed?]: New variable. [with-store/maybe]: New macro. Use it instead of 'with-store', and remove 'with-build-handler' form. --- guix/scripts/environment.scm | 169 +++++++++++++++++++---------------- 1 file changed, 93 insertions(+), 76 deletions(-) diff --git a/guix/scripts/environment.scm b/guix/scripts/environment.scm index 32f376fdd2..e23d52df39 100644 --- a/guix/scripts/environment.scm +++ b/guix/scripts/environment.scm @@ -691,6 +691,26 @@ (define (guix-environment* opts) (mappings (pick-all opts 'file-system-mapping)) (white-list (pick-all opts 'inherit-regexp))) + (define store-needed? + ;; Whether connecting to the daemon is needed. + (or container? (not profile))) + + (define-syntax-rule (with-store/maybe store exp ...) + ;; Evaluate EXP... with STORE bound to a connection, unless + ;; STORE-NEEDED? is false, in which case STORE is bound to #f. + (let ((proc (lambda (store) exp ...))) + (if store-needed? + (with-store s + (set-build-options-from-command-line s opts) + (with-build-handler (build-notifier #:use-substitutes? + (assoc-ref opts 'substitutes?) + #:verbosity + (assoc-ref opts 'verbosity) + #:dry-run? + (assoc-ref opts 'dry-run?)) + (proc s))) + (proc #f)))) + (when container? (assert-container-features)) (when (and (not container?) link-prof?) @@ -701,88 +721,85 @@ (define (guix-environment* opts) (leave (G_ "--no-cwd cannot be used without --container~%"))) - (with-store store - (with-build-handler (build-notifier #:use-substitutes? - (assoc-ref opts 'substitutes?) - #:verbosity - (assoc-ref opts 'verbosity) - #:dry-run? - (assoc-ref opts 'dry-run?)) - (with-status-verbosity (assoc-ref opts 'verbosity) - (define manifest-from-opts - (options/resolve-packages store opts)) + (with-store/maybe store + (with-status-verbosity (assoc-ref opts 'verbosity) + (define manifest-from-opts + (options/resolve-packages store opts)) - (define manifest - (if profile - (profile-manifest profile) - manifest-from-opts)) + (define manifest + (if profile + (profile-manifest profile) + manifest-from-opts)) - (when (and profile - (> (length (manifest-entries manifest-from-opts)) 0)) - (leave (G_ "'--profile' cannot be used with package options~%"))) + (when (and profile + (> (length (manifest-entries manifest-from-opts)) 0)) + (leave (G_ "'--profile' cannot be used with package options~%"))) - (when (null? (manifest-entries manifest)) - (warning (G_ "no packages specified; creating an empty environment~%"))) + (when (null? (manifest-entries manifest)) + (warning (G_ "no packages specified; creating an empty environment~%"))) - (set-build-options-from-command-line store opts) + ;; Use the bootstrap Guile when requested. + (parameterize ((%graft? (assoc-ref opts 'graft?)) + (%guile-for-build + (and store-needed? + (package-derivation + store + (if bootstrap? + %bootstrap-guile + (default-guile)))))) + (run-with-store store + ;; Containers need a Bourne shell at /bin/sh. + (mlet* %store-monad ((bash (environment-bash container? + bootstrap? + system)) + (prof-drv (if profile + (return #f) + (manifest->derivation + manifest system bootstrap?))) + (profile -> (if profile + (readlink* profile) + (derivation->output-path prof-drv))) + (gc-root -> (assoc-ref opts 'gc-root))) - ;; Use the bootstrap Guile when requested. - (parameterize ((%graft? (assoc-ref opts 'graft?)) - (%guile-for-build - (and (or container? (not profile)) - (package-derivation - store - (if bootstrap? - %bootstrap-guile - (default-guile)))))) - (run-with-store store - ;; Containers need a Bourne shell at /bin/sh. - (mlet* %store-monad ((bash (environment-bash container? - bootstrap? - system)) - (prof-drv (if profile - (return #f) - (manifest->derivation - manifest system bootstrap?))) - (profile -> (if profile - (readlink* profile) - (derivation->output-path prof-drv))) - (gc-root -> (assoc-ref opts 'gc-root))) - - ;; First build the inputs. This is necessary even for - ;; --search-paths. Additionally, we might need to build bash for - ;; a container. - (mbegin %store-monad + ;; First build the inputs. This is necessary even for + ;; --search-paths. Additionally, we might need to build bash for + ;; a container. + (mbegin %store-monad + (mwhen store-needed? (built-derivations (append (if prof-drv (list prof-drv) '()) - (if (derivation? bash) (list bash) '()))) - (mwhen gc-root - (register-gc-root profile gc-root)) + (if (derivation? bash) (list bash) '())))) + (mwhen gc-root + (register-gc-root profile gc-root)) - (cond - ((assoc-ref opts 'search-paths) - (show-search-paths profile manifest #:pure? pure?) - (return #t)) - (container? - (let ((bash-binary - (if bootstrap? - (derivation->output-path bash) - (string-append (derivation->output-path bash) - "/bin/sh")))) - (launch-environment/container #:command command - #:bash bash-binary - #:user user - #:user-mappings mappings - #:profile profile - #:manifest manifest - #:white-list white-list - #:link-profile? link-prof? - #:network? network? - #:map-cwd? (not no-cwd?)))) + (cond + ((assoc-ref opts 'search-paths) + (show-search-paths profile manifest #:pure? pure?) + (return #t)) + (container? + (let ((bash-binary + (if bootstrap? + (derivation->output-path bash) + (string-append (derivation->output-path bash) + "/bin/sh")))) + (launch-environment/container #:command command + #:bash bash-binary + #:user user + #:user-mappings mappings + #:profile profile + #:manifest manifest + #:white-list white-list + #:link-profile? link-prof? + #:network? network? + #:map-cwd? (not no-cwd?)))) - (else - (return - (exit/status - (launch-environment/fork command profile manifest - #:white-list white-list - #:pure? pure?))))))))))))))) + (else + (return + (exit/status + (launch-environment/fork command profile manifest + #:white-list white-list + #:pure? pure?)))))))))))))) + +;;; Local Variables: +;;; (put 'with-store/maybe 'scheme-indent-function 1) +;;; End: From patchwork Mon Oct 11 21:38:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Ludovic_Court=C3=A8s?= X-Patchwork-Id: 33775 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 057B627BBE3; Mon, 11 Oct 2021 22:44:17 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_MSPIKE_H2,SPF_HELO_PASS autolearn=unavailable autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id D036227BBE1 for ; Mon, 11 Oct 2021 22:44:16 +0100 (BST) Received: from localhost ([::1]:40920 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ma35j-00028F-St for patchwork@mira.cbaines.net; Mon, 11 Oct 2021 17:44:15 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47530) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ma30j-0006bz-B4 for guix-patches@gnu.org; Mon, 11 Oct 2021 17:39:05 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:49616) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1ma30j-0004rv-2y for guix-patches@gnu.org; Mon, 11 Oct 2021 17:39:05 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1ma30j-0001QJ-0g for guix-patches@gnu.org; Mon, 11 Oct 2021 17:39:05 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#50960] [PATCH v2 09/11] environment: Autoload some modules. Resent-From: Ludovic =?utf-8?q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Mon, 11 Oct 2021 21:39:04 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 50960 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 50960@debbugs.gnu.org Cc: Ludovic =?utf-8?q?Court=C3=A8s?= Received: via spool by 50960-submit@debbugs.gnu.org id=B50960.16339883335377 (code B ref 50960); Mon, 11 Oct 2021 21:39:04 +0000 Received: (at 50960) by debbugs.gnu.org; 11 Oct 2021 21:38:53 +0000 Received: from localhost ([127.0.0.1]:32916 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ma30W-0001OZ-SI for submit@debbugs.gnu.org; Mon, 11 Oct 2021 17:38:53 -0400 Received: from eggs.gnu.org ([209.51.188.92]:45510) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ma30G-0001Mg-32 for 50960@debbugs.gnu.org; Mon, 11 Oct 2021 17:38:36 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:40580) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ma30A-0004aC-Pe; Mon, 11 Oct 2021 17:38:30 -0400 Received: from 91-160-117-201.subs.proxad.net ([91.160.117.201]:53321 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1ma30A-0007Y0-GT; Mon, 11 Oct 2021 17:38:30 -0400 From: Ludovic =?utf-8?q?Court=C3=A8s?= Date: Mon, 11 Oct 2021 23:38:07 +0200 Message-Id: <20211011213809.17482-10-ludo@gnu.org> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20211011213809.17482-1-ludo@gnu.org> References: <20211002102116.27726-1-ludo@gnu.org> <20211011213809.17482-1-ludo@gnu.org> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches This further speeds up the 'guix environment -p PROFILE' case. * guix/scripts/environment.scm: Autoload a bunch of modules. --- guix/scripts/environment.scm | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-) diff --git a/guix/scripts/environment.scm b/guix/scripts/environment.scm index e23d52df39..05a43659da 100644 --- a/guix/scripts/environment.scm +++ b/guix/scripts/environment.scm @@ -34,15 +34,18 @@ (define-module (guix scripts environment) #:use-module (guix scripts) #:use-module (guix scripts build) #:use-module (guix transformations) - #:use-module (gnu build linux-container) - #:use-module (gnu build accounts) - #:use-module ((guix build syscalls) #:select (set-network-interface-up)) - #:use-module (gnu system linux-container) + #:autoload (gnu build linux-container) (call-with-container %namespaces + user-namespace-supported? + unprivileged-user-namespace-supported? + setgroups-supported?) + #:autoload (gnu build accounts) (password-entry group-entry + password-entry-name password-entry-directory + write-passwd write-group) + #:autoload (guix build syscalls) (set-network-interface-up) #:use-module (gnu system file-systems) - #:use-module (gnu packages) - #:use-module (gnu packages bash) - #:use-module ((gnu packages bootstrap) - #:select (bootstrap-executable %bootstrap-guile)) + #:autoload (gnu packages) (specification->package+output) + #:autoload (gnu packages bash) (bash) + #:autoload (gnu packages bootstrap) (bootstrap-executable %bootstrap-guile) #:use-module (ice-9 match) #:use-module (srfi srfi-1) #:use-module (srfi srfi-11) From patchwork Mon Oct 11 21:38:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Ludovic_Court=C3=A8s?= X-Patchwork-Id: 33769 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id CD01627BBE3; Mon, 11 Oct 2021 22:39:18 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_MSPIKE_H2,SPF_HELO_PASS autolearn=unavailable autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 8FE5027BBE1 for ; Mon, 11 Oct 2021 22:39:18 +0100 (BST) Received: from localhost ([::1]:35422 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ma30v-0006hE-GP for patchwork@mira.cbaines.net; Mon, 11 Oct 2021 17:39:17 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47532) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ma30j-0006c0-NK for guix-patches@gnu.org; Mon, 11 Oct 2021 17:39:06 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:49617) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1ma30j-0004s0-Eo for guix-patches@gnu.org; Mon, 11 Oct 2021 17:39:05 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1ma30j-0001QQ-Ch for guix-patches@gnu.org; Mon, 11 Oct 2021 17:39:05 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#50960] [PATCH v2 10/11] cache: Gracefully handle non-existent cache. Resent-From: Ludovic =?utf-8?q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Mon, 11 Oct 2021 21:39:05 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 50960 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 50960@debbugs.gnu.org Cc: Ludovic =?utf-8?q?Court=C3=A8s?= Received: via spool by 50960-submit@debbugs.gnu.org id=B50960.16339883335384 (code B ref 50960); Mon, 11 Oct 2021 21:39:05 +0000 Received: (at 50960) by debbugs.gnu.org; 11 Oct 2021 21:38:53 +0000 Received: from localhost ([127.0.0.1]:32918 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ma30X-0001Og-5v for submit@debbugs.gnu.org; Mon, 11 Oct 2021 17:38:53 -0400 Received: from eggs.gnu.org ([209.51.188.92]:45516) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ma30G-0001Mn-N5 for 50960@debbugs.gnu.org; Mon, 11 Oct 2021 17:38:36 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:40584) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ma30B-0004ag-EC; Mon, 11 Oct 2021 17:38:31 -0400 Received: from 91-160-117-201.subs.proxad.net ([91.160.117.201]:53321 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1ma30B-0007Y0-4u; Mon, 11 Oct 2021 17:38:31 -0400 From: Ludovic =?utf-8?q?Court=C3=A8s?= Date: Mon, 11 Oct 2021 23:38:08 +0200 Message-Id: <20211011213809.17482-11-ludo@gnu.org> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20211011213809.17482-1-ludo@gnu.org> References: <20211002102116.27726-1-ludo@gnu.org> <20211011213809.17482-1-ludo@gnu.org> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches * guix/cache.scm (maybe-remove-expired-cache-entries): Ignore ENOENT when writing EXPIRY-FILE. --- guix/cache.scm | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/guix/cache.scm b/guix/cache.scm index 0401a9d428..51009809bd 100644 --- a/guix/cache.scm +++ b/guix/cache.scm @@ -101,7 +101,13 @@ (define last-expiry-date #:now now #:entry-expiration entry-expiration #:delete-entry delete-entry) - (call-with-output-file expiry-file - (cute write (time-second now) <>)))) + (catch 'system-error + (lambda () + (call-with-output-file expiry-file + (cute write (time-second now) <>))) + (lambda args + ;; ENOENT means CACHE does not exist. + (unless (= ENOENT (system-error-errno args)) + (apply throw args)))))) ;;; cache.scm ends here From patchwork Mon Oct 11 21:38:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Ludovic_Court=C3=A8s?= X-Patchwork-Id: 33777 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 528C427BBE4; Mon, 11 Oct 2021 22:44:26 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id D8F3827BBE1 for ; Mon, 11 Oct 2021 22:44:25 +0100 (BST) Received: from localhost ([::1]:41440 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ma35s-0002Vj-Vx for patchwork@mira.cbaines.net; Mon, 11 Oct 2021 17:44:25 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47534) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ma30k-0006c9-3b for guix-patches@gnu.org; Mon, 11 Oct 2021 17:39:06 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:49618) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1ma30j-0004sM-Re for guix-patches@gnu.org; Mon, 11 Oct 2021 17:39:05 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1ma30j-0001QX-PB for guix-patches@gnu.org; Mon, 11 Oct 2021 17:39:05 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#50960] [PATCH v2 11/11] shell: Maintain a profile cache. Resent-From: Ludovic =?utf-8?q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Mon, 11 Oct 2021 21:39:05 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 50960 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 50960@debbugs.gnu.org Cc: Ludovic =?utf-8?q?Court=C3=A8s?= Received: via spool by 50960-submit@debbugs.gnu.org id=B50960.16339883345391 (code B ref 50960); Mon, 11 Oct 2021 21:39:05 +0000 Received: (at 50960) by debbugs.gnu.org; 11 Oct 2021 21:38:54 +0000 Received: from localhost ([127.0.0.1]:32920 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ma30X-0001On-EU for submit@debbugs.gnu.org; Mon, 11 Oct 2021 17:38:53 -0400 Received: from eggs.gnu.org ([209.51.188.92]:45520) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ma30H-0001Mo-As for 50960@debbugs.gnu.org; Mon, 11 Oct 2021 17:38:38 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:40586) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ma30C-0004ax-31; Mon, 11 Oct 2021 17:38:32 -0400 Received: from 91-160-117-201.subs.proxad.net ([91.160.117.201]:53321 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1ma30B-0007Y0-Py; Mon, 11 Oct 2021 17:38:32 -0400 From: Ludovic =?utf-8?q?Court=C3=A8s?= Date: Mon, 11 Oct 2021 23:38:09 +0200 Message-Id: <20211011213809.17482-12-ludo@gnu.org> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20211011213809.17482-1-ludo@gnu.org> References: <20211002102116.27726-1-ludo@gnu.org> <20211011213809.17482-1-ludo@gnu.org> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches shell: Maintain a profile cache. With this change, running "guix shell" (no arguments) is equivalent to: guix environment -r ~/.cache/guix/profiles/some-root -l guix.scm This is the cache miss. On cache hit, it's equivalent to: guix environment -p ~/.cache/guix/profiles/some-root ... which can run in 0.1s. * guix/scripts/shell.scm (options-with-caching): New procedure. (parse-args): Use it. (%profile-cache-directory): New variable. (profile-cache-key, profile-cached-gc-root): New procedures. (show-help, %options): Add '--rebuild-cache'. (guix-shell)[cache-entries, entry-expiration]: New procedures. Add call to 'maybe-remove-expired-cache-entries'. * doc/guix.texi (Invoking guix shell): Document '--rebuild-cache'. --- doc/guix.texi | 11 ++++ guix/scripts/shell.scm | 127 ++++++++++++++++++++++++++++++++++++++--- 2 files changed, 130 insertions(+), 8 deletions(-) diff --git a/doc/guix.texi b/doc/guix.texi index b95025a39f..f3be6b5085 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -5768,6 +5768,17 @@ This is similar to the same-named option in @command{guix package} (@pxref{profile-manifest, @option{--manifest}}) and uses the same manifest files. +@item --rebuild-cache +When using @option{--manifest}, @option{--file}, or when invoked without +arguments, @command{guix shell} caches the environment so that +subsequent uses are instantaneous. The cache is invalidated anytime the +file is modified. + +The @option{--rebuild-cache} forces the cached environment to be +refreshed even if the file has not changed. This is useful if the +@command{guix.scm} or @command{manifest.scm} has external dependencies, +or if its behavior depends, say, on environment variables. + @item --pure Unset existing environment variables when building the new environment, except those specified with @option{--preserve} (see below). This has the effect of diff --git a/guix/scripts/shell.scm b/guix/scripts/shell.scm index 45fd536145..4062e8155d 100644 --- a/guix/scripts/shell.scm +++ b/guix/scripts/shell.scm @@ -31,7 +31,15 @@ (define-module (guix scripts shell) #:use-module (srfi srfi-71) #:use-module (ice-9 match) #:autoload (ice-9 rdelim) (read-line) - #:autoload (guix utils) (config-directory) + #:autoload (guix base32) (bytevector->base32-string) + #:autoload (rnrs bytevectors) (string->utf8) + #:autoload (guix utils) (config-directory cache-directory) + #:autoload (guix describe) (current-channels) + #:autoload (guix channels) (channel-commit) + #:autoload (gcrypt hash) (sha256) + #:use-module ((guix build utils) #:select (mkdir-p)) + #:use-module (guix cache) + #:use-module ((ice-9 ftw) #:select (scandir)) #:export (guix-shell)) (define (show-help) @@ -48,6 +56,8 @@ (define (show-help) FILE evaluates to")) (display (G_ " -q inhibit loading of 'guix.scm' and 'manifest.scm'")) + (display (G_ " + --rebuild-cache rebuild cached environment, if any")) (show-environment-options-help) (newline) @@ -109,7 +119,10 @@ (define %options result))) (option '(#\q) #f #f (lambda (opt name arg result) - (alist-cons 'explicit-loading? #t result)))) + (alist-cons 'explicit-loading? #t result))) + (option '("rebuild-cache") #f #f + (lambda (opt name arg result) + (alist-cons 'rebuild-cache? #t result)))) (filter-map (lambda (opt) (and (not (any (lambda (name) (member name to-remove)) @@ -132,11 +145,12 @@ (define (handle-argument arg result) (let ((args command (break (cut string=? "--" <>) args))) (let ((opts (parse-command-line args %options (list %default-options) #:argument-handler handle-argument))) - (auto-detect-manifest - (match command - (() opts) - (("--") opts) - (("--" command ...) (alist-cons 'exec command opts))))))) + (options-with-caching + (auto-detect-manifest + (match command + (() opts) + (("--") opts) + (("--" command ...) (alist-cons 'exec command opts)))))))) (define (find-file-in-parent-directories candidates) "Find one of CANDIDATES in the current directory or one of its ancestors." @@ -187,6 +201,53 @@ (define (authorized-shell-directory? directory) line)))))))))) (const #f))) +(define (options-with-caching opts) + "If OPTS contains exactly one 'load' or one 'manifest' key, automatically +add a 'profile' key (when a profile for that file is already in cache) or a +'gc-root' key (to add the profile to cache)." + (define (single-file-for-caching opts) + (let loop ((opts opts) + (file #f)) + (match opts + (() file) + ((('package . _) . _) #f) + ((('load . ('package candidate)) . rest) + (and (not file) (loop rest candidate))) + ((('manifest . candidate) . rest) + (and (not file) (loop rest candidate))) + ((('expression . _) . _) #f) + ((_ . rest) (loop rest file))))) + + ;; Check whether there's a single 'load' or 'manifest' option. When that is + ;; the case, arrange to automatically cache the resulting profile. + (match (single-file-for-caching opts) + (#f opts) + (file + (let* ((root (profile-cached-gc-root file)) + (stat (and root (false-if-exception (lstat root))))) + (if (and (not (assoc-ref opts 'rebuild-cache?)) + stat + (<= (stat:mtime ((@ (guile) stat) file)) + (stat:mtime stat))) + (let ((now (current-time))) + ;; Update the atime on ROOT to reflect usage. + (utime root + now (stat:mtime stat) 0 (stat:mtimensec stat) + AT_SYMLINK_NOFOLLOW) + (alist-cons 'profile root + (remove (match-lambda + (('load . _) #t) + (('manifest . _) #t) + (_ #f)) + opts))) ;load right away + (if (and root (not (assq-ref opts 'gc-root))) + (begin + (if stat + (delete-file root) + (mkdir-p (dirname root))) + (alist-cons 'gc-root root opts)) + opts)))))) + (define (auto-detect-manifest opts) "If OPTS do not specify packages or a manifest, load a \"guix.scm\" or \"manifest.scm\" file from the current directory or one of its ancestors. @@ -236,9 +297,59 @@ (define disallow-implicit-load? (authorized-directory-file))) opts)))))) + +;;; +;;; Profile cache. +;;; + +(define %profile-cache-directory + ;; Directory where profiles created by 'guix shell' alone (without extra + ;; options) are cached. + (make-parameter (string-append (cache-directory #:ensure? #f) + "/profiles"))) + +(define (profile-cache-key file) + "Return the cache key for the profile corresponding to FILE, a 'guix.scm' or +'manifest.scm' file, or #f if we lack channel information." + (match (current-channels) + (() #f) + (((= channel-commit commits) ...) + (let ((stat (stat file))) + (bytevector->base32-string + ;; Since FILE is not canonicalized, only include the device/inode + ;; numbers. XXX: In some rare cases involving Btrfs and NFS, this can + ;; be insufficient: . + (sha256 (string->utf8 + (string-append (string-join commits) ":" + (number->string (stat:dev stat)) ":" + (number->string (stat:ino stat)))))))))) + +(define (profile-cached-gc-root file) + "Return the cached GC root for FILE, a 'guix.scm' or 'manifest.scm' file, or +#f if we lack information to cache it." + (match (profile-cache-key file) + (#f #f) + (key (string-append (%profile-cache-directory) "/" key)))) + (define-command (guix-shell . args) (category development) (synopsis "spawn one-off software environments") - (guix-environment* (parse-args args))) + (define (cache-entries directory) + (filter-map (match-lambda + ((or "." "..") #f) + (file (string-append directory "/" file))) + (or (scandir directory) '()))) + + (define* (entry-expiration file) + ;; Return the time at which FILE, a cached profile, is considered expired. + (match (false-if-exception (lstat file)) + (#f 0) ;FILE may have been deleted in the meantime + (st (+ (stat:atime st) (* 60 60 24 7))))) + + (let ((result (guix-environment* (parse-args args)))) + (maybe-remove-expired-cache-entries (%profile-cache-directory) + cache-entries + #:entry-expiration entry-expiration) + result))