From patchwork Mon Mar 14 21:51:46 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Ludovic_Court=C3=A8s?= X-Patchwork-Id: 37797 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 8D76727BBEA; Mon, 14 Mar 2022 21:53:35 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-3.7 required=5.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H5,RCVD_IN_MSPIKE_WL, SPF_HELO_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 1AE0927BBE9 for ; Mon, 14 Mar 2022 21:53:34 +0000 (GMT) Received: from localhost ([::1]:60092 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nTsdB-0005BW-2A for patchwork@mira.cbaines.net; Mon, 14 Mar 2022 17:53:33 -0400 Received: from eggs.gnu.org ([209.51.188.92]:54250) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nTscg-00056Z-Cu for guix-patches@gnu.org; Mon, 14 Mar 2022 17:53:02 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:52759) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1nTscg-0001NY-3j for guix-patches@gnu.org; Mon, 14 Mar 2022 17:53:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1nTscg-00036l-2g for guix-patches@gnu.org; Mon, 14 Mar 2022 17:53:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#54393] [PATCH 2/2] Add 'guix manifest'. Resent-From: Ludovic =?utf-8?q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Mon, 14 Mar 2022 21:53:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 54393 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 54393@debbugs.gnu.org Cc: Ludovic =?utf-8?q?Court=C3=A8s?= Received: via spool by 54393-submit@debbugs.gnu.org id=B54393.164729473611878 (code B ref 54393); Mon, 14 Mar 2022 21:53:02 +0000 Received: (at 54393) by debbugs.gnu.org; 14 Mar 2022 21:52:16 +0000 Received: from localhost ([127.0.0.1]:46655 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nTsbl-00035H-7x for submit@debbugs.gnu.org; Mon, 14 Mar 2022 17:52:16 -0400 Received: from eggs.gnu.org ([209.51.188.92]:47890) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nTsbh-00034N-93 for 54393@debbugs.gnu.org; Mon, 14 Mar 2022 17:52:03 -0400 Received: from [2001:470:142:3::e] (port=52432 helo=fencepost.gnu.org) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nTsbb-00019J-NT; Mon, 14 Mar 2022 17:51:55 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:References:In-Reply-To:Date:Subject:To: From; bh=6zqjGpniIhydCdbvKDp7LwPDK+N2lWYYWJy4s0/FHg0=; b=hRRMjPbtvUrUz5+a6rFe YrWzOjbQi9nmU6OM5g4Hzh/0eN/wFU0+ycfzCSJvVZ5ndQbhRlq+njkG9WxES+c/4o0gHiP/ky8ud jk6ptFQJNl95ctdMgYLqn9zOD1HelZnn9B905qjZcLz+5/clRcfzt0s8rbdcGZ2xBCzD5WZvG8NZE 25aT+jN9I7SlRKIFAIenWKC3JwsN4rkU7evjp4NZ0ZRP7BBQbuQ/b64yp9sqs/IdmbkKZBuB3U7xs qAA7w9B3PsNrr9NJRD8+qnSXFaeRVdlf9boLU+1Y+cYPbQe/luiIAZs9KolIr8fm4+GtbjWhZnxHj 67J4TS88MRCc5Q==; Received: from 91-160-117-201.subs.proxad.net ([91.160.117.201]:62373 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1nTsbb-00006I-BF; Mon, 14 Mar 2022 17:51:55 -0400 From: Ludovic =?utf-8?q?Court=C3=A8s?= Date: Mon, 14 Mar 2022 22:51:46 +0100 Message-Id: <20220314215146.24490-2-ludo@gnu.org> X-Mailer: git-send-email 2.34.0 In-Reply-To: <20220314215146.24490-1-ludo@gnu.org> References: <20220314215146.24490-1-ludo@gnu.org> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches From: Ludovic Courtès * guix/scripts/manifest.scm: New file. * po/guix/POTFILES.in: Add it. * tests/guix-manifest.sh: New file. * Makefile.am (MODULES, SH_TESTS): Add them. * doc/guix.texi (Invoking guix manifest): New section. (Invoking guix package): Refer to it. (Invoking guix shell): Likewise. (Invoking guix environment): Likewise. (Invoking guix pack): Likewise. --- Makefile.am | 2 + doc/guix.texi | 146 +++++++++++++++++++++++++++++++- guix/scripts/manifest.scm | 174 ++++++++++++++++++++++++++++++++++++++ po/guix/POTFILES.in | 1 + tests/guix-manifest.sh | 76 +++++++++++++++++ 5 files changed, 398 insertions(+), 1 deletion(-) create mode 100644 guix/scripts/manifest.scm create mode 100644 tests/guix-manifest.sh diff --git a/Makefile.am b/Makefile.am index 7402c89b62..40b6c75e23 100644 --- a/Makefile.am +++ b/Makefile.am @@ -318,6 +318,7 @@ MODULES = \ guix/scripts/import/pypi.scm \ guix/scripts/import/stackage.scm \ guix/scripts/import/texlive.scm \ + guix/scripts/manifest.scm \ guix/scripts/environment.scm \ guix/scripts/shell.scm \ guix/scripts/publish.scm \ @@ -568,6 +569,7 @@ SH_TESTS = \ tests/guix-environment.sh \ tests/guix-environment-container.sh \ tests/guix-shell.sh \ + tests/guix-manifest.sh \ tests/guix-graph.sh \ tests/guix-describe.sh \ tests/guix-repl.sh \ diff --git a/doc/guix.texi b/doc/guix.texi index dbe281ead7..4dc3c8b1fc 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -233,6 +233,7 @@ Package Management * Invoking guix package:: Package installation, removal, etc. * Substitutes:: Downloading pre-built binaries. * Packages with Multiple Outputs:: Single source package, multiple outputs. +* Invoking guix manifest:: Producing environment declarations. * Invoking guix gc:: Running the garbage collector. * Invoking guix pull:: Fetching the latest Guix and distribution. * Invoking guix time-machine:: Running an older revision of Guix. @@ -3042,6 +3043,7 @@ guix install emacs-guix * Invoking guix package:: Package installation, removal, etc. * Substitutes:: Downloading pre-built binaries. * Packages with Multiple Outputs:: Single source package, multiple outputs. +* Invoking guix manifest:: Producing environment declarations. * Invoking guix gc:: Running the garbage collector. * Invoking guix pull:: Fetching the latest Guix and distribution. * Invoking guix time-machine:: Running an older revision of Guix. @@ -3412,7 +3414,9 @@ The example above gives you all the software required to develop Emacs, similar to what @command{guix environment emacs} provides. @xref{export-manifest, @option{--export-manifest}}, to learn how to -obtain a manifest file from an existing profile. +obtain a manifest file from an existing profile, and @pxref{Invoking +guix manifest} on how to generate a manifest file from a list of package +specs and command-line options. @item --roll-back @cindex rolling back @@ -4159,6 +4163,137 @@ Files}). The outputs of a packages are listed in the third column of the output of @command{guix package --list-available} (@pxref{Invoking guix package}). +@node Invoking guix manifest +@section Invoking @command{guix manifest} + +@cindex manifest, generating +The @command{guix manifest} command outputs a @dfn{manifest} +corresponding to the packages and options specified on the command line. +Manifests are code snippets that @emph{declare} the set of packages you +want to deploy---you can view them as a more expressive form of what you +pass on the command line to @command{guix install}, @command{guix +shell}, etc. All these commands accept a @option{--manifest} (or +@option{-m}) option that allows you to pass them a manifest. For +non-trivial package sets and customizations, you'll find that using a +manifest rather than a long command line is often more convenient. + +But how do you go from that long command line you're familiar with to +that appealing but possibly intimidating ``manifest'' thing? The +@command{guix manifest} command is your companion on this journey: it +essentially ``translates'' command-line arguments into manifests. + +Let's look at a few examples. What's a manifest corresponding to a +basic list of package specifications? We can figure out by running, +say: + +@example +guix manifest coreutils grep sed +@end example + +@noindent +... which outputs this manifest: + +@lisp +(specifications->manifest + (list "coreutils" "grep" "sed")) +@end lisp + +The manifest constructs a list containing the three @dfn{package specs} +and passes it to the @code{specifications->manifest} procedure, which +returns a manifest corresponding of the three designated packages. + +@quotation Note +Manifests are @emph{symbolic}: they refer to packages by their +specification (name and optionally version), which denote different +packages over time---@code{coreutils} above might refer to version 8.32 +today and to 9.0 six months from now. + +To ``pin'' a package set to a specific revision, you will additionally +need a @dfn{channel file} as produced by @command{guix describe -f +channels} (@pxref{Invoking guix describe}). +@end quotation + +That one was easy, but @command{guix manifest} can also handle more +complex cases. For example, consider the manifest for the development +environment of Guile, with the addition of Git: + +@example +guix manifest -D guile git +@end example + +@noindent +This gives us: + +@example +(concatenate-manifests + (list (specifications->manifest (list "git")) + (package->development-manifest + (specification->package "guile")))) +@end example + +The @command{guix manifest} command also takes care package +transformation options, producing a manifest that faithfully reapplies +them (@pxref{Package Transformation Options}): + +@example +guix manifest intel-mpi-benchmarks --with-input=openmpi=mpich +@end example + +@noindent +... yields: + +@lisp +(use-modules (guix transformations)) + +(define transform1 + (options->transformation + '((with-input . "openmpi=mpich")))) + +(packages->manifest + (list (transform1 + (specification->package "intel-mpi-benchmarks")))) +@end lisp + +Convenient, no? You can take the output of @command{guix manifest} +as-is, store it in a file, and enjoy it. But you can also view it as +raw material that you can modify to refine the expression of the +environment you want to deploy. + +The general syntax is: + +@example +guix manifest [@var{options}] @var{spec}@dots{} +@end example + +@noindent +... where each @var{spec} denotes a package and (optionally) its output, +such as @code{emacs}, @code{gcc-toolchain@@8}, or +@code{git:send-email}. The available options are: + +@table @option +@item --development +@itemx -D +Consider the environment needed to @emph{develop} the following package +rather than the package itself. + +For instance, to obtain a manifest representing the environment to +develop Elixir (and not Elixir itself), with the addition of Nano, run: + +@example +guix manifest -D elixir nano +@end example + +In this example, @option{-D} affects @code{elixir}, not @code{nano}. + +@item --manifest=@var{file} +@itemx -m @var{file} +Read the manifest in @var{file} and combine it with other options to +produce the resulting manifest. +@end table + +Additionally, @command{guix manifest} understands all the package +transformation options (@pxref{Package Transformation Options}). + @node Invoking guix gc @section Invoking @command{guix gc} @@ -5847,6 +5982,9 @@ This is similar to the same-named option in @command{guix package} (@pxref{profile-manifest, @option{--manifest}}) and uses the same manifest files. +@xref{Invoking guix manifest}, for information on how to ``convert'' +command-line options into a manifest. + @item --profile=@var{profile} @itemx -p @var{profile} Create an environment containing the packages installed in @var{profile}. @@ -6234,6 +6372,9 @@ This is similar to the same-named option in @command{guix package} (@pxref{profile-manifest, @option{--manifest}}) and uses the same manifest files. +@xref{Invoking guix manifest}, for information on how to ``convert'' +command-line options into a manifest. + @item --ad-hoc Include all specified packages in the resulting environment, as if an @i{ad hoc} package were defined with them as inputs. This option is @@ -6692,6 +6833,9 @@ for use on machines that do not have Guix installed. Note that you can specify @emph{either} a manifest file @emph{or} a list of packages, but not both. +@xref{Invoking guix manifest}, for information on how to ``convert'' +command-line options into a manifest. + @item --system=@var{system} @itemx -s @var{system} Attempt to build for @var{system}---e.g., @code{i686-linux}---instead of diff --git a/guix/scripts/manifest.scm b/guix/scripts/manifest.scm new file mode 100644 index 0000000000..fea5d130c3 --- /dev/null +++ b/guix/scripts/manifest.scm @@ -0,0 +1,174 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright © 2022 Ludovic Courtès +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see . + +(define-module (guix scripts manifest) + #:use-module (guix ui) + #:use-module ((guix diagnostics) #:select (location)) + #:use-module (guix scripts environment) + #:use-module (guix transformations) + #:use-module (guix scripts) + #:use-module (guix packages) + #:use-module (guix profiles) + #:autoload (gnu packages) (specifications->manifest + specification->package + package-unique-version-prefix) + #:use-module (srfi srfi-1) + #:use-module (srfi srfi-26) + #:use-module (srfi srfi-37) + #:use-module (srfi srfi-71) + #:use-module (ice-9 match) + #:autoload (ice-9 pretty-print) (pretty-print) + #:export (guix-manifest)) + +(define (show-help) + (display (G_ "Usage: guix manifest [OPTION] SPECS... +Print a manifest corresponding to the given package SPECS.\n")) + (newline) + + (display (G_ " + -D, --development include the development inputs of the next package")) + (display (G_ " + -m, --manifest=FILE create environment with the manifest from FILE")) + + (newline) + (show-transformation-options-help) + (newline) + (display (G_ " + -h, --help display this help and exit")) + (display (G_ " + -V, --version display version information and exit")) + (newline) + (show-bug-report-information)) + +(define %options + ;; Specification of the command-line options. + (cons* (option '(#\h "help") #f #f + (lambda args + (show-help) + (exit 0))) + (option '(#\V "version") #f #f + (lambda args + (show-version-and-exit "guix manifest"))) + + (option '(#\D "development") #f #f + (lambda (opt name arg result) + (alist-cons 'development? #t result))) + (option '(#\m "manifest") #t #f + (lambda (opt name arg result) + (alist-cons 'manifest arg result))) + + %transformation-options)) + +(define %default-options + ;; Default option alist. + '()) + +(define (load-manifest file) ;TODO: factorize + "Load the user-profile manifest (Scheme code) from FILE and return it." + (let ((user-module (make-user-module '((guix profiles) (gnu))))) + (load* file user-module))) + +(define (manifest-entry-version-prefix entry) + "Search among all the versions of ENTRY's package that are available, and +return the shortest unambiguous version prefix for this package." + (package-unique-version-prefix (manifest-entry-name entry) + (manifest-entry-version entry))) + +(define (manifest->code* manifest extra-manifests) + "Like 'manifest->code', but insert a 'concatenate-manifests' call that +concatenates MANIFESTS, a list of expressions." + (if (null? (manifest-entries manifest)) + (match extra-manifests + ((one) one) + (lst `(concatenate-manifests ,@extra-manifests))) + (match (manifest->code manifest + #:entry-package-version + manifest-entry-version-prefix) + (('begin exp ... last) + `(begin + ,@exp + ,(match extra-manifests + (() last) + (_ `(concatenate-manifests + (list ,last ,@extra-manifests))))))))) + + +(define-command (guix-manifest . args) + (category development) + (synopsis "turn command-line arguments into a manifest") + + (define (manifest-lift proc) + (lambda (entry) + (match (manifest-entry-item entry) + ((? package? p) + (manifest-entry + (inherit (package->manifest-entry (proc p))) + (output (manifest-entry-output entry)))) + (_ + entry)))) + + (define (handle-argument arg result) + (if (assoc-ref result 'development?) + (alist-cons 'development-inputs arg + (alist-delete 'development? result)) + (alist-cons 'argument arg result))) + + (with-error-handling + (let* ((opts (parse-command-line args %options (list %default-options) + #:build-options? #f + #:argument-handler handle-argument)) + (transform (options->transformation opts)) + (specs (reverse + (filter-map (match-lambda + (('argument . spec) spec) + (_ #f)) + opts))) + (extras (reverse + (filter-map (match-lambda + (('development-inputs . spec) + ;; Make sure SPEC is valid. + (specification->package spec) + + ;; XXX: This is an approximation: + ;; transformation options are not + ;; applied. + `(package->development-manifest + (specification->package ,spec))) + (_ #f)) + opts))) + (manifest (concatenate-manifests + (cons (map-manifest-entries + (manifest-lift transform) + (specifications->manifest specs)) + (filter-map (match-lambda + (('manifest . file) + (load-manifest file)) + (_ #f)) + opts))))) + (display (G_ "\ +;; What follows is a \"manifest\" equivalent to the command line you gave. +;; You can store it in a file that you may then pass to any 'guix' command +;; that accepts a '--manifest' (or '-m') option.\n")) + (match (manifest->code* manifest extras) + (('begin exp ...) + (for-each (lambda (exp) + (newline) + (pretty-print exp)) + exp)) + (exp + (pretty-print exp)))))) diff --git a/po/guix/POTFILES.in b/po/guix/POTFILES.in index d97ba8c209..5b8eadf884 100644 --- a/po/guix/POTFILES.in +++ b/po/guix/POTFILES.in @@ -99,6 +99,7 @@ guix/scripts/weather.scm guix/scripts/describe.scm guix/scripts/processes.scm guix/scripts/deploy.scm +guix/scripts/manifest.scm guix/gexp.scm guix/gnu-maintenance.scm guix/scripts/container.scm diff --git a/tests/guix-manifest.sh b/tests/guix-manifest.sh new file mode 100644 index 0000000000..de82815ba0 --- /dev/null +++ b/tests/guix-manifest.sh @@ -0,0 +1,76 @@ +# GNU Guix --- Functional package management for GNU +# Copyright © 2022 Ludovic Courtès +# +# This file is part of GNU Guix. +# +# GNU Guix is free software; you can redistribute it and/or modify it +# under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 3 of the License, or (at +# your option) any later version. +# +# GNU Guix is distributed in the hope that it will be useful, but +# WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with GNU Guix. If not, see . + +# +# Test 'guix manifest'. +# + +guix manifest --version + +tmpdir="t-guix-manifest-$$" +trap 'rm -r "$tmpdir"' EXIT +mkdir "$tmpdir" + +manifest="$tmpdir/manifest.scm" + +# Basics. +guix manifest guile-bootstrap > "$manifest" +test "$(guix build -m "$manifest")" = "$(guix build guile-bootstrap)" + +guix shell -m "$manifest" --bootstrap -- \ + "$SHELL" -c 'guix package --export-manifest -p "$GUIX_ENVIRONMENT"' > \ + "$manifest.second" +for m in "$manifest" "$manifest.second" +do + grep -v '^;' < "$m" > "$m.new" # filter out comments + mv "$m.new" "$m" +done + +cat "$manifest" +cat "$manifest.second" + +cmp "$manifest" "$manifest.second" + +# Package transformation option. +guix manifest guile guix --with-latest=guile-json > "$manifest" +grep 'options->transformation' "$manifest" +grep '(with-latest . "guile-json")' "$manifest" + +# Development manifest. +guix manifest -D guile git > "$manifest" +grep 'package->development-manifest' "$manifest" +grep '"guile"' "$manifest" +guix build -m "$manifest" -d | \ + grep "$(guix build -e '(@@ (gnu packages commencement) gcc-final)' -d)" +guix build -m "$manifest" -d | \ + grep "$(guix build git -d)" + +# Test various combinations to make sure generated code uses interfaces +# correctly. +for options in \ + "coreutils grep sed" \ + "gsl openblas gcc-toolchain --tune" \ + "guile -m $manifest.previous" \ + "git:send-email gdb guile:debug" \ + "git -D coreutils" +do + guix manifest $options > "$manifest" + cat "$manifest" + guix shell -m "$manifest" -n + mv "$manifest" "$manifest.previous" +done