From patchwork Tue Nov 14 13:35:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Ludovic_Court=C3=A8s?= X-Patchwork-Id: 56452 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 557BE27BBEA; Tue, 14 Nov 2023 13:36:40 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,SPF_HELO_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 2995E27BBE2 for ; Tue, 14 Nov 2023 13:36:39 +0000 (GMT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1r2tab-00070v-Kv; Tue, 14 Nov 2023 08:36:25 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1r2taZ-00070T-WE for guix-patches@gnu.org; Tue, 14 Nov 2023 08:36:24 -0500 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1r2taZ-00039p-5H; Tue, 14 Nov 2023 08:36:23 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1r2tbC-0004fE-4q; Tue, 14 Nov 2023 08:37:02 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#67172] [PATCH 1/2] gexp: Add compiler for . Resent-From: Ludovic =?utf-8?q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix@cbaines.net, dev@jpoiret.xyz, ludo@gnu.org, othacehe@gnu.org, rekado@elephly.net, zimon.toutoune@gmail.com, me@tobias.gr, guix-patches@gnu.org Resent-Date: Tue, 14 Nov 2023 13:37:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 67172 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 67172@debbugs.gnu.org Cc: Ludovic =?utf-8?q?Court=C3=A8s?= , Christopher Baines , Josselin Poiret , Ludovic =?utf-8?q?Court=C3=A8s?= , Mathieu Othacehe , Ricardo Wurmus , Simon Tournier , Tobias Geerinckx-Rice X-Debbugs-Original-Xcc: Christopher Baines , Josselin Poiret , Ludovic =?utf-8?q?Court=C3=A8s?= , Mathieu Othacehe , Ricardo Wurmus , Simon Tournier , Tobias Geerinckx-Rice Received: via spool by 67172-submit@debbugs.gnu.org id=B67172.169996902117922 (code B ref 67172); Tue, 14 Nov 2023 13:37:02 +0000 Received: (at 67172) by debbugs.gnu.org; 14 Nov 2023 13:37:01 +0000 Received: from localhost ([127.0.0.1]:60495 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r2tbB-0004ez-3M for submit@debbugs.gnu.org; Tue, 14 Nov 2023 08:37:01 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:53364) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r2tb8-0004ek-Cr for 67172@debbugs.gnu.org; Tue, 14 Nov 2023 08:36:59 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1r2taL-000382-1L; Tue, 14 Nov 2023 08:36:09 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:References:In-Reply-To:Date:Subject:To: From; bh=2tfQ0dQSlWrbaookOBSUTMiBfVQUuJ6VHs6Fq3Q0FkE=; b=S24OSB+Sr9JkEjZKmZBP 4UXvCwqvrFYeWKR8fLJwY643ZClKNVId9UukbMzTtTbZyu3JO9GHyqB3mSzdJJHlndvZTgWl80JcM gHgJlG2rGdnqwEEx19IaZb6dbOQPG7joHDqncMzfqCLYH7wwE2F5dzKEa80h7Esc1kXmDKaxwDem+ wbSaE6UxArKVqnFPSyZYd0s7PTXACOL9WU6LC9i2KSYC3OVZUQQCH7NelX98z2zIUfLZjqrrcz9QW nITxiX7AGWDWxTxSl9qmdxCp9sKSDArvegds2ZYguZpgrjiWQBA0ILW14nW6FbGQ9oKhSo5WPrIha zUA0FFimxxDVaQ==; From: Ludovic =?utf-8?q?Court=C3=A8s?= Date: Tue, 14 Nov 2023 14:35:47 +0100 Message-ID: X-Mailer: git-send-email 2.41.0 In-Reply-To: References: MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org X-getmail-retrieved-from-mailbox: Patches * guix/gexp.scm (gexp-input-compiler): New procedure. * tests/gexp.scm ("gexp references non-existent output") ("gexp-input, as first-class input"): New tests. * doc/guix.texi (G-Expressions): Document it. Change-Id: I95b58d6e4d77a54364026b4324fbb00125a9402e Reviewed-by: Maxim Cournoyer --- doc/guix.texi | 38 ++++++++++++++++++++++++++++++++++++++ guix/gexp.scm | 19 ++++++++++++++++++- tests/gexp.scm | 26 +++++++++++++++++++++++++- 3 files changed, 81 insertions(+), 2 deletions(-) diff --git a/doc/guix.texi b/doc/guix.texi index 9f06f1c325..8492f0ada3 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -12096,6 +12096,11 @@ G-Expressions @var{output} of @var{obj}---this is useful when @var{obj} produces multiple outputs (@pxref{Packages with Multiple Outputs}). +Sometimes a gexp unconditionally refers to the @code{"out"} output, but +the user of that gexp would still like to insert a reference to another +output. The @code{gexp-input} procedure aims to address that. +@xref{gexp-input}. + @item #+@var{obj} @itemx #+@var{obj}:output @itemx (ungexp-native @var{obj}) @@ -12489,6 +12494,39 @@ G-Expressions of Coreutils, regardless of the current value of @code{%current-system}. @end defmac +@anchor{gexp-input} +@deffn {Procedure} gexp-input @var{obj} [@var{output}] [#:native? #f] +Return a @dfn{gexp input} record for the given @var{output} of file-like +object @var{obj}, with @code{#:native?} determining whether this is a +native reference (as with @code{ungexp-native}) or not. + +This procedure is helpful when you want to pass a reference to a +specific output of an object to some procedure that may not know about +that output. For example, assume you have this procedure, which takes +one file-like object: + +@lisp +(define (make-symlink target) + (computed-file "the-symlink" + #~(symlink #$target #$output))) +@end lisp + +Here @code{make-symlink} can only ever refer to the default output of +@var{target}---the @code{"out"} output (@pxref{Packages with Multiple +Outputs}). To have it refer to, say, the @code{"lib"} output of the +@code{hwloc} package, you can call it like so: + +@lisp +(make-symlink (gexp-input hwloc "lib")) +@end lisp + +You can also compose it like any other file-like object: + +@lisp +(make-symlink + (file-append (gexp-input hwloc "lib") "/lib/libhwloc.so")) +@end lisp +@end deffn Of course, in addition to gexps embedded in ``host'' code, there are also modules containing build tools. To make it clear that they are diff --git a/guix/gexp.scm b/guix/gexp.scm index 0fe4f1c98a..a7f4256d24 100644 --- a/guix/gexp.scm +++ b/guix/gexp.scm @@ -1,5 +1,5 @@ ;;; GNU Guix --- Functional package management for GNU -;;; Copyright © 2014-2022 Ludovic Courtès +;;; Copyright © 2014-2023 Ludovic Courtès ;;; Copyright © 2018 Clément Lassieur ;;; Copyright © 2018 Jan Nieuwenhuizen ;;; Copyright © 2019, 2020 Mathieu Othacehe @@ -775,6 +775,23 @@ (define* (gexp-input thing ;convenience procedure whether this should be considered a \"native\" input or not." (%gexp-input thing output native?)) +;; Allow s to be used within gexps. This is useful when willing +;; to force a specific reference to an object, as in (gexp-input hwloc "bin"), +;; which forces a reference to the "bin" output of 'hwloc' instead of leaving +;; it up to the recipient to pick the right output. +(define-gexp-compiler gexp-input-compiler + compiler => (lambda (obj system target) + (match obj + (($ thing output native?) + (lower-object thing system + #:target (and (not native?) target))))) + expander => (lambda (obj lowered output/ignored) + (match obj + (($ thing output native?) + (let ((expand (or (lookup-expander thing) + (lookup-expander lowered)))) + (expand thing lowered output)))))) + ;; Reference to one of the derivation's outputs, for gexps used in ;; derivations. (define-record-type diff --git a/tests/gexp.scm b/tests/gexp.scm index 7a90f8dcbf..a3147405d7 100644 --- a/tests/gexp.scm +++ b/tests/gexp.scm @@ -1,5 +1,5 @@ ;;; GNU Guix --- Functional package management for GNU -;;; Copyright © 2014-2022 Ludovic Courtès +;;; Copyright © 2014-2023 Ludovic Courtès ;;; Copyright © 2021-2022 Maxime Devos ;;; ;;; This file is part of GNU Guix. @@ -393,6 +393,30 @@ (define %extension-package (list item)) (null? (lowered-gexp-inputs lexp))))) +(test-equal "gexp references non-existent output" + "no-default-output" + (guard (c ((derivation-missing-output-error? c) + (derivation-name (derivation-error-derivation c)))) + (let* ((obj (computed-file "no-default-output" + #~(mkdir #$output:bar))) + (exp #~(symlink #$obj #$output)) + (drv (run-with-store %store (lower-gexp exp)))) + (pk 'oops! drv #f)))) + +(test-assert "gexp-input, as first-class input" + ;; Insert a record in a gexp as a way to specify which output + ;; of OBJ should be used. + (let* ((obj (computed-file "foo" #~(mkdir #$output:bar))) + (exp #~(list #$(gexp-input obj "bar"))) + (drv (run-with-store %store (lower-object obj))) + (item (derivation->output-path drv "bar")) + (lexp (run-with-store %store (lower-gexp exp)))) + (and (match (lowered-gexp-inputs lexp) + ((input) + (eq? (derivation-input-derivation input) drv))) + (equal? (lowered-gexp-sexp lexp) + `(list ,item))))) + (test-assertm "with-parameters for %current-system" (mlet* %store-monad ((system -> (match (%current-system) ("aarch64-linux" "x86_64-linux") From patchwork Tue Nov 14 13:35:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Ludovic_Court=C3=A8s?= X-Patchwork-Id: 56453 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 8D5C627BBEA; Tue, 14 Nov 2023 13:37:39 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,SPF_HELO_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id A4DCB27BBE2 for ; Tue, 14 Nov 2023 13:37:38 +0000 (GMT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1r2tbg-0007hQ-7S; Tue, 14 Nov 2023 08:37:32 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1r2tbX-0007Xn-Gu for guix-patches@gnu.org; Tue, 14 Nov 2023 08:37:24 -0500 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1r2tbV-0003Jf-Lz; Tue, 14 Nov 2023 08:37:23 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1r2tc9-0004gu-Kd; Tue, 14 Nov 2023 08:38:01 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#67172] [PATCH 2/2] gexp: #:references-graphs accepts and honors records. Resent-From: Ludovic =?utf-8?q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix@cbaines.net, dev@jpoiret.xyz, ludo@gnu.org, othacehe@gnu.org, rekado@elephly.net, zimon.toutoune@gmail.com, me@tobias.gr, guix-patches@gnu.org Resent-Date: Tue, 14 Nov 2023 13:38:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 67172 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 67172@debbugs.gnu.org Cc: Ludovic =?utf-8?q?Court=C3=A8s?= , Christopher Baines , Josselin Poiret , Ludovic =?utf-8?q?Court=C3=A8s?= , Mathieu Othacehe , Ricardo Wurmus , Simon Tournier , Tobias Geerinckx-Rice X-Debbugs-Original-Xcc: Christopher Baines , Josselin Poiret , Ludovic =?utf-8?q?Court=C3=A8s?= , Mathieu Othacehe , Ricardo Wurmus , Simon Tournier , Tobias Geerinckx-Rice Received: via spool by 67172-submit@debbugs.gnu.org id=B67172.169996904417988 (code B ref 67172); Tue, 14 Nov 2023 13:38:01 +0000 Received: (at 67172) by debbugs.gnu.org; 14 Nov 2023 13:37:24 +0000 Received: from localhost ([127.0.0.1]:60505 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r2tbX-0004g4-Ky for submit@debbugs.gnu.org; Tue, 14 Nov 2023 08:37:23 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:34728) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r2tbV-0004fp-GU for 67172@debbugs.gnu.org; Tue, 14 Nov 2023 08:37:21 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1r2tah-0003Ae-Kt; Tue, 14 Nov 2023 08:36:33 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:References:In-Reply-To:Date:Subject:To: From; bh=ok/uAKjMCy/oQGrwNt/rf/KNm6Bb0NevxF0AwmWp/K0=; b=mL/rg/mKv/D8KZVzr+lG RTuarVYGKPh+rqGoXtF1Agua3f+T4/4fAfcvurnth0TET9ChjeNpxQXlipFFRLkrTWeAs0Df/Isi5 Fd2cvGSFSZjSEiZAmH701FRPgRAwhk8VkYZEXTfzx7UXtC9O3zsP//m/1UyMcLmOmgkrsV4Ck/1dR IkALdGrKt8IQGsFdR0PKNR/2uv/NYaPcRhaE8jKX3LlLUjvcIYivM3U/Gbc50ZApDfe0juP9piQVI 0K6ZNlksf6Lh8OdXtQACRqDVL1Ob3quEm7Rk/Kv9oMI+mnn1kkibhS99OT5hDH007SMo3LZaWWVzb +193STFPojOIjw==; From: Ludovic =?utf-8?q?Court=C3=A8s?= Date: Tue, 14 Nov 2023 14:35:48 +0100 Message-ID: X-Mailer: git-send-email 2.41.0 In-Reply-To: References: MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org X-getmail-retrieved-from-mailbox: Patches * guix/gexp.scm (lower-reference-graphs)[tuple->gexp-input]: Add ‘gexp-input?’ case. (gexp->derivation): Update docstring. * doc/guix.texi (G-Expressions): Adjust accordingly. * tests/gexp.scm ("references-file, non-default output"): New test. Change-Id: I595cb75da0867ab8ab44552887dc06ed1d23315e Reviewed-by: Maxim Cournoyer --- doc/guix.texi | 7 +++---- guix/gexp.scm | 12 ++++++++---- tests/gexp.scm | 30 ++++++++++++++++++++++++++++++ 3 files changed, 41 insertions(+), 8 deletions(-) diff --git a/doc/guix.texi b/doc/guix.texi index 8492f0ada3..5f90ec6eb4 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -12213,10 +12213,9 @@ G-Expressions following forms: @example -(@var{file-name} @var{package}) -(@var{file-name} @var{package} @var{output}) -(@var{file-name} @var{derivation}) -(@var{file-name} @var{derivation} @var{output}) +(@var{file-name} @var{obj}) +(@var{file-name} @var{obj} @var{output}) +(@var{file-name} @var{gexp-input}) (@var{file-name} @var{store-item}) @end example diff --git a/guix/gexp.scm b/guix/gexp.scm index a7f4256d24..29819878fa 100644 --- a/guix/gexp.scm +++ b/guix/gexp.scm @@ -934,6 +934,11 @@ (define* (lower-reference-graphs graphs #:key system target) corresponding or store item." (define tuple->gexp-input (match-lambda + (((? gexp-input? input)) + ;; This case lets users specify the output of interest more + ;; conveniently, for instance by passing (gexp-input hwloc "lib") to + ;; the 'references-file' procedure. + input) ((thing) (%gexp-input thing "out" (not target))) ((thing output) @@ -1152,10 +1157,9 @@ (define* (gexp->derivation name exp When REFERENCES-GRAPHS is true, it must be a list of tuples of one of the following forms: - (FILE-NAME PACKAGE) - (FILE-NAME PACKAGE OUTPUT) - (FILE-NAME DERIVATION) - (FILE-NAME DERIVATION OUTPUT) + (FILE-NAME OBJ) + (FILE-NAME OBJ OUTPUT) + (FILE-NAME GEXP-INPUT) (FILE-NAME STORE-ITEM) The right-hand-side of each element of REFERENCES-GRAPHS is automatically made diff --git a/tests/gexp.scm b/tests/gexp.scm index a3147405d7..481755138e 100644 --- a/tests/gexp.scm +++ b/tests/gexp.scm @@ -1651,6 +1651,36 @@ (define shebang read) refs))))))) +(test-assertm "references-file, non-default output" + (let* ((exp #~(begin + (mkdir #$output) + (symlink #$%bootstrap-guile #$output:extra))) + (computed (computed-file "computed" exp + #:guile %bootstrap-guile)) + (refs1 (references-file computed + #:guile %bootstrap-guile)) + ;; Wrap COMPUTE in 'gexp-input' to get the "extra" output. + (refs2 (references-file (gexp-input computed "extra") + #:guile %bootstrap-guile))) + (mlet* %store-monad ((drv0 (lower-object %bootstrap-guile)) + (drv1 (lower-object computed)) + (drv2 (lower-object refs2)) + (drv3 (lower-object refs1))) + (mbegin %store-monad + (built-derivations (list drv2 drv3)) + (mlet %store-monad ((refs ((store-lift requisites) + (list (derivation->output-path + drv1 "extra"))))) + (return + (and (lset= string=? + (call-with-input-file (derivation->output-path drv2) + read) + refs) + (lset= string=? + (call-with-input-file (derivation->output-path drv3) + read) + (list (derivation->output-path drv1)))))))))) + (test-assert "lower-object & gexp-input-error?" (guard (c ((gexp-input-error? c) (gexp-error-invalid-input c)))