From patchwork Wed Sep 23 16:23:16 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Ludovic_Court=C3=A8s?= X-Patchwork-Id: 24262 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 99CCC27BBE8; Wed, 23 Sep 2020 17:27:55 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_MSPIKE_H4,RCVD_IN_MSPIKE_WL,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 5FC6427BBE7 for ; Wed, 23 Sep 2020 17:27:54 +0100 (BST) Received: from localhost ([::1]:34178 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kL7cX-0006Y1-Ge for patchwork@mira.cbaines.net; Wed, 23 Sep 2020 12:27:53 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51620) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kL7Yp-00023G-Jp for guix-patches@gnu.org; Wed, 23 Sep 2020 12:24:03 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:54313) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kL7Yp-0000Nz-9B for guix-patches@gnu.org; Wed, 23 Sep 2020 12:24:03 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1kL7Yp-0001Z0-60 for guix-patches@gnu.org; Wed, 23 Sep 2020 12:24:03 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#43578] [PATCH 2/4] packages: 'package-input-rewriting/spec' can rewrite implicit dependencies. Resent-From: Ludovic =?utf-8?q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Wed, 23 Sep 2020 16:24:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 43578 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 43578@debbugs.gnu.org Cc: Ludovic =?utf-8?q?Court=C3=A8s?= Received: via spool by 43578-submit@debbugs.gnu.org id=B43578.16008782175936 (code B ref 43578); Wed, 23 Sep 2020 16:24:03 +0000 Received: (at 43578) by debbugs.gnu.org; 23 Sep 2020 16:23:37 +0000 Received: from localhost ([127.0.0.1]:37620 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kL7YO-0001Xe-9X for submit@debbugs.gnu.org; Wed, 23 Sep 2020 12:23:36 -0400 Received: from eggs.gnu.org ([209.51.188.92]:44658) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kL7YK-0001Wt-9S for 43578@debbugs.gnu.org; Wed, 23 Sep 2020 12:23:33 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:47177) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kL7YE-0000Jp-Sc; Wed, 23 Sep 2020 12:23:26 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=52578 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1kL7YD-0002gP-G2; Wed, 23 Sep 2020 12:23:26 -0400 From: Ludovic =?utf-8?q?Court=C3=A8s?= Date: Wed, 23 Sep 2020 18:23:16 +0200 Message-Id: <20200923162318.2800-2-ludo@gnu.org> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200923162318.2800-1-ludo@gnu.org> References: <20200923162318.2800-1-ludo@gnu.org> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches With this change, '--with-input', '--with-graft', etc. also apply to implicit dependencies. Thus, it's now possible to do: guix build python-itsdangerous --with-input=python-wrapper=python@2 or: guix build hello --with-graft=glibc=glibc@2.29 Additionally, before, implicit inputs were not rewritten, which could lead to duplicates in the output of 'bag-transitive-inputs' (packages that are not 'eq?' but lead to the same derivation). This in turn would lead to unnecessary rebuilds when using '--with-input' & co. This change fixes it by ensuring even implicit inputs are rewritten. Fixes . * guix/packages.scm (package-input-rewriting/spec): Add #:deep? defaulting to #true, and pass it to 'package-mapping'. [replacement-property]: New variable. [rewrite]: Check that property and set it on the result of PROC. [cut?]: New procedure. * tests/packages.scm ("package-input-rewriting/spec"): Ensure implicit inputs were unchanged. ("package-input-rewriting/spec, partial match"): Pass #:deep? #f. ("package-input-rewriting/spec, deep") ("package-input-rewriting/spec, no duplicates"): New tests. (package/inherit): Move before use. * tests/guix-build.sh: Add tests. * tests/scripts-build.scm ("options->transformation, with-graft"): Compare dependencies by package name or derivation file name. * doc/guix.texi (Defining Packages): Adjust accordingly. --- doc/guix.texi | 13 ++++---- guix/packages.scm | 53 +++++++++++++++++++++------------ tests/guix-build.sh | 11 +++++++ tests/packages.scm | 66 +++++++++++++++++++++++++++++++++++++++-- tests/scripts-build.scm | 12 ++++++-- 5 files changed, 124 insertions(+), 31 deletions(-) diff --git a/doc/guix.texi b/doc/guix.texi index 4595008c4f..e9e1d122ab 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -6270,12 +6270,13 @@ This is exactly what the @option{--with-input} command-line option does The following variant of @code{package-input-rewriting} can match packages to be replaced by name rather than by identity. -@deffn {Scheme Procedure} package-input-rewriting/spec @var{replacements} -Return a procedure that, given a package, applies the given @var{replacements} to -all the package graph (excluding implicit inputs). @var{replacements} is a list of -spec/procedures pair; each spec is a package specification such as @code{"gcc"} or -@code{"guile@@2"}, and each procedure takes a matching package and returns a -replacement for that package. +@deffn {Scheme Procedure} package-input-rewriting/spec @var{replacements} [#:deep? #t] +Return a procedure that, given a package, applies the given +@var{replacements} to all the package graph, including implicit inputs +unless @var{deep?} is false. @var{replacements} is a list of +spec/procedures pair; each spec is a package specification such as +@code{"gcc"} or @code{"guile@@2"}, and each procedure takes a matching +package and returns a replacement for that package. @end deffn The example above could be rewritten this way: diff --git a/guix/packages.scm b/guix/packages.scm index 171fd048ef..f696945e30 100644 --- a/guix/packages.scm +++ b/guix/packages.scm @@ -422,6 +422,16 @@ name of its URI." package) 16))))) +(define-syntax-rule (package/inherit p overrides ...) + "Like (package (inherit P) OVERRIDES ...), except that the same +transformation is done to the package replacement, if any. P must be a bare +identifier, and will be bound to either P or its replacement when evaluating +OVERRIDES." + (let loop ((p p)) + (package (inherit p) + overrides ... + (replacement (and=> (package-replacement p) loop))))) + (define (package-upstream-name package) "Return the upstream name of PACKAGE, which could be different from the name it has in Guix." @@ -1051,12 +1061,12 @@ package and returns its new name after rewrite." (package-mapping rewrite (cut assq <> replacements))) -(define (package-input-rewriting/spec replacements) +(define* (package-input-rewriting/spec replacements #:key (deep? #t)) "Return a procedure that, given a package, applies the given REPLACEMENTS to -all the package graph (excluding implicit inputs). REPLACEMENTS is a list of -spec/procedures pair; each spec is a package specification such as \"gcc\" or -\"guile@2\", and each procedure takes a matching package and returns a -replacement for that package." +all the package graph, including implicit inputs unless DEEP? is false. +REPLACEMENTS is a list of spec/procedures pair; each spec is a package +specification such as \"gcc\" or \"guile@2\", and each procedure takes a +matching package and returns a replacement for that package." (define table (fold (lambda (replacement table) (match replacement @@ -1081,22 +1091,27 @@ replacement for that package." (package-name package) table)) - (define (rewrite package) - (match (find-replacement package) - (#f package) - (proc (proc package)))) + (define replacement-property + (gensym " package-replacement")) - (package-mapping rewrite find-replacement)) + (define (rewrite p) + (if (assq-ref (package-properties p) replacement-property) + p + (match (find-replacement p) + (#f p) + (proc + (let ((new (proc p))) + ;; Mark NEW as already processed. + (package/inherit new + (properties `((,replacement-property . #t) + ,@(package-properties new))))))))) -(define-syntax-rule (package/inherit p overrides ...) - "Like (package (inherit P) OVERRIDES ...), except that the same -transformation is done to the package replacement, if any. P must be a bare -identifier, and will be bound to either P or its replacement when evaluating -OVERRIDES." - (let loop ((p p)) - (package (inherit p) - overrides ... - (replacement (and=> (package-replacement p) loop))))) + (define (cut? p) + (or (assq-ref (package-properties p) replacement-property) + (find-replacement p))) + + (package-mapping rewrite cut? + #:deep? deep?)) ;;; diff --git a/tests/guix-build.sh b/tests/guix-build.sh index 6c08857358..ec2f736ccb 100644 --- a/tests/guix-build.sh +++ b/tests/guix-build.sh @@ -259,6 +259,17 @@ drv1=`guix build guile -d` drv2=`guix build guile --with-input=gimp=ruby -d` test "$drv1" = "$drv2" +# See . +drv1=`guix build glib -d` +drv2=`guix build glib -d --with-input=libreoffice=inkscape` +test "$drv1" = "$drv2" + +# Rewriting implicit inputs. +drv1=`guix build hello -d` +drv2=`guix build hello -d --with-input=gcc=gcc-toolchain` +test "$drv1" != "$drv2" +guix gc -R "$drv2" | grep `guix build -d gcc-toolchain` + if guix build guile --with-input=libunistring=something-really-silly then false; else true; fi diff --git a/tests/packages.scm b/tests/packages.scm index f33332a461..6fa4ad2f1b 100644 --- a/tests/packages.scm +++ b/tests/packages.scm @@ -38,6 +38,7 @@ #:use-module (guix build-system) #:use-module (guix build-system trivial) #:use-module (guix build-system gnu) + #:use-module (guix build-system python) #:use-module (guix memoization) #:use-module (guix profiles) #:use-module (guix scripts package) @@ -45,6 +46,7 @@ #:use-module (gnu packages base) #:use-module (gnu packages guile) #:use-module (gnu packages bootstrap) + #:use-module (gnu packages python) #:use-module (gnu packages version-control) #:use-module (gnu packages xml) #:use-module (srfi srfi-1) @@ -1262,7 +1264,8 @@ ("baz" ,dep))))) (rewrite (package-input-rewriting/spec `(("coreutils" . ,(const sed)) - ("grep" . ,(const findutils))))) + ("grep" . ,(const findutils))) + #:deep? #f)) (p1 (rewrite p0)) (p2 (rewrite p0))) (and (not (eq? p1 p0)) @@ -1279,7 +1282,11 @@ (match (package-native-inputs dep3) ((("x" dep)) (string=? (package-full-name dep) - (package-full-name findutils)))))))))) + (package-full-name findutils))))))) + + ;; Make sure implicit inputs were left unchanged. + (equal? (drop (bag-direct-inputs (package->bag p1)) 3) + (drop (bag-direct-inputs (package->bag p0)) 3))))) (test-assert "package-input-rewriting/spec, partial match" (let* ((dep (dummy-package "chbouib" @@ -1290,7 +1297,8 @@ ("bar" ,dep))))) (rewrite (package-input-rewriting/spec `(("chbouib@123" . ,(const sed)) ;not matched - ("grep" . ,(const findutils))))) + ("grep" . ,(const findutils))) + #:deep? #f)) (p1 (rewrite p0))) (and (not (eq? p1 p0)) (string=? "example" (package-name p1)) @@ -1304,6 +1312,58 @@ (string=? (package-full-name dep) (package-full-name findutils)))))))))) +(test-assert "package-input-rewriting/spec, deep" + (let* ((dep (dummy-package "chbouib")) + (p0 (dummy-package "example" + (build-system gnu-build-system) + (inputs `(("dep" ,dep))))) + (rewrite (package-input-rewriting/spec + `(("tar" . ,(const sed)) + ("gzip" . ,(const findutils))))) + (p1 (rewrite p0)) + (p2 (rewrite p0))) + (and (not (eq? p1 p0)) + (eq? p1 p2) ;memoization + (string=? "example" (package-name p1)) + (match (package-inputs p1) + ((("dep" dep1)) + (and (string=? (package-full-name dep1) + (package-full-name dep)) + (eq? dep1 (rewrite dep))))) ;memoization + + ;; Make sure implicit inputs were replaced. + (match (bag-direct-inputs (package->bag p1)) + ((("dep" dep1) ("tar" tar) ("gzip" gzip) _ ...) + (and (eq? dep1 (rewrite dep)) + (string=? (package-full-name tar) + (package-full-name sed)) + (string=? (package-full-name gzip) + (package-full-name findutils)))))))) + +(test-assert "package-input-rewriting/spec, no duplicates" + ;; Ensure that deep input rewriting does not forget implicit inputs. Doing + ;; so could lead to duplicates in a package's inputs: in the example below, + ;; P0's transitive inputs would contain one rewritten "python" and one + ;; original "python". These two "python" packages are thus not 'eq?' but + ;; they lower to the same derivation. See , + ;; which can be reproduced by passing #:deep? #f. + (let* ((dep0 (dummy-package "dep0" + (build-system trivial-build-system) + (propagated-inputs `(("python" ,python))))) + (p0 (dummy-package "chbouib" + (build-system python-build-system) + (arguments `(#:python ,python)) + (inputs `(("dep0" ,dep0))))) + (rewrite (package-input-rewriting/spec '() #:deep? #t)) + (p1 (rewrite p0)) + (bag1 (package->bag p1)) + (pythons (filter-map (match-lambda + (("python" python) python) + (_ #f)) + (bag-transitive-inputs bag1)))) + (match (delete-duplicates pythons eq?) + ((p) (eq? p (rewrite python)))))) + (test-equal "package-patched-vulnerabilities" '(("CVE-2015-1234") ("CVE-2016-1234" "CVE-2018-4567") diff --git a/tests/scripts-build.scm b/tests/scripts-build.scm index 32876e956a..3491610754 100644 --- a/tests/scripts-build.scm +++ b/tests/scripts-build.scm @@ -1,5 +1,5 @@ ;;; GNU Guix --- Functional package management for GNU -;;; Copyright © 2016, 2017, 2019 Ludovic Courtès +;;; Copyright © 2016, 2017, 2019, 2020 Ludovic Courtès ;;; ;;; This file is part of GNU Guix. ;;; @@ -19,6 +19,7 @@ (define-module (test-scripts-build) #:use-module (guix tests) #:use-module (guix store) + #:use-module (guix derivations) #:use-module (guix packages) #:use-module (guix git-download) #:use-module (guix scripts build) @@ -163,11 +164,16 @@ ((("foo" dep1) ("bar" dep2)) (and (string=? (package-full-name dep1) (package-full-name grep)) - (eq? (package-replacement dep1) findutils) + (string=? (package-full-name (package-replacement dep1)) + (package-full-name findutils)) (string=? (package-name dep2) "chbouib") (match (package-native-inputs dep2) ((("x" dep)) - (eq? (package-replacement dep) findutils))))))))))) + (with-store store + (string=? (derivation-file-name + (package-derivation store findutils)) + (derivation-file-name + (package-derivation store dep)))))))))))))) (test-equal "options->transformation, with-branch" (git-checkout (url "https://example.org") From patchwork Wed Sep 23 16:23:17 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Ludovic_Court=C3=A8s?= X-Patchwork-Id: 24264 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 0A45927BBE8; Wed, 23 Sep 2020 17:28:23 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_MSPIKE_H4,RCVD_IN_MSPIKE_WL autolearn=unavailable autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 9CA0E27BBE7 for ; Wed, 23 Sep 2020 17:28:22 +0100 (BST) Received: from localhost ([::1]:35798 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kL7cz-0007Br-PC for patchwork@mira.cbaines.net; Wed, 23 Sep 2020 12:28:21 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51630) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kL7Yq-000258-BU for guix-patches@gnu.org; Wed, 23 Sep 2020 12:24:04 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:54315) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kL7Yq-0000OB-25 for guix-patches@gnu.org; Wed, 23 Sep 2020 12:24:04 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1kL7Yp-0001ZF-VQ for guix-patches@gnu.org; Wed, 23 Sep 2020 12:24:03 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#43578] [PATCH 3/4] packages: 'package-mapping' correctly recurses into 'replacement'. Resent-From: Ludovic =?utf-8?q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Wed, 23 Sep 2020 16:24:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 43578 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 43578@debbugs.gnu.org Cc: Ludovic =?utf-8?q?Court=C3=A8s?= Received: via spool by 43578-submit@debbugs.gnu.org id=B43578.16008782175949 (code B ref 43578); Wed, 23 Sep 2020 16:24:03 +0000 Received: (at 43578) by debbugs.gnu.org; 23 Sep 2020 16:23:37 +0000 Received: from localhost ([127.0.0.1]:37624 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kL7YP-0001Xo-D2 for submit@debbugs.gnu.org; Wed, 23 Sep 2020 12:23:37 -0400 Received: from eggs.gnu.org ([209.51.188.92]:44666) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kL7YM-0001Wx-MV for 43578@debbugs.gnu.org; Wed, 23 Sep 2020 12:23:35 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:47178) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kL7YF-0000Jw-Vw; Wed, 23 Sep 2020 12:23:28 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=52578 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1kL7YF-0002gP-1w; Wed, 23 Sep 2020 12:23:27 -0400 From: Ludovic =?utf-8?q?Court=C3=A8s?= Date: Wed, 23 Sep 2020 18:23:17 +0200 Message-Id: <20200923162318.2800-3-ludo@gnu.org> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200923162318.2800-1-ludo@gnu.org> References: <20200923162318.2800-1-ludo@gnu.org> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches Previously, something like: guix build glib --with-graft=glibc=glibc@2.29 would produce a result showing that rewriting rules were not applied to libx11@1.6.A (a replacement). * guix/packages.scm (package-mapping): Call REPLACE instead of PROC to 'replacement'. * tests/packages.scm ("package-input-rewriting/spec, graft"): New test. --- guix/packages.scm | 2 +- tests/packages.scm | 27 +++++++++++++++++++++++++++ 2 files changed, 28 insertions(+), 1 deletion(-) diff --git a/guix/packages.scm b/guix/packages.scm index f696945e30..0d0d7492b6 100644 --- a/guix/packages.scm +++ b/guix/packages.scm @@ -1037,7 +1037,7 @@ applied to implicit inputs as well." (inputs (map rewrite (package-inputs p))) (native-inputs (map rewrite (package-native-inputs p))) (propagated-inputs (map rewrite (package-propagated-inputs p))) - (replacement (and=> (package-replacement p) proc)) + (replacement (and=> (package-replacement p) replace)) (properties `((,mapping-property . #t) ,@(package-properties p)))))))) diff --git a/tests/packages.scm b/tests/packages.scm index 6fa4ad2f1b..e31dea6f72 100644 --- a/tests/packages.scm +++ b/tests/packages.scm @@ -1364,6 +1364,33 @@ (match (delete-duplicates pythons eq?) ((p) (eq? p (rewrite python)))))) +(test-equal "package-input-rewriting/spec, graft" + (derivation-file-name (package-derivation %store sed)) + + ;; Make sure replacements are rewritten. + (let* ((dep0 (dummy-package "dep" + (version "1") + (build-system trivial-build-system) + (inputs `(("coreutils" ,coreutils))))) + (dep1 (dummy-package "dep" + (version "0") + (build-system trivial-build-system) + (replacement dep0))) + (p0 (dummy-package "p" + (build-system trivial-build-system) + (inputs `(("dep" ,dep1))))) + (rewrite (package-input-rewriting/spec + `(("coreutils" . ,(const sed))))) + (p1 (rewrite p0))) + (match (package-inputs p1) + ((("dep" dep)) + (match (package-inputs (package-replacement dep)) + ((("coreutils" coreutils)) + ;; COREUTILS is not 'eq?' to SED, so the most reliable way to check + ;; for equality is to lower to a derivation. + (derivation-file-name + (package-derivation %store coreutils)))))))) + (test-equal "package-patched-vulnerabilities" '(("CVE-2015-1234") ("CVE-2016-1234" "CVE-2018-4567") From patchwork Wed Sep 23 16:23:18 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Ludovic_Court=C3=A8s?= X-Patchwork-Id: 24263 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 9A10727BBE8; Wed, 23 Sep 2020 17:28:06 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_MSPIKE_H4,RCVD_IN_MSPIKE_WL autolearn=unavailable autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id DE33327BBE7 for ; Wed, 23 Sep 2020 17:28:05 +0100 (BST) Received: from localhost ([::1]:34992 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kL7cj-0006rz-3n for patchwork@mira.cbaines.net; Wed, 23 Sep 2020 12:28:05 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51624) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kL7Yq-00024P-24 for guix-patches@gnu.org; Wed, 23 Sep 2020 12:24:04 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:54314) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kL7Yp-0000O5-Ma for guix-patches@gnu.org; Wed, 23 Sep 2020 12:24:03 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1kL7Yp-0001Z8-JA for guix-patches@gnu.org; Wed, 23 Sep 2020 12:24:03 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#43578] [PATCH 4/4] packages: 'package-input-rewriting' has a #:deep? parameter. Resent-From: Ludovic =?utf-8?q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Wed, 23 Sep 2020 16:24:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 43578 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 43578@debbugs.gnu.org Cc: Ludovic =?utf-8?q?Court=C3=A8s?= Received: via spool by 43578-submit@debbugs.gnu.org id=B43578.16008782175943 (code B ref 43578); Wed, 23 Sep 2020 16:24:03 +0000 Received: (at 43578) by debbugs.gnu.org; 23 Sep 2020 16:23:37 +0000 Received: from localhost ([127.0.0.1]:37622 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kL7YO-0001Xh-UY for submit@debbugs.gnu.org; Wed, 23 Sep 2020 12:23:37 -0400 Received: from eggs.gnu.org ([209.51.188.92]:44662) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kL7YM-0001Ww-Ls for 43578@debbugs.gnu.org; Wed, 23 Sep 2020 12:23:35 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:47179) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kL7YG-0000Jx-Of; Wed, 23 Sep 2020 12:23:29 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=52578 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1kL7YG-0002gP-9w; Wed, 23 Sep 2020 12:23:28 -0400 From: Ludovic =?utf-8?q?Court=C3=A8s?= Date: Wed, 23 Sep 2020 18:23:18 +0200 Message-Id: <20200923162318.2800-4-ludo@gnu.org> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200923162318.2800-1-ludo@gnu.org> References: <20200923162318.2800-1-ludo@gnu.org> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches * guix/packages.scm (package-input-rewriting): Add #:deep? and pass it to 'package-mapping'. [replacement-property]: New variable. [rewrite]: Check it. [cut?]: New procedure. * tests/packages.scm ("package-input-rewriting"): Pass #:deep? #f and ensure implicit inputs were not rewritten. Avoid 'eq?' comparisons. ("package-input-rewriting, deep"): New test. * gnu/packages/guile.scm (package-for-guile-2.0, package-for-guile-3.0): Pass #:deep? #f. --- doc/guix.texi | 10 +++++----- gnu/packages/guile.scm | 6 ++++-- guix/packages.scm | 35 +++++++++++++++++++++++++---------- tests/packages.scm | 20 ++++++++++++++++++-- 4 files changed, 52 insertions(+), 19 deletions(-) diff --git a/doc/guix.texi b/doc/guix.texi index e9e1d122ab..193529bbb1 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -6236,12 +6236,12 @@ transformation is @dfn{input rewriting}, whereby the dependency tree of a package is rewritten by replacing specific inputs by others: @deffn {Scheme Procedure} package-input-rewriting @var{replacements} @ - [@var{rewrite-name}] + [@var{rewrite-name}] [#:deep? #t] Return a procedure that, when passed a package, replaces its direct and -indirect dependencies (but not its implicit inputs) according to -@var{replacements}. @var{replacements} is a list of package pairs; the -first element of each pair is the package to replace, and the second one -is the replacement. +indirect dependencies, including implicit inputs when @var{deep?} is +true, according to @var{replacements}. @var{replacements} is a list of +package pairs; the first element of each pair is the package to replace, +and the second one is the replacement. Optionally, @var{rewrite-name} is a one-argument procedure that takes the name of a package and returns its new name after rewrite. diff --git a/gnu/packages/guile.scm b/gnu/packages/guile.scm index c59daeebe2..280053bf06 100644 --- a/gnu/packages/guile.scm +++ b/gnu/packages/guile.scm @@ -420,11 +420,13 @@ GNU@tie{}Guile. Use the @code{(ice-9 readline)} module and call its ;; A procedure that rewrites the dependency tree of the given package to use ;; GUILE-2.0 instead of GUILE-3.0. (package-input-rewriting `((,guile-3.0 . ,guile-2.0)) - (guile-variant-package-name "guile2.0"))) + (guile-variant-package-name "guile2.0") + #:deep? #f)) (define package-for-guile-2.2 (package-input-rewriting `((,guile-3.0 . ,guile-2.2)) - (guile-variant-package-name "guile2.2"))) + (guile-variant-package-name "guile2.2") + #:deep? #f)) (define-syntax define-deprecated-guile3.0-package (lambda (s) diff --git a/guix/packages.scm b/guix/packages.scm index 0d0d7492b6..4f2bb432be 100644 --- a/guix/packages.scm +++ b/guix/packages.scm @@ -1044,22 +1044,37 @@ applied to implicit inputs as well." replace) (define* (package-input-rewriting replacements - #:optional (rewrite-name identity)) + #:optional (rewrite-name identity) + #:key (deep? #t)) "Return a procedure that, when passed a package, replaces its direct and -indirect dependencies (but not its implicit inputs) according to REPLACEMENTS. -REPLACEMENTS is a list of package pairs; the first element of each pair is the -package to replace, and the second one is the replacement. +indirect dependencies, including implicit inputs when DEEP? is true, according +to REPLACEMENTS. REPLACEMENTS is a list of package pairs; the first element +of each pair is the package to replace, and the second one is the replacement. Optionally, REWRITE-NAME is a one-argument procedure that takes the name of a package and returns its new name after rewrite." + (define replacement-property + ;; Property to tag right-hand sides in REPLACEMENTS. + (gensym " package-replacement")) + (define (rewrite p) - (match (assq-ref replacements p) - (#f (package - (inherit p) - (name (rewrite-name (package-name p))))) - (new new))) + (if (assq-ref (package-properties p) replacement-property) + p + (match (assq-ref replacements p) + (#f (package/inherit p + (name (rewrite-name (package-name p))))) + (new (if deep? + (package/inherit new + (properties `((,replacement-property . #t) + ,@(package-properties new)))) + new))))) - (package-mapping rewrite (cut assq <> replacements))) + (define (cut? p) + (or (assq-ref (package-properties p) replacement-property) + (assq-ref replacements p))) + + (package-mapping rewrite cut? + #:deep? deep?)) (define* (package-input-rewriting/spec replacements #:key (deep? #t)) "Return a procedure that, given a package, applies the given REPLACEMENTS to diff --git a/tests/packages.scm b/tests/packages.scm index e31dea6f72..af8941c2e2 100644 --- a/tests/packages.scm +++ b/tests/packages.scm @@ -1239,7 +1239,8 @@ ("baz" ,dep))))) (rewrite (package-input-rewriting `((,coreutils . ,sed) (,grep . ,findutils)) - (cut string-append "r-" <>))) + (cut string-append "r-" <>) + #:deep? #f)) (p1 (rewrite p0)) (p2 (rewrite p0))) (and (not (eq? p1 p0)) @@ -1253,7 +1254,22 @@ (eq? dep3 (rewrite dep)) ;memoization (match (package-native-inputs dep3) ((("x" dep)) - (eq? dep findutils))))))))) + (eq? dep findutils)))))) + + ;; Make sure implicit inputs were left unchanged. + (equal? (drop (bag-direct-inputs (package->bag p1)) 3) + (drop (bag-direct-inputs (package->bag p0)) 3))))) + +(test-eq "package-input-rewriting, deep" + (derivation-file-name (package-derivation %store sed)) + (let* ((p0 (dummy-package "chbouib" + (build-system python-build-system) + (arguments `(#:python ,python)))) + (rewrite (package-input-rewriting `((,python . ,sed)))) + (p1 (rewrite p0))) + (match (bag-direct-inputs (package->bag p1)) + ((("python" python) _ ...) + (derivation-file-name (package-derivation %store python)))))) (test-assert "package-input-rewriting/spec" (let* ((dep (dummy-package "chbouib"