From patchwork Wed Sep 23 16:23:15 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Ludovic_Court=C3=A8s?= X-Patchwork-Id: 24261 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id BB08927BBE8; Wed, 23 Sep 2020 17:27:47 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_MSPIKE_H4,RCVD_IN_MSPIKE_WL autolearn=unavailable autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id F26B727BBE7 for ; Wed, 23 Sep 2020 17:27:46 +0100 (BST) Received: from localhost ([::1]:33304 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kL7cP-00067h-Vi for patchwork@mira.cbaines.net; Wed, 23 Sep 2020 12:27:45 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51618) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kL7Yp-00022E-6V for guix-patches@gnu.org; Wed, 23 Sep 2020 12:24:03 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:54312) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kL7Yo-0000Nu-SF for guix-patches@gnu.org; Wed, 23 Sep 2020 12:24:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1kL7Yo-0001Yt-PP for guix-patches@gnu.org; Wed, 23 Sep 2020 12:24:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#43578] [PATCH 1/4] packages: 'package-mapping' can recurse on implicit inputs. References: <20200923161253.2378-1-ludo@gnu.org> In-Reply-To: <20200923161253.2378-1-ludo@gnu.org> Resent-From: Ludovic =?utf-8?q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Wed, 23 Sep 2020 16:24:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 43578 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 43578@debbugs.gnu.org Cc: Ludovic =?utf-8?q?Court=C3=A8s?= Received: via spool by 43578-submit@debbugs.gnu.org id=B43578.16008782135912 (code B ref 43578); Wed, 23 Sep 2020 16:24:02 +0000 Received: (at 43578) by debbugs.gnu.org; 23 Sep 2020 16:23:33 +0000 Received: from localhost ([127.0.0.1]:37616 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kL7YK-0001XD-Lt for submit@debbugs.gnu.org; Wed, 23 Sep 2020 12:23:33 -0400 Received: from eggs.gnu.org ([209.51.188.92]:44652) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kL7YI-0001Wq-Qs for 43578@debbugs.gnu.org; Wed, 23 Sep 2020 12:23:31 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:47175) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kL7YD-0000Is-Cx; Wed, 23 Sep 2020 12:23:25 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=52578 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1kL7YB-0002gP-NW; Wed, 23 Sep 2020 12:23:24 -0400 From: Ludovic =?utf-8?q?Court=C3=A8s?= Date: Wed, 23 Sep 2020 18:23:15 +0200 Message-Id: <20200923162318.2800-1-ludo@gnu.org> X-Mailer: git-send-email 2.28.0 MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches * guix/packages.scm (build-system-with-package-mapping): New procedure. (package-mapping): Add #:deep? and honor it. * tests/packages.scm ("package-mapping"): Compare the direct inputs of the bag of P0 and that of P1. ("package-mapping, deep"): New test. --- doc/guix.texi | 5 ++-- guix/packages.scm | 65 +++++++++++++++++++++++++++++++++++----------- tests/packages.scm | 36 ++++++++++++++++++++++++- 3 files changed, 88 insertions(+), 18 deletions(-) diff --git a/doc/guix.texi b/doc/guix.texi index f7e2204b53..4595008c4f 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -6294,10 +6294,11 @@ A more generic procedure to rewrite a package dependency graph is @code{package-mapping}: it supports arbitrary changes to nodes in the graph. -@deffn {Scheme Procedure} package-mapping @var{proc} [@var{cut?}] +@deffn {Scheme Procedure} package-mapping @var{proc} [@var{cut?}] [#:deep? #f] Return a procedure that, given a package, applies @var{proc} to all the packages depended on and returns the resulting package. The procedure stops recursion -when @var{cut?} returns true for a given package. +when @var{cut?} returns true for a given package. When @var{deep?} is true, @var{proc} is +applied to implicit inputs as well. @end deffn @menu diff --git a/guix/packages.scm b/guix/packages.scm index 6598bd3149..171fd048ef 100644 --- a/guix/packages.scm +++ b/guix/packages.scm @@ -968,10 +968,31 @@ packages they depend on, recursively." (vhash-consq package #t visited) (fold set-insert closure dependencies)))))))) -(define* (package-mapping proc #:optional (cut? (const #f))) +(define (build-system-with-package-mapping bs rewrite) + "Return a variant of BS, a build system, that rewrites a bag's inputs by +passing them through REWRITE, a procedure that takes an input tuplet and +returns a \"rewritten\" input tuplet." + (define lower + (build-system-lower bs)) + + (define (lower* . args) + (let ((lowered (apply lower args))) + (bag + (inherit lowered) + (build-inputs (map rewrite (bag-build-inputs lowered))) + (host-inputs (map rewrite (bag-host-inputs lowered))) + (target-inputs (map rewrite (bag-target-inputs lowered)))))) + + (build-system + (inherit bs) + (lower lower*))) + +(define* (package-mapping proc #:optional (cut? (const #f)) + #:key deep?) "Return a procedure that, given a package, applies PROC to all the packages depended on and returns the resulting package. The procedure stops recursion -when CUT? returns true for a given package." +when CUT? returns true for a given package. When DEEP? is true, PROC is +applied to implicit inputs as well." (define (rewrite input) (match input ((label (? package? package) outputs ...) @@ -980,21 +1001,35 @@ when CUT? returns true for a given package." (_ input))) + (define mapping-property + ;; Property indicating whether the package has already been processed. + (gensym " package-mapping-done")) + (define replace (mlambdaq (p) - ;; Return a variant of P with PROC applied to P and its explicit - ;; dependencies, recursively. Memoize the transformations. Failing to - ;; do that, we would build a huge object graph with lots of duplicates, - ;; which in turns prevents us from benefiting from memoization in - ;; 'package-derivation'. - (let ((p (proc p))) - (package - (inherit p) - (location (package-location p)) - (inputs (map rewrite (package-inputs p))) - (native-inputs (map rewrite (package-native-inputs p))) - (propagated-inputs (map rewrite (package-propagated-inputs p))) - (replacement (and=> (package-replacement p) proc)))))) + ;; If P is the result of a previous call, return it. + (if (assq-ref (package-properties p) mapping-property) + p + + ;; Return a variant of P with PROC applied to P and its explicit + ;; dependencies, recursively. Memoize the transformations. Failing + ;; to do that, we would build a huge object graph with lots of + ;; duplicates, which in turns prevents us from benefiting from + ;; memoization in 'package-derivation'. + (let ((p (proc p))) + (package + (inherit p) + (location (package-location p)) + (build-system (if deep? + (build-system-with-package-mapping + (package-build-system p) rewrite) + (package-build-system p))) + (inputs (map rewrite (package-inputs p))) + (native-inputs (map rewrite (package-native-inputs p))) + (propagated-inputs (map rewrite (package-propagated-inputs p))) + (replacement (and=> (package-replacement p) proc)) + (properties `((,mapping-property . #t) + ,@(package-properties p)))))))) replace) diff --git a/tests/packages.scm b/tests/packages.scm index cbd0503733..f33332a461 100644 --- a/tests/packages.scm +++ b/tests/packages.scm @@ -1172,15 +1172,24 @@ (let* ((dep (dummy-package "chbouib" (native-inputs `(("x" ,grep))))) (p0 (dummy-package "example" + (source 77) (inputs `(("foo" ,coreutils) ("bar" ,grep) ("baz" ,dep))))) (transform (lambda (p) (package (inherit p) (source 42)))) (rewrite (package-mapping transform)) - (p1 (rewrite p0))) + (p1 (rewrite p0)) + (bag0 (package->bag p0)) + (bag1 (package->bag p1))) (and (eq? p1 (rewrite p0)) (eqv? 42 (package-source p1)) + + ;; Implicit inputs should be left unchanged (skip "source", "foo", + ;; "bar", and "baz" in this comparison). + (equal? (drop (bag-direct-inputs bag0) 4) + (drop (bag-direct-inputs bag1) 4)) + (match (package-inputs p1) ((("foo" dep1) ("bar" dep2) ("baz" dep3)) (and (eq? dep1 (rewrite coreutils)) ;memoization @@ -1194,6 +1203,31 @@ (and (eq? dep (rewrite grep)) (package-source dep)))))))))) +(test-equal "package-mapping, deep" + '(42) + (let* ((p0 (dummy-package "example" + (inputs `(("foo" ,coreutils) + ("bar" ,grep))))) + (transform (lambda (p) + (package (inherit p) (source 42)))) + (rewrite (package-mapping transform #:deep? #t)) + (p1 (rewrite p0)) + (bag (package->bag p1))) + (and (eq? p1 (rewrite p0)) + (match (bag-direct-inputs bag) + ((("source" 42) ("foo" dep1) ("bar" dep2) rest ..1) + (and (eq? dep1 (rewrite coreutils)) ;memoization + (eq? dep2 (rewrite grep)) + (= 42 (package-source dep1)) + (= 42 (package-source dep2)) + + ;; Check that implicit inputs of P0 also got rewritten. + (delete-duplicates + (map (match-lambda + ((_ package . _) + (package-source package))) + rest)))))))) + (test-assert "package-input-rewriting" (let* ((dep (dummy-package "chbouib" (native-inputs `(("x" ,grep)))))