From patchwork Fri Sep 11 05:16:28 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jesse Gibbons X-Patchwork-Id: 23992 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 5739327BBE7; Fri, 11 Sep 2020 06:17:09 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H4, RCVD_IN_MSPIKE_WL,T_DKIM_INVALID,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id A520727BBE8 for ; Fri, 11 Sep 2020 06:17:08 +0100 (BST) Received: from localhost ([::1]:50464 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kGbQp-0005qc-TD for patchwork@mira.cbaines.net; Fri, 11 Sep 2020 01:17:07 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51368) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kGbQk-0005qH-Lb for guix-patches@gnu.org; Fri, 11 Sep 2020 01:17:02 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:58721) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kGbQk-0001nS-Cs for guix-patches@gnu.org; Fri, 11 Sep 2020 01:17:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1kGbQk-0005nd-6Y for guix-patches@gnu.org; Fri, 11 Sep 2020 01:17:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#43193] [PATCH] guix: Add --with-dependency-source option Resent-From: Jesse Gibbons Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Fri, 11 Sep 2020 05:17:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 43193 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Ludovic =?utf-8?q?Court=C3=A8s?= Cc: 43193@debbugs.gnu.org Received: via spool by 43193-submit@debbugs.gnu.org id=B43193.159980141822281 (code B ref 43193); Fri, 11 Sep 2020 05:17:02 +0000 Received: (at 43193) by debbugs.gnu.org; 11 Sep 2020 05:16:58 +0000 Received: from localhost ([127.0.0.1]:42034 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kGbQf-0005nI-NV for submit@debbugs.gnu.org; Fri, 11 Sep 2020 01:16:58 -0400 Received: from mail-pf1-f194.google.com ([209.85.210.194]:34171) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kGbQe-0005n5-57 for 43193@debbugs.gnu.org; Fri, 11 Sep 2020 01:16:56 -0400 Received: by mail-pf1-f194.google.com with SMTP id v196so6504898pfc.1 for <43193@debbugs.gnu.org>; Thu, 10 Sep 2020 22:16:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language; bh=xLILgtYB5U7mNFbrseGmfu+0IeQ96pscWWvGULjwTmY=; b=QC7FXqzPEnw4KtRzxkypKv3Cpg9zCpsEbvGIUbSUsmYLoPJtNLWUJwuIyYxGmF+Xa9 PQX4CuqDy5/CCfhtMWYogLDO6S01cgShmefXTiCjYM5c1dlpqkMRoa7raZLdGjbvH0JT vvtyqmnauoMeCdOhHjkZoaBOBxJ8xXbpXVt0KwvYnaOuIfXVnRvboIsrpd5z37pbVqO/ CmNqaKR0OGWuYQ3XCmpfT5a/XHkKT/n2B6EC6gpM0NdTvgmRH+O9xPd+unyfAjWR/Ona HainYhZpfIXrYRv3aRHlVMMlEs4P4BRhtlFZ3/HVHpuBWOOPkRLK3Y34pKULx5eKzka6 Mwbg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language; bh=xLILgtYB5U7mNFbrseGmfu+0IeQ96pscWWvGULjwTmY=; b=anekytNYn8w1FJZ3buQVJx9Ej5RQzjTnlDA5CWsPU72EpQ6uMm04oKmmUnoniGxFgx hQo/gtebT3zM/xtf44RnrOsLxQMHrdqs8jKWqa02dMTrqFf3sjkzA7r6WQ+87TghysZY AmU+9cTL7YC59Fzie1yneEb4JfJj2gwZmNr3Ov+2wobc51bUH/XN/paKgqswwD7TuKkI /KyZddz+W2/qQRkgRcChCZ+76LzMSLPYkoMFFvaeAer6nkeqfXehHzTacOiVBrQ5jtfD /TOwsmEy42gFM1giginplw9iQj6splTQ5kVpr8MtVybvL+VbcWHGu+nu9t5uqv0Kt8bN 7d0A== X-Gm-Message-State: AOAM5303hobN0WGy0Du9sLAc9ldR4sl+NT8lJEW2bZ0uuulxII2AuvwH ijCO3kgb+gB9SnEluTHR9nqKBCmdfd60Vg== X-Google-Smtp-Source: ABdhPJyWhXEVx17lXTvEKre8I6i84OfkIP7+cEfBixXIkYYxkGFgNHeEWpwamUu6DFplZ3AphWVNwQ== X-Received: by 2002:a63:fc18:: with SMTP id j24mr445875pgi.452.1599801409763; Thu, 10 Sep 2020 22:16:49 -0700 (PDT) Received: from [192.168.1.25] ([38.141.58.134]) by smtp.gmail.com with ESMTPSA id ca6sm642804pjb.53.2020.09.10.22.16.48 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 10 Sep 2020 22:16:48 -0700 (PDT) References: <9c3a00ba-19bf-a8e9-8803-d150e91f1b6e@gmail.com> <87y2lige5l.fsf@gnu.org> From: Jesse Gibbons Message-ID: <7f52c72f-c280-b585-d1ad-ca012f804910@gmail.com> Date: Thu, 10 Sep 2020 23:16:28 -0600 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Icedove/68.12.0 MIME-Version: 1.0 In-Reply-To: <87y2lige5l.fsf@gnu.org> Content-Language: en-US X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches On 9/10/20 3:43 AM, Ludovic Courtès wrote: > Hi Jesse, > > Jesse Gibbons skribis: > >>     The attached patch adds support for the --with-dependency-source >> common build option. It can be used to specify the sources of >> dependencies of specified packages. With this step, we can close >> #42155. This is also a step in closing #43061. > Excellent! > >> Note that I suggested making a new >> --with-dependency-source=package=source build option in response to >> #42155 and nobody responded with an objection. So I am sending this >> patch with the assumption that there are no objections to this new >> build option, and that if a member of the community wants to >> completely replace the behavior of --with-source with the behavior of >> the new option, that person can do the work required to not break >> --with-source=source. > OK. Like I wrote at , I wouldn’t > mind simply calling this new option ‘--with-source’. What we’d lose by > doing so is the warning you get if you do > ‘--with-source=does-not-exist=whatever’, saying the option had no > effect, but that’s about it. The new ‘--with-source’ behavior > (recursive) would be consistent with the other options, which, to me, is > more important. I agree that '--with-source' is a better name for the option, but I don't want to lose that particular functionality. I worked a little more to alter "--with-source" while still preserving the simple '--with-source=source' option, because once it's committed to master, it will be difficult to turn back and get the ideal implementation. The result is a bit dirty and should be refactored and cleaned, but at least it works. Attached is the updated patch. > > WDYT? > >> >From 91a89277067fd454ad77edb3a09ed06382f3694c Mon Sep 17 00:00:00 2001 >> From: Jesse Gibbons >> Date: Thu, 3 Sep 2020 17:45:08 -0600 >> Subject: [PATCH v1 1/1] guix: Add --with-dependency-source option >> >> * guix/scripts/build.scm: (transform-package-inputs/source): new >> function >> (evaluate-source-replacement-specs): new function >> (%transformations): add with-dependency-source option >> (%transformation-options): add with-dependency-source-option >> (show-transformation-options-help): document --with-dependency-source > [...] > >> +(define (evaluate-source-replacement-specs specs proc) >> + "Parse SPECS, a list of strings like \"guile=/path/to/source\", and return a >> +list of package pairs, where (PROC PACKAGE URL) returns the replacement package. >> +Raise an error if an element of SPECS uses invalid syntax, or if a package it >> +refers to could not be found." >> + (map (lambda (spec) >> + (match (string-tokenize spec %not-equal) >> + ((spec url) >> + (define (replace old) >> + (proc old url)) >> + (cons spec replace)) >> + (x >> + (leave (G_ "invalid replacement specification: ~s~%") spec)))) > ^ > Add “source” here. It’s always a good idea to not have the exact same > error message in several places. :-) Fixed. > Could you: > > 1. adjust doc/guix.texi accordingly? That is, if we rename this new > option to ‘--with-source’, simply add a note stating that it’s > recursive. I included this in the attached patch. > 2. add a test to tests/guix-build.sh? There are already --with-source > tests in other files. You can mimic them, essentially to make sure > the option has an effect. > 3. optionally add an entry as a separate commit to > etc/news.scm. I can do that for you if you want. > Do you still think the tests should be updated and this change should be announced in the news file? I'm willing to do these. > Thanks! > > Ludo’. -Jesse From 2786da1e7011c59f08fc150dfa284f35bc0ed093 Mon Sep 17 00:00:00 2001 From: Jesse Gibbons Date: Thu, 3 Sep 2020 17:45:08 -0600 Subject: [PATCH 1/1] guix: Make --with-source option recursive * guix/scripts/build.scm: (transform-package-inputs/source): new function (evaluate-source-replacement-specs): new function (%transformations): change with-source to use evaluate-source-replacement-specs *doc/guix.texi (Package Transformation Options): Document it. --- doc/guix.texi | 3 ++- guix/scripts/build.scm | 39 ++++++++++++++++++++++++++++++++++++++- 2 files changed, 40 insertions(+), 2 deletions(-) diff --git a/doc/guix.texi b/doc/guix.texi index 1d6782e6fa..4036861c23 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -9129,7 +9129,8 @@ without having to type in the definitions of package variants @itemx --with-source=@var{package}=@var{source} @itemx --with-source=@var{package}@@@var{version}=@var{source} Use @var{source} as the source of @var{package}, and @var{version} as -its version number. +its version number. This replacement is applied recursively on all +dependencies only if PACKAGE is specified. @var{source} must be a file name or a URL, as for @command{guix download} (@pxref{Invoking guix download}). diff --git a/guix/scripts/build.scm b/guix/scripts/build.scm index 6286a43c02..095457b174 100644 --- a/guix/scripts/build.scm +++ b/guix/scripts/build.scm @@ -280,6 +280,28 @@ current 'gnutls' package, after which version 3.5.4 is grafted onto them." (rewrite obj) obj)))) +(define (transform-package-inputs/source replacement-specs) + "Return a procedure that, when passed a package, replaces its direct +dependencies according to REPLACEMENT-SPECS. REPLACEMENT-SPECS is a list of +strings like \"guile=/path/to/source\" or +\"guile=https://www.example.com/guile-source.tar.gz\" meaning that, any +dependency on a package called \"guile\" must be replaced with a dependency on a +\"guile\" built with the source at the specified location." + (match (string-tokenize (car replacement-specs) %not-equal) + ((spec url) + (lambda (store obj) + (let* ((replacements (evaluate-source-replacement-specs replacement-specs + (lambda (old url) + (package-with-source store old url)))) + (rewrite (package-input-rewriting/spec replacements)) + (rewrite* (lambda (obj) + (rewrite obj)))) + (if (package? obj) + (rewrite* obj) + obj)))) + ((url) + (transform-package-source replacement-specs)))) + (define %not-equal (char-set-complement (char-set #\=))) @@ -314,6 +336,21 @@ syntax, or if a package it refers to could not be found." (leave (G_ "invalid replacement specification: ~s~%") spec)))) specs)) +(define (evaluate-source-replacement-specs specs proc) + "Parse SPECS, a list of strings like \"guile=/path/to/source\", and return a +list of package pairs, where (PROC PACKAGE URL) returns the replacement package. +Raise an error if an element of SPECS uses invalid syntax, or if a package it +refers to could not be found." + (map (lambda (spec) + (match (string-tokenize spec %not-equal) + ((spec url) + (define (replace old) + (proc old url)) + (cons spec replace)) + (x + (leave (G_ "invalid source replacement specification: ~s~%") spec)))) + specs)) + (define (transform-package-source-branch replacement-specs) "Return a procedure that, when passed a package, replaces its direct dependencies according to REPLACEMENT-SPECS. REPLACEMENT-SPECS is a list of @@ -398,7 +435,7 @@ a checkout of the Git repository at the given URL." ;; key used in the option alist, and the cdr is the transformation ;; procedure; it is called with two arguments: the store, and a list of ;; things to build. - `((with-source . ,transform-package-source) + `((with-source . ,transform-package-inputs/source) (with-input . ,transform-package-inputs) (with-graft . ,transform-package-inputs/graft) (with-branch . ,transform-package-source-branch) -- 2.28.0