From patchwork Wed Mar 13 10:47:46 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Ludovic_Court=C3=A8s?= X-Patchwork-Id: 1414 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 5338016CED; Wed, 13 Mar 2019 11:05:06 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTP id 8969516CBB for ; Wed, 13 Mar 2019 11:05:05 +0000 (GMT) Received: from localhost ([127.0.0.1]:42298 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1h41h3-0006qv-37 for patchwork@mira.cbaines.net; Wed, 13 Mar 2019 07:05:05 -0400 Received: from eggs.gnu.org ([209.51.188.92]:42807) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1h41gj-0006cY-Si for guix-patches@gnu.org; Wed, 13 Mar 2019 07:04:52 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1h41RW-0000Xp-MV for guix-patches@gnu.org; Wed, 13 Mar 2019 06:49:04 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:55789) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1h41RW-0000Xa-Fh for guix-patches@gnu.org; Wed, 13 Mar 2019 06:49:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1h41RW-0002Qe-DW for guix-patches@gnu.org; Wed, 13 Mar 2019 06:49:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#34838] [PATCH 1/6] guix build: Add '--with-git-url'. References: <20190313095755.15378-1-ludo@gnu.org> In-Reply-To: <20190313095755.15378-1-ludo@gnu.org> Resent-From: Ludovic =?utf-8?q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Wed, 13 Mar 2019 10:49:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 34838 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 34838@debbugs.gnu.org Received: via spool by 34838-submit@debbugs.gnu.org id=B34838.15524740859236 (code B ref 34838); Wed, 13 Mar 2019 10:49:02 +0000 Received: (at 34838) by debbugs.gnu.org; 13 Mar 2019 10:48:05 +0000 Received: from localhost ([127.0.0.1]:41095 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1h41Qa-0002Oo-OX for submit@debbugs.gnu.org; Wed, 13 Mar 2019 06:48:05 -0400 Received: from hera.aquilenet.fr ([185.233.100.1]:38394) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1h41QW-0002NO-Ex for 34838@debbugs.gnu.org; Wed, 13 Mar 2019 06:48:01 -0400 Received: from localhost (localhost [127.0.0.1]) by hera.aquilenet.fr (Postfix) with ESMTP id C6A64D625; Wed, 13 Mar 2019 11:47:58 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at aquilenet.fr Received: from hera.aquilenet.fr ([127.0.0.1]) by localhost (hera.aquilenet.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id GyDAaYmxVfcR; Wed, 13 Mar 2019 11:47:56 +0100 (CET) Received: from gnu.org (unknown [IPv6:2a01:e0a:1d:7270:af76:b9b:ca24:c465]) by hera.aquilenet.fr (Postfix) with ESMTPSA id D00DFD61D; Wed, 13 Mar 2019 11:47:55 +0100 (CET) From: Ludovic =?utf-8?q?Court=C3=A8s?= Date: Wed, 13 Mar 2019 11:47:46 +0100 Message-Id: <20190313104751.20758-1-ludo@gnu.org> X-Mailer: git-send-email 2.21.0 MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.51.188.43 X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Ludovic =?utf-8?q?Court=C3=A8s?= Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches From: Ludovic Courtès * guix/scripts/build.scm (%not-equal): New variable. (evaluate-git-replacement-specs): Use it instead of local variable 'not-equal'. (transform-package-source-git-url): New procedure. (%transformations): Add 'with-git-url'. (%transformation-options, show-transformation-options-help): Add '--with-git-url'. * tests/scripts-build.scm ("options->transformation, with-git-url"): New test. --- doc/guix.texi | 36 ++++++++++++++++++++----------- guix/scripts/build.scm | 47 +++++++++++++++++++++++++++++++++++------ tests/scripts-build.scm | 24 ++++++++++++++++++++- 3 files changed, 87 insertions(+), 20 deletions(-) diff --git a/doc/guix.texi b/doc/guix.texi index 82cf2babb2..42885577be 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -7694,9 +7694,32 @@ must be compatible. If @var{replacement} is somehow incompatible with @var{package}, then the resulting package may be unusable. Use with care! -@item --with-branch=@var{package}=@var{branch} +@item --with-git-url=@var{package}=@var{url} @cindex Git, using the latest commit @cindex latest commit, building +Build @var{package} from the latest commit of the @code{master} branch of the +Git repository at @var{url}. + +For example, the following commands builds the GNU C Library (glibc) straight +from its Git repository instead of building the currently-packaged release: + +@example +guix build glibc \ + --with-git-url=glibc=git://sourceware.org/git/glibc.git +@end example + +@cindex continuous integration +Obviously, since it uses the latest commit of the given branch, the result of +such a command varies over time. Nevertheless it is a convenient way to +rebuild entire software stacks against the latest commit of one or more +packages. This is particularly useful in the context of continuous +integration (CI). + +Checkouts are kept in a cache under @file{~/.cache/guix/checkouts} to speed up +consecutive accesses to the same repository. You may want to clean it up once +in a while to save disk space. + +@item --with-branch=@var{package}=@var{branch} Build @var{package} from the latest commit of @var{branch}. The @code{source} field of @var{package} must be an origin with the @code{git-fetch} method (@pxref{origin Reference}) or a @code{git-checkout} object; the repository URL @@ -7712,17 +7735,6 @@ specific @code{guile-sqlite3} build: guix build --with-branch=guile-sqlite3=master cuirass @end example -@cindex continuous integration -Obviously, since it uses the latest commit of the given branch, the result of -such a command varies over time. Nevertheless it is a convenient way to -rebuild entire software stacks against the latest commit of one or more -packages. This is particularly useful in the context of continuous -integration (CI). - -Checkouts are kept in a cache under @file{~/.cache/guix/checkouts} to speed up -consecutive accesses to the same repository. You may want to clean it up once -in a while to save disk space. - @item --with-commit=@var{package}=@var{commit} This is similar to @code{--with-branch}, except that it builds from @var{commit} rather than the tip of a branch. @var{commit} must be a valid diff --git a/guix/scripts/build.scm b/guix/scripts/build.scm index 6b29c470fb..5883dbfb44 100644 --- a/guix/scripts/build.scm +++ b/guix/scripts/build.scm @@ -272,16 +272,16 @@ current 'gnutls' package, after which version 3.5.4 is grafted onto them." (rewrite obj) obj)))) +(define %not-equal + (char-set-complement (char-set #\=))) + (define (evaluate-git-replacement-specs specs proc) "Parse SPECS, a list of strings like \"guile=stable-2.2\", and return a list of package pairs, where (PROC PACKAGE URL BRANCH-OR-COMMIT) returns the replacement package. Raise an error if an element of SPECS uses invalid syntax, or if a package it refers to could not be found." - (define not-equal - (char-set-complement (char-set #\=))) - (map (lambda (spec) - (match (string-tokenize spec not-equal) + (match (string-tokenize spec %not-equal) ((name branch-or-commit) (let* ((old (specification->package name)) (source (package-source old)) @@ -341,6 +341,33 @@ strings like \"guile-next=cabba9e\" meaning that packages are built using (rewrite obj) obj)))) +(define (transform-package-source-git-url replacement-specs) + "Return a procedure that, when passed a package, replaces its dependencies +according to REPLACEMENT-SPECS. REPLACEMENT-SPECS is a list of strings like +\"guile-json=https://gitthing.com/…\" meaning that packages are built using +a checkout of the Git repository at the given URL." + ;; FIXME: Currently this cannot be combined with '--with-branch' or + ;; '--with-commit' because they all transform "from scratch". + (define replacements + (map (lambda (spec) + (match (string-tokenize spec %not-equal) + ((name url) + (let* ((old (specification->package name)) + (new (package + (inherit old) + (source (git-checkout (url url) + (recursive? #t)))))) + (cons old new))))) + replacement-specs)) + + (define rewrite + (package-input-rewriting replacements)) + + (lambda (store obj) + (if (package? obj) + (rewrite obj) + obj))) + (define %transformations ;; Transformations that can be applied to things to build. The car is the ;; key used in the option alist, and the cdr is the transformation @@ -350,7 +377,8 @@ strings like \"guile-next=cabba9e\" meaning that packages are built using (with-input . ,transform-package-inputs) (with-graft . ,transform-package-inputs/graft) (with-branch . ,transform-package-source-branch) - (with-commit . ,transform-package-source-commit))) + (with-commit . ,transform-package-source-commit) + (with-git-url . ,transform-package-source-git-url))) (define %transformation-options ;; The command-line interface to the above transformations. @@ -368,7 +396,9 @@ strings like \"guile-next=cabba9e\" meaning that packages are built using (option '("with-branch") #t #f (parser 'with-branch)) (option '("with-commit") #t #f - (parser 'with-commit))))) + (parser 'with-commit)) + (option '("with-git-url") #t #f + (parser 'with-git-url))))) (define (show-transformation-options-help) (display (G_ " @@ -385,7 +415,10 @@ strings like \"guile-next=cabba9e\" meaning that packages are built using build PACKAGE from the latest commit of BRANCH")) (display (G_ " --with-commit=PACKAGE=COMMIT - build PACKAGE from COMMIT"))) + build PACKAGE from COMMIT")) + (display (G_ " + --with-git-url=PACKAGE=URL + build PACKAGE from the repository at URL"))) (define (options->transformation opts) diff --git a/tests/scripts-build.scm b/tests/scripts-build.scm index 190426ed06..54681274b9 100644 --- a/tests/scripts-build.scm +++ b/tests/scripts-build.scm @@ -1,5 +1,5 @@ ;;; GNU Guix --- Functional package management for GNU -;;; Copyright © 2016, 2017 Ludovic Courtès +;;; Copyright © 2016, 2017, 2019 Ludovic Courtès ;;; ;;; This file is part of GNU Guix. ;;; @@ -23,6 +23,7 @@ #:use-module (guix scripts build) #:use-module (guix ui) #:use-module (guix utils) + #:use-module (guix git) #:use-module (gnu packages) #:use-module (gnu packages base) #:use-module (gnu packages busybox) @@ -164,4 +165,25 @@ ((("x" dep)) (eq? (package-replacement dep) findutils))))))))))) +(test-equal "options->transformation, with-git-url" + (let ((source (git-checkout (url "https://example.org") + (recursive? #t)))) + (list source source)) + (let* ((p (dummy-package "guix.scm" + (inputs `(("foo" ,grep) + ("bar" ,(dummy-package "chbouib" + (native-inputs `(("x" ,grep))))))))) + (t (options->transformation '((with-git-url . "grep=https://example.org"))))) + (with-store store + (let ((new (t store p))) + (and (not (eq? new p)) + (match (package-inputs new) + ((("foo" dep1) ("bar" dep2)) + (and (string=? (package-full-name dep1) + (package-full-name grep)) + (string=? (package-name dep2) "chbouib") + (match (package-native-inputs dep2) + ((("x" dep3)) + (map package-source (list dep1 dep3)))))))))))) + (test-end)