From patchwork Mon Sep 11 14:25:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Ludovic_Court=C3=A8s?= X-Patchwork-Id: 53725 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id D87D827BBEA; Mon, 11 Sep 2023 15:26:30 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,SPF_HELO_PASS autolearn=unavailable autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id AA31C27BBE9 for ; Mon, 11 Sep 2023 15:26:29 +0100 (BST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qfhrc-0003sw-Nc; Mon, 11 Sep 2023 10:26:08 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qfhrX-0003qX-Ex for guix-patches@gnu.org; Mon, 11 Sep 2023 10:26:03 -0400 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qfhrW-00035t-4p; Mon, 11 Sep 2023 10:26:03 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1qfhrV-0006gz-R0; Mon, 11 Sep 2023 10:26:01 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#65866] [PATCH 1/8] git-download: Move fallback code to (guix build git). Resent-From: Ludovic =?utf-8?q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix@cbaines.net, dev@jpoiret.xyz, ludo@gnu.org, othacehe@gnu.org, rekado@elephly.net, zimon.toutoune@gmail.com, me@tobias.gr, guix-patches@gnu.org Resent-Date: Mon, 11 Sep 2023 14:26:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 65866 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 65866@debbugs.gnu.org Cc: Ludovic =?utf-8?q?Court=C3=A8s?= , Christopher Baines , Josselin Poiret , Ludovic =?utf-8?q?Court=C3=A8s?= , Mathieu Othacehe , Ricardo Wurmus , Simon Tournier , Tobias Geerinckx-Rice X-Debbugs-Original-Xcc: Christopher Baines , Josselin Poiret , Ludovic =?utf-8?q?Court=C3=A8s?= , Mathieu Othacehe , Ricardo Wurmus , Simon Tournier , Tobias Geerinckx-Rice Received: via spool by 65866-submit@debbugs.gnu.org id=B65866.169444235125649 (code B ref 65866); Mon, 11 Sep 2023 14:26:01 +0000 Received: (at 65866) by debbugs.gnu.org; 11 Sep 2023 14:25:51 +0000 Received: from localhost ([127.0.0.1]:54260 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qfhrL-0006fW-1i for submit@debbugs.gnu.org; Mon, 11 Sep 2023 10:25:51 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54974) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qfhrJ-0006f7-4k for 65866@debbugs.gnu.org; Mon, 11 Sep 2023 10:25:49 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qfhr9-0002xL-Lx; Mon, 11 Sep 2023 10:25:39 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:References:In-Reply-To:Date:Subject:To: From; bh=NhkD4LZUlsDDA0ql2tBJERZrS5hCR8P2YozzIJfod8I=; b=BlO+gHNPGdjPFbkWr6nz pzBLctux9qRsmOWlilsAMK5arxWUvitoXbjuhHAxm7dkL+dAT6DfFh+zTSu6ZMmXc5O6+Uxnp6ddl qjsMKIigHK5CR30NghNL8zKHBLlyXWjFl0Bcrb+A7sbW1DACjrolJq5Xsmg4mXHZDWftP7ELl0N2y bHAaDkfTiNH5OxQSRuZ/8XlQ7mshdXx2COVOhXsveAwHUqp/LJAD0XLQF5KLS73ZpIqEigW41S/oE jWsTQhkCIoSwx95S7xzWhbsW88b3jjzHx2KpqYH0riz/xK2s2ZjahCHolH7tFPpdRc89045Mscvqm UKi/45n7RK1oWw==; From: Ludovic =?utf-8?q?Court=C3=A8s?= Date: Mon, 11 Sep 2023 16:25:19 +0200 Message-ID: X-Mailer: git-send-email 2.41.0 In-Reply-To: References: MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org X-getmail-retrieved-from-mailbox: Patches * guix/build/git.scm (git-fetch-with-fallback): New procedure, with code taken from… * guix/git-download.scm (git-fetch): … here. [modules]: Remove modules that are no longer directly used in ‘build’. [build]: Use ‘git-fetch-with-fallback’. --- guix/build/git.scm | 44 ++++++++++++++++++++++++++++++++++++++-- guix/git-download.scm | 47 ++++++++----------------------------------- 2 files changed, 50 insertions(+), 41 deletions(-) diff --git a/guix/build/git.scm b/guix/build/git.scm index deda10fee8..0ff263c81b 100644 --- a/guix/build/git.scm +++ b/guix/build/git.scm @@ -1,5 +1,5 @@ ;;; GNU Guix --- Functional package management for GNU -;;; Copyright © 2014, 2016, 2019 Ludovic Courtès +;;; Copyright © 2014, 2016, 2019, 2023 Ludovic Courtès ;;; ;;; This file is part of GNU Guix. ;;; @@ -18,9 +18,12 @@ (define-module (guix build git) #:use-module (guix build utils) + #:autoload (guix build download-nar) (download-nar) + #:autoload (guix swh) (%verify-swh-certificate? swh-download) #:use-module (srfi srfi-34) #:use-module (ice-9 format) - #:export (git-fetch)) + #:export (git-fetch + git-fetch-with-fallback)) ;;; Commentary: ;;; @@ -76,4 +79,41 @@ (define* (git-fetch url commit directory (delete-file-recursively ".git") #t))) + +(define* (git-fetch-with-fallback url commit directory + #:key (git-command "git") recursive?) + "Like 'git-fetch', fetch COMMIT from URL into DIRECTORY, but fall back to +alternative methods when fetching from URL fails: attempt to download a nar, +and if that also fails, download from the Software Heritage archive." + (or (git-fetch url commit directory + #:recursive? recursive? + #:git-command git-command) + (download-nar directory) + + ;; As a last resort, attempt to download from Software Heritage. + ;; Disable X.509 certificate verification to avoid depending + ;; on nss-certs--we're authenticating the checkout anyway. + ;; XXX: Currently recursive checkouts are not supported. + (and (not recursive?) + (parameterize ((%verify-swh-certificate? #f)) + (format (current-error-port) + "Trying to download from Software Heritage...~%") + + (swh-download url commit directory) + (when (file-exists? + (string-append directory "/.gitattributes")) + ;; Perform CR/LF conversion and other changes + ;; specificied by '.gitattributes'. + (invoke git-command "-C" directory "init") + (invoke git-command "-C" directory "config" "--local" + "user.email" "you@example.org") + (invoke git-command "-C" directory "config" "--local" + "user.name" "Your Name") + (invoke git-command "-C" directory "add" ".") + (invoke git-command "-C" directory "commit" "-am" "init") + (invoke git-command "-C" directory "read-tree" "--empty") + (invoke git-command "-C" directory "reset" "--hard") + (delete-file-recursively + (string-append directory "/.git"))))))) + ;;; git.scm ends here diff --git a/guix/git-download.scm b/guix/git-download.scm index d88f4c40ee..8989b1b463 100644 --- a/guix/git-download.scm +++ b/guix/git-download.scm @@ -116,19 +116,16 @@ (define* (git-fetch ref hash-algo hash (define modules (delete '(guix config) (source-module-closure '((guix build git) - (guix build utils) - (guix build download-nar) - (guix swh))))) + (guix build utils))))) (define build (with-imported-modules modules - (with-extensions (list guile-json gnutls ;for (guix swh) + (with-extensions (list guile-json gnutls ;for (guix swh) guile-lzlib) #~(begin (use-modules (guix build git) - (guix build utils) - (guix build download-nar) - (guix swh) + ((guix build utils) + #:select (set-path-environment-variable)) (ice-9 match)) (define recursive? @@ -151,38 +148,10 @@ (define* (git-fetch ref hash-algo hash (setvbuf (current-output-port) 'line) (setvbuf (current-error-port) 'line) - (or (git-fetch (getenv "git url") (getenv "git commit") - #$output - #:recursive? recursive? - #:git-command "git") - (download-nar #$output) - - ;; As a last resort, attempt to download from Software Heritage. - ;; Disable X.509 certificate verification to avoid depending - ;; on nss-certs--we're authenticating the checkout anyway. - ;; XXX: Currently recursive checkouts are not supported. - (and (not recursive?) - (parameterize ((%verify-swh-certificate? #f)) - (format (current-error-port) - "Trying to download from Software Heritage...~%") - - (swh-download (getenv "git url") (getenv "git commit") - #$output) - (when (file-exists? - (string-append #$output "/.gitattributes")) - ;; Perform CR/LF conversion and other changes - ;; specificied by '.gitattributes'. - (invoke "git" "-C" #$output "init") - (invoke "git" "-C" #$output "config" "--local" - "user.email" "you@example.org") - (invoke "git" "-C" #$output "config" "--local" - "user.name" "Your Name") - (invoke "git" "-C" #$output "add" ".") - (invoke "git" "-C" #$output "commit" "-am" "init") - (invoke "git" "-C" #$output "read-tree" "--empty") - (invoke "git" "-C" #$output "reset" "--hard") - (delete-file-recursively - (string-append #$output "/.git")))))))))) + (git-fetch-with-fallback (getenv "git url") (getenv "git commit") + #$output + #:recursive? recursive? + #:git-command "git"))))) (mlet %store-monad ((guile (package->derivation guile system))) (gexp->derivation (or name "git-checkout") build From patchwork Mon Sep 11 14:25:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Ludovic_Court=C3=A8s?= X-Patchwork-Id: 53729 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id CB50527BBEA; Mon, 11 Sep 2023 15:26:43 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,SPF_HELO_PASS autolearn=unavailable autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 5889F27BBE2 for ; Mon, 11 Sep 2023 15:26:43 +0100 (BST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qfhrZ-0003r2-F9; Mon, 11 Sep 2023 10:26:05 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qfhrV-0003ox-6r for guix-patches@gnu.org; Mon, 11 Sep 2023 10:26:01 -0400 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qfhrU-00035h-Tq; Mon, 11 Sep 2023 10:26:00 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1qfhrW-0006h5-8i; Mon, 11 Sep 2023 10:26:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#65866] [PATCH 2/8] git-download: Honor the =?utf-8?b?4oCYR1VJ?= =?utf-8?b?WF9ET1dOTE9BRF9GQUxMQkFDS19URVNU4oCZ?= environment variable. Resent-From: Ludovic =?utf-8?q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix@cbaines.net, dev@jpoiret.xyz, ludo@gnu.org, othacehe@gnu.org, rekado@elephly.net, zimon.toutoune@gmail.com, me@tobias.gr, guix-patches@gnu.org Resent-Date: Mon, 11 Sep 2023 14:26:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 65866 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 65866@debbugs.gnu.org Cc: Ludovic =?utf-8?q?Court=C3=A8s?= , Christopher Baines , Josselin Poiret , Ludovic =?utf-8?q?Court=C3=A8s?= , Mathieu Othacehe , Ricardo Wurmus , Simon Tournier , Tobias Geerinckx-Rice X-Debbugs-Original-Xcc: Christopher Baines , Josselin Poiret , Ludovic =?utf-8?q?Court=C3=A8s?= , Mathieu Othacehe , Ricardo Wurmus , Simon Tournier , Tobias Geerinckx-Rice Received: via spool by 65866-submit@debbugs.gnu.org id=B65866.169444235225655 (code B ref 65866); Mon, 11 Sep 2023 14:26:02 +0000 Received: (at 65866) by debbugs.gnu.org; 11 Sep 2023 14:25:52 +0000 Received: from localhost ([127.0.0.1]:54262 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qfhrL-0006fe-OL for submit@debbugs.gnu.org; Mon, 11 Sep 2023 10:25:52 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54982) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qfhrK-0006f9-1B for 65866@debbugs.gnu.org; Mon, 11 Sep 2023 10:25:50 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qfhrA-0002yL-Ke; Mon, 11 Sep 2023 10:25:40 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:References:In-Reply-To:Date:Subject:To: From; bh=lbJIuSS34XKou1KAAp0q0w8yQNp1j9kF2Ik0svE2RAw=; b=kZJ47piQeRVOU/HkuP+U zzJ4xJEQlFz/qec3U8lI/jslJ+ZLqBvUqz9gqHdz2LrE1k+bOdG83e5zVYQbDFgWh4Adb7Pz/CywA ek/L7cE6aSavlkJUime6vaXeB0GhqBybDge2w4Smk6/K7d24dHYl4tC5DN3jdr3nxBPL0syQaj1Df sNwqUi4c1ZAbXxZgW4X5nZvcidTnEmlgNkfdaf26Q6qyDeWEZTC+jjmB1IF08Etw33sYVFw4b2na5 ptWmOJ+kC8l5KXubtOo5Ck5F05yxUwXy1fv2o6vplknXASqX0UEZviyLhgL7ITdKPCVhNwpPUP/cZ 0fss5/F6Ujymnw==; From: Ludovic =?utf-8?q?Court=C3=A8s?= Date: Mon, 11 Sep 2023 16:25:20 +0200 Message-ID: <2cc7a7d7d2d1c801ec4529b18f2a526d8d2b07ef.1694441831.git.ludo@gnu.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: References: MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org X-getmail-retrieved-from-mailbox: Patches * guix/git-download.scm (git-fetch): Honor ‘%download-fallback-test’. --- guix/git-download.scm | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/guix/git-download.scm b/guix/git-download.scm index 8989b1b463..f1f19397c6 100644 --- a/guix/git-download.scm +++ b/guix/git-download.scm @@ -28,6 +28,7 @@ (define-module (guix git-download) #:use-module (guix packages) #:use-module (guix modules) #:autoload (guix build-system gnu) (standard-packages) + #:autoload (guix download) (%download-fallback-test) #:autoload (git bindings) (libgit2-init!) #:autoload (git repository) (repository-open repository-close! @@ -161,7 +162,11 @@ (define* (git-fetch ref hash-algo hash ;; downloads. #:script-name "git-download" #:env-vars - `(("git url" . ,(git-reference-url ref)) + `(("git url" . ,(match (%download-fallback-test) + ('content-addressed-mirrors + "https://example.org/does-not-exist") + (_ + (git-reference-url ref)))) ("git commit" . ,(git-reference-commit ref)) ("git recursive?" . ,(object->string (git-reference-recursive? ref)))) From patchwork Mon Sep 11 14:25:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Ludovic_Court=C3=A8s?= X-Patchwork-Id: 53726 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 1CCD727BBE9; Mon, 11 Sep 2023 15:26:32 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,SPF_HELO_PASS autolearn=unavailable autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 8CDBB27BBE2 for ; Mon, 11 Sep 2023 15:26:31 +0100 (BST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qfhra-0003r5-U8; Mon, 11 Sep 2023 10:26:06 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qfhrX-0003qW-E3 for guix-patches@gnu.org; Mon, 11 Sep 2023 10:26:03 -0400 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qfhrV-00035q-Nv; Mon, 11 Sep 2023 10:26:01 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1qfhrW-0006hG-QT; Mon, 11 Sep 2023 10:26:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#65866] [PATCH 3/8] perform-download: Remove unused one-argument clause. Resent-From: Ludovic =?utf-8?q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix@cbaines.net, dev@jpoiret.xyz, ludo@gnu.org, othacehe@gnu.org, rekado@elephly.net, zimon.toutoune@gmail.com, me@tobias.gr, guix-patches@gnu.org Resent-Date: Mon, 11 Sep 2023 14:26:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 65866 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 65866@debbugs.gnu.org Cc: Ludovic =?utf-8?q?Court=C3=A8s?= , Christopher Baines , Josselin Poiret , Ludovic =?utf-8?q?Court=C3=A8s?= , Mathieu Othacehe , Ricardo Wurmus , Simon Tournier , Tobias Geerinckx-Rice X-Debbugs-Original-Xcc: Christopher Baines , Josselin Poiret , Ludovic =?utf-8?q?Court=C3=A8s?= , Mathieu Othacehe , Ricardo Wurmus , Simon Tournier , Tobias Geerinckx-Rice Received: via spool by 65866-submit@debbugs.gnu.org id=B65866.169444235825696 (code B ref 65866); Mon, 11 Sep 2023 14:26:02 +0000 Received: (at 65866) by debbugs.gnu.org; 11 Sep 2023 14:25:58 +0000 Received: from localhost ([127.0.0.1]:54274 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qfhrQ-0006gA-43 for submit@debbugs.gnu.org; Mon, 11 Sep 2023 10:25:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54994) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qfhrK-0006fA-Tt for 65866@debbugs.gnu.org; Mon, 11 Sep 2023 10:25:51 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qfhrB-0002zP-HR; Mon, 11 Sep 2023 10:25:41 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:References:In-Reply-To:Date:Subject:To: From; bh=8VpxIxGrW5Ejr0OFvwQSmvFszi43poboHugffNEPfik=; b=fcnKcHe20tnhWkXLT8d1 vWW/aQb7DryvAfqlcykaLxvQloLxyOiTPLN5Qk4eOT6k4AjmKEJIFi8+Zqq/AtYZKkuL75XefTYWo ptCLGHGSJoRItbC17+padWbtZ89McznXKRIyCzbPpORdAi5EKKczgt60eL1ErX7endI1J6K9X84dY NdTntlPsf3X/WoK4x5DwoHWP8KOIqPwU6ktxRJOf1MmfBTqf8mZ/4rgHzCDXeDIvNvYe91IUjvrqV VxAqOydcni5yp8ZgdqzO3NDEnTs9jA0TASqkr5GWVkqwJPXAkzoSUsjRXilmPXpFxhka4HhpfG4Q9 oPYwuQNvLW7C6g==; From: Ludovic =?utf-8?q?Court=C3=A8s?= Date: Mon, 11 Sep 2023 16:25:21 +0200 Message-ID: X-Mailer: git-send-email 2.41.0 In-Reply-To: References: MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org X-getmail-retrieved-from-mailbox: Patches Code in ‘builtins.cc’ only ever invokes ‘guix perform-download’ with two arguments. * guix/scripts/perform-download.scm (guix-perform-download): Remove unused one-argument clause. --- guix/scripts/perform-download.scm | 9 ++------- 1 file changed, 2 insertions(+), 7 deletions(-) diff --git a/guix/scripts/perform-download.scm b/guix/scripts/perform-download.scm index 6889bcef79..c8f044e82e 100644 --- a/guix/scripts/perform-download.scm +++ b/guix/scripts/perform-download.scm @@ -120,13 +120,8 @@ (define-command (guix-perform-download . args) (match args (((? derivation-path? drv) (? store-path? output)) (assert-low-privileges) - (perform-download (read-derivation-from-file drv) - output - #:print-build-trace? print-build-trace?)) - (((? derivation-path? drv)) ;backward compatibility - (assert-low-privileges) - (perform-download (read-derivation-from-file drv) - #:print-build-trace? print-build-trace?)) + (let ((drv (read-derivation-from-file drv))) + (perform-download drv output #:print-build-trace? print-build-trace?))) (("--version") (show-version-and-exit)) (x From patchwork Mon Sep 11 14:25:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Ludovic_Court=C3=A8s?= X-Patchwork-Id: 53728 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id A77F927BBE2; Mon, 11 Sep 2023 15:26:40 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,SPF_HELO_PASS autolearn=unavailable autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 538D227BBEA for ; Mon, 11 Sep 2023 15:26:38 +0100 (BST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qfhrX-0003qI-7U; Mon, 11 Sep 2023 10:26:03 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qfhrT-0003oS-Fa for guix-patches@gnu.org; Mon, 11 Sep 2023 10:25:59 -0400 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qfhrT-00035S-6W; Mon, 11 Sep 2023 10:25:59 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1qfhrX-0006hO-7Q; Mon, 11 Sep 2023 10:26:03 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#65866] [PATCH 4/8] daemon: Add =?utf-8?b?4oCcZ2l0LWRvd25sb2Fk?= =?utf-8?b?4oCd?= built-in builder. Resent-From: Ludovic =?utf-8?q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix@cbaines.net, dev@jpoiret.xyz, ludo@gnu.org, othacehe@gnu.org, rekado@elephly.net, zimon.toutoune@gmail.com, me@tobias.gr, guix-patches@gnu.org Resent-Date: Mon, 11 Sep 2023 14:26:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 65866 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 65866@debbugs.gnu.org Cc: Ludovic =?utf-8?q?Court=C3=A8s?= , Christopher Baines , Josselin Poiret , Ludovic =?utf-8?q?Court=C3=A8s?= , Mathieu Othacehe , Ricardo Wurmus , Simon Tournier , Tobias Geerinckx-Rice X-Debbugs-Original-Xcc: Christopher Baines , Josselin Poiret , Ludovic =?utf-8?q?Court=C3=A8s?= , Mathieu Othacehe , Ricardo Wurmus , Simon Tournier , Tobias Geerinckx-Rice Received: via spool by 65866-submit@debbugs.gnu.org id=B65866.169444235925704 (code B ref 65866); Mon, 11 Sep 2023 14:26:03 +0000 Received: (at 65866) by debbugs.gnu.org; 11 Sep 2023 14:25:59 +0000 Received: from localhost ([127.0.0.1]:54276 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qfhrS-0006gP-MT for submit@debbugs.gnu.org; Mon, 11 Sep 2023 10:25:59 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55002) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qfhrM-0006fF-QW for 65866@debbugs.gnu.org; Mon, 11 Sep 2023 10:25:53 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qfhrC-0002zo-HX; Mon, 11 Sep 2023 10:25:42 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:References:In-Reply-To:Date:Subject:To: From; bh=99DJrUUViq7vxNIOv5rl+/tDT4n8RnWxwe8vYvOdTYY=; b=SmO+MULr6zuErUGBxYWQ 4NFu1jaKbhmVJdtar2NLvbY3+gAaBY6E1lgfxnSz7ZaS9jI03sNI+vQ+8cfJawun82NxH/LGDHItc YSgcwzszb9CW8hZS7qZ+gPW64CZgGPU9L5s6tvGURPcRW1BQL5ZJ9eCfUjvX/QaAOTycGYHJuKXem 5z1L4n7m7SmlnyGgq9qWGJQayds/9l5zu7eLujHraSbQz00RgDndmLSNBCUHsIInht13zTHXQaVSk /2u9qDe+bX3nEdaBbg6ozJs2ixYzX0OqL6dZQv2micbTZXqldF/kXyUONWPhX4DzgiJqt7WbQn7XB d2cbcJNN8HokxQ==; From: Ludovic =?utf-8?q?Court=C3=A8s?= Date: Mon, 11 Sep 2023 16:25:22 +0200 Message-ID: <3c42634cb47dd7eaa81a198bc2d097ca74a973ed.1694441831.git.ludo@gnu.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: References: MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org X-getmail-retrieved-from-mailbox: Patches From: Ludovic Courtès The new builder makes it possible to break cycles that occurs when the fixed-output derivation for the source of a dependency of ‘git’ would itself depend on ‘git’. * guix/scripts/perform-download.scm (perform-git-download): New procedure. (perform-download): Move fixed-output derivation check to… (guix-perform-download): … here. Invoke ‘perform-download’ or ‘perform-git-download’ depending on what ‘derivation-builder’ returns. * nix/libstore/builtins.cc (builtins): Add “git-download”. * tests/derivations.scm ("built-in-builders"): Update. ("'git-download' built-in builder") ("'git-download' built-in builder, invalid hash") ("'git-download' built-in builder, invalid commit") ("'git-download' built-in builder, not found"): New tests. --- guix/scripts/perform-download.scm | 52 +++++++++++++--- nix/libstore/builtins.cc | 5 +- tests/derivations.scm | 100 +++++++++++++++++++++++++++++- 3 files changed, 145 insertions(+), 12 deletions(-) diff --git a/guix/scripts/perform-download.scm b/guix/scripts/perform-download.scm index c8f044e82e..a287e97528 100644 --- a/guix/scripts/perform-download.scm +++ b/guix/scripts/perform-download.scm @@ -1,5 +1,5 @@ ;;; GNU Guix --- Functional package management for GNU -;;; Copyright © 2016, 2017, 2018, 2020 Ludovic Courtès +;;; Copyright © 2016-2018, 2020, 2023 Ludovic Courtès ;;; ;;; This file is part of GNU Guix. ;;; @@ -21,7 +21,8 @@ (define-module (guix scripts perform-download) #:use-module (guix scripts) #:use-module (guix derivations) #:use-module ((guix store) #:select (derivation-path? store-path?)) - #:use-module (guix build download) + #:autoload (guix build download) (url-fetch) + #:autoload (guix build git) (git-fetch-with-fallback) #:use-module (ice-9 match) #:export (guix-perform-download)) @@ -64,10 +65,6 @@ (define* (perform-download drv #:optional output (drv-output (assoc-ref (derivation-outputs drv) "out")) (algo (derivation-output-hash-algo drv-output)) (hash (derivation-output-hash drv-output))) - (unless (and algo hash) - (leave (G_ "~a is not a fixed-output derivation~%") - (derivation-file-name drv))) - ;; We're invoked by the daemon, which gives us write access to OUTPUT. (when (url-fetch url output #:print-build-trace? print-build-trace? @@ -92,6 +89,33 @@ (define* (perform-download drv #:optional output (when (and executable (string=? executable "1")) (chmod output #o755)))))) +(define* (perform-git-download drv #:optional output + #:key print-build-trace?) + "Perform the download described by DRV, a fixed-output derivation, to +OUTPUT. + +Note: Unless OUTPUT is #f, we don't read the value of 'out' in DRV since the +actual output is different from that when we're doing a 'bmCheck' or +'bmRepair' build." + (derivation-let drv ((output* "out") + (url "url") + (commit "commit") + (recursive? "recursive?")) + (unless url + (leave (G_ "~a: missing Git URL~%") (derivation-file-name drv))) + (unless commit + (leave (G_ "~a: missing Git commit~%") (derivation-file-name drv))) + + (let* ((output (or output output*)) + (url (call-with-input-string url read)) + (recursive? (and recursive? + (call-with-input-string recursive? read))) + (drv-output (assoc-ref (derivation-outputs drv) "out")) + (algo (derivation-output-hash-algo drv-output)) + (hash (derivation-output-hash drv-output))) + (git-fetch-with-fallback url commit output + #:recursive? recursive?)))) + (define (assert-low-privileges) (when (zero? (getuid)) (leave (G_ "refusing to run with elevated privileges (UID ~a)~%") @@ -120,8 +144,20 @@ (define-command (guix-perform-download . args) (match args (((? derivation-path? drv) (? store-path? output)) (assert-low-privileges) - (let ((drv (read-derivation-from-file drv))) - (perform-download drv output #:print-build-trace? print-build-trace?))) + (let* ((drv (read-derivation-from-file drv)) + (download (match (derivation-builder drv) + ("builtin:download" perform-download) + ("builtin:git-download" perform-git-download) + (unknown (leave (G_ "~a: unknown builtin builder") + unknown)))) + (drv-output (assoc-ref (derivation-outputs drv) "out")) + (algo (derivation-output-hash-algo drv-output)) + (hash (derivation-output-hash drv-output))) + (unless (and hash algo) + (leave (G_ "~a is not a fixed-output derivation~%") + (derivation-file-name drv))) + + (download drv output #:print-build-trace? print-build-trace?))) (("--version") (show-version-and-exit)) (x diff --git a/nix/libstore/builtins.cc b/nix/libstore/builtins.cc index 4111ac4760..6bf467354a 100644 --- a/nix/libstore/builtins.cc +++ b/nix/libstore/builtins.cc @@ -1,5 +1,5 @@ /* GNU Guix --- Functional package management for GNU - Copyright (C) 2016, 2017, 2018, 2019 Ludovic Courtès + Copyright (C) 2016-2019, 2023 Ludovic Courtès This file is part of GNU Guix. @@ -58,7 +58,8 @@ static void builtinDownload(const Derivation &drv, static const std::map builtins = { - { "download", builtinDownload } + { "download", builtinDownload }, + { "git-download", builtinDownload } }; derivationBuilder lookupBuiltinBuilder(const std::string & name) diff --git a/tests/derivations.scm b/tests/derivations.scm index 66c777cfe7..e1312bd46b 100644 --- a/tests/derivations.scm +++ b/tests/derivations.scm @@ -24,10 +24,15 @@ (define-module (test-derivations) #:use-module (guix utils) #:use-module ((gcrypt hash) #:prefix gcrypt:) #:use-module (guix base32) + #:use-module ((guix git) #:select (with-repository)) #:use-module (guix tests) + #:use-module (guix tests git) #:use-module (guix tests http) #:use-module ((guix packages) #:select (package-derivation base32)) - #:use-module ((guix build utils) #:select (executable-file?)) + #:use-module ((guix build utils) #:select (executable-file? which)) + #:use-module ((guix hash) #:select (file-hash*)) + #:use-module ((git oid) #:select (oid->string)) + #:use-module ((git reference) #:select (reference-name->oid)) #:use-module (gnu packages bootstrap) #:use-module ((gnu packages guile) #:select (guile-1.8)) #:use-module (srfi srfi-1) @@ -195,7 +200,7 @@ (define* (directory-contents dir #:optional (slurp get-bytevector-all)) (stat:ino (lstat file2)))))))) (test-equal "built-in-builders" - '("download") + '("download" "git-download") (built-in-builders %store)) (test-assert "unknown built-in builder" @@ -290,6 +295,97 @@ (define* (directory-contents dir #:optional (slurp get-bytevector-all)) get-string-all) text)))))) +;; 'with-temporary-git-repository' relies on the 'git' command. +(unless (which (git-command)) (test-skip 1)) +(test-equal "'git-download' built-in builder" + `(("/a.txt" . "AAA") + ("/b.scm" . "#t")) + (let ((nonce (random-text))) + (with-temporary-git-repository directory + `((add "a.txt" "AAA") + (add "b.scm" "#t") + (commit ,nonce)) + (let* ((commit (with-repository directory repository + (oid->string + (reference-name->oid repository "HEAD")))) + (drv (derivation %store "git-download" + "builtin:git-download" '() + #:env-vars + `(("url" + . ,(object->string + (string-append "file://" directory))) + ("commit" . ,commit)) + #:hash-algo 'sha256 + #:hash (file-hash* directory + #:algorithm + (gcrypt:hash-algorithm + gcrypt:sha256) + #:recursive? #t) + #:recursive? #t))) + (build-derivations %store (list drv)) + (directory-contents (derivation->output-path drv) get-string-all))))) + +(unless (which (git-command)) (test-skip 1)) +(test-assert "'git-download' built-in builder, invalid hash" + (with-temporary-git-repository directory + `((add "a.txt" "AAA") + (add "b.scm" "#t") + (commit "Commit!")) + (let* ((commit (with-repository directory repository + (oid->string + (reference-name->oid repository "HEAD")))) + (drv (derivation %store "git-download" + "builtin:git-download" '() + #:env-vars + `(("url" + . ,(object->string + (string-append "file://" directory))) + ("commit" . ,commit)) + #:hash-algo 'sha256 + #:hash (gcrypt:sha256 #vu8()) + #:recursive? #t))) + (guard (c ((store-protocol-error? c) + (string-contains (store-protocol-error-message c) "failed"))) + (build-derivations %store (list drv)) + #f)))) + +(unless (which (git-command)) (test-skip 1)) +(test-assert "'git-download' built-in builder, invalid commit" + (with-temporary-git-repository directory + `((add "a.txt" "AAA") + (add "b.scm" "#t") + (commit "Commit!")) + (let* ((drv (derivation %store "git-download" + "builtin:git-download" '() + #:env-vars + `(("url" + . ,(object->string + (string-append "file://" directory))) + ("commit" + . "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa")) + #:hash-algo 'sha256 + #:hash (gcrypt:sha256 #vu8()) + #:recursive? #t))) + (guard (c ((store-protocol-error? c) + (string-contains (store-protocol-error-message c) "failed"))) + (build-derivations %store (list drv)) + #f)))) + +(test-assert "'git-download' built-in builder, not found" + (let* ((drv (derivation %store "git-download" + "builtin:git-download" '() + #:env-vars + `(("url" . "file:///does-not-exist.git") + ("commit" + . "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa")) + #:hash-algo 'sha256 + #:hash (gcrypt:sha256 #vu8()) + #:recursive? #t))) + (guard (c ((store-protocol-error? c) + (string-contains (store-protocol-error-message c) "failed"))) + (build-derivations %store (list drv)) + #f))) + (test-equal "derivation-name" "foo-0.0" (let ((drv (derivation %store "foo-0.0" %bash '()))) From patchwork Mon Sep 11 14:25:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Ludovic_Court=C3=A8s?= X-Patchwork-Id: 53730 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id CD70527BBEA; Mon, 11 Sep 2023 15:27:10 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,SPF_HELO_PASS autolearn=unavailable autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id D739827BBE2 for ; Mon, 11 Sep 2023 15:27:06 +0100 (BST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qfhsS-0004wT-RL; Mon, 11 Sep 2023 10:27:01 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qfhsR-0004vw-Hz for guix-patches@gnu.org; Mon, 11 Sep 2023 10:26:59 -0400 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qfhsR-0003DZ-9V; Mon, 11 Sep 2023 10:26:59 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1qfhsU-0006jf-Fk; Mon, 11 Sep 2023 10:27:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#65866] [PATCH 5/8] build: Add dependency on Git. Resent-From: Ludovic =?utf-8?q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix@cbaines.net, dev@jpoiret.xyz, ludo@gnu.org, othacehe@gnu.org, rekado@elephly.net, zimon.toutoune@gmail.com, me@tobias.gr, guix-patches@gnu.org Resent-Date: Mon, 11 Sep 2023 14:27:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 65866 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 65866@debbugs.gnu.org Cc: Ludovic =?utf-8?q?Court=C3=A8s?= , Christopher Baines , Josselin Poiret , Ludovic =?utf-8?q?Court=C3=A8s?= , Mathieu Othacehe , Ricardo Wurmus , Simon Tournier , Tobias Geerinckx-Rice X-Debbugs-Original-Xcc: Christopher Baines , Josselin Poiret , Ludovic =?utf-8?q?Court=C3=A8s?= , Mathieu Othacehe , Ricardo Wurmus , Simon Tournier , Tobias Geerinckx-Rice Received: via spool by 65866-submit@debbugs.gnu.org id=B65866.169444236625819 (code B ref 65866); Mon, 11 Sep 2023 14:27:02 +0000 Received: (at 65866) by debbugs.gnu.org; 11 Sep 2023 14:26:06 +0000 Received: from localhost ([127.0.0.1]:54310 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qfhrZ-0006iE-Ea for submit@debbugs.gnu.org; Mon, 11 Sep 2023 10:26:06 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54996) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qfhrM-0006fE-Nl for 65866@debbugs.gnu.org; Mon, 11 Sep 2023 10:25:56 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qfhrD-000312-BV; Mon, 11 Sep 2023 10:25:43 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:References:In-Reply-To:Date:Subject:To: From; bh=kdp9llTbF8+3aSxiW2tzTYl/MmzrtCqW5SxqJyuizdM=; b=d+aomIVl1LJSbNczrvGW UQj43n9l5aVNFvL6AtcDLyJEFBi80mjkZio1zbGYuMqSOttlIU0LvZ4tKPwukb0UpmRmYS/0whxHt ZM3yyz54PxzpkbPd/7Oe25qx1HFWMsTP6MbPkEiIsJS7ukZgF39SH/XEK6TdoSZVeLp+PQBw5MyR9 M+lVLxuSYv6M01hKDkAy4SG87iiLc0jhcwH/e/leVuEswr3UIC2o45f6TYTTb8b87Pq0Ila07X2cC MXN4YIM5p4GMM0u+JnEV7Ai0TO0U3lYZgmfOqA4wlpteXvmd0bBywPlJcT3lXQ4OiyEggCaZGd69M gbHb4AFHsd7ZWg==; From: Ludovic =?utf-8?q?Court=C3=A8s?= Date: Mon, 11 Sep 2023 16:25:23 +0200 Message-ID: <4eca94501c2c1e9986e1f718eeccb3eb9276dcd4.1694441831.git.ludo@gnu.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: References: MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org X-getmail-retrieved-from-mailbox: Patches * configure.ac: Check for ‘git’ and substitute ‘GIT’. * guix/config.scm.in (%git): New variable. * guix/self.scm (compiled-guix): Define ‘git’ and pass it to ‘make-config.scm’. (make-config.scm): Add #:git; emit a ‘%git’ variable. * doc/guix.texi (Requirements): Add it. --- configure.ac | 7 +++++++ doc/guix.texi | 1 + guix/config.scm.in | 6 +++++- guix/self.scm | 10 +++++++++- 4 files changed, 22 insertions(+), 2 deletions(-) diff --git a/configure.ac b/configure.ac index 92dede8014..d817f620cf 100644 --- a/configure.ac +++ b/configure.ac @@ -201,6 +201,13 @@ AC_SUBST([GZIP]) AC_SUBST([BZIP2]) AC_SUBST([XZ]) +dnl Git is now required for the "builtin:git-download" derivation builder. +AC_PATH_PROG([GIT], [git]) +if test "x$GIT" = "x"; then + AC_MSG_ERROR([Git is missing; please install it.]) +fi +AC_SUBST([GIT]) + LIBGCRYPT_LIBDIR="no" LIBGCRYPT_PREFIX="no" diff --git a/doc/guix.texi b/doc/guix.texi index 339dcb2a41..a2520ce89d 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -1011,6 +1011,7 @@ Requirements @item @uref{https://gitlab.com/guile-git/guile-git, Guile-Git}, version 0.5.0 or later; +@item @uref{https://git-scm.com, Git} (yes, both!); @item @uref{https://savannah.nongnu.org/projects/guile-json/, Guile-JSON} 4.3.0 or later; @item @url{https://www.gnu.org/software/make/, GNU Make}. diff --git a/guix/config.scm.in b/guix/config.scm.in index d582d91d74..62e15dd713 100644 --- a/guix/config.scm.in +++ b/guix/config.scm.in @@ -1,5 +1,5 @@ ;;; GNU Guix --- Functional package management for GNU -;;; Copyright © 2012, 2013, 2014, 2015, 2016, 2018, 2019, 2021 Ludovic Courtès +;;; Copyright © 2012-2016, 2018-2019, 2021, 2023 Ludovic Courtès ;;; Copyright © 2017 Caleb Ristvedt ;;; ;;; This file is part of GNU Guix. @@ -35,6 +35,7 @@ (define-module (guix config) %config-directory %system + %git %gzip %bzip2 %xz)) @@ -109,6 +110,9 @@ (define %config-directory (define %system "@guix_system@") +(define %git + "@GIT@") + (define %gzip "@GZIP@") diff --git a/guix/self.scm b/guix/self.scm index 81a36e007f..41c5f40786 100644 --- a/guix/self.scm +++ b/guix/self.scm @@ -68,6 +68,7 @@ (define %packages ("gzip" . ,(ref 'compression 'gzip)) ("bzip2" . ,(ref 'compression 'bzip2)) ("xz" . ,(ref 'compression 'xz)) + ("git-minimal" . ,(ref 'version-control 'git-minimal)) ("po4a" . ,(ref 'gettext 'po4a)) ("gettext-minimal" . ,(ref 'gettext 'gettext-minimal)) ("gcc-toolchain" . ,(ref 'commencement 'gcc-toolchain)) @@ -825,6 +826,9 @@ (define* (compiled-guix source #:key (define guile-lzma (specification->package "guile-lzma")) + (define git + (specification->package "git-minimal")) + (define dependencies (append-map transitive-package-dependencies (list guile-gcrypt guile-gnutls guile-git guile-avahi @@ -998,6 +1002,7 @@ (define* (compiled-guix source #:key => ,(make-config.scm #:gzip gzip #:bzip2 bzip2 #:xz xz + #:git git #:package-name %guix-package-name #:package-version @@ -1103,7 +1108,7 @@ (define %default-config-variables (%storedir . "/gnu/store") (%sysconfdir . "/etc"))) -(define* (make-config.scm #:key gzip xz bzip2 +(define* (make-config.scm #:key gzip xz bzip2 git (package-name "GNU Guix") (package-version "0") (channel-metadata #f) @@ -1133,6 +1138,7 @@ (define* (make-config.scm #:key gzip xz bzip2 %state-directory %store-database-directory %config-directory + %git %gzip %bzip2 %xz)) @@ -1175,6 +1181,8 @@ (define* (make-config.scm #:key gzip xz bzip2 ;; information is used by (guix describe). '#$channel-metadata) + (define %git + #+(and git (file-append git "/bin/git"))) (define %gzip #+(and gzip (file-append gzip "/bin/gzip"))) (define %bzip2 From patchwork Mon Sep 11 14:25:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Ludovic_Court=C3=A8s?= X-Patchwork-Id: 53724 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id A49ED27BBEA; Mon, 11 Sep 2023 15:26:18 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,SPF_HELO_PASS autolearn=ham autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 18A3F27BBE2 for ; Mon, 11 Sep 2023 15:26:18 +0100 (BST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qfhrc-0003tB-Q5; Mon, 11 Sep 2023 10:26:08 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qfhrX-0003qY-FD for guix-patches@gnu.org; Mon, 11 Sep 2023 10:26:03 -0400 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qfhrV-00035r-W0; Mon, 11 Sep 2023 10:26:03 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1qfhrX-0006hY-Jc; Mon, 11 Sep 2023 10:26:03 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#65866] [PATCH 6/8] perform-git-download: Use the =?utf-8?b?4oCYZ2l04oCZ?= command captured at configure time. Resent-From: Ludovic =?utf-8?q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix@cbaines.net, dev@jpoiret.xyz, ludo@gnu.org, othacehe@gnu.org, rekado@elephly.net, zimon.toutoune@gmail.com, me@tobias.gr, guix-patches@gnu.org Resent-Date: Mon, 11 Sep 2023 14:26:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 65866 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 65866@debbugs.gnu.org Cc: Ludovic =?utf-8?q?Court=C3=A8s?= , Christopher Baines , Josselin Poiret , Ludovic =?utf-8?q?Court=C3=A8s?= , Mathieu Othacehe , Ricardo Wurmus , Simon Tournier , Tobias Geerinckx-Rice X-Debbugs-Original-Xcc: Christopher Baines , Josselin Poiret , Ludovic =?utf-8?q?Court=C3=A8s?= , Mathieu Othacehe , Ricardo Wurmus , Simon Tournier , Tobias Geerinckx-Rice Received: via spool by 65866-submit@debbugs.gnu.org id=B65866.169444236025711 (code B ref 65866); Mon, 11 Sep 2023 14:26:03 +0000 Received: (at 65866) by debbugs.gnu.org; 11 Sep 2023 14:26:00 +0000 Received: from localhost ([127.0.0.1]:54278 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qfhrT-0006gX-G4 for submit@debbugs.gnu.org; Mon, 11 Sep 2023 10:25:59 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55018) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qfhrN-0006fH-Jd for 65866@debbugs.gnu.org; Mon, 11 Sep 2023 10:25:54 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qfhrE-00033D-7r; Mon, 11 Sep 2023 10:25:44 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:References:In-Reply-To:Date:Subject:To: From; bh=2EsfrtzZMQfiz6o+ksU68WCBCRlX5ph0lDFzZywzfbI=; b=d9AICfwDAeCc9S32AQW8 1aqVhm5KnPbH8gt6xDnOID5QSa6JtfCO0c3M2tKO34bFpssaUL4kcZZAB6BRDItHC/9Mb+6xAHWFn P6QUfggP4GjOWo2Ry+i/9HU/E/1HGD9/kXOm3/4iDUBUEBRyTkUXGHpyZ73AsC3q9W6aQbhG8RQkU 7beL21wOKdQsVydNzAHlg+Ku5QJMaDM04enXvov8Z/jtFDlwJnaytQbnPtmQ8X4kpXanUEl0xKGyd kUJXRslqzJj/HR64WJy9/uPrzMLGHSPS+x8sYr0qB4E8y2a+yT/8ibyqAvnB7FkaF0w4FfRtBbIHL 3Rp3th/Ng/TVmQ==; From: Ludovic =?utf-8?q?Court=C3=A8s?= Date: Mon, 11 Sep 2023 16:25:24 +0200 Message-ID: <52b761e331150a637bddf696469c5e265646d461.1694441831.git.ludo@gnu.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: References: MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org X-getmail-retrieved-from-mailbox: Patches * guix/scripts/perform-download.scm (perform-git-download): Pass #:git-command to ‘git-fetch-with-fallback’. --- guix/scripts/perform-download.scm | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/guix/scripts/perform-download.scm b/guix/scripts/perform-download.scm index a287e97528..e1c584fbda 100644 --- a/guix/scripts/perform-download.scm +++ b/guix/scripts/perform-download.scm @@ -23,6 +23,7 @@ (define-module (guix scripts perform-download) #:use-module ((guix store) #:select (derivation-path? store-path?)) #:autoload (guix build download) (url-fetch) #:autoload (guix build git) (git-fetch-with-fallback) + #:autoload (guix config) (%git) #:use-module (ice-9 match) #:export (guix-perform-download)) @@ -114,7 +115,8 @@ (define* (perform-git-download drv #:optional output (algo (derivation-output-hash-algo drv-output)) (hash (derivation-output-hash drv-output))) (git-fetch-with-fallback url commit output - #:recursive? recursive?)))) + #:recursive? recursive? + #:git-command %git)))) (define (assert-low-privileges) (when (zero? (getuid)) From patchwork Mon Sep 11 14:25:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Ludovic_Court=C3=A8s?= X-Patchwork-Id: 53727 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 07C0B27BBE9; Mon, 11 Sep 2023 15:26:39 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,SPF_HELO_PASS autolearn=unavailable autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id B9E9F27BBE2 for ; Mon, 11 Sep 2023 15:26:37 +0100 (BST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qfhrc-0003sJ-KD; Mon, 11 Sep 2023 10:26:08 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qfhrX-0003qZ-GG for guix-patches@gnu.org; Mon, 11 Sep 2023 10:26:03 -0400 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qfhrW-00035s-0k; Mon, 11 Sep 2023 10:26:03 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1qfhrX-0006hj-VX; Mon, 11 Sep 2023 10:26:03 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#65866] [PATCH 7/8] git-download: Use =?utf-8?b?4oCcYnVpbHRp?= =?utf-8?b?bjpnaXQtZG93bmxvYWTigJ0=?= when available. Resent-From: Ludovic =?utf-8?q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix@cbaines.net, dev@jpoiret.xyz, ludo@gnu.org, othacehe@gnu.org, rekado@elephly.net, zimon.toutoune@gmail.com, me@tobias.gr, guix-patches@gnu.org Resent-Date: Mon, 11 Sep 2023 14:26:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 65866 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 65866@debbugs.gnu.org Cc: Ludovic =?utf-8?q?Court=C3=A8s?= , Christopher Baines , Josselin Poiret , Ludovic =?utf-8?q?Court=C3=A8s?= , Mathieu Othacehe , Ricardo Wurmus , Simon Tournier , Tobias Geerinckx-Rice X-Debbugs-Original-Xcc: Christopher Baines , Josselin Poiret , Ludovic =?utf-8?q?Court=C3=A8s?= , Mathieu Othacehe , Ricardo Wurmus , Simon Tournier , Tobias Geerinckx-Rice Received: via spool by 65866-submit@debbugs.gnu.org id=B65866.169444236025717 (code B ref 65866); Mon, 11 Sep 2023 14:26:03 +0000 Received: (at 65866) by debbugs.gnu.org; 11 Sep 2023 14:26:00 +0000 Received: from localhost ([127.0.0.1]:54280 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qfhrT-0006ge-QM for submit@debbugs.gnu.org; Mon, 11 Sep 2023 10:26:00 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55022) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qfhrO-0006fJ-G1 for 65866@debbugs.gnu.org; Mon, 11 Sep 2023 10:25:55 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qfhrF-000344-3n; Mon, 11 Sep 2023 10:25:45 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:References:In-Reply-To:Date:Subject:To: From; bh=ShZItNlvYjaRrNo4JPlFc63xoz7be61ooWEzJHPU0v4=; b=oHz5EoLWBmpAliEnOeLs QaqCICjXjhsBGU2R5y13a00M9MZTquQrf2G/MwdumOXgCo6WVwW6H+LNQoeStmf8/3jxuZM1EVI+B ZdJvDOI8CYZ1LzKNLec4T3mcScJkNSrl22/p9/mx7BiQR3SyiTx8hw/WAI8TcPO6AMpYy08ykLz0T vgXZ6oWBJfcjOrlj6M+eDr3chUCZDHUqY/Ul0sdGh9QbSNJOltn3GyKWQbVcSgODlB65odEHlClzu dumDECqVnrin3wRDY1pYuSB+n1Eq/dR0d/uhhi/2gt7cW7UXNIpsgkAY+FfR5dag4wQA6r3/mhod3 VKh1/qCMbTKJYQ==; From: Ludovic =?utf-8?q?Court=C3=A8s?= Date: Mon, 11 Sep 2023 16:25:25 +0200 Message-ID: <2cd5b127be6d64e640e569f262cef3bbb89f58a6.1694441831.git.ludo@gnu.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: References: MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org X-getmail-retrieved-from-mailbox: Patches Fixes . Longer-term this will remove Git from the derivation graph when its sole use is to perform a checkout for a fixed-output derivation, thereby breaking dependency cycles that can arise in these situations. * guix/git-download.scm (git-fetch): Rename to… (git-fetch/in-band): … this. Deal with GIT or GUILE being #f. (git-fetch/built-in, built-in-builders*, git-fetch): New procedures. * tests/builders.scm ("git-fetch, file URI"): New test. --- guix/git-download.scm | 68 +++++++++++++++++++++++++++++++++++++------ tests/builders.scm | 29 +++++++++++++++++- 2 files changed, 87 insertions(+), 10 deletions(-) diff --git a/guix/git-download.scm b/guix/git-download.scm index f1f19397c6..505dff0a89 100644 --- a/guix/git-download.scm +++ b/guix/git-download.scm @@ -27,6 +27,7 @@ (define-module (guix git-download) #:use-module (guix records) #:use-module (guix packages) #:use-module (guix modules) + #:use-module ((guix derivations) #:select (raw-derivation)) #:autoload (guix build-system gnu) (standard-packages) #:autoload (guix download) (%download-fallback-test) #:autoload (git bindings) (libgit2-init!) @@ -78,15 +79,19 @@ (define (git-package) (let ((distro (resolve-interface '(gnu packages version-control)))) (module-ref distro 'git-minimal))) -(define* (git-fetch ref hash-algo hash - #:optional name - #:key (system (%current-system)) (guile (default-guile)) - (git (git-package))) - "Return a fixed-output derivation that fetches REF, a -object. The output is expected to have recursive hash HASH of type -HASH-ALGO (a symbol). Use NAME as the file name, or a generic name if #f." +(define* (git-fetch/in-band ref hash-algo hash + #:optional name + #:key (system (%current-system)) + (guile (default-guile)) + (git (git-package))) + "Return a fixed-output derivation that performs a Git checkout of REF, using +GIT and GUILE (thus, said derivation depends on GIT and GUILE). + +This method is deprecated in favor of the \"builtin:git-download\" builder. +It will be removed when versions of guix-daemon implementing +\"builtin:git-download\" will be sufficiently widespread." (define inputs - `(("git" ,git) + `(("git" ,(or git (git-package))) ;; When doing 'git clone --recursive', we need sed, grep, etc. to be ;; available so that 'git submodule' works. @@ -154,7 +159,8 @@ (define* (git-fetch ref hash-algo hash #:recursive? recursive? #:git-command "git"))))) - (mlet %store-monad ((guile (package->derivation guile system))) + (mlet %store-monad ((guile (package->derivation (or guile (default-guile)) + system))) (gexp->derivation (or name "git-checkout") build ;; Use environment variables and a fixed script name so @@ -181,6 +187,50 @@ (define* (git-fetch ref hash-algo hash #:recursive? #t #:guile-for-build guile))) +(define* (git-fetch/built-in ref hash-algo hash + #:optional name + #:key (system (%current-system))) + "Return a fixed-output derivation without any dependency that performs a Git +checkout of REF, using the \"builtin:git-download\" derivation builder." + (raw-derivation (or name "git-checkout") "builtin:git-download" '() + #:system system + #:hash-algo hash-algo + #:hash hash + #:recursive? #t + #:env-vars + `(("url" . ,(object->string + (match (%download-fallback-test) + ('content-addressed-mirrors + "https://example.org/does-not-exist") + (_ + (git-reference-url ref))))) + ("commit" . ,(git-reference-commit ref)) + ("recursive?" . ,(object->string + (git-reference-recursive? ref)))) + #:leaked-env-vars '("http_proxy" "https_proxy" + "LC_ALL" "LC_MESSAGES" "LANG" + "COLUMNS") + #:local-build? #t)) + +(define built-in-builders* + (store-lift built-in-builders)) + +(define* (git-fetch ref hash-algo hash + #:optional name + #:key (system (%current-system)) + guile git) + "Return a fixed-output derivation that fetches REF, a +object. The output is expected to have recursive hash HASH of type +HASH-ALGO (a symbol). Use NAME as the file name, or a generic name if #f." + (mlet %store-monad ((builtins (built-in-builders*))) + (if (member "git-download" builtins) + (git-fetch/built-in ref hash-algo hash name + #:system system) + (git-fetch/in-band ref hash-algo hash name + #:system system + #:guile guile + #:git git)))) + (define (git-version version revision commit) "Return the version string for packages using git-download." ;; git-version is almost exclusively executed while modules are being loaded. diff --git a/tests/builders.scm b/tests/builders.scm index 0b5577c7a3..619caa5f31 100644 --- a/tests/builders.scm +++ b/tests/builders.scm @@ -1,5 +1,5 @@ ;;; GNU Guix --- Functional package management for GNU -;;; Copyright © 2012, 2013, 2014, 2015, 2018, 2019, 2021 Ludovic Courtès +;;; Copyright © 2012-2015, 2018-2019, 2021, 2023 Ludovic Courtès ;;; Copyright © 2021 Lars-Dominik Braun ;;; ;;; This file is part of GNU Guix. @@ -20,6 +20,7 @@ (define-module (tests builders) #:use-module (guix download) + #:use-module (guix git-download) #:use-module (guix build-system) #:use-module (guix build-system gnu) #:use-module (guix build gnu-build-system) @@ -31,9 +32,12 @@ (define-module (tests builders) #:use-module (guix base32) #:use-module (guix derivations) #:use-module (gcrypt hash) + #:use-module ((guix hash) #:select (file-hash*)) #:use-module (guix tests) + #:use-module (guix tests git) #:use-module (guix packages) #:use-module (gnu packages bootstrap) + #:use-module ((ice-9 ftw) #:select (scandir)) #:use-module (ice-9 match) #:use-module (ice-9 textual-ports) #:use-module (srfi srfi-1) @@ -84,6 +88,29 @@ (define url-fetch* (and (file-exists? out) (valid-path? %store out)))) +(test-equal "git-fetch, file URI" + '("." ".." "a.txt" "b.scm") + (let ((nonce (random-text))) + (with-temporary-git-repository directory + `((add "a.txt" ,nonce) + (add "b.scm" "#t") + (commit "Commit.") + (tag "v1.0.0" "The tag.")) + (run-with-store %store + (mlet* %store-monad ((hash + -> (file-hash* directory + #:algorithm (hash-algorithm sha256) + #:recursive? #t)) + (drv (git-fetch + (git-reference + (url (string-append "file://" directory)) + (commit "v1.0.0")) + 'sha256 hash + "git-fetch-test"))) + (mbegin %store-monad + (built-derivations (list drv)) + (return (scandir (derivation->output-path drv))))))))) + (test-assert "gnu-build-system" (build-system? gnu-build-system)) From patchwork Mon Sep 11 14:25:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Ludovic_Court=C3=A8s?= X-Patchwork-Id: 53731 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id B42FD27BBE9; Mon, 11 Sep 2023 15:27:21 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,SPF_HELO_PASS autolearn=unavailable autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 3F1F227BBE2 for ; Mon, 11 Sep 2023 15:27:19 +0100 (BST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qfhsS-0004wH-IO; Mon, 11 Sep 2023 10:27:00 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qfhsR-0004vp-7x for guix-patches@gnu.org; Mon, 11 Sep 2023 10:26:59 -0400 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qfhsQ-0003DU-VI for guix-patches@gnu.org; Mon, 11 Sep 2023 10:26:58 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1qfhsU-0006jo-W2 for guix-patches@gnu.org; Mon, 11 Sep 2023 10:27:03 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#65866] [PATCH 8/8] tests: Assume =?utf-8?b?4oCYZ2l04oCZ?= is always available. Resent-From: Ludovic =?utf-8?q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Mon, 11 Sep 2023 14:27:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 65866 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 65866@debbugs.gnu.org Cc: Ludovic =?utf-8?q?Court=C3=A8s?= Received: via spool by 65866-submit@debbugs.gnu.org id=B65866.169444236725827 (code B ref 65866); Mon, 11 Sep 2023 14:27:02 +0000 Received: (at 65866) by debbugs.gnu.org; 11 Sep 2023 14:26:07 +0000 Received: from localhost ([127.0.0.1]:54319 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qfhra-0006iO-Au for submit@debbugs.gnu.org; Mon, 11 Sep 2023 10:26:07 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55028) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qfhrP-0006fU-Df for 65866@debbugs.gnu.org; Mon, 11 Sep 2023 10:25:56 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qfhrG-00034U-1C; Mon, 11 Sep 2023 10:25:46 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:References:In-Reply-To:Date:Subject:To: From; bh=u9+hTSEMumwCQQ/u1GucMUnU5Oqp6LOTsklYGo7TPpc=; b=qTADEbxmokHSswl6hdhb N8E4qb8u1SltapQ0dBY9TvnTYsGIoySbLMXkJHERyV9M2DZYY5RkmtMZcbpNsWGQXXlX4MTAfT8d7 gRAwysyPTxcUHugqGYZcY1CrCcZb3KG73dkgXxWLgsLrOfSwOYHQlGyG1I5Bso+4JAYdN6qaGCcL1 OacqJoVaGOBqN7rZAJMl7fijWOziv8ivkTfoiHqVucAAEOFQUpOUaKpicLyHWYDGJWUGqToHFpKey QJgpnDipHm2m+uY0Bq02CDjdvWi+ZE9mvYjzhIpPoJ5JHFWiCcIm+xi9fMz5K0cjqEvezLYir6CFX 1EUdRmY0QF4m1w==; From: Ludovic =?utf-8?q?Court=C3=A8s?= Date: Mon, 11 Sep 2023 16:25:26 +0200 Message-ID: <1488a25806497439ab0b586df11ee689dd7fb446.1694441831.git.ludo@gnu.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: References: MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org X-getmail-retrieved-from-mailbox: Patches * tests/channels.scm (gpg+git-available?): Check for ‘gpg-command’ only. Remove all ‘test-skip’ statements. * tests/derivations.scm: Likewise. * tests/git-authenticate.scm: Likewise. * tests/git.scm: Likewise. * tests/import-git.scm: Likewise. --- tests/channels.scm | 7 +------ tests/derivations.scm | 6 +----- tests/git-authenticate.scm | 1 - tests/git.scm | 10 ---------- tests/import-git.scm | 18 ------------------ 5 files changed, 2 insertions(+), 40 deletions(-) diff --git a/tests/channels.scm b/tests/channels.scm index 62312e240c..6c4276deb4 100644 --- a/tests/channels.scm +++ b/tests/channels.scm @@ -50,7 +50,7 @@ (define-module (test-channels) #:use-module (ice-9 match)) (define (gpg+git-available?) - (and (which (git-command)) + (and #t ;'git' is always available (which (gpg-command)) (which (gpgconf-command)))) (define commit-id-string @@ -196,7 +196,6 @@ (define channel-metadata-dependencies "abc1234"))) instances))))))) -(unless (which (git-command)) (test-skip 1)) (test-equal "latest-channel-instances #:validate-pull" 'descendant @@ -306,7 +305,6 @@ (define channel-metadata-dependencies (depends? drv3 (list drv2 drv0) (list)))))))) -(unless (which (git-command)) (test-skip 1)) (test-equal "channel-news, no news" '() (with-temporary-git-repository directory @@ -318,7 +316,6 @@ (define channel-metadata-dependencies (latest (reference-name->oid repository "HEAD"))) (channel-news-for-commit channel (oid->string latest)))))) -(unless (which (git-command)) (test-skip 1)) (test-assert "channel-news, one entry" (with-temporary-git-repository directory `((add ".guix-channel" @@ -406,7 +403,6 @@ (define channel-metadata-dependencies (channel-news-for-commit channel commit5 commit1)) '(#f "tag-for-first-news-entry"))))))) -(unless (which (git-command)) (test-skip 1)) (test-assert "channel-news, annotated tag" (with-temporary-git-repository directory `((add ".guix-channel" @@ -453,7 +449,6 @@ (define channel-metadata-dependencies (channel-news-for-commit channel commit2)) (list commit1))))))) -(unless (which (git-command)) (test-skip 1)) (test-assert "latest-channel-instances, missing introduction for 'guix'" (with-temporary-git-repository directory '((add "a.txt" "A") diff --git a/tests/derivations.scm b/tests/derivations.scm index e1312bd46b..0e87778981 100644 --- a/tests/derivations.scm +++ b/tests/derivations.scm @@ -29,7 +29,7 @@ (define-module (test-derivations) #:use-module (guix tests git) #:use-module (guix tests http) #:use-module ((guix packages) #:select (package-derivation base32)) - #:use-module ((guix build utils) #:select (executable-file? which)) + #:use-module ((guix build utils) #:select (executable-file?)) #:use-module ((guix hash) #:select (file-hash*)) #:use-module ((git oid) #:select (oid->string)) #:use-module ((git reference) #:select (reference-name->oid)) @@ -295,8 +295,6 @@ (define* (directory-contents dir #:optional (slurp get-bytevector-all)) get-string-all) text)))))) -;; 'with-temporary-git-repository' relies on the 'git' command. -(unless (which (git-command)) (test-skip 1)) (test-equal "'git-download' built-in builder" `(("/a.txt" . "AAA") ("/b.scm" . "#t")) @@ -325,7 +323,6 @@ (define* (directory-contents dir #:optional (slurp get-bytevector-all)) (build-derivations %store (list drv)) (directory-contents (derivation->output-path drv) get-string-all))))) -(unless (which (git-command)) (test-skip 1)) (test-assert "'git-download' built-in builder, invalid hash" (with-temporary-git-repository directory `((add "a.txt" "AAA") @@ -349,7 +346,6 @@ (define* (directory-contents dir #:optional (slurp get-bytevector-all)) (build-derivations %store (list drv)) #f)))) -(unless (which (git-command)) (test-skip 1)) (test-assert "'git-download' built-in builder, invalid commit" (with-temporary-git-repository directory `((add "a.txt" "AAA") diff --git a/tests/git-authenticate.scm b/tests/git-authenticate.scm index c063920c12..4de223d422 100644 --- a/tests/git-authenticate.scm +++ b/tests/git-authenticate.scm @@ -44,7 +44,6 @@ (define (gpg+git-available?) (test-begin "git-authenticate") -(unless (which (git-command)) (test-skip 1)) (test-assert "unsigned commits" (with-temporary-git-repository directory '((add "a.txt" "A") diff --git a/tests/git.scm b/tests/git.scm index 9c944d65b1..ad43435b67 100644 --- a/tests/git.scm +++ b/tests/git.scm @@ -21,7 +21,6 @@ (define-module (test-git) #:use-module (git) #:use-module (guix git) #:use-module (guix tests git) - #:use-module (guix build utils) #:use-module ((guix utils) #:select (call-with-temporary-directory)) #:use-module (srfi srfi-1) #:use-module (srfi srfi-64) @@ -33,8 +32,6 @@ (define-module (test-git) (test-begin "git") -;; 'with-temporary-git-repository' relies on the 'git' command. -(unless (which (git-command)) (test-skip 1)) (test-assert "commit-difference, linear history" (with-temporary-git-repository directory '((add "a.txt" "A") @@ -61,7 +58,6 @@ (define-module (test-git) ;; empty list. (null? (commit-difference commit1 commit4))))))) -(unless (which (git-command)) (test-skip 1)) (test-assert "commit-difference, fork" (with-temporary-git-repository directory '((add "a.txt" "A") @@ -101,7 +97,6 @@ (define-module (test-git) (lset= eq? (commit-difference master4 master2) (list master4 merge master3 devel1 devel2))))))) -(unless (which (git-command)) (test-skip 1)) (test-assert "commit-difference, excluded commits" (with-temporary-git-repository directory '((add "a.txt" "A") @@ -126,7 +121,6 @@ (define-module (test-git) (list commit4)) (null? (commit-difference commit4 commit1 (list commit5)))))))) -(unless (which (git-command)) (test-skip 1)) (test-equal "commit-relation" '(self ;master3 master3 ancestor ;master1 master3 @@ -166,7 +160,6 @@ (define-module (test-git) (commit-relation master1 merge) (commit-relation merge master1)))))) -(unless (which (git-command)) (test-skip 1)) (test-equal "commit-descendant?" '((master3 master3 => #t) (master1 master3 => #f) @@ -216,7 +209,6 @@ (define-module (test-git) (master1 merge) (merge master1))))))) -(unless (which (git-command)) (test-skip 1)) (test-equal "remote-refs" '("refs/heads/develop" "refs/heads/master" "refs/tags/v1.0" "refs/tags/v1.1") @@ -231,7 +223,6 @@ (define-module (test-git) (tag "v1.1" "release-1.1")) (remote-refs directory))) -(unless (which (git-command)) (test-skip 1)) (test-equal "remote-refs: only tags" '("refs/tags/v1.0" "refs/tags/v1.1") (with-temporary-git-repository directory @@ -243,7 +234,6 @@ (define-module (test-git) (tag "v1.1" "Release 1.1")) (remote-refs directory #:tags? #t))) -(unless (which (git-command)) (test-skip 1)) (test-assert "update-cached-checkout, tag" (call-with-temporary-directory (lambda (cache) diff --git a/tests/import-git.scm b/tests/import-git.scm index f1bce154bb..20255dedb3 100644 --- a/tests/import-git.scm +++ b/tests/import-git.scm @@ -24,7 +24,6 @@ (define-module (test-import-git) #:use-module (guix import git) #:use-module (guix git-download) #:use-module (guix tests git) - #:use-module (guix build utils) #:use-module (srfi srfi-1) #:use-module (srfi srfi-64)) @@ -46,7 +45,6 @@ (define* (make-package directory version #:optional (properties '())) (base32 "0000000000000000000000000000000000000000000000000000")))))) -(unless (which (git-command)) (test-skip 1)) (test-equal "latest-git-tag-version: no custom prefix, suffix, and delimiter" "1.0.1" (with-temporary-git-repository directory @@ -56,7 +54,6 @@ (define* (make-package directory version #:optional (properties '())) (let ((package (make-package directory "1.0.0"))) (latest-git-tag-version package)))) -(unless (which (git-command)) (test-skip 1)) (test-equal "latest-git-tag-version: custom prefix, no suffix and delimiter" "1.0.1" (with-temporary-git-repository directory @@ -67,7 +64,6 @@ (define* (make-package directory version #:optional (properties '())) '((release-tag-prefix . "prefix-"))))) (latest-git-tag-version package)))) -(unless (which (git-command)) (test-skip 1)) (test-equal "latest-git-tag-version: custom suffix, no prefix and delimiter" "1.0.1" (with-temporary-git-repository directory @@ -78,7 +74,6 @@ (define* (make-package directory version #:optional (properties '())) '((release-tag-suffix . "-suffix-[0-9]*"))))) (latest-git-tag-version package)))) -(unless (which (git-command)) (test-skip 1)) (test-equal "latest-git-tag-version: custom delimiter, no prefix and suffix" "2021.09.07" (with-temporary-git-repository directory @@ -89,7 +84,6 @@ (define* (make-package directory version #:optional (properties '())) '((release-tag-version-delimiter . "-"))))) (latest-git-tag-version package)))) -(unless (which (git-command)) (test-skip 1)) (test-equal "latest-git-tag-version: empty delimiter, no prefix and suffix" "20210907" (with-temporary-git-repository directory @@ -100,7 +94,6 @@ (define* (make-package directory version #:optional (properties '())) '((release-tag-version-delimiter . ""))))) (latest-git-tag-version package)))) -(unless (which (git-command)) (test-skip 1)) (test-equal "latest-git-tag-version: custom prefix and suffix, no delimiter" "2.0.0" (with-temporary-git-repository directory @@ -112,7 +105,6 @@ (define* (make-package directory version #:optional (properties '())) (release-tag-suffix . "suffix-[0-9]"))))) (latest-git-tag-version package)))) -(unless (which (git-command)) (test-skip 1)) (test-equal "latest-git-tag-version: custom prefix, suffix, and delimiter" "2.0.0" (with-temporary-git-repository directory @@ -125,7 +117,6 @@ (define* (make-package directory version #:optional (properties '())) (release-tag-version-delimiter . "_"))))) (latest-git-tag-version package)))) -(unless (which (git-command)) (test-skip 1)) (test-equal "latest-git-tag-version: only pre-releases available" #f (with-temporary-git-repository directory @@ -135,7 +126,6 @@ (define* (make-package directory version #:optional (properties '())) (let ((package (make-package directory "1.0.0"))) (latest-git-tag-version package)))) -(unless (which (git-command)) (test-skip 1)) (test-equal "latest-git-tag-version: accept pre-releases" "2.0.0-rc1" (with-temporary-git-repository directory @@ -146,7 +136,6 @@ (define* (make-package directory version #:optional (properties '())) '((accept-pre-releases? . #t))))) (latest-git-tag-version package)))) -(unless (which (git-command)) (test-skip 1)) (test-equal "latest-git-tag-version: accept pre-releases, and custom prefix" "2.0.0-rc1" (with-temporary-git-repository directory @@ -158,7 +147,6 @@ (define* (make-package directory version #:optional (properties '())) (release-tag-prefix . "version-"))))) (latest-git-tag-version package)))) -(unless (which (git-command)) (test-skip 1)) (test-equal "latest-git-tag-version: accept pre-releases, and custom suffix" "2.0.0-rc1" (with-temporary-git-repository directory @@ -170,7 +158,6 @@ (define* (make-package directory version #:optional (properties '())) (release-tag-suffix . "-suffix"))))) (latest-git-tag-version package)))) -(unless (which (git-command)) (test-skip 1)) (test-equal "latest-git-tag-version: accept pre-releases, delimiter conflicts with pre-release part" "2.0.0_alpha" (with-temporary-git-repository directory @@ -182,7 +169,6 @@ (define* (make-package directory version #:optional (properties '())) (release-tag-version-delimiter . "_"))))) (latest-git-tag-version package)))) -(unless (which (git-command)) (test-skip 1)) (test-equal "latest-git-tag-version: accept pre-releases, and custom suffix and prefix" "2.0.0-alpha" (with-temporary-git-repository directory @@ -195,7 +181,6 @@ (define* (make-package directory version #:optional (properties '())) (release-tag-suffix . "-suffix"))))) (latest-git-tag-version package)))) -(unless (which (git-command)) (test-skip 1)) (test-equal "latest-git-tag-version: accept pre-releases, and custom suffix, prefix, and delimiter" "2.0.0-alpha" (with-temporary-git-repository directory @@ -209,7 +194,6 @@ (define* (make-package directory version #:optional (properties '())) (release-tag-version-delimiter . "-"))))) (latest-git-tag-version package)))) -(unless (which (git-command)) (test-skip 1)) (test-equal "latest-git-tag-version: accept pre-releases, no delimiter, and custom suffix, prefix" "2alpha" (with-temporary-git-repository directory @@ -223,7 +207,6 @@ (define* (make-package directory version #:optional (properties '())) (release-tag-version-delimiter . ""))))) (latest-git-tag-version package)))) -(unless (which (git-command)) (test-skip 1)) (test-equal "latest-git-tag-version: no tags found" #f (with-temporary-git-repository directory @@ -232,7 +215,6 @@ (define* (make-package directory version #:optional (properties '())) (let ((package (make-package directory "1.0.0"))) (latest-git-tag-version package)))) -(unless (which (git-command)) (test-skip 1)) (test-equal "latest-git-tag-version: no valid tags found" #f (with-temporary-git-repository directory