From patchwork Mon Sep 11 14:25:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Ludovic_Court=C3=A8s?= X-Patchwork-Id: 53727 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 07C0B27BBE9; Mon, 11 Sep 2023 15:26:39 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,SPF_HELO_PASS autolearn=unavailable autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id B9E9F27BBE2 for ; Mon, 11 Sep 2023 15:26:37 +0100 (BST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qfhrc-0003sJ-KD; Mon, 11 Sep 2023 10:26:08 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qfhrX-0003qZ-GG for guix-patches@gnu.org; Mon, 11 Sep 2023 10:26:03 -0400 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qfhrW-00035s-0k; Mon, 11 Sep 2023 10:26:03 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1qfhrX-0006hj-VX; Mon, 11 Sep 2023 10:26:03 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#65866] [PATCH 7/8] git-download: Use =?utf-8?b?4oCcYnVpbHRp?= =?utf-8?b?bjpnaXQtZG93bmxvYWTigJ0=?= when available. Resent-From: Ludovic =?utf-8?q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix@cbaines.net, dev@jpoiret.xyz, ludo@gnu.org, othacehe@gnu.org, rekado@elephly.net, zimon.toutoune@gmail.com, me@tobias.gr, guix-patches@gnu.org Resent-Date: Mon, 11 Sep 2023 14:26:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 65866 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 65866@debbugs.gnu.org Cc: Ludovic =?utf-8?q?Court=C3=A8s?= , Christopher Baines , Josselin Poiret , Ludovic =?utf-8?q?Court=C3=A8s?= , Mathieu Othacehe , Ricardo Wurmus , Simon Tournier , Tobias Geerinckx-Rice X-Debbugs-Original-Xcc: Christopher Baines , Josselin Poiret , Ludovic =?utf-8?q?Court=C3=A8s?= , Mathieu Othacehe , Ricardo Wurmus , Simon Tournier , Tobias Geerinckx-Rice Received: via spool by 65866-submit@debbugs.gnu.org id=B65866.169444236025717 (code B ref 65866); Mon, 11 Sep 2023 14:26:03 +0000 Received: (at 65866) by debbugs.gnu.org; 11 Sep 2023 14:26:00 +0000 Received: from localhost ([127.0.0.1]:54280 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qfhrT-0006ge-QM for submit@debbugs.gnu.org; Mon, 11 Sep 2023 10:26:00 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55022) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qfhrO-0006fJ-G1 for 65866@debbugs.gnu.org; Mon, 11 Sep 2023 10:25:55 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qfhrF-000344-3n; Mon, 11 Sep 2023 10:25:45 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:References:In-Reply-To:Date:Subject:To: From; bh=ShZItNlvYjaRrNo4JPlFc63xoz7be61ooWEzJHPU0v4=; b=oHz5EoLWBmpAliEnOeLs QaqCICjXjhsBGU2R5y13a00M9MZTquQrf2G/MwdumOXgCo6WVwW6H+LNQoeStmf8/3jxuZM1EVI+B ZdJvDOI8CYZ1LzKNLec4T3mcScJkNSrl22/p9/mx7BiQR3SyiTx8hw/WAI8TcPO6AMpYy08ykLz0T vgXZ6oWBJfcjOrlj6M+eDr3chUCZDHUqY/Ul0sdGh9QbSNJOltn3GyKWQbVcSgODlB65odEHlClzu dumDECqVnrin3wRDY1pYuSB+n1Eq/dR0d/uhhi/2gt7cW7UXNIpsgkAY+FfR5dag4wQA6r3/mhod3 VKh1/qCMbTKJYQ==; From: Ludovic =?utf-8?q?Court=C3=A8s?= Date: Mon, 11 Sep 2023 16:25:25 +0200 Message-ID: <2cd5b127be6d64e640e569f262cef3bbb89f58a6.1694441831.git.ludo@gnu.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: References: MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org X-getmail-retrieved-from-mailbox: Patches Fixes . Longer-term this will remove Git from the derivation graph when its sole use is to perform a checkout for a fixed-output derivation, thereby breaking dependency cycles that can arise in these situations. * guix/git-download.scm (git-fetch): Rename to… (git-fetch/in-band): … this. Deal with GIT or GUILE being #f. (git-fetch/built-in, built-in-builders*, git-fetch): New procedures. * tests/builders.scm ("git-fetch, file URI"): New test. --- guix/git-download.scm | 68 +++++++++++++++++++++++++++++++++++++------ tests/builders.scm | 29 +++++++++++++++++- 2 files changed, 87 insertions(+), 10 deletions(-) diff --git a/guix/git-download.scm b/guix/git-download.scm index f1f19397c6..505dff0a89 100644 --- a/guix/git-download.scm +++ b/guix/git-download.scm @@ -27,6 +27,7 @@ (define-module (guix git-download) #:use-module (guix records) #:use-module (guix packages) #:use-module (guix modules) + #:use-module ((guix derivations) #:select (raw-derivation)) #:autoload (guix build-system gnu) (standard-packages) #:autoload (guix download) (%download-fallback-test) #:autoload (git bindings) (libgit2-init!) @@ -78,15 +79,19 @@ (define (git-package) (let ((distro (resolve-interface '(gnu packages version-control)))) (module-ref distro 'git-minimal))) -(define* (git-fetch ref hash-algo hash - #:optional name - #:key (system (%current-system)) (guile (default-guile)) - (git (git-package))) - "Return a fixed-output derivation that fetches REF, a -object. The output is expected to have recursive hash HASH of type -HASH-ALGO (a symbol). Use NAME as the file name, or a generic name if #f." +(define* (git-fetch/in-band ref hash-algo hash + #:optional name + #:key (system (%current-system)) + (guile (default-guile)) + (git (git-package))) + "Return a fixed-output derivation that performs a Git checkout of REF, using +GIT and GUILE (thus, said derivation depends on GIT and GUILE). + +This method is deprecated in favor of the \"builtin:git-download\" builder. +It will be removed when versions of guix-daemon implementing +\"builtin:git-download\" will be sufficiently widespread." (define inputs - `(("git" ,git) + `(("git" ,(or git (git-package))) ;; When doing 'git clone --recursive', we need sed, grep, etc. to be ;; available so that 'git submodule' works. @@ -154,7 +159,8 @@ (define* (git-fetch ref hash-algo hash #:recursive? recursive? #:git-command "git"))))) - (mlet %store-monad ((guile (package->derivation guile system))) + (mlet %store-monad ((guile (package->derivation (or guile (default-guile)) + system))) (gexp->derivation (or name "git-checkout") build ;; Use environment variables and a fixed script name so @@ -181,6 +187,50 @@ (define* (git-fetch ref hash-algo hash #:recursive? #t #:guile-for-build guile))) +(define* (git-fetch/built-in ref hash-algo hash + #:optional name + #:key (system (%current-system))) + "Return a fixed-output derivation without any dependency that performs a Git +checkout of REF, using the \"builtin:git-download\" derivation builder." + (raw-derivation (or name "git-checkout") "builtin:git-download" '() + #:system system + #:hash-algo hash-algo + #:hash hash + #:recursive? #t + #:env-vars + `(("url" . ,(object->string + (match (%download-fallback-test) + ('content-addressed-mirrors + "https://example.org/does-not-exist") + (_ + (git-reference-url ref))))) + ("commit" . ,(git-reference-commit ref)) + ("recursive?" . ,(object->string + (git-reference-recursive? ref)))) + #:leaked-env-vars '("http_proxy" "https_proxy" + "LC_ALL" "LC_MESSAGES" "LANG" + "COLUMNS") + #:local-build? #t)) + +(define built-in-builders* + (store-lift built-in-builders)) + +(define* (git-fetch ref hash-algo hash + #:optional name + #:key (system (%current-system)) + guile git) + "Return a fixed-output derivation that fetches REF, a +object. The output is expected to have recursive hash HASH of type +HASH-ALGO (a symbol). Use NAME as the file name, or a generic name if #f." + (mlet %store-monad ((builtins (built-in-builders*))) + (if (member "git-download" builtins) + (git-fetch/built-in ref hash-algo hash name + #:system system) + (git-fetch/in-band ref hash-algo hash name + #:system system + #:guile guile + #:git git)))) + (define (git-version version revision commit) "Return the version string for packages using git-download." ;; git-version is almost exclusively executed while modules are being loaded. diff --git a/tests/builders.scm b/tests/builders.scm index 0b5577c7a3..619caa5f31 100644 --- a/tests/builders.scm +++ b/tests/builders.scm @@ -1,5 +1,5 @@ ;;; GNU Guix --- Functional package management for GNU -;;; Copyright © 2012, 2013, 2014, 2015, 2018, 2019, 2021 Ludovic Courtès +;;; Copyright © 2012-2015, 2018-2019, 2021, 2023 Ludovic Courtès ;;; Copyright © 2021 Lars-Dominik Braun ;;; ;;; This file is part of GNU Guix. @@ -20,6 +20,7 @@ (define-module (tests builders) #:use-module (guix download) + #:use-module (guix git-download) #:use-module (guix build-system) #:use-module (guix build-system gnu) #:use-module (guix build gnu-build-system) @@ -31,9 +32,12 @@ (define-module (tests builders) #:use-module (guix base32) #:use-module (guix derivations) #:use-module (gcrypt hash) + #:use-module ((guix hash) #:select (file-hash*)) #:use-module (guix tests) + #:use-module (guix tests git) #:use-module (guix packages) #:use-module (gnu packages bootstrap) + #:use-module ((ice-9 ftw) #:select (scandir)) #:use-module (ice-9 match) #:use-module (ice-9 textual-ports) #:use-module (srfi srfi-1) @@ -84,6 +88,29 @@ (define url-fetch* (and (file-exists? out) (valid-path? %store out)))) +(test-equal "git-fetch, file URI" + '("." ".." "a.txt" "b.scm") + (let ((nonce (random-text))) + (with-temporary-git-repository directory + `((add "a.txt" ,nonce) + (add "b.scm" "#t") + (commit "Commit.") + (tag "v1.0.0" "The tag.")) + (run-with-store %store + (mlet* %store-monad ((hash + -> (file-hash* directory + #:algorithm (hash-algorithm sha256) + #:recursive? #t)) + (drv (git-fetch + (git-reference + (url (string-append "file://" directory)) + (commit "v1.0.0")) + 'sha256 hash + "git-fetch-test"))) + (mbegin %store-monad + (built-derivations (list drv)) + (return (scandir (derivation->output-path drv))))))))) + (test-assert "gnu-build-system" (build-system? gnu-build-system))