From patchwork Mon Nov 19 16:24:09 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Ludovic_Court=C3=A8s?= X-Patchwork-Id: 231 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 7A29C167ED; Mon, 19 Nov 2018 16:25:15 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_HI, URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) by mira.cbaines.net (Postfix) with ESMTPS id 18BD8167E8 for ; Mon, 19 Nov 2018 16:25:15 +0000 (GMT) Received: from localhost ([::1]:57588 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gOmMM-0003Yd-D9 for patchwork@mira.cbaines.net; Mon, 19 Nov 2018 11:25:14 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:54596) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gOmMG-0003W9-Kd for guix-patches@gnu.org; Mon, 19 Nov 2018 11:25:09 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gOmMA-0005NU-US for guix-patches@gnu.org; Mon, 19 Nov 2018 11:25:08 -0500 Received: from debbugs.gnu.org ([208.118.235.43]:56866) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gOmMA-0005N7-Qi for guix-patches@gnu.org; Mon, 19 Nov 2018 11:25:02 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1gOmMA-0007my-Ib for guix-patches@gnu.org; Mon, 19 Nov 2018 11:25:02 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#33432] [PATCH 2/2] git-download: Download from Software Heritage as a last resort. Resent-From: Ludovic =?utf-8?q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Mon, 19 Nov 2018 16:25:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33432 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 33432@debbugs.gnu.org Received: via spool by 33432-submit@debbugs.gnu.org id=B33432.154264467029883 (code B ref 33432); Mon, 19 Nov 2018 16:25:02 +0000 Received: (at 33432) by debbugs.gnu.org; 19 Nov 2018 16:24:30 +0000 Received: from localhost ([127.0.0.1]:32889 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gOmLd-0007lu-NH for submit@debbugs.gnu.org; Mon, 19 Nov 2018 11:24:29 -0500 Received: from eggs.gnu.org ([208.118.235.92]:44714) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gOmLb-0007lb-Ms for 33432@debbugs.gnu.org; Mon, 19 Nov 2018 11:24:28 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gOmLV-0004jg-SQ for 33432@debbugs.gnu.org; Mon, 19 Nov 2018 11:24:22 -0500 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:54063) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gOmLV-0004jS-P5; Mon, 19 Nov 2018 11:24:21 -0500 Received: from [2001:660:6102:320:e120:2c8f:8909:cdfe] (port=60932 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1gOmLV-000700-GF; Mon, 19 Nov 2018 11:24:21 -0500 From: Ludovic =?utf-8?q?Court=C3=A8s?= Date: Mon, 19 Nov 2018 17:24:09 +0100 Message-Id: <20181119162409.8130-2-ludo@gnu.org> X-Mailer: git-send-email 2.19.1 In-Reply-To: <20181119162409.8130-1-ludo@gnu.org> References: <20181119162409.8130-1-ludo@gnu.org> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 208.118.235.43 X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Ludovic =?utf-8?q?Court=C3=A8s?= Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches From: Ludovic Courtès * guix/git-download.scm (git-fetch)[inputs]: Add gzip and tar when 'git-reference-recursive?' is false. [guile-json, gnutls]: New variables. [modules]: Add (guix swh). [build]: Wrap in 'with-extensions'. Add call to 'swh-download'. --- guix/git-download.scm | 64 +++++++++++++++++++++++++++++-------------- 1 file changed, 44 insertions(+), 20 deletions(-) diff --git a/guix/git-download.scm b/guix/git-download.scm index fa94fad8f8..2689658af8 100644 --- a/guix/git-download.scm +++ b/guix/git-download.scm @@ -74,11 +74,22 @@ HASH-ALGO (a symbol). Use NAME as the file name, or a generic name if #f." ;; available so that 'git submodule' works. (if (git-reference-recursive? ref) (standard-packages) - '())) + + ;; The 'swh-download' procedure requires tar and gzip. + `(("gzip" ,(module-ref (resolve-interface '(gnu packages compression)) + 'gzip)) + ("tar" ,(module-ref (resolve-interface '(gnu packages base)) + 'tar))))) (define zlib (module-ref (resolve-interface '(gnu packages compression)) 'zlib)) + (define guile-json + (module-ref (resolve-interface '(gnu packages guile)) 'guile-json)) + + (define gnutls + (module-ref (resolve-interface '(gnu packages tls)) 'gnutls)) + (define config.scm (scheme-file "config.scm" #~(begin @@ -93,30 +104,43 @@ HASH-ALGO (a symbol). Use NAME as the file name, or a generic name if #f." (delete '(guix config) (source-module-closure '((guix build git) (guix build utils) - (guix build download-nar)))))) + (guix build download-nar) + (guix swh)))))) (define build (with-imported-modules modules - #~(begin - (use-modules (guix build git) - (guix build utils) - (guix build download-nar) - (ice-9 match)) + (with-extensions (list guile-json gnutls) ;for (guix swh) + #~(begin + (use-modules (guix build git) + (guix build utils) + (guix build download-nar) + (guix swh) + (ice-9 match)) - ;; The 'git submodule' commands expects Coreutils, sed, - ;; grep, etc. to be in $PATH. - (set-path-environment-variable "PATH" '("bin") - (match '#+inputs - (((names dirs outputs ...) ...) - dirs))) + (define recursive? + (call-with-input-string (getenv "git recursive?") read)) - (or (git-fetch (getenv "git url") (getenv "git commit") - #$output - #:recursive? (call-with-input-string - (getenv "git recursive?") - read) - #:git-command (string-append #+git "/bin/git")) - (download-nar #$output))))) + ;; The 'git submodule' commands expects Coreutils, sed, + ;; grep, etc. to be in $PATH. + (set-path-environment-variable "PATH" '("bin") + (match '#+inputs + (((names dirs outputs ...) ...) + dirs))) + + (setvbuf (current-output-port) 'line) + (setvbuf (current-error-port) 'line) + + (or (git-fetch (getenv "git url") (getenv "git commit") + #$output + #:recursive? recursive? + #:git-command (string-append #+git "/bin/git")) + (download-nar #$output) + + ;; As a last resort, attempt to download from Software Heritage. + ;; XXX: Currently recursive checkouts are not supported. + (and (not recursive?) + (swh-download (getenv "git url") (getenv "git commit") + #$output))))))) (mlet %store-monad ((guile (package->derivation guile system))) (gexp->derivation (or name "git-checkout") build