From patchwork Thu May 15 08:16:32 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?J=C3=B8rgen_Kvalsvik?= X-Patchwork-Id: 42625 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 26E7127BC49; Thu, 15 May 2025 09:19:59 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-6.4 required=5.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_BLOCKED, RCVD_IN_VALIDITY_CERTIFIED,RCVD_IN_VALIDITY_RPBL,RCVD_IN_VALIDITY_SAFE, SPF_HELO_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 036D527BC4A for ; Thu, 15 May 2025 09:19:58 +0100 (BST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uFTnf-0005oB-9I; Thu, 15 May 2025 04:18:43 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uFTn3-0004cb-Rh for guix-patches@gnu.org; Thu, 15 May 2025 04:18:10 -0400 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1uFTn0-0002K1-MR for guix-patches@gnu.org; Thu, 15 May 2025 04:18:03 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=debbugs.gnu.org; s=debbugs-gnu-org; h=MIME-Version:References:In-Reply-To:Date:From:To:Subject; bh=5C+7J5y/Scdtphpvqi3YDdZElj3wse6xAVaDOfsflq4=; b=Hr+i4uSzz2cYr+jUm+wZ/1aSEIk+/WOTpsRFXdfd+sxs/Nrwi3q7V1FM/2msZPkMUvmDIJpmdTFofVaITE1pkmozEdwn2AzZ9DapdEi4UXzdFx0e/6pWV+kM4c6RnrCIDWTVxG0FuvzCJwcayYWQEfBJc3+2NbAYr9HWlj1Wx29eGflAnrjcSYGu3uqe7IVOdM4P2cMQJAt8igovhc7bpSdQvniDHuYCXnvF+tjTxRNrxYJ2A+bAsYLODd/spEy086Taup91lwG831hdFlDgGFIX4deOiDEecsvo3jeMNxo/y/VLJ6KmlIDeRKeeOx149l7bIBJknIGX091Ev1rzEQ==; Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1uFTn0-0004rF-E0 for guix-patches@gnu.org; Thu, 15 May 2025 04:18:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#78404] [PATCH 1/4] guix: Add downloader for go modules from GOPROXY Resent-From: =?utf-8?q?J=C3=B8rgen?= Kvalsvik Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Thu, 15 May 2025 08:18:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 78404 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 78404@debbugs.gnu.org Cc: =?utf-8?q?J=C3=B8rgen?= Kvalsvik Received: via spool by 78404-submit@debbugs.gnu.org id=B78404.174729705618605 (code B ref 78404); Thu, 15 May 2025 08:18:02 +0000 Received: (at 78404) by debbugs.gnu.org; 15 May 2025 08:17:36 +0000 Received: from localhost ([127.0.0.1]:50790 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1uFTma-0004q0-35 for submit@debbugs.gnu.org; Thu, 15 May 2025 04:17:36 -0400 Received: from mx.kolabnow.com ([212.103.80.154]:59870) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1uFTlq-0004nd-0B for 78404@debbugs.gnu.org; Thu, 15 May 2025 04:16:53 -0400 Received: from localhost (unknown [127.0.0.1]) by mx.kolabnow.com (Postfix) with ESMTP id 5375330011B9 for <78404@debbugs.gnu.org>; Thu, 15 May 2025 10:16:43 +0200 (CEST) Authentication-Results: ext-mx-out013.mykolab.com (amavis); dkim=pass reason="pass (just generated, assumed good)" header.d=lambda.is DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=lambda.is; h= content-transfer-encoding:content-type:content-type:mime-version :references:in-reply-to:message-id:date:date:subject:subject :from:from:received:received:received; s=dkim2; t=1747297002; x= 1749111403; bh=5C+7J5y/Scdtphpvqi3YDdZElj3wse6xAVaDOfsflq4=; b=H /RC4VPsF37Z6ooUpON+855AsIcrw6CaIAf9JHaidjwlBqWjJ8G7I3Wpfu71Mzhqd j6czoePgx/irD7O0xoVBa8+2bWJpe/tXW6TsyiZz/kwMcFQqItv1TAs5ue2EpP1Z jPyjM/FkcDyVigm6if4Gl1ux/E9ZT0zB26HPzVMqrt2C8V4hXTnCGrOsEWt/fXyV +8Xer8EiQ1EARuiCPE663ufrBaP8/ZQ7ICSdDB9A15ggpPntJl6qVtEEMSqwqBaB n8WvktO3mD2P0jvh0Y6w6F7qAd+Yhj+Gvq+UEml+TJ8nypzB+PUFbtIIIzPHc70r 2fvULkZhrhtPpDwBHNJ5w== X-Virus-Scanned: amavis at mykolab.com Received: from mx.kolabnow.com ([127.0.0.1]) by localhost (ext-mx-out013.mykolab.com [127.0.0.1]) (amavis, port 10024) with ESMTP id jSfWVgmWnhBA for <78404@debbugs.gnu.org>; Thu, 15 May 2025 10:16:42 +0200 (CEST) Received: from int-mx011.mykolab.com (unknown [10.9.13.11]) by mx.kolabnow.com (Postfix) with ESMTPS id 067443001022 for <78404@debbugs.gnu.org>; Thu, 15 May 2025 10:16:41 +0200 (CEST) Received: from ext-subm010.mykolab.com (unknown [10.9.6.10]) by int-mx011.mykolab.com (Postfix) with ESMTPS id CA01B323EF18 for <78404@debbugs.gnu.org>; Thu, 15 May 2025 10:16:41 +0200 (CEST) From: =?utf-8?q?J=C3=B8rgen?= Kvalsvik Date: Thu, 15 May 2025 10:16:32 +0200 Message-Id: <20250515081635.575459-2-j@lambda.is> In-Reply-To: <20250515081635.575459-1-j@lambda.is> References: <20250515081635.575459-1-j@lambda.is> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org X-getmail-retrieved-from-mailbox: Patches Add a new downloader which implements, approximately, the download step of go get $module. This is a convenient way of downloading zips with go modules by just specifying the version and import path, as an alternative to git clone, or awkward https:// fetches. This is particularly useful for sources that are processed before release (like autotools generated files in tarballs) or generated modules. * guix/go-mod-download.scm: New file. * Makefile.am (MODULES): Add it. Change-Id: Ibb3b3ee70833fd0ea0c64278c95b8cb96a0be639 --- Makefile.am | 1 + guix/go-mod-download.scm | 126 +++++++++++++++++++++++++++++++++++++++ 2 files changed, 127 insertions(+) create mode 100644 guix/go-mod-download.scm diff --git a/Makefile.am b/Makefile.am index ec5220333e..b5fb81f412 100644 --- a/Makefile.am +++ b/Makefile.am @@ -102,6 +102,7 @@ MODULES = \ guix/android-repo-download.scm \ guix/bzr-download.scm \ guix/git-download.scm \ + guix/go-mod-download.scm \ guix/hg-download.scm \ guix/hash.scm \ guix/swh.scm \ diff --git a/guix/go-mod-download.scm b/guix/go-mod-download.scm new file mode 100644 index 0000000000..7024362318 --- /dev/null +++ b/guix/go-mod-download.scm @@ -0,0 +1,126 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright © 2025 Jørgen Kvalsvik +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see . + +(define-module (guix go-mod-download) + #:use-module (guix gexp) + #:use-module (guix store) + #:use-module (guix packages) + #:use-module (guix monads) + #:use-module (guix records) + #:use-module (guix modules) + #:use-module (guix download) + #:use-module (ice-9 string-fun) + #:use-module (ice-9 regex) + #:autoload (guix build-system gnu) (standard-packages) + #:export (go-mod-reference + go-mod-reference? + go-mod-reference-path + go-mod-reference-version + go-mod-fetch)) + +;;; Commentary: +;;; +;;; An method that fetches a go module [1] from a GOPROXY. A go +;;; module is usually identified as a vcs (usually git) repository, +;;; e.g. github.com/calmh/du or golang.org/x/net. +;;; +;;; This is mostly a regular http(s) fetch some custom url building. Unless +;;; goproxy is specified, it fetches from the default goproxy +;;; https://proxy.golang.org. This is mostly just a convenience -- the same +;;; code could be fetched directly, but sometimes libraries are only +;;; practically available through a goproxy. Such a module would be +;;; https://pkg.go.dev/buf.build/gen/go/bufbuild/protovalidate/protocolbuffers/go +;;; +;;; [1] https://go.dev/ref/mod +;;; +;;; Code: + +(define-record-type* + go-mod-reference make-go-mod-reference + go-mod-reference? + (path go-mod-reference-path) + (version go-mod-reference-version) + (goproxy go-mod-reference-goproxy (default "https://proxy.golang.org"))) + +(define (default-unzip) + "Return the 'unzip' package. This is a lazy reference so that we don't +depend on (gnu packages compression)." + (module-ref (resolve-interface '(gnu packages compression)) 'unzip)) + +;; Fetch a go module e.g. golang.org/x/net from a goproxy. +(define* (go-mod-fetch ref hash-algo hash + #:optional name + #:key (system (%current-system)) + (guile (default-guile)) + (unzip (default-unzip))) + (define inputs + `(("unzip" ,unzip) + ,@(standard-packages))) + + (define (go-path-escape path) + "Escape a module path by replacing every uppercase letter with an +exclamation mark followed with its lowercase equivalent, as per the module +Escaped Paths specification (see: +https://godoc.org/golang.org/x/mod/module#hdr-Escaped_Paths)." + (define (escape occurrence) + (string-append "!" (string-downcase (match:substring occurrence)))) + (regexp-substitute/global #f "[A-Z]" path 'pre escape 'post)) + + (define modules + (source-module-closure '((guix build utils)))) + + (define (build mod.zip) + (with-imported-modules modules + #~(begin + (use-modules (guix build utils)) + (let* ((pkg-path (getenv "go-mod path")) + (pkg-version (getenv "go-mod version")) + (pkg-root (string-append pkg-path "@v" pkg-version)) + (go.mod (string-append pkg-root "/go.mod"))) + + (invoke (string-append #+unzip "/bin/unzip") "-q" #$mod.zip) + ;; The sources in the zip are in the subdir + ;; $path@v$version/, but we want our sources at root. + (copy-recursively pkg-root #$output))))) + + (define path-as-store-name + (string-append + (string-replace-substring (go-mod-reference-path ref) "/" "-") + "-" (go-mod-reference-version ref))) + + (define url/zip + (format #f "~a/~a/@v/v~a.zip" + (go-mod-reference-goproxy ref) + (go-path-escape (go-mod-reference-path ref)) + (go-mod-reference-version ref))) + + (mlet* %store-monad ((guile-for-build (package->derivation guile system)) + (mod (url-fetch url/zip hash-algo hash + (or name (string-append path-as-store-name ".zip")) + #:system system + #:guile guile))) + (gexp->derivation (or name path-as-store-name) (build mod) + #:script-name "go-mod-fetch" + #:env-vars + `(("go-mod path" . ,(go-mod-reference-path ref)) + ("go-mod version" . ,(go-mod-reference-version ref))) + #:leaked-env-vars '("http_proxy" "https_proxy" + "LC_ALL" "LC_MESSAGES" "LANG" + "COLUMNS") + #:system system + #:local-build? #t)))