From patchwork Tue May 13 09:55:21 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?J=C3=B8rgen_Kvalsvik?= X-Patchwork-Id: 42574 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 0DBE327BC4D; Tue, 13 May 2025 10:56:39 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-6.4 required=5.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_BLOCKED, RCVD_IN_VALIDITY_CERTIFIED,RCVD_IN_VALIDITY_RPBL,RCVD_IN_VALIDITY_SAFE, SPF_HELO_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 2AAFA27BC4B for ; Tue, 13 May 2025 10:56:38 +0100 (BST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uEmMq-0005zt-El; Tue, 13 May 2025 05:56:08 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uEmMl-0005zV-J5 for guix-patches@gnu.org; Tue, 13 May 2025 05:56:04 -0400 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1uEmMk-0006Mr-Rv for guix-patches@gnu.org; Tue, 13 May 2025 05:56:03 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=debbugs.gnu.org; s=debbugs-gnu-org; h=MIME-Version:Date:From:To:In-Reply-To:References:Subject; bh=5C+7J5y/Scdtphpvqi3YDdZElj3wse6xAVaDOfsflq4=; b=rsugxJHGjGYs29+6uZoo36Dj8sxLexatm0h2MDox+Ngr0FfcqvYagTvBNjwOsireawkJy94lPKtwYWYpWL9RrWVENfygXIq9IIvfbdtGA9poHEecLh9tEU51nR5EjDS1peFbuejpShmssBTXj3yFDUBwLRQ8sEpBb8tInQ4WNFO8suW3VMa8btBsjtrx6uj8hmyMy3V4c67l8GqPbhFDYZ/us7+mS0DP9flrK+ZpkL6ohPCXAgyR3FfECS9G9ecObN5erQJ8C+A2Jk9r9UyMUYH76iTx5bp0aOtbqlLMsom60ZlApm5LoM9pBeL+Wr+a0h9ddDJY28dwL9zV7SKojQ==; Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1uEmMk-0000RT-Me for guix-patches@gnu.org; Tue, 13 May 2025 05:56:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#78404] [PATCH 1/2] guix: Add downloader for go modules from GOPROXY References: <20250513094023.348947-1-j@lambda.is> In-Reply-To: <20250513094023.348947-1-j@lambda.is> Resent-From: =?utf-8?q?J=C3=B8rgen?= Kvalsvik Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Tue, 13 May 2025 09:56:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 78404 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 78404@debbugs.gnu.org Cc: =?utf-8?q?J=C3=B8rgen?= Kvalsvik , steve@futurile.net Received: via spool by 78404-submit@debbugs.gnu.org id=B78404.17471301431651 (code B ref 78404); Tue, 13 May 2025 09:56:02 +0000 Received: (at 78404) by debbugs.gnu.org; 13 May 2025 09:55:43 +0000 Received: from localhost ([127.0.0.1]:58112 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1uEmMQ-0000QT-UE for submit@debbugs.gnu.org; Tue, 13 May 2025 05:55:43 -0400 Received: from mx.kolabnow.com ([212.103.80.154]:60818) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1uEmMJ-0000Pv-9R for 78404@debbugs.gnu.org; Tue, 13 May 2025 05:55:38 -0400 Received: from localhost (unknown [127.0.0.1]) by mx.kolabnow.com (Postfix) with ESMTP id AEF3630051EA; Tue, 13 May 2025 11:55:28 +0200 (CEST) Authentication-Results: ext-mx-out013.mykolab.com (amavis); dkim=pass reason="pass (just generated, assumed good)" header.d=lambda.is DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=lambda.is; h= content-transfer-encoding:content-type:content-type:mime-version :message-id:date:date:subject:subject:from:from:received :received:received; s=dkim2; t=1747130128; x=1748944529; bh=5C+7 J5y/Scdtphpvqi3YDdZElj3wse6xAVaDOfsflq4=; b=F94OWYuzYegJzQCHCf8e UUiGzw4W3z+CxOYXEAqoZAjqYBYKOYaKsB70qkTKHd7fHGdsthPU0A4tVqpdgm2n p4AO+FTI89uoVqFvGajSY2G307XFPHXrKXt8GV1eoh1WhmN3ndmXNKniPTG3rXgk E17pamfYT4BUntfEy2pPbHuYPqTDLN9JnBMnWvVc0tN7NDOXoSdGv4+tAuNhBFTX m4g7w8BtXhmEde4UIVby0qqT6yIBnzebq9AbO1lHGVbEO0/mjitL9uhQgJ9As3Ay JeVno5i2P6b4WoaxJ+zsDcdHZ/LIw2sXhlg9RHOoH3EXD7gBA42Pidhsj8L8drkh Dg== X-Virus-Scanned: amavis at mykolab.com Received: from mx.kolabnow.com ([127.0.0.1]) by localhost (ext-mx-out013.mykolab.com [127.0.0.1]) (amavis, port 10024) with ESMTP id QgJYZEJlkwfJ; Tue, 13 May 2025 11:55:28 +0200 (CEST) Received: from int-mx009.mykolab.com (unknown [10.9.13.9]) by mx.kolabnow.com (Postfix) with ESMTPS id E736930051E3; Tue, 13 May 2025 11:55:27 +0200 (CEST) Received: from ext-subm010.mykolab.com (unknown [10.9.6.10]) by int-mx009.mykolab.com (Postfix) with ESMTPS id 9974620C19A5; Tue, 13 May 2025 11:55:27 +0200 (CEST) From: =?utf-8?q?J=C3=B8rgen?= Kvalsvik Date: Tue, 13 May 2025 11:55:21 +0200 Message-Id: <20250513095522.4313-1-j@lambda.is> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org X-getmail-retrieved-from-mailbox: Patches Add a new downloader which implements, approximately, the download step of go get $module. This is a convenient way of downloading zips with go modules by just specifying the version and import path, as an alternative to git clone, or awkward https:// fetches. This is particularly useful for sources that are processed before release (like autotools generated files in tarballs) or generated modules. * guix/go-mod-download.scm: New file. * Makefile.am (MODULES): Add it. Change-Id: Ibb3b3ee70833fd0ea0c64278c95b8cb96a0be639 --- Makefile.am | 1 + guix/go-mod-download.scm | 126 +++++++++++++++++++++++++++++++++++++++ 2 files changed, 127 insertions(+) create mode 100644 guix/go-mod-download.scm diff --git a/Makefile.am b/Makefile.am index ec5220333e..b5fb81f412 100644 --- a/Makefile.am +++ b/Makefile.am @@ -102,6 +102,7 @@ MODULES = \ guix/android-repo-download.scm \ guix/bzr-download.scm \ guix/git-download.scm \ + guix/go-mod-download.scm \ guix/hg-download.scm \ guix/hash.scm \ guix/swh.scm \ diff --git a/guix/go-mod-download.scm b/guix/go-mod-download.scm new file mode 100644 index 0000000000..7024362318 --- /dev/null +++ b/guix/go-mod-download.scm @@ -0,0 +1,126 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright © 2025 Jørgen Kvalsvik +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see . + +(define-module (guix go-mod-download) + #:use-module (guix gexp) + #:use-module (guix store) + #:use-module (guix packages) + #:use-module (guix monads) + #:use-module (guix records) + #:use-module (guix modules) + #:use-module (guix download) + #:use-module (ice-9 string-fun) + #:use-module (ice-9 regex) + #:autoload (guix build-system gnu) (standard-packages) + #:export (go-mod-reference + go-mod-reference? + go-mod-reference-path + go-mod-reference-version + go-mod-fetch)) + +;;; Commentary: +;;; +;;; An method that fetches a go module [1] from a GOPROXY. A go +;;; module is usually identified as a vcs (usually git) repository, +;;; e.g. github.com/calmh/du or golang.org/x/net. +;;; +;;; This is mostly a regular http(s) fetch some custom url building. Unless +;;; goproxy is specified, it fetches from the default goproxy +;;; https://proxy.golang.org. This is mostly just a convenience -- the same +;;; code could be fetched directly, but sometimes libraries are only +;;; practically available through a goproxy. Such a module would be +;;; https://pkg.go.dev/buf.build/gen/go/bufbuild/protovalidate/protocolbuffers/go +;;; +;;; [1] https://go.dev/ref/mod +;;; +;;; Code: + +(define-record-type* + go-mod-reference make-go-mod-reference + go-mod-reference? + (path go-mod-reference-path) + (version go-mod-reference-version) + (goproxy go-mod-reference-goproxy (default "https://proxy.golang.org"))) + +(define (default-unzip) + "Return the 'unzip' package. This is a lazy reference so that we don't +depend on (gnu packages compression)." + (module-ref (resolve-interface '(gnu packages compression)) 'unzip)) + +;; Fetch a go module e.g. golang.org/x/net from a goproxy. +(define* (go-mod-fetch ref hash-algo hash + #:optional name + #:key (system (%current-system)) + (guile (default-guile)) + (unzip (default-unzip))) + (define inputs + `(("unzip" ,unzip) + ,@(standard-packages))) + + (define (go-path-escape path) + "Escape a module path by replacing every uppercase letter with an +exclamation mark followed with its lowercase equivalent, as per the module +Escaped Paths specification (see: +https://godoc.org/golang.org/x/mod/module#hdr-Escaped_Paths)." + (define (escape occurrence) + (string-append "!" (string-downcase (match:substring occurrence)))) + (regexp-substitute/global #f "[A-Z]" path 'pre escape 'post)) + + (define modules + (source-module-closure '((guix build utils)))) + + (define (build mod.zip) + (with-imported-modules modules + #~(begin + (use-modules (guix build utils)) + (let* ((pkg-path (getenv "go-mod path")) + (pkg-version (getenv "go-mod version")) + (pkg-root (string-append pkg-path "@v" pkg-version)) + (go.mod (string-append pkg-root "/go.mod"))) + + (invoke (string-append #+unzip "/bin/unzip") "-q" #$mod.zip) + ;; The sources in the zip are in the subdir + ;; $path@v$version/, but we want our sources at root. + (copy-recursively pkg-root #$output))))) + + (define path-as-store-name + (string-append + (string-replace-substring (go-mod-reference-path ref) "/" "-") + "-" (go-mod-reference-version ref))) + + (define url/zip + (format #f "~a/~a/@v/v~a.zip" + (go-mod-reference-goproxy ref) + (go-path-escape (go-mod-reference-path ref)) + (go-mod-reference-version ref))) + + (mlet* %store-monad ((guile-for-build (package->derivation guile system)) + (mod (url-fetch url/zip hash-algo hash + (or name (string-append path-as-store-name ".zip")) + #:system system + #:guile guile))) + (gexp->derivation (or name path-as-store-name) (build mod) + #:script-name "go-mod-fetch" + #:env-vars + `(("go-mod path" . ,(go-mod-reference-path ref)) + ("go-mod version" . ,(go-mod-reference-version ref))) + #:leaked-env-vars '("http_proxy" "https_proxy" + "LC_ALL" "LC_MESSAGES" "LANG" + "COLUMNS") + #:system system + #:local-build? #t)))