From patchwork Sat Mar 16 10:30:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christina O'Donnell X-Patchwork-Id: 61868 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 4DA7927BBEC; Sat, 16 Mar 2024 10:31:36 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=BAYES_00,MAILING_LIST_MULTI, SPF_HELO_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id F2CDF27BBEA for ; Sat, 16 Mar 2024 10:31:34 +0000 (GMT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rlRK4-0001td-1p; Sat, 16 Mar 2024 06:31:28 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rlRK2-0001t8-Ll for guix-patches@gnu.org; Sat, 16 Mar 2024 06:31:26 -0400 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rlRK2-0001QZ-0z for guix-patches@gnu.org; Sat, 16 Mar 2024 06:31:26 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1rlRKc-00061Z-PW; Sat, 16 Mar 2024 06:32:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#69827] [PATCH 2/3] import/go: Account for monorepo modules in the Go importer. Resent-From: Christina O'Donnell Original-Sender: "Debbugs-submit" Resent-CC: cox.katherine.e+guix@gmail.com, sharlatanus@gmail.com, guix-patches@gnu.org Resent-Date: Sat, 16 Mar 2024 10:32:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 69827 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 69827@debbugs.gnu.org Cc: Christina O'Donnell , Katherine Cox-Buday , Sharlatan Hellseher X-Debbugs-Original-Xcc: Katherine Cox-Buday , Sharlatan Hellseher Received: via spool by 69827-submit@debbugs.gnu.org id=B69827.171058507823050 (code B ref 69827); Sat, 16 Mar 2024 10:32:02 +0000 Received: (at 69827) by debbugs.gnu.org; 16 Mar 2024 10:31:18 +0000 Received: from localhost ([127.0.0.1]:54886 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rlRJt-0005zh-TS for submit@debbugs.gnu.org; Sat, 16 Mar 2024 06:31:18 -0400 Received: from vmi993448.contaboserver.net ([194.163.141.236]:40062 helo=mutix.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rlRJk-0005zJ-NX for 69827@debbugs.gnu.org; Sat, 16 Mar 2024 06:31:09 -0400 Received: from [86.132.246.87] (host81-147-82-218.range81-147.btcentralplus.com [81.147.82.218]) (Authenticated sender: cdo) by mutix.org (Postfix) with ESMTPSA id 29418A63582; Sat, 16 Mar 2024 11:30:31 +0100 (CET) From: Christina O'Donnell Date: Sat, 16 Mar 2024 10:30:21 +0000 Message-ID: X-Mailer: git-send-email 2.41.0 In-Reply-To: References: MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org X-getmail-retrieved-from-mailbox: Patches This change allows for importing of modules situated in repos where they're not at the root directory. * guix/import/go.scm (go-module-verstion-string): Delete (go-module-version-info): New procedure (module-path->repository-root): Add parameter (vcs->origin): Add parameter (path-diff): New procedure Change-Id: If50f7a951ce8e527e5ea44ed24db10d6a3676ff0 --- guix/import/go.scm | 71 +++++++++++++++++++++++++++++++++------------- 1 file changed, 51 insertions(+), 20 deletions(-) diff --git a/guix/import/go.scm b/guix/import/go.scm index dd9298808d..8276797d9a 100644 --- a/guix/import/go.scm +++ b/guix/import/go.scm @@ -122,15 +122,14 @@ (define (go-path-escape path) (define (go.pkg.dev-info name) (http-fetch* (string-append "https://pkg.go.dev/" name))) -(define* (go-module-version-string goproxy name #:key version) - "Fetch the version string of the latest version for NAME from the given +(define* (go-module-version-info goproxy name #:key version) + "Fetch a JSON object encoding about the lastest version for NAME from the given GOPROXY server, or for VERSION when specified." (let ((file (if version (string-append "@v/" version ".info") "@latest"))) - (assoc-ref (json-fetch* (format #f "~a/~a/~a" - goproxy (go-path-escape name) file)) - "Version"))) + (json-fetch* (format #f "~a/~a/~a" + goproxy (go-path-escape name) file)))) (define* (go-module-available-versions goproxy name) "Retrieve the available versions for a given module from the module proxy. @@ -140,8 +139,12 @@ (define* (go-module-available-versions goproxy name) (body (http-fetch* url)) (versions (remove string-null? (string-split body #\newline)))) (if (null? versions) - (list (go-module-version-string goproxy name)) ;latest version - versions))) + (begin + ;; If we haven't recieved any versions, look in the version-info json + ;; object and return a one-element list if found. + (or (and=> (assoc-ref (go-module-version-info goproxy name) "Version") + list)))) + versions)) (define (go-package-licenses name) "Retrieve the list of licenses that apply to NAME, a Go package or module @@ -431,7 +434,7 @@ (define known-vcs (/[A-Za-z0-9_.\\-]+)*$" 'git))) -(define (module-path->repository-root module-path) +(define (module-path->repository-root module-path version-info) "Infer the repository root from a module path. Go modules can be defined at any level of a repository tree, but querying for the meta tag usually can only be done from the web page at the root of the repository, @@ -452,6 +455,17 @@ (define (module-path->repository-root module-path) (lambda (vcs) (match:substring (regexp-exec (vcs-root-regex vcs) module-path) 1))) + (and=> (assoc-ref version-info "Origin") + (lambda (origin) + (and=> (assoc-ref origin "Subdir") + (lambda (subdir) + ;; If version-info contains a 'subdir' and that is a suffix, + ;; then the repo-root can be found by stripping off the + ;; suffix. + (if (string-suffix? (string-append "/" subdir) module-path) + (string-drop-right module-path + (+ 1 (string-length subdir))) + #f))))) (vcs-qualified-module-path->root-repo-url module-path) module-path)) @@ -534,13 +548,21 @@ (define* (git-checkout-hash url reference algorithm) `(tag-or-commit . ,reference))))) (file-hash* checkout #:algorithm algorithm #:recursive? #true))) -(define (vcs->origin vcs-type vcs-repo-url version) +(define (vcs->origin vcs-type vcs-repo-url version subdir) "Generate the `origin' block of a package depending on what type of source control system is being used." (case vcs-type ((git) - (let ((plain-version? (string=? version (go-version->git-ref version))) - (v-prefixed? (string-prefix? "v" version))) + (let* ((plain-version? (string=? version (go-version->git-ref version + #:subdir subdir))) + (v-prefixed? (string-prefix? "v" version)) + ;; This is done because the version field of the package, + ;; which the generated quoted expression refers to, has been + ;; stripped of any 'v' prefixed. + (version-expr (if (and plain-version? v-prefixed?) + '(string-append "v" version) + `(go-version->git-ref version + ,@(if subdir `(#:subdir ,subdir) '()))))) `(origin (method git-fetch) (uri (git-reference @@ -548,14 +570,13 @@ (define (vcs->origin vcs-type vcs-repo-url version) ;; This is done because the version field of the package, ;; which the generated quoted expression refers to, has been ;; stripped of any 'v' prefixed. - (commit ,(if (and plain-version? v-prefixed?) - '(string-append "v" version) - '(go-version->git-ref version))))) + (commit ,version-expr))) (file-name (git-file-name name version)) (sha256 (base32 ,(bytevector->nix-base32-string - (git-checkout-hash vcs-repo-url (go-version->git-ref version) + (git-checkout-hash vcs-repo-url (go-version->git-ref version + #:subdir subdir) (hash-algorithm sha256)))))))) ((hg) `(origin @@ -612,6 +633,12 @@ (define (validate-version version available-versions module-path) (map strip-v-prefix available-versions))))))))) +(define (path-diff parent child) + (if (and (string-prefix? parent child) (not (string=? parent child))) + (let ((parent-len (string-length parent))) + (string-trim (substring child parent-len) (char-set #\/))) + #f)) + (define* (go-module->guix-package module-path #:key (goproxy "https://proxy.golang.org") version @@ -623,9 +650,11 @@ (define* (go-module->guix-package module-path #:key (let* ((available-versions (go-module-available-versions goproxy module-path)) (version* (validate-version (or (and version (ensure-v-prefix version)) - (go-module-version-string goproxy module-path)) ;latest + (assoc-ref (go-module-version-info goproxy module-path) + "Version")) ;latest available-versions module-path)) + (version-info (go-module-version-info goproxy module-path #:version version*)) (content (fetch-go.mod goproxy module-path version*)) (min-go-version (second (go.mod-go-version (parse-go.mod content)))) (dependencies+versions (go.mod-requirements (parse-go.mod content))) @@ -634,11 +663,13 @@ (define* (go-module->guix-package module-path #:key (map car dependencies+versions))) (module-path-sans-suffix (match:prefix (string-match "([\\./]v[0-9]+)?$" module-path))) - (guix-name (go-module->guix-package-name module-path)) - (root-module-path (module-path->repository-root module-path)) + (guix-name (go-module->guix-package-name module-path-sans-suffix )) + (root-module-path (module-path->repository-root module-path-sans-suffix + version-info)) ;; The VCS type and URL are not included in goproxy information. For ;; this we need to fetch it from the official module page. (meta-data (fetch-module-meta-data root-module-path)) + (subdir (path-diff root-module-path module-path-sans-suffix)) (vcs-type (module-meta-vcs meta-data)) (vcs-repo-url (module-meta-data-repo-url meta-data goproxy)) (synopsis (go-package-synopsis module-path)) @@ -649,14 +680,14 @@ (define* (go-module->guix-package module-path #:key (name ,guix-name) (version ,(strip-v-prefix version*)) (source - ,(vcs->origin vcs-type vcs-repo-url version*)) + ,(vcs->origin vcs-type vcs-repo-url version* subdir)) (build-system go-build-system) (arguments (list ,@(if (version>? min-go-version (package-version (go-package))) `(#:go ,(string->number min-go-version)) '()) #:import-path ,module-path - ,@(if (string=? module-path-sans-suffix root-module-path) + ,@(if (string=? module-path root-module-path) '() `(#:unpack-path ,root-module-path)))) ,@(maybe-propagated-inputs