From patchwork Sat Feb 3 23:07:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Nicolas Graves X-Patchwork-Id: 22784 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 808B727BBEA; Sat, 3 Feb 2024 23:09:31 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-3.9 required=5.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_MSPIKE_H5,RCVD_IN_MSPIKE_WL,SPF_HELO_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 51A0F27BBE9 for ; Sat, 3 Feb 2024 23:09:28 +0000 (GMT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rWP82-0008CF-Rh; Sat, 03 Feb 2024 18:08:54 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rWP7z-0008Ah-5V for guix-patches@gnu.org; Sat, 03 Feb 2024 18:08:51 -0500 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rWP7y-0008Dz-TG for guix-patches@gnu.org; Sat, 03 Feb 2024 18:08:50 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1rWP8A-0001HG-95 for guix-patches@gnu.org; Sat, 03 Feb 2024 18:09:02 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#62202] [PATCH v4 1/6] import: utils: Add function git->origin. References: <871qlq89kz.fsf@ngraves.fr> In-Reply-To: <871qlq89kz.fsf@ngraves.fr> Resent-From: Nicolas Graves Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sat, 03 Feb 2024 23:09:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 62202 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch moreinfo To: 62202@debbugs.gnu.org Cc: ngraves@ngraves.fr, zimoun.toutoune@gmail.com Received: via spool by 62202-submit@debbugs.gnu.org id=B62202.17070017104826 (code B ref 62202); Sat, 03 Feb 2024 23:09:02 +0000 Received: (at 62202) by debbugs.gnu.org; 3 Feb 2024 23:08:30 +0000 Received: from localhost ([127.0.0.1]:47244 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rWP7c-0001FX-1Q for submit@debbugs.gnu.org; Sat, 03 Feb 2024 18:08:29 -0500 Received: from 5.mo576.mail-out.ovh.net ([46.105.43.105]:55199) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rWP7Y-0001FD-7g for 62202@debbugs.gnu.org; Sat, 03 Feb 2024 18:08:27 -0500 Received: from director9.ghost.mail-out.ovh.net (unknown [10.109.139.43]) by mo576.mail-out.ovh.net (Postfix) with ESMTP id 135DE30869 for <62202@debbugs.gnu.org>; Sat, 3 Feb 2024 23:08:10 +0000 (UTC) Received: from ghost-submission-6684bf9d7b-sb92c (unknown [10.110.178.25]) by director9.ghost.mail-out.ovh.net (Postfix) with ESMTPS id 4F5CC1FD59; Sat, 3 Feb 2024 23:08:10 +0000 (UTC) Received: from ngraves.fr ([37.59.142.107]) by ghost-submission-6684bf9d7b-sb92c with ESMTPSA id YZ6MD1rHvmW7PAAAkXgBxg (envelope-from ); Sat, 03 Feb 2024 23:08:10 +0000 Authentication-Results: garm.ovh; auth=pass (GARM-107S001b9bb2030-5a2b-4bdf-a53a-435e179e130d, CDD90146079FDB87F723A7A7FD21F5F4569C6C59) smtp.auth=ngraves@ngraves.fr X-OVh-ClientIp: 81.67.146.208 Date: Sun, 4 Feb 2024 00:07:11 +0100 Message-ID: <20240203230807.25751-1-ngraves@ngraves.fr> X-Mailer: git-send-email 2.41.0 MIME-Version: 1.0 X-Ovh-Tracer-Id: 14364793964043625063 X-VR-SPAMSTATE: OK X-VR-SPAMSCORE: 0 X-VR-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrgedvkedrfedujedgtdejucetufdoteggodetrfdotffvucfrrhhofhhilhgvmecuqfggjfdpvefjgfevmfevgfenuceurghilhhouhhtmecuhedttdenucenucfjughrpefhvfevufffkffogggtgfesthekredtredtjeenucfhrhhomheppfhitgholhgrshcuifhrrghvvghsuceonhhgrhgrvhgvshesnhhgrhgrvhgvshdrfhhrqeenucggtffrrghtthgvrhhnpefgfeejveevtdfgieekieejueffhefhtdetudffgedvkedvudejgeejieffvdetffenucffohhmrghinhepghhithhlrggsrdgtohhmpdhsphgugidrohhrghdpghhithhhuhgsrdgtohhmnecukfhppeduvdejrddtrddtrddupdekuddrieejrddugeeirddvtdekpdefjedrheelrddugedvrddutdejnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehinhgvthepuddvjedrtddrtddruddpmhgrihhlfhhrohhmpeeonhhgrhgrvhgvshesnhhgrhgrvhgvshdrfhhrqedpnhgspghrtghpthhtohepuddprhgtphhtthhopeeivddvtddvseguvggssghughhsrdhgnhhurdhorhhgpdfovfetjfhoshhtpehmohehjeeipdhmohguvgepshhmthhpohhuth X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-to: Nicolas Graves X-ACL-Warn: , Nicolas Graves via Guix-patches X-Patchwork-Original-From: Nicolas Graves via Guix-patches via From: Nicolas Graves Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org X-getmail-retrieved-from-mailbox: Patches * guix/import/utils.scm: (git->origin): Add function. * guix/import/elpa.scm (download-git-repository): Remove function download-git-repository. (git-repository->origin): Remove function git-repository->origin. (ref): Add function ref. (melpa-recipe->origin): Use functions git->origin and ref. * guix/import/go.scm (git-checkout-hash): Remove function git-checkout-hash. (transform-version): Add function transform-version. (vcs->origin): Use functions git->origin and transform-version. Add optional argument transform-version. * tests/import/go.scm (go-module->guix-package): Adapt test case to changes in guix/import/go.scm. * guix/import/minetest.scm (download-git-repository): Remove function download-git-repository. (make-minetest-sexp): Use function git->origin. * tests/minetest.scm (make-package-sexp): Use function git->origin. (example-package): Adapt test-case to git->origin. * guix/import/composer.scm (make-php-sexp): Use function git->origin. Change-Id: Ied05a63bdd60fbafe26fbbb4e115ff6f0bb9db3c --- guix/import/composer.scm | 86 ++++++++++++++-------------------------- guix/import/elpa.scm | 43 ++++++-------------- guix/import/go.scm | 57 ++++++++------------------ guix/import/minetest.scm | 28 ++----------- guix/import/utils.scm | 39 ++++++++++++++++++ tests/go.scm | 29 ++++++++++---- tests/minetest.scm | 15 ++----- 7 files changed, 127 insertions(+), 170 deletions(-) diff --git a/guix/import/composer.scm b/guix/import/composer.scm index 1ad608964b..dabc5423ed 100644 --- a/guix/import/composer.scm +++ b/guix/import/composer.scm @@ -19,22 +19,17 @@ (define-module (guix import composer) #:use-module (ice-9 match) #:use-module (json) - #:use-module (guix hash) - #:use-module (guix base32) - #:use-module (guix build git) - #:use-module (guix build utils) - #:use-module (guix build-system) #:use-module (guix build-system composer) + #:use-module ((guix download) #:select (download-to-store)) #:use-module (guix import json) #:use-module (guix import utils) #:use-module ((guix licenses) #:prefix license:) #:use-module (guix memoization) #:use-module (guix packages) - #:use-module (guix serialization) + #:use-module (guix store) #:use-module (guix upstream) #:use-module (guix utils) #:use-module (srfi srfi-1) - #:use-module (srfi srfi-11) #:use-module (srfi srfi-26) #:export (composer->guix-package %composer-updater @@ -141,55 +136,34 @@ (define (make-php-sexp composer-package) (dependencies (map php-package-name (composer-package-require composer-package))) (dev-dependencies (map php-package-name - (composer-package-dev-require composer-package))) - (git? (equal? (composer-source-type source) "git"))) - ((if git? call-with-temporary-directory call-with-temporary-output-file) - (lambda* (temp #:optional port) - (and (if git? - (begin - (mkdir-p temp) - (git-fetch (composer-source-url source) - (composer-source-reference source) - temp)) - (url-fetch (composer-source-url source) temp)) - `(package - (name ,(composer-package-name composer-package)) - (version ,(composer-package-version composer-package)) - (source - (origin - ,@(if git? - `((method git-fetch) - (uri (git-reference - (url ,(if (string-suffix? - ".git" - (composer-source-url source)) - (string-drop-right - (composer-source-url source) - (string-length ".git")) - (composer-source-url source))) - (commit ,(composer-source-reference source)))) - (file-name (git-file-name name version)) - (sha256 - (base32 - ,(bytevector->nix-base32-string - (file-hash* temp))))) - `((method url-fetch) - (uri ,(composer-source-url source)) - (sha256 (base32 ,(guix-hash-url temp))))))) - (build-system composer-build-system) - ,@(if (null? dependencies) - '() - `((inputs - (list ,@(map string->symbol dependencies))))) - ,@(if (null? dev-dependencies) - '() - `((native-inputs - (list ,@(map string->symbol dev-dependencies))))) - (synopsis "") - (description ,(composer-package-description composer-package)) - (home-page ,(composer-package-homepage composer-package)) - (license ,(or (composer-package-license composer-package) - 'unknown-license!)))))))) + (composer-package-dev-require composer-package)))) + `(package + (name ,(composer-package-name composer-package)) + (version ,(composer-package-version composer-package)) + (source + ,(if (string= (composer-source-type source) "git") + (git->origin (composer-source-url source) + `(tag-or-commit . ,(composer-source-reference source))) + (let* ((source (composer-source-url source)) + (tarball (with-store store (download-to-store store source)))) + `(origin + (method url-fetch) + (uri ,source) + (sha256 (base32 ,(guix-hash-url tarball))))))) + (build-system composer-build-system) + ,@(if (null? dependencies) + '() + `((inputs + (list ,@(map string->symbol dependencies))))) + ,@(if (null? dev-dependencies) + '() + `((native-inputs + (list ,@(map string->symbol dev-dependencies))))) + (synopsis "") + (description ,(composer-package-description composer-package)) + (home-page ,(composer-package-homepage composer-package)) + (license ,(or (composer-package-license composer-package) + 'unknown-license!))))) (define composer->guix-package (memoize diff --git a/guix/import/elpa.scm b/guix/import/elpa.scm index d1855b3698..a755387242 100644 --- a/guix/import/elpa.scm +++ b/guix/import/elpa.scm @@ -8,6 +8,7 @@ ;;; Copyright © 2021 Sarah Morgensen ;;; Copyright © 2021 Simon Tournier ;;; Copyright © 2022 Hartmut Goebel +;;; Copyright © 2023 Nicolas Graves ;;; ;;; This file is part of GNU Guix. ;;; @@ -208,11 +209,6 @@ (define* (fetch-elpa-package name #:optional (repo 'gnu)) url))) (_ #f)))) -(define* (download-git-repository url ref) - "Fetch the given REF from the Git repository at URL." - (with-store store - (latest-repository-commit store url #:ref ref))) - (define (package-name->melpa-recipe package-name) "Fetch the MELPA recipe for PACKAGE-NAME, represented as an alist from keywords to values." @@ -232,28 +228,15 @@ (define (data->recipe data) (close-port port) (data->recipe (cons ':name data)))) -(define (git-repository->origin recipe url) - "Fetch origin details from the Git repository at URL for the provided MELPA -RECIPE." - (define ref - (cond - ((assoc-ref recipe #:branch) - => (lambda (branch) (cons 'branch branch))) - ((assoc-ref recipe #:commit) - => (lambda (commit) (cons 'commit commit))) - (else - '()))) - - (let-values (((directory commit) (download-git-repository url ref))) - `(origin - (method git-fetch) - (uri (git-reference - (url ,url) - (commit ,commit))) - (sha256 - (base32 - ,(bytevector->nix-base32-string - (file-hash* directory #:recursive? #true))))))) +(define (ref recipe) + "Create REF from MELPA RECIPE." + (cond + ((assoc-ref recipe #:branch) + => (lambda (branch) (cons 'branch branch))) + ((assoc-ref recipe #:commit) + => (lambda (commit) (cons 'commit commit))) + (else + '()))) (define* (melpa-recipe->origin recipe) "Fetch origin details from the MELPA recipe and associated repository for @@ -264,9 +247,9 @@ (define (gitlab-repo->url repo) (string-append "https://gitlab.com/" repo ".git")) (match (assq-ref recipe ':fetcher) - ('github (git-repository->origin recipe (github-repo->url (assq-ref recipe ':repo)))) - ('gitlab (git-repository->origin recipe (gitlab-repo->url (assq-ref recipe ':repo)))) - ('git (git-repository->origin recipe (assq-ref recipe ':url))) + ('github (git->origin (github-repo->url (assq-ref recipe ':repo)) (ref recipe))) + ('gitlab (git->origin (gitlab-repo->url (assq-ref recipe ':repo)) (ref recipe))) + ('git (git->origin (assq-ref recipe ':url) (ref recipe))) (#f #f) ; if we're not using melpa then this stops us printing a warning (_ (warning (G_ "unsupported MELPA fetcher: ~a, falling back to unstable MELPA source~%") (assq-ref recipe ':fetcher)) diff --git a/guix/import/go.scm b/guix/import/go.scm index dd9298808d..6e2ce2ed00 100644 --- a/guix/import/go.scm +++ b/guix/import/go.scm @@ -8,6 +8,7 @@ ;;; Copyright © 2021 Sarah Morgensen ;;; Copyright © 2021 Simon Tournier ;;; Copyright © 2023 Efraim Flashner +;;; Copyright © 2023 Nicolas Graves ;;; ;;; This file is part of GNU Guix. ;;; @@ -514,49 +515,24 @@ (define (module-meta-data-repo-url meta-data goproxy) goproxy (module-meta-repo-root meta-data))) -(define* (git-checkout-hash url reference algorithm) - "Return the ALGORITHM hash of the checkout of URL at REFERENCE, a commit or -tag." - (define cache - (string-append (or (getenv "TMPDIR") "/tmp") - "/guix-import-go-" - (passwd:name (getpwuid (getuid))))) +;; This is done because the version field of the package, which the generated +;; quoted expression refers to, has been stripped of any 'v' prefixed. +(define (transform-version version) + (let ((plain-version? (string=? version (go-version->git-ref version))) + (v-prefixed? (string-prefix? "v" version))) + (if (and plain-version? v-prefixed?) + '(string-append "v" version) + '(go-version->git-ref version)))) - ;; Use a custom cache to avoid cluttering the default one under - ;; ~/.cache/guix, but choose one under /tmp so that it's persistent across - ;; subsequent "guix import" invocations. - (mkdir-p cache) - (chmod cache #o700) - (let-values (((checkout commit _) - (parameterize ((%repository-cache-directory cache)) - (update-cached-checkout url - #:ref - `(tag-or-commit . ,reference))))) - (file-hash* checkout #:algorithm algorithm #:recursive? #true))) - -(define (vcs->origin vcs-type vcs-repo-url version) +(define* (vcs->origin vcs-type vcs-repo-url version + #:key (transform-version #f)) "Generate the `origin' block of a package depending on what type of source -control system is being used." +control system is being used. Optionally use the function TRANSFORM-VERSION +which takes version as an input." (case vcs-type ((git) - (let ((plain-version? (string=? version (go-version->git-ref version))) - (v-prefixed? (string-prefix? "v" version))) - `(origin - (method git-fetch) - (uri (git-reference - (url ,vcs-repo-url) - ;; This is done because the version field of the package, - ;; which the generated quoted expression refers to, has been - ;; stripped of any 'v' prefixed. - (commit ,(if (and plain-version? v-prefixed?) - '(string-append "v" version) - '(go-version->git-ref version))))) - (file-name (git-file-name name version)) - (sha256 - (base32 - ,(bytevector->nix-base32-string - (git-checkout-hash vcs-repo-url (go-version->git-ref version) - (hash-algorithm sha256)))))))) + (git->origin vcs-repo-url `(tag-or-commit . ,version) + #:ref->commit transform-version)) ((hg) `(origin (method hg-fetch) @@ -649,7 +625,8 @@ (define* (go-module->guix-package module-path #:key (name ,guix-name) (version ,(strip-v-prefix version*)) (source - ,(vcs->origin vcs-type vcs-repo-url version*)) + ,(vcs->origin vcs-type vcs-repo-url version* + #:transform-version transform-version)) (build-system go-build-system) (arguments (list ,@(if (version>? min-go-version (package-version (go-package))) diff --git a/guix/import/minetest.scm b/guix/import/minetest.scm index 5ea6e023ce..65ef242431 100644 --- a/guix/import/minetest.scm +++ b/guix/import/minetest.scm @@ -1,6 +1,7 @@ ;;; GNU Guix --- Functional package management for GNU ;;; Copyright © 2021, 2022 Maxime Devos ;;; Copyright © 2022 Hartmut Goebel +;;; Copyright © 2023 Nicolas Graves ;;; ;;; This file is part of GNU Guix. ;;; @@ -32,7 +33,6 @@ (define-module (guix import minetest) #:use-module (guix import utils) #:use-module (guix import json) #:use-module (json) - #:use-module (guix base32) #:use-module (guix git) #:use-module ((guix git-download) #:prefix download:) #:use-module (guix hash) @@ -277,12 +277,6 @@ (define url (string-append (%contentdb-api) "packages/?type=" type -;; XXX copied from (guix import elpa) -(define* (download-git-repository url ref) - "Fetch the given REF from the Git repository at URL." - (with-store store - (latest-repository-commit store url #:ref ref))) - (define (make-minetest-sexp author/name version repository commit inputs home-page synopsis description media-license license) @@ -293,24 +287,8 @@ (define (make-minetest-sexp author/name version repository commit (name ,(contentdb->package-name author/name)) (version ,version) (source - (origin - (method git-fetch) - (uri (git-reference - (url ,repository) - (commit ,commit))) - (sha256 - (base32 - ;; The git commit is not always available. - ,(and commit - (bytevector->nix-base32-string - (file-hash* - (download-git-repository repository - `(commit . ,commit)) - ;; 'download-git-repository' already filtered out the '.git' - ;; directory. - #:select? (const #true) - #:recursive? #true))))) - (file-name (git-file-name name version)))) + ,(git->origin + repository `(tag-or-commit . ,commit) #:ref->commit #t)) (build-system minetest-mod-build-system) ,@(maybe-propagated-inputs (map contentdb->package-name inputs)) (home-page ,home-page) diff --git a/guix/import/utils.scm b/guix/import/utils.scm index 0cf52cdbde..47254539a1 100644 --- a/guix/import/utils.scm +++ b/guix/import/utils.scm @@ -13,6 +13,7 @@ ;;; Copyright © 2022 Alice Brenon ;;; Copyright © 2022 Kyle Meyer ;;; Copyright © 2022 Philip McGrath +;;; Copyright © 2023 Nicolas Graves ;;; ;;; This file is part of GNU Guix. ;;; @@ -39,6 +40,8 @@ (define-module (guix import utils) #:use-module (guix packages) #:use-module (guix discovery) #:use-module (guix build-system) + #:use-module (guix git) + #:use-module (guix hash) #:use-module ((guix i18n) #:select (G_)) #:use-module (guix store) #:use-module (guix download) @@ -63,6 +66,7 @@ (define-module (guix import utils) url-fetch guix-hash-url + git->origin package-names->package-inputs maybe-inputs @@ -161,6 +165,41 @@ (define (guix-hash-url filename) "Return the hash of FILENAME in nix-base32 format." (bytevector->nix-base32-string (file-sha256 filename))) +(define* (git->origin repo-url ref #:key (ref->commit #f)) + "Returns a generated `origin' block of a package depending on the git source +control system, and the directory in the store where the package has been +downloaded, in case further processing is necessary. REPO-URL or REF can be +null. REF->COMMIT can be a function or #t, in which case the commit matching +ref is used. If REF->COMMIT is not used, the value inside REF is used." + (let* ((version (and (pair? ref) (cdr ref))) + (directory commit + (if version + (with-store store + (latest-repository-commit store repo-url + #:ref (if version ref '()))) + (values #f #f))) + (vcommit (match ref->commit + (#t commit) + (#f version) + ((? procedure?) (ref->commit version)) + (_ #f)))) + (values + `(origin + (method git-fetch) + (uri (git-reference + (url ,(and (not (eq? repo-url 'null)) repo-url)) + (commit ,vcommit))) + (file-name (git-file-name name version)) + (sha256 + (base32 + ,(and version ; Version or commit is not always available. + (bytevector->nix-base32-string + (file-hash* directory + ;; 'git-fetch' already filtered out '.git'. + #:select? (const #true) + #:recursive? #true)))))) + directory))) + (define %spdx-license-identifiers ;; https://spdx.org/licenses/ ;; The gfl1.0, nmap, repoze diff --git a/tests/go.scm b/tests/go.scm index d2e8846b30..4644e1bd1c 100644 --- a/tests/go.scm +++ b/tests/go.scm @@ -1,6 +1,7 @@ ;;; GNU Guix --- Functional package management for GNU ;;; Copyright © 2021 François Joulaud ;;; Copyright © 2021 Sarah Morgensen +;;; Copyright © 2023 Nicolas Graves ;;; ;;; This file is part of GNU Guix. ;;; @@ -24,7 +25,6 @@ (define-module (tests-import-go) #:use-module (guix base32) #:use-module (guix build-system go) #:use-module (guix import go) - #:use-module (guix base32) #:use-module ((guix utils) #:select (call-with-temporary-directory)) #:use-module (guix tests) #:use-module (ice-9 match) @@ -403,13 +403,26 @@ (define (mock-http-get testcase) (mock-http-get fixtures-go-check-test)) (mock ((guix http-client) http-fetch (mock-http-fetch fixtures-go-check-test)) - (mock ((guix git) update-cached-checkout - (lambda* (url #:key ref) - ;; Return an empty directory and its hash. - (values checkout - (nix-base32-string->bytevector - "0sjjj9z1dhilhpc8pq4154czrb79z9cm044jvn75kxcjv6v5l2m5") - #f))) + (mock ((guix import utils) git->origin + ;; Mock an empty directory by replacing hash. + (lambda* (repo-url ref #:key (ref->commit #f)) + (let* ((version (if (pair? ref) + (cdr ref) + #f)) + (vcommit (match ref->commit + (#t commit) + (#f version) + ((? procedure?) (ref->commit version)) + (_ #f)))) + `(origin + (method git-fetch) + (uri (git-reference + (url ,(and (not (eq? repo-url 'null)) repo-url)) + (commit ,vcommit))) + (file-name (git-file-name name version)) + (sha256 + (base32 + "0sjjj9z1dhilhpc8pq4154czrb79z9cm044jvn75kxcjv6v5l2m5")))))) (go-module->guix-package* "github.com/go-check/check"))))))) (test-end "go") diff --git a/tests/minetest.scm b/tests/minetest.scm index 78469bf95b..7ff72dbfdc 100644 --- a/tests/minetest.scm +++ b/tests/minetest.scm @@ -1,5 +1,6 @@ ;;; GNU Guix --- Functional package management for GNU ;;; Copyright © 2021 Maxime Devos +;;; Copyright © 2023 Nicolas Graves ;;; ;;; This file is part of GNU Guix. ;;; @@ -57,15 +58,7 @@ (define* (make-package-sexp #:key `(package (name ,guix-name) (version ,version) - (source - (origin - (method git-fetch) - (uri (git-reference - (url ,(and (not (eq? repo 'null)) repo)) - (commit #f))) - (sha256 - (base32 #f)) - (file-name (git-file-name name version)))) + (source ,(git->origin repo #f)) (build-system minetest-mod-build-system) ,@(maybe-propagated-inputs inputs) (home-page ,home-page) @@ -419,8 +412,8 @@ (define* (example-package #:key (uri (git-reference (url repo) (commit commit #;"808f9ffbd3106da4c92d2367b118b98196c9e81e"))) - (sha256 #f) ; not important for the following tests - (file-name (git-file-name name version))) + (file-name (git-file-name name version)) + (sha256 #f)) ; not important for the following tests source)) (build-system minetest-mod-build-system) (license #f) From patchwork Sat Feb 3 23:07:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Nicolas Graves X-Patchwork-Id: 22785 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id DDFEB27BBE9; Sat, 3 Feb 2024 23:09:33 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-3.8 required=5.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_MSPIKE_H5,RCVD_IN_MSPIKE_WL,SPF_HELO_PASS,URIBL_BLOCKED, URIBL_SBL_A autolearn=ham autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id D761827BBEB for ; Sat, 3 Feb 2024 23:09:28 +0000 (GMT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rWP83-0008Ce-2t; Sat, 03 Feb 2024 18:08:55 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rWP7z-0008Ao-EX for guix-patches@gnu.org; Sat, 03 Feb 2024 18:08:51 -0500 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rWP7z-0008E4-6D for guix-patches@gnu.org; Sat, 03 Feb 2024 18:08:51 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1rWP8A-0001HP-O6 for guix-patches@gnu.org; Sat, 03 Feb 2024 18:09:02 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#62202] [PATCH v4 2/6] import: Add juliahub importer. Resent-From: Nicolas Graves Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sat, 03 Feb 2024 23:09:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 62202 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch moreinfo To: 62202@debbugs.gnu.org Cc: ngraves@ngraves.fr, zimoun.toutoune@gmail.com Received: via spool by 62202-submit@debbugs.gnu.org id=B62202.17070017114838 (code B ref 62202); Sat, 03 Feb 2024 23:09:02 +0000 Received: (at 62202) by debbugs.gnu.org; 3 Feb 2024 23:08:31 +0000 Received: from localhost ([127.0.0.1]:47247 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rWP7e-0001Fp-1Y for submit@debbugs.gnu.org; Sat, 03 Feb 2024 18:08:31 -0500 Received: from 8.mo550.mail-out.ovh.net ([178.33.110.239]:55535) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rWP7Z-0001FF-Lz for 62202@debbugs.gnu.org; Sat, 03 Feb 2024 18:08:27 -0500 Received: from director4.ghost.mail-out.ovh.net (unknown [10.108.2.141]) by mo550.mail-out.ovh.net (Postfix) with ESMTP id 4A5D028469 for <62202@debbugs.gnu.org>; Sat, 3 Feb 2024 23:08:12 +0000 (UTC) Received: from ghost-submission-6684bf9d7b-phmvh (unknown [10.110.178.62]) by director4.ghost.mail-out.ovh.net (Postfix) with ESMTPS id EB0111FD68; Sat, 3 Feb 2024 23:08:11 +0000 (UTC) Received: from ngraves.fr ([37.59.142.106]) by ghost-submission-6684bf9d7b-phmvh with ESMTPSA id 7XcnC1vHvmWjAwAAGBOpXA (envelope-from ); Sat, 03 Feb 2024 23:08:11 +0000 Authentication-Results: garm.ovh; auth=pass (GARM-106R006f6be1dce-99ec-4fa1-a0ee-43630bf37bba, CDD90146079FDB87F723A7A7FD21F5F4569C6C59) smtp.auth=ngraves@ngraves.fr X-OVh-ClientIp: 81.67.146.208 Date: Sun, 4 Feb 2024 00:07:12 +0100 Message-ID: <20240203230807.25751-2-ngraves@ngraves.fr> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20240203230807.25751-1-ngraves@ngraves.fr> References: <20240203230807.25751-1-ngraves@ngraves.fr> MIME-Version: 1.0 X-Ovh-Tracer-Id: 14365356912783057511 X-VR-SPAMSTATE: OK X-VR-SPAMSCORE: 0 X-VR-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrgedvkedrfedujedgtdejucetufdoteggodetrfdotffvucfrrhhofhhilhgvmecuqfggjfdpvefjgfevmfevgfenuceurghilhhouhhtmecuhedttdenucenucfjughrpefhvfevufffkffojghfgggtgfesthekredtredtjeenucfhrhhomheppfhitgholhgrshcuifhrrghvvghsuceonhhgrhgrvhgvshesnhhgrhgrvhgvshdrfhhrqeenucggtffrrghtthgvrhhnpeevkeetkefffffgheekgeekhfeugedvheejteelheejudeffeffgfelleeghffhieenucffohhmrghinhepghhithhhuhgsrdgtohhmpdhjuhhlihgrhhhusgdrtghomhdpghhnuhdrohhrghdpghhithhhuhgsuhhsvghrtghonhhtvghnthdrtghomhdpjhhulhhirghhuhgsrdhorhhgnecukfhppeduvdejrddtrddtrddupdekuddrieejrddugeeirddvtdekpdefjedrheelrddugedvrddutdeinecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehinhgvthepuddvjedrtddrtddruddpmhgrihhlfhhrohhmpeeonhhgrhgrvhgvshesnhhgrhgrvhgvshdrfhhrqedpnhgspghrtghpthhtohepuddprhgtphhtthhopeeivddvtddvseguvggssghughhsrdhgnhhurdhorhhgpdfovfetjfhoshhtpehmohehhedtpdhmohguvgepshhmthhpohhuth X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-to: Nicolas Graves X-ACL-Warn: , Nicolas Graves via Guix-patches X-Patchwork-Original-From: Nicolas Graves via Guix-patches via From: Nicolas Graves Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org X-getmail-retrieved-from-mailbox: Patches --- doc/guix.texi | 27 +++ guix/import/juliahub.scm | 309 +++++++++++++++++++++++++++++++ guix/scripts/import.scm | 2 +- guix/scripts/import/juliahub.scm | 107 +++++++++++ 4 files changed, 444 insertions(+), 1 deletion(-) create mode 100644 guix/import/juliahub.scm create mode 100644 guix/scripts/import/juliahub.scm diff --git a/doc/guix.texi b/doc/guix.texi index c71d7e94cf..6baa726517 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -14782,6 +14782,33 @@ guix import hexpm cf@@0.3.0 Additional options include: +@table @code +@item --recursive +@itemx -r +Traverse the dependency graph of the given upstream package recursively +and generate package expressions for all those packages that are not yet +in Guix. +@end table + +@item juliahub +@cindex juliahub +Import metadata from both the General +@uref{https://github.com/JuliaRegistries/General} and Juliahub +@uref{https://juliahub.com} Julia package repositories, as in this +example: + +@example +guix import juliahub Cthulhu@@2.8.9 +@end example + +The supplied package name must have the same case as in the +aforementioned package repositories, and the version used must be an +exact version (e.g. @code{2.8.9} instead of @code{2.8}). The command +will also fail in the case of a Julia package that doesn't use a git +tag. + +Additional options include: + @table @code @item --recursive @itemx -r diff --git a/guix/import/juliahub.scm b/guix/import/juliahub.scm new file mode 100644 index 0000000000..ab838b6035 --- /dev/null +++ b/guix/import/juliahub.scm @@ -0,0 +1,309 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright © 2023 Nicolas Graves +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see . + +(define-module (guix import juliahub) + #:use-module (ice-9 textual-ports) + #:use-module (ice-9 regex) + #:use-module (ice-9 match) + #:use-module (srfi srfi-1) + #:use-module (srfi srfi-9) + #:use-module (srfi srfi-71) + #:use-module (guix http-client) + #:use-module (guix git) + #:use-module (guix import utils) + #:use-module (guix import json) + #:use-module (guix base32) + #:use-module (guix packages) + #:use-module (guix upstream) + #:use-module (json) + #:use-module ((guix licenses) #:prefix license:) + + #:export (juliahub->guix-package + %juliahub-updater + juliahub-recursive-import)) + + +;; JuliaHub API. +(define (juliahub-redirect-uri name) + (let* ((url (string-append "https://docs.juliahub.com/" name "/")) + (port (http-fetch url #:text? #t)) + (_ (get-line port)) + (meta (get-line port)) + (regex "url=[a-zA-Z0-9]{5}\\/[0-9\\.]*") + (redirect (match:substring (string-match regex meta)))) + (close-port port) + (string-drop redirect 4))) + +(define (juliahub-url name) + (let* ((url (string-append "https://docs.juliahub.com/" name "/")) + (uri (juliahub-redirect-uri name))) + (string-append url uri "/"))) + +;; General package repository. +(define %general-base-url + "https://raw.githubusercontent.com/JuliaRegistries/General/master/") + +(define (general-url package-name file) + (let ((folder (string-capitalize (string-take package-name 1)))) + (string-append + %general-base-url folder "/" package-name "/" file))) + +(define (ini-line->alist line) + (let* ((l (string-split line #\=)) + (attribute (string->symbol (string-drop-right (car l) 1))) + (value (string-drop (string-drop-right (cadr l) 1) 2))) + `(,attribute . ,value))) + +(define (ini-fetch url) + (let* ((port (http-fetch url #:text? #t)) + (raw (get-string-all port)) + (lines (drop-right (string-split raw #\newline) 1))) + (close-port port) + (map ini-line->alist lines))) + +;; Filtering out julia-stdlibs. +;; To update them, see file sysimg.jl. +(define %julia-stdlibs + (list "julia" + "ArgTools" + "Artifacts" + "Base64" + "CRC32c" + "FileWatching" + "Libdl" + "Logging" + "Mmap" + "NetworkOptions" + "SHA" + "Serialization" + "Sockets" + "Unicode" + "DelimitedFiles" + "LinearAlgebra" + "Markdown" + "Printf" + "Random" + "Tar" + "Dates" + "Distributed" + "Future" + "InteractiveUtils" + "LibGit2" + "Profile" + "SparseArrays" + "UUIDs" + "REPL" + "SharedArrays" + "Statistics" + "SuiteSparse" + "TOML" + "Test" + "LibCURL" + "Downloads" + "Pkg" + "LazyArtifacts")) + +;; Julia package. +(define-json-mapping make-juliahub-package juliahub-package? + json->juliahub-package + (homepage juliahub-package-homepage) ;string + (readme juliahub-package-readme) ;string + (version juliahub-package-version) ;string + (description juliahub-package-description) ;string + (dependencies + juliahub-package-dependencies "deps" + json->juliahub-dependencies) ;list of + (url juliahub-package-url) ;string + (uuid juliahub-package-uuid) ;string + (license juliahub-package-license)) ;string + +(define-json-mapping + make-juliahub-dependency juliahub-dependency? + json->juliahub-dependency + (direct? juliahub-dependency-direct? "direct") ;boolean + (name juliahub-dependency-name) ;string + (uuid juliahub-dependency-uuid) ;string + (versions juliahub-dependency-versions "versions" vector->list)) ;list of strings + +(define (julia-name->guix-name name) + (string-append "julia-" (snake-case name))) + +(define* (juliahub-fetch name #:key (version #f)) + "Return a record for package NAME, or #f on failure." + (let* ((uri (juliahub-redirect-uri name)) + (slug (string-take uri 5)) + (url (if version + (string-append "https://docs.juliahub.com/" name "/" + slug "/" version "/pkg.json") + (string-append (juliahub-url name) "pkg.json")))) + (and=> (json-fetch url) json->juliahub-package))) + +(define (make-julia-sexp name version source home-page synopsis description + direct-dependencies test-dependencies-names license) + "Return the `package' s-expression for a Julia package with the given NAME, +VERSION, SOURCE, HOME-PAGE, DESCRIPTION, DIRECT-DEPENDENCIES, +TEST-DEPENDENCIES-NAMES and LICENSE." + `(package + (name ,(julia-name->guix-name name)) + (version ,version) + (source ,source) + (build-system julia-build-system) + ,@(if (null? direct-dependencies) + '() + `((propagated-inputs + (list ,@(map (compose string->symbol + julia-name->guix-name + juliahub-dependency-name) + direct-dependencies))))) + ,@(if (null? test-dependencies-names) + '() + `((native-inputs + (list ,@(map (compose string->symbol julia-name->guix-name) + test-dependencies-names))))) + (synopsis ,synopsis) + (description ,description) + (home-page ,home-page) + (license ,(if license (spdx-string->license license) #f)))) + +;; Dependencies helpers. +(define (json->juliahub-dependencies vector) + (if (vector? vector) + (filter-map + (lambda (el) + (let ((dep (json->juliahub-dependency el))) + (if (and (juliahub-dependency-direct? dep) + (not (member (juliahub-dependency-name dep) + %julia-stdlibs))) + dep + #f))) + (vector->list vector)))) + +(define (parse-test-dependencies directory) + (let* ((port (open-input-file (string-append directory "/Project.toml"))) + (project.toml (get-string-all port)) + (regex "\ntest = \\[.*\\]") + (deps (match:substring (string-match regex project.toml))) + (pure (string-delete (list->char-set (list #\" #\ )) deps))) + (close-port port) + (filter (lambda (x) (not (member x %julia-stdlibs))) + (string-split (string-drop (string-drop-right pure 1) 7) #\,)))) + +;; Juliahub may be more up-to-date than the General registry or the actual git +;; tag (it seems around 6 hours pass between the time a commit is supplied to +;; JuliaRegistrator as a release, and the time Julia TagBot Github Action makes +;; the git tag). We have no simple way to get the commit of the latest-version. +;; Thus the best simple thing we can do is get the latest-git-tag, and import +;; this version instead. We do this by parsing Package.toml in the General +;; registry, and then getting the refs of the git repo supplied by this +;; file. Parsing this file is also necessary if the package is in a subdir of a +;; git repository, because the information isn't present in Juliahub. + +;; There's a last case where some Julia packages are not based on a particular +;; git tag. In this case, the script fails, but it seems quite rare. We could +;; introduce the tree-commit which is available in the Versions.toml file in the +;; General repository. This can be used to identify the state of a repository, +;; since we have a unique hash of the listing of files and directories. + +(define (latest-git-tag repo) + (let* ((last-ref (last (remote-refs repo #:tags? #t))) + (last-git-tag (last (string-split last-ref #\/)))) + (string-drop last-git-tag 1))) + +(define* (juliahub->guix-package package-name + #:key version #:allow-other-keys) + "Fetch the metadata for PACKAGE-NAME from juliahub.org, and return the +`package' s-expression corresponding to that package, or #f on failure. +Optionally include a VERSION string to fetch a specific version juliahub." + (let* ((package-toml (ini-fetch (general-url package-name "Package.toml"))) + (subdir (assoc-ref package-toml 'subdir)) + (tag (latest-git-tag (assoc-ref package-toml 'repo))) + (package (if version + (juliahub-fetch package-name #:version version) + (if tag + (juliahub-fetch package-name #:version tag) + (juliahub-fetch package-name))))) + (if package + (let* ((source directory + (git->origin + (juliahub-package-url package) + `(tag-or-commit + . ,(string-append + "v" (juliahub-package-version package))))) + (direct-dependencies + (filter juliahub-dependency-direct? + (juliahub-package-dependencies package))) + (dependencies-names (map juliahub-dependency-name + direct-dependencies)) + (test-dependencies-names + (if subdir + (parse-test-dependencies + (string-append subdir "/" directory)) + (parse-test-dependencies directory))) + (homepage (juliahub-package-homepage package))) + (values (make-julia-sexp + package-name + (juliahub-package-version package) + source + (match homepage + ("" (juliahub-package-url package)) + ((? string?) homepage) + (_ (juliahub-package-url package))) + (juliahub-package-description package) + (beautify-description + (juliahub-package-readme package)) + direct-dependencies + test-dependencies-names + (juliahub-package-license package)) + (append dependencies-names test-dependencies-names))) + (values #f '())))) + +;; We must use the url to get a name with the true case of juliahub/general. +(define (guix-package->juliahub-name package) + (let* ((url (juliahub-package-url package)) + (git-name (last (string-split url #\/))) + (ungitted-name (if (string-suffix? ".git" git-name) + (string-drop-right git-name 4) + git-name)) + (package-name (if (string-suffix? ".jl" ungitted-name) + (string-drop-right ungitted-name 4) + ungitted-name))) + package-name)) + +(define* (import-release package #:key (version #f)) + "Return an for the latest release of PACKAGE." + (let* ((package-name (guix-package->juliahub-name package)) + (package (juliahub-fetch package-name)) + (version (or version (juliahub-package-version package)))) + (upstream-source + (package (package-name package)) + (version version) + (urls (list (juliahub-package-url package)))))) + +(define %juliahub-updater + (upstream-updater + (name 'juliahub) + (description "Updater for Juliahub packages") + (pred juliahub-package?) + (import import-release))) + +(define* (juliahub-recursive-import package-name #:optional version) + (recursive-import package-name + #:repo '() + #:repo->guix-package juliahub->guix-package + #:guix-name julia-name->guix-name + #:version version)) diff --git a/guix/scripts/import.scm b/guix/scripts/import.scm index d2a1cee56e..8926c9610f 100644 --- a/guix/scripts/import.scm +++ b/guix/scripts/import.scm @@ -47,7 +47,7 @@ (define %standard-import-options '()) (define importers '("gnu" "pypi" "cpan" "hackage" "stackage" "egg" "elpa" "gem" "go" "cran" "crate" "texlive" "json" "opam" - "minetest" "elm" "hexpm" "composer")) + "minetest" "elm" "hexpm" "composer" "juliahub")) (define (resolve-importer name) (let ((module (resolve-interface diff --git a/guix/scripts/import/juliahub.scm b/guix/scripts/import/juliahub.scm new file mode 100644 index 0000000000..1317c67aa3 --- /dev/null +++ b/guix/scripts/import/juliahub.scm @@ -0,0 +1,107 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright © 2023 Nicolas Graves +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see . + +(define-module (guix scripts import juliahub) + #:use-module (guix ui) + #:use-module (guix utils) + #:use-module (guix scripts) + #:use-module (guix import juliahub) + #:use-module (guix scripts import) + #:use-module (srfi srfi-1) + #:use-module (srfi srfi-11) + #:use-module (srfi srfi-37) + #:use-module (ice-9 match) + #:use-module (ice-9 format) + #:use-module (ice-9 receive) + #:export (guix-import-juliahub)) + + +;;; +;;; Command-line options. +;;; + +(define %default-options + '()) + +(define (show-help) + (display (G_ "Usage: guix import juliahub PACKAGE-NAME[@VERSION] Import and +convert the Julia package for PACKAGE-NAME. Optionally, a version can be +specified after the at-sign (@) character.\n")) + (display (G_ " + -h, --help display this help and exit")) + (display (G_ " + -V, --version display version information and exit")) + (display (G_ " + -r, --recursive generate package expressions for all Gem packages\ + that are not yet in Guix")) + (newline) + (show-bug-report-information)) + +(define %options + ;; Specification of the command-line options. + (cons* (option '(#\h "help") #f #f + (lambda args + (show-help) + (exit 0))) + (option '(#\V "version") #f #f + (lambda args + (show-version-and-exit "guix import gem"))) + (option '(#\r "recursive") #f #f + (lambda (opt name arg result) + (alist-cons 'recursive #t result))) + %standard-import-options)) + + +;;; +;;; Entry point. +;;; + +(define (guix-import-juliahub . args) + (define (parse-options) + ;; Return the alist of option values. + (parse-command-line args %options (list %default-options) + #:build-options? #f)) + + (let* ((opts (parse-options)) + (args (filter-map (match-lambda + (('argument . value) + value) + (_ #f)) + (reverse opts)))) + (match args + ((spec) + (receive (package-name package-version) + (package-name->name+version spec) + (let ((code (if (assoc-ref opts 'recursive) + (map (match-lambda + ((and ('package ('name name) . rest) pkg) + `(define-public ,(string->symbol name) + ,pkg)) + (_ #f)) + (juliahub-recursive-import package-name package-version)) + (let ((sexp (juliahub->guix-package package-name #:version package-version))) + (if sexp sexp #f))))) + (match code + ((or #f '(#f)) + (leave (G_ "failed to download meta-data for package '~a'~%") + package-name)) + (_ code))))) + (() + (leave (G_ "too few arguments~%"))) + ((many ...) + (leave (G_ "too many arguments~%")))))) From patchwork Sat Feb 3 23:07:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Graves X-Patchwork-Id: 22782 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id A678C27BBED; Sat, 3 Feb 2024 23:09:21 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-3.9 required=5.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_MSPIKE_H5,RCVD_IN_MSPIKE_WL,SPF_HELO_PASS autolearn=ham autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 4932B27BBEC for ; Sat, 3 Feb 2024 23:09:20 +0000 (GMT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rWP81-0008BF-0g; Sat, 03 Feb 2024 18:08:53 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rWP7z-0008Aw-Rs for guix-patches@gnu.org; Sat, 03 Feb 2024 18:08:51 -0500 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rWP7z-0008ED-K0 for guix-patches@gnu.org; Sat, 03 Feb 2024 18:08:51 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1rWP8B-0001HW-4z for guix-patches@gnu.org; Sat, 03 Feb 2024 18:09:03 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#62202] [PATCH v4 3/6] import: juliahub: Beautify description. Resent-From: Nicolas Graves Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sat, 03 Feb 2024 23:09:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 62202 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch moreinfo To: 62202@debbugs.gnu.org Cc: ngraves@ngraves.fr, zimoun.toutoune@gmail.com Received: via spool by 62202-submit@debbugs.gnu.org id=B62202.17070017114846 (code B ref 62202); Sat, 03 Feb 2024 23:09:03 +0000 Received: (at 62202) by debbugs.gnu.org; 3 Feb 2024 23:08:31 +0000 Received: from localhost ([127.0.0.1]:47250 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rWP7f-0001G4-Gm for submit@debbugs.gnu.org; Sat, 03 Feb 2024 18:08:31 -0500 Received: from 9.mo584.mail-out.ovh.net ([46.105.40.176]:53255) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rWP7b-0001FI-5O for 62202@debbugs.gnu.org; Sat, 03 Feb 2024 18:08:28 -0500 Received: from director10.ghost.mail-out.ovh.net (unknown [10.109.176.72]) by mo584.mail-out.ovh.net (Postfix) with ESMTP id BBD8E25BDC for <62202@debbugs.gnu.org>; Sat, 3 Feb 2024 23:08:13 +0000 (UTC) Received: from ghost-submission-6684bf9d7b-xfz5d (unknown [10.108.42.33]) by director10.ghost.mail-out.ovh.net (Postfix) with ESMTPS id 565DD1FDEC; Sat, 3 Feb 2024 23:08:13 +0000 (UTC) Received: from ngraves.fr ([37.59.142.103]) by ghost-submission-6684bf9d7b-xfz5d with ESMTPSA id 12T/C13HvmUznggA0jpIEA (envelope-from ); Sat, 03 Feb 2024 23:08:13 +0000 Authentication-Results: garm.ovh; auth=pass (GARM-103G00512037912-6b0e-4dfc-af55-9d499f043c10, CDD90146079FDB87F723A7A7FD21F5F4569C6C59) smtp.auth=ngraves@ngraves.fr X-OVh-ClientIp: 81.67.146.208 Date: Sun, 4 Feb 2024 00:07:13 +0100 Message-ID: <20240203230807.25751-3-ngraves@ngraves.fr> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20240203230807.25751-1-ngraves@ngraves.fr> References: <20240203230807.25751-1-ngraves@ngraves.fr> MIME-Version: 1.0 X-Ovh-Tracer-Id: 14365638387764814439 X-VR-SPAMSTATE: OK X-VR-SPAMSCORE: 0 X-VR-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrgedvkedrfedujedgtdejucetufdoteggodetrfdotffvucfrrhhofhhilhgvmecuqfggjfdpvefjgfevmfevgfenuceurghilhhouhhtmecuhedttdenucenucfjughrpefhvfevufffkffojghfggfgsedtkeertdertddtnecuhfhrohhmpefpihgtohhlrghsucfirhgrvhgvshcuoehnghhrrghvvghssehnghhrrghvvghsrdhfrheqnecuggftrfgrthhtvghrnhepleffjeetueethfefkeffffefvddukeejkefgleduiedthfekvefhiedvhfffgeegnecukfhppeduvdejrddtrddtrddupdekuddrieejrddugeeirddvtdekpdefjedrheelrddugedvrddutdefnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehinhgvthepuddvjedrtddrtddruddpmhgrihhlfhhrohhmpeeonhhgrhgrvhgvshesnhhgrhgrvhgvshdrfhhrqedpnhgspghrtghpthhtohepuddprhgtphhtthhopeeivddvtddvseguvggssghughhsrdhgnhhurdhorhhgpdfovfetjfhoshhtpehmohehkeegpdhmohguvgepshhmthhpohhuth X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-to: Nicolas Graves X-ACL-Warn: , Nicolas Graves via Guix-patches X-Patchwork-Original-From: Nicolas Graves via Guix-patches via From: Nicolas Graves Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org X-getmail-retrieved-from-mailbox: Patches --- guix/import/juliahub.scm | 53 ++++++++++++++++++++++++++++++++++++++-- 1 file changed, 51 insertions(+), 2 deletions(-) diff --git a/guix/import/juliahub.scm b/guix/import/juliahub.scm index ab838b6035..e4540de06d 100644 --- a/guix/import/juliahub.scm +++ b/guix/import/juliahub.scm @@ -30,8 +30,10 @@ (define-module (guix import juliahub) #:use-module (guix base32) #:use-module (guix packages) #:use-module (guix upstream) - #:use-module (json) #:use-module ((guix licenses) #:prefix license:) + #:use-module (json) + #:use-module (htmlprag) + #:use-module (sxml transform) #:export (juliahub->guix-package %juliahub-updater @@ -76,6 +78,53 @@ (define (ini-fetch url) (close-port port) (map ini-line->alist lines))) +;; Beautify description. +(define %juliahub-beautify-description-rules + `((h1 *preorder* . ,(lambda args #f)) + (h2 *preorder* . ,(lambda args #f)) + (h3 *preorder* . ,(lambda args #f)) + (h4 *preorder* . ,(lambda args #f)) + (h5 *preorder* . ,(lambda args #f)) + (h6 *preorder* . ,(lambda args #f)) + (hr *preorder* . ,(lambda args #f)) + (span *preorder* . ,(lambda args #f)) + (img *preorder* . ,(lambda args #f)) + (pre *preorder* . ,(lambda args #f)) + (div *preorder* . ,(lambda args #f)) + (table *preorder* . ,(lambda args #f)) + (imgalt *preorder* . ,(lambda args #f)) + (@ *preorder* . ,(lambda args #f)) + (*TOP* . ,(lambda args (cdr args))) + (p . ,(lambda args (cdr args))) + (em . ,(lambda args (cdr args))) + (strong . ,(lambda args (cdr args))) + (a . ,(lambda args + (match args + ((tag link ref) + (if ref ref #f)) + (_ #f)))) + (ul . ,(lambda args + `("@itemize" ,@(cdr args) "\n@end itemize"))) + (ol . ,(lambda args + `("@enumerate" ,@(cdr args) "@end enumerate"))) + (blockquote . ,(lambda args + `("@quotation" ,@(cdr args) "@end quotation"))) + (li . ,(lambda args + `("\n@item" ,@(cdr args)))) + (code . ,(lambda args + `("@code{" ,@(cdr args) "}"))) + (*text* . ,(lambda (tag x) x)) + (*default* . ,(lambda (tag . body) + (cons tag body))))) + +(define (juliahub-beautify-description description) + (string-join + (filter (lambda (x) (if (equal? x " ") #f x)) + (flatten + (pre-post-order (html->sxml description) + %juliahub-beautify-description-rules))) + " ")) + ;; Filtering out julia-stdlibs. ;; To update them, see file sysimg.jl. (define %julia-stdlibs @@ -264,7 +313,7 @@ (define* (juliahub->guix-package package-name ((? string?) homepage) (_ (juliahub-package-url package))) (juliahub-package-description package) - (beautify-description + ((compose beautify-description juliahub-beautify-description) (juliahub-package-readme package)) direct-dependencies test-dependencies-names From patchwork Sat Feb 3 23:07:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Graves X-Patchwork-Id: 22781 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id D6A2427BBED; Sat, 3 Feb 2024 23:09:19 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-3.9 required=5.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_MSPIKE_H5,RCVD_IN_MSPIKE_WL,SPF_HELO_PASS autolearn=ham autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 0DA7F27BBEB for ; Sat, 3 Feb 2024 23:09:18 +0000 (GMT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rWP82-0008C5-83; Sat, 03 Feb 2024 18:08:54 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rWP80-0008B5-9f for guix-patches@gnu.org; Sat, 03 Feb 2024 18:08:52 -0500 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rWP80-0008EJ-1h for guix-patches@gnu.org; Sat, 03 Feb 2024 18:08:52 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1rWP8B-0001He-JB for guix-patches@gnu.org; Sat, 03 Feb 2024 18:09:03 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#62202] [PATCH v4 4/6] import: utils: Rule out texinfo common syntax from @ escape. Resent-From: Nicolas Graves Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sat, 03 Feb 2024 23:09:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 62202 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch moreinfo To: 62202@debbugs.gnu.org Cc: ngraves@ngraves.fr, zimoun.toutoune@gmail.com Received: via spool by 62202-submit@debbugs.gnu.org id=B62202.17070017124855 (code B ref 62202); Sat, 03 Feb 2024 23:09:03 +0000 Received: (at 62202) by debbugs.gnu.org; 3 Feb 2024 23:08:32 +0000 Received: from localhost ([127.0.0.1]:47252 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rWP7f-0001G7-Tc for submit@debbugs.gnu.org; Sat, 03 Feb 2024 18:08:32 -0500 Received: from 11.mo584.mail-out.ovh.net ([46.105.34.195]:39809) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rWP7b-0001FK-SZ for 62202@debbugs.gnu.org; Sat, 03 Feb 2024 18:08:28 -0500 Received: from director10.ghost.mail-out.ovh.net (unknown [10.108.17.34]) by mo584.mail-out.ovh.net (Postfix) with ESMTP id 32FA725BCC for <62202@debbugs.gnu.org>; Sat, 3 Feb 2024 23:08:15 +0000 (UTC) Received: from ghost-submission-6684bf9d7b-f7jcc (unknown [10.110.168.153]) by director10.ghost.mail-out.ovh.net (Postfix) with ESMTPS id D9B741FDEC; Sat, 3 Feb 2024 23:08:14 +0000 (UTC) Received: from ngraves.fr ([37.59.142.99]) by ghost-submission-6684bf9d7b-f7jcc with ESMTPSA id 3jLSK17HvmXKigYAqzIo+Q (envelope-from ); Sat, 03 Feb 2024 23:08:14 +0000 Authentication-Results: garm.ovh; auth=pass (GARM-99G003feae5ee9-f3cd-49c3-ab89-dd51beb4a2ee, CDD90146079FDB87F723A7A7FD21F5F4569C6C59) smtp.auth=ngraves@ngraves.fr X-OVh-ClientIp: 81.67.146.208 Date: Sun, 4 Feb 2024 00:07:14 +0100 Message-ID: <20240203230807.25751-4-ngraves@ngraves.fr> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20240203230807.25751-1-ngraves@ngraves.fr> References: <20240203230807.25751-1-ngraves@ngraves.fr> MIME-Version: 1.0 X-Ovh-Tracer-Id: 14366201339026268775 X-VR-SPAMSTATE: OK X-VR-SPAMSCORE: 0 X-VR-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrgedvkedrfedujedgtdejucetufdoteggodetrfdotffvucfrrhhofhhilhgvmecuqfggjfdpvefjgfevmfevgfenuceurghilhhouhhtmecuhedttdenucenucfjughrpefhvfevufffkffojghfggfgsedtkeertdertddtnecuhfhrohhmpefpihgtohhlrghsucfirhgrvhgvshcuoehnghhrrghvvghssehnghhrrghvvghsrdhfrheqnecuggftrfgrthhtvghrnhepleffjeetueethfefkeffffefvddukeejkefgleduiedthfekvefhiedvhfffgeegnecukfhppeduvdejrddtrddtrddupdekuddrieejrddugeeirddvtdekpdefjedrheelrddugedvrdelleenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepihhnvghtpeduvdejrddtrddtrddupdhmrghilhhfrhhomhepoehnghhrrghvvghssehnghhrrghvvghsrdhfrheqpdhnsggprhgtphhtthhopedupdhrtghpthhtohepiedvvddtvdesuggvsggsuhhgshdrghhnuhdrohhrghdpoffvtefjohhsthepmhhoheekgedpmhhouggvpehsmhhtphhouhht X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-to: Nicolas Graves X-ACL-Warn: , Nicolas Graves via Guix-patches X-Patchwork-Original-From: Nicolas Graves via Guix-patches via From: Nicolas Graves Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org X-getmail-retrieved-from-mailbox: Patches --- guix/import/utils.scm | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/guix/import/utils.scm b/guix/import/utils.scm index 47254539a1..57e4ec0ce7 100644 --- a/guix/import/utils.scm +++ b/guix/import/utils.scm @@ -375,7 +375,13 @@ (define* (beautify-description description #:optional (length 80)) (cut string-trim-both <> #\') ;; Escape single @ to prevent it from being understood as ;; invalid Texinfo syntax. - (cut regexp-substitute/global #f "@" <> 'pre "@@" 'post) + (lambda (word) + (if ; Rule out some valid Texinfo syntax. + (member word '("@itemize" "@item" "@end" "@quotation" + "@enumerate" "@code" "@code{")) + word + ((cut regexp-substitute/global + #f "@" <> 'pre "@@" 'post) word))) ;; Wrap camelCase or PascalCase words in @code{...}. (lambda (word) (let ((pattern (make-regexp "([A-Z][a-z]+[A-Z]|[a-z]+[A-Z])"))) From patchwork Sat Feb 3 23:07:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Graves X-Patchwork-Id: 22780 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id A8FA027BBE9; Sat, 3 Feb 2024 23:09:18 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-3.9 required=5.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_MSPIKE_H5,RCVD_IN_MSPIKE_WL,SPF_HELO_PASS autolearn=unavailable autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 96A6227BBEA for ; Sat, 3 Feb 2024 23:09:17 +0000 (GMT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rWP81-0008Be-Kc; Sat, 03 Feb 2024 18:08:53 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rWP80-0008BD-NP for guix-patches@gnu.org; Sat, 03 Feb 2024 18:08:52 -0500 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rWP80-0008EP-Fh for guix-patches@gnu.org; Sat, 03 Feb 2024 18:08:52 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1rWP8C-0001Hl-1C for guix-patches@gnu.org; Sat, 03 Feb 2024 18:09:04 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#62202] [PATCH v4 5/6] tests: go: Add mock-git->origin function. Resent-From: Nicolas Graves Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sat, 03 Feb 2024 23:09:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 62202 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch moreinfo To: 62202@debbugs.gnu.org Cc: ngraves@ngraves.fr, zimoun.toutoune@gmail.com Received: via spool by 62202-submit@debbugs.gnu.org id=B62202.17070017134867 (code B ref 62202); Sat, 03 Feb 2024 23:09:03 +0000 Received: (at 62202) by debbugs.gnu.org; 3 Feb 2024 23:08:33 +0000 Received: from localhost ([127.0.0.1]:47254 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rWP7h-0001GQ-7W for submit@debbugs.gnu.org; Sat, 03 Feb 2024 18:08:33 -0500 Received: from 15.mo584.mail-out.ovh.net ([91.121.62.11]:44939) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rWP7d-0001Fd-2K for 62202@debbugs.gnu.org; Sat, 03 Feb 2024 18:08:30 -0500 Received: from director10.ghost.mail-out.ovh.net (unknown [10.108.17.34]) by mo584.mail-out.ovh.net (Postfix) with ESMTP id 3B5F425CB8 for <62202@debbugs.gnu.org>; Sat, 3 Feb 2024 23:08:16 +0000 (UTC) Received: from ghost-submission-6684bf9d7b-l9gd2 (unknown [10.110.96.132]) by director10.ghost.mail-out.ovh.net (Postfix) with ESMTPS id E4AA11FDEC; Sat, 3 Feb 2024 23:08:15 +0000 (UTC) Received: from ngraves.fr ([37.59.142.109]) by ghost-submission-6684bf9d7b-l9gd2 with ESMTPSA id 7uFfNV/HvmUrtQUAtmFz7A (envelope-from ); Sat, 03 Feb 2024 23:08:15 +0000 Authentication-Results: garm.ovh; auth=pass (GARM-109S003b116b371-a2ff-41e2-a39e-32a64ccadfe2, CDD90146079FDB87F723A7A7FD21F5F4569C6C59) smtp.auth=ngraves@ngraves.fr X-OVh-ClientIp: 81.67.146.208 Date: Sun, 4 Feb 2024 00:07:15 +0100 Message-ID: <20240203230807.25751-5-ngraves@ngraves.fr> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20240203230807.25751-1-ngraves@ngraves.fr> References: <20240203230807.25751-1-ngraves@ngraves.fr> MIME-Version: 1.0 X-Ovh-Tracer-Id: 14366482811726389863 X-VR-SPAMSTATE: OK X-VR-SPAMSCORE: 0 X-VR-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrgedvkedrfedujedgtdejucetufdoteggodetrfdotffvucfrrhhofhhilhgvmecuqfggjfdpvefjgfevmfevgfenuceurghilhhouhhtmecuhedttdenucenucfjughrpefhvfevufffkffojghfggfgsedtkeertdertddtnecuhfhrohhmpefpihgtohhlrghsucfirhgrvhgvshcuoehnghhrrghvvghssehnghhrrghvvghsrdhfrheqnecuggftrfgrthhtvghrnhepvdehleeiffehtedvlefhffffjeefgfduhfetkeevheeiteduiedugfekuedtheejnecuffhomhgrihhnpehgihhthhhusgdrtghomhenucfkphepuddvjedrtddrtddruddpkedurdeijedrudegiedrvddtkedpfeejrdehledrudegvddruddtleenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepihhnvghtpeduvdejrddtrddtrddupdhmrghilhhfrhhomhepoehnghhrrghvvghssehnghhrrghvvghsrdhfrheqpdhnsggprhgtphhtthhopedupdhrtghpthhtohepiedvvddtvdesuggvsggsuhhgshdrghhnuhdrohhrghdpoffvtefjohhsthepmhhoheekgedpmhhouggvpehsmhhtphhouhht X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-to: Nicolas Graves X-ACL-Warn: , Nicolas Graves via Guix-patches X-Patchwork-Original-From: Nicolas Graves via Guix-patches via From: Nicolas Graves Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org X-getmail-retrieved-from-mailbox: Patches * tests/go.scm (mock-git->origin): Add function. (go-module->guix-package test): Use mock-git->origin. Change-Id: I9f9de814fc6c9dcf9772507c9e5bf32c16d81784 --- tests/go.scm | 41 +++++++++++++++++++++-------------------- 1 file changed, 21 insertions(+), 20 deletions(-) diff --git a/tests/go.scm b/tests/go.scm index 4644e1bd1c..cca27506e6 100644 --- a/tests/go.scm +++ b/tests/go.scm @@ -371,6 +371,26 @@ (define (mock-http-get testcase) (values response-header body) (error "mocked http-get Unexpected URL: " url))))) +;; Mock an empty directory by replacing hash. +(define* (mock-git->origin repo-url ref #:key (ref->commit #f)) + (let* ((version (if (pair? ref) + (cdr ref) + #f)) + (vcommit (match ref->commit + (#t commit) + (#f version) + ((? procedure?) (ref->commit version)) + (_ #f)))) + `(origin + (method git-fetch) + (uri (git-reference + (url ,(and (not (eq? repo-url 'null)) repo-url)) + (commit ,vcommit))) + (file-name (git-file-name name version)) + (sha256 + (base32 + "0sjjj9z1dhilhpc8pq4154czrb79z9cm044jvn75kxcjv6v5l2m5"))))) + (test-equal "go-module->guix-package" '(package (name "go-github-com-go-check-check") @@ -403,26 +423,7 @@ (define (mock-http-get testcase) (mock-http-get fixtures-go-check-test)) (mock ((guix http-client) http-fetch (mock-http-fetch fixtures-go-check-test)) - (mock ((guix import utils) git->origin - ;; Mock an empty directory by replacing hash. - (lambda* (repo-url ref #:key (ref->commit #f)) - (let* ((version (if (pair? ref) - (cdr ref) - #f)) - (vcommit (match ref->commit - (#t commit) - (#f version) - ((? procedure?) (ref->commit version)) - (_ #f)))) - `(origin - (method git-fetch) - (uri (git-reference - (url ,(and (not (eq? repo-url 'null)) repo-url)) - (commit ,vcommit))) - (file-name (git-file-name name version)) - (sha256 - (base32 - "0sjjj9z1dhilhpc8pq4154czrb79z9cm044jvn75kxcjv6v5l2m5")))))) + (mock ((guix import utils) git->origin mock-git->origin) (go-module->guix-package* "github.com/go-check/check"))))))) (test-end "go") From patchwork Sat Feb 3 23:07:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Nicolas Graves X-Patchwork-Id: 22783 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id DE35627BBEA; Sat, 3 Feb 2024 23:09:22 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-3.8 required=5.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_MSPIKE_H5,RCVD_IN_MSPIKE_WL,SPF_HELO_PASS,URIBL_BLOCKED, URIBL_SBL_A autolearn=unavailable autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 0FFBA27BBE9 for ; Sat, 3 Feb 2024 23:09:19 +0000 (GMT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rWP83-0008Cj-RA; Sat, 03 Feb 2024 18:08:55 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rWP81-0008BS-FZ for guix-patches@gnu.org; Sat, 03 Feb 2024 18:08:53 -0500 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rWP81-0008EV-7N for guix-patches@gnu.org; Sat, 03 Feb 2024 18:08:53 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1rWP8C-0001Hs-FA for guix-patches@gnu.org; Sat, 03 Feb 2024 18:09:04 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#62202] [PATCH v4 6/6] tests: juliahub: Add unit tests for (guix import juliahub). Resent-From: Nicolas Graves Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sat, 03 Feb 2024 23:09:04 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 62202 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch moreinfo To: 62202@debbugs.gnu.org Cc: ngraves@ngraves.fr, zimoun.toutoune@gmail.com Received: via spool by 62202-submit@debbugs.gnu.org id=B62202.17070017144873 (code B ref 62202); Sat, 03 Feb 2024 23:09:04 +0000 Received: (at 62202) by debbugs.gnu.org; 3 Feb 2024 23:08:34 +0000 Received: from localhost ([127.0.0.1]:47256 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rWP7h-0001GS-Jh for submit@debbugs.gnu.org; Sat, 03 Feb 2024 18:08:34 -0500 Received: from 12.mo550.mail-out.ovh.net ([87.98.162.229]:34175) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rWP7e-0001Fk-Fy for 62202@debbugs.gnu.org; Sat, 03 Feb 2024 18:08:32 -0500 Received: from director2.ghost.mail-out.ovh.net (unknown [10.109.148.79]) by mo550.mail-out.ovh.net (Postfix) with ESMTP id 2257E283D2 for <62202@debbugs.gnu.org>; Sat, 3 Feb 2024 23:08:17 +0000 (UTC) Received: from ghost-submission-6684bf9d7b-wmfjk (unknown [10.110.96.132]) by director2.ghost.mail-out.ovh.net (Postfix) with ESMTPS id D4AA11FD2B; Sat, 3 Feb 2024 23:08:16 +0000 (UTC) Received: from ngraves.fr ([37.59.142.99]) by ghost-submission-6684bf9d7b-wmfjk with ESMTPSA id RY8lMGDHvmVLPhsAW0a7IA (envelope-from ); Sat, 03 Feb 2024 23:08:16 +0000 Authentication-Results: garm.ovh; auth=pass (GARM-99G0037b55a605-8e13-411a-b187-b40de83c08fb, CDD90146079FDB87F723A7A7FD21F5F4569C6C59) smtp.auth=ngraves@ngraves.fr X-OVh-ClientIp: 81.67.146.208 Date: Sun, 4 Feb 2024 00:07:16 +0100 Message-ID: <20240203230807.25751-6-ngraves@ngraves.fr> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20240203230807.25751-1-ngraves@ngraves.fr> References: <20240203230807.25751-1-ngraves@ngraves.fr> MIME-Version: 1.0 X-Ovh-Tracer-Id: 14366764287976268391 X-VR-SPAMSTATE: OK X-VR-SPAMSCORE: 0 X-VR-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrgedvkedrfedujedgtdejucetufdoteggodetrfdotffvucfrrhhofhhilhgvmecuqfggjfdpvefjgfevmfevgfenuceurghilhhouhhtmecuhedttdenucenucfjughrpefhvfevufffkffojghfgggtgfesthekredtredtjeenucfhrhhomheppfhitgholhgrshcuifhrrghvvghsuceonhhgrhgrvhgvshesnhhgrhgrvhgvshdrfhhrqeenucggtffrrghtthgvrhhnpedvvddvgedtudffgeegfeeluddtieevvdevuefgledtveduteetgedutdfgleffueenucffohhmrghinhepghhnuhdrohhrghdphhhtthhpqdhfvghttghhuhhnvgigphgvtghtvgguuhhrlhhurhhluggvfhhinhgvmhhotghkqdhhthhtphdqghgvthhtvghsthgtrghsvghlrghmsggurghurhhlrdhrvghsthdpghhithhhuhgsrdgtohhmpdhrvghpohhrthdrmhgupdgrmhgriihonhgrfihsrdgtohhmpdhlihgtvghnshgvpghurhhlrdguohgtshdplhhitggvnhhsvgdrmhgupdhjuhhlihgrhhhusgdrtghomhdpghhithhhuhgsuhhsvghrtghonhhtvghnthdrtghomhdpthhomhhlrdhnrghmvgenucfkphepuddvjedrtddrtddruddpkedurdeijedrudegiedrvddtkedpfeejrdehledrudegvddrleelnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehinhgvthepuddvjedrtddrtddruddpmhgrihhlfhhrohhmpeeonhhgrhgrvhgvshesnhhgrhgrvhgvshdrfhhrqedpnhgspghrtg hpthhtohepuddprhgtphhtthhopeeivddvtddvseguvggssghughhsrdhgnhhurdhorhhgpdfovfetjfhoshhtpehmohehhedtpdhmohguvgepshhmthhpohhuth X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-to: Nicolas Graves X-ACL-Warn: , Nicolas Graves via Guix-patches X-Patchwork-Original-From: Nicolas Graves via Guix-patches via From: Nicolas Graves Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org X-getmail-retrieved-from-mailbox: Patches * tests/juliahub.scm : Add unit tests juliahub-redirect, julia-general-registry-parsing, juliahub-fetch. Change-Id: Ief5133b49a15d0bfc72bb357321126d296ba2802 --- tests/juliahub.scm | 185 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 185 insertions(+) create mode 100644 tests/juliahub.scm diff --git a/tests/juliahub.scm b/tests/juliahub.scm new file mode 100644 index 0000000000..21fdc3a4eb --- /dev/null +++ b/tests/juliahub.scm @@ -0,0 +1,185 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright © 2024 Nicolas Graves +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see . + +;;; Summary +;; Tests for guix/import/juliahub.scm + +(define-module (tests-import-juliahub) + #:use-module (guix base32) + #:use-module (json) + #:use-module (guix import juliahub) + #:use-module (guix import utils) + #:use-module ((guix utils) #:select (call-with-temporary-directory)) + #:use-module (guix tests) + #:use-module (ice-9 match) + #:use-module (srfi srfi-19) + #:use-module (srfi srfi-64) + #:use-module (srfi srfi-71) + #:use-module (web response)) + +;; XXX: Copied from tests/go.scm + +(define (mock-http-fetch testcase) + (lambda (url . rest) + (let ((body (assoc-ref testcase url))) + (if body + (open-input-string body) + (error "mocked http-fetch Unexpected URL: " url))))) + +(define (mock-http-get testcase) + (lambda (url . rest) + (let ((body (assoc-ref testcase url)) + (response-header + (build-response + #:version '(1 . 1) + #:code 200 + #:reason-phrase "Ok" + #:headers `( + (content-type text/html (charset . "utf-8")) + (date . ,(make-date 0 10 58 12 6 3 2021 0)) + (transfer-encoding (chunked))) + #:port #f + #:validate-headers? #t))) + (if body + (values response-header body) + (error "mocked http-get Unexpected URL: " url))))) + +;; Mock an empty directory by replacing hash. +(define* (mock-git->origin repo-url ref #:key (ref->commit #f)) + (let* ((version (if (pair? ref) + (cdr ref) + #f)) + (vcommit (match ref->commit + (#t commit) + (#f version) + ((? procedure?) (ref->commit version)) + (_ #f)))) + `(origin + (method git-fetch) + (uri (git-reference + (url ,(and (not (eq? repo-url 'null)) repo-url)) + (commit ,vcommit))) + (file-name (git-file-name name version)) + (sha256 + (base32 + "0sjjj9z1dhilhpc8pq4154czrb79z9cm044jvn75kxcjv6v5l2m5"))))) + +;; Fixtures. + +(define fixture-pkg.json + (scm->json-string + `(("hosted_uri" . "https://git.savannah.gnu.org/cgit/MyPackage.git") + ("installable" . "missing") + ("license" . "GPL-3.0") + ("tags" . #("my-tag")) + ("uuid" . "aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa") + ("url" . "https://git.savannah.gnu.org/cgit/MyPackage.git") + ("contributors" . #((("url" . "https://github.com/me") ("type" . "User") ("contributions" . 1) ("name" . "me")))) + ("success" . #t) + ("reversedeps" . #()) + ("pkgeval" + ("report_url" . "https://github.com/JuliaCI/NanosoldierReports/blob/master/pkgeval/by_date/2024-02/01/report.md") + ("version" ("minor" . 18) ("patch" . 1) ("build" . #()) ("prerelease" . #()) ("major" . 1)) + ("reason" . "time_limit") + ("log_url" . "https://s3.amazonaws.com/julialang-reports/nanosoldier/pkgeval/by_date/2024-02/01/MyPackage.jl.primary.log") + ("status" . "fail") + ("duration" . 0)) + ("stargazers_count" . 0) + ("deps" . #((("slug" . "3FQLY") + ("registry" . "") + ("versions" . #("0.0.0" "1.6.0-1")) + ("uuid" . "de0858da-6303-5e67-8744-51eddeeeb8d7") + ("name" . "Printf") + ("direct" . #t)) + (("slug" . "SIw1t") + ("registry" . "") + ("versions" . #("*" "0.0.0" "1")) + ("uuid" . "cf7118a7-6976-5b1a-9a39-7adc72f591a4") + ("name" . "UUIDs") + ("direct" . #f)) + (("slug" . "THl1k") + ("registry" . "General") + ("versions" . #("0.7" "1" "1-1.10" "1.0-1.5" "1.3.0-1" "1.6.0-1" "1.9.0-1" "1.9.4-1")) + ("uuid" . "1222c4b2-2114-5bfd-aeef-88e4692bbb3e") + ("name" . "julia") + ("direct" . #f)) + (("slug" . "THl1k") + ("registry" . "General") + ("versions" . #("1.6.0-1")) + ("uuid" . "1222c4b2-2114-5bfd-aeef-88e4692bbb3e") + ("name" . "julia") + ("direct" . #t)))) + ("description" . "My description") + ("version" . "1.0.0") + ("documenter_errored" . "missing") + ("slug" . "MySlg") + ("owner" . "me") + ("release_date" . "Feb 2024") + ("name" . "MyPackage.jl") + ("readme" . "My readme") + ("homepage" . "https://git.savannah.gnu.org/cgit/MyPackage.git") + ("doctype" . "hosted") + ("license_url" . "/docs/MyPackage/MySlg/1.0.0/_packagesource/LICENSE.md")))) + +(define fixtures-juliahub-check-test + `(("https://docs.juliahub.com/MyPackage/" . "\ + + +") + ("https://raw.githubusercontent.com/JuliaRegistries/General/master\ +/M/MyPackage/Package.toml" . "\ +name = \"MyPackage\" +uuid = \"aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa\" +repo = \"https://git.savannah.gnu.org/cgit/MyPackage.git\" +") + ("https://docs.juliahub.com/MyPackage/MySlg/1.0.0/pkg.json" . ,fixture-pkg.json))) + +(test-begin "juliahub") + +;;; Unit tests for (guix import juliahub) + +(test-equal "juliahub-redirect" + "https://docs.juliahub.com/MyPackage/MySlg/1.0.0/" + (mock ((web client) http-get + (mock-http-get fixtures-juliahub-check-test)) + (mock ((guix http-client) http-fetch + (mock-http-fetch fixtures-juliahub-check-test)) + ((@@ (guix import juliahub) juliahub-url) "MyPackage")))) + +(test-equal "julia-general-registry-parsing" + "https://git.savannah.gnu.org/cgit/MyPackage.git" + (mock ((web client) http-get + (mock-http-get fixtures-juliahub-check-test)) + (mock ((guix http-client) http-fetch + (mock-http-fetch fixtures-juliahub-check-test)) + (assoc-ref + ((@@ (guix import juliahub) ini-fetch) + ((@@ (guix import juliahub) general-url) "MyPackage" "Package.toml")) + 'repo)))) + +(test-equal "juliahub-fetch" + #t + (mock ((web client) http-get + (mock-http-get fixtures-juliahub-check-test)) + (mock ((guix http-client) http-fetch + (mock-http-fetch fixtures-juliahub-check-test)) + (mock ((guix import utils) git->origin mock-git->origin) + ((@@ (guix import juliahub) juliahub-package?) + ((@@ (guix import juliahub) juliahub-fetch) "MyPackage")))))) + +(test-end "juliahub")