From patchwork Thu Dec 21 22:01:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: David Elsing X-Patchwork-Id: 57970 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 966A327BBE2; Thu, 21 Dec 2023 22:03:36 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,SPF_HELO_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 3E01C27BBEA for ; Thu, 21 Dec 2023 22:03:32 +0000 (GMT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rGR88-0004gc-9x; Thu, 21 Dec 2023 17:03:00 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rGR87-0004fp-44 for guix-patches@gnu.org; Thu, 21 Dec 2023 17:02:59 -0500 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rGR86-0005JB-Rx for guix-patches@gnu.org; Thu, 21 Dec 2023 17:02:58 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1rGR8A-0004iF-VC for guix-patches@gnu.org; Thu, 21 Dec 2023 17:03:02 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#67960] [PATCH 2/4] import: crate: Optionally import dev-dependencies recursively. Resent-From: David Elsing Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Thu, 21 Dec 2023 22:03:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 67960 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 67960@debbugs.gnu.org Cc: David Elsing Received: via spool by 67960-submit@debbugs.gnu.org id=B67960.170319617118076 (code B ref 67960); Thu, 21 Dec 2023 22:03:02 +0000 Received: (at 67960) by debbugs.gnu.org; 21 Dec 2023 22:02:51 +0000 Received: from localhost ([127.0.0.1]:45620 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rGR7x-0004hT-Oo for submit@debbugs.gnu.org; Thu, 21 Dec 2023 17:02:50 -0500 Received: from mout02.posteo.de ([185.67.36.66]:59121) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rGR7p-0004gc-8w for 67960@debbugs.gnu.org; Thu, 21 Dec 2023 17:02:44 -0500 Received: from submission (posteo.de [185.67.36.169]) by mout02.posteo.de (Postfix) with ESMTPS id 7761B240103 for <67960@debbugs.gnu.org>; Thu, 21 Dec 2023 23:02:31 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1703196151; bh=ypS3YGMTYlH2BzJ9VbUqH7laHNlWak9KlOqJOPwdIWs=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version: Content-Transfer-Encoding:From; b=CeoODpYeDxc0jgbYBP5Kz5L0DcXcQtaIPC2CTeMz0JuYYuWXLjR57ukZZJAxW1jwt K83p5dYTA9N2Z0v4JaFN1FipKx30MwRc9Ym09RyIoUy1kfOU83f3AFmO7PYYAkcLmA q2dsYTjSpgXNqVDPcK4LWQAmI6//lMIv/m4guKR4whoGV6lVV7vat1YijMxTAI5Izr Eini62hHYKEmkAwsN0uh6WM8r5AqJ5tz6XqRIdqAPvFGrtrmvoMbX9GFXLhsuh62Lc IkQlfrCoTI56H9uglFw4bpVRjtl6s8LoHGU3HMru2ysaruj038m2GzL9szQk7QTT0n bbeDabPuHBC6w== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4Sx4D70WZvz6tvh; Thu, 21 Dec 2023 23:02:31 +0100 (CET) From: David Elsing Date: Thu, 21 Dec 2023 22:01:50 +0000 Message-ID: <068ee00edf887f21bea0267064f729ecaff05190.1703195451.git.david.elsing@posteo.net> In-Reply-To: References: MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org X-getmail-retrieved-from-mailbox: Patches If --recursive-dev-dependencies is specified, development dependencies are also included for all recursivly imported packages. * doc/guix.texi (Invoking guix import): Mention --recursive-dev-dependencies. * guix/import/crate.scm (crate-recursive-import): Add recursive-dev-dependencies? argument. * guix/scripts/import/crate.scm (show-help, guix-import-crate): Add "--recursive-dev-dependencies". * tests/crate.scm: Test both #f and #t for #:recursive-dev-dependencies? in the 'cargo-recursive-import' test. (test-root-dependencies): Add intermediate-c as dev-dependency. (test-intermediate-c-crate,test-intermediate-c-dependencies): New variables. --- doc/guix.texi | 4 + guix/import/crate.scm | 7 +- guix/scripts/import/crate.scm | 12 +- tests/crate.scm | 228 +++++++++++++++++++++++++++++++++- 4 files changed, 244 insertions(+), 7 deletions(-) diff --git a/doc/guix.texi b/doc/guix.texi index b742a3d5b2..a19671643b 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -14512,6 +14512,10 @@ Additional options include: Traverse the dependency graph of the given upstream package recursively and generate package expressions for all those packages that are not yet in Guix. +@item --recursive-dev-dependencies +If @option{--recursive} is specified, also the recursively imported +packages contain their development dependencies, which are recursively +imported as well. @end table @item elm diff --git a/guix/import/crate.scm b/guix/import/crate.scm index 07874bdb26..db5461312f 100644 --- a/guix/import/crate.scm +++ b/guix/import/crate.scm @@ -328,14 +328,17 @@ (define (sort-map-dependencies deps) (append cargo-inputs cargo-development-inputs))) (values #f '()))) -(define* (crate-recursive-import crate-name #:key version) +(define* (crate-recursive-import + crate-name #:key version recursive-dev-dependencies?) (recursive-import crate-name #:repo->guix-package (let ((crate->guix-package* (memoize crate->guix-package))) (lambda* params ;; download development dependencies only for the top level package - (let ((include-dev-deps? (equal? (car params) crate-name))) + (let ((include-dev-deps? + (or (equal? (car params) crate-name) + recursive-dev-dependencies?))) (apply crate->guix-package* (append params `(#:include-dev-deps? ,include-dev-deps?)))))) #:version version diff --git a/guix/scripts/import/crate.scm b/guix/scripts/import/crate.scm index 038faa87db..b13b6636a6 100644 --- a/guix/scripts/import/crate.scm +++ b/guix/scripts/import/crate.scm @@ -5,6 +5,7 @@ ;;; Copyright © 2019, 2020 Martin Becze ;;; Copyright © 2021 Sarah Morgensen ;;; Copyright © 2023 Simon Tournier +;;; Copyright © 2023 David Elsing ;;; ;;; This file is part of GNU Guix. ;;; @@ -47,6 +48,9 @@ (define (show-help) Import and convert the crates.io package for PACKAGE-NAME.\n")) (display (G_ " -r, --recursive import packages recursively")) + (display (G_ " + --recursive-dev-dependencies + include dev-dependencies recursively")) (newline) (display (G_ " -h, --help display this help and exit")) @@ -67,6 +71,9 @@ (define %options (option '(#\r "recursive") #f #f (lambda (opt name arg result) (alist-cons 'recursive #t result))) + (option '("recursive-dev-dependencies") #f #f + (lambda (opt name arg result) + (alist-cons 'recursive-dev-dependencies #t result))) %standard-import-options)) @@ -92,7 +99,10 @@ (define-values (name version) (package-name->name+version spec)) (match (if (assoc-ref opts 'recursive) - (crate-recursive-import name #:version version) + (crate-recursive-import + name #:version version + #:recursive-dev-dependencies? + (assoc-ref opts 'recursive-dev-dependencies)) (crate->guix-package name #:version version #:include-dev-deps? #t)) ((or #f '()) (leave (G_ "failed to download meta-data for package '~a'~%") diff --git a/tests/crate.scm b/tests/crate.scm index 5aea5efaf3..1b9ad88358 100644 --- a/tests/crate.scm +++ b/tests/crate.scm @@ -4,6 +4,7 @@ ;;; Copyright © 2019, 2020, 2022 Ludovic Courtès ;;; Copyright © 2020 Martin Becze ;;; Copyright © 2023 Efraim Flashner +;;; Copyright © 2023 David Elsing ;;; ;;; This file is part of GNU Guix. ;;; @@ -40,10 +41,11 @@ (define-module (test-crate) ;; ;; root-1.0.0 ;; root-1.0.4 -;; intermediate-a 1.0.42 -;; intermeidate-b ^1.0.0 +;; intermediate-a 1.0.42 +;; intermediate-b ^1.0.0 ;; leaf-alice ^0.7 -;; leaf-bob ^3 +;; leaf-bob ^3 +;; intermediate-c 1 (dev-dependency) ;; ;; intermediate-a-1.0.40 ;; intermediate-a-1.0.42 @@ -55,6 +57,9 @@ (define-module (test-crate) ;; intermediate-b-1.2.3 ;; leaf-bob 3.0.1 ;; +;; intermediate-c-1.0.1 +;; leaf-alice 0.7.5 (dev-dependency) +;; ;; leaf-alice-0.7.3 ;; leaf-alice-0.7.5 ;; @@ -164,6 +169,11 @@ (define test-root-dependencies \"crate_id\": \"leaf-bob\", \"kind\": \"normal\", \"req\": \"^3\" + }, + { + \"crate_id\": \"intermediate-c\", + \"kind\": \"dev\", + \"req\": \"1\" } ] }") @@ -262,6 +272,40 @@ (define test-intermediate-b-dependencies ] }") +(define test-intermediate-c-crate + "{ + \"crate\": { + \"max_version\": \"1.0.1\", + \"name\": \"intermediate-c\", + \"description\": \"summary\", + \"homepage\": \"http://example.com\", + \"repository\": \"http://example.com\", + \"keywords\": [\"dummy\", \"test\"], + \"categories\": [\"test\"], + \"actual_versions\": [ + { \"id\": 234290, + \"num\": \"1.0.1\", + \"license\": \"MIT OR Apache-2.0\", + \"links\": { + \"dependencies\": \"/api/v1/crates/intermediate-c/1.0.1/dependencies\" + }, + \"yanked\": false + } + ] + } +}") + +(define test-intermediate-c-dependencies + "{ + \"dependencies\": [ + { + \"crate_id\": \"leaf-alice\", + \"kind\": \"dev\", + \"req\": \"0.7.5\" + } + ] +}") + (define test-leaf-alice-crate "{ \"crate\": { @@ -430,6 +474,15 @@ (define have-guile-semver? (open-input-string "empty file\n")) ("https://crates.io/api/v1/crates/intermediate-b/1.2.3/dependencies" (open-input-string test-intermediate-b-dependencies)) + ("https://crates.io/api/v1/crates/intermediate-c" + (open-input-string test-intermediate-c-crate)) + ("https://crates.io/api/v1/crates/intermediate-c/1.0.1/download" + (set! test-source-hash + (bytevector->nix-base32-string + (sha256 (string->bytevector "empty file\n" "utf-8")))) + (open-input-string "empty file\n")) + ("https://crates.io/api/v1/crates/intermediate-c/1.0.1/dependencies" + (open-input-string test-intermediate-c-dependencies)) ("https://crates.io/api/v1/crates/leaf-alice" (open-input-string test-leaf-alice-crate)) ("https://crates.io/api/v1/crates/leaf-alice/0.7.5/download" @@ -452,7 +505,27 @@ (define have-guile-semver? (match (crate-recursive-import "root") ;; rust-intermediate-b has no dependency on the rust-leaf-alice ;; package, so this is a valid ordering - (((define-public 'rust-leaf-alice-0.7 + (((define-public 'rust-intermediate-c-1 + (package + (name "rust-intermediate-c") + (version "1.0.1") + (source + (origin + (method url-fetch) + (uri (crate-uri "intermediate-c" version)) + (file-name + (string-append name "-" version ".tar.gz")) + (sha256 + (base32 + (? string? hash))))) + (build-system cargo-build-system) + (arguments + ('quasiquote (#:skip-build? #t))) + (home-page "http://example.com") + (synopsis "summary") + (description "summary") + (license (list license:expat license:asl2.0)))) + (define-public 'rust-leaf-alice-0.7 (package (name "rust-leaf-alice") (version "0.7.5") @@ -562,11 +635,158 @@ (define-public 'rust-root-1 ('unquote rust-intermediate-b-1)) ("rust-leaf-alice" ('unquote 'rust-leaf-alice-0.7)) + ("rust-leaf-bob" + ('unquote rust-leaf-bob-3))) + #:cargo-development-inputs + (("rust-intermediate-c" + ('unquote rust-intermediate-c-1)))))) + (home-page "http://example.com") + (synopsis "summary") + (description "summary") + (license (list license:expat license:asl2.0))))) + #t) + (x + (pk 'fail x #f))) + (match (crate-recursive-import "root" + #:recursive-dev-dependencies? #t) + ;; rust-intermediate-b has no dependency on the rust-leaf-alice + ;; package, so this is a valid ordering + (((define-public 'rust-intermediate-c-1 + (package + (name "rust-intermediate-c") + (version "1.0.1") + (source + (origin + (method url-fetch) + (uri (crate-uri "intermediate-c" version)) + (file-name + (string-append name "-" version ".tar.gz")) + (sha256 + (base32 + (? string? hash))))) + (build-system cargo-build-system) + (arguments + ('quasiquote (#:cargo-development-inputs + (("rust-leaf-alice" + ('unquote rust-leaf-alice-0.7)))))) + (home-page "http://example.com") + (synopsis "summary") + (description "summary") + (license (list license:expat license:asl2.0)))) + (define-public 'rust-leaf-alice-0.7 + (package + (name "rust-leaf-alice") + (version "0.7.5") + (source + (origin + (method url-fetch) + (uri (crate-uri "leaf-alice" version)) + (file-name + (string-append name "-" version ".tar.gz")) + (sha256 + (base32 + (? string? hash))))) + (build-system cargo-build-system) + (home-page "http://example.com") + (synopsis "summary") + (description "summary") + (license (list license:expat license:asl2.0)))) + (define-public 'rust-leaf-bob-3 + (package + (name "rust-leaf-bob") + (version "3.0.1") + (source + (origin + (method url-fetch) + (uri (crate-uri "leaf-bob" version)) + (file-name + (string-append name "-" version ".tar.gz")) + (sha256 + (base32 + (? string? hash))))) + (build-system cargo-build-system) + (home-page "http://example.com") + (synopsis "summary") + (description "summary") + (license (list license:expat license:asl2.0)))) + (define-public 'rust-intermediate-b-1 + (package + (name "rust-intermediate-b") + (version "1.2.3") + (source + (origin + (method url-fetch) + (uri (crate-uri "intermediate-b" version)) + (file-name + (string-append name "-" version ".tar.gz")) + (sha256 + (base32 + (? string? hash))))) + (build-system cargo-build-system) + (arguments + ('quasiquote (#:cargo-inputs + (("rust-leaf-bob" + ('unquote rust-leaf-bob-3)))))) + (home-page "http://example.com") + (synopsis "summary") + (description "summary") + (license (list license:expat license:asl2.0)))) + (define-public 'rust-intermediate-a-1 + (package + (name "rust-intermediate-a") + (version "1.0.42") + (source + (origin + (method url-fetch) + (uri (crate-uri "intermediate-a" version)) + (file-name + (string-append name "-" version ".tar.gz")) + (sha256 + (base32 + (? string? hash))))) + (build-system cargo-build-system) + (arguments + ('quasiquote (#:cargo-inputs + (("rust-intermediate-b" + ('unquote rust-intermediate-b-1)) + ("rust-leaf-alice" + ('unquote 'rust-leaf-alice-0.7)) ("rust-leaf-bob" ('unquote rust-leaf-bob-3)))))) (home-page "http://example.com") (synopsis "summary") (description "summary") + (license (list license:expat license:asl2.0)))) + (define-public 'rust-root-1 + (package + (name "rust-root") + (version "1.0.4") + (source + (origin + (method url-fetch) + (uri (crate-uri "root" version)) + (file-name + (string-append name "-" version ".tar.gz")) + (sha256 + (base32 + (? string? hash))))) + (build-system cargo-build-system) + (arguments + ('quasiquote (#:cargo-inputs + (("rust-intermediate-a" + ('unquote rust-intermediate-a-1)) + ("rust-intermediate-b" + ('unquote rust-intermediate-b-1)) + ("rust-leaf-alice" + ('unquote 'rust-leaf-alice-0.7)) + ("rust-leaf-bob" + ('unquote rust-leaf-bob-3))) + #:cargo-development-inputs + (("rust-intermediate-c" + ('unquote rust-intermediate-c-1)))))) + (home-page "http://example.com") + (synopsis "summary") + (description "summary") (license (list license:expat license:asl2.0))))) #t) (x