From patchwork Fri Dec 6 18:21:33 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Martin Becze X-Patchwork-Id: 16404 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 1A2251785E; Fri, 6 Dec 2019 18:59:00 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,T_DKIM_INVALID, URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTP id BC7F41785C for ; Fri, 6 Dec 2019 18:58:59 +0000 (GMT) Received: from localhost ([::1]:44078 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1idIod-0006Yq-9v for patchwork@mira.cbaines.net; Fri, 06 Dec 2019 13:58:59 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:49962) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1idIEu-0004MX-NS for guix-patches@gnu.org; Fri, 06 Dec 2019 13:22:06 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1idIEs-0003lT-Sm for guix-patches@gnu.org; Fri, 06 Dec 2019 13:22:04 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:41064) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1idIEs-0003kp-Ou for guix-patches@gnu.org; Fri, 06 Dec 2019 13:22:02 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1idIEs-00054a-KT for guix-patches@gnu.org; Fri, 06 Dec 2019 13:22:02 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#38408] [PATCH v3 1/5] gnu: added new function, find-packages-by-name*/direct Resent-From: Martin Becze Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Fri, 06 Dec 2019 18:22:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 38408 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 38408@debbugs.gnu.org Cc: Martin Becze Received: via spool by 38408-submit@debbugs.gnu.org id=B38408.157565651019443 (code B ref 38408); Fri, 06 Dec 2019 18:22:02 +0000 Received: (at 38408) by debbugs.gnu.org; 6 Dec 2019 18:21:50 +0000 Received: from localhost ([127.0.0.1]:47028 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1idIEf-00053Q-Me for submit@debbugs.gnu.org; Fri, 06 Dec 2019 13:21:50 -0500 Received: from mx1.riseup.net ([198.252.153.129]:41854) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1idIEc-00052b-6L for 38408@debbugs.gnu.org; Fri, 06 Dec 2019 13:21:47 -0500 Received: from capuchin.riseup.net (capuchin-pn.riseup.net [10.0.1.176]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client CN "*.riseup.net", Issuer "Sectigo RSA Domain Validation Secure Server CA" (not verified)) by mx1.riseup.net (Postfix) with ESMTPS id 47V1Bd5my7zFc8L for <38408@debbugs.gnu.org>; Fri, 6 Dec 2019 10:21:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=riseup.net; s=squak; t=1575656505; bh=chL81QND2PvQrUFrNgy/ahQ3nrmiGlgQD4PnAhtyUtw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=aNld9npm25JM9+J4r4ihVIdHTpUcWCSIpbto9QDHfSEetI/6kh3aTVh2xlfew8HPs WhqrGQ6xn5kJqFJac8MUhOel551uF8CZHCrydsMd/vT6xbavN3Po4w7P29gMgKOKbq Mk2il0VPvj3kD0Y9b7IhbMX12fp8PRmdxY6U4aYM= X-Riseup-User-ID: 01300253ABDCBE7EEBA63884393B1C8352BA0A1AC2E89DF5E50076C8AFE96C94 Received: from [127.0.0.1] (localhost [127.0.0.1]) by capuchin.riseup.net (Postfix) with ESMTPSA id 47V1Bd1lmvz8tYt; Fri, 6 Dec 2019 10:21:45 -0800 (PST) From: Martin Becze Date: Fri, 6 Dec 2019 13:21:33 -0500 Message-Id: In-Reply-To: References: MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.51.188.43 X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches * gnu/packages.scm (find-packages-by-naem*/direct) --- gnu/packages.scm | 41 +++++++++++++++++++++++++++++++++++++++++ tests/packages.scm | 13 +++++++++++++ 2 files changed, 54 insertions(+) diff --git a/gnu/packages.scm b/gnu/packages.scm index 959777ff8f..cca2a393e5 100644 --- a/gnu/packages.scm +++ b/gnu/packages.scm @@ -4,6 +4,7 @@ ;;; Copyright © 2014 Eric Bavier ;;; Copyright © 2016, 2017 Alex Kost ;;; Copyright © 2016 Mathieu Lirzin +;;; Copyright © 2019 Martin Becze ;;; ;;; This file is part of GNU Guix. ;;; @@ -52,7 +53,9 @@ %default-package-module-path fold-packages + fold-packages* fold-available-packages + find-packages-by-name*/direct find-newest-available-packages find-packages-by-name @@ -250,6 +253,23 @@ is guaranteed to never traverse the same package twice." init modules)) +(define* (fold-packages* proc init + #:optional + (modules (all-modules (%package-module-path) + #:warn + warn-about-load-error)) + #:key (select? (negate hidden-package?))) + "Call (PROC PACKAGE RESULT) for each available package defined in one of +MODULES that matches SELECT?, using INIT as the initial value of RESULT. It +is guaranteed to never traverse the same package twice." + (fold-module-public-variables* (lambda (module symbol var result) + (let ((object (variable-ref var))) + (if (and (package? object) (select? object)) + (proc module symbol object result) + result))) + init + modules)) + (define %package-cache-file ;; Location of the package cache. "/lib/guix/package.cache") @@ -297,6 +317,27 @@ decreasing version order." matching) matching))))) +(define find-packages-by-name*/direct ;bypass the cache + (let ((packages (delay + (fold-packages* (lambda (mod sym p r) + (vhash-cons (package-name p) (list mod sym p) r)) + vlist-null))) + (version>? (match-lambda* + (((_ _ versions) ..1) + (apply version>? (map package-version versions)))))) + (lambda* (name #:optional version) + "Return the list of ( ) with the given NAME. If + VERSION is not #f, then only return packages whose version is prefixed by + VERSION, sorted in decreasing version order." + (let ((matching (sort (vhash-fold* cons '() name (force packages)) + version>?))) + (if version + (filter (match-lambda + ((_ _ package) + (version-prefix? version (package-version package)))) + matching) + matching))))) + (define (cache-lookup cache name) "Lookup package NAME in CACHE. Return a list sorted in increasing version order." diff --git a/tests/packages.scm b/tests/packages.scm index 423c5061aa..9f02b0d5d2 100644 --- a/tests/packages.scm +++ b/tests/packages.scm @@ -1,6 +1,7 @@ ;;; GNU Guix --- Functional package management for GNU ;;; Copyright © 2012, 2013, 2014, 2015, 2016, 2017, 2018, 2019 Ludovic Courtès ;;; Copyright © Jan (janneke) Nieuwenhuizen +;;; Copyright © 2019 Martin Becze ;;; ;;; This file is part of GNU Guix. ;;; @@ -1135,11 +1136,23 @@ (((? (cut eq? hello <>))) #t) (wrong (pk 'find-packages-by-name wrong #f)))) +(test-assert "find-packages-by-name*/direct" + (match (find-packages-by-name*/direct "hello") + ((((? (cut eq? (resolve-interface '(gnu packages base)) <>)) + (? (cut eq? 'hello <>)) + (? (cut eq? hello <>)))) #t))) + (test-assert "find-packages-by-name with version" (match (find-packages-by-name "hello" (package-version hello)) (((? (cut eq? hello <>))) #t) (wrong (pk 'find-packages-by-name wrong #f)))) +(test-assert "find-packages-by-name*/direct with version" + (match (find-packages-by-name*/direct "hello" (package-version hello)) + ((((? (cut eq? (resolve-interface '(gnu packages base)) <>)) + (? (cut eq? 'hello <>)) + (? (cut eq? hello <>)))) #t))) + (test-equal "find-packages-by-name with cache" (find-packages-by-name "guile") (call-with-temporary-directory From patchwork Fri Dec 6 18:21:34 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Martin Becze X-Patchwork-Id: 16401 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 899891785D; Fri, 6 Dec 2019 18:27:33 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,T_DKIM_INVALID, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTP id D49FC17838 for ; Fri, 6 Dec 2019 18:27:32 +0000 (GMT) Received: from localhost ([::1]:43442 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1idIKC-0002KC-C1 for patchwork@mira.cbaines.net; Fri, 06 Dec 2019 13:27:32 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:50353) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1idIEx-0004R5-Cr for guix-patches@gnu.org; Fri, 06 Dec 2019 13:22:09 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1idIEt-0003nh-Fy for guix-patches@gnu.org; Fri, 06 Dec 2019 13:22:07 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:41065) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1idIEt-0003mU-6o for guix-patches@gnu.org; Fri, 06 Dec 2019 13:22:03 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1idIEt-00054i-2L for guix-patches@gnu.org; Fri, 06 Dec 2019 13:22:03 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#38408] [PATCH v3 2/5] gnu: added new procedure, recusive-import-semver Resent-From: Martin Becze Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Fri, 06 Dec 2019 18:22:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 38408 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 38408@debbugs.gnu.org Cc: Martin Becze Received: via spool by 38408-submit@debbugs.gnu.org id=B38408.157565651119450 (code B ref 38408); Fri, 06 Dec 2019 18:22:03 +0000 Received: (at 38408) by debbugs.gnu.org; 6 Dec 2019 18:21:51 +0000 Received: from localhost ([127.0.0.1]:47030 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1idIEg-00053Y-5A for submit@debbugs.gnu.org; Fri, 06 Dec 2019 13:21:50 -0500 Received: from mx1.riseup.net ([198.252.153.129]:41866) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1idIEd-00052n-45 for 38408@debbugs.gnu.org; Fri, 06 Dec 2019 13:21:48 -0500 Received: from capuchin.riseup.net (capuchin-pn.riseup.net [10.0.1.176]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client CN "*.riseup.net", Issuer "Sectigo RSA Domain Validation Secure Server CA" (not verified)) by mx1.riseup.net (Postfix) with ESMTPS id 47V1Bf4p0DzDrRH for <38408@debbugs.gnu.org>; Fri, 6 Dec 2019 10:21:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=riseup.net; s=squak; t=1575656506; bh=eXiKp6xJqqFQPNzckywNUZngJI+P5LxkAMwTMvBbN2M=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=i+Yf6qUdEZrgIIrRM30/Xntka+E7+Nh7QCRFLAmAdtwivotrEeBD2THTsakG+ZowN xgQyI17OmAwWIH0O7nFegXE8T2yIqj2+k9lM+tUOUM8Qi5ZlmZ/SY22+2E1Q59N4d+ cD60nlRSpqrgPmyUTkNTSKNdjD302qb2h9dDcln4= X-Riseup-User-ID: 9041C2E2D04DA28126AF8E936EA566DD23F20DF015BBBBB46601093772B2B41F Received: from [127.0.0.1] (localhost [127.0.0.1]) by capuchin.riseup.net (Postfix) with ESMTPSA id 47V1Bd6yJwz8tXp; Fri, 6 Dec 2019 10:21:45 -0800 (PST) From: Martin Becze Date: Fri, 6 Dec 2019 13:21:34 -0500 Message-Id: <18d8555e374e1d7682c5f7231be25dd572dacbf0.1575656092.git.mjbecze@riseup.net> In-Reply-To: References: MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.51.188.43 X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches * gnu/packages.scm (recusive-import-semver): New Procedure * gnu/packages.scm (package->definition)[arguments]: New argument, "latest" * tests/import-utils.scm: tests for recusive-import-semver --- guix/import/utils.scm | 181 +++++++++++++++++++++++++++++++++++++++-- tests/import-utils.scm | 162 ++++++++++++++++++++++++++++++++++++ 2 files changed, 336 insertions(+), 7 deletions(-) diff --git a/guix/import/utils.scm b/guix/import/utils.scm index 4694b6e7ef..6932614f8e 100644 --- a/guix/import/utils.scm +++ b/guix/import/utils.scm @@ -5,6 +5,7 @@ ;;; Copyright © 2017, 2019 Ricardo Wurmus ;;; Copyright © 2018 Oleg Pykhalov ;;; Copyright © 2019 Robert Vollmert +;;; Copyright © 2019 Martin Becze ;;; ;;; This file is part of GNU Guix. ;;; @@ -32,6 +33,7 @@ #:use-module (guix discovery) #:use-module (guix build-system) #:use-module (guix gexp) + #:use-module (guix memoization) #:use-module (guix store) #:use-module (guix download) #:use-module (gnu packages) @@ -43,6 +45,8 @@ #:use-module (srfi srfi-11) #:use-module (srfi srfi-26) #:use-module (srfi srfi-41) + #:use-module (semver) + #:use-module (semver ranges) #:export (factorize-uri flatten @@ -69,7 +73,8 @@ guix-name - recursive-import)) + recursive-import + recursive-import-semver)) (define (factorize-uri uri version) "Factorize URI, a package tarball URI as a string, such that any occurrences @@ -257,13 +262,15 @@ package definition." ((package-inputs ...) `((native-inputs (,'quasiquote ,package-inputs)))))) -(define (package->definition guix-package) +(define* (package->definition guix-package #:optional (latest #t)) (match guix-package - (('package ('name (? string? name)) _ ...) - `(define-public ,(string->symbol name) - ,guix-package)) - (('let anything ('package ('name (? string? name)) _ ...)) - `(define-public ,(string->symbol name) + ((or + ('package ('name name) ('version version) . rest) + ('let _ ('package ('name name) ('version version) . rest))) + + `(define-public ,(string->symbol (if latest + name + (string-append name "-" version))) ,guix-package)))) (define (build-system-modules) @@ -414,3 +421,163 @@ dependencies." step ;; initial state (step initial-state))) + +(define* (recursive-import-semver #:key name + (version #f) + name->metadata + metadata->package + metadata-versions + package-dependencies + dependency-name + dependency-range + guix-name + make-sexp) + "Generates a stream of package expressions for the dependencies of the given +NAME and VERSION. The dependencies will be resolved using semantic versioning. +This procedure makes the assumption that most package repositories will, for a +given package provide some on that package that includes what +versions of the package that are available and a list of dependencies for each +version. Dependencies are assumed to be composed of a NAME, a semantic RANGE and +other data. + +This procedure takes the following keys: + NAME - The name of the package to import + VERSION - The version of the package to import + NAME->METADATA - A procedure that takes a NAME of a package and returns that +package's + METADATA->PACKAGE A procedure that takes a package's and VERSION +and returns the for the given VERSION + METADATA-VERSIONS A procedure that that takes a packages and +returns a list of version as strings that are available for the given package + PACKAGE-DEPENDENCIES a procedure that returns a list of given a + + DEPENDENCY-NAME A procedure that takes a and returns the its name + DEPENDENCY-RANGE A procedure that takes a and returns that +decency's range as a string + GUIX-NAME A procedure that take a NAME and returns the Guix version of it + MAKE-SEXP A procedure that takes , and a list of pairs +containing (EXPORT-NAME ), returning the package expression as an +s-expression" + (define mem-name->metadata (memoize name->metadata)) + + (define (latest? versions version) + (equal? (car versions) version)) + + (define (sorted-versions metadata) + (sort (metadata-versions metadata) version>?)) + + (define (name->versions name) + (sorted-versions (mem-name->metadata name))) + + (define (semver-range-contains-string? range version) + (semver-range-contains? range + (string->semver version))) + + (define (guix-export-name name version) + (let ((versions (name->versions name)) + (name (guix-name name))) + (if (latest? versions version) + name + (string-append name "-" version)))) + + ;; checks to see if we already defined or want to define a dep + (define (find-known name range known) + (match + (find + (match-lambda ((dep-name version) + (and + (string=? dep-name name) + (semver-range-contains-string? range version)))) + known) + + (#f #f) + ((name version) (list (guix-export-name name version) version #f))) + ) + + ;; searches searches for a package in guix + (define (find-locally name range) + (match + (find + (match-lambda ((_ _ package) + (semver-range-contains-string? + range + (package-version package)))) + (find-packages-by-name*/direct (guix-name name))) + (#f #f) + ((_ export-symbol package) (list + (symbol->string export-symbol) + (package-version package) #f)))) + + ;; searches for a package in some external repo + (define (find-remote name range) + (let* ((versions (name->versions name)) + (version (find + (lambda (ver) + (semver-range-contains-string? range ver)) + versions)) + (export-name (guix-export-name name version))) + `(,export-name ,version #t))) + + + (define (find-dep-version dep known-deps) + (let* ((name (dependency-name dep)) + (range (string->semver-range (dependency-range dep))) + (export-name-version-needed + (or (find-known name range known-deps) + (find-locally name range) + (find-remote name range)))) + `(,name ,@export-name-version-needed ,dep) + )) + + (define (make-package-definition name version known-deps) + (let* ((metadata (mem-name->metadata name)) + (versions (sorted-versions metadata)) + (package (metadata->package metadata version)) + (deps (map (lambda (dep) + (find-dep-version dep known-deps)) + (package-dependencies package))) + (sexp + (make-sexp metadata package + (map + (match-lambda ((_ export-symbol _ _ dep) + (list export-symbol dep))) + deps)))) + (values + (package->definition sexp (latest? versions version)) + (filter-map + (match-lambda ((name _ version need? dep) + (if need? + (list name version) + #f))) + deps)))) + + (define initial-state + (list #f + (list + ;; packages to find + (list name (if version + version + (car (name->versions name))))) + ;; packages that have been found + (list))) + + (define (step state) + (match state + ((prev ((next-name next-version) . rest) done) + (receive (package dependencies) + (make-package-definition next-name next-version + (append done rest `((,next-name ,next-version)))) + (list + package + (append rest dependencies) + (cons (list next-name next-version) done)))) + ((prev '() done) + (list #f '() done)))) + + (stream-unfold + ;; map: produce a stream element + (match-lambda ((latest queue done) latest)) + ;; predicate + (match-lambda ((latest queue done) latest)) + step + (step initial-state))) diff --git a/tests/import-utils.scm b/tests/import-utils.scm index c3ab25d788..4ed3a5e1da 100644 --- a/tests/import-utils.scm +++ b/tests/import-utils.scm @@ -1,6 +1,7 @@ ;;; GNU Guix --- Functional package management for GNU ;;; Copyright © 2015, 2017 Ricardo Wurmus ;;; Copyright © 2016 Ben Woodcroft +;;; Copyright © 2016 Martin Becze ;;; ;;; This file is part of GNU Guix. ;;; @@ -24,6 +25,10 @@ #:use-module (guix packages) #:use-module (guix build-system) #:use-module (gnu packages) + #:use-module (ice-9 match) + #:use-module (srfi srfi-1) + #:use-module (srfi srfi-9) + #:use-module (srfi srfi-41) #:use-module (srfi srfi-64)) (test-begin "import-utils") @@ -120,4 +125,161 @@ ("license" . #f)))) (package-native-inputs (alist->package meta)))) +(define-record-type + (make-metadata name versions) + metadata? + (name metadata-name) + (versions metadata-versions)) + +(define-record-type + (make-package version dependencies) + package? + (version package-version) + (dependencies package-dependencies)) + +(define-record-type + (make-dependency name range) + dependency? + (name dependency-name) + (range dependency-range)) + +(define (metadata-semver-versions metadata) + (map (lambda (p) + (package-version p)) + (metadata-versions metadata))) + +(define (metadata->package metadata version) + (find + (lambda (package) + (equal? (package-version package) version)) + (metadata-versions metadata))) + +(define (make-sexp metadata package dependencies) + `(package + (name ,(guix-name (metadata-name metadata))) + (version ,(package-version package)) + (dependcies ,(map + (match-lambda ((public-name dep) + (list (guix-name (dependency-name dep)) public-name))) + dependencies)))) + +(define (guix-name name) + (string-append "test-" name)) + +(define packages + `(("no-deps" . (("1.0.0" . ()) ("0.1.0" . ()))) + ("one-dep" . (("1.0.0" . (("no-deps" "^1.0"))) + ("0.1.0" . (("no-deps" "^0.1.0"))))) + ("shared-dep" . (("1.0.0" . (("one-dep" "^0.1.0") + ("no-deps" "*"))))) + ("recursive" . (("1.0.0" . (("recursive" "=1.0.0"))))) + ("already-packaged" . (("1.0.0" . (("rust" "~1.28"))))))) + +(define (name->metadata name) + (let ((versions (assoc-ref packages name))) + (make-metadata name + (map + (match-lambda + ((version . deps) + (make-package version + (map + (lambda (name-range) + (apply make-dependency name-range)) + deps)))) + versions)))) + +(define* (test-recursive-importer name version #:optional (guix-name guix-name)) + (recursive-import-semver #:name name + #:version version + #:name->metadata name->metadata + #:metadata->package metadata->package + #:metadata-versions metadata-semver-versions + #:package-dependencies package-dependencies + #:dependency-name dependency-name + #:dependency-range dependency-range + #:guix-name guix-name + #:make-sexp make-sexp)) + +(test-equal "recursive import test with no dependencies" + `((define-public test-no-deps + (package + (name "test-no-deps") + (version "1.0.0") + (dependcies ())))) + (stream->list (test-recursive-importer "no-deps" "1.0.0"))) + +(test-equal "recursive import test with one dependencies" + `((define-public test-one-dep + (package + (name "test-one-dep") + (version "1.0.0") + (dependcies (("test-no-deps" "test-no-deps"))))) + (define-public test-no-deps + (package + (name "test-no-deps") + (version "1.0.0") + (dependcies ())))) + (stream->list (test-recursive-importer "one-dep" "1.0.0"))) + +(test-equal "recursive import test with recursuve dependencies" + `((define-public test-recursive + (package + (name "test-recursive") + (version "1.0.0") + (dependcies (("test-recursive" "test-recursive")))))) + (stream->list (test-recursive-importer "recursive" "1.0.0"))) + +(test-equal "recursive import test with no dependencies using an old version" + `((define-public test-no-deps-0.1.0 + (package + (name "test-no-deps") + (version "0.1.0") + (dependcies ())))) + (stream->list (test-recursive-importer "no-deps" "0.1.0"))) + +(test-equal "recursive import test with one dependencies unsing an old version" + `((define-public test-one-dep-0.1.0 + (package + (name "test-one-dep") + (version "0.1.0") + (dependcies (("test-no-deps" "test-no-deps-0.1.0"))))) + (define-public test-no-deps-0.1.0 + (package + (name "test-no-deps") + (version "0.1.0") + (dependcies ())))) + (stream->list (test-recursive-importer "one-dep" "0.1.0"))) + +(test-equal "recursive import test with with dependency that is already in the repo" + `((define-public test-already-packaged + (package (name "test-already-packaged") + (version "1.0.0") + (dependcies + (("test-rust" "rust-1.28")))))) + (stream->list (test-recursive-importer "already-packaged" "1.0.0" identity))) + +(test-equal "shared dependencies" + `((define-public test-shared-dep + (package + (name "test-shared-dep") + (version "1.0.0") + (dependcies (("test-one-dep" "test-one-dep-0.1.0") + ("test-no-deps" "test-no-deps"))))) + (define-public test-one-dep-0.1.0 + (package + (name "test-one-dep") + (version "0.1.0") + (dependcies (("test-no-deps" "test-no-deps-0.1.0"))))) + (define-public test-no-deps + (package + (name "test-no-deps") + (version "1.0.0") + (dependcies ()))) + (define-public test-no-deps-0.1.0 + (package + (name "test-no-deps") + (version "0.1.0") + (dependcies())))) + (stream->list (test-recursive-importer "shared-dep" "1.0.0"))) + (test-end "import-utils") From patchwork Fri Dec 6 18:21:35 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Martin Becze X-Patchwork-Id: 16400 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 9C05D1785C; Fri, 6 Dec 2019 18:27:14 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,T_DKIM_INVALID, URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTP id 19A6C17833 for ; Fri, 6 Dec 2019 18:27:14 +0000 (GMT) Received: from localhost ([::1]:43432 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1idIJt-00025F-3M for patchwork@mira.cbaines.net; Fri, 06 Dec 2019 13:27:13 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:50133) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1idIEw-0004Oa-0m for guix-patches@gnu.org; Fri, 06 Dec 2019 13:22:07 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1idIEt-0003oz-OT for guix-patches@gnu.org; Fri, 06 Dec 2019 13:22:05 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:41066) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1idIEt-0003nt-IO for guix-patches@gnu.org; Fri, 06 Dec 2019 13:22:03 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1idIEt-00054p-Eo for guix-patches@gnu.org; Fri, 06 Dec 2019 13:22:03 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#38408] [PATCH v3 3/5] Rewrote some of guix/import/crate.scm to use recursive-import-semver and updated script and test. Resent-From: Martin Becze Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Fri, 06 Dec 2019 18:22:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 38408 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 38408@debbugs.gnu.org Cc: Martin Becze Received: via spool by 38408-submit@debbugs.gnu.org id=B38408.157565651219461 (code B ref 38408); Fri, 06 Dec 2019 18:22:03 +0000 Received: (at 38408) by debbugs.gnu.org; 6 Dec 2019 18:21:52 +0000 Received: from localhost ([127.0.0.1]:47032 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1idIEi-00053m-1P for submit@debbugs.gnu.org; Fri, 06 Dec 2019 13:21:52 -0500 Received: from mx1.riseup.net ([198.252.153.129]:41876) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1idIEd-00052u-Q3 for 38408@debbugs.gnu.org; Fri, 06 Dec 2019 13:21:48 -0500 Received: from capuchin.riseup.net (capuchin-pn.riseup.net [10.0.1.176]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client CN "*.riseup.net", Issuer "Sectigo RSA Domain Validation Secure Server CA" (not verified)) by mx1.riseup.net (Postfix) with ESMTPS id 47V1Bg2yg8zFcpd for <38408@debbugs.gnu.org>; Fri, 6 Dec 2019 10:21:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=riseup.net; s=squak; t=1575656507; bh=/mXFFri2VLK/k5oza9VQ6xurOa3MksoTDnW4B/KLRgU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=UAKf/q95ZXqhWRgVOje3GYhgT1LCO1xxI7j1BhvD698VMsskcJf5cVrQA75wvVWNs /uWbMO86XIAMgV+uayqpxUX5dN1tgLdUFyxCbeX/LKvA6AR1X8D/KA2KKXripiOCfZ GHcx3WD95SmRMMD5uMYhAp+k2Ctm/uPaTLPJ12AI= X-Riseup-User-ID: C46F093E526AEA372FF0A0C87EDB9D310FCDA19AB3B7A5CE8AE047D4B3A0061F Received: from [127.0.0.1] (localhost [127.0.0.1]) by capuchin.riseup.net (Postfix) with ESMTPSA id 47V1Bf5xncz8tXp; Fri, 6 Dec 2019 10:21:46 -0800 (PST) From: Martin Becze Date: Fri, 6 Dec 2019 13:21:35 -0500 Message-Id: <7cccd4231b9c1327d59ff39daf2aa7dd47a2e765.1575656092.git.mjbecze@riseup.net> In-Reply-To: References: MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.51.188.43 X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches * guix/import/crate.scm (make-crate-sexp): Use as args * guix/import/crate.scm (crate->crate-version): New Procedure * guix/import/crate.scm (crate->versions): New Procedure * guix/import/crate.scm (crate-recursive-import): Updated to user recursive-import-semver * guix/scripts/import/crate.scm (guix-import-crate): Remove `define-public` generation from UI * guix/tests/crate.scm: Updated tests --- guix/import/crate.scm | 165 ++++++++++++++++++---------------- guix/scripts/import/crate.scm | 9 +- tests/crate.scm | 2 +- 3 files changed, 91 insertions(+), 85 deletions(-) diff --git a/guix/import/crate.scm b/guix/import/crate.scm index 8dc014d232..da92c43b8c 100644 --- a/guix/import/crate.scm +++ b/guix/import/crate.scm @@ -38,6 +38,7 @@ #:use-module (srfi srfi-1) #:use-module (srfi srfi-2) #:use-module (srfi srfi-26) + #:use-module (srfi srfi-71) #:export (crate->guix-package guix-package->crate-name crate-recursive-import @@ -85,7 +86,7 @@ crate-dependency? json->crate-dependency (id crate-dependency-id "crate_id") ;string - (kind crate-dependency-kind "kind" ;'normal | 'dev + (kind crate-dependency-kind "kind" ;'normal | 'dev | 'build string->symbol) (requirement crate-dependency-requirement "req")) ;string @@ -111,7 +112,9 @@ record or #f if it was not found." (url (string-append (%crate-base-url) path))) (match (assoc-ref (or (json-fetch url) '()) "dependencies") ((? vector? vector) - (map json->crate-dependency (vector->list vector))) + (filter (lambda (dep) + (not (eq? (crate-dependency-kind dep) 'dev))) + (map json->crate-dependency (vector->list vector)))) (_ '())))) @@ -141,62 +144,84 @@ record or #f if it was not found." ((args ...) `((arguments (,'quasiquote ,args)))))) -(define* (make-crate-sexp #:key name version cargo-inputs cargo-development-inputs - home-page synopsis description license - #:allow-other-keys) - "Return the `package' s-expression for a rust package with the given NAME, -VERSION, CARGO-INPUTS, CARGO-DEVELOPMENT-INPUTS, HOME-PAGE, SYNOPSIS, DESCRIPTION, -and LICENSE." - (let* ((port (http-fetch (crate-uri name version))) +(define (make-crate-sexp crate version* dependencies) + "Return the `package' s-expression for a rust package given , + and a list of " + (define normal-dependency? + (match-lambda ((_ dep) (not (eq? (crate-dependency-kind dep) 'dev))))) + + (define (string->license string) + (match (regexp-exec %dual-license-rx string) + (#f (list (spdx-string->license string))) + (m (list (spdx-string->license (match:substring m 1)) + (spdx-string->license (match:substring m 2)))))) + + (let* ((dep-crates dev-dep-crates (partition normal-dependency? dependencies)) + (cargo-inputs (sort (unzip1 dep-crates) + string-ci (crate-version-license version*) + string->license)) + (port (http-fetch (crate-uri name version)) ) (guix-name (crate-name->package-name name)) - (cargo-inputs (map crate-name->package-name cargo-inputs)) - (cargo-development-inputs (map crate-name->package-name - cargo-development-inputs)) (pkg `(package - (name ,guix-name) - (version ,version) - (source (origin - (method url-fetch) - (uri (crate-uri ,name version)) - (file-name (string-append name "-" version ".tar.gz")) - (sha256 - (base32 - ,(bytevector->nix-base32-string (port-sha256 port)))))) - (build-system cargo-build-system) - ,@(maybe-arguments (append (maybe-cargo-inputs cargo-inputs) - (maybe-cargo-development-inputs - cargo-development-inputs))) - (home-page ,(match home-page - (() "") - (_ home-page))) - (synopsis ,synopsis) - (description ,(beautify-description description)) - (license ,(match license - (() #f) - ((license) license) - (_ `(list ,@license))))))) - (close-port port) - pkg)) + (name ,guix-name) + (version ,version) + (source (origin + (method url-fetch) + (uri (crate-uri ,name version)) + (file-name (string-append name "-" version ".crate")) + (sha256 + (base32 + ,(bytevector->nix-base32-string (port-sha256 port)))))) + (build-system cargo-build-system) + ,@(maybe-arguments (append (maybe-cargo-inputs cargo-inputs) + (maybe-cargo-development-inputs + cargo-development-inputs))) + (home-page ,(match home-page + (() "") + (_ home-page))) + (synopsis ,synopsis) + (description ,(beautify-description description)) + (license ,(match license + (() #f) + ((license) license) + (_ `(list ,@license))))))) + + (close-port port) + pkg)) (define %dual-license-rx ;; Dual licensing is represented by a string such as "MIT OR Apache-2.0". ;; This regexp matches that. (make-regexp "^(.*) OR (.*)$")) +(define (crate->crate-version crate version-number) + "returns the for a given CRATE and VERSION-NUMBER" + (find (lambda (version) + (string=? (crate-version-number version) + version-number)) + (crate-versions crate))) + +(define (crate->versions crate) + "Returns a list of versions for a given CRATE" + (map (lambda (version) + (crate-version-number version)) + (crate-versions crate))) + (define* (crate->guix-package crate-name #:optional version) "Fetch the metadata for CRATE-NAME from crates.io, and return the `package' s-expression corresponding to that package, or #f on failure. When VERSION is specified, attempt to fetch that version; otherwise fetch the latest version of CRATE-NAME." - (define (string->license string) - (match (regexp-exec %dual-license-rx string) - (#f (list (spdx-string->license string))) - (m (list (spdx-string->license (match:substring m 1)) - (spdx-string->license (match:substring m 2)))))) - - (define (normal-dependency? dependency) - (eq? (crate-dependency-kind dependency) 'normal)) - (define crate (lookup-crate crate-name)) @@ -205,38 +230,27 @@ latest version of CRATE-NAME." (crate-latest-version crate))) (define version* - (find (lambda (version) - (string=? (crate-version-number version) - version-number)) - (crate-versions crate))) + (crate->crate-version crate version-number)) - (and crate version* - (let* ((dependencies (crate-version-dependencies version*)) - (dep-crates (filter normal-dependency? dependencies)) - (dev-dep-crates (remove normal-dependency? dependencies)) - (cargo-inputs (sort (map crate-dependency-id dep-crates) - string-ci (crate-version-license version*) - string->license)) - (append cargo-inputs cargo-development-inputs))))) + (define dependencies (map + (lambda (dep) + (list (crate-name->package-name + (crate-dependency-id dep)) dep)) + (crate-version-dependencies version*))) + (make-crate-sexp crate version* dependencies)) -(define (crate-recursive-import crate-name) - (recursive-import crate-name #f - #:repo->guix-package (lambda (name repo) - (crate->guix-package name)) - #:guix-name crate-name->package-name)) +(define* (crate-recursive-import name #:optional version) + (recursive-import-semver + #:name name + #:version version + #:name->metadata lookup-crate + #:metadata->package crate->crate-version + #:metadata-versions crate->versions + #:package-dependencies crate-version-dependencies + #:dependency-name crate-dependency-id + #:dependency-range crate-dependency-requirement + #:guix-name crate-name->package-name + #:make-sexp make-crate-sexp)) (define (guix-package->crate-name package) "Return the crate name of PACKAGE." @@ -285,4 +299,3 @@ latest version of CRATE-NAME." (description "Updater for crates.io packages") (pred crate-package?) (latest latest-release))) - diff --git a/guix/scripts/import/crate.scm b/guix/scripts/import/crate.scm index 4690cceb4d..85ae6fbe59 100644 --- a/guix/scripts/import/crate.scm +++ b/guix/scripts/import/crate.scm @@ -96,14 +96,7 @@ Import and convert the crate.io package for PACKAGE-NAME.\n")) (package-name->name+version spec)) (if (assoc-ref opts 'recursive) - (map (match-lambda - ((and ('package ('name name) . rest) pkg) - `(define-public ,(string->symbol name) - ,pkg)) - (_ #f)) - (reverse - (stream->list - (crate-recursive-import name)))) + (stream->list (crate-recursive-import name version)) (let ((sexp (crate->guix-package name version))) (unless sexp (leave (G_ "failed to download meta-data for package '~a'~%") diff --git a/tests/crate.scm b/tests/crate.scm index c14862ad9f..b77cbb08c6 100644 --- a/tests/crate.scm +++ b/tests/crate.scm @@ -95,7 +95,7 @@ ('source ('origin ('method 'url-fetch) ('uri ('crate-uri "foo" 'version)) - ('file-name ('string-append 'name "-" 'version ".tar.gz")) + ('file-name ('string-append 'name "-" 'version ".crate")) ('sha256 ('base32 (? string? hash))))) From patchwork Fri Dec 6 18:21:36 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Martin Becze X-Patchwork-Id: 16405 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id E3E441785E; Fri, 6 Dec 2019 18:59:05 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,T_DKIM_INVALID autolearn=unavailable autolearn_force=no version=3.4.0 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTP id AEA1F1785C for ; Fri, 6 Dec 2019 18:59:05 +0000 (GMT) Received: from localhost ([::1]:44082 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1idIoj-0006gr-7E for patchwork@mira.cbaines.net; Fri, 06 Dec 2019 13:59:05 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:50073) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1idIEv-0004Nj-IV for guix-patches@gnu.org; Fri, 06 Dec 2019 13:22:06 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1idIEu-0003qY-59 for guix-patches@gnu.org; Fri, 06 Dec 2019 13:22:05 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:41067) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1idIEu-0003po-07 for guix-patches@gnu.org; Fri, 06 Dec 2019 13:22:04 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1idIEt-00054w-Rx for guix-patches@gnu.org; Fri, 06 Dec 2019 13:22:03 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#38408] [PATCH v3 4/5] added "#:skip-build? #t" to the output of (make-crate-sexp). Most the the packages imported will be libaries and won't need to build. The top level package will build them though. Resent-From: Martin Becze Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Fri, 06 Dec 2019 18:22:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 38408 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 38408@debbugs.gnu.org Cc: Martin Becze Received: via spool by 38408-submit@debbugs.gnu.org id=B38408.157565651319468 (code B ref 38408); Fri, 06 Dec 2019 18:22:03 +0000 Received: (at 38408) by debbugs.gnu.org; 6 Dec 2019 18:21:53 +0000 Received: from localhost ([127.0.0.1]:47034 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1idIEi-00053q-Mp for submit@debbugs.gnu.org; Fri, 06 Dec 2019 13:21:52 -0500 Received: from mx1.riseup.net ([198.252.153.129]:41892) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1idIEe-000536-Gd for 38408@debbugs.gnu.org; Fri, 06 Dec 2019 13:21:48 -0500 Received: from capuchin.riseup.net (capuchin-pn.riseup.net [10.0.1.176]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client CN "*.riseup.net", Issuer "Sectigo RSA Domain Validation Secure Server CA" (not verified)) by mx1.riseup.net (Postfix) with ESMTPS id 47V1Bh10kwzFcqf for <38408@debbugs.gnu.org>; Fri, 6 Dec 2019 10:21:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=riseup.net; s=squak; t=1575656508; bh=dCIdzcohllKhuGZdXzOmICN0KPZQt8uvs6Lpd8lm9oE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=KT/PaKke41+1QX9KUifRcCFfrY6q8QDwB6aw3ub6uvvlgxTv6pRckaIJd1GoHm/Vx T+pUBlXZXLOdncGbLGFlyCgOU82haFQ/hrKxpYUdKaKBWQiOtHtOI/d61IczJGLuXD GxWWPKbMpNj1EGMLmSraTn4rH44v9oBopf82JPEA= X-Riseup-User-ID: BEE56343BFF31AA3425503BF1F4D995DB9C329AB77454A685D9B34C484F5A1D6 Received: from [127.0.0.1] (localhost [127.0.0.1]) by capuchin.riseup.net (Postfix) with ESMTPSA id 47V1Bg4Fc3z8tXp; Fri, 6 Dec 2019 10:21:47 -0800 (PST) From: Martin Becze Date: Fri, 6 Dec 2019 13:21:36 -0500 Message-Id: In-Reply-To: References: MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.51.188.43 X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches * guix/import/crate.scm (make-crate-sexp): added "#:skip-build? #t" to the output --- guix/import/crate.scm | 3 ++- tests/crate.scm | 3 ++- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/guix/import/crate.scm b/guix/import/crate.scm index da92c43b8c..5683369b7a 100644 --- a/guix/import/crate.scm +++ b/guix/import/crate.scm @@ -183,7 +183,8 @@ record or #f if it was not found." (base32 ,(bytevector->nix-base32-string (port-sha256 port)))))) (build-system cargo-build-system) - ,@(maybe-arguments (append (maybe-cargo-inputs cargo-inputs) + ,@(maybe-arguments (append `(#:skip-build? #t) + (maybe-cargo-inputs cargo-inputs) (maybe-cargo-development-inputs cargo-development-inputs))) (home-page ,(match home-page diff --git a/tests/crate.scm b/tests/crate.scm index b77cbb08c6..64e5b6932e 100644 --- a/tests/crate.scm +++ b/tests/crate.scm @@ -102,7 +102,8 @@ ('build-system 'cargo-build-system) ('arguments ('quasiquote - ('#:cargo-inputs (("rust-bar" ('unquote rust-bar)))))) + ('#:skip-build? #t + #:cargo-inputs (("rust-bar" ('unquote rust-bar)))))) ('home-page "http://example.com") ('synopsis "summary") ('description "summary") From patchwork Fri Dec 6 18:21:37 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Martin Becze X-Patchwork-Id: 16402 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id A37D81785C; Fri, 6 Dec 2019 18:29:46 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,T_DKIM_INVALID autolearn=unavailable autolearn_force=no version=3.4.0 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTP id 6EECE17833 for ; Fri, 6 Dec 2019 18:29:46 +0000 (GMT) Received: from localhost ([::1]:43484 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1idIML-000517-Uh for patchwork@mira.cbaines.net; Fri, 06 Dec 2019 13:29:45 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:50109) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1idIEv-0004OC-Ru for guix-patches@gnu.org; Fri, 06 Dec 2019 13:22:06 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1idIEu-0003rb-IT for guix-patches@gnu.org; Fri, 06 Dec 2019 13:22:05 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:41068) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1idIEu-0003rJ-C7 for guix-patches@gnu.org; Fri, 06 Dec 2019 13:22:04 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1idIEu-000554-9O for guix-patches@gnu.org; Fri, 06 Dec 2019 13:22:04 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#38408] [PATCH v3 5/5] guix: crate: Depublicated dependencies Resent-From: Martin Becze Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Fri, 06 Dec 2019 18:22:04 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 38408 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 38408@debbugs.gnu.org Cc: Martin Becze Received: via spool by 38408-submit@debbugs.gnu.org id=B38408.157565651319475 (code B ref 38408); Fri, 06 Dec 2019 18:22:04 +0000 Received: (at 38408) by debbugs.gnu.org; 6 Dec 2019 18:21:53 +0000 Received: from localhost ([127.0.0.1]:47036 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1idIEi-00053x-Vt for submit@debbugs.gnu.org; Fri, 06 Dec 2019 13:21:53 -0500 Received: from mx1.riseup.net ([198.252.153.129]:41896) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1idIEf-00053I-7e for 38408@debbugs.gnu.org; Fri, 06 Dec 2019 13:21:49 -0500 Received: from capuchin.riseup.net (capuchin-pn.riseup.net [10.0.1.176]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client CN "*.riseup.net", Issuer "Sectigo RSA Domain Validation Secure Server CA" (not verified)) by mx1.riseup.net (Postfix) with ESMTPS id 47V1Bh69HXzDt64 for <38408@debbugs.gnu.org>; Fri, 6 Dec 2019 10:21:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=riseup.net; s=squak; t=1575656508; bh=s7kVPwUcU8b/+mRMpysYnE5JwybAezSwWf79j/xYYaU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=AJkOo2HDM7xQMLOxnW1bWgB5d7Sb0gFoAWnTNTHmJYfglxb97YahbM8n7v7jBneeV NmiU5m7B2NtAVyG4R481v5bLSYbYMyG306nD71hsADs90hf4B70F3Uk5Qd3TwTnlQx fIDcRVFA+/ulPT71aZnfRIEkrR+aCXVs1V99vb+c= X-Riseup-User-ID: 435AD2D2182CCCFD11D5C5BC508140666D027EB730C5BA66DF2CC9D4C9AFEE1F Received: from [127.0.0.1] (localhost [127.0.0.1]) by capuchin.riseup.net (Postfix) with ESMTPSA id 47V1Bh2KPlz8syM; Fri, 6 Dec 2019 10:21:48 -0800 (PST) From: Martin Becze Date: Fri, 6 Dec 2019 13:21:37 -0500 Message-Id: In-Reply-To: References: MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.51.188.43 X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches * guix/import/crate.scm: (crate-version-dependencies): dedup deps --- guix/import/crate.scm | 28 ++++++++++++++++++++-------- 1 file changed, 20 insertions(+), 8 deletions(-) diff --git a/guix/import/crate.scm b/guix/import/crate.scm index 5683369b7a..f3c36ba516 100644 --- a/guix/import/crate.scm +++ b/guix/import/crate.scm @@ -109,14 +109,26 @@ record or #f if it was not found." "Return the list of records of VERSION, a ." (let* ((path (assoc-ref (crate-version-links version) "dependencies")) - (url (string-append (%crate-base-url) path))) - (match (assoc-ref (or (json-fetch url) '()) "dependencies") - ((? vector? vector) - (filter (lambda (dep) - (not (eq? (crate-dependency-kind dep) 'dev))) - (map json->crate-dependency (vector->list vector)))) - (_ - '())))) + (url (string-append (%crate-base-url) path)) + (deps-list (match (assoc-ref (or (json-fetch url) '()) "dependencies") + ((? vector? vector) (vector->list vector)) + (_ + '()))) + ;; turn the raw list into 's and remove dev depenedencies + (deps (filter-map (lambda (json) + (let ((dep (json->crate-dependency json))) + (if (eq? (crate-dependency-kind dep) 'dev) + #f + dep))) + deps-list)) + ;; split normal and build dependencies + (deps-normal deps-build (partition (lambda (dep) + (eq? (crate-dependency-kind dep) 'normal)) + deps))) + ;;remove duplicate normal and build dependencies + (lset-union (lambda (a b) + (string= (crate-dependency-id a) (crate-dependency-id a))) + deps-normal deps-build))) ;;;