From patchwork Mon Sep 27 20:11:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Xinglu Chen X-Patchwork-Id: 33364 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 927E127BBE3; Mon, 27 Sep 2021 21:19:04 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,T_DKIM_INVALID, URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id E29B127BBE1 for ; Mon, 27 Sep 2021 21:19:03 +0100 (BST) Received: from localhost ([::1]:34126 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mUx5a-0005Is-US for patchwork@mira.cbaines.net; Mon, 27 Sep 2021 16:19:03 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38716) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mUwyo-0003wC-IZ for guix-patches@gnu.org; Mon, 27 Sep 2021 16:12:04 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:59639) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mUwyo-0001xG-BO for guix-patches@gnu.org; Mon, 27 Sep 2021 16:12:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1mUwyo-0003FV-6s for guix-patches@gnu.org; Mon, 27 Sep 2021 16:12:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#50854] [PATCH 1/2] import: pypi: Honor the 'upstream-name' package property. Resent-From: Xinglu Chen Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Mon, 27 Sep 2021 20:12:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 50854 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 50854@debbugs.gnu.org Received: via spool by 50854-submit@debbugs.gnu.org id=B50854.163277349812447 (code B ref 50854); Mon, 27 Sep 2021 20:12:02 +0000 Received: (at 50854) by debbugs.gnu.org; 27 Sep 2021 20:11:38 +0000 Received: from localhost ([127.0.0.1]:42948 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mUwyJ-0003ET-N3 for submit@debbugs.gnu.org; Mon, 27 Sep 2021 16:11:38 -0400 Received: from h87-96-130-155.cust.a3fiber.se ([87.96.130.155]:60122 helo=mail.yoctocell.xyz) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mUwyH-0003ED-4g for 50854@debbugs.gnu.org; Mon, 27 Sep 2021 16:11:30 -0400 From: Xinglu Chen DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yoctocell.xyz; s=mail; t=1632773481; bh=AMD08MdRIDPBMKnVZNSvVlmQtS2LM7ZucuK20cHwnU0=; h=From:To:Subject:In-Reply-To:References:Date; b=FyDAFr53R24FpHFC5EJFkS9e0s/xnAUp1jeMFLMMiefzIOyrMxHQzcg2uKCKvkUvX GO6Z9tlP+0JFtIVhkSlf1Tpfnou3gpFMaGQqTBf9Z6dcg0ipOu6IzAFbGvjhpc0M63 owZwN3VdNTCMxaHg4H7y/ZFfvAjrtIhG05XkbRR0= In-Reply-To: References: Message-Id: <45f4a38589689ee3c640ff54618405a6eb7c4f40.1632772958.git.public@yoctocell.xyz> Date: Mon, 27 Sep 2021 22:11:20 +0200 MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches Previously, when a PyPI package had a “-” followed by one or more digits in its name, e.g., “AV-98”, the importer would interpret “98” as the version of the package and thus mistake the “AV-98” package for the “av” package on PyPI. $ ./pre-inst-env guix refresh av-98 following redirection to `https://pypi.org/pypi/av/json'... /home/yoctocell/src/guix/gnu/packages/web-browsers.scm:914:13: av-98 would be upgraded from 1.0.1 to 8.0.3 Setting the ‘upstream-name’ property to “AV-98” would solve the problem. $ ./pre-inst-env guix refresh av-98 /home/yoctocell/src/guix/gnu/packages/web-browsers.scm:914:13: 1.0.1 is already the latest version of av-98 * guix/import/pypi.scm (guix-package->pypi-name): Honor ‘upstream-name’ property. (make-pypi-sexp): Set ‘upstream-name’ property when appropriate. * tests/pypi.scm (test-json): Rename to ... (test-json-1): ... this. (test-json-2): New variable ("guix-package->pypi-name, honor 'upstream-name'"): New test. ("pypi->guix-package, package name contains \"-\" followed by digits"): Likewise. --- guix/import/pypi.scm | 20 +++++--- tests/pypi.scm | 106 ++++++++++++++++++++++++++++++++++++++++--- 2 files changed, 113 insertions(+), 13 deletions(-) diff --git a/guix/import/pypi.scm b/guix/import/pypi.scm index 6731d50891..b7859c8341 100644 --- a/guix/import/pypi.scm +++ b/guix/import/pypi.scm @@ -9,6 +9,7 @@ ;;; Copyright © 2020 Lars-Dominik Braun ;;; Copyright © 2020 Arun Isaac ;;; Copyright © 2020 Martin Becze +;;; Copyright © 2021 Xinglu Chen ;;; ;;; This file is part of GNU Guix. ;;; @@ -163,12 +164,13 @@ (define (url->pypi-name url) (hyphen-package-name->name+version (basename (file-sans-extension url)))) - (match (and=> (package-source package) origin-uri) - ((? string? url) - (url->pypi-name url)) - ((lst ...) - (any url->pypi-name lst)) - (#f #f))) + (or (assoc-ref (package-properties package) 'upstream-name) + (match (and=> (package-source package) origin-uri) + ((? string? url) + (url->pypi-name url)) + ((lst ...) + (any url->pypi-name lst)) + (#f #f)))) (define (wheel-url->extracted-directory wheel-url) (match (string-split (basename wheel-url) #\-) @@ -423,6 +425,11 @@ (define (make-pypi-sexp name version source-url wheel-url home-page synopsis description license) "Return the `package' s-expression for a python package with the given NAME, VERSION, SOURCE-URL, HOME-PAGE, SYNOPSIS, DESCRIPTION, and LICENSE." + (define (maybe-upstream-name name) + (if (string-match ".*\\-[0-9]+" (pk name)) + `((properties ,`'(("upstream-name" . ,name)))) + '())) + (call-with-temporary-output-file (lambda (temp port) (and (url-fetch source-url temp) @@ -461,6 +468,7 @@ (define (make-pypi-sexp name version source-url wheel-url home-page synopsis (sha256 (base32 ,(guix-hash-url temp))))) + ,@(maybe-upstream-name name) (build-system python-build-system) ,@(maybe-inputs required-inputs 'propagated-inputs) ,@(maybe-inputs native-inputs 'native-inputs) diff --git a/tests/pypi.scm b/tests/pypi.scm index f421d6d9df..70f4298a90 100644 --- a/tests/pypi.scm +++ b/tests/pypi.scm @@ -2,6 +2,7 @@ ;;; Copyright © 2014 David Thompson ;;; Copyright © 2016 Ricardo Wurmus ;;; Copyright © 2019 Maxim Cournoyer +;;; Copyright © 2021 Xinglu Chen ;;; ;;; This file is part of GNU Guix. ;;; @@ -29,7 +30,7 @@ (define-module (test-pypi) #:use-module (srfi srfi-64) #:use-module (ice-9 match)) -(define test-json +(define test-json-1 "{ \"info\": { \"version\": \"1.0.0\", @@ -57,6 +58,34 @@ (define test-json } }") +(define test-json-2 + "{ + \"info\": { + \"version\": \"1.0.0\", + \"name\": \"foo-99\", + \"license\": \"GNU LGPL\", + \"summary\": \"summary\", + \"home_page\": \"http://example.com\", + \"classifiers\": [], + \"download_url\": \"\" + }, + \"urls\": [], + \"releases\": { + \"1.0.0\": [ + { + \"url\": \"https://example.com/foo-99-1.0.0.egg\", + \"packagetype\": \"bdist_egg\" + }, { + \"url\": \"https://example.com/foo-99-1.0.0.tar.gz\", + \"packagetype\": \"sdist\" + }, { + \"url\": \"https://example.com/foo-99-1.0.0-py2.py3-none-any.whl\", + \"packagetype\": \"bdist_wheel\" + } + ] + } +}") + (define test-source-hash "") @@ -147,6 +176,13 @@ (define test-metadata-with-extras-jedi "\ (uri (list "https://bitheap.org/cram/cram-0.7.tar.gz" (pypi-uri "cram" "0.7")))))))) +(test-equal "guix-package->pypi-name, honor 'upstream-name'" + "bar-3" + (guix-package->pypi-name + (dummy-package "foo" + (properties + '((upstream-name . "bar-3")))))) + (test-equal "specification->requirement-name" '("Fizzy" "PickyThing" "SomethingWithMarker" "requests" "pip") (map specification->requirement-name test-specifications)) @@ -198,8 +234,8 @@ (define test-metadata-with-extras-jedi "\ (lambda (url . rest) (match url ("https://pypi.org/pypi/foo/json" - (values (open-input-string test-json) - (string-length test-json))) + (values (open-input-string test-json-1) + (string-length test-json-1))) ("https://example.com/foo-1.0.0-py2.py3-none-any.whl" #f) (_ (error "Unexpected URL: " url))))) (match (pypi->guix-package "foo") @@ -264,8 +300,8 @@ (define test-metadata-with-extras-jedi "\ (lambda (url . rest) (match url ("https://pypi.org/pypi/foo/json" - (values (open-input-string test-json) - (string-length test-json))) + (values (open-input-string test-json-1) + (string-length test-json-1))) ("https://example.com/foo-1.0.0-py2.py3-none-any.whl" #f) (_ (error "Unexpected URL: " url))))) ;; Not clearing the memoization cache here would mean returning the value @@ -317,8 +353,8 @@ (define test-metadata-with-extras-jedi "\ (lambda (url . rest) (match url ("https://pypi.org/pypi/foo/json" - (values (open-input-string test-json) - (string-length test-json))) + (values (open-input-string test-json-1) + (string-length test-json-1))) ("https://example.com/foo-1.0.0-py2.py3-none-any.whl" #f) (_ (error "Unexpected URL: " url))))) ;; Not clearing the memoization cache here would mean returning the value @@ -345,4 +381,60 @@ (define test-metadata-with-extras-jedi "\ (x (pk 'fail x #f)))))) +(test-assert "pypi->guix-package, package name contains \"-\" followed by digits" + ;; Replace network resources with sample data. + (mock ((guix import utils) url-fetch + (lambda (url file-name) + (match url + ("https://example.com/foo-99-1.0.0.tar.gz" + (begin + ;; Unusual requires.txt location should still be found. + (mkdir-p "foo-99-1.0.0/src/bizarre.egg-info") + (with-output-to-file "foo-99-1.0.0/src/bizarre.egg-info/requires.txt" + (lambda () + (display test-requires.txt))) + (parameterize ((current-output-port (%make-void-port "rw+"))) + (system* "tar" "czvf" file-name "foo-99-1.0.0/")) + (delete-file-recursively "foo-99-1.0.0") + (set! test-source-hash + (call-with-input-file file-name port-sha256)))) + ("https://example.com/foo-99-1.0.0-py2.py3-none-any.whl" #f) + (_ (error "Unexpected URL: " url))))) + (mock ((guix http-client) http-fetch + (lambda (url . rest) + (match url + ("https://pypi.org/pypi/foo-99/json" + (values (open-input-string test-json-2) + (string-length test-json-2))) + ("https://example.com/foo-99-1.0.0-py2.py3-none-any.whl" #f) + (_ (error "Unexpected URL: " url))))) + (match (pypi->guix-package "foo-99") + (('package + ('name "python-foo-99") + ('version "1.0.0") + ('source ('origin + ('method 'url-fetch) + ('uri ('pypi-uri "foo-99" 'version)) + ('sha256 + ('base32 + (? string? hash))))) + ('properties ('quote (("upstream-name" . "foo-99")))) + ('build-system 'python-build-system) + ('propagated-inputs + ('quasiquote + (("python-bar" ('unquote 'python-bar)) + ("python-foo" ('unquote 'python-foo))))) + ('native-inputs + ('quasiquote + (("python-pytest" ('unquote 'python-pytest))))) + ('home-page "http://example.com") + ('synopsis "summary") + ('description "summary") + ('license 'license:lgpl2.0)) + (string=? (bytevector->nix-base32-string + test-source-hash) + hash)) + (x + (pk 'fail x #f)))))) + (test-end "pypi") From patchwork Mon Sep 27 20:11:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Xinglu Chen X-Patchwork-Id: 33363 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id CAD1B27BBE3; Mon, 27 Sep 2021 21:12:29 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,T_DKIM_INVALID, URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 95CAF27BBE1 for ; Mon, 27 Sep 2021 21:12:29 +0100 (BST) Received: from localhost ([::1]:48156 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mUwzE-0003zx-J2 for patchwork@mira.cbaines.net; Mon, 27 Sep 2021 16:12:28 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38718) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mUwyp-0003wG-5E for guix-patches@gnu.org; Mon, 27 Sep 2021 16:12:04 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:59640) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mUwyo-0001xs-OG for guix-patches@gnu.org; Mon, 27 Sep 2021 16:12:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1mUwyo-0003Fc-K5 for guix-patches@gnu.org; Mon, 27 Sep 2021 16:12:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#50854] [PATCH 2/2] gnu: av-98: Set 'upstream-name' property. Resent-From: Xinglu Chen Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Mon, 27 Sep 2021 20:12:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 50854 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 50854@debbugs.gnu.org Received: via spool by 50854-submit@debbugs.gnu.org id=B50854.163277349812453 (code B ref 50854); Mon, 27 Sep 2021 20:12:02 +0000 Received: (at 50854) by debbugs.gnu.org; 27 Sep 2021 20:11:38 +0000 Received: from localhost ([127.0.0.1]:42951 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mUwyQ-0003Ei-Ee for submit@debbugs.gnu.org; Mon, 27 Sep 2021 16:11:38 -0400 Received: from h87-96-130-155.cust.a3fiber.se ([87.96.130.155]:60128 helo=mail.yoctocell.xyz) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mUwyM-0003EK-JB for 50854@debbugs.gnu.org; Mon, 27 Sep 2021 16:11:35 -0400 From: Xinglu Chen DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yoctocell.xyz; s=mail; t=1632773487; bh=669pH3xEPv3NtGY0ZcR3vZM1E/GHBosOdyHQNb5nClI=; h=From:To:Subject:In-Reply-To:References:Date; b=M7TqPfnLaxdM44owN8seaUgBmFlwDGe9lv1V1SYcNDv3qmoXRoLcB0PuhKPrf9zQc bRmBsmxjGwJEx8S3MMoAE1FVo7OovjtYaUrbnoRwDMM72j2udMLsNjT2Ty2lo32a1r kYJr/l313BV4u4HGoI0NkbkNtyw2B6dNONzNIz0Q= In-Reply-To: References: Message-Id: <2547a5160ecf1a25cad90479c09135e676a30f33.1632772958.git.public@yoctocell.xyz> Date: Mon, 27 Sep 2021 22:11:26 +0200 MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches * gnu/packages/web-browsers.scm (av-98)[properties]: Set ‘upstream-name’. --- gnu/packages/web-browsers.scm | 2 ++ 1 file changed, 2 insertions(+) diff --git a/gnu/packages/web-browsers.scm b/gnu/packages/web-browsers.scm index f0aefa3eb4..596d9f7f2b 100644 --- a/gnu/packages/web-browsers.scm +++ b/gnu/packages/web-browsers.scm @@ -912,6 +912,8 @@ (define-public av-98 (package (name "av-98") (version "1.0.1") + (properties + '((upstream-name . "AV-98"))) (source (origin (method url-fetch)