From patchwork Thu Apr 25 15:59:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Nicolas Graves X-Patchwork-Id: 63364 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 866AC27BBE2; Thu, 25 Apr 2024 17:01:19 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=BAYES_00,MAILING_LIST_MULTI, SPF_HELO_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 9882527BBE9 for ; Thu, 25 Apr 2024 17:01:17 +0100 (BST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1s01Wk-0002E2-Tf; Thu, 25 Apr 2024 12:00:50 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1s01Wi-0002D2-Bl for guix-patches@gnu.org; Thu, 25 Apr 2024 12:00:48 -0400 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1s01Wi-0003mL-2S for guix-patches@gnu.org; Thu, 25 Apr 2024 12:00:48 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1s01Wz-0007ct-Mg for guix-patches@gnu.org; Thu, 25 Apr 2024 12:01:05 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#70570] [PATCH 1/2] guix: import: pypi: Ignore pypi-ignored-inputs. References: <20240425155529.30747-1-ngraves@ngraves.fr> In-Reply-To: <20240425155529.30747-1-ngraves@ngraves.fr> Resent-From: Nicolas Graves Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Thu, 25 Apr 2024 16:01:05 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 70570 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 70570@debbugs.gnu.org Cc: ngraves@ngraves.fr Received: via spool by 70570-submit@debbugs.gnu.org id=B70570.171406084729094 (code B ref 70570); Thu, 25 Apr 2024 16:01:05 +0000 Received: (at 70570) by debbugs.gnu.org; 25 Apr 2024 16:00:47 +0000 Received: from localhost ([127.0.0.1]:33138 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1s01Wb-0007Y9-HC for submit@debbugs.gnu.org; Thu, 25 Apr 2024 12:00:46 -0400 Received: from 5.mo560.mail-out.ovh.net ([87.98.181.248]:41125) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1s01WV-0007WV-BU for 70570@debbugs.gnu.org; Thu, 25 Apr 2024 12:00:39 -0400 Received: from director5.ghost.mail-out.ovh.net (unknown [10.109.139.129]) by mo560.mail-out.ovh.net (Postfix) with ESMTP id 4VQLCz06RDz1Zs5 for <70570@debbugs.gnu.org>; Thu, 25 Apr 2024 16:00:14 +0000 (UTC) Received: from ghost-submission-6684bf9d7b-vpbkv (unknown [10.110.96.9]) by director5.ghost.mail-out.ovh.net (Postfix) with ESMTPS id 90BBB1FE5E; Thu, 25 Apr 2024 16:00:14 +0000 (UTC) Received: from ngraves.fr ([37.59.142.109]) by ghost-submission-6684bf9d7b-vpbkv with ESMTPSA id SVMnHA5+KmYuIAEABnI3AA (envelope-from ); Thu, 25 Apr 2024 16:00:14 +0000 Authentication-Results: garm.ovh; auth=pass (GARM-109S0037da76618-70fc-49d8-b0bc-befd460d2feb, AD8D7EE5F14CB41C4DCF674C83E011AE46681836) smtp.auth=ngraves@ngraves.fr X-OVh-ClientIp: 80.215.5.23 Date: Thu, 25 Apr 2024 17:59:45 +0200 Message-ID: <20240425160010.6243-1-ngraves@ngraves.fr> X-Mailer: git-send-email 2.41.0 MIME-Version: 1.0 X-Ovh-Tracer-Id: 9083197499245978338 X-VR-SPAMSTATE: OK X-VR-SPAMSCORE: 0 X-VR-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrgedvledrudeljedgleeiucetufdoteggodetrfdotffvucfrrhhofhhilhgvmecuqfggjfdpvefjgfevmfevgfenuceurghilhhouhhtmecuhedttdenucenucfjughrpefhvfevufffkffogggtgfesthekredtredtjeenucfhrhhomheppfhitgholhgrshcuifhrrghvvghsuceonhhgrhgrvhgvshesnhhgrhgrvhgvshdrfhhrqeenucggtffrrghtthgvrhhnpeegtdekudeuveeuteegueevffetieegtefgjedtudetgeeigfeuueeftdeiveetgfenucffohhmrghinhepphihphhirdhorhhgnecukfhppeduvdejrddtrddtrddupdektddrvdduhedrhedrvdefpdefjedrheelrddugedvrddutdelnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehinhgvthepuddvjedrtddrtddruddpmhgrihhlfhhrohhmpehnghhrrghvvghssehnghhrrghvvghsrdhfrhdpnhgspghrtghpthhtohepuddprhgtphhtthhopeejtdehjedtseguvggssghughhsrdhgnhhurdhorhhgpdfovfetjfhoshhtpehmohehiedtpdhmohguvgepshhmthhpohhuth X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-to: Nicolas Graves X-ACL-Warn: , Nicolas Graves via Guix-patches X-Patchwork-Original-From: Nicolas Graves via Guix-patches via From: Nicolas Graves Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org X-getmail-retrieved-from-mailbox: Patches * guix/import/pypi.scm (pypi-ignored-inputs): New variable. (compute-inputs): Use it. * tests/pypi.scm (parse-requires.txt): Add ignored input to test the feature. * guix/lint.scm (check-inputs-should-be-native): Adapt list. (check-inputs-should-not-be-an-input-at-all): Use pypi-ignored-list. Change-Id: I774e526c5a090026e778ee44049637174a1dca95 --- guix/import/pypi.scm | 20 +++++++++++++++++--- guix/lint.scm | 11 ++++++----- tests/pypi.scm | 3 ++- 3 files changed, 25 insertions(+), 9 deletions(-) diff --git a/guix/import/pypi.scm b/guix/import/pypi.scm index 6719fde330..a43d4eca1d 100644 --- a/guix/import/pypi.scm +++ b/guix/import/pypi.scm @@ -14,6 +14,7 @@ ;;; Copyright © 2022 Vivien Kraus ;;; Copyright © 2021 Simon Tournier ;;; Copyright © 2022 Hartmut Goebel +;;; Copyright © 2024 Nicolas Graves ;;; ;;; This file is part of GNU Guix. ;;; @@ -61,6 +62,7 @@ (define-module (guix import pypi) #:use-module (guix upstream) #:use-module ((guix licenses) #:prefix license:) #:export (%pypi-base-url + pypi-ignored-inputs parse-requires.txt parse-wheel-metadata specification->requirement-name @@ -77,6 +79,17 @@ (define %pypi-base-url ;; Base URL of the PyPI API. (make-parameter "https://pypi.org/pypi/")) +(define pypi-ignored-inputs + ;; This list contains packages that are useful for development or quality + ;; testing, but that most of the time are not necessary to have as an input. + (list "argparse" ; native + "codecov" "coverage" ; coverage + "black" "isort" "pycodestyle" "pep8" ; style + "pyflakes" "flake8" "pylint" "mypy" ; style+lint + "coveralls" "twine" ; upload integration tools + "pytest-isort" "pytest-flake8" "pytest-cov" "pytest-black" + "pytest-pep8" "pytest-mypy" "pytest-pep8" "pre-commit")) ; variants + (define non-empty-string-or-false (match-lambda ("" #f) @@ -424,9 +437,10 @@ (define (compute-inputs source-url wheel-url archive) "Given the SOURCE-URL and WHEEL-URL of an already downloaded ARCHIVE, return the corresponding list of records." (define (requirements->upstream-inputs deps type) - (filter-map (match-lambda - ("argparse" #f) - (name (upstream-input + (filter-map (lambda (name) + (if (member name pypi-ignored-inputs) + #f + (upstream-input (name name) (downstream-name (python->package-name name)) (type type)))) diff --git a/guix/lint.scm b/guix/lint.scm index 68d532968d..9867b71cfd 100644 --- a/guix/lint.scm +++ b/guix/lint.scm @@ -71,6 +71,7 @@ (define-module (guix lint) hg-reference-url) #:autoload (guix bzr-download) (bzr-reference? bzr-reference-url) + #:use-module ((guix import pypi) #:select (pypi-ignored-inputs)) #:use-module (guix import stackage) #:use-module (ice-9 match) #:use-module (ice-9 regex) @@ -557,14 +558,12 @@ (define (check-inputs-should-be-native package) "m4" "qttools-5" "yasm" "nasm" "fasm" - "python-coverage" "python-cython" "python-docutils" "python-mock" "python-nose" "python-pbr" "python-pytest" - "python-pytest-cov" "python-setuptools-scm" "python-sphinx" "scdoc" @@ -586,9 +585,11 @@ (define (check-inputs-should-be-native package) (define (check-inputs-should-not-be-an-input-at-all package) ;; Emit a warning if some inputs of PACKAGE are likely to should not be ;; an input at all. - (let ((input-names '("python-setuptools" - "python-pip" - "python-pre-commit"))) + (let ((input-names (append + '("python-setuptools" + "python-pip" + "python-pre-commit") + pypi-ignored-inputs))) (map (lambda (input) (make-warning package diff --git a/tests/pypi.scm b/tests/pypi.scm index 42b39cde73..fe01ab3beb 100644 --- a/tests/pypi.scm +++ b/tests/pypi.scm @@ -97,6 +97,7 @@ (define test-requires.txt "\ [test] pytest (>=2.5.0) +pytest-cov # read but ignored ") ;; Beaker contains only optional dependencies. @@ -244,7 +245,7 @@ (define-syntax-rule (with-pypi responses body ...) (map specification->requirement-name test-specifications)) (test-equal "parse-requires.txt" - (list '("foo" "bar") '("pytest")) + (list '("foo" "bar") '("pytest" "pytest-cov")) (mock ((ice-9 ports) call-with-input-file call-with-input-string) (parse-requires.txt test-requires.txt))) From patchwork Thu Apr 25 15:59:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Nicolas Graves X-Patchwork-Id: 63365 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 4B94E27BBEA; Thu, 25 Apr 2024 17:02:28 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=BAYES_00,MAILING_LIST_MULTI, SPF_HELO_PASS autolearn=ham autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 69AB427BBE2 for ; Thu, 25 Apr 2024 17:02:27 +0100 (BST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1s01Xj-0002pN-VP; Thu, 25 Apr 2024 12:01:52 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1s01Xh-0002o6-S4 for guix-patches@gnu.org; Thu, 25 Apr 2024 12:01:49 -0400 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1s01Xh-0004aM-JC for guix-patches@gnu.org; Thu, 25 Apr 2024 12:01:49 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1s01Xy-0007pX-JD for guix-patches@gnu.org; Thu, 25 Apr 2024 12:02:06 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#70570] [PATCH 2/2] guix: pyproject-build-system: Ignore unwanted pytest flags. Resent-From: Nicolas Graves Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Thu, 25 Apr 2024 16:02:05 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 70570 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 70570@debbugs.gnu.org Cc: ngraves@ngraves.fr Received: via spool by 70570-submit@debbugs.gnu.org id=B70570.171406087829485 (code B ref 70570); Thu, 25 Apr 2024 16:02:05 +0000 Received: (at 70570) by debbugs.gnu.org; 25 Apr 2024 16:01:18 +0000 Received: from localhost ([127.0.0.1]:33141 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1s01X6-0007eU-JT for submit@debbugs.gnu.org; Thu, 25 Apr 2024 12:01:17 -0400 Received: from 9.mo575.mail-out.ovh.net ([46.105.78.111]:38727) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1s01WX-0007X4-HA for 70570@debbugs.gnu.org; Thu, 25 Apr 2024 12:00:42 -0400 Received: from director7.ghost.mail-out.ovh.net (unknown [10.109.139.129]) by mo575.mail-out.ovh.net (Postfix) with ESMTP id 4VQLD16NXjz1ZCn for <70570@debbugs.gnu.org>; Thu, 25 Apr 2024 16:00:17 +0000 (UTC) Received: from ghost-submission-6684bf9d7b-82vqs (unknown [10.111.182.237]) by director7.ghost.mail-out.ovh.net (Postfix) with ESMTPS id 6A61A1FD2A; Thu, 25 Apr 2024 16:00:17 +0000 (UTC) Received: from ngraves.fr ([37.59.142.96]) by ghost-submission-6684bf9d7b-82vqs with ESMTPSA id YJQ5FBF+KmYXAwgAseltsA (envelope-from ); Thu, 25 Apr 2024 16:00:17 +0000 Authentication-Results: garm.ovh; auth=pass (GARM-96R001d6bd04c7-908d-463d-afd0-3b33328a9bb0, AD8D7EE5F14CB41C4DCF674C83E011AE46681836) smtp.auth=ngraves@ngraves.fr X-OVh-ClientIp: 80.215.5.23 Date: Thu, 25 Apr 2024 17:59:46 +0200 Message-ID: <20240425160010.6243-2-ngraves@ngraves.fr> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20240425160010.6243-1-ngraves@ngraves.fr> References: <20240425160010.6243-1-ngraves@ngraves.fr> MIME-Version: 1.0 X-Ovh-Tracer-Id: 9084041924015547106 X-VR-SPAMSTATE: OK X-VR-SPAMSCORE: 0 X-VR-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrgedvledrudeljedgleeiucetufdoteggodetrfdotffvucfrrhhofhhilhgvmecuqfggjfdpvefjgfevmfevgfenuceurghilhhouhhtmecuhedttdenucenucfjughrpefhvfevufffkffojghfgggtgfesthekredtredtjeenucfhrhhomheppfhitgholhgrshcuifhrrghvvghsuceonhhgrhgrvhgvshesnhhgrhgrvhgvshdrfhhrqeenucggtffrrghtthgvrhhnpedvleelgeeffeetleekueegvdfhtedvueffiefhkedtleejkeduveelgfdufeduleenucfkphepuddvjedrtddrtddruddpkedtrddvudehrdehrddvfedpfeejrdehledrudegvddrleeinecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehinhgvthepuddvjedrtddrtddruddpmhgrihhlfhhrohhmpehnghhrrghvvghssehnghhrrghvvghsrdhfrhdpnhgspghrtghpthhtohepuddprhgtphhtthhopeejtdehjedtseguvggssghughhsrdhgnhhurdhorhhgpdfovfetjfhoshhtpehmohehjeehpdhmohguvgepshhmthhpohhuth X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-to: Nicolas Graves X-ACL-Warn: , Nicolas Graves via Guix-patches X-Patchwork-Original-From: Nicolas Graves via Guix-patches via From: Nicolas Graves Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org X-getmail-retrieved-from-mailbox: Patches * guix/build/pyproject-build-system.scm : Ignore unwanted pytest flags. Change-Id: Ib9f1602e5af11227e5b7ce124f0f9be4fa2b78e4 --- guix/build/pyproject-build-system.scm | 88 ++++++++++++++++++++++++++- 1 file changed, 86 insertions(+), 2 deletions(-) diff --git a/guix/build/pyproject-build-system.scm b/guix/build/pyproject-build-system.scm index 947d240114..3ca3c76c1c 100644 --- a/guix/build/pyproject-build-system.scm +++ b/guix/build/pyproject-build-system.scm @@ -1,6 +1,7 @@ ;;; GNU Guix --- Functional package management for GNU ;;; Copyright © 2021 Lars-Dominik Braun ;;; Copyright © 2022 Marius Bakke +;;; Copyright © 2024 Nicolas Graves ;;; ;;; This file is part of GNU Guix. ;;; @@ -142,7 +143,86 @@ (define* (build #:key outputs build-backend backend-path configure-flags #:allow wheel-dir config-settings))) -(define* (check #:key tests? test-backend test-flags #:allow-other-keys) +;; Pytest plugin to filter out arguments to ignore. +(define pytest-default-ignore-alist + '(("cov" . ("--cov" "--cov-reset" "--cov-report" "--cov-config" + "--no-cov-on-fail" "--no-cov" "--cov-fail-under" + "--cov-append" "--cov-branch" "--cov-context")) + ("mypy" . ("--mypy" "--mypy-config-file" "--mypy-ignore-missing-imports")) + ("isort" . ("--isort")) + ("flake8" . ("--flake8")) + ("black" . ("--black")) + ("flakes" . ("--flakes")) + ("pep8" . ("--pep8")))) + +;; Allow guix to ignore these options when underlying pytest package is not +;; an input. These flags are not necessary to properly run tests. +(define (pytest-ignore-options-plugin flags) + "This function converts an list of flags in a string that can + be instantiated as a python pytest plugin." + (format #f "\ +import pytest + +def pytest_addoption(parser): + group = parser.getgroup('guix','Guix ignored options') + options = [~{~s, ~}] + for option in options: + group.addoption(option, action='append', nargs='*')" + flags)) + +(define (call-with-guix-pytest-plugin inputs thunk) + (let* ((former-path (getenv "PYTHONPATH")) + (input-names + (filter (match-lambda + (((name . _) ...) + (if (string-prefix? "python-pytest-" name) + name + #f)) + ( _ #f)) + inputs)) + (filtered-flags + (filter identity + (append-map + (match-lambda + ((group . flags) + (if (member (string-append "python-pytest-" group) + input-names) + (list #f) + flags)) + (_ (list #f))) + pytest-default-ignore-alist)))) + (dynamic-wind + (lambda () + (setenv "PYTHONPATH" + (string-append + (if former-path + (string-append former-path ":") + "") + ".guix-pytest")) + (setenv "PYTEST_PLUGINS" + (string-append + (if (getenv "PYTEST_PLUGINS") + (string-append former-path ",") + "") + "pytest_guix_plugin")) + (mkdir-p ".guix-pytest") + (with-output-to-file ".guix-pytest/__init__.py" + (lambda _ (display ""))) + (with-output-to-file ".guix-pytest/pytest_guix_plugin.py" + (lambda _ + (display (pytest-ignore-options-plugin filtered-flags))))) + thunk + (lambda () + (setenv "PYTHONPATH" former-path) + (unsetenv "PYTEST_PLUGINS") + (when (file-exists? ".guix-pytest") + (delete-file-recursively ".guix-pytest")))))) + +(define-syntax-rule (with-guix-pytest-plugin inputs exp ...) + "Evaluate EXP in a context where the Guix pytest plugin is added." + (call-with-guix-pytest-plugin inputs (lambda () exp ...))) + +(define* (check #:key inputs tests? test-backend test-flags #:allow-other-keys) "Run the test suite of a given Python package." (if tests? ;; Unfortunately with PEP 517 there is no common method to specify test @@ -165,7 +245,8 @@ (define* (check #:key tests? test-backend test-flags #:allow-other-keys) (format #t "Using ~a~%" use-test-backend) (match use-test-backend ('pytest - (apply invoke pytest "-vv" test-flags)) + (with-guix-pytest-plugin inputs + (apply invoke pytest "-vv" test-flags))) ('nose (apply invoke nosetests "-v" test-flags)) ('nose2 @@ -386,3 +467,6 @@ (define* (pyproject-build #:key inputs (phases %standard-phases) (apply python:python-build #:inputs inputs #:phases phases args)) ;;; pyproject-build-system.scm ends here +;;; Local Variables: +;;; eval: (put 'with-guix-pytest-plugin 'scheme-indent-function 1) +;;; End: