From patchwork Tue Jul 6 12:16:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Lars-Dominik Braun X-Patchwork-Id: 31194 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 6F26927BC81; Tue, 6 Jul 2021 13:17:59 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, MAILING_LIST_MULTI,SPF_HELO_PASS,T_DKIM_INVALID,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 39D9C27BC78 for ; Tue, 6 Jul 2021 13:17:58 +0100 (BST) Received: from localhost ([::1]:59042 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m0k1V-0004Gx-6v for patchwork@mira.cbaines.net; Tue, 06 Jul 2021 08:17:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:44596) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m0k0g-0004AQ-Cu for guix-patches@gnu.org; Tue, 06 Jul 2021 08:17:06 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:36012) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1m0k0c-00082W-C0 for guix-patches@gnu.org; Tue, 06 Jul 2021 08:17:05 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1m0k0c-0000Ca-4F for guix-patches@gnu.org; Tue, 06 Jul 2021 08:17:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#46848] Questions regarding Python packaging Resent-From: Lars-Dominik Braun Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Tue, 06 Jul 2021 12:17:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 46848 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Hartmut Goebel Cc: guix-devel@gnu.org, 46848@debbugs.gnu.org Received: via spool by 46848-submit@debbugs.gnu.org id=B46848.1625573790736 (code B ref 46848); Tue, 06 Jul 2021 12:17:02 +0000 Received: (at 46848) by debbugs.gnu.org; 6 Jul 2021 12:16:30 +0000 Received: from localhost ([127.0.0.1]:47558 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1m0jzw-0000Be-I2 for submit@debbugs.gnu.org; Tue, 06 Jul 2021 08:16:30 -0400 Received: from mout-p-201.mailbox.org ([80.241.56.171]:24830) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1m0jzs-0000BO-Ok for 46848@debbugs.gnu.org; Tue, 06 Jul 2021 08:16:19 -0400 Received: from smtp1.mailbox.org (smtp1.mailbox.org [IPv6:2001:67c:2050:105:465:1:1:0]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-384) server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mout-p-201.mailbox.org (Postfix) with ESMTPS id 4GK1k268HnzQkBv; Tue, 6 Jul 2021 14:16:10 +0200 (CEST) X-Virus-Scanned: amavisd-new at heinlein-support.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6xq.net; s=MBO0001; t=1625573768; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=kaG6VNJJty9xD8VZJalEGuS/bvkEBLt3J6gywW9g/FE=; b=AS+BvMta1I60rCfI01/ndRjOLeRwAARp3HNWyYP55B51p2FZtH+1ujMJV08uesabLosyca UHPt8FA8btqcaQzlTHUENdKkEivGrLyzMHnaVsQXI+dUBFno/chXDz22eIDkDJMRqgrsDO Tr4Yq9OUQGf3sAxWpsPnf0PK37oSjBZAaC6905stBE9cOA8KHhgwsT4AMcRVUHJnpUhTSW n8LWgLfm73SX0f7etcu+HEK9y4tZGkiguIX/dAtIt1I8227TgV6BuHwtO1PX9EzG4uSRQ8 fp9vz22xUaehrwixtxGMcmrZl7hbaAQYYEj8Yg3s6edW+DaHlf6YIqDbAmk7aw== Received: from smtp1.mailbox.org ([80.241.60.240]) by gerste.heinlein-support.de (gerste.heinlein-support.de [91.198.250.173]) (amavisd-new, port 10030) with ESMTP id HlXG7a2YG9nd; Tue, 6 Jul 2021 14:16:05 +0200 (CEST) Date: Tue, 6 Jul 2021 14:16:03 +0200 From: Lars-Dominik Braun Message-ID: References: <1609946775.8blxygrg9p.astroid@rafflesia.none> <1611303651.35tpgtn1z1.astroid@melmoth.none> <1622997703.qcpe1ehxem.astroid@melmoth.none> <520a5492-6467-bbfc-3252-f17a5cc5d16f@crazy-compilers.com> <681eb450-0185-a465-3ed1-8446e8ad0974@crazy-compilers.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-MBO-SPAM-Probability: X-Rspamd-Score: -3.11 / 15.00 / 15.00 X-Rspamd-Queue-Id: 97B8718B4 X-Rspamd-UID: 6ba2c3 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches Hi again, > No, try > > git clone https://github.com/pypa/pep517.git > cd pep517 > pip wheel --use-pep517 -v . > > which has no setup.py and uses flit instead. pip can build it, because > it supports PEP 517-based builds. As I said, if we decide to keep it > bundled with our python package, there’s no good reason to choose > pypa-build. I now have a proof of concept for a pip-based python-build-system, see 1st patch. The changeset is quite small, but it does not handle testing at all right now. Note that alot of setuptools-based packages lack a dependency on python-wheel (2nd patch; pardon the whitespace errors), which is required when using pyproject.toml (and also declared there, but our Python importer cannot read that file yet). But – as expected – that’s really the only issue I’ve encountered while rebuilding alot of Python packages so far. Cheers, Lars From 7e6d12aac228f369a1f09870529dc003a8ad356e Mon Sep 17 00:00:00 2001 From: Lars-Dominik Braun Date: Tue, 6 Jul 2021 14:14:21 +0200 Subject: [PATCH 2/2] dirty: Fix build errors. --- gnu/packages/build-tools.scm | 2 ++ gnu/packages/check.scm | 16 ++++++++---- gnu/packages/python-build.scm | 47 ++++++++++++++++++++++++++++++++++ gnu/packages/python-crypto.scm | 9 ++++--- gnu/packages/python-xyz.scm | 44 ++++++++++++++++--------------- gnu/packages/time.scm | 4 ++- 6 files changed, 92 insertions(+), 30 deletions(-) diff --git a/gnu/packages/build-tools.scm b/gnu/packages/build-tools.scm index dc2411801c..6c29fb9bdc 100644 --- a/gnu/packages/build-tools.scm +++ b/gnu/packages/build-tools.scm @@ -50,6 +50,7 @@ #:use-module (gnu packages pretty-print) #:use-module (gnu packages protobuf) #:use-module (gnu packages python) + #:use-module (gnu packages python-build) #:use-module (gnu packages python-crypto) #:use-module (gnu packages python-web) #:use-module (gnu packages python-xyz) @@ -237,6 +238,7 @@ files and generates build instructions for the Ninja build system.") (delete 'wrap)))) (inputs `(("ninja" ,ninja))) (propagated-inputs `(("python" ,python))) + (native-inputs `(("python-wheel" ,python-wheel))) (home-page "https://mesonbuild.com/") (synopsis "Build system designed to be fast and user-friendly") (description diff --git a/gnu/packages/check.scm b/gnu/packages/check.scm index 11117b88e4..12f17c41c6 100644 --- a/gnu/packages/check.scm +++ b/gnu/packages/check.scm @@ -975,6 +975,7 @@ standard library.") ("python-mock" ,python-mock) ("python-pytest" ,python-pytest-bootstrap) ("python-setuptools-scm" ,python-setuptools-scm) + ("python-wheel" ,python-wheel) ("python-toml" ,python-toml) ("python-xmlschema" ,python-xmlschema))) (home-page "https://docs.pytest.org/en/latest/") @@ -1028,7 +1029,8 @@ and many external plugins.") (name "python-pytest-bootstrap") (native-inputs `(("python-iniconfig" ,python-iniconfig) ("python-setuptools-scm" ,python-setuptools-scm) - ("python-toml" ,python-toml))) + ("python-toml" ,python-toml) + ("python-wheel" ,python-wheel))) (arguments `(#:tests? #f)) (properties `((python2-variant . ,(delay python2-pytest-bootstrap)))))) @@ -1113,7 +1115,8 @@ supports coverage of subprocesses.") (format #t "test suite not run~%")) #t))))) (native-inputs - `(("python-setuptools-scm" ,python-setuptools-scm))) + `(("python-setuptools-scm" ,python-setuptools-scm) + ("python-wheel" ,python-wheel))) (home-page "https://github.com/pytest-dev/pytest-runner") (synopsis "Invoke py.test as a distutils command") (description @@ -1236,7 +1239,8 @@ same arguments.") (when tests? (invoke "py.test" "-v"))))))) (native-inputs - `(("python-setuptools-scm" ,python-setuptools-scm))) + `(("python-setuptools-scm" ,python-setuptools-scm) + ("python-wheel" ,python-wheel))) (propagated-inputs `(("python-execnet" ,python-execnet) ("python-pytest" ,python-pytest) @@ -1302,7 +1306,8 @@ timeout has been exceeded.") (build-system python-build-system) (native-inputs `(("python-pytest" ,python-pytest) - ("python-setuptools-scm" ,python-setuptools-scm))) + ("python-setuptools-scm" ,python-setuptools-scm) + ("python-wheel" ,python-wheel))) (home-page "https://github.com/pytest-dev/pytest-forked") (synopsis @@ -2486,7 +2491,8 @@ create data based on random numbers and yet remain repeatable.") (build-system python-build-system) (native-inputs `(("python-mock" ,python-mock) - ("python-pytest" ,python-pytest))) + ("python-pytest" ,python-pytest) + ("python-wheel" ,python-wheel))) (propagated-inputs `(("python-six" ,python-six) ("python-dateutil" ,python-dateutil))) diff --git a/gnu/packages/python-build.scm b/gnu/packages/python-build.scm index 140629ca43..913d1b04ce 100644 --- a/gnu/packages/python-build.scm +++ b/gnu/packages/python-build.scm @@ -89,6 +89,52 @@ installed with a newer @code{pip} or with wheel's own command line utility.") Language (TOML) configuration files.") (license license:expat))) +(define-public python-pytoml + (package + (name "python-pytoml") + (version "0.1.21") + (source + (origin + (method url-fetch) + (uri (pypi-uri "pytoml" version)) + (sha256 + (base32 + "1rv1byiw82k7mj6aprcrqi2vdabs801y97xhfnrz7kxds34ggv4f")))) + (build-system python-build-system) + (home-page "https://github.com/avakar/pytoml") + (synopsis "Parser for TOML") + (description "This package provides a Python parser for TOML-0.4.0.") + (license license:expat))) + +(define-public python-flit-core + (package + (name "python-flit-core") + (version "3.2.0") + (source + (origin + (method url-fetch) + (uri (pypi-uri "flit-core" version)) + (sha256 + (base32 + "0cclv7v4cmzi457bzqsx9ig8ir1ha3ip8h1kx8qfy95wbmfg51zz")))) + (build-system python-build-system) + (arguments + `(#:phases + (modify-phases %standard-phases + ;; Requirements refer to a specific version of dependencies, + ;; which are too old. So we patch to refer to any later version. + (add-after 'unpack 'use-local-sourcetree + (lambda _ + (setenv "GUIX_PYTHONPATH" (string-append (getenv "GUIX_PYTHONPATH") ":" (getcwd)))))))) + (propagated-inputs + `(("python-toml" ,python-toml))) + (home-page "https://github.com/takluyver/flit") + (synopsis + "Distribution-building parts of Flit. See flit package for more information") + (description + "Distribution-building parts of Flit. See flit package for more information") + (license #f))) + (define-public python-pep517-bootstrap (hidden-package (package @@ -104,6 +150,7 @@ Language (TOML) configuration files.") (build-system python-build-system) (arguments `(#:tests? #f)) ;to avoid circular dependencies + (native-inputs `(("python-flit-core" ,python-flit-core))) (propagated-inputs `(("python-toml" ,python-toml) ("python-wheel" ,python-wheel))) diff --git a/gnu/packages/python-crypto.scm b/gnu/packages/python-crypto.scm index 39b194b25e..e40a60a9ca 100644 --- a/gnu/packages/python-crypto.scm +++ b/gnu/packages/python-crypto.scm @@ -124,7 +124,8 @@ with what is used by the Bitcoin network.") (build-system python-build-system) (native-inputs `(("python-pycparser" ,python-pycparser) - ("python-pytest" ,python-pytest))) + ("python-pytest" ,python-pytest) + ("python-wheel" ,python-wheel))) (propagated-inputs `(("python-cffi" ,python-cffi) ("python-six" ,python-six))) @@ -544,7 +545,8 @@ is used by the Requests library to verify HTTPS requests.") ("python-hypothesis" ,python-hypothesis) ("python-pretend" ,python-pretend) ("python-pytz" ,python-pytz) - ("python-pytest" ,python-pytest))) + ("python-pytest" ,python-pytest) + ("python-wheel" ,python-wheel))) (home-page "https://github.com/pyca/cryptography") (synopsis "Cryptographic recipes and primitives for Python") (description @@ -611,7 +613,8 @@ message digests and key derivation functions.") `(("libfaketime" ,libfaketime) ("python-flaky" ,python-flaky) ("python-pretend" ,python-pretend) - ("python-pytest" ,python-pytest))) + ("python-pytest" ,python-pytest) + ("python-wheel" ,python-wheel))) (home-page "https://github.com/pyca/pyopenssl") (synopsis "Python wrapper module around the OpenSSL library") (description diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm index 7a2b9eecc6..59dbe45a4f 100644 --- a/gnu/packages/python-xyz.scm +++ b/gnu/packages/python-xyz.scm @@ -2307,6 +2307,7 @@ and is not compatible with JSON.") (invoke "python" "bootstrap.py" "build/scons" "DEVELOPER=guix") (chdir "build/scons") #t))))) + (native-inputs `(("python-wheel" ,python-wheel))) (home-page "https://scons.org/") (synopsis "Software construction tool written in Python") (description @@ -3161,7 +3162,8 @@ e.g. filters, callbacks and errbacks can all be promises.") (native-inputs `(("python-mock" ,python-mock) ("python-pytest" ,python-pytest) - ("python-setuptools-scm" ,python-setuptools-scm))) + ("python-setuptools-scm" ,python-setuptools-scm) + ("python-wheel" ,python-wheel))) (propagated-inputs `(("python-appdirs" ,python-appdirs) ("python-distlib" ,python-distlib/next) @@ -6498,6 +6500,7 @@ child application and control it as if a human were typing commands.") (base32 "0ahlrxxkx2xhmxskx57gc96w3bdndflxx30304ihvm7ds136nny8")))) (build-system python-build-system) + (native-inputs `(("python-wheel" ,python-wheel))) (home-page "https://github.com/pypa/setuptools_scm/") (synopsis "Manage Python package versions in SCM metadata") (description @@ -8284,7 +8287,8 @@ PEP 8.") (native-inputs `(("python-mock" ,python-mock) ("python-pytest" ,python-pytest) - ("python-testpath" ,python-testpath))) + ("python-testpath" ,python-testpath) + ,@(package-native-inputs python-pep517-bootstrap))) (properties `((python2-variant . ,(delay python2-pep517)))))) ;; Skip the tests so we don't create a cyclical dependency with pytest. @@ -8752,6 +8756,7 @@ output.") (sha256 (base32 "1p9p7mn8x2j9psc4jxab98897v4i9s4fliyfw8rp8v4bx1n7pjj2")))) (build-system python-build-system) + (native-inputs `(("python-poetry-core" ,python-poetry-core))) (home-page "https://github.com/sdispater/crashtest") (synopsis "Manage Python errors with ease") (description @@ -9891,20 +9896,7 @@ Jupyter Notebook format and Python APIs for working with notebooks.") (base32 "0lc4si3xb7hza424414rdqdc3vng3kcrph8jbvjqb32spqddf3f7")))) (build-system python-build-system) - ;; The package does not come with a setup.py file, so we have to generate - ;; one ourselves. - (arguments - `(#:tests? #f - #:phases - (modify-phases %standard-phases - (add-after 'unpack 'create-setup.py - (lambda _ - (call-with-output-file "setup.py" - (lambda (port) - (format port "\ -from setuptools import setup -setup(name='entrypoints', version='~a', py_modules=['entrypoints']) -" ,version)))))))) + (native-inputs `(("python-flit" ,python-flit))) (home-page "https://github.com/takluyver/entrypoints") (synopsis "Discover and load entry points from installed Python packages") (description "Entry points are a way for Python packages to advertise @@ -11176,7 +11168,8 @@ library as well as on the command line.") "1c35qyhvy27q9ih9n899f3h4sdnpgq027dbiilly2qb5cvgarchm")))) (build-system python-build-system) (native-inputs - `(("python-setuptools-scm" ,python-setuptools-scm))) + `(("python-setuptools-scm" ,python-setuptools-scm) + ("python-wheel" ,python-wheel))) (synopsis "Plugin and hook calling mechanism for Python") (description "Pluggy is an extraction of the plugin manager as used by Pytest but stripped of Pytest specific details.") @@ -14657,7 +14650,8 @@ than during a preprocessing step).") "17xbrgi23l87yg6h0qcknssp2q812miiy33qw6v45v5gx0jwv5xh")))) (build-system python-build-system) (propagated-inputs - `(("python-setuptools-scm" ,python-setuptools-scm))) + `(("python-setuptools-scm" ,python-setuptools-scm) + ("python-wheel" ,python-wheel))) (home-page "https://github.com/marcelm/xopen/") (synopsis "Open compressed files transparently") (description "This module provides an @code{xopen} function that works like @@ -15194,7 +15188,8 @@ and bit flag values.") (lambda _ (invoke "pytest")))))) (native-inputs - `(("python-coverage" ,python-coverage) + `(("python-wheel" ,python-wheel) + ("python-coverage" ,python-coverage) ("python-hypothesis" ,python-hypothesis) ("python-pympler" ,python-pympler) ("python-pytest" ,python-pytest) @@ -15215,7 +15210,7 @@ protocols.") (package (inherit python-attrs) (name "python-attrs-bootstrap") - (native-inputs `()) + (native-inputs `(("python-wheel" ,python-wheel))) (arguments `(#:tests? #f)))) (define-public python2-attrs-bootstrap @@ -15606,6 +15601,7 @@ in other versions.") (base32 "0ckzngs3scaa1mcfmsi1w40a1l8cxxnncscrxzjjwjyisx8z0fmw")))) (build-system python-build-system) + (native-inputs `(("python-wheel" ,python-wheel))) (home-page "https://github.com/RonnyPfannschmidt/iniconfig") (synopsis "Simple INI-file parser") (description "The @code{iniconfig} package provides a small and simple @@ -20784,7 +20780,8 @@ register custom encoders and decoders.") (native-inputs `(("double-conversion" ,double-conversion) ("python-setuptools-scm" ,python-setuptools-scm) - ("python-pytest" ,python-pytest))) + ("python-pytest" ,python-pytest) + ("python-wheel" ,python-wheel))) (home-page "https://github.com/ultrajson/ultrajson") (synopsis "Ultra fast JSON encoder and decoder for Python") (description @@ -20980,6 +20977,11 @@ the syntactic logic to configure and launch jobs in an execution environment.") (build-system python-build-system) (arguments `(#:tests? #f)) ; XXX: Check requires network access. + (propagated-inputs + `(("python-docutils" ,python-docutils) + ("python-pytoml" ,python-pytoml) + ("python-requests" ,python-requests) + ("python-flit-core" ,python-flit-core))) (home-page "https://flit.readthedocs.io/") (synopsis "Simple packaging tool for simple packages") diff --git a/gnu/packages/time.scm b/gnu/packages/time.scm index 9b006a5438..60a0584202 100644 --- a/gnu/packages/time.scm +++ b/gnu/packages/time.scm @@ -46,6 +46,7 @@ #:use-module (gnu packages compression) #:use-module (gnu packages perl) #:use-module (gnu packages python) + #:use-module (gnu packages python-build) #:use-module (gnu packages python-xyz)) (define-public time @@ -194,7 +195,8 @@ Pendulum instances.") (native-inputs `(("python-pytest" ,python-pytest) ("python-pytest-cov" ,python-pytest-cov) - ("python-setuptools-scm" ,python-setuptools-scm))) + ("python-setuptools-scm" ,python-setuptools-scm) + ("python-wheel" ,python-wheel))) (propagated-inputs `(("python-six" ,python-six))) (home-page "https://dateutil.readthedocs.io/en/stable/") -- 2.31.1