From patchwork Sun Nov 15 12:39:11 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Lars-Dominik Braun X-Patchwork-Id: 25140 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 5E89527BBF5; Sun, 15 Nov 2020 12:41:13 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H4,RCVD_IN_MSPIKE_WL,SPF_HELO_PASS, T_DKIM_INVALID,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 14A4527BBF4 for ; Sun, 15 Nov 2020 12:41:11 +0000 (GMT) Received: from localhost ([::1]:34596 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1keHLC-0000RO-3Q for patchwork@mira.cbaines.net; Sun, 15 Nov 2020 07:41:10 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:37530) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1keHK6-0000Q2-62 for guix-patches@gnu.org; Sun, 15 Nov 2020 07:40:05 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:41163) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1keHK5-000495-SB for guix-patches@gnu.org; Sun, 15 Nov 2020 07:40:01 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1keHK5-0001Vb-Nt for guix-patches@gnu.org; Sun, 15 Nov 2020 07:40:01 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#44656] [PATCH] Upgrade pypy3 Resent-From: Lars-Dominik Braun Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sun, 15 Nov 2020 12:40:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 44656 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 44656@debbugs.gnu.org X-Debbugs-Original-To: guix-patches@gnu.org Received: via spool by submit@debbugs.gnu.org id=B.16054439835767 (code B ref -1); Sun, 15 Nov 2020 12:40:01 +0000 Received: (at submit) by debbugs.gnu.org; 15 Nov 2020 12:39:43 +0000 Received: from localhost ([127.0.0.1]:52709 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1keHJb-0001Ug-PB for submit@debbugs.gnu.org; Sun, 15 Nov 2020 07:39:43 -0500 Received: from lists.gnu.org ([209.51.188.17]:51586) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1keHJX-0001UX-H2 for submit@debbugs.gnu.org; Sun, 15 Nov 2020 07:39:30 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:37508) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1keHJX-0000MQ-9n for guix-patches@gnu.org; Sun, 15 Nov 2020 07:39:27 -0500 Received: from mout-p-202.mailbox.org ([80.241.56.172]:36728) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_CHACHA20_POLY1305:256) (Exim 4.90_1) (envelope-from ) id 1keHJS-0003xj-4X for guix-patches@gnu.org; Sun, 15 Nov 2020 07:39:26 -0500 Received: from smtp1.mailbox.org (smtp1.mailbox.org [IPv6:2001:67c:2050:105:465:1:1:0]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-384) server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mout-p-202.mailbox.org (Postfix) with ESMTPS id 4CYsGD653nzQkHd for ; Sun, 15 Nov 2020 13:39:16 +0100 (CET) X-Virus-Scanned: amavisd-new at heinlein-support.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6xq.net; s=MBO0001; t=1605443954; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=oyJIDXJI4qiuWp2UrmXpB4Y1dFtMBJ7E18KONefv5jM=; b=iliSyyIgYdZLLP1XSXwdB0F8Q1+VTffHOBaouE1aQ2N15DCVA9roj/2S31MatTZkAbkuwk pXfkuc3cpJHB4X7J+LLJmVR2n2BUd7HTTWx0KeL5ikuvj9NGxgmTZjqcg4tzPrDYA15vKj fHCKyHN5Y9fD4EBTDrp7/mxiCmuFc5D7ZTSxkaEnp4ed4DRarAOtUVbqSWOzraqSrCavMB ycwdb94aKLmQ/oWUKsYe86tH4zNSpV5Di8QxYDNLRcDOBwqqhxzkYGyFrLBS1ag1iIiN05 o70DpI6w9tyFfOK7oF+UJCmrrH4ZIAmapDRAHqft/xHwfCMBlBH8OHWikuCkNA== Received: from smtp1.mailbox.org ([80.241.60.240]) by spamfilter06.heinlein-hosting.de (spamfilter06.heinlein-hosting.de [80.241.56.125]) (amavisd-new, port 10030) with ESMTP id d69lETnppCky for ; Sun, 15 Nov 2020 13:39:12 +0100 (CET) Date: Sun, 15 Nov 2020 13:39:11 +0100 From: Lars-Dominik Braun Message-ID: <20201115123911.GA1956@noor.fritz.box> MIME-Version: 1.0 Content-Disposition: inline X-MBO-SPAM-Probability: X-Rspamd-Score: -10.75 / 15.00 / 15.00 X-Rspamd-Queue-Id: 9687E1831 X-Rspamd-UID: c28279 Received-SPF: pass client-ip=80.241.56.172; envelope-from=lars@6xq.net; helo=mout-p-202.mailbox.org X-detected-operating-system: by eggs.gnu.org: First seen = 2020/11/15 07:39:17 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches Hi, the first patch in this series upgrades pypy3 to 7.3.2 and the second adjusts several aspects of the package. Lars From caf5e4bf9acd10f91de2e6a9c60185475144a1b9 Mon Sep 17 00:00:00 2001 From: Lars-Dominik Braun Date: Sun, 15 Nov 2020 10:54:26 +0100 Subject: [PATCH 2/2] gnu: pypy3: Various package fixes. * gnu/packages/python.scm (pypy3) [patches]: Add new patch. [inputs]: Remove bash-minimal and add nss-certs. [native-inputs]: Remove nss-certs here. [arguments]: Use gdbm compat library, add 2to3 binary. [native-search-paths]: Add search path. * gnu/packages/patches/pypy3-7.3.1-ssl-paths.patch: New file. * gnu/local.mk: Add it. --- gnu/local.mk | 1 + .../patches/pypy3-7.3.1-ssl-paths.patch | 41 +++++++ gnu/packages/python.scm | 109 ++++++++++++------ 3 files changed, 116 insertions(+), 35 deletions(-) create mode 100644 gnu/packages/patches/pypy3-7.3.1-ssl-paths.patch diff --git a/gnu/local.mk b/gnu/local.mk index 91a3295e75..b644391a11 100644 --- a/gnu/local.mk +++ b/gnu/local.mk @@ -1536,6 +1536,7 @@ dist_patch_DATA = \ %D%/packages/patches/python-unittest2-remove-argparse.patch \ %D%/packages/patches/python-waitress-fix-tests.patch \ %D%/packages/patches/pypy3-7.3.1-fix-tests.patch \ + %D%/packages/patches/pypy3-7.3.1-ssl-paths.patch \ %D%/packages/patches/qemu-build-info-manual.patch \ %D%/packages/patches/qemu-glibc-2.27.patch \ %D%/packages/patches/qrcodegen-cpp-make-install.patch \ diff --git a/gnu/packages/patches/pypy3-7.3.1-ssl-paths.patch b/gnu/packages/patches/pypy3-7.3.1-ssl-paths.patch new file mode 100644 index 0000000000..d21133b4ae --- /dev/null +++ b/gnu/packages/patches/pypy3-7.3.1-ssl-paths.patch @@ -0,0 +1,41 @@ +Fix default certificate search path, still allowing the user to override it +with environment variables. + +--- a/lib_pypy/_cffi_ssl/_stdssl/__init__.py ++++ b/lib_pypy/_cffi_ssl/_stdssl/__init__.py +@@ -1679,20 +1679,9 @@ def get_default_verify_paths(): + https://golang.org/src/crypto/x509/root_linux.go (for the files) + ''' + certFiles = [ +- "/etc/ssl/certs/ca-certificates.crt", # Debian/Ubuntu/Gentoo etc. +- "/etc/pki/tls/certs/ca-bundle.crt", # Fedora/RHEL 6 +- "/etc/ssl/ca-bundle.pem", # OpenSUSE +- "/etc/pki/tls/cacert.pem", # OpenELEC +- "/etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem", # CentOS/RHEL 7 +- "/etc/ssl/cert.pem", # Alpine Linux + ] + certDirectories = [ +- "/etc/ssl/certs", # SLES10/SLES11 +- "/system/etc/security/cacerts", # Android +- "/usr/local/share/certs", # FreeBSD +- "/etc/pki/tls/certs", # Fedora/RHEL +- "/etc/openssl/certs", # NetBSD +- "/var/ssl/certs", # AIX ++ "@GUIX_CERT_PATH@", + ] + + # optimization: reuse the values from a local varaible +@@ -1707,9 +1696,10 @@ def get_default_verify_paths(): + ofile = _cstr_decode_fs(lib.X509_get_default_cert_file()) + odir = _cstr_decode_fs(lib.X509_get_default_cert_dir()) + +- if os.path.exists(ofile) and os.path.exists(odir): +- get_default_verify_paths.retval = (ofile_env, ofile, odir_env, odir) +- return get_default_verify_paths.retval ++ if not os.path.exists(ofile): ++ ofile = None ++ if not os.path.exists(odir): ++ odir = None + + # OpenSSL didn't supply the goods. Try some other options + for f in certFiles: diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm index 8ef8ae2e1d..c0bd3335e3 100644 --- a/gnu/packages/python.scm +++ b/gnu/packages/python.scm @@ -696,7 +696,8 @@ ease from the desktop to a microcontroller or embedded system.") (sha256 (base32 "03f1fdw6yk2mypa9pbmgk26r8y1hhmw801l6g36zry9zsvz7aqgx")) - (patches (search-patches "pypy3-7.3.1-fix-tests.patch")))) + (patches (search-patches "pypy3-7.3.1-fix-tests.patch" + "pypy3-7.3.1-ssl-paths.patch")))) (build-system gnu-build-system) (native-inputs `(("python-2" ,python-2) @@ -704,13 +705,13 @@ ease from the desktop to a microcontroller or embedded system.") ("tar" ,tar) ; Required for package.py ("python2-pycparser" ,python2-pycparser) ("python2-hypothesis" ,python2-hypothesis) - ("nss-certs" ,nss-certs) ; For ssl tests ("gzip" ,gzip))) (inputs `(("libffi" ,libffi) ("zlib" ,zlib) ("ncurses" ,ncurses) ("openssl" ,openssl) + ("nss-certs" ,nss-certs) ; For ssl module ("expat" ,expat) ("bzip2" ,bzip2) ("sqlite" ,sqlite) @@ -718,10 +719,9 @@ ease from the desktop to a microcontroller or embedded system.") ("tcl" ,tcl) ("tk" ,tk) ("glibc" ,glibc) - ("bash-minimal" ,bash-minimal) ; Used as /bin/sh ("xz" ,xz))) ; liblzma (arguments - `(#:tests? #f ;FIXME: Disabled for now, there are many tests failing. + `(#:tests? #f ;FIXME: Disabled for now, there are many tests failing. #:modules ((ice-9 ftw) (ice-9 match) (guix build utils) (guix build gnu-build-system)) #:phases (modify-phases %standard-phases @@ -750,6 +750,11 @@ ease from the desktop to a microcontroller or embedded system.") (substitute* '("lib_pypy/_curses_build.py") ;; Find curses (("/usr/local") (assoc-ref inputs "ncurses"))) + (substitute* '("lib_pypy/_dbm.py") + ;; Use gdbm compat library, so we don’t need to pull in bdb + (("ctypes.util.find_library\\('db'\\)") + (string-append "'" (assoc-ref inputs "gdbm") + "/lib/libgdbm_compat.so'"))) (substitute* '("lib_pypy/_sqlite3_build.py") ;; Always use search paths (("sys\\.platform\\.startswith\\('freebsd'\\)") "True") @@ -761,11 +766,18 @@ ease from the desktop to a microcontroller or embedded system.") "/lib/libsqlite3.so.0'"))) (substitute* '("lib-python/3/subprocess.py") ;; Fix shell path - (("/bin/sh") - (string-append (assoc-ref inputs "bash-minimal") "/bin/sh"))) + (("/bin/sh") (which "sh"))) (substitute* '("lib-python/3/distutils/unixccompiler.py") ;; gcc-toolchain does not provide symlink cc -> gcc (("\"cc\"") "\"gcc\"")) + (substitute* '("lib_pypy/_cffi_ssl/_stdssl/__init__.py") + ;; Add nss-certs to default certificate search path, + ;; otherwise every packages has to specify nss-certs and + ;; openssl as input to set the proper env variables. + ;; Depends on -ssl-paths.patch. + (("@GUIX_CERT_PATH@") + (string-append (assoc-ref inputs "nss-certs") + "/etc/ssl/certs"))) #t)) (add-after 'unpack 'set-source-file-times-to-1980 @@ -785,7 +797,8 @@ ease from the desktop to a microcontroller or embedded system.") (string-append "--make-jobs=" (number->string (parallel-job-count))) "-Ojit" - "targetpypystandalone")) + "targetpypystandalone" + "--allworkingmodules")) ;; Build c modules and package everything, so tests work. (with-directory-excursion "pypy/tool/release" (unsetenv "PYTHONPATH") ; Do not use the system’s python libs: @@ -793,7 +806,12 @@ ease from the desktop to a microcontroller or embedded system.") ; attribute 'IntFlag' (invoke "python2" "package.py" "--archive-name" "pypy-dist" - "--builddir" (getcwd))))) + "--builddir" (getcwd)) + ;; install pip and setuptools into the dist directory. + ;; XXX: Breaks virtualenv, because it does not set +w + ;; on files copied from the store. + ;(invoke "pypy-dist/bin/pypy3" "-m" "ensurepip") + ))) (replace 'check (lambda* (#:key tests? #:allow-other-keys) (if tests? @@ -811,32 +829,54 @@ ease from the desktop to a microcontroller or embedded system.") #t)) (replace 'install (lambda* (#:key inputs outputs #:allow-other-keys) - (with-directory-excursion "pypy/tool/release" - ;; Delete test data. - (for-each - (lambda (x) - (delete-file-recursively (string-append - "pypy-dist/lib-python/3/" x))) - '("tkinter/test" - "test" - "sqlite3/test" - "lib2to3/tests" - "idlelib/idle_test" - "distutils/tests" - "ctypes/test" - "unittest/test")) - ;; Patch shebang referencing python2 - (substitute* '("pypy-dist/lib-python/3/cgi.py" - "pypy-dist/lib-python/3/encodings/rot_13.py") - (("#!.+/bin/python") - (string-append "#!" (assoc-ref outputs "out") "/bin/pypy3"))) - (with-fluids ((%default-port-encoding "ISO-8859-1")) - (substitute* '("pypy-dist/lib_pypy/_md5.py" - "pypy-dist/lib_pypy/_sha1.py") - (("#!.+/bin/python") - (string-append "#!" (assoc-ref outputs "out") "/bin/pypy3")))) - (copy-recursively "pypy-dist" (assoc-ref outputs "out"))) - #t))))) + (let* ((out (assoc-ref outputs "out")) + (bin-pypy3 (string-append out "/bin/pypy3")) + (shebang-match-python "#!.+/bin/python") + (shebang-pypy3 (string-append "#!" bin-pypy3)) + (dist-dir "pypy/tool/release/pypy-dist")) + (with-directory-excursion dist-dir + ;; Delete test data. + (for-each + (lambda (x) + (delete-file-recursively (string-append + "lib-python/3/" x))) + '("tkinter/test" + "test" + "sqlite3/test" + "lib2to3/tests" + "idlelib/idle_test" + "distutils/tests" + "ctypes/test" + "unittest/test")) + ;; Patch shebang referencing python2 + (substitute* '("lib-python/3/cgi.py" + "lib-python/3/encodings/rot_13.py") + ((shebang-match-python) shebang-pypy3)) + (with-fluids ((%default-port-encoding "ISO-8859-1")) + (substitute* '("lib_pypy/_md5.py" + "lib_pypy/_sha1.py") + ((shebang-match-python) shebang-pypy3)))) + (copy-recursively dist-dir out) + ;; Make sure pypy3 is callable as python/python3, so we + ;; don’t have to patch every single package. + (symlink bin-pypy3 (string-append out "/bin/python")) + (symlink bin-pypy3 (string-append out "/bin/python3")) + ;; 2to3 is missing from pypy3, create it. + (let ((2to3 (string-append out "/bin/2to3"))) + (call-with-output-file 2to3 + (lambda (port) + (format port "#!~a~%" (string-append out "/bin/pypy3")) + (format port " +import sys +from lib2to3.main import main + +sys.exit(main('lib2to3.fixes'))"))) + (chmod 2to3 #o755)) + #t)))))) + (native-search-paths + (list (search-path-specification + (variable "PYTHONPATH") + (files '("lib/pypy3.6/site-packages"))))) (home-page "https://www.pypy.org/") (synopsis "Python implementation with just-in-time compilation") (description "PyPy is a faster, alternative implementation of the Python @@ -849,4 +889,3 @@ Python code natively, including C extensions.") license:bsd-3 ; lib_pypy/cffi/_pycparser/ply/ (license:non-copyleft "http://www.unicode.org/copyright.html"))))) - -- 2.26.2