| Message ID | 20191114084453.27042-1-tanguy@bioneland.org |
|---|---|
| State | Accepted |
| Headers | show |
| Series | [bug#38208] gnu: Add python-3.8. | expand |
Hello Tanguy, Tanguy Le Carrour <tanguy@bioneland.org> writes: > * gnu/packages/python.scm (python-3.8): New public variable. > [source]: Add patches to skip four tests. > * gnu/packages/patches/python-3.8-search-paths.patch: New file. > * gnu/packages/patches/python-3.8-fix-tests.patch: New file. Thank you for this! > ++ @unittest.skipIf(True, "This fails for unknown reasons on Guix") > + def test_keyboard_interrupt_exit_code(self): > + """KeyboardInterrupt triggers exit via SIGINT.""" > + process = subprocess.run( I'm fairly confident that these failures are because stdin is not a TTY. Can you try changing to this? @unittest.skipUnless(sys.stdin.isatty(), "KeyboardInterrupts require a TTY device") It would be good to submit a patch along these lines upstream if that works. > +(define-public python-3.8 > + (package > + (inherit python-3.7) > + (version "3.8.0") > + (source > + (origin > + (inherit (package-source python-3.7)) > + (uri (string-append "https://www.python.org/ftp/python/" > + version "/Python-" version ".tar.xz")) > + (sha256 (base32 "110d0did9rxn7rg85kf2fwli5hqq44xv2d8bi7d92m7v2d728mmk")) > + (patches (search-patches > + "python-3.8-search-paths.patch" > + "python-3-fix-tests.patch" > + "python-3.8-fix-tests.patch" > + "python-3-deterministic-build-info.patch")) > + (snippet > + '(begin > + ;; Delete the bundled copy of libexpat. > + (delete-file-recursively "Modules/expat") > + (substitute* "Modules/Setup" > + ;; Link Expat instead of embedding the bundled one. > + (("^#pyexpat.*") "pyexpat pyexpat.c -lexpat\n")) > + #t)))))) One final question... Do you know how stable the Python ABI is? Are packages compiled with 3.7 guaranteed to work with 3.8? I wonder if we should 1) rename it to 'python-next', and 2) override 'native-search-paths' so that it searches 'lib/python3.8/site-packages'. My concern is that e.g. 'guix environment --ad-hoc python python-numpy' might stop working unless we rename it, and/or adjust search paths. If Python 3.8 is fully backwards compatible, we probably don't have to do anything, otherwise we should adjust accordingly. Thoughts?
Hi Marius, Le 11/15, Marius Bakke a écrit : > Tanguy Le Carrour <tanguy@bioneland.org> writes: > > ++ @unittest.skipIf(True, "This fails for unknown reasons on Guix") > > + def test_keyboard_interrupt_exit_code(self): > > + """KeyboardInterrupt triggers exit via SIGINT.""" > > + process = subprocess.run( > > I'm fairly confident that these failures are because stdin is not a > TTY. Can you try changing to this? > > @unittest.skipUnless(sys.stdin.isatty(), "KeyboardInterrupts require a TTY device") > > It would be good to submit a patch along these lines upstream if that > works. Thanks for the snippet! It works! I've been trying for 2 days to register on https://bugs.python.org to submit this patch, but failed! :-( I'll keep on trying with different email addresses… Does this mean that we have to wait for upstream to integrate it, or can we use the patched wersion for Guix? Anyway, I'll update this patch according to your suggestion and push it. > > +(define-public python-3.8 > > + (package > > + (inherit python-3.7) > > + (version "3.8.0") > > + (source > > + (origin > > + (inherit (package-source python-3.7)) > > + (uri (string-append "https://www.python.org/ftp/python/" > > + version "/Python-" version ".tar.xz")) > > + (sha256 (base32 "110d0did9rxn7rg85kf2fwli5hqq44xv2d8bi7d92m7v2d728mmk")) > > + (patches (search-patches > > + "python-3.8-search-paths.patch" > > + "python-3-fix-tests.patch" > > + "python-3.8-fix-tests.patch" > > + "python-3-deterministic-build-info.patch")) > > + (snippet > > + '(begin > > + ;; Delete the bundled copy of libexpat. > > + (delete-file-recursively "Modules/expat") > > + (substitute* "Modules/Setup" > > + ;; Link Expat instead of embedding the bundled one. > > + (("^#pyexpat.*") "pyexpat pyexpat.c -lexpat\n")) > > + #t)))))) > > One final question... Do you know how stable the Python ABI is? Are > packages compiled with 3.7 guaranteed to work with 3.8? > > I wonder if we should 1) rename it to 'python-next', and 2) override > 'native-search-paths' so that it searches 'lib/python3.8/site-packages'. > > My concern is that e.g. 'guix environment --ad-hoc python python-numpy' > might stop working unless we rename it, and/or adjust search paths. > > If Python 3.8 is fully backwards compatible, we probably don't have to > do anything, otherwise we should adjust accordingly. I don't know anything about ABI, but what I read here: https://www.python.org/dev/peps/pep-0384/ So I guess it's been taken into consideration! I might have some time later this week to test using Python 3.8 as default Python. I'll let you know! Regards,
Patch submitted upstream: https://bugs.python.org/issue38845
Tanguy Le Carrour <tanguy@bioneland.org> writes: > Hi Marius, > > > Le 11/15, Marius Bakke a écrit : >> Tanguy Le Carrour <tanguy@bioneland.org> writes: >> > ++ @unittest.skipIf(True, "This fails for unknown reasons on Guix") >> > + def test_keyboard_interrupt_exit_code(self): >> > + """KeyboardInterrupt triggers exit via SIGINT.""" >> > + process = subprocess.run( >> >> I'm fairly confident that these failures are because stdin is not a >> TTY. Can you try changing to this? >> >> @unittest.skipUnless(sys.stdin.isatty(), "KeyboardInterrupts require a TTY device") >> >> It would be good to submit a patch along these lines upstream if that >> works. > > Thanks for the snippet! It works! > > I've been trying for 2 days to register on https://bugs.python.org to submit > this patch, but failed! :-( > I'll keep on trying with different email addresses… Excellent, thank you very much for taking care of this. There are five other tests skipped for the same reason in 'python-3-fix-tests.patch', can you include those in the upstream pull request? > Does this mean that we have to wait for upstream to integrate it, or can > we use the patched wersion for Guix? I think the patch looks OK, but looking forward to hear upstreams take on it. In any case, it should not be a blocker. > Anyway, I'll update this patch according to your suggestion and push it. Push it to this bug report, presumably? :-) >> One final question... Do you know how stable the Python ABI is? Are >> packages compiled with 3.7 guaranteed to work with 3.8? >> >> I wonder if we should 1) rename it to 'python-next', and 2) override >> 'native-search-paths' so that it searches 'lib/python3.8/site-packages'. >> >> My concern is that e.g. 'guix environment --ad-hoc python python-numpy' >> might stop working unless we rename it, and/or adjust search paths. >> >> If Python 3.8 is fully backwards compatible, we probably don't have to >> do anything, otherwise we should adjust accordingly. > > I don't know anything about ABI, but what I read here: > https://www.python.org/dev/peps/pep-0384/ > So I guess it's been taken into consideration! Cool, thanks for the link. So it is safe to swap out Python 3.7 with 3.8 "in-place". However I think we should still call it differently, because the new Python won't necessarily be compatible with all the manifests/environments out there that reference Python by name. So if you send a patch that adds (name "python-next") to the inherited Python package, I think we are all set. Thank you! > I might have some time later this week to test using Python 3.8 as > default Python. I'll let you know! Great, good luck! I suspect you'll have to update many of the 'core' Python packages along the way; feel free to drop by #guix on Freenode if you get stuck anywhere. :-)
diff --git a/gnu/packages/patches/python-3.8-fix-tests.patch b/gnu/packages/patches/python-3.8-fix-tests.patch new file mode 100644 index 0000000000..bea311992a --- /dev/null +++ b/gnu/packages/patches/python-3.8-fix-tests.patch @@ -0,0 +1,42 @@ +diff --git a/Lib/test/_test_multiprocessing.py b/Lib/test/_test_multiprocessing.py +index 1474624..887f8ee 100644 +--- a/Lib/test/_test_multiprocessing.py ++++ b/Lib/test/_test_multiprocessing.py +@@ -3801,6 +3801,7 @@ class _TestSharedMemory(BaseTestCase): + sms.close() + + @unittest.skipIf(os.name != "posix", "not feasible in non-posix platforms") ++ @unittest.skipIf(True, "This fails for unknown reasons on Guix") + def test_shared_memory_SharedMemoryServer_ignores_sigint(self): + # bpo-36368: protect SharedMemoryManager server process from + # KeyboardInterrupt signals. +diff --git a/Lib/test/test_signal.py b/Lib/test/test_signal.py +index d41e94b..a1c15e7 100644 +--- a/Lib/test/test_signal.py ++++ b/Lib/test/test_signal.py +@@ -78,6 +78,7 @@ class PosixTests(unittest.TestCase): + self.assertLess(len(s), signal.NSIG) + + @unittest.skipUnless(sys.executable, "sys.executable required.") ++ @unittest.skipIf(True, "This fails for unknown reasons on Guix") + def test_keyboard_interrupt_exit_code(self): + """KeyboardInterrupt triggers exit via SIGINT.""" + process = subprocess.run( +@@ -128,6 +129,7 @@ class WindowsSignalTests(unittest.TestCase): + signal.signal(7, handler) + + @unittest.skipUnless(sys.executable, "sys.executable required.") ++ @unittest.skipIf(True, "This fails for unknown reasons on Guix") + def test_keyboard_interrupt_exit_code(self): + """KeyboardInterrupt triggers an exit using STATUS_CONTROL_C_EXIT.""" + # We don't test via os.kill(os.getpid(), signal.CTRL_C_EVENT) here +@@ -1245,6 +1247,7 @@ class StressTest(unittest.TestCase): + + class RaiseSignalTest(unittest.TestCase): + ++ @unittest.skipIf(True, "This fails for unknown reasons on Guix") + def test_sigint(self): + with self.assertRaises(KeyboardInterrupt): + signal.raise_signal(signal.SIGINT) +-- +2.23.0 diff --git a/gnu/packages/patches/python-3.8-search-paths.patch b/gnu/packages/patches/python-3.8-search-paths.patch new file mode 100644 index 0000000000..22e102ccbe --- /dev/null +++ b/gnu/packages/patches/python-3.8-search-paths.patch @@ -0,0 +1,17 @@ +diff --git a/setup.py b/setup.py +index 20d7f35..5751083 100644 +--- a/setup.py ++++ b/setup.py +@@ -676,8 +676,8 @@ class PyBuildExt(build_ext): + # if a file is found in one of those directories, it can + # be assumed that no additional -I,-L directives are needed. + if not CROSS_COMPILING: +- self.lib_dirs = self.compiler.library_dirs + system_lib_dirs +- self.inc_dirs = self.compiler.include_dirs + system_include_dirs ++ self.lib_dirs = os.getenv('LIBRARY_PATH', '').split(os.pathsep) ++ self.inc_dirs = os.getenv('CPATH', '').split(os.pathsep) + else: + # Add the sysroot paths. 'sysroot' is a compiler option used to + # set the logical path of the standard system headers and +-- +2.23.0 diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm index 644c9d7666..8a2d3e9721 100644 --- a/gnu/packages/python.scm +++ b/gnu/packages/python.scm @@ -57,6 +57,7 @@ ;;; Copyright © 2018, 2019 Maxim Cournoyer <maxim.cournoyer@gmail.com> ;;; Copyright © 2018 Luther Thompson <lutheroto@gmail.com> ;;; Copyright © 2018 Vagrant Cascadian <vagrant@debian.org> +;;; Copyright © 2019 Tanguy Le Carrour <tanguy@bioneland.org> ;;; ;;; This file is part of GNU Guix. ;;; @@ -408,6 +409,30 @@ data types.") (version-major+minor version) "/site-packages")))))))) +(define-public python-3.8 + (package + (inherit python-3.7) + (version "3.8.0") + (source + (origin + (inherit (package-source python-3.7)) + (uri (string-append "https://www.python.org/ftp/python/" + version "/Python-" version ".tar.xz")) + (sha256 (base32 "110d0did9rxn7rg85kf2fwli5hqq44xv2d8bi7d92m7v2d728mmk")) + (patches (search-patches + "python-3.8-search-paths.patch" + "python-3-fix-tests.patch" + "python-3.8-fix-tests.patch" + "python-3-deterministic-build-info.patch")) + (snippet + '(begin + ;; Delete the bundled copy of libexpat. + (delete-file-recursively "Modules/expat") + (substitute* "Modules/Setup" + ;; Link Expat instead of embedding the bundled one. + (("^#pyexpat.*") "pyexpat pyexpat.c -lexpat\n")) + #t)))))) + ;; Current 3.x version. (define-public python-3 python-3.7)