From patchwork Wed Mar 6 12:55:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Tanguy LE CARROUR X-Patchwork-Id: 61408 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id D9EAA27BBE9; Wed, 6 Mar 2024 12:58:07 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,SPF_HELO_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id C113D27BBE2 for ; Wed, 6 Mar 2024 12:58:02 +0000 (GMT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rhqqB-0007mf-OR; Wed, 06 Mar 2024 07:57:49 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rhqpz-0007gZ-3F for guix-patches@gnu.org; Wed, 06 Mar 2024 07:57:35 -0500 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rhqpx-00081V-Ni; Wed, 06 Mar 2024 07:57:34 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1rhqqQ-0005Ww-Jn; Wed, 06 Mar 2024 07:58:02 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#60240] [PATCH v3] gnu: Add python-3.12 and python-next. References: <20221221153230.27764-1-tanguy@bioneland.org> In-Reply-To: <20221221153230.27764-1-tanguy@bioneland.org> Resent-From: Tanguy Le Carrour Original-Sender: "Debbugs-submit" Resent-CC: lars@6xq.net, marius@gnu.org, me@bonfacemunyoki.com, sharlatanus@gmail.com, jgart@dismail.de, guix-patches@gnu.org Resent-Date: Wed, 06 Mar 2024 12:58:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 60240 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 60240@debbugs.gnu.org Cc: Tanguy Le Carrour , Lars-Dominik Braun , Lars-Dominik Braun , Marius Bakke , Munyoki Kilyungi , Sharlatan Hellseher , jgart X-Debbugs-Original-Xcc: Lars-Dominik Braun , Marius Bakke , Munyoki Kilyungi , Sharlatan Hellseher , jgart Received: via spool by 60240-submit@debbugs.gnu.org id=B60240.170972983021188 (code B ref 60240); Wed, 06 Mar 2024 12:58:02 +0000 Received: (at 60240) by debbugs.gnu.org; 6 Mar 2024 12:57:10 +0000 Received: from localhost ([127.0.0.1]:49996 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rhqpZ-0005Vd-1f for submit@debbugs.gnu.org; Wed, 06 Mar 2024 07:57:10 -0500 Received: from smtp-8faf.mail.infomaniak.ch ([83.166.143.175]:47663) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rhqpT-0005Uz-Pw for 60240@debbugs.gnu.org; Wed, 06 Mar 2024 07:57:07 -0500 Received: from smtp-3-0001.mail.infomaniak.ch (smtp-3-0001.mail.infomaniak.ch [10.4.36.108]) by smtp-4-3000.mail.infomaniak.ch (Postfix) with ESMTPS id 4TqXVy4xRbzDgY; Wed, 6 Mar 2024 13:56:26 +0100 (CET) Received: from unknown by smtp-3-0001.mail.infomaniak.ch (Postfix) with ESMTPA id 4TqXVy0rYpzMpnPm; Wed, 6 Mar 2024 13:56:26 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=bioneland.org; s=20230804; t=1709729786; bh=TRVVzNb9G3OWD05r+fpQqHrH/OVW05QxGQVrrR2kqC8=; h=From:To:Cc:Subject:Date:From; b=HSwfLSB77CgMr6/pb3SgA6LacWt7Z0lBuv5l3BbDLzAO9W3U7hal6y/XvuxcX0Dam PZDbz09p06DDJGGRmHoQFRUidwgAznYuT4xmiHU3Wg8GPP6D5Y/xSOFFQbaZbTgwbl wTVFglgm8JOkEThdtfG/Y4BuYx9zYNeMAkn0lUHnpAdnKeaErp7ndm0rn/WTfMmCDn leBFhy8lVQ3agxb0KqYCjNXov+y7HN52j5BAIEHnaD2Q3DAXp/yEwbXKt05tw9W+1T 7cOcspYUy975slMggWsPO3ztRk2gscgvr6k5G1E/K40Cxi47hwyEe0le/VDKssKJwW MMalOhVgGl9ww== From: Tanguy Le Carrour Date: Wed, 6 Mar 2024 13:55:56 +0100 Message-ID: <796e983ade5fec88ff68f68fa30385aa0283839b.1709729756.git.tanguy@bioneland.org> X-Mailer: git-send-email 2.41.0 MIME-Version: 1.0 X-Infomaniak-Routing: alpha X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org X-getmail-retrieved-from-mailbox: Patches Patch v3 changes the name of the package to `python-next` and actually exclude Tcl/Tk from `out`. Thanks Lars! * gnu/packages/python.scm (python-3.12, python-next): New variables. * gnu/packages/patches/python-3.12-fix-tests.patch: New file. Change-Id: Ie393b732a8863569578e72e62603b75a1655a34e --- .../patches/python-3.12-fix-tests.patch | 334 +++++++++++++++ gnu/packages/python.scm | 385 +++++++++++++++++- 2 files changed, 718 insertions(+), 1 deletion(-) create mode 100644 gnu/packages/patches/python-3.12-fix-tests.patch base-commit: f3ea06baca2b96cd20170616fd5da13ec2daac54 diff --git a/gnu/packages/patches/python-3.12-fix-tests.patch b/gnu/packages/patches/python-3.12-fix-tests.patch new file mode 100644 index 0000000000..fa5c8027ce --- /dev/null +++ b/gnu/packages/patches/python-3.12-fix-tests.patch @@ -0,0 +1,334 @@ +From f0698133e7d6c353a3e6ae0fc62e57ba558a9bc0 Mon Sep 17 00:00:00 2001 +From: Maxim Cournoyer +Date: Wed, 28 Oct 2020 22:55:05 -0400 +Subject: [PATCH] Skip problematic Python 3 tests in Guix. + +A subset of the hunks in this patch is tracked upstream at +https://bugs.python.org/issue38845, which was contributed by Tanguy Le +Carrour . + +diff --git a/Lib/test/_test_multiprocessing.py b/Lib/test/_test_multiprocessing.py +index e42c7ab4bd..8087c84dab 100644 +--- a/Lib/test/_test_multiprocessing.py ++++ b/Lib/test/_test_multiprocessing.py +@@ -1695,6 +1695,7 @@ def _test_wait_result(cls, c, pid): + if pid is not None: + os.kill(pid, signal.SIGINT) + ++ @unittest.skipIf(True, "This fails for unknown reasons on Guix") + def test_wait_result(self): + if isinstance(self, ProcessesMixin) and sys.platform != 'win32': + pid = os.getpid() +@@ -4150,6 +4151,7 @@ def test_shared_memory_across_processes(self): + sms.close() + + @unittest.skipIf(os.name != "posix", "not feasible in non-posix platforms") ++ @unittest.skipUnless(sys.stdin.isatty(), "KeyboardInterrupts require a TTY device") + def test_shared_memory_SharedMemoryServer_ignores_sigint(self): + # bpo-36368: protect SharedMemoryManager server process from + # KeyboardInterrupt signals. +diff --git a/Lib/test/test_asyncio/test_base_events.py b/Lib/test/test_asyncio/test_base_events.py +index 85c8152d49..e35cfffe84 100644 +--- a/Lib/test/test_asyncio/test_base_events.py ++++ b/Lib/test/test_asyncio/test_base_events.py +@@ -1377,6 +1377,8 @@ def test_create_connection_no_inet_pton(self, m_socket): + self._test_create_connection_ip_addr(m_socket, False) + + @patch_socket ++ @unittest.skipUnless(support.is_resource_enabled('network'), ++ 'network is not enabled') + def test_create_connection_service_name(self, m_socket): + m_socket.getaddrinfo = socket.getaddrinfo + sock = m_socket.socket.return_value +diff --git a/Lib/test/test_ctypes/test_find.py b/Lib/test/test_ctypes/test_find.py +index a41e94971d..1291af3057 100644 +--- a/Lib/test/test_ctypes/test_find.py ++++ b/Lib/test/test_ctypes/test_find.py +@@ -117,6 +117,7 @@ def test_find_library_with_gcc(self): + with unittest.mock.patch("ctypes.util._findSoname_ldconfig", lambda *args: None): + self.assertNotEqual(find_library('c'), None) + ++ @unittest.skipIf(True, 'Fails on Guix.') + def test_find_library_with_ld(self): + with unittest.mock.patch("ctypes.util._findSoname_ldconfig", lambda *args: None), \ + unittest.mock.patch("ctypes.util._findLib_gcc", lambda *args: None): +diff --git a/Lib/test/test_generators.py b/Lib/test/test_generators.py +index 1ee9958445..ab6b41befe 100644 +--- a/Lib/test/test_generators.py ++++ b/Lib/test/test_generators.py +@@ -34,6 +34,7 @@ def generator2(self): + else: + return "FAILED" + ++ @unittest.skipIf(True, 'Keyboard interrupts do not work in the Guix build environment') + def test_raise_and_yield_from(self): + gen = self.generator1() + gen.send(None) +diff --git a/Lib/test/test_pathlib.py b/Lib/test/test_pathlib.py +index ec105ae1a0..ae4c5b672e 100644 +--- a/Lib/test/test_pathlib.py ++++ b/Lib/test/test_pathlib.py +@@ -3069,6 +3069,7 @@ def test_rglob(self): + 'pwd module does not expose getpwall()') + @unittest.skipIf(sys.platform == "vxworks", + "no home directory on VxWorks") ++ @unittest.skipIf(True, "Guix builder home is '/' which causes trouble for these tests") + def test_expanduser(self): + P = self.cls + import_helper.import_module('pwd') +diff --git a/Lib/test/test_pdb.py b/Lib/test/test_pdb.py +index 51b844262e..004d3133cf 100644 +--- a/Lib/test/test_pdb.py ++++ b/Lib/test/test_pdb.py +@@ -1580,6 +1580,7 @@ def test_pdb_next_command_subiterator(): + (Pdb) continue + """ + ++@unittest.skipIf(True, 'Fails on Guix… but skipIf not taken into account?!') + def test_pdb_issue_20766(): + """Test for reference leaks when the SIGINT handler is set. + +@@ -1598,11 +1599,11 @@ def test_pdb_issue_20766(): + > (6)test_function() + -> print('pdb %d: %s' % (i, sess._previous_sigint_handler)) + (Pdb) continue +- pdb 1: ++ pdb 1: 1 + > (6)test_function() + -> print('pdb %d: %s' % (i, sess._previous_sigint_handler)) + (Pdb) continue +- pdb 2: ++ pdb 2: 1 + """ + + def test_pdb_issue_43318(): +diff --git a/Lib/test/test_regrtest.py b/Lib/test/test_regrtest.py +index 2ab6f6a986..8cf6b4d1c8 100644 +--- a/Lib/test/test_regrtest.py ++++ b/Lib/test/test_regrtest.py +@@ -1049,6 +1049,7 @@ def test_fromfile(self): + output = self.run_tests('--fromfile', filename) + self.check_executed_tests(output, tests, stats=stats) + ++ @unittest.skipIf(True, 'Keyboard interrupts do not work in the Guix build environment.') + def test_interrupted(self): + code = TEST_INTERRUPTED + test = self.create_test('sigint', code=code) +@@ -1066,6 +1067,7 @@ def test_slowest(self): + % (self.TESTNAME_REGEX, len(tests))) + self.check_line(output, regex) + ++ @unittest.skipIf(True, 'Keyboard interrupts do not work in the Guix build environment.') + def test_slowest_interrupted(self): + # Issue #25373: test --slowest with an interrupted test + code = TEST_INTERRUPTED +diff --git a/Lib/test/test_resource.py b/Lib/test/test_resource.py +index 317e7ca8f8..7f272daf24 100644 +--- a/Lib/test/test_resource.py ++++ b/Lib/test/test_resource.py +@@ -151,6 +151,7 @@ def test_freebsd_contants(self): + + @unittest.skipUnless(hasattr(resource, 'prlimit'), 'no prlimit') + @support.requires_linux_version(2, 6, 36) ++ @unittest.skipIf(True, "Bug: the PermissionError is not raised") + def test_prlimit(self): + self.assertRaises(TypeError, resource.prlimit) + self.assertRaises(ProcessLookupError, resource.prlimit, +diff --git a/Lib/test/test_shutil.py b/Lib/test/test_shutil.py +index bf60f37934..5e3a96380a 100644 +--- a/Lib/test/test_shutil.py ++++ b/Lib/test/test_shutil.py +@@ -1743,6 +1743,7 @@ def test_make_archive(self): + base_name = os.path.join(tmpdir, 'archive') + self.assertRaises(ValueError, make_archive, base_name, 'xxx') + ++ @unittest.skipIf(True, "The Guix build container has no root user") + @support.requires_zlib() + def test_make_archive_owner_group(self): + # testing make_archive with owner and group, with various combinations +@@ -1771,6 +1772,7 @@ def test_make_archive_owner_group(self): + self.assertTrue(os.path.isfile(res)) + + ++ @unittest.skipIf(True, "The Guix build container has no root user") + @support.requires_zlib() + @unittest.skipUnless(UID_GID_SUPPORT, "Requires grp and pwd support") + def test_tarfile_root_owner(self): +diff --git a/Lib/test/test_signal.py b/Lib/test/test_signal.py +index 637a0ca3b3..2fb804e340 100644 +--- a/Lib/test/test_signal.py ++++ b/Lib/test/test_signal.py +@@ -160,6 +160,7 @@ def test_valid_signals(self): + self.assertLess(signum, signal.NSIG) + + @unittest.skipUnless(sys.executable, "sys.executable required.") ++ @unittest.skipUnless(sys.stdin.isatty(), "KeyboardInterrupts require a TTY device") + @support.requires_subprocess() + def test_keyboard_interrupt_exit_code(self): + """KeyboardInterrupt triggers exit via SIGINT.""" +@@ -211,6 +212,7 @@ def test_issue9324(self): + signal.signal(7, handler) + + @unittest.skipUnless(sys.executable, "sys.executable required.") ++ @unittest.skipUnless(sys.stdin.isatty(), "KeyboardInterrupts require a TTY device") + @support.requires_subprocess() + def test_keyboard_interrupt_exit_code(self): + """KeyboardInterrupt triggers an exit using STATUS_CONTROL_C_EXIT.""" +@@ -1407,6 +1409,7 @@ def cycle_handlers(): + + class RaiseSignalTest(unittest.TestCase): + ++ @unittest.skipUnless(sys.stdin.isatty(), "KeyboardInterrupts require a TTY device") + def test_sigint(self): + with self.assertRaises(KeyboardInterrupt): + signal.raise_signal(signal.SIGINT) +@@ -1452,6 +1455,7 @@ def __del__(self): + + class PidfdSignalTest(unittest.TestCase): + ++ @unittest.skipUnless(sys.stdin.isatty(), "KeyboardInterrupts require a TTY device") + @unittest.skipUnless( + hasattr(signal, "pidfd_send_signal"), + "pidfd support not built in", +diff --git a/Lib/test/test_socket.py b/Lib/test/test_socket.py +index 4eb5af99d6..9c7b8f6dbc 100644 +--- a/Lib/test/test_socket.py ++++ b/Lib/test/test_socket.py +@@ -1016,6 +1016,8 @@ def testHostnameRes(self): + if not fqhn in all_host_names: + self.fail("Error testing host resolution mechanisms. (fqdn: %s, all: %s)" % (fqhn, repr(all_host_names))) + ++ @unittest.skipUnless(support.is_resource_enabled('network'), ++ 'network is not enabled') + def test_host_resolution(self): + for addr in [socket_helper.HOSTv4, '10.0.0.1', '255.255.255.255']: + self.assertEqual(socket.gethostbyname(addr), addr) +@@ -1161,6 +1163,8 @@ def testNtoHErrors(self): + self.assertRaises(OverflowError, socket.ntohl, k) + self.assertRaises(OverflowError, socket.htonl, k) + ++ @unittest.skipUnless(os.path.exists("/etc/services"), ++ "getservbyname uses /etc/services, which is not in the chroot") + def testGetServBy(self): + eq = self.assertEqual + # Find one service that exists, then check all the related interfaces. +@@ -1521,6 +1525,8 @@ def test_sio_loopback_fast_path(self): + raise + self.assertRaises(TypeError, s.ioctl, socket.SIO_LOOPBACK_FAST_PATH, None) + ++ @unittest.skipUnless(os.path.exists("/etc/gai.conf"), ++ "getaddrinfo() will fail") + def testGetaddrinfo(self): + try: + socket.getaddrinfo('localhost', 80) +@@ -1653,6 +1659,8 @@ def test_getnameinfo(self): + # only IP addresses are allowed + self.assertRaises(OSError, socket.getnameinfo, ('mail.python.org',0), 0) + ++ @unittest.skipUnless(os.path.exists("/etc/gai.conf"), ++ "getaddrinfo() will fail") + @unittest.skipUnless(support.is_resource_enabled('network'), + 'network is not enabled') + def test_idna(self): +diff --git a/Lib/test/test_spwd.py b/Lib/test/test_spwd.py +index 50766c2548..0c7eb7a83a 100644 +--- a/Lib/test/test_spwd.py ++++ b/Lib/test/test_spwd.py +@@ -9,8 +9,7 @@ + spwd = import_helper.import_module('spwd') + + +-@unittest.skipUnless(hasattr(os, 'geteuid') and os.geteuid() == 0, +- 'root privileges required') ++@unittest.skipUnless(os.path.exists("/etc/shadow"), 'spwd tests require /etc/shadow') + class TestSpwdRoot(unittest.TestCase): + + def test_getspall(self): +@@ -60,8 +59,7 @@ def test_getspnam(self): + self.assertRaises(TypeError, spwd.getspnam, bytes_name) + + +-@unittest.skipUnless(hasattr(os, 'geteuid') and os.geteuid() != 0, +- 'non-root user required') ++@unittest.skipUnless(os.path.exists("/etc/shadow"), 'spwd tests require /etc/shadow') + class TestSpwdNonRoot(unittest.TestCase): + + def test_getspnam_exception(self): +diff --git a/Lib/test/test_tarfile.py b/Lib/test/test_tarfile.py +index 71489ea493..33351919fe 100644 +--- a/Lib/test/test_tarfile.py ++++ b/Lib/test/test_tarfile.py +@@ -2911,9 +2911,12 @@ def root_is_uid_gid_0(): + import pwd, grp + except ImportError: + return False +- if pwd.getpwuid(0)[0] != 'root': +- return False +- if grp.getgrgid(0)[0] != 'root': ++ try: ++ if pwd.getpwuid(0)[0] != 'root': ++ return False ++ if grp.getgrgid(0)[0] != 'root': ++ return False ++ except KeyError: + return False + return True + +diff --git a/Lib/test/test_threading.py b/Lib/test/test_threading.py +index 00d9e591c7..2515603715 100644 +--- a/Lib/test/test_threading.py ++++ b/Lib/test/test_threading.py +@@ -1962,6 +1962,7 @@ def check_interrupt_main_noerror(self, signum): + # Restore original handler + signal.signal(signum, handler) + ++ @unittest.skipIf(True, 'Keyboard interrupts do not work in the Guix build container.') + def test_interrupt_main_subthread(self): + # Calling start_new_thread with a function that executes interrupt_main + # should raise KeyboardInterrupt upon completion. +@@ -1973,6 +1974,8 @@ def call_interrupt(): + t.join() + t.join() + ++ ++ @unittest.skipIf(True, 'Keyboard interrupts do not work in the Guix build container.') + def test_interrupt_main_mainthread(self): + # Make sure that if interrupt_main is called in main thread that + # KeyboardInterrupt is raised instantly. +diff --git a/Lib/test/test_tools/test_freeze.py b/Lib/test/test_tools/test_freeze.py +index 0e7ed67de7..6539a2983b 100644 +--- a/Lib/test/test_tools/test_freeze.py ++++ b/Lib/test/test_tools/test_freeze.py +@@ -23,6 +23,7 @@ + 'test is too slow with PGO') + class TestFreeze(unittest.TestCase): + ++ @unittest.skipIf(True, 'Fails on Guix.') + @support.requires_resource('cpu') # Building Python is slow + def test_freeze_simple_script(self): + script = textwrap.dedent(""" +diff --git a/Lib/test/test_unicodedata.py b/Lib/test/test_unicodedata.py +index 515c3840cb..a96dfad0fe 100644 +--- a/Lib/test/test_unicodedata.py ++++ b/Lib/test/test_unicodedata.py +@@ -342,6 +342,7 @@ def test_linebreak_7643(self): + self.assertEqual(len(lines), 1, + r"\u%.4x should not be a linebreak" % i) + ++@requires_resource('network') + class NormalizationTest(unittest.TestCase): + @staticmethod + def check_version(testfile): +diff --git a/Tools/scripts/run_tests.py b/Tools/scripts/run_tests.py +index 445a34ae3e..8f750537c3 100644 +--- a/Tools/scripts/run_tests.py ++++ b/Tools/scripts/run_tests.py +@@ -69,7 +69,7 @@ def main(regrtest_args): + else: + args.extend(['-j', '0']) # Use all CPU cores + if not any(is_resource_use_flag(arg) for arg in regrtest_args): +- args.extend(['-u', 'all,-largefile,-audio,-gui']) ++ args.extend(['-u', 'all,-largefile,-audio,-gui,-network']) + + if cross_compile and hostrunner: + # If HOSTRUNNER is set and -p/--python option is not given, then diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm index 51d5f598d7..25fe83ea69 100644 --- a/gnu/packages/python.scm +++ b/gnu/packages/python.scm @@ -55,7 +55,7 @@ ;;; Copyright © 2018, 2019, 2020, 2021 Maxim Cournoyer ;;; Copyright © 2018 Luther Thompson ;;; Copyright © 2018 Vagrant Cascadian -;;; Copyright © 2019 Tanguy Le Carrour +;;; Copyright © 2019, 2024 Tanguy Le Carrour ;;; Copyright © 2020, 2023 Janneke Nieuwenhuizen ;;; Copyright © 2020, 2021 Greg Hogan ;;; Copyright © 2022 Philip McGrath @@ -596,6 +596,389 @@ (define-public python-3.10 (variable "PYTHONTZPATH") (files (list "share/zoneinfo"))))))) +(define-public python-3.12 + (package + (name "python-next") + (version "3.12.2") + (source + (origin + (method url-fetch) + (uri (string-append "https://www.python.org/ftp/python/" version + "/Python-" version ".tar.xz")) + (sha256 + (base32 "0w6qyfhc912xxav9x9pifwca40b4l49vy52wai9j0gc1mhni2a5y")) + (patches (search-patches "python-3-deterministic-build-info.patch" + "python-3.12-fix-tests.patch" + "python-3-hurd-configure.patch")) + (modules '((guix build utils))) + (snippet '(begin + ;; Delete the bundled copy of libexpat. + (delete-file-recursively "Modules/expat") + (substitute* "Modules/Setup" + ;; Link Expat instead of embedding the bundled one. + (("^#pyexpat.*") + "pyexpat pyexpat.c -lexpat\n")) + ;; Delete windows binaries + (for-each delete-file + (find-files "Lib/distutils/command" "\\.exe$")))))) + (outputs '("out" "tk" ;tkinter; adds 50 MiB to the closure + "idle")) ;programming environment; weighs 5MB + (build-system gnu-build-system) + (arguments + `(#:test-target "test" + #:configure-flags (list "--enable-shared" ;allow embedding + "--with-system-expat" ;for XML support + "--with-system-ffi" ;build ctypes + "--with-ensurepip=install" ;install pip and setuptools + "--with-computed-gotos" ;main interpreter loop optimization + "--enable-unicode=ucs4" + "--without-static-libpython" + + ;; FIXME: These flags makes Python significantly faster, but + ;; leads to non-reproducible binaries. + ;; "--with-lto" ;increase size by 20MB, but 15% speedup + ;; "--enable-optimizations" + + ;; Prevent the installed _sysconfigdata.py from retaining a reference + ;; to coreutils. + "INSTALL=install -c" + "MKDIR_P=mkdir -p" + + ;; Disable runtime check failing if cross-compiling, see: + ;; https://lists.yoctoproject.org/pipermail/poky/2013-June/008997.html + ,@(if (%current-target-system) + '("ac_cv_buggy_getaddrinfo=no" + "ac_cv_file__dev_ptmx=no" + "ac_cv_file__dev_ptc=no") + '()) + ;; -fno-semantic-interposition reinstates some optimizations by gcc + ;; leading to around 15% speedup. This is the default starting from + ;; python 3.10. + "CFLAGS=-fno-semantic-interposition" + (string-append "LDFLAGS=-Wl,-rpath=" + (assoc-ref %outputs "out") + "/lib" + " -fno-semantic-interposition")) + ;; With no -j argument tests use all available cpus, so provide one. + #:make-flags (list (string-append (format #f "TESTOPTS=-j~d" + (parallel-job-count)) + ;; those tests fail on low-memory systems + " --exclude test_mmap test_socket test_threading test_asyncio" + ,@(if (system-hurd?) + '(" test_posix" ;multiple errors + " test_time" + " test_pty" + " test_shutil" + " test_tempfile" ;chflags: invalid argument: + ;; tbv14c9t/dir0/dir0/dir0/test0.txt + " test_os" ;stty: 'standard input': + ;; Inappropriate ioctl for device + " test_openpty" ;No such file or directory + " test_selectors" ;assertEqual(NUM_FDS // 2, len(fds)) + ;; 32752 != 4 + " test_compileall" ;multiple errors + " test_poll" ;list index out of range + " test_subprocess" ;runs over 10min + " test_asyncore" ;multiple errors + " test_threadsignals" + " test_eintr" ;Process return code is -14 + " test_io" ;multiple errors + " test_logging" + " test_signal" + " test_flags" ;ERROR + " test_bidirectional_pty" + " test_create_unix_connection" + " test_unix_sock_client_ops" + " test_open_unix_connection" + " test_open_unix_connection_error" + " test_read_pty_output" + " test_write_pty" + " test_concurrent_futures" ;freeze + " test_venv" ;freeze + " test_multiprocessing_forkserver" ;runs over 10min + " test_multiprocessing_spawn" ;runs over 10min + " test_builtin" + " test_capi" + " test_dbm_ndbm" + " test_exceptions" + " test_faulthandler" + " test_getopt" + " test_importlib" + " test_json" + " test_multiprocessing_fork" + " test_multiprocessing_main_handling" + " test_pdb " + " test_regrtest" + " test_sqlite") + '()))) + + #:modules ((ice-9 ftw) + (ice-9 match) + (guix build utils) + (guix build gnu-build-system)) + + #:phases (modify-phases %standard-phases + ,@(if (system-hurd?) + `((add-after 'unpack + 'disable-multi-processing + (lambda _ + (substitute* "Makefile.pre.in" + (("-j0") + "-j1"))))) + '()) + (add-before 'configure 'patch-lib-shells + (lambda _ + ;; This variable is used in setup.py to enable cross compilation + ;; specific switches. As it is not set properly by configure + ;; script, set it manually. + ,@(if (%current-target-system) + '((setenv "_PYTHON_HOST_PLATFORM" "")) + '()) + ;; Filter for existing files, since some may not exist in all + ;; versions of python that are built with this recipe. + (substitute* (filter file-exists? + '("Lib/subprocess.py" + "Lib/popen2.py" + "Lib/distutils/tests/test_spawn.py" + "Lib/test/support/__init__.py" + "Lib/test/test_subprocess.py")) + (("/bin/sh") + (which "sh"))))) + (add-before 'configure 'do-not-record-configure-flags + (lambda* (#:key configure-flags #:allow-other-keys) + ;; Remove configure flags from the installed '_sysconfigdata.py' + ;; and 'Makefile' so we don't end up keeping references to the + ;; build tools. + ;; + ;; Preserve at least '--with-system-ffi' since otherwise the + ;; thing tries to build libffi, fails, and we end up with a + ;; Python that lacks ctypes. + (substitute* "configure" + (("^CONFIG_ARGS=.*$") + (format #f "CONFIG_ARGS='~a'\n" + (if (member "--with-system-ffi" + configure-flags) + "--with-system-ffi" "")))))) + (add-before 'check 'pre-check + (lambda _ + ;; 'Lib/test/test_site.py' needs a valid $HOME + (setenv "HOME" + (getcwd)))) + (add-after 'unpack 'set-source-file-times-to-1980 + ;; XXX One of the tests uses a ZIP library to pack up some of the + ;; source tree, and fails with "ZIP does not support timestamps + ;; before 1980". Work around this by setting the file times in the + ;; source tree to sometime in early 1980. + (lambda _ + (let ((circa-1980 (* 10 366 24 60 60))) + (ftw "." + (lambda (file stat flag) + (utime file circa-1980 circa-1980) #t))))) + (add-after 'unpack 'remove-windows-binaries + (lambda _ + ;; Delete .exe from embedded .whl (zip) files + (for-each (lambda (whl) + (let ((dir "whl-content") + (circa-1980 (* 10 366 24 60 60))) + (mkdir-p dir) + (with-directory-excursion dir + (let ((whl (string-append "../" whl))) + (invoke "unzip" whl) + (for-each delete-file + (find-files "." "\\.exe$")) + (delete-file whl) + ;; Reset timestamps to prevent them from ending + ;; up in the Zip archive. + (ftw "." + (lambda (file stat flag) + (utime file circa-1980 + circa-1980) #t)) + (apply invoke "zip" "-X" whl + (find-files "." + #:directories? #t)))) + (delete-file-recursively dir))) + (find-files "Lib/ensurepip" "\\.whl$")))) + (add-after 'install 'remove-tests + ;; Remove 25 MiB of unneeded unit tests. Keep test_support.* + ;; because these files are used by some libraries out there. + (lambda* (#:key outputs #:allow-other-keys) + (let ((out (assoc-ref outputs "out"))) + (match (scandir (string-append out "/lib") + (lambda (name) + (string-prefix? "python" name))) + ((pythonX.Y) + (let ((testdir (string-append out "/lib/" pythonX.Y + "/test"))) + (with-directory-excursion testdir + (for-each delete-file-recursively + (scandir testdir + (match-lambda + ((or "." "..") + #f) + ("support" #f) + (file (not (string-prefix? + "test_support." + file)))))) + (call-with-output-file "__init__.py" + (const #t)))) + (let ((libdir (string-append out "/lib/" pythonX.Y))) + (for-each (lambda (directory) + (let ((dir (string-append libdir "/" + directory))) + (when (file-exists? dir) + (delete-file-recursively dir)))) + '("email/test" "ctypes/test" + "unittest/test" + "tkinter/test" + "sqlite3/test" + "bsddb/test" + "lib-tk/test" + "json/tests" + "distutils/tests")))))))) + (add-after 'remove-tests 'move-tk-inter + (lambda* (#:key outputs inputs #:allow-other-keys) + ;; When Tkinter support is built move it to a separate output so + ;; that the main output doesn't contain a reference to Tcl/Tk. + (let ((out (assoc-ref outputs "out")) + (tk (assoc-ref outputs "tk"))) + (when tk + (match (find-files out "tkinter.*\\.so") + ((tkinter.so) + ;; The .so is in OUT/lib/pythonX.Y/lib-dynload, but we + ;; want it under TK/lib/pythonX.Y/site-packages. + (let* ((len (string-length out)) + (target (string-append tk "/" + (string-drop (dirname + (dirname + tkinter.so)) + len) + "/site-packages"))) + (install-file tkinter.so target) + (delete-file tkinter.so)))) + ;; Remove explicit store path references. + (let ((tcl (assoc-ref inputs "tcl")) + (tk (assoc-ref inputs "tk"))) + (substitute* (find-files (string-append out "/lib") + "^(_sysconfigdata_.*\\.py|Makefile)$") + (((string-append "-L" tk "/lib")) "") + (((string-append "-L" tcl "/lib")) ""))))))) + (add-after 'move-tk-inter 'move-idle + (lambda* (#:key outputs #:allow-other-keys) + ;; when idle is built, move it to a separate output to save some + ;; space (5MB) + (let ((out (assoc-ref outputs "out")) + (idle (assoc-ref outputs "idle"))) + (when idle + (for-each (lambda (file) + (let ((target (string-append idle + "/bin/" + (basename + file)))) + (install-file file + (dirname target)) + (delete-file file))) + (find-files (string-append out "/bin") + "^idle")) + (match (find-files out "^idlelib$" + #:directories? #t) + ((idlelib) + (let* ((len (string-length out)) + (target (string-append idle "/" + (string-drop + idlelib len) + "/site-packages"))) + (mkdir-p (dirname target)) + (rename-file idlelib target)))))))) + (add-after 'move-idle 'rebuild-bytecode + (lambda* (#:key outputs #:allow-other-keys) + (let ((out (assoc-ref outputs "out"))) + ;; Disable hash randomization to ensure the generated .pycs + ;; are reproducible. + (setenv "PYTHONHASHSEED" "0") + + (for-each (lambda (output) + ;; XXX: Delete existing pycs generated by the build + ;; system beforehand because the -f argument does + ;; not necessarily overwrite all files, leading to + ;; indeterministic results. + (for-each (lambda (pyc) + (delete-file pyc)) + (find-files output "\\.pyc$")) + + (apply invoke + `(,,(if (%current-target-system) + "python3" + '(string-append out + "/bin/python3")) "-m" + "compileall" + "-o" + "0" + "-o" + "1" + "-o" + "2" + "-f" ;force rebuild + "--invalidation-mode=unchecked-hash" + ;; Don't build lib2to3, because it's + ;; Python 2 code. + "-x" + "lib2to3/.*" + ,output))) + (map cdr outputs))))) + (add-before 'check 'set-TZDIR + (lambda* (#:key inputs native-inputs #:allow-other-keys) + ;; test_email requires the Olson time zone database. + (setenv "TZDIR" + (string-append (assoc-ref (or native-inputs + inputs) "tzdata") + "/share/zoneinfo")))) + (add-after 'install 'install-sitecustomize.py + ,(customize-site version))))) + (inputs (list bzip2 + expat + gdbm + libffi ;for ctypes + sqlite ;for sqlite extension + openssl + readline + zlib + tcl + tk)) ;for tkinter + (native-inputs `(("tzdata" ,tzdata-for-tests) + ("unzip" ,unzip) + ("zip" ,(@ (gnu packages compression) zip)) + ("pkg-config" ,pkg-config) + ("sitecustomize.py" ,(local-file (search-auxiliary-file + "python/sitecustomize.py"))) + ;; When cross-compiling, a native version of Python itself is needed. + ,@(if (%current-target-system) + `(("python" ,this-package) + ("which" ,which)) + '()))) + (native-search-paths + (list (guix-pythonpath-search-path version) + ;; Used to locate tzdata by the zoneinfo module introduced in + ;; Python 3.9. + (search-path-specification + (variable "PYTHONTZPATH") + (files (list "share/zoneinfo"))))) + (home-page "https://www.python.org") + (synopsis "High-level, dynamically-typed programming language") + (description + "Python is a remarkably powerful dynamic programming language that +is used in a wide variety of application domains. Some of its key +distinguishing features include: clear, readable syntax; strong +introspection capabilities; intuitive object orientation; natural +expression of procedural code; full modularity, supporting hierarchical +packages; exception-based error handling; and very high level dynamic +data types.") + (properties '((cpe-name . "python"))) + (license license:psfl))) + + +;; Next 3.x version. +(define-public python-next python-3.12) + ;; Current 3.x version. (define-public python-3 python-3.10)