From patchwork Mon Apr 25 03:57:03 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxim Cournoyer X-Patchwork-Id: 38875 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id E6EF827BBE9; Mon, 25 Apr 2022 05:24:57 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FROM,MAILING_LIST_MULTI, SPF_HELO_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id F235727BBE9 for ; Mon, 25 Apr 2022 05:24:56 +0100 (BST) Received: from localhost ([::1]:49626 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1niqHQ-0004te-5H for patchwork@mira.cbaines.net; Mon, 25 Apr 2022 00:24:56 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38984) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nipwG-0008Cy-8D for guix-patches@gnu.org; Mon, 25 Apr 2022 00:03:06 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:38846) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1nipwF-0001sl-Vf for guix-patches@gnu.org; Mon, 25 Apr 2022 00:03:03 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1nipwF-0008KI-SX for guix-patches@gnu.org; Mon, 25 Apr 2022 00:03:03 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#55104] [PATCH 097/232] gnu: Add python-debugpy. Resent-From: Maxim Cournoyer Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Mon, 25 Apr 2022 04:03:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 55104 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 55104@debbugs.gnu.org Cc: Maxim Cournoyer Received: via spool by 55104-submit@debbugs.gnu.org id=B55104.165085935831938 (code B ref 55104); Mon, 25 Apr 2022 04:03:03 +0000 Received: (at 55104) by debbugs.gnu.org; 25 Apr 2022 04:02:38 +0000 Received: from localhost ([127.0.0.1]:60972 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nipvp-0008Iy-4K for submit@debbugs.gnu.org; Mon, 25 Apr 2022 00:02:37 -0400 Received: from mail-qt1-f178.google.com ([209.85.160.178]:36745) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nipuF-0007v1-7t for 55104@debbugs.gnu.org; Mon, 25 Apr 2022 00:01:05 -0400 Received: by mail-qt1-f178.google.com with SMTP id bb21so9571812qtb.3 for <55104@debbugs.gnu.org>; Sun, 24 Apr 2022 21:00:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=QfT6yytZbWl5bmvyVOztbk1VwCUDYcfUbwrhNuhqT2Y=; b=IHPTMUE0UqrwC5VemBk93mGXecCjyb+Q0Al2LqBB2KLbnSlocM3g0piPOAhR37nuRG FyCEfSTjjfrHG3GNaZG/VdikxgWSErH+HXUjlAnH2XbNq6ZD+yCJrU6P14/V5CXOhxuB ixZiNsb7vNUnD2XX0D5dfcFtWpi1GGOPiesoPh8aC8r6rdOmmE9NPQ8b93i25ZfIcYI0 20Rj6Yij+eVSMFSWywcktii1sSiqtHdTPsCSjwpkslmTFlTV4W18oqZkODpKfKgYJoQr 523qlIwMqtOeDYqqRccC2g+rsMqxWX7XTGunsvStq8bw9qg7H5mdZ+w8XMg7+9ryr2r6 SUsg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=QfT6yytZbWl5bmvyVOztbk1VwCUDYcfUbwrhNuhqT2Y=; b=HQdf+2la30X1PQd0XNFMbAN6DQsWqR/1l1hTAQOUCrUP11THdEEnDoceWDWjzKkDRV OF0P5rba7W4er+0qCvfzBqc1W5eGVIeMG4mLJrthpXOGgPYJw+6Yua7KWyJgsUwiPWTX OUWBLRbotM9D2nhmNJKU5F6vfr/KL47j9kgitTJFrbuDvXEdzju+l+DxfDGiiQr2MFNo 1GVeXGxuozfKW6jT3pJG7KrCErpRUWTQUn5kjn0aJQRCqIrLnrJfFLMAK09CuqorteiF snyj/RLv2bM1DG1EY+gt66cA4RcFjRGr8QnBPPuNpS9KWHQjJTy7F7zj3qFFUSPZzXxE BERg== X-Gm-Message-State: AOAM531Fxiro3yg/jQyvoK7FCVWFvCQRHQ34DhEMdqnf7DM3JVL0nSYc 2DyA0S2mdjxl7OAFjUVkpLQFdXljRo86ZA== X-Google-Smtp-Source: ABdhPJxF5Tr2eteNNm+q5nzGbuljTY0IxzuKTFFsPEwPcW0HsbXfjvwKDA5+w7WtjKX8NNqKWR+8SQ== X-Received: by 2002:ac8:5848:0:b0:2f3:6f14:8244 with SMTP id h8-20020ac85848000000b002f36f148244mr68893qth.584.1650859253529; Sun, 24 Apr 2022 21:00:53 -0700 (PDT) Received: from localhost.localdomain (dsl-150-232.b2b2c.ca. [66.158.150.232]) by smtp.gmail.com with ESMTPSA id w82-20020a376255000000b0069ee3f0ae63sm4478142qkb.45.2022.04.24.21.00.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 24 Apr 2022 21:00:52 -0700 (PDT) From: Maxim Cournoyer Date: Sun, 24 Apr 2022 23:57:03 -0400 Message-Id: <20220425035918.25683-97-maxim.cournoyer@gmail.com> X-Mailer: git-send-email 2.34.0 In-Reply-To: <20220425035918.25683-1-maxim.cournoyer@gmail.com> References: <20220425035918.25683-1-maxim.cournoyer@gmail.com> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches * gnu/packages/python-xyz.scm (python-debugpy): New variable. * gnu/packages/patches/python-debugpy-unbundle-pydevd.patch: New file. * gnu/local.mk: Register it. --- gnu/local.mk | 1 + .../python-debugpy-unbundle-pydevd.patch | 254 ++++++++++++++++++ gnu/packages/python-xyz.scm | 72 +++++ 3 files changed, 327 insertions(+) create mode 100644 gnu/packages/patches/python-debugpy-unbundle-pydevd.patch diff --git a/gnu/local.mk b/gnu/local.mk index 702c430623..0cad8fc7fa 100644 --- a/gnu/local.mk +++ b/gnu/local.mk @@ -1679,6 +1679,7 @@ dist_patch_DATA = \ %D%/packages/patches/python-argcomplete-1.11.1-fish31.patch \ %D%/packages/patches/python-cross-compile.patch \ %D%/packages/patches/python-configobj-setuptools.patch \ + %D%/packages/patches/python-debugpy-unbundle-pydevd.patch \ %D%/packages/patches/python-docopt-pytest6-compat.patch \ %D%/packages/patches/python-execnet-read-only-fix.patch \ %D%/packages/patches/python-fixtures-remove-monkeypatch-test.patch \ diff --git a/gnu/packages/patches/python-debugpy-unbundle-pydevd.patch b/gnu/packages/patches/python-debugpy-unbundle-pydevd.patch new file mode 100644 index 0000000000..7a6ad54489 --- /dev/null +++ b/gnu/packages/patches/python-debugpy-unbundle-pydevd.patch @@ -0,0 +1,254 @@ +Allow using pydevd as a regular dependency. +Submitted upstream at: https://github.com/microsoft/debugpy/pull/902 + +diff --git a/setup.py b/setup.py +index 5fc40070..3a530a29 100644 +--- a/setup.py ++++ b/setup.py +@@ -11,6 +11,9 @@ import subprocess + import sys + + ++DEBUGPY_BUNDLING_DISABLED = bool(os.getenv('DEBUGPY_BUNDLING_DISABLED')) ++ ++ + sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) + import versioneer # noqa + +@@ -18,12 +21,15 @@ del sys.path[0] + + sys.path.insert(0, os.path.join(os.path.dirname(os.path.abspath(__file__)), "src")) + import debugpy +-import debugpy._vendored ++ ++if not DEBUGPY_BUNDLING_DISABLED: ++ import debugpy._vendored + + del sys.path[0] + + +-PYDEVD_ROOT = debugpy._vendored.project_root("pydevd") ++PYDEVD_ROOT = (None if DEBUGPY_BUNDLING_DISABLED else ++ debugpy._vendored.project_root("pydevd")) + DEBUGBY_ROOT = os.path.dirname(os.path.abspath(debugpy.__file__)) + + +@@ -67,7 +73,7 @@ def iter_vendored_files(): + # relevant setuptools versions. + class ExtModules(list): + def __bool__(self): +- return True ++ return not DEBUGPY_BUNDLING_DISABLED + + + def override_build(cmds): +@@ -133,9 +139,24 @@ with open("DESCRIPTION.md", "r") as fh: + + + if __name__ == "__main__": +- if not os.getenv("SKIP_CYTHON_BUILD"): ++ if not (os.getenv("SKIP_CYTHON_BUILD") or DEBUGPY_BUNDLING_DISABLED): + cython_build() + ++ # Etch bundling status in the source. ++ if debugpy.__bundling_disabled__ != DEBUGPY_BUNDLING_DISABLED: ++ ++ with open(os.path.join(DEBUGBY_ROOT, '__init__.py'), 'r') as f: ++ lines = f.readlines() ++ with open(os.path.join(DEBUGBY_ROOT, '__init__.py'), 'w') as f: ++ edited = [] ++ for line in lines: ++ if line.startswith('__bundling_disabled__'): ++ edited.append( ++ f'__bundling_disabled__ = {DEBUGPY_BUNDLING_DISABLED}\n') ++ else: ++ edited.append(line) ++ f.writelines(edited) ++ + extras = {} + platforms = get_buildplatform() + if platforms is not None: +@@ -145,6 +166,18 @@ if __name__ == "__main__": + override_build(cmds) + override_build_py(cmds) + ++ data = {"debugpy": ["ThirdPartyNotices.txt"]} ++ packages = [ ++ "debugpy", ++ "debugpy.adapter", ++ "debugpy.common", ++ "debugpy.launcher", ++ "debugpy.server", ++ ] ++ if not DEBUGPY_BUNDLING_DISABLED: ++ data.update({"debugpy._vendored": list(iter_vendored_files())}) ++ packages.append("debugpy._vendored") ++ + setuptools.setup( + name="debugpy", + version=versioneer.get_version(), +@@ -173,20 +206,10 @@ if __name__ == "__main__": + "License :: OSI Approved :: MIT License", + ], + package_dir={"": "src"}, +- packages=[ +- "debugpy", +- "debugpy.adapter", +- "debugpy.common", +- "debugpy.launcher", +- "debugpy.server", +- "debugpy._vendored", +- ], +- package_data={ +- "debugpy": ["ThirdPartyNotices.txt"], +- "debugpy._vendored": list(iter_vendored_files()), +- }, ++ packages=packages, ++ package_data=data, + ext_modules=ExtModules(), +- has_ext_modules=lambda: True, ++ has_ext_modules=lambda: not DEBUGPY_BUNDLING_DISABLED, + cmdclass=cmds, + **extras + ) +diff --git a/src/debugpy/__init__.py b/src/debugpy/__init__.py +index baa5a7c5..7b7a29aa 100644 +--- a/src/debugpy/__init__.py ++++ b/src/debugpy/__init__.py +@@ -206,6 +206,8 @@ def trace_this_thread(should_trace): + + __version__ = _version.get_versions()["version"] + ++__bundling_disabled__ = False ++ + # Force absolute path on Python 2. + __file__ = os.path.abspath(__file__) + +diff --git a/src/debugpy/server/__init__.py b/src/debugpy/server/__init__.py +index e6a1ad66..5f29a87a 100644 +--- a/src/debugpy/server/__init__.py ++++ b/src/debugpy/server/__init__.py +@@ -4,6 +4,50 @@ + + from __future__ import absolute_import, division, print_function, unicode_literals + ++from importlib import import_module ++import os ++ + # "force_pydevd" must be imported first to ensure (via side effects) + # that the debugpy-vendored copy of pydevd gets used. +-import debugpy._vendored.force_pydevd # noqa ++import debugpy ++if debugpy.__bundling_disabled__: ++ # Do what force_pydevd.py does, but using the system-provided ++ # pydevd. ++ ++ # XXX: This is copied here so that the whole '_vendored' directory ++ # can be deleted when DEBUGPY_BUNDLING_DISABLED is set. ++ ++ # If debugpy logging is enabled, enable it for pydevd as well ++ if "DEBUGPY_LOG_DIR" in os.environ: ++ os.environ[str("PYDEVD_DEBUG")] = str("True") ++ os.environ[str("PYDEVD_DEBUG_FILE")] = \ ++ os.environ["DEBUGPY_LOG_DIR"] + str("/debugpy.pydevd.log") ++ ++ # Work around https://github.com/microsoft/debugpy/issues/346. ++ # Disable pydevd frame-eval optimizations only if unset, to allow opt-in. ++ if "PYDEVD_USE_FRAME_EVAL" not in os.environ: ++ os.environ[str("PYDEVD_USE_FRAME_EVAL")] = str("NO") ++ ++ # Constants must be set before importing any other pydevd module ++ # due to heavy use of "from" in them. ++ pydevd_constants = import_module('_pydevd_bundle.pydevd_constants') ++ # The default pydevd value is 1000. ++ pydevd_constants.MAXIMUM_VARIABLE_REPRESENTATION_SIZE = 2 ** 32 ++ ++ # When pydevd is imported it sets the breakpoint behavior, but it needs to be ++ # overridden because by default pydevd will connect to the remote debugger using ++ # its own custom protocol rather than DAP. ++ import pydevd # noqa ++ import debugpy # noqa ++ ++ def debugpy_breakpointhook(): ++ debugpy.breakpoint() ++ ++ pydevd.install_breakpointhook(debugpy_breakpointhook) ++ ++ # Ensure that pydevd uses JSON protocol ++ from _pydevd_bundle import pydevd_constants ++ from _pydevd_bundle import pydevd_defaults ++ pydevd_defaults.PydevdCustomization.DEFAULT_PROTOCOL = pydevd_constants.HTTP_JSON_PROTOCOL ++else: ++ import debugpy._vendored.force_pydevd # noqa +diff --git a/src/debugpy/server/attach_pid_injected.py b/src/debugpy/server/attach_pid_injected.py +index e6345996..87cfdd53 100644 +--- a/src/debugpy/server/attach_pid_injected.py ++++ b/src/debugpy/server/attach_pid_injected.py +@@ -8,6 +8,7 @@ from __future__ import absolute_import, division, print_function, unicode_litera + + import os + ++import debugpy + + __file__ = os.path.abspath(__file__) + _debugpy_dir = os.path.dirname(os.path.dirname(os.path.dirname(__file__))) +@@ -30,25 +31,29 @@ def attach(setup): + def on_critical(msg): + print(msg, file=sys.stderr) + +- pydevd_attach_to_process_path = os.path.join( +- _debugpy_dir, +- "debugpy", +- "_vendored", +- "pydevd", +- "pydevd_attach_to_process", +- ) +- assert os.path.exists(pydevd_attach_to_process_path) +- sys.path.insert(0, pydevd_attach_to_process_path) +- +- # NOTE: that it's not a part of the pydevd PYTHONPATH +- import attach_script ++ if debugpy.__bundling_disabled__: ++ from pydevd_attach_to_process import attach_script ++ else: ++ pydevd_attach_to_process_path = os.path.join( ++ _debugpy_dir, ++ "debugpy", ++ "_vendored", ++ "pydevd", ++ "pydevd_attach_to_process", ++ ) ++ assert os.path.exists(pydevd_attach_to_process_path) ++ sys.path.insert(0, pydevd_attach_to_process_path) ++ ++ # NOTE: that it's not a part of the pydevd PYTHONPATH ++ import attach_script + + attach_script.fix_main_thread_id( + on_warn=on_warn, on_exception=on_exception, on_critical=on_critical + ) + +- # NOTE: At this point it should be safe to remove this. +- sys.path.remove(pydevd_attach_to_process_path) ++ if not debugpy.__bundling_disabled__: ++ # NOTE: At this point it should be safe to remove this. ++ sys.path.remove(pydevd_attach_to_process_path) + except: + import traceback + +diff --git a/tests/tests/test_vendoring.py b/tests/tests/test_vendoring.py +index dd6c4269..28c03702 100644 +--- a/tests/tests/test_vendoring.py ++++ b/tests/tests/test_vendoring.py +@@ -1,3 +1,8 @@ ++import pytest ++ ++import debugpy ++ ++@pytest.mark.skipif(debugpy.__bundling_disabled__, reason='Bundling disabled') + def test_vendoring(pyfile): + @pyfile + def import_debugpy(): +-- +2.34.0 + diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm index 83c2d472c9..d7a9603e02 100644 --- a/gnu/packages/python-xyz.scm +++ b/gnu/packages/python-xyz.scm @@ -13118,6 +13118,78 @@ (define-public python-pydevd and other @acronym{IDEs, Integrated Development Environments}.") (license license:epl1.0)))) +(define-public python-debugpy + (package + (name "python-debugpy") + (version "1.6.0") + (source + (origin + (method git-fetch) + (uri (git-reference ;no tests in PyPI archive + (url "https://github.com/microsoft/debugpy") + (commit (string-append "v" version)))) + (file-name (git-file-name name version)) + (modules '((guix build utils))) + ;; Remove the bundled PyDev-Debugger copy, including its pre-built + ;; attach binary. + (snippet '(delete-file-recursively "src/debugpy/_vendored")) + (patches (search-patches "python-debugpy-unbundle-pydevd.patch")) + (sha256 + (base32 + "1dpfzs3p51648i7f3fz8dw5d0vrj39iwn1jhn0226idc02ybyqih")))) + (build-system python-build-system) + (arguments + (list + #:phases + #~(modify-phases %standard-phases + (add-after 'unpack 'patch-sh-in-tests + (lambda _ + (substitute* "tests/debugpy/test_run.py" + (("#!/bin/sh") + (string-append "#!" (which "sh")))))) + (add-after 'unpack 'fix-version + ;; Versioneer is useless when there is no git metadata. + (lambda _ + (substitute* "setup.py" + (("version=versioneer.get_version\\(),") + (format #f "version=~s," #$version))))) + (add-before 'build 'configure + (lambda _ + ;; This adjusts the behavior of debugpy to load pydevd from + ;; Python site packages. + (setenv "DEBUGPY_BUNDLING_DISABLED" "1"))) + (replace 'check + (lambda* (#:key tests? #:allow-other-keys) + (invoke "pytest" "-vv" + "-n" (number->string (parallel-job-count)) + "-k" + (string-append + ;; These tests cannot be run in parallel because their + ;; test data would not be copied by xdist and lead to + ;; import errors. (see: + ;; https://github.com/microsoft/debugpy/issues/342 and + ;; https://github.com/microsoft/debugpy/issues/880). + "not test_custom_python_args " + "and not test_autokill "))))))) + (native-inputs + ;; See: https://raw.githubusercontent.com/microsoft/debugpy/ + ;; main/tests/requirements.txt. + (list python-django + python-gevent + python-flask + python-psutil + python-pytest + python-pytest-cov + python-pytest-timeout + python-pytest-xdist + python-requests)) + (propagated-inputs (list python-pydevd)) + (home-page "https://aka.ms/debugpy") + (synopsis "Debug Adapter Protocol Python implementation") + (description "An implementation of the Debug Adapter Protocol for +Python.") + (license license:expat))) + (define-public python-debian (package (name "python-debian")