| Message ID | 20210122131858.11206-1-maxim.cournoyer@gmail.com |
|---|---|
| State | Accepted |
| Headers | show |
| Series | [bug#46028,core-updates,v2] gnu: python: Replace PYTHONPATH by GUIX_PYTHONPATH_X_Y. | expand |
| Context | Check | Description |
|---|---|---|
| cbaines/applying patch | fail | View Laminar job |
| cbaines/issue | success | View issue |
Hi, Maxim Cournoyer <maxim.cournoyer@gmail.com> writes: > Using PYTHONPATH as a mean to discover the Python packages had the following > issues: > > 1. It is not versioned, so different versions of Python would clash if > installed in a shared profile. > > 2. It would interfere with the host Python site on foreign > distributions, sometimes preventing a a user to login their GDM > session (!). > > 3. It would take precedence over user installed Python packages > installed through pip. > > 4. It would leak into Python virtualenvs, which are supposed to create > isolated Python environments. I've adapted the most problematic uses of PYTHONPATH in the code base (mostly (getenv "PYTHONPATH"), which would typically now return #f and pushed the branch to cu/farewell-to-pythonpath (it contains ~160 commits, so I thought it'd be easier to review as a branch). Thanks! Maxim
Maxim Cournoyer <maxim.cournoyer@gmail.com> writes: > Hi, > > Maxim Cournoyer <maxim.cournoyer@gmail.com> writes: > >> Using PYTHONPATH as a mean to discover the Python packages had the following >> issues: >> >> 1. It is not versioned, so different versions of Python would clash if >> installed in a shared profile. >> >> 2. It would interfere with the host Python site on foreign >> distributions, sometimes preventing a a user to login their GDM >> session (!). >> >> 3. It would take precedence over user installed Python packages >> installed through pip. >> >> 4. It would leak into Python virtualenvs, which are supposed to create >> isolated Python environments. > > I've adapted the most problematic uses of PYTHONPATH in the code base > (mostly (getenv "PYTHONPATH"), which would typically now return #f and > pushed the branch to cu/farewell-to-pythonpath (it contains ~160 > commits, so I thought it'd be easier to review as a branch). There's now a third version of this that I pushed to the cu/farewell-to-pythonpath branch. It now uses GUIX_PYTHONPATH instead of GUIX_PYTHONPATH_X_Y, which makes it easier to use in any package definition (as there's no longer a need to import the (guix build python-build-system) module for packages not using the Python build system). I'll merge it soon if nobody has an objection. Thanks, Maxim
Hello, Maxim Cournoyer <maxim.cournoyer@gmail.com> writes: > Maxim Cournoyer <maxim.cournoyer@gmail.com> writes: > >> Hi, >> >> Maxim Cournoyer <maxim.cournoyer@gmail.com> writes: >> >>> Using PYTHONPATH as a mean to discover the Python packages had the following >>> issues: >>> >>> 1. It is not versioned, so different versions of Python would clash if >>> installed in a shared profile. >>> >>> 2. It would interfere with the host Python site on foreign >>> distributions, sometimes preventing a a user to login their GDM >>> session (!). >>> >>> 3. It would take precedence over user installed Python packages >>> installed through pip. >>> >>> 4. It would leak into Python virtualenvs, which are supposed to create >>> isolated Python environments. >> >> I've adapted the most problematic uses of PYTHONPATH in the code base >> (mostly (getenv "PYTHONPATH"), which would typically now return #f and >> pushed the branch to cu/farewell-to-pythonpath (it contains ~160 >> commits, so I thought it'd be easier to review as a branch). > > There's now a third version of this that I pushed to the > cu/farewell-to-pythonpath branch. It now uses GUIX_PYTHONPATH instead > of GUIX_PYTHONPATH_X_Y, which makes it easier to use in any package > definition (as there's no longer a need to import the (guix build > python-build-system) module for packages not using the Python build > system). > > I'll merge it soon if nobody has an objection. I've now pushed this to core-updates. Closing! Maxim
diff --git a/gnu/packages/commencement.scm b/gnu/packages/commencement.scm index 0713bcb843..6d7593e223 100644 --- a/gnu/packages/commencement.scm +++ b/gnu/packages/commencement.scm @@ -3176,6 +3176,7 @@ memoized as a function of '%current-system'." ;; built, since it requires Linux headers. (("'linux', ") "")))) (delete 'set-TZDIR) + (replace 'customize-site ,(customize-site version)) ,@(if (hurd-system?) `((add-before 'build 'fix-regen (lambda* (#:key inputs #:allow-other-keys) @@ -3184,7 +3185,8 @@ memoized as a function of '%current-system'." (("/usr/include/") (string-append libc "/include/"))))))) '()))) - ((#:tests? _ #f) #f)))))) + ((#:tests? _ #f) #f)))) + (native-search-paths (list (guix-pythonpath-search-path version))))) (define/system-dependent ld-wrapper-boot0 ;; The first 'ld' wrapper, defined with 'define/system-dependent' because diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm index d3df1a4dab..c3453d63a9 100644 --- a/gnu/packages/python.scm +++ b/gnu/packages/python.scm @@ -102,7 +102,44 @@ #:use-module (guix build-system gnu) #:use-module (guix build-system trivial) #:use-module (srfi srfi-1) - #:use-module (srfi srfi-26)) + #:use-module (srfi srfi-26) + #:export (customize-site + guix-pythonpath-search-path)) + +(define* (customize-site version) + "Generate a sitecustomize.py phase, using VERSION." + `(lambda* (#:key outputs #:allow-other-keys) + (let* ((out (assoc-ref outputs "out")) + (site-packages (string-append + out "/lib/python" + ,(version-major+minor version) + "/site-packages")) + (pythonpath (format #f "GUIX_PYTHONPATH_~a_~a" + ,(version-major version) + ,(version-minor version))) + (sitecustomize.py (string-append site-packages + "/sitecustomize.py"))) + (with-output-to-file sitecustomize.py + (lambda _ + (format #t "\ +import os +import sys + +# Insert the GUIX_PYTHONPATH_X_Y values just before Python's own site. +sys_path_absolute = [os.path.realpath(p) for p in sys.path] +index = sys_path_absolute.index(~s) +sys.path = sys.path[:index] + os.environ[~s].split(':') + sys.path[index:] +" site-packages pythonpath)))))) + +(define (guix-pythonpath-search-path version) + "Generate a GUIX_PYTHONPATH_X_Y search path specification, using VERSION." + (search-path-specification + (variable (format #f "GUIX_PYTHONPATH_~a_~a" + (version-major version) + (version-minor version))) + (files (list (string-append "lib/python" + (version-major+minor version) + "/site-packages"))))) (define-public python-2.7 (package @@ -313,7 +350,9 @@ "/site-packages"))) (install-file tkinter.so target) (delete-file tkinter.so))))) - #t)))))) + #t))) + (add-after 'install 'customize-site + ,(customize-site version))))) (inputs `(("bzip2" ,bzip2) ("expat" ,expat) @@ -332,10 +371,7 @@ `(("python2" ,this-package) ("which" ,which)) '()))) - (native-search-paths - (list (search-path-specification - (variable "PYTHONPATH") - (files '("lib/python2.7/site-packages"))))) + (native-search-paths (list (guix-pythonpath-search-path version))) (home-page "https://www.python.org") (synopsis "High-level, dynamically-typed programming language") (description @@ -463,7 +499,8 @@ data types.") "-x" "lib2to3/.*" ,out)))) (list "none" "-O" "-OO")) - #t))))))) + #t))) + (replace 'customize-site ,(customize-site version)))))) (native-inputs `(("tzdata" ,tzdata-for-tests) ,@(if (%current-target-system) @@ -471,11 +508,7 @@ data types.") '()) ,@(package-native-inputs python-2))) (native-search-paths - (list (search-path-specification - (variable "PYTHONPATH") - (files (list (string-append "lib/python" - (version-major+minor version) - "/site-packages")))) + (list (guix-pythonpath-search-path version) ;; Used to locate tzdata by the zoneinfo module introduced in ;; Python 3.9. (search-path-specification
Using PYTHONPATH as a mean to discover the Python packages had the following issues: 1. It is not versioned, so different versions of Python would clash if installed in a shared profile. 2. It would interfere with the host Python site on foreign distributions, sometimes preventing a a user to login their GDM session (!). 3. It would take precedence over user installed Python packages installed through pip. 4. It would leak into Python virtualenvs, which are supposed to create isolated Python environments. This changes fixes the above issues by making use of a sitecustomize.py module. The newly introduced GUIX_PYTHONPATH_X_Y environment variable, where X is the major and Y the minor version numbers, is read from the environment and spliced in 'sys.path' just before Python's own site location, which provides the expected behavior. * gnu/packages/python.scm (customize-site) (guix-pythonpath-search-path): New procedures. * gnu/packages/python.scm (python-2.7)[phases]{customize-site}: New phase. [native-search-paths]: Adjust. * gnu/packages/python.scm (python-3.9)[native-search-paths]: Likewise. [phases]{customize-site}: Override the inherited phase, so that it uses the correct version. * gnu/packages/commencement.scm (python-boot0) [phases]{customize-site}: Likewise. [native-search-paths]: Override the inherited one. --- gnu/packages/commencement.scm | 4 ++- gnu/packages/python.scm | 57 +++++++++++++++++++++++++++-------- 2 files changed, 48 insertions(+), 13 deletions(-)