From patchwork Sat Jan 22 10:47:57 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Ludovic_Court=C3=A8s?= X-Patchwork-Id: 36741 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 12D5227BBE9; Sat, 22 Jan 2022 10:49:12 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-3.7 required=5.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS, URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 1CB4D27BBEA for ; Sat, 22 Jan 2022 10:49:11 +0000 (GMT) Received: from localhost ([::1]:36024 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nBDxG-0002oQ-7a for patchwork@mira.cbaines.net; Sat, 22 Jan 2022 05:49:10 -0500 Received: from eggs.gnu.org ([209.51.188.92]:42388) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nBDx8-0002n1-Pl for guix-patches@gnu.org; Sat, 22 Jan 2022 05:49:02 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:42887) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1nBDx8-0003Iu-GC for guix-patches@gnu.org; Sat, 22 Jan 2022 05:49:02 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1nBDx8-0005gP-FW for guix-patches@gnu.org; Sat, 22 Jan 2022 05:49:02 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#53439] [PATCH] doc: Document search paths. Resent-From: Ludovic =?utf-8?q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sat, 22 Jan 2022 10:49:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 53439 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 53439@debbugs.gnu.org Cc: Ludovic =?utf-8?q?Court=C3=A8s?= X-Debbugs-Original-To: guix-patches@gnu.org Received: via spool by submit@debbugs.gnu.org id=B.164284850321796 (code B ref -1); Sat, 22 Jan 2022 10:49:02 +0000 Received: (at submit) by debbugs.gnu.org; 22 Jan 2022 10:48:23 +0000 Received: from localhost ([127.0.0.1]:35790 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nBDwU-0005fU-SQ for submit@debbugs.gnu.org; Sat, 22 Jan 2022 05:48:23 -0500 Received: from lists.gnu.org ([209.51.188.17]:37584) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nBDwS-0005fL-Vr for submit@debbugs.gnu.org; Sat, 22 Jan 2022 05:48:21 -0500 Received: from eggs.gnu.org ([209.51.188.92]:42252) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nBDwP-0002Qi-AV for guix-patches@gnu.org; Sat, 22 Jan 2022 05:48:19 -0500 Received: from [2001:470:142:3::e] (port=49056 helo=fencepost.gnu.org) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nBDwL-0003Dx-Vn; Sat, 22 Jan 2022 05:48:15 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:Date:Subject:To:From:in-reply-to: references; bh=fDvbvVhwWUelPeqLVS4qGhK7ugGBRAlCKOf/SomEpC4=; b=FJ1dg4eR4u9HHh i+cLE3yKABemYCpGvTuPrwtFQ6NULrvyFwBRc0Dx/Q/NwSKidQGuqAu7VaXbxBzJGdbsSg02Sqv+r IW5+6LdnImPwUKf9u4Pp/5avp3mLVwGWW0WYsrEfXhlyJgMd2jh9tcddjN8lZ+5RgIs26soe1odJc o3Y1pCxeUbRC1ACe/bKhtBLsb1gXxGxXLo3/lpTzdtLN5+8b4XR5VCpDBe4TdenoCopT1iVVfSQ/0 l1tCqeHXPWi5DS3URnaiuceVa28OhNUMS0k0dClAkxatpznQMsi2BELAmNVJe6QB9eZghfnfoj+AS bfvLJ7oHgpyBMQXjD3Tg==; Received: from 91-160-117-201.subs.proxad.net ([91.160.117.201]:60140 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1nBDwH-0000O4-TN; Sat, 22 Jan 2022 05:48:12 -0500 From: Ludovic =?utf-8?q?Court=C3=A8s?= Date: Sat, 22 Jan 2022 11:47:57 +0100 Message-Id: <20220122104757.1837-1-ludo@gnu.org> X-Mailer: git-send-email 2.34.0 MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches * doc/guix.texi (package Reference): Link to "Search Paths". (Build Phases): Mention 'set-paths' phase. (Search Paths): New node. --- doc/guix.texi | 184 +++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 183 insertions(+), 1 deletion(-) Hi! Here's a long overdue piece of documentation. Thoughts? Ludo'. base-commit: ee6bf00b2d89f6acab55b7a82896d99e39c1229b diff --git a/doc/guix.texi b/doc/guix.texi index 912a8e3c5a..15d887576d 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -279,6 +279,7 @@ Programming Interface * Build Systems:: Specifying how packages are built. * Build Phases:: Phases of the build process of a package. * Build Utilities:: Helpers for your package definitions and more. +* Search Paths:: Declaring search path environment variables. * The Store:: Manipulating the package store. * Derivations:: Low-level interface to package derivations. * The Store Monad:: Purely functional interface to the store. @@ -6859,6 +6860,7 @@ package definitions. * Build Systems:: Specifying how packages are built. * Build Phases:: Phases of the build process of a package. * Build Utilities:: Helpers for your package definitions and more. +* Search Paths:: Declaring search path environment variables. * The Store:: Manipulating the package store. * Derivations:: Low-level interface to package derivations. * The Store Monad:: Purely functional interface to the store. @@ -7239,7 +7241,8 @@ Outputs}, for typical uses of additional outputs. @item @code{native-search-paths} (default: @code{'()}) @itemx @code{search-paths} (default: @code{'()}) A list of @code{search-path-specification} objects describing -search-path environment variables honored by the package. +search-path environment variables honored by the package. @xref{Search +Paths}, for more on search path specifications. @item @code{replacement} (default: @code{#f}) This must be either @code{#f} or a package object that will be used as a @@ -8900,6 +8903,10 @@ standard list of phases. For @code{gnu-build-system}, the main build phases are the following: @table @code +@item set-paths +Define search path environment variables for all the input packages, +including @env{PATH} (@pxref{Search Paths}). + @item unpack Unpack the source tarball, and change the current directory to the extracted source tree. If the source is actually a directory, copy it @@ -9393,6 +9400,181 @@ executable files to be installed: @c TODO: Add more examples. +@node Search Paths +@section Search Paths + +@cindex search path +Many programs and libraries look for input data in a @dfn{search path}, +a list of directories: shells like Bash look for executables in the +command search path, a C compiler looks for @file{.h} files in its +header search path, and the Python interpreter looks for @file{.py} +files in its search path, the spell checker has a search path for +dictionaries, and so on. + +Search paths can usually be defined or overridden @i{via} environment +variables (@pxref{Environment Variables,,, libc, The GNU C Library +Reference Manual}). For example, the search paths mentioned above can +be changed by defining the @env{PATH}, @env{C_INCLUDE_PATH}, +@env{PYTHONPATH} (or @env{GUIX_PYTHONPATH}), and @env{DICPATH} +environment variables---you know, all these something-PATH variables +that you need to get right or things ``won't be found''. + +You may have noticed from the command line that Guix ``knows'' which +search path environment variables should be defined, and how. When you +install packages in your default profile, the file +@file{~/.guix-profile/etc/profile} is created, which you can ``source'' +from the shell to set those variables. Likewise, if you ask +@command{guix shell} to create an environment containing Python and +NumPy, a Python library, and if you pass it the @option{--search-paths} +option, it will tell you about @env{PATH} and @env{GUIX_PYTHONPATH} +(@pxref{Invoking guix shell}): + +@example +$ guix shell python python-numpy --pure --search-paths +export PATH="/gnu/store/@dots{}-profile/bin" +export GUIX_PYTHONPATH="/gnu/store/@dots{}-profile/lib/python3.9/site-packages" +@end example + +When you omit @option{--search-paths}, it defines these environment +variables right away, such that Python can readily find NumPy: + +@example +$ guix shell python python-numpy -- python3 +Python 3.9.6 (default, Jan 1 1970, 00:00:01) +[GCC 10.3.0] on linux +Type "help", "copyright", "credits" or "license" for more information. +>>> import numpy +>>> numpy.version.version +'1.20.3' +@end example + +For this to work, the definition of the @code{python} package +@emph{declares} the search path it cares about and its associated +environment variable, @env{GUIX_PYTHONPATH}. It looks like this: + +@lisp +(package + (name "python") + (version "3.9.9") + ;; some fields omitted... + (native-search-paths + (list (search-path-specification + (variable "GUIX_PYTHONPATH") + (files (list "lib/python/3.9/site-packages")))))) +@end lisp + +What this @code{native-search-paths} field says is that, when the +@code{python} package is used, the @env{GUIX_PYTHONPATH} environment +variable must be defined to include all the +@file{lib/python/3.9/site-packages} sub-directories encountered in its +environment. (The @code{native-} bit means that, if we are in a +cross-compilation environment, only native inputs may be added to the +search path.) In the NumPy example above, the profile where +@code{python} appears contains exactly one such sub-directory, and +@env{GUIX_PYTHONPATH} is set to that. When there are several +@file{lib/python/3.9/site-packages}---this is the case in package build +environments---they are all added to @env{GUIX_PYTHONPATH}, separated by +colons (@code{:}). + +@quotation Note +Notice that @env{GUIX_PYTHONPATH} is specified as part of the definition +of the @code{python} package, and @emph{not} as part of that of +@code{python-numpy}. This is because this environment variable +``belongs'' to Python, not NumPy: Python actually reads the value of +that variable and honors it. + +Corollary: if you create a profile that does not contain @code{python}, +@code{GUIX_PYTHONPATH} will @emph{not} be defined, even if it contains +packages that provide @file{.py} files: + +@example +$ guix shell python-numpy --search-paths --pure +export PATH="/gnu/store/@dots{}-profile/bin" +@end example + +This makes a lot of sense if we look at this profile in isolation: no +software in this profile would read @env{GUIX_PYTHONPATH}. +@end quotation + +Of course, there are many variations on that theme: some packages honor +more than one search path, some use separators other than colon, some +accumulate several directories in their search path, and so on. A more +complex example is the search path of libxml2: the value of the +@env{XML_CATALOG_FILES} environment variable is space-separated, it must +contain a list of @file{catalog.xml} files (not directories), which are +to be found in @file{xml} sub-directories---nothing less. The search +path specification looks like this: + +@lisp +(define libxml2 + (package + (name "libxml2") + ;; some fields omitted + (native-search-paths + (list (search-path-specification + (variable "XML_CATALOG_FILES") + (separator " ") + (files '("xml")) + (file-pattern "^catalog\\.xml$") + (file-type 'regular)))))) +@end lisp + +Worry not, search path specifications are usually not this tricky. + +The @code{(guix search-paths)} module defines the data type of search +path specifications and a number of helper procedures. Below is the +reference of search path specifications. + +@deftp {Data Type} search-path-specification +The data type for search path specifications. + +@table @asis +@item @code{variable} +The name of the environment variable for this search path (a string). + +@item @code{files} +The list of sub-directories (strings) that should be added to the search +path. + +@item @code{separator} (default: @code{":"}) +The string used to separate search path components. + +As a special case, a @code{separator} value of @code{#f} specifies a +``single-component search path''---in other words, a search path that +cannot contain more than one element. This is useful in some cases, +such as the @code{SSL_CERT_DIR} variable (honored by OpenSSL, cURL, and +a few other packages) or the @code{ASPELL_DICT_DIR} variable (honored by +the GNU Aspell spell checker), both of which must point to a single +directory. + +@item @code{file-type} (default: @code{'directory}) +The type of file being matched. + +In the libxml2 example above, we would match regular files. + +@item @code{file-pattern} (default: @code{#f}) +When true, this is a regular expression specifying files to be matched +@emph{within} the sub-directories specified by the @code{files} field. + +Again, the libxml2 example shows a situation where this is needed. +@end table +@end deftp + +How do you turn search path specifications on one hand and a bunch of +directories on the other hand in a set of environment variable +definitions? That's the job of @code{evaluate-search-paths}. + +@deffn {Scheme Procedure} evaluate-search-paths @var{search-paths} @ + @var{directories} [@var{getenv}] +Evaluate @var{search-paths}, a list of search-path specifications, for +@var{directories}, a list of directory names, and return a list of +specification/value pairs. Use @var{getenv} to determine the current +settings and report only settings not already effective. +@end deffn + +The @code{(guix profiles)} provides a higher-level helper procedure, +@code{load-profile}, that sets the environment variables of a profile. + @node The Store @section The Store