diff mbox series

[bug#69587] doc: Add “Source Tree Structure” section.

Message ID c692a389c094959f7235ab2191d4769cca036a8f.1709742909.git.ludo@gnu.org
State New
Headers show
Series [bug#69587] doc: Add “Source Tree Structure” section. | expand

Commit Message

Ludovic Courtès March 6, 2024, 4:37 p.m. UTC
* doc/contributing.texi (Source Tree Structure): New node.
* doc/guix.texi (Programming Interface): Add cross-reference.

Change-Id: I141a1f4d806ae5f72c7a246e18c14dc63056a607
---
 doc/contributing.texi | 235 +++++++++++++++++++++++++++++++++++++++++-
 doc/guix.texi         |   3 +-
 2 files changed, 236 insertions(+), 2 deletions(-)

Hello there!

This is a long overdue section for contributors, very much inspired
by Josselin's talk last year:

  https://10years.guix.gnu.org/video/guixy-guile-the-derivation-factory-a-tour-of-the-guix-source-tree/

Thoughts?

Ludo'.


base-commit: 9d9bb8955a939b89b5b28f1071b70ed9f9a54f8c

Comments

pelzflorian (Florian Pelz) March 8, 2024, 6:01 p.m. UTC | #1
Hi Ludo.

Good that I now have finally watched Josselin’s talk.

This section seems to be a good idea in general, but the examples are an
opinionated selection.  I believe the list should be small.  Listing
individual modules directly in ~/src/guix/guix directory is not useful
to newcomers or often self-explanatory.  gnu/installer, gnu/system,
gnu/build, gnu/machine is not useful to know.  po is better not talked
about, otherwise people edit translations directly in po, when they
should use Weblate.

Ludovic Courtès <ludo@gnu.org> writes:
> +The following table gives an overview of the main directories and what
> +they contain.  Remember that in Guile, each module name is derived from
> +its file name---e.g., the module in file@file{guix/packages.scm} is

file@file should be @file


> +called @code{(guix packages)}.
> +
> +@table @file
> +@item guix
> +This is the location of core Guix mechanisms.  A few examples:

What does core mean?  Perhaps (guix …) should be listed after (gnu …)
and defined as the Guix mechanisms that do not belong in gnu?  Not quite
sure either.

> […]
> +With all this, a fair chunk of your operating system is at your
> +fingertips!  Beyond @command{grep} and @command{git grep}, @pxref{The
> +Perfect Setup} on how to navigate code from your editor, and
> +@pxref{Using Guix Interactively} for information on how to use
> +interactively use Scheme modules.  Enjoy!

Remove duplicate “use”.

Regards,
Florian
Ludovic Courtès March 8, 2024, 10:06 p.m. UTC | #2
Hello!

"pelzflorian (Florian Pelz)" <pelzflorian@pelzflorian.de> skribis:

> This section seems to be a good idea in general, but the examples are an
> opinionated selection.  I believe the list should be small.  Listing
> individual modules directly in ~/src/guix/guix directory is not useful
> to newcomers or often self-explanatory.  gnu/installer, gnu/system,
> gnu/build, gnu/machine is not useful to know.

Are you suggesting to remove the examples?

If that’s the case, we might as well not add the section or it would
become a bullet list with three items (guix/, gnu/, and tests/), no?

The selection is opinionated, indeed.  My thought was that it was a way
to connect the dots among things presented in the manual and to provide
a “guided tour” pretty much like Josselin’s talk.

> po is better not talked about, otherwise people edit translations
> directly in po, when they should use Weblate.

Not sure; there’s a link to “Translating Guix” that explains what to do.

>> +@table @file
>> +@item guix
>> +This is the location of core Guix mechanisms.  A few examples:
>
> What does core mean?

The examples were meant to illustrate what is meant by “core”.  Do you
think some other adjective or a longer description would help?

> Perhaps (guix …) should be listed after (gnu …)  and defined as the
> Guix mechanisms that do not belong in gnu?  Not quite sure either.

The order I chose is (roughly) from lower-level to higher-level:

(guix store) -> (guix derivation) -> (guix packages) -> …
… -> (gnu packages) -> (gnu system) -> …

Does that make sense?

Thanks for your feedback!

Ludo’.
pelzflorian (Florian Pelz) March 9, 2024, 2:38 p.m. UTC | #3
Hi Ludo.

Josselin’s talk is different in that it is a talk of more than 30
minutes.  In so much time, it can give more detailed guidance to almost
the whole guix source tree, even including build-aux and nix.  Josselin
also gives hints to use git grep (like you) but also to read the
commentary at the top of the file.  This may be a helpful hint to
someone starting out, but someone starting out maybe does not want to
read as much as a complete talk.  If they wanted it all, then better
link to Josselin’s talk.

Ludovic Courtès <ludo@gnu.org> writes:
> The order I chose is (roughly) from lower-level to higher-level:
>
> (guix store) -> (guix derivation) -> (guix packages) -> …
> … -> (gnu packages) -> (gnu system) -> …
>
> Does that make sense?

In your section the modules directly in (guix …) appeared unsorted to
me.  Could you explicitly state this order in the manual section?

Nice things like (guix swh) or (gnu system), (gnu build), (gnu
installer), (gnu machine), or po, still seem not useful for the general
populace to me.


> Are you suggesting to remove the examples?

I like tests and gnu/tests.  Also that your section lists (guix
build-system …) and the occasional duality with (guix build
…-build-system), Rightfully you also list non-dual utilities like (guix
build syscalls) to make clear not all in (guix build) is about build
systems.

The explanation about guix/scripts containing the entry points is
useful.

>> What does core mean?
>
> The examples were meant to illustrate what is meant by “core”.  Do you
> think some other adjective or a longer description would help?
>
>> Perhaps (guix …) should be listed after (gnu …)  and defined as the
>> Guix mechanisms that do not belong in gnu?  Not quite sure either.

Josselin called the distinction between (guix …) and (gnu …) murky,
explaining that most of (guix …) must not import (gnu …) except by
module-ref, while (guix scripts …) and such can just use-modules (gnu
…).  To me, gnu/packages.scm looks like core as well, but it rightfully
is in gnu.

Regards,
Florian
diff mbox series

Patch

diff --git a/doc/contributing.texi b/doc/contributing.texi
index a7d91724fb..ff7065ad2a 100644
--- a/doc/contributing.texi
+++ b/doc/contributing.texi
@@ -23,7 +23,8 @@  Contributing
 * Building from Git::           The latest and greatest.
 * Running Guix Before It Is Installed::  Hacker tricks.
 * The Perfect Setup::           The right tools.
-* Alternative Setups::          Other posible tools that do the job.
+* Alternative Setups::          Other possible tools that do the job.
+* Source Tree Structure::       Source code guided tour.
 * Packaging Guidelines::        Growing the distribution.
 * Coding Style::                Hygiene of the contributor.
 * Submitting Patches::          Share your work.
@@ -546,6 +547,238 @@  Vim and NeoVim
 process and inject your code there live (sadly it's not packaged in Guix yet).
 
 
+@node Source Tree Structure
+@section Source Tree Structure
+
+@cindex structure, of the source tree
+Now that we're all set, let's look at the source tree.  Overall, the
+Guix source tree contains almost exclusively Guile @dfn{modules}, each
+of which can be seen as an independent library (@pxref{Modules,,, guile,
+GNU Guile Reference Manual}).
+
+The following table gives an overview of the main directories and what
+they contain.  Remember that in Guile, each module name is derived from
+its file name---e.g., the module in file@file{guix/packages.scm} is
+called @code{(guix packages)}.
+
+@table @file
+@item guix
+This is the location of core Guix mechanisms.  A few examples:
+
+@table @code
+@item (guix store)
+Connecting to and interacting with the build daemon (@pxref{The Store}).
+@item (guix derivations)
+Creating derivations (@pxref{Derivations}).
+@item (guix gexps)
+Writing G-expressions (@pxref{G-Expressions}).
+@item (guix packages)
+Defining packages and origins (@pxref{package Reference}).
+@item (guix download)
+@itemx (guix git-download)
+The @code{url-fetch} and @code{git-fetch} origin download methods
+(@pxref{origin Reference}).
+@item (guix swh)
+Fetching source code from the
+@uref{https://archive.softwareheritage.org,Software Heritage archive}.
+@item (guix search-paths)
+Implementing search paths (@pxref{Search Paths}).
+@item (guix build-system)
+The build system interface (@pxref{Build Systems}).
+@item (guix profiles)
+Implementing profiles.
+@end table
+
+@cindex build system, directory structure
+@item guix/build-system
+This directory contains specific build system implementations
+(@pxref{Build Systems}), such as:
+
+@table @code
+@item (guix build-system gnu)
+the GNU build system;
+@item (guix build-system cmake)
+the CMake build system;
+@item (guix build-system pyproject)
+The Python ``pyproject'' build system.
+@end table
+
+@item guix/build
+This contains code generally used on the ``build side''
+(@pxref{G-Expressions, strata of code}).  This includes code used to
+build packages or other operating system components, as well as
+utilities:
+
+@table @code
+@item (guix build utils)
+Utilities for package definitions and more (@pxref{Build Utilities}).
+@item (guix build gnu-build-system)
+@itemx (guix build cmake-build-system)
+@itemx (guix build pyproject-build-system)
+Implementation of build systems, and in particular definition of their
+build phases (@pxref{Build Phases}).
+@item (guix build syscalls)
+Interface to the C library and to Linux system calls.
+@end table
+
+@cindex command-line tools, as Guile modules
+@cindex command modules
+@item guix/scripts
+This contains modules corresponding to @command{guix} sub-commands.  For
+example, the @code{(guix scripts shell)} module exports the
+@code{guix-shell} procedure, which directly corresponds to the
+@command{guix shell} command (@pxref{Invoking guix shell}).
+
+@cindex importer modules
+@item guix/import
+This contains supporting code for the importers and updaters
+(@pxref{Invoking guix import}, and @pxref{Invoking guix refresh}).  For
+example, @code{(guix import pypi)} defines the interface to PyPI, which
+is used by the @code{guix import pypi} command.
+@end table
+
+The directories we have seen so far all live under @file{guix/}.  The
+other important place is the @code{gnu/} directory, which contains
+primarily package definitions as well as libraries and tools for Guix
+System (@pxref{System Configuration}) and Guix Home (@pxref{Home
+Configuration}).
+
+@table @file
+@cindex package modules
+@item gnu/packages
+This is by far the most crowded directory of the source tree: it
+contains @dfn{package modules} that export package definitions
+(@pxref{Package Modules}).  A few examples:
+
+@table @code
+@item (gnu packages base)
+Module providing ``base'' packages: @code{glibc}, @code{coreutils},
+@code{grep}, etc.
+@item (gnu packages guile)
+Guile and core Guile packages.
+@item (gnu packages linux)
+The Linux-libre kernel and related packages.
+@item (gnu packages python)
+Python and core Python packages.
+@item (gnu packages python-xyz)
+Miscellaneous Python packages (we were not very creative).
+@end table
+
+In any case, you can jump to a package definition using @command{guix
+edit} (@pxref{Invoking guix edit}) and view its location with
+@command{guix show} (@pxref{Invoking guix package}).
+
+@findex search-patches
+@item gnu/packages/patches
+This directory contains patches applied against packages and obtained
+using the @code{search-patches} procedure.
+
+@item gnu/services
+This contains service definitions, primarily for Guix System
+(@pxref{Services}) but some of them are adapted and reused for Guix Home
+as we will see below.  Examples:
+
+@table @code
+@item (gnu services)
+The service framework itself, which defines the service and service type
+data types (@pxref{Service Composition}).
+@item (gnu services base)
+``Base'' services (@pxref{Base Services}).
+@item (gnu services desktop)
+``Desktop'' services (@pxref{Desktop Services}).
+@item (gnu services shepherd)
+Support for Shepherd services (@pxref{Shepherd Services}).
+@end table
+
+You can jump to a service definition using @command{guix system edit}
+and view its location with @command{guix system search} (@pxref{Invoking
+guix system}).
+
+@item gnu/system
+These are core Guix System modules, such as:
+
+@table @code
+@item (gnu system)
+Defines @code{operating-system} (@pxref{operating-system Reference}).
+@item (gnu system file-systems)
+Defines @code{file-system} (@pxref{File Systems}).
+@item (gnu system mapped-devices)
+Defines @code{mapped-device} (@pxref{Mapped Devices}).
+@end table
+
+@item gnu/build
+These are modules that are either used on the ``build side'' when
+building operating systems or packages, or at run time by operating
+systems.
+
+@table @code
+@item (gnu build accounts)
+Creating @file{/etc/passwd}, @file{/etc/shadow}, etc. (@pxref{User
+Accounts}).
+@item (gnu build activation)
+Activating an operating system at boot time or reconfiguration time.
+@item (gnu build file-systems)
+Searching, checking, and mounting file systems.
+@item (gnu build linux-boot)
+@itemx (gnu build hurd-boot)
+Booting GNU/Linux and GNU/Hurd operating systems.
+@item (gnu build linux-initrd)
+Creating a Linux initial RAM disk (@pxref{Initial RAM Disk}).
+@end table
+
+@item gnu/home
+This contains all things Guix Home (@pxref{Home Configuration});
+examples:
+
+@table @code
+@item (gnu home services)
+Core services such as @code{home-files-service-type}.
+@item (gnu home services ssh)
+SSH-related services (@pxref{Secure Shell}).
+@end table
+
+@item gnu/installer
+This contains the text-mode graphical system installer (@pxref{Guided
+Graphical Installation}).
+
+@item gnu/machine
+These are the @dfn{machine abstractions} used by @command{guix deploy}
+(@pxref{Invoking guix deploy}).
+
+@item gnu/tests
+This contains system tests---tests that spawn virtual machines to check
+that system services work as expected (@pxref{Running the Test Suite}).
+@end table
+
+Last, there's also a few directories that contain files that are
+@emph{not} Guile modules:
+
+@table @file
+@item tests
+These are unit tests, each file corresponding more or less to one
+module, in particular @code{(guix @dots{})} modules (@pxref{Running the
+Test Suite}).
+
+@item doc
+This is the documentation in the form of Texinfo files: this manual and
+the Cookbook.
+
+@item po
+This is the location of translations of Guix itself, of package synopses
+and descriptions, of the manual, and of the cookbook (@pxref{Translating
+Guix}).
+
+@item etc
+Miscellaneous files: shell completions, support for systemd and other
+init systems, Git hooks, etc.
+@end table
+
+With all this, a fair chunk of your operating system is at your
+fingertips!  Beyond @command{grep} and @command{git grep}, @pxref{The
+Perfect Setup} on how to navigate code from your editor, and
+@pxref{Using Guix Interactively} for information on how to use
+interactively use Scheme modules.  Enjoy!
+
 @node Packaging Guidelines
 @section Packaging Guidelines
 
diff --git a/doc/guix.texi b/doc/guix.texi
index 87fe9f803c..fa26d78187 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -7677,7 +7677,8 @@  Programming Interface
 that build results @emph{derive} from them.
 
 This chapter describes all these APIs in turn, starting from high-level
-package definitions.
+package definitions.  @xref{Source Tree Structure}, for a more general
+overview of the source code.
 
 @menu
 * Package Modules::             Packages from the programmer's viewpoint.