[bug#77093,rust-team,v2,17/17] doc: Document lockfile importer based Rust packaging workflow.

  * doc/contributing.texi (Packaging Guidelines)[Rust Crates]: Update
documentation.

Change-Id: Ic0c6378cf5f5df97d6f8bdd040b486be62c7bddc
---
 doc/contributing.texi | 415 +++++++++++++++++++++++++++++++++++++++---
 1 file changed, 390 insertions(+), 25 deletions(-)
  

On Tue, Mar 18, 2025 at 10:27:00PM +0800, Hilton Chain wrote:
> * doc/contributing.texi (Packaging Guidelines)[Rust Crates]: Update
> documentation.
> 
> Change-Id: Ic0c6378cf5f5df97d6f8bdd040b486be62c7bddc
> ---
>  doc/contributing.texi | 415 +++++++++++++++++++++++++++++++++++++++---
>  1 file changed, 390 insertions(+), 25 deletions(-)
> 
> diff --git a/doc/contributing.texi b/doc/contributing.texi
> index ab4f30d54b..837074dead 100644
> --- a/doc/contributing.texi
> +++ b/doc/contributing.texi
> @@ -1600,34 +1600,399 @@ Rust Crates
>  @subsection Rust Crates
>  
>  @cindex rust
> -Rust programs standing for themselves are named as any other package, using the
> -lowercase upstream name.
> +As currently there's no achievable way to reuse Rust packages as pre-compiled
> +inputs for other packages at a distribution scale, and our previous approach on
> +making all Rust dependencies as full packages doesn't cope well with Rust
> +ecosystem thus causing issues on both contribution and maintenance sides, we
> +have switched to a different packaging model.
> +
> +Rust programs (binary crates) and dependencies (library crates) are treated
> +separately.  We put our main efforts into programs and only package Rust
> +dependencies as sources, utilizing automation with a manual focus on unbundling
> +vendored dependencies.  The following paragraphs will explain them and give
> +several examples.
> +
> +Rust programs are treated like any other package and named using the lowercase
> +upstream name.  When using the Cargo build system (@pxref{Build Systems,
> +@code{cargo-build-system}}), Rust programs should have @code{#:install-source?}
> +argument set to @code{#f}, as this argument only makes sense for dependencies.
> +When the package source is a Cargo workspace, @code{#:cargo-install-paths} must
> +be set to enable relevant support.
> +
> +Rust dependencies are managed in two modules:
>  
> -To prevent namespace collisions we prefix all other Rust packages with the
> -@code{rust-} prefix.  The name should be changed to lowercase as appropriate and
> -dashes should remain in place.
> +@enumerate
> +@item
> +@code{(gnu packages rust-crates)}, storing source definitions imported from Rust
> +programs' @file{Cargo.lock} via the lockfile importer (@pxref{Invoking guix
> +import, crate, @code{--lockfile=@var{file}}}).
> +
> +Imported definitions must be checked and have vendored sources unbundled before
> +contributing to Guix.  Naturally, this module serves as a store for both sources
> +and unbundling strategies.
> +
> +This module is managed by the Rust team (@pxref{Teams}) to ensure there's always
> +one version containing all changes from other branches, so that the maintained
> +version can be used directly in case of merge conflicts, thus coordination is
> +required for other committers to modify it.
> +
> +Guix source ships template @file{etc/teams/rust/rust-crates.tmpl} and cleanup
> +script @file{etc/teams/rust/cleanup-crates.sh} for this module.
> +
> +@item
> +@code{(gnu packages rust-sources)}, storing more complex definitions that need
> +to be full packages.  This includes Rust dependencies requiring external inputs
> +to unbundle and Cargo workspaces.
> +
> +These dependencies should have the @code{#:skip-build?} argument set to
> +@code{#t}.  For Cargo workspaces, @code{#:cargo-package-crates} must be set.
> +
> +Since they are added manually, the following naming convention applies:
> +
> +To prevent namespace collisions they are named with @code{rust-} prefix.  The
> +name should be changed to lowercase as appropriate and dashes should remain in
> +place.
>  
>  In the rust ecosystem it is common for multiple incompatible versions of a
> -package to be used at any given time, so all package definitions should have a
> -versioned suffix.  The versioned suffix is the left-most non-zero digit (and
> -any leading zeros, of course).  This follows the ``caret'' version scheme
> -intended by Cargo.  Examples@: @code{rust-clap-2}, @code{rust-rand-0.6}.
> -
> -Because of the difficulty in reusing rust packages as pre-compiled inputs for
> -other packages the Cargo build system (@pxref{Build Systems,
> -@code{cargo-build-system}}) presents the @code{#:cargo-inputs} and
> -@code{cargo-development-inputs} keywords as build system arguments.  It would be
> -helpful to think of these as similar to @code{propagated-inputs} and
> -@code{native-inputs}.  Rust @code{dependencies} and @code{build-dependencies}
> -should go in @code{#:cargo-inputs}, and @code{dev-dependencies} should go in
> -@code{#:cargo-development-inputs}.  If a Rust package links to other libraries
> -then the standard placement in @code{inputs} and the like should be used.
> -
> -Care should be taken to ensure the correct version of dependencies are used; to
> -this end we try to refrain from skipping the tests or using @code{#:skip-build?}
> -when possible.  Of course this is not always possible, as the package may be
> -developed for a different Operating System, depend on features from the Nightly
> -Rust compiler, or the test suite may have atrophied since it was released.
> +package to be used at any given time, so all dependencies should have a
> +versioned suffix.  The versioned suffix is the left-most non-zero digit (and any
> +leading zeros, of course).  This follows the ``caret'' version scheme intended
> +by Cargo.  Examples@: @code{rust-clap-2}, @code{rust-rand-0.6}.
> +
> +In practice we are usually packaging development snapshots of Rust dependencies
> +specifically for some Rust programs, and can't simply identity them by version.
> +In this case we can use a @code{for-@var{program}} suffix.  Examples@:
> +@code{rust-pipewire-for-niri}, @code{rust-pubgrub-for-uv}.
> +@end enumerate
> +
> +Let's demonstrate the packaging workflow by examples, note that package-specific
> +issues are not involved here.
> +
> +In preparation, we'll add the following packages to our environment:
> +
> +@example
> +guix shell rust rust:cargo cargo-audit cargo-license
> +@end example

I was able to run cargo-audit as 'cargo-audit audit ...' but I wouldn't
actually recommend that to anyone.  I'm not sure we actually need rust
in the shell but I don't have a strong preference.  More thinking aloud,
perhaps we should adjust rust:cargo to automatically pull in rust:out?

I also find it amusing to suggest using cargo-audit when it's not (yet)
in guix and we immediately follow up with importing it.

I could see an argument for putting this part in the cookbook instead of
here but I feel strongly that it should be here.

> +
> +Example 1: @code{cargo-audit}, which is published on the crates.io Rust package
> +repository @uref{https://crates.io, crates.io}.
> +
> +@enumerate
> +@item
> +We can generate a draft definition via the crates.io importer (@pxref{Invoking
> +guix import, crate}).  In the end we'll have the following definiton:
> +
> +@lisp
> +(define-public cargo-audit
> +  (package
> +    (name "cargo-audit")
> +    (version "0.21.2")
> +    (source
> +     (origin
> +       (method url-fetch)
> +       (uri (crate-uri "cargo-audit" version))
> +       (file-name (string-append name "-" version ".tar.gz"))
> +       (sha256
> +        (base32 "1a00yqpckkw86zh2hg7ra82c5fx0ird5766dyynimbvqiwg2ps0n"))))
> +    (build-system cargo-build-system)
> +    (arguments (list #:install-source? #f))
> +    (inputs (cargo-inputs 'cargo-audit))
> +    (home-page "https://rustsec.org/")
> +    (synopsis "Audit Cargo.lock for crates with security vulnerabilities")
> +    (description
> +     "This package provides a Cargo subcommand, @@command@{cargo audit@}, to
> +audit @@file@{Cargo.lock@} for crates with security vulnerabilities.")
> +    (license (list license:asl2.0 license:expat))))
> +@end lisp
> +
> +@code{cargo-inputs} is a procedure defined in @code{guix build-system cargo},
> +facilitating dependency modification.  @code{'cargo-audit} used here must be a
> +unique identifier, usually same to the program's variable name.
> +
> +@item
> +Unpack package source and navigate to the unpacked directory, then execute the
> +following commands:
> +
> +@example
> +cargo generate-lockfile
> +cargo audit
> +cargo license
> +@end example
> +
> +@command{cargo generate-lockfile} updates dependencies to compatible versions,
> +@command{cargo audit} checks known vulnerabilities and @command{cargo license}
> +checks licenses of all dependencies.
> +
> +We must have an acceptable output of @command{cargo audit} and ensure all
> +dependencies are licensed with our supported licenses (@pxref{Defining Packages,
> +@code{license}}).
> +
> +@item
> +Import dependencies from previously generated lockfile:
> +
> +@example
> +guix import --insert=gnu/packages/rust-crates.scm \
> +    crate --lockfile=/path/to/Cargo.lock cargo-audit
> +@end example
> +
> +@code{cargo-audit} used here must be consistent with the identifier used for
> +@code{cargo-inputs}.
> +
> +At this stage, @code{cargo-audit} is buildable.
> +
> +@item
> +Finally we'll unbundle vendored sources.  The lockfile importer inserts
> +@code{TODO:} comments to dependencies with high probability of bundled sources.
> +@code{cargo-build-system} also performs additional check in its
> +@code{check-for-pregenerated-files} phase:
> +
> +@example
> +$ ./pre-inst-env guix build cargo-audit
> +@dots{}
> +starting phase `check-for-pregenerated-files'
> +Searching for binary files...
> +./guix-vendor/rust-addr2line-0.21.0.tar.gz/rustfmt.toml
> +./guix-vendor/rust-arc-swap-1.7.1.tar.gz/rustfmt.toml
> +./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/dictionary-rust
> +./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/dictionary-rust-other
> +./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/lib.rs.zst
> +./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/long-window-size-lib.rs.zst
> +@dots{}
> +@end example
> +
> +Although dependencies in @code{(gnu packages rust-crates)} are not exported, we
> +can still select them in Guix command-line interface through expression:
> +
> +@example
> +guix build --expression='(@@@@ (gnu packages rust-crates) rust-ring-0.17.14)'
> +@end example
> +
> +For most dependencies, a snippet is sufficient:
> +
> +@lisp
> +(define rust-bzip2-sys-0.1.13+1.0.8
> +  (crate-source "bzip2-sys" "0.1.13+1.0.8"
> +                "056c39pgjh4272bdslv445f5ry64xvb0f7nph3z7860ln8rzynr2"
> +                #:snippet
> +                '(begin
> +                   (delete-file-recursively "bzip2-1.0.8")
> +                   (delete-file "build.rs")
> +                   ;; Inspired by Debian's patch.
> +                   (with-output-to-file "build.rs"
> +                     (lambda _
> +                       (format #t "fn main() @{~@@
> +                        println!(\"cargo:rustc-link-lib=bz2\");~@@
> +                        @}~%"))))))
> +@end lisp
> +
> +In a more complex case, where unbundling one dependency requires other packages,
> +we should package the dependency in @code{(gnu packages rust-sources)} first and
> +point the imported definition to a symbol with the same name.
> +
> +For example we have defined a @code{rust-ring-0.17} in @code{(gnu packages
> +rust-sources)}, then the imported definition in @code{(gnu packages
> +rust-crates)} should be modified to point to it:
> +
> +@lisp
> +(define rust-ring-0.17.14 'rust-ring-0.17)
> +@end lisp
> +
> +When one dependency can be safely removed, modify it to @code{#f}.
> +
> +@lisp
> +(define rust-openssl-src-300.4.2+3.4.1 #f)
> +@end lisp
> +
> +Such modifications are processed by @code{cargo-inputs} procedure.
> +
> +@code{cargo-inputs} can also be configured by keywoard arguments
> +@code{#:crates-module} (default: @code{(gnu packages rust-crates)}, module to
> +resolve imported definitions) and @code{#:sources-module} (default: @code{(gnu
> +packages rust-sources)}, module to resolve modified symbols), which are useful
> +to maintain an independent channel.
> +@end enumerate
> +
> +Example 2: @code{niri}, which is not available on crates.io and depends on
> +development snapshots (also Cargo workspaces in this example).
> +
> +@enumerate
> +@item
> +As we can't ensure compatibility of a development snapshot, before executing
> +@command{cargo generate-lockfile}, we should modify @file{Cargo.toml} to pin it
> +in a known working revision.
> +
> +To use our packaged development snapshots, it's also necessary to modify
> +@file{Cargo.toml} in build environment, the substitution pattern is
> +package-specific.
> +
> +@code{cargo-inputs} returns a list, all list operations apply to it too.
> +
> +@lisp
> +(define-public niri
> +  (package
> +   (name "niri")
> +   (version "25.02")
> +   (source (origin
> +             (method git-fetch)
> +             (uri (git-reference
> +                   (url "https://github.com/YaLTeR/niri")
> +                   (commit (string-append "v" version))))
> +             (file-name (git-file-name name version))
> +             (sha256
> +              (base32
> +               "0vzskaalcz6pcml687n54adjddzgf5r07gggc4fhfsa08h1wfd4r"))))
> +   (build-system cargo-build-system)
> +   (arguments
> +    (list #:install-source? #f
> +          #:phases
> +          #~(modify-phases %standard-phases
> +              (add-after 'unpack 'use-guix-vendored-dependencies
> +                (lambda _
> +                  (substitute* "Cargo.toml"
> +                    (("# version =.*")
> +                     "version = \"*\"")
> +                    (("git.*optional")
> +                     "version = \"*\", optional")
> +                    (("^git = .*")
> +                     "")))))))
> +   (native-inputs
> +    (list pkg-config))
> +   (inputs
> +    (cons* clang
> +           libdisplay-info
> +           libinput-minimal
> +           libseat
> +           libxkbcommon
> +           mesa
> +           pango
> +           pipewire
> +           wayland
> +           (cargo-inputs 'niri)))
> +   (home-page "https://github.com/YaLTeR/niri")
> +   (synopsis "Scrollable-tiling Wayland compositor")
> +   (description
> +    "Niri is a scrollable-tiling Wayland compositor which arranges windows in a
> +scrollable format.  It is considered stable for daily use and performs most
> +functions expected of a Wayland compositor.")
> +   (license license:gpl3)))
> +@end lisp
> +
> +@item
> +@code{niri} also has Cargo workspace dependencies.  When packaging a Cargo
> +workspace, build argument @code{#:cargo-package-crates} is required.
> +
> +@lisp
> +(define-public rust-pipewire-for-niri
> +  (let ((commit "fd3d8f7861a29c2eeaa4c393402e013578bb36d9")
> +        (revision "0"))
> +    (package
> +      (name "rust-pipewire")
> +      (version (git-version "0.8.0" revision commit))
> +      (source
> +       (origin
> +         (method git-fetch)
> +         (uri (git-reference
> +               (url "https://gitlab.freedesktop.org/pipewire/pipewire-rs.git")
> +               (commit commit)))
> +         (file-name (git-file-name name version))
> +         (sha256
> +          (base32 "1hzyhz7xg0mz8a5y9j6yil513p1m610q3j9pzf6q55vdh5mcn79v"))))
> +      (build-system cargo-build-system)
> +      (arguments
> +       (list #:skip-build? #t
> +             #:cargo-package-crates
> +             ''("libspa-sys" "libspa" "pipewire-sys" "pipewire")))
> +      (inputs (cargo-inputs 'rust-pipewire-for-niri))

should these be inputs or propagated-inputs? in rust-ring-0.17 in
rust-sources they are propagated. I would think inputs would be enough
since the Cargo.lock file will let us know which other crates are
necessary for actually building it as a dependency of that package.

> +      (home-page "https://pipewire.org/")
> +      (synopsis "Rust bindings for PipeWire")
> +      (description "This package provides Rust bindings for PipeWire.")
> +      (license license:expat))))
> +@end lisp
> +
> +Don't forget to modify all workspace members in @code{(gnu packages
> +rust-crates)}:
> +
> +@lisp
> +(define rust-pipewire-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
> +(define rust-pipewire-sys-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
> +@dots{}
> +(define rust-libspa-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
> +(define rust-libspa-sys-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
> +@end lisp
> +@end enumerate
> +
> +Example 3: @code{fish}, which combines two build systems.
> +
> +@enumerate
> +@item
> +When building Rust packages in other build systems, we need to add @code{rust}
> +and @code{rust:cargo} to @code{native-inputs}, import and use modules from both
> +build systems, and apply necessary build phases from @code{cargo-build-system}.
> +
> +@lisp
> +(define-public fish
> +  (package
> +    (name "fish")
> +    (version "4.0.0")
> +    (source
> +     (origin
> +       (method url-fetch)
> +       (uri (string-append "https://github.com/fish-shell/fish-shell/"
> +                           "releases/download/" version "/"
> +                           "fish-" version ".tar.xz"))
> +       (sha256
> +        (base32 "1wv9kjwg6ax8m2f85i58l9f9cndshn1f15n8skc68w1mf3cmpnig"))))
> +    (build-system cmake-build-system)
> +    (inputs
> +     (cons* fish-foreign-env ncurses pcre2
> +            python                    ;for fish_config and manpage completions
> +            (cargo-inputs 'fish)))
> +    (native-inputs
> +     (list doxygen groff                ;for 'fish --help'
> +           pkg-config
> +           procps                       ;for the test suite
> +           rust
> +           `(,rust "cargo")))
> +    (arguments
> +     (list #:imported-modules
> +           (append %cargo-build-system-modules
> +                   %cmake-build-system-modules)
> +           #:modules
> +           (((guix build cargo-build-system) #:prefix cargo:)
> +            (guix build cmake-build-system)
> +            (guix build utils))
> +           #:phases
> +           #~(modify-phases %standard-phases
> +               (add-after 'unpack 'use-guix-vendored-dependencies
> +                 (lambda _
> +                   (substitute* "Cargo.toml"
> +                     (("git.*tag.*,")
> +                      "version = \"*\","))))
> +               (add-after 'unpack 'prepare-cargo-build-system
> +                 (lambda args
> +                   (for-each
> +                    (lambda (phase)
> +                      (format #t "Running cargo phase: ~a~%" phase)
> +                      (apply (assoc-ref cargo:%standard-phases phase)
> +                             args))
> +                    '(unpack-rust-crates
> +                      configure
> +                      check-for-pregenerated-files
> +                      patch-cargo-checksums)))))))
> +    (synopsis "The friendly interactive shell")
> +    (description
> +     "Fish (friendly interactive shell) is a shell focused on interactive use,
> +discoverability, and friendliness.  Fish has very user-friendly and powerful
> +tab-completion, including descriptions of every completion, completion of
> +strings with wildcards, and many completions for specific commands.  It also
> +has extensive and discoverable help.  A special @@command@{help@} command gives
> +access to all the fish documentation in your web browser.  Other features
> +include smart terminal handling based on terminfo, an easy to search history,
> +and syntax highlighting.")
> +    (home-page "https://fishshell.com/")
> +    (license license:gpl2)))
> +@end lisp
> +@end enumerate
>  
>  
>  @node Elm Packages
> -- 
> 2.48.1
>
  

On Wed, 19 Mar 2025 14:52:32 +0800,
Efraim Flashner wrote:
>
> > +Let's demonstrate the packaging workflow by examples, note that package-specific
> > +issues are not involved here.
> > +
> > +In preparation, we'll add the following packages to our environment:
> > +
> > +@example
> > +guix shell rust rust:cargo cargo-audit cargo-license
> > +@end example
>
> I was able to run cargo-audit as 'cargo-audit audit ...' but I wouldn't
> actually recommend that to anyone.  I'm not sure we actually need rust
> in the shell but I don't have a strong preference.  More thinking aloud,

‘cargo generate-lockfile’ sometimes requires rust.

> perhaps we should adjust rust:cargo to automatically pull in rust:out?

Sounds reasonable.

> I also find it amusing to suggest using cargo-audit when it's not (yet)
> in guix and we immediately follow up with importing it.

XD yes, it's funny to use cargo-audit audit cargo-audit, and cargo-license to
check license of cargo-license.

> I could see an argument for putting this part in the cookbook instead of
> here but I feel strongly that it should be here.
> > [...]
> > +@item
> > +@code{niri} also has Cargo workspace dependencies.  When packaging a Cargo
> > +workspace, build argument @code{#:cargo-package-crates} is required.
> > +
> > +@lisp
> > +(define-public rust-pipewire-for-niri
> > +  (let ((commit "fd3d8f7861a29c2eeaa4c393402e013578bb36d9")
> > +        (revision "0"))
> > +    (package
> > +      (name "rust-pipewire")
> > +      (version (git-version "0.8.0" revision commit))
> > +      (source
> > +       (origin
> > +         (method git-fetch)
> > +         (uri (git-reference
> > +               (url "https://gitlab.freedesktop.org/pipewire/pipewire-rs.git")
> > +               (commit commit)))
> > +         (file-name (git-file-name name version))
> > +         (sha256
> > +          (base32 "1hzyhz7xg0mz8a5y9j6yil513p1m610q3j9pzf6q55vdh5mcn79v"))))
> > +      (build-system cargo-build-system)
> > +      (arguments
> > +       (list #:skip-build? #t
> > +             #:cargo-package-crates
> > +             ''("libspa-sys" "libspa" "pipewire-sys" "pipewire")))
> > +      (inputs (cargo-inputs 'rust-pipewire-for-niri))
>
> should these be inputs or propagated-inputs? in rust-ring-0.17 in
> rust-sources they are propagated. I would think inputs would be enough
> since the Cargo.lock file will let us know which other crates are
> necessary for actually building it as a dependency of that package.

For rust-ring-0.17 the propagation is necessary, I think this is because our
packaged version is a bit older than the one used in generated lockfile.
  

Hi,

Hilton Chain <hako@ultrarare.space> writes:

> * doc/contributing.texi (Packaging Guidelines)[Rust Crates]: Update
> documentation.
>
> Change-Id: Ic0c6378cf5f5df97d6f8bdd040b486be62c7bddc
> ---
>  doc/contributing.texi | 415 +++++++++++++++++++++++++++++++++++++++---
>  1 file changed, 390 insertions(+), 25 deletions(-)
>
> diff --git a/doc/contributing.texi b/doc/contributing.texi
> index ab4f30d54b..837074dead 100644
> --- a/doc/contributing.texi
> +++ b/doc/contributing.texi
> @@ -1600,34 +1600,399 @@ Rust Crates
>  @subsection Rust Crates
>  
>  @cindex rust
> -Rust programs standing for themselves are named as any other package, using the
> -lowercase upstream name.
> +As currently there's no achievable way to reuse Rust packages as pre-compiled
> +inputs for other packages at a distribution scale, and our previous approach on
> +making all Rust dependencies as full packages doesn't cope well with Rust
> +ecosystem thus causing issues on both contribution and maintenance sides, we
> +have switched to a different packaging model.

I'd drop the above paragraph; it won't age well.

> +Rust programs (binary crates) and dependencies (library crates) are treated
> +separately.  We put our main efforts into programs and only package Rust
> +dependencies as sources, utilizing automation with a manual focus on unbundling
> +vendored dependencies.  The following paragraphs will explain them and give
> +several examples.
> +
> +Rust programs are treated like any other package and named using the lowercase
> +upstream name.  When using the Cargo build system (@pxref{Build Systems,
> +@code{cargo-build-system}}), Rust programs should have @code{#:install-source?}
> +argument set to @code{#f}, as this argument only makes sense for dependencies.
> +When the package source is a Cargo workspace, @code{#:cargo-install-paths} must
> +be set to enable relevant support.
> +
> +Rust dependencies are managed in two modules:
>  
> -To prevent namespace collisions we prefix all other Rust packages with the
> -@code{rust-} prefix.  The name should be changed to lowercase as appropriate and
> -dashes should remain in place.
> +@enumerate
> +@item
> +@code{(gnu packages rust-crates)}, storing source definitions imported from Rust
> +programs' @file{Cargo.lock} via the lockfile importer (@pxref{Invoking guix
> +import, crate, @code{--lockfile=@var{file}}}).
> +
> +Imported definitions must be checked and have vendored sources unbundled before
> +contributing to Guix.  Naturally, this module serves as a store for both sources

s/contributing/being contributed/.  The next sentence (Naturally, ...)
seems superfluous to me.

> +This module is managed by the Rust team (@pxref{Teams}) to ensure there's always
> +one version containing all changes from other branches, so that the maintained
> +version can be used directly in case of merge conflicts, thus coordination is
> +required for other committers to modify it.

I think the above could be simplified like: This module is managed by
the Rust team, via the @file{etc/teams/rust/rust-crates.tmpl} template
file and @file{etc/teams/rust/rust-crates.tmpl} cleanup script.

[...]

> +@item
> +@code{(gnu packages rust-sources)}, storing more complex definitions that need
> +to be full packages.  This includes Rust dependencies requiring external inputs
> +to unbundle and Cargo workspaces.
> +
> +These dependencies should have the @code{#:skip-build?} argument set to
> +@code{#t}.  For Cargo workspaces, @code{#:cargo-package-crates} must be set.
> +
> +Since they are added manually, the following naming convention applies:
> +
> +To prevent namespace collisions they are named with @code{rust-} prefix.  The
> +name should be changed to lowercase as appropriate and dashes should remain in
> +place.
>  
>  In the rust ecosystem it is common for multiple incompatible versions of a
> -package to be used at any given time, so all package definitions should have a
> -versioned suffix.  The versioned suffix is the left-most non-zero digit (and
> -any leading zeros, of course).  This follows the ``caret'' version scheme
> -intended by Cargo.  Examples@: @code{rust-clap-2}, @code{rust-rand-0.6}.
> -
> -Because of the difficulty in reusing rust packages as pre-compiled inputs for
> -other packages the Cargo build system (@pxref{Build Systems,
> -@code{cargo-build-system}}) presents the @code{#:cargo-inputs} and
> -@code{cargo-development-inputs} keywords as build system arguments.  It would be
> -helpful to think of these as similar to @code{propagated-inputs} and
> -@code{native-inputs}.  Rust @code{dependencies} and @code{build-dependencies}
> -should go in @code{#:cargo-inputs}, and @code{dev-dependencies} should go in
> -@code{#:cargo-development-inputs}.  If a Rust package links to other libraries
> -then the standard placement in @code{inputs} and the like should be used.
> -
> -Care should be taken to ensure the correct version of dependencies are used; to
> -this end we try to refrain from skipping the tests or using @code{#:skip-build?}
> -when possible.  Of course this is not always possible, as the package may be
> -developed for a different Operating System, depend on features from the Nightly
> -Rust compiler, or the test suite may have atrophied since it was released.
> +package to be used at any given time, so all dependencies should have a
> +versioned suffix.  The versioned suffix is the left-most non-zero digit (and any
> +leading zeros, of course).  This follows the ``caret'' version scheme intended
> +by Cargo.  Examples@: @code{rust-clap-2}, @code{rust-rand-0.6}.

@: is not needed before a colon.  Refer to (info "(texinfo) Not Ending a Sentence").

> +In practice we are usually packaging development snapshots of Rust dependencies
> +specifically for some Rust programs, and can't simply identity them by version.
> +In this case we can use a @code{for-@var{program}} suffix.  Examples@:

I'd join the last sentence with the previous one, e.g. [...] suffix, for
example: [...]

> +@code{rust-pipewire-for-niri}, @code{rust-pubgrub-for-uv}.
> +@end enumerate
> +
> +Let's demonstrate the packaging workflow by examples, note that package-specific
> +issues are not involved here.

I'd drop the part after examples.  

> +In preparation, we'll add the following packages to our environment:
> +
> +@example
> +guix shell rust rust:cargo cargo-audit cargo-license
> +@end example
> +
> +Example 1: @code{cargo-audit}, which is published on the crates.io Rust package
> +repository @uref{https://crates.io, crates.io}.

I'd replace the textual crates.io with the later @uref, end place the
period after 'repository'.

This verbose how-to Rust packaging, while useful, would better belong in
the cookbook, in my opinion.  The GNU Guix Cookbook document section can
then be referenced from here, for example:

@ref{Rust Packaging Examples,,,guix-cookbook,GNU Guix Cookbook} for real
life examples of packaging Rust applications.

> +@enumerate
> +@item
> +We can generate a draft definition via the crates.io importer (@pxref{Invoking
> +guix import, crate}).  In the end we'll have the following definiton:
> +
> +@lisp
> +(define-public cargo-audit
> +  (package
> +    (name "cargo-audit")
> +    (version "0.21.2")
> +    (source
> +     (origin
> +       (method url-fetch)
> +       (uri (crate-uri "cargo-audit" version))
> +       (file-name (string-append name "-" version ".tar.gz"))
> +       (sha256
> +        (base32 "1a00yqpckkw86zh2hg7ra82c5fx0ird5766dyynimbvqiwg2ps0n"))))
> +    (build-system cargo-build-system)
> +    (arguments (list #:install-source? #f))
> +    (inputs (cargo-inputs 'cargo-audit))
> +    (home-page "https://rustsec.org/")
> +    (synopsis "Audit Cargo.lock for crates with security vulnerabilities")
> +    (description
> +     "This package provides a Cargo subcommand, @@command@{cargo audit@}, to
> +audit @@file@{Cargo.lock@} for crates with security vulnerabilities.")
> +    (license (list license:asl2.0 license:expat))))
> +@end lisp
> +
> +@code{cargo-inputs} is a procedure defined in @code{guix build-system cargo},
> +facilitating dependency modification.  @code{'cargo-audit} used here must be a
> +unique identifier, usually same to the program's variable name.

I'd reword to: [...], usually matching the variable name of the package.

> +
> +@item
> +Unpack package source and navigate to the unpacked directory, then execute the
> +following commands:
> +
> +@example
> +cargo generate-lockfile
> +cargo audit
> +cargo license
> +@end example
> +
> +@command{cargo generate-lockfile} updates dependencies to compatible versions,
> +@command{cargo audit} checks known vulnerabilities and @command{cargo license}
> +checks licenses of all dependencies.
> +
> +We must have an acceptable output of @command{cargo audit} and ensure all
> +dependencies are licensed with our supported licenses (@pxref{Defining Packages,
> +@code{license}}).
> +
> +@item
> +Import dependencies from previously generated lockfile:
> +
> +@example
> +guix import --insert=gnu/packages/rust-crates.scm \
> +    crate --lockfile=/path/to/Cargo.lock cargo-audit
> +@end example
> +
> +@code{cargo-audit} used here must be consistent with the identifier used for
> +@code{cargo-inputs}.
> +
> +At this stage, @code{cargo-audit} is buildable.
> +
> +@item
> +Finally we'll unbundle vendored sources.  The lockfile importer inserts
> +@code{TODO:} comments to dependencies with high probability of bundled sources.
> +@code{cargo-build-system} also performs additional check in its
> +@code{check-for-pregenerated-files} phase:
> +
> +@example
> +$ ./pre-inst-env guix build cargo-audit
> +@dots{}
> +starting phase `check-for-pregenerated-files'
> +Searching for binary files...
> +./guix-vendor/rust-addr2line-0.21.0.tar.gz/rustfmt.toml
> +./guix-vendor/rust-arc-swap-1.7.1.tar.gz/rustfmt.toml
> +./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/dictionary-rust
> +./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/dictionary-rust-other
> +./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/lib.rs.zst
> +./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/long-window-size-lib.rs.zst
> +@dots{}
> +@end example
> +
> +Although dependencies in @code{(gnu packages rust-crates)} are not exported, we
> +can still select them in Guix command-line interface through expression:

select them *via* the Guix command-line interface [...]

> +
> +@example
> +guix build --expression='(@@@@ (gnu packages rust-crates) rust-ring-0.17.14)'
> +@end example
> +
> +For most dependencies, a snippet is sufficient:
> +
> +@lisp
> +(define rust-bzip2-sys-0.1.13+1.0.8
> +  (crate-source "bzip2-sys" "0.1.13+1.0.8"
> +                "056c39pgjh4272bdslv445f5ry64xvb0f7nph3z7860ln8rzynr2"
> +                #:snippet
> +                '(begin
> +                   (delete-file-recursively "bzip2-1.0.8")
> +                   (delete-file "build.rs")
> +                   ;; Inspired by Debian's patch.
> +                   (with-output-to-file "build.rs"
> +                     (lambda _
> +                       (format #t "fn main() @{~@@
> +                        println!(\"cargo:rustc-link-lib=bz2\");~@@
> +                        @}~%"))))))
> +@end lisp
> +
> +In a more complex case, where unbundling one dependency requires other packages,
> +we should package the dependency in @code{(gnu packages rust-sources)} first and
> +point the imported definition to a symbol with the same name.
> +
> +For example we have defined a @code{rust-ring-0.17} in @code{(gnu packages
> +rust-sources)}, then the imported definition in @code{(gnu packages
> +rust-crates)} should be modified to point to it:
> +
> +@lisp
> +(define rust-ring-0.17.14 'rust-ring-0.17)
> +@end lisp
> +
> +When one dependency can be safely removed, modify it to @code{#f}.
> +
> +@lisp
> +(define rust-openssl-src-300.4.2+3.4.1 #f)
> +@end lisp
> +
> +Such modifications are processed by @code{cargo-inputs} procedure.
> +
> +@code{cargo-inputs} can also be configured by keywoard arguments
> +@code{#:crates-module} (default: @code{(gnu packages rust-crates)}, module to
> +resolve imported definitions) and @code{#:sources-module} (default: @code{(gnu
> +packages rust-sources)}, module to resolve modified symbols), which are useful
> +to maintain an independent channel.
> +@end enumerate
> +
> +Example 2: @code{niri}, which is not available on crates.io and depends on

It seems to me these examples should have their own structure node
(subsection or subsubsection, as appropriate).

> +development snapshots (also Cargo workspaces in this example).
> +
> +@enumerate
> +@item
> +As we can't ensure compatibility of a development snapshot, before executing
> +@command{cargo generate-lockfile}, we should modify @file{Cargo.toml} to pin it
> +in a known working revision.
> +
> +To use our packaged development snapshots, it's also necessary to modify
> +@file{Cargo.toml} in build environment, the substitution pattern is
> +package-specific.

The above sentence appears incomplete or disconnected.  Perhaps, 

  To use our packaged development snapshots, it's also necessary to modify
  @file{Cargo.toml} in a phase, with a package-specific substitution
  pattern.

> +@code{cargo-inputs} returns a list, all list operations apply to it too.

This appears to be out of context?

> +
> +@lisp
> +(define-public niri
> +  (package
> +   (name "niri")
> +   (version "25.02")
> +   (source (origin
> +             (method git-fetch)
> +             (uri (git-reference
> +                   (url "https://github.com/YaLTeR/niri")
> +                   (commit (string-append "v" version))))
> +             (file-name (git-file-name name version))
> +             (sha256
> +              (base32
> +               "0vzskaalcz6pcml687n54adjddzgf5r07gggc4fhfsa08h1wfd4r"))))
> +   (build-system cargo-build-system)
> +   (arguments
> +    (list #:install-source? #f
> +          #:phases
> +          #~(modify-phases %standard-phases
> +              (add-after 'unpack 'use-guix-vendored-dependencies
> +                (lambda _
> +                  (substitute* "Cargo.toml"
> +                    (("# version =.*")
> +                     "version = \"*\"")
> +                    (("git.*optional")
> +                     "version = \"*\", optional")
> +                    (("^git = .*")
> +                     "")))))))
> +   (native-inputs
> +    (list pkg-config))
> +   (inputs
> +    (cons* clang
> +           libdisplay-info
> +           libinput-minimal
> +           libseat
> +           libxkbcommon
> +           mesa
> +           pango
> +           pipewire
> +           wayland
> +           (cargo-inputs 'niri)))
> +   (home-page "https://github.com/YaLTeR/niri")
> +   (synopsis "Scrollable-tiling Wayland compositor")
> +   (description
> +    "Niri is a scrollable-tiling Wayland compositor which arranges windows in a
> +scrollable format.  It is considered stable for daily use and performs most
> +functions expected of a Wayland compositor.")
> +   (license license:gpl3)))

I think this should be gpl3+, as the license file carries the original
'or any later version' and apparently no further directions to limit
gpl3 only.

> +@end lisp
> +
> +@item
> +@code{niri} also has Cargo workspace dependencies.  When packaging a Cargo
> +workspace, build argument @code{#:cargo-package-crates} is required.

Hm.  Does that mean we're still in a situation where some inputs are not
true Guix inputs, and some Rust-packaging oddity that our tooling
doesn't handle (e.g. 'guix graph' ?).

> +@lisp
> +(define-public rust-pipewire-for-niri
> +  (let ((commit "fd3d8f7861a29c2eeaa4c393402e013578bb36d9")
> +        (revision "0"))
> +    (package
> +      (name "rust-pipewire")
> +      (version (git-version "0.8.0" revision commit))
> +      (source
> +       (origin
> +         (method git-fetch)
> +         (uri (git-reference
> +               (url "https://gitlab.freedesktop.org/pipewire/pipewire-rs.git")
> +               (commit commit)))
> +         (file-name (git-file-name name version))
> +         (sha256
> +          (base32 "1hzyhz7xg0mz8a5y9j6yil513p1m610q3j9pzf6q55vdh5mcn79v"))))
> +      (build-system cargo-build-system)
> +      (arguments
> +       (list #:skip-build? #t
> +             #:cargo-package-crates
> +             ''("libspa-sys" "libspa" "pipewire-sys" "pipewire")))
> +      (inputs (cargo-inputs 'rust-pipewire-for-niri))
> +      (home-page "https://pipewire.org/")
> +      (synopsis "Rust bindings for PipeWire")
> +      (description "This package provides Rust bindings for PipeWire.")
> +      (license license:expat))))
> +@end lisp
> +
> +Don't forget to modify all workspace members in @code{(gnu packages
> +rust-crates)}:
> +
> +@lisp
> +(define rust-pipewire-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
> +(define rust-pipewire-sys-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
> +@dots{}
> +(define rust-libspa-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
> +(define rust-libspa-sys-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
> +@end lisp
> +@end enumerate
> +
> +Example 3: @code{fish}, which combines two build systems.
> +
> +@enumerate
> +@item
> +When building Rust packages in other build systems, we need to add @code{rust}
> +and @code{rust:cargo} to @code{native-inputs}, import and use modules from both
> +build systems, and apply necessary build phases from @code{cargo-build-system}.
> +
> +@lisp
> +(define-public fish
> +  (package
> +    (name "fish")
> +    (version "4.0.0")
> +    (source
> +     (origin
> +       (method url-fetch)
> +       (uri (string-append "https://github.com/fish-shell/fish-shell/"
> +                           "releases/download/" version "/"
> +                           "fish-" version ".tar.xz"))
> +       (sha256
> +        (base32 "1wv9kjwg6ax8m2f85i58l9f9cndshn1f15n8skc68w1mf3cmpnig"))))
> +    (build-system cmake-build-system)
> +    (inputs
> +     (cons* fish-foreign-env ncurses pcre2
> +            python                    ;for fish_config and manpage completions
> +            (cargo-inputs 'fish)))
> +    (native-inputs
> +     (list doxygen groff                ;for 'fish --help'
> +           pkg-config
> +           procps                       ;for the test suite
> +           rust
> +           `(,rust "cargo")))
> +    (arguments
> +     (list #:imported-modules
> +           (append %cargo-build-system-modules
> +                   %cmake-build-system-modules)
> +           #:modules
> +           (((guix build cargo-build-system) #:prefix cargo:)
> +            (guix build cmake-build-system)
> +            (guix build utils))
> +           #:phases
> +           #~(modify-phases %standard-phases
> +               (add-after 'unpack 'use-guix-vendored-dependencies
> +                 (lambda _
> +                   (substitute* "Cargo.toml"
> +                     (("git.*tag.*,")
> +                      "version = \"*\","))))
> +               (add-after 'unpack 'prepare-cargo-build-system
> +                 (lambda args
> +                   (for-each
> +                    (lambda (phase)
> +                      (format #t "Running cargo phase: ~a~%" phase)
> +                      (apply (assoc-ref cargo:%standard-phases phase)
> +                             args))
> +                    '(unpack-rust-crates
> +                      configure
> +                      check-for-pregenerated-files
> +                      patch-cargo-checksums)))))))
> +    (synopsis "The friendly interactive shell")
> +    (description
> +     "Fish (friendly interactive shell) is a shell focused on interactive use,
> +discoverability, and friendliness.  Fish has very user-friendly and powerful
> +tab-completion, including descriptions of every completion, completion of
> +strings with wildcards, and many completions for specific commands.  It also
> +has extensive and discoverable help.  A special @@command@{help@} command gives
> +access to all the fish documentation in your web browser.  Other features
> +include smart terminal handling based on terminfo, an easy to search history,
> +and syntax highlighting.")
> +    (home-page "https://fishshell.com/")
> +    (license license:gpl2)))
> +@end lisp
> +@end enumerate

Thanks for taking the time to author all of this!
  

On Wed, 19 Mar 2025 21:41:39 +0800,
Maxim Cournoyer wrote:
>
> [...]
> > +@end lisp
> > +
> > +@item
> > +@code{niri} also has Cargo workspace dependencies.  When packaging a Cargo
> > +workspace, build argument @code{#:cargo-package-crates} is required.
>
> Hm.  Does that mean we're still in a situation where some inputs are not
> true Guix inputs, and some Rust-packaging oddity that our tooling
> doesn't handle (e.g. 'guix graph' ?).

This argument is meant to specify a order, workspace members may depend on each
other, so the dependencies should be packaged and put into "guix-vendor" first.

The naming might be confusing but I can't come up with a better one :(

cargo- : cargo-build-system specific
package- : package phase
crates : list of crate names

Rust <package>s will have all their Rust dependencies in inputs field but most
of them are origins, so there won't be much difference in terms of ‘guix graph’.

--8<---------------cut here---------------start------------->8---
$ guix graph niri | grep '"rust-.*'
  "140032307245408" [label = "rust-bindgen-cli@0.71.1", shape = box, fontname = sans];
  "140032307245936" [label = "rust-cbindgen@0.26.0", shape = box, fontname = sans];
  "140032240396352" [label = "rust-ring@0.17.8", shape = box, fontname = sans];
  "140032307276192" [label = "rust-cargo-c@0.10.9+cargo-0.85.0", shape = box, fontname = sans];
  "140032240396176" [label = "rust-smithay@0.4.0-0.0cd3345", shape = box, fontname = sans];
  "140032240396880" [label = "rust-pipewire@0.8.0-0.fd3d8f7", shape = box, fontname = sans];
$ guix graph niri --type=bag-with-origins | grep '"rust-.*'
  "/gnu/store/kzl3bv8yfczwg18kcx649p7nmf8n2m55-rust-bindgen-cli-0.71.1.drv" [label = "rust-bindgen-cli@0.71.1", shape = box, fontname = sans];
  "/gnu/store/d28511hifkbpzr8hrnngb5hlsy7js2na-rust-windows-x86-64-msvc-0.52.6.tar.zst.drv" [label = "rust-windows-x86-64-msvc-0.52.6.tar.gz", shape = box, fontname = sans];
  "/gnu/store/7kh3innc661lf8fcya6nziglv93hy2yl-rust-windows-x86-64-gnullvm-0.52.6.tar.zst.drv" [label = "rust-windows-x86-64-gnullvm-0.52.6.tar.gz", shape = box, fontname = sans];
  "/gnu/store/fc73asklfa1q29g46bc8gpkggqwpc83w-rust-windows-x86-64-gnu-0.52.6.tar.zst.drv" [label = "rust-windows-x86-64-gnu-0.52.6.tar.gz", shape = box, fontname = sans];
  "/gnu/store/ca8laa955hw96r49dg5pzsl5pjxqs37j-rust-windows-i686-msvc-0.52.6.tar.zst.drv" [label = "rust-windows-i686-msvc-0.52.6.tar.gz", shape = box, fontname = sans];
  "/gnu/store/9czqdmfglvmag8xi69a4jyabblhff72j-rust-windows-i686-gnullvm-0.52.6.tar.zst.drv" [label = "rust-windows-i686-gnullvm-0.52.6.tar.gz", shape = box, fontname = sans];
  "/gnu/store/wjmvcdbqjm1zxqaih1c5f52d7990h40p-rust-windows-i686-gnu-0.52.6.tar.zst.drv" [label = "rust-windows-i686-gnu-0.52.6.tar.gz", shape = box, fontname = sans];
  "/gnu/store/w16561c65nzsqmif3ww634mijfk835gl-rust-windows-aarch64-msvc-0.52.6.tar.zst.drv" [label = "rust-windows-aarch64-msvc-0.52.6.tar.gz", shape = box, fontname = sans];
  "/gnu/store/aaw14iijajsxv5s3p6g58py1ippbr3ql-rust-windows-aarch64-gnullvm-0.52.6.tar.zst.drv" [label = "rust-windows-aarch64-gnullvm-0.52.6.tar.gz", shape = box, fontname = sans];
  "/gnu/store/xp4i48sdij8qmq2xwpq8dq5h4ax0mzgs-rust-windows-targets-0.52.6.tar.gz.drv" [label = "rust-windows-targets-0.52.6.tar.gz", shape = box, fontname = sans];
  "/gnu/store/wlfg3iy55ph6ffwp96sqyzg22isx8573-rust-windows-sys-0.59.0.tar.gz.drv" [label = "rust-windows-sys-0.59.0.tar.gz", shape = box, fontname = sans];
  "/gnu/store/q8x8721s5r6rb8nrb19jkwbszrqih26f-rust-winapi-util-0.1.9.tar.gz.drv" [label = "rust-winapi-util-0.1.9.tar.gz", shape = box, fontname = sans];
  "/gnu/store/qynjbqr3md18m6q4ynrpvglfa8xjys0d-rust-utf8parse-0.2.2.tar.gz.drv" [label = "rust-utf8parse-0.2.2.tar.gz", shape = box, fontname = sans];
[...]
--8<---------------cut here---------------end--------------->8---

Thanks for the review!  I'll prepare v3 soon :)