[bug#77093,rust-team,v2,00/17] New Rust packaging workflow based on lockfile importer.

Message ID cover.1742306960.git.hako@ultrarare.space
State New
Headers

Commit Message

Hilton Chain March 18, 2025, 2:26 p.m. UTC
  V1 -> V2:
  * Drop jemalloc environment variable patch.
  * Remove use of RUSTC_BOOTSTRAP.
  * Adjust documentation.
  Diff inserted at end.

Efraim Flashner (1):
  build/cargo: Don't try to unpack sanity-check.py.

Hilton Chain (16):
  build/cargo: Pass ‘--offline’ to cargo.
  build/cargo: Print out all non-empty binary files.
  build-system: cargo: Support packaging Cargo workspace.
  build-system: cargo: Support installing Cargo workspace.
  build/cargo: Set default value of arguments for build phases.
  build/cargo: Support non-workspace directory source inputs.
  scripts: import: Document argument for ‘--insert’ option in help
    message.
  scripts: import: Add two newlines for ‘--insert’ option.
  scripts: import: Support expressions defined by 'define.
  scripts: import: Pass "--insert" to importers.
  scripts: import: Skip existing definition for ‘--insert’ option.
  import: crate: crate-name->package-name: Move to (guix build-system
    cargo).
  build-system: cargo: Add ‘cargo-inputs’.
  import: crate: Add Cargo.lock parser.
  import: crate: Add ‘--lockfile’ option.
  doc: Document lockfile importer based Rust packaging workflow.

 Makefile.am                       |   1 +
 doc/contributing.texi             | 415 ++++++++++++++++++++++++++++--
 doc/guix.texi                     |  24 ++
 etc/teams/rust/cleanup-crates.sh  |  38 +++
 etc/teams/rust/rust-crates.tmpl   |  42 +++
 gnu/local.mk                      |   2 +
 gnu/packages/rust-crates.scm      |  42 +++
 gnu/packages/rust-sources.scm     |  29 +++
 guix/build-system/cargo.scm       |  57 +++-
 guix/build/cargo-build-system.scm | 151 ++++++++---
 guix/import/crate.scm             |  81 +++++-
 guix/import/crate/cargo-lock.scm  | 105 ++++++++
 guix/scripts/import.scm           |  53 ++--
 guix/scripts/import/crate.scm     |  58 ++++-
 guix/utils.scm                    |  27 +-
 tests/crate.scm                   |  88 +++++++
 16 files changed, 1107 insertions(+), 106 deletions(-)
 create mode 100755 etc/teams/rust/cleanup-crates.sh
 create mode 100644 etc/teams/rust/rust-crates.tmpl
 create mode 100644 gnu/packages/rust-crates.scm
 create mode 100644 gnu/packages/rust-sources.scm
 create mode 100644 guix/import/crate/cargo-lock.scm


base-commit: b54a9ca849f013300c633fb79d80bc754f6b28a2
--
2.48.1

--8<---------------cut here---------------start------------->8---
--8<---------------cut here---------------end--------------->8---
  

Patch

diff --git a/doc/contributing.texi b/doc/contributing.texi
index e7c5116a3d..837074dead 100644
--- a/doc/contributing.texi
+++ b/doc/contributing.texi
@@ -1607,15 +1607,15 @@  Rust Crates
 have switched to a different packaging model.

 Rust programs (binary crates) and dependencies (library crates) are treated
-separately.  We put main efforts into programs and only package Rust
-dependencies as sources, utilizing automation with manual focus on unbundling
+separately.  We put our main efforts into programs and only package Rust
+dependencies as sources, utilizing automation with a manual focus on unbundling
 vendored dependencies.  The following paragraphs will explain them and give
 several examples.

 Rust programs are treated like any other package and named using the lowercase
 upstream name.  When using the Cargo build system (@pxref{Build Systems,
 @code{cargo-build-system}}), Rust programs should have @code{#:install-source?}
-argument set to @code{#f}, as this argument only makes sense to dependencies.
+argument set to @code{#f}, as this argument only makes sense for dependencies.
 When the package source is a Cargo workspace, @code{#:cargo-install-paths} must
 be set to enable relevant support.

@@ -1631,21 +1631,21 @@  Rust Crates
 contributing to Guix.  Naturally, this module serves as a store for both sources
 and unbundling strategies.

-This module is managed by Rust team (@pxref{Teams}) to ensure there's always one
-version containing all changes from other branches, so that the maintained
+This module is managed by the Rust team (@pxref{Teams}) to ensure there's always
+one version containing all changes from other branches, so that the maintained
 version can be used directly in case of merge conflicts, thus coordination is
 required for other committers to modify it.

 Guix source ships template @file{etc/teams/rust/rust-crates.tmpl} and cleanup
-script @file{etc/teams/rust/cleanup-crates.sh} for this moudle.
+script @file{etc/teams/rust/cleanup-crates.sh} for this module.

 @item
 @code{(gnu packages rust-sources)}, storing more complex definitions that need
 to be full packages.  This includes Rust dependencies requiring external inputs
 to unbundle and Cargo workspaces.

-These dependencies should have @code{#:skip-build?} argument set to @code{#t}.
-For Cargo workspaces, @code{#:cargo-package-crates} must be set.
+These dependencies should have the @code{#:skip-build?} argument set to
+@code{#t}.  For Cargo workspaces, @code{#:cargo-package-crates} must be set.

 Since they are added manually, the following naming convention applies:

@@ -1668,7 +1668,7 @@  Rust Crates
 Let's demonstrate the packaging workflow by examples, note that package-specific
 issues are not involved here.

-In preparation, we'll adding the following packages to our environment:
+In preparation, we'll add the following packages to our environment:

 @example
 guix shell rust rust:cargo cargo-audit cargo-license
@@ -1747,7 +1747,7 @@  Rust Crates
 @code{check-for-pregenerated-files} phase:

 @example
-$ guix build cargo-audit
+$ ./pre-inst-env guix build cargo-audit
 @dots{}
 starting phase `check-for-pregenerated-files'
 Searching for binary files...
diff --git a/doc/guix.texi b/doc/guix.texi
index 029c8cf59c..218c2ba630 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -9365,14 +9365,14 @@  Build Systems
 sources, to ease in future hacking on rust packages.

 This build system supports cargo workspaces.  Parameter
-@code{#:cargo-package-crates} (default: @code{''()}) allows specifying
-names of library crates to package in @code{package} phase.  Specified
-crates are packaged from left to right, in case there's dependency among
-them.  For example, specifying @code{''("pcre2-sys" "pcre2")} will
-package @code{"pcre2-sys"} first and then @code{"pcre2"}.  Parameter
-@code{#:cargo-install-paths} (default: @code{''()}) allows specifying
-paths of binary crates to install in @code{install} phase,
-@code{''("crates/atuin")}, for example.
+@code{#:cargo-package-crates} (default: @code{''()}) allows specifying names of
+library crates to package in the @code{package} phase.  Specified crates are
+packaged from left to right, in case there's dependency among them.  For
+example, specifying @code{''("pcre2-sys" "pcre2")} will package
+@code{"pcre2-sys"} first and then @code{"pcre2"}.  Parameter
+@code{#:cargo-install-paths} (default: @code{''()}) allows specifying paths of
+binary crates to install in the @code{install} phase, @code{''("crates/atuin")},
+for example.
 @end defvar

 @defvar chicken-build-system
@@ -14705,19 +14705,18 @@  Invoking guix import
 a comment.
 @item --lockfile=@var{file}
 @itemx -f @var{file}
-When @option{--lockfile} is specified, importer will ignore other
-options and won't output package definitions, instead generating source
-definition for all dependencies in @var{file}, a @file{Cargo.lock} file.
-For example:
+When @option{--lockfile} is specified, the importer will ignore other options
+and won't output package definitions, instead generating source definitions for
+all dependencies in @var{file}, a @file{Cargo.lock} file.  For example:

 @example
 guix import crate --lockfile=/path/to/Cargo.lock my-package
 @end example

-generates sources from @file{/path/to/Cargo.lock}, and a list
-@code{my-package-cargo-inputs} referencing these sources.  The generated
-list is intended for package's base @code{inputs}, replacing
-@code{#:cargo-inputs} and @code{#:cargo-development-inputs}.
+generates sources from @file{/path/to/Cargo.lock} and a list
+@code{my-package-cargo-inputs} referencing these sources.  The generated list is
+intended for the package's @code{inputs}, replacing @code{#:cargo-inputs} and
+@code{#:cargo-development-inputs}.
 @end table

 @item elm
diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
index b547421b88..41adc03752 100644
--- a/guix/build/cargo-build-system.scm
+++ b/guix/build/cargo-build-system.scm
@@ -248,12 +248,6 @@  (define* (configure #:key inputs
   (setenv "LIBSSH2_SYS_USE_PKG_CONFIG" "1")
   (setenv "SODIUM_USE_PKG_CONFIG" "1")
   (setenv "ZSTD_SYS_USE_PKG_CONFIG" "1")
-  ;; This flag is needed when not using the bundled jemalloc.
-  ;; https://github.com/tikv/jemallocator/issues/19
-  (setenv "CARGO_FEATURE_UNPREFIXED_MALLOC_ON_SUPPORTED_PLATFORMS" "1")
-  (when (assoc-ref inputs "jemalloc")
-    (setenv "JEMALLOC_OVERRIDE"
-            (string-append (assoc-ref inputs "jemalloc") "/lib/libjemalloc.so")))
   (when (assoc-ref inputs "openssl")
     (setenv "OPENSSL_DIR" (assoc-ref inputs "openssl")))
   (when (assoc-ref inputs "gettext")
@@ -356,23 +350,17 @@  (define* (package #:key

         (if (null? cargo-package-crates)
             (apply invoke `("cargo" "package" "--offline" ,@cargo-package-flags))
-            ;; Use unstable feature ‘--registry’.
             (begin
-              (setenv "RUSTC_BOOTSTRAP" "1")
               (for-each
                (lambda (pkg)
                  (apply invoke "cargo" "package" "--offline" "--package" pkg
-                        "--registry" "crates-io" "-Z" "package-workspace"
                         cargo-package-flags)
                  (for-each
                   (lambda (crate)
                     (invoke "tar" "xzf" crate "-C" vendor-dir))
-                  (begin
-                    (delete-file-recursively "target/package/tmp-registry")
-                    (find-files "target/package" "\\.crate$")))
+                  (find-files "target/package" "\\.crate$"))
                  (patch-cargo-checksums #:vendor-dir vendor-dir))
-               cargo-package-crates)
-              (unsetenv "RUSTC_BOOTSTRAP")))
+               cargo-package-crates)))

         ;; Then unpack the crate, reset the timestamp of all contained files, and
         ;; repack them.  This is necessary to ensure that they are reproducible.
diff --git a/guix/scripts/import.scm b/guix/scripts/import.scm
index d8e1ed3e6f..f6e2985ed9 100644
--- a/guix/scripts/import.scm
+++ b/guix/scripts/import.scm
@@ -126,13 +126,12 @@  (define-command (guix-import . args)
      (show-version-and-exit "guix import"))
     ((or ("-i" file importer args ...)
          ("--insert" file importer args ...))
-     (let* ((definer?
-              (cut member
-                   <>
-                   `(,@(if (member importer '("crate"))
-                           '(define)
-                           '())
-                     define-public)))
+     (let* ((importer-definers
+             `(,@(if (member importer '("crate"))
+                     '(define)
+                     '())
+               define-public))
+            (definer? (cut member <> importer-definers))
             (find-and-insert
              (lambda (expr)
                (match expr
diff --git a/tests/crate.scm b/tests/crate.scm
index d46f753f9c..9b7066c3b1 100644
--- a/tests/crate.scm
+++ b/tests/crate.scm
@@ -478,7 +478,7 @@  (define rust-leaf-bob-3.0.2-yanked
     (license #f)))

 (define temp-file
-  (string-append "t-utils-" (number->string (getpid))))
+  (string-append "t-crate-" (number->string (getpid))))

 
 (test-begin "crate")