| Message ID | 1bbbeed9c7c6e50464bee042e8dd06e9cd62c4c2.1690094273.git.fries1234@protonmail.com |
|---|---|
| State | New |
| Headers | show |
| Series | [bug#64804] gnu: rust: Update to Rust 1.71.0 | expand |
Hi Fries, While I'm not a Rust packager (and therefore may miss some stuff during review), there are a few glaring issues which stand out for me. Also forgive the formatting; not having the message proper to reply to complicates things. First and foremost, Rust has 11052 reverse dependencies and this patch would rebuild 25834 packages. This should be submitted to the core-updates branch. I'm not sure whether this patch should be resubmitted for it or if others can just keep this in mind when considering merging it. The next major thing is that this should be split into multiple patches. Make each discrete new Rust in its own patch, for example. Ideally, each patch would be atomic so that only a subset could be applied and packages would still build fine. If I were doing it, I would commit each intermediate version of Rust and bump the default Rust version with that commit. You may also be able to get away with adding each intermediate Rust and only bumping the version of the default rust for the last one; others will have to weigh in. See Contributing > Submitting Patches > Sending a Patch Series > Multiple Patches in the manual for how to submit such a patch series if you need it. Make sure your commit messages match the proper style; see https://www.gnu.org/prep/standards/html_node/Change-Logs.html#Change-Logs and the commit history. There's one point where you add three new, empty lines near the beginning of the patch. Prune those. Where you write new modify-phases forms and new snippets, use g-expressions. > + [target."cfg(windows)".dependencies.windows-sys] > + version = "0.45.0" Shouldn't we be able to remove this altogether somehow since we're not the target operating system? > + ;; Rust 1.70 uses the rustix library which on Linux, it defaults to > + ;; using outline ASM which without the cc cargo feature enabled, it > + ;; will expect a precompiled binary library. This patch will enable the cargo > + ;; cc feature flag inside the fd-lock vendored Cargo.toml file, which is the > + ;; crate that uses rustix. This comment is difficult to parse. Could it be reworded more clearly? Perhaps, "Rust 1.70 adds the rustix library which uses outline ASM. The vendored fd-lock crate uses rustix. It expects a precompiled binary library without the "cc" Cargo feature enabled. This patch enables the "cc" feature flag inside the vendored fd-lock Cargo.toml file." That said, fd-lock is already packaged in Guix; the vendored version should be avoided. > + (("(checksum = )\".*\"" all name) > + (string-append name "\"" ,%cargo-reference-hash "\""))) Clever. > + ;;; Function to make creating a list to ignore tests a bit easier. This should probably be a docstring instead of a comment, and it should describe what the function does - "Accept a list of strings containing test names, and return a list of forms for skipping those tests" maybe. > + (define (make-ignore-test-list strs) > + (map (lambda (str) > + (let ((ignore-string (format #f "#[ignore]\n~a" str))) > + `((,str) ,ignore-string))) > + strs)) A few things here. Firstly, I'm not sure that this procedure should exist. Ideally we'd like to fix rather than ignore as many tests as possible - although I see you're using this to rewrite existing stuff more cleanly. Either way, break this whole function and its usage out into a separate patch. Move it to be near other helper functions, not in the middle of package definitions. Also perhaps make it a bit cleaner like: > (define (ignore-rust-tests strs) > (map (lambda (str) > `((,str) ,(string-append "#[ignore]\n" str)) > strs)) It's worth noting this is essentially a non-hygienic macro; maybe look into rewriting it as a macro instead. See https://spritely.institute/static/papers/scheme-primer.html#scheme-extensibility for an introduction to the ideas behind macros, and https://www.gnu.org/software/guile/manual/html_node/Macros.html - especially https://www.gnu.org/software/guile/manual/html_node/Syntax-Rules.html. I've also found the (incorrectly) linked and very excellent http://www.phyast.pitt.edu/~micheles/syntax-rules.pdf (WARNING! IS A PDF! also available from https://gist.github.com/jgarte/beb03e000943b7426f00b3d04ed01262 (WARNING! IS GITHUB!)) to be incredibly helpful. Ideally instead of using `(substitutes* <file> ...)` you'd simply have some thing like `(ignore-rust-tests <file> '(<str> <strs> ...))` which produces the right code. Note that if you rewrite this as a macro, you won't be able to use docstrings. > + ;; Gitoxide tests seem to require the internet to run > + ;; and Guix build containers don't have the internet. You can just say, "Gitoxide tests require the network" or similar. > - (substitute* > - "src/tools/cargo/tests/testsuite/init/simple_hg_ignore_exists/mod.rs" > - (("fn simple_hg_ignore_exists") > - "#[ignore]\nfn simple_hg_ignore_exists")) > (substitute* > "src/tools/cargo/tests/testsuite/init/mercurial_autodetect/mod.rs" > - (("fn mercurial_autodetect") > - "#[ignore]\nfn mercurial_autodetect")))) > + ,@(make-ignore-test-list '("fn case"))) > + (substitute* > + "src/tools/cargo/tests/testsuite/init/simple_hg/mod.rs" > + ,@(make-ignore-test-list '("fn case"))) > + (substitute* > + "src/tools/cargo/tests/testsuite/init/simple_hg_ignore_exists/mod.rs" > + ,@(make-ignore-test-list '("fn case"))) In this code, you move down the substitute* call on simple_hg_ignore_exists/mod.rs and add code above it. This produces a larger diff than necessary. Avoid changing code order like this without a good reason. > + ;; Append the default output's lib folder to the RUSTFLAGS > + ;; environment variable. this lets programs that depend on > + ;; rustc's shared libraries like rustfmt work. Minor writing stuff. Capitalize the 't' in "this;" move "like rustfmt" to after "programs." > + (setenv "RUSTFLAGS" > + (format #f "-C link-arg=-Wl,-rpath,~a/lib" > + (assoc-ref outputs "out"))) I'm not sure that this is the best way to do this. Setting build flags in an environment variable just feels wrong. I'll let someone with more knowledge of the domain weigh in, but I do want to mention it. I don't think any of the rest of this review is blocking, but you may want to consider it anyway. I don't see "format" used in package definitions very often; I don't know if there's some reason to oppose it, but it's worth considering just using "string-append" instead. Make sure this builds on architectures besides x86_64 if you can. I see in the commit history for Rust there seem to be some issues around aarch64 and riscv64 support in particular, so testing for those platforms may reveal more issues. Make sure to run "guix style" and "guix lint". You may want to have a separate patch, either before or after your changes, to port all of the default rust package's modify-phases to g-expressions; "guix style -S arguments" should help there. You'll need to wait for review from experts on the subject, and obviously someone with commit, but this is very impressive for a first patch. Rust has been languishing for a while and it's clearly not just because it's not used. Well done. - Juli
Hello, it would be wonderful if this could be merged. I am trying to package broot (command line file manager) and one of it's dependency requires rust 1.70.0. Pretty please.
Also bump, We shouldn't be 4 versions behind upstream as a lot of newer rust apps/programs require on what used to be unstable features in older/current versions of rust. This is currently a stopgap for me when packaging swww, a wayland compositor wallpaper changer. Thanks all.
Hi, such rust related changes are targeted at the rust team branch. IIRC the rust team is a one-man army, so it absolutely possible that things take a while :/. But reading #64804, Efraim already bumped rust to 1.70 on the rust-team branch, so it will be available after the next rust-team merge. In the meantime you'd have to use the rust-team branch if you really want to use rust-1.70. mfg² Milan Svoboda <milan.svoboda@centrum.cz> writes: > Hello, > > it would be wonderful if this could be merged. I am trying to package broot > (command line file manager) and one of it's dependency requires rust 1.70.0. > > Pretty please.
diff --git a/gnu/packages/patches/rust-1.70-fix-rustix-build.patch b/gnu/packages/patches/rust-1.70-fix-rustix-build.patch new file mode 100644 index 0000000..a7e2003 --- /dev/null +++ b/gnu/packages/patches/rust-1.70-fix-rustix-build.patch @@ -0,0 +1,20 @@ +--- a/vendor/fd-lock/Cargo.toml 2023-05-31 14:44:48.000000000 -0700 ++++ b/vendor/fd-lock/Cargo.toml 2023-07-14 21:19:34.637702319 -0700 +@@ -45,7 +45,7 @@ + + [target."cfg(unix)".dependencies.rustix] + version = "0.37.0" +-features = ["fs"] ++features = ["fs", "cc"] + + [target."cfg(windows)".dependencies.windows-sys] + version = "0.45.0" +--- a/src/bootstrap/Cargo.lock 2023-07-11 20:32:40.000000000 -0700 ++++ b/src/bootstrap/Cargo.lock 2023-07-14 22:41:53.269284713 -0700 +@@ -618,6 +618,7 @@ + dependencies = [ + "bitflags", ++ "cc", + "errno", + "io-lifetimes", + "libc", diff --git a/gnu/packages/rust.scm b/gnu/packages/rust.scm index 8e106a9..d489de9 100644 --- a/gnu/packages/rust.scm +++ b/gnu/packages/rust.scm @@ -16,6 +16,7 @@ ;;; Copyright © 2022 Zheng Junjie <873216071@qq.com> ;;; Copyright © 2022 Jim Newsome <jnewsome@torproject.org> ;;; Copyright © 2022 Mark H Weaver <mhw@netris.org> +;;; Copyright © 2023 Fries <fries1234@protonmail.com> ;;; ;;; This file is part of GNU Guix. ;;; @@ -122,6 +123,9 @@ (define* (rust-bootstrapped-package base-rust version checksum) (alist-replace "rustc-bootstrap" (list base-rust) (package-native-inputs base-rust)))))) + + + ;;; Note: mrustc's only purpose is to be able to bootstap Rust; it's designed ;;; to be used in source form. (define %mrustc-commit "597593aba86fa2edbea80c6e09f0b1b2a480722d") @@ -705,6 +709,81 @@ (define rust-1.68 (rust-bootstrapped-package rust-1.67 "1.68.2" "15ifyd5jj8rd979dkakp887hgmhndr68pqaqvd2hqkfdywirqcwk")) +(define rust-1.69 + (let ((base-rust + (rust-bootstrapped-package + rust-1.68 "1.69.0" "03zn7kx5bi5mdfsqfccj4h8gd6abm7spj0kjsfxwlv5dcwc9f1gv"))) + (package + (inherit base-rust) + (source + (origin + (inherit (package-source base-rust)) + (snippet + '(begin + (for-each delete-file-recursively + '("src/llvm-project" + "vendor/tikv-jemalloc-sys/jemalloc")) + ;; Also remove the bundled (mostly Windows) libraries. + (for-each delete-file + (find-files "vendor" ".*\\.(a|dll|exe|lib)$"))))))))) + +(define rust-1.70 + (let ((base-rust + (rust-bootstrapped-package + rust-1.69 "1.70.0" "0z6j7d0ni0rmfznv0w3mrf882m11kyh51g2bxkj40l3s1c0axgxj"))) + (package + (inherit base-rust) + (source + (origin + (inherit (package-source base-rust)) + ;; Rust 1.70 uses the rustix library which on Linux, it defaults to + ;; using outline ASM which without the cc cargo feature enabled, it + ;; will expect a precompiled binary library. This patch will enable the cargo + ;; cc feature flag inside the fd-lock vendored Cargo.toml file, which is the + ;; crate that uses rustix. + (patches (search-patches "rust-1.70-fix-rustix-build.patch")) + (patch-flags '("-p1")))) + (arguments + (substitute-keyword-arguments (package-arguments base-rust) + ((#:phases phases) + `(modify-phases ,phases + (replace 'build + (lambda* (#:key parallel-build? #:allow-other-keys) + (let ((job-spec (string-append + "-j" (if parallel-build? + (number->string (parallel-job-count)) + "1")))) + (invoke "./x.py" job-spec "build" "--stage=1" + "library/std" + "src/tools/cargo"))))))))))) + +(define rust-1.71 + (let ((base-rust + (rust-bootstrapped-package + rust-1.70 "1.71.0" "15jc0d13cmrh2xvpkyyvsbwgn3w4klqiwf2wlgzfp22mvjmy8rx6"))) + (package + (inherit base-rust) + (arguments + (substitute-keyword-arguments (package-arguments base-rust) + ((#:phases phases) + `(modify-phases ,phases + (replace 'patch-cargo-checksums + (lambda _ + (substitute* '("Cargo.lock" + "src/bootstrap/Cargo.lock" + "src/tools/rust-analyzer/Cargo.lock" + "src/tools/cargo/Cargo.lock") + (("(checksum = )\".*\"" all name) + (string-append name "\"" ,%cargo-reference-hash "\""))) + (generate-all-checksums "vendor")))))))))) + +;;; Function to make creating a list to ignore tests a bit easier. +(define (make-ignore-test-list strs) + (map (lambda (str) + (let ((ignore-string (format #f "#[ignore]\n~a" str))) + `((,str) ,ignore-string))) + strs)) + ;;; Note: Only the latest versions of Rust are supported and tested. The ;;; intermediate rusts are built for bootstrapping purposes and should not ;;; be relied upon. This is to ease maintenance and reduce the time @@ -713,7 +792,7 @@ (define rust-1.68 ;;; Here we take the latest included Rust, make it public, and re-enable tests ;;; and extra components such as rustfmt. (define-public rust - (let ((base-rust rust-1.67)) + (let ((base-rust rust-1.71)) (package (inherit base-rust) (outputs (cons "rustfmt" (package-outputs base-rust))) @@ -748,23 +827,43 @@ (define-public rust (which "env"))))) (add-after 'unpack 'disable-tests-requiring-git (lambda _ - (substitute* "src/tools/cargo/tests/testsuite/new.rs" - (("fn author_prefers_cargo") - "#[ignore]\nfn author_prefers_cargo") - (("fn finds_author_git") - "#[ignore]\nfn finds_author_git") - (("fn finds_local_author_git") - "#[ignore]\nfn finds_local_author_git")))) + (substitute* "src/tools/cargo/tests/testsuite/git.rs" + ,@(make-ignore-test-list + '("fn fetch_downloads_with_git2_first_then_with_gitoxide_and_vice_versa" + "fn git_fetch_cli_env_clean" + "fn git_with_cli_force" + "fn use_the_cli"))) + ;; Gitoxide tests seem to require the internet to run + ;; and Guix build containers don't have the internet. + (substitute* "src/tools/cargo/tests/testsuite/git_shallow.rs" + ,@(make-ignore-test-list + '("fn gitoxide_clones_git_dependency_with_shallow_protocol_and_git2_is_used_for_followup_fetches" + "fn gitoxide_clones_registry_with_shallow_protocol_and_aborts_and_updates_again" + "fn gitoxide_clones_registry_with_shallow_protocol_and_follow_up_fetch_maintains_shallowness" + "fn gitoxide_clones_registry_with_shallow_protocol_and_follow_up_with_git2_fetch" + "fn gitoxide_clones_registry_without_shallow_protocol_and_follow_up_fetch_uses_shallowness" + "fn gitoxide_clones_shallow_two_revs_same_deps" + "fn gitoxide_git_dependencies_switch_from_branch_to_rev" + "fn gitoxide_shallow_clone_followed_by_non_shallow_update" + "fn shallow_deps_work_with_revisions_and_branches_mixed_on_same_dependency"))) + (substitute* "src/tools/cargo/tests/testsuite/offline.rs" + ,@(make-ignore-test-list '("fn gitoxide_cargo_compile_offline_with_cached_git_dep_shallow_dep"))) + (substitute* "src/tools/cargo/tests/testsuite/patch.rs" + ,@(make-ignore-test-list '("fn gitoxide_clones_shallow_old_git_patch"))))) (add-after 'unpack 'disable-tests-requiring-mercurial (lambda _ - (substitute* - "src/tools/cargo/tests/testsuite/init/simple_hg_ignore_exists/mod.rs" - (("fn simple_hg_ignore_exists") - "#[ignore]\nfn simple_hg_ignore_exists")) (substitute* "src/tools/cargo/tests/testsuite/init/mercurial_autodetect/mod.rs" - (("fn mercurial_autodetect") - "#[ignore]\nfn mercurial_autodetect")))) + ,@(make-ignore-test-list '("fn case"))) + (substitute* + "src/tools/cargo/tests/testsuite/init/simple_hg/mod.rs" + ,@(make-ignore-test-list '("fn case"))) + (substitute* + "src/tools/cargo/tests/testsuite/init/simple_hg_ignore_exists/mod.rs" + ,@(make-ignore-test-list '("fn case"))) + (substitute* + "src/tools/cargo/tests/testsuite/new.rs" + ,@(make-ignore-test-list '("fn simple_hg"))))) (add-after 'unpack 'disable-tests-broken-on-aarch64 (lambda _ (with-directory-excursion "src/tools/cargo/tests/testsuite/" @@ -807,7 +906,7 @@ (define-public rust ;; We skip the test since it's drastically unlikely Guix's ;; packaging will introduce a bug here. (lambda _ - (delete-file "src/test/ui/parser/shebang/sneaky-attrib.rs"))) + (delete-file "tests/ui/parser/shebang/sneaky-attrib.rs"))) (add-after 'unpack 'patch-process-tests (lambda* (#:key inputs #:allow-other-keys) (let ((bash (assoc-ref inputs "bash"))) @@ -840,11 +939,17 @@ (define-public rust "gdb = \"" gdb "/bin/gdb\"\n")))))) (replace 'build ;; Phase overridden to also build rustfmt. - (lambda* (#:key parallel-build? #:allow-other-keys) + (lambda* (#:key parallel-build? outputs #:allow-other-keys) (let ((job-spec (string-append "-j" (if parallel-build? (number->string (parallel-job-count)) "1")))) + ;; Append the default output's lib folder to the RUSTFLAGS + ;; environment variable. this lets programs that depend on + ;; rustc's shared libraries like rustfmt work. + (setenv "RUSTFLAGS" + (format #f "-C link-arg=-Wl,-rpath,~a/lib" + (assoc-ref outputs "out"))) (invoke "./x.py" job-spec "build" "library/std" ;rustc "src/tools/cargo"