| Message ID | f0cc4fd9538ab9dd8c7649efbc457f26cf536b98.1634185715.git.leo@famulari.name |
|---|---|
| State | Accepted |
| Headers | show |
| Series | [bug#51198] gnu: Add b2sum. | expand |
| Context | Check | Description |
|---|---|---|
| cbaines/comparison | success | View comparision |
| cbaines/git branch | success | View Git branch |
| cbaines/applying patch | success | View Laminar job |
| cbaines/issue | success | View issue |
Pushed as 3d51fd19faee59d9455eebbb957cf6498ecc1220
Leo Famulari <leo@famulari.name> writes: > * gnu/packages/crypto.scm (b2sum): New variable. Hi, I guix pulled and wanted to try this, but I get >> b2sum --help > Illegal instruction (I'm on an old X200 thinkpad). Might it be that build is not deterministic (and we should disable some optimization)? Thanks, Nicolò
On Wed, Oct 27, 2021 at 12:26:01AM +0200, Nicolò Balzarotti wrote: > Leo Famulari <leo@famulari.name> writes: > > > * gnu/packages/crypto.scm (b2sum): New variable. > > Hi, I guix pulled and wanted to try this, but I get > > >> b2sum --help > > Illegal instruction > > (I'm on an old X200 thinkpad). Might it be that build is not > deterministic (and we should disable some optimization)? I guess it's because the package built with "-march=native". What's the right choice for this parameter? https://gcc.gnu.org/onlinedocs/gcc/x86-Options.html CC-ing Mark Weaver because he often has good advice on this subject. 'gnu/packages' does include some "-march=" examples.
On Tue, Oct 26, 2021 at 06:53:55PM -0400, Leo Famulari wrote: > I guess it's because the package built with "-march=native". > > What's the right choice for this parameter? > > https://gcc.gnu.org/onlinedocs/gcc/x86-Options.html > > CC-ing Mark Weaver because he often has good advice on this subject. > > 'gnu/packages' does include some "-march=" examples. Er, CC-ing for real now
On Wed, Oct 27, 2021 at 12:26:01AM +0200, Nicolò Balzarotti wrote: > (I'm on an old X200 thinkpad). Might it be that build is not > deterministic (and we should disable some optimization)? As a test, I replaced march=native with march=x86-64. My benchmark is hashing a ~7GB tree of files [0]. Once the files are cached by the kernel: "march=native": ~13 seconds "march=x86-64": ~20 seconds I wonder if some of the other values of -march are faster... Or maybe the package could use #:substitutable #f, since b2sum is a very quick build. Although BLAKE3 / b3sum [1] is radically faster, b2sum is nice because it's not in Rust, which is still suboptimal on Guix. With b3sum, my benchmark takes less than 2 seconds! [0] `LC_ALL=C; find "$1" -type f -print0 | sort -z | xargs -0 b2sum | time b2sum` [1] https://bugs.gnu.org/51289
Hi Leo, Leo Famulari <leo@famulari.name> writes: > On Wed, Oct 27, 2021 at 12:26:01AM +0200, Nicolò Balzarotti wrote: >> (I'm on an old X200 thinkpad). Might it be that build is not >> deterministic (and we should disable some optimization)? > > As a test, I replaced march=native with march=x86-64. > > [...] > > Or maybe the package could use #:substitutable #f, since b2sum is a very > quick build. This blog post [fn:1] on guix-hpc address the "Pre-built binaries vs. performance" dilemma. I guess the easiest way is to provide a variant (b2sum-avx or something like that) with avx enabled. Else, I'd just go with the unoptimized version as it happens for many other packages, but let's hear from others. Thanks, Nicolò [fn:1] https://hpc.guix.info/blog/2018/01/pre-built-binaries-vs-performance/
On Wed, Oct 27, 2021 at 12:26:01AM +0200, Nicolò Balzarotti wrote: > Leo Famulari <leo@famulari.name> writes: > > > * gnu/packages/crypto.scm (b2sum): New variable. > > Hi, I guix pulled and wanted to try this, but I get > > >> b2sum --help > > Illegal instruction > > (I'm on an old X200 thinkpad). Might it be that build is not > deterministic (and we should disable some optimization)? I just pushed commit 386adb6df0e00e7170df0cdfaf0e04c38d0f3e11, which removes the "march=native" compiler flag, so that the package should work generically. https://git.savannah.gnu.org/cgit/guix.git/commit/?id=386adb6df0e00e7170df0cdfaf0e04c38d0f3e11 This commit also enables the 'tunable?' property of the package, so that one can use this program with the highest performance in a way that is tailored to their hardware. Just add the --tune command-line option to any Guix command that provides this package and it should do the right thing. Thanks Nicolò for reporting the problem and thanks to Ludovic for developing the solution.
diff --git a/gnu/packages/crypto.scm b/gnu/packages/crypto.scm index 64cdf48d5d..9d16d64ed2 100644 --- a/gnu/packages/crypto.scm +++ b/gnu/packages/crypto.scm @@ -1,7 +1,7 @@ ;;; GNU Guix --- Functional package management for GNU ;;; Copyright © 2014 David Thompson <davet@gnu.org> ;;; Copyright © 2015, 2017, 2018, 2019 Ricardo Wurmus <rekado@elephly.net> -;;; Copyright © 2016, 2017, 2018, 2019 Leo Famulari <leo@famulari.name> +;;; Copyright © 2016, 2017, 2018, 2019, 2021 Leo Famulari <leo@famulari.name> ;;; Copyright © 2016 Lukas Gradl <lgradl@openmailbox> ;;; Copyright © 2016–2021 Tobias Geerinckx-Rice <me@tobias.gr> ;;; Copyright © 2016, 2017 Nikita <nikita@n0.is> @@ -838,6 +838,42 @@ (define-public libb2 BLAKE.") (license license:public-domain))) +(define-public b2sum + ;; Upstream doesn't seem to use a versioned release workflow, so build from + ;; a recent commit. + (let ((commit "54f4faa4c16ea34bcd59d16e8da46a64b259fc07") + (revision "0")) + (package + (name "b2sum") + (version (git-version "20190724" revision commit)) + (source (origin + (method git-fetch) + (uri (git-reference + (url "https://github.com/BLAKE2/BLAKE2") + (commit commit))) + (file-name (git-file-name name version)) + (sha256 + (base32 "04z631v0vzl52g73v390ask5fnzi5wg83lcjkjhpmmymaz0jn152")))) + (build-system gnu-build-system) + (arguments + `(#:make-flags (list (string-append "CC=" ,(cc-for-target)) + (string-append "PREFIX=" (assoc-ref %outputs "out"))) + #:tests? #f ; No test suite + #:phases + (modify-phases %standard-phases + ;; XXX Can add-before and delete be expected to work like this? + (add-before 'configure 'change-directory + (lambda _ + (chdir "b2sum"))) + (delete 'configure)))) + (home-page "https://www.blake2.net/") + (synopsis "BLAKE2 checksum tool") + (description "BLAKE2 is a cryptographic hash function faster than MD5, +SHA-1, SHA-2, and SHA-3, yet is at least as secure as SHA-3.") + ;; You may also choose to redistribute this program as Apache 2.0 or the + ;; OpenSSL license. See 'b2sum/b2sum.c' in the source distribution. + (license license:cc0)))) + (define-public rhash (package (name "rhash")