From patchwork Sun Dec 27 14:38:06 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Ludovic_Court=C3=A8s?= X-Patchwork-Id: 25966 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 69FE827BC05; Sun, 27 Dec 2020 14:39:12 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_MSPIKE_H4,RCVD_IN_MSPIKE_WL,SPF_HELO_PASS autolearn=unavailable autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id AF9FB1794D for ; Sun, 27 Dec 2020 14:39:11 +0000 (GMT) Received: from localhost ([::1]:34396 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ktXCQ-0003hm-SW for patchwork@mira.cbaines.net; Sun, 27 Dec 2020 09:39:10 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:55162) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ktXCI-0003fx-U3 for guix-patches@gnu.org; Sun, 27 Dec 2020 09:39:02 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:48959) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1ktXCI-0003Oj-MQ for guix-patches@gnu.org; Sun, 27 Dec 2020 09:39:02 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1ktXCI-0000Je-Kp for guix-patches@gnu.org; Sun, 27 Dec 2020 09:39:02 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#45460] [PATCH 2/5] utils: Support zstd compression via Guile-zstd. Resent-From: Ludovic =?utf-8?q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sun, 27 Dec 2020 14:39:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 45460 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 45460@debbugs.gnu.org Received: via spool by 45460-submit@debbugs.gnu.org id=B45460.16090799031119 (code B ref 45460); Sun, 27 Dec 2020 14:39:02 +0000 Received: (at 45460) by debbugs.gnu.org; 27 Dec 2020 14:38:23 +0000 Received: from localhost ([127.0.0.1]:60496 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ktXBf-0000Hu-1k for submit@debbugs.gnu.org; Sun, 27 Dec 2020 09:38:23 -0500 Received: from eggs.gnu.org ([209.51.188.92]:33738) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ktXBd-0000HT-6W for 45460@debbugs.gnu.org; Sun, 27 Dec 2020 09:38:21 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]:49700) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ktXBY-00039k-09; Sun, 27 Dec 2020 09:38:16 -0500 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=51504 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1ktXBX-0001Rq-GK; Sun, 27 Dec 2020 09:38:15 -0500 From: Ludovic =?utf-8?q?Court=C3=A8s?= Date: Sun, 27 Dec 2020 15:38:06 +0100 Message-Id: <20201227143809.18554-2-ludo@gnu.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201227143809.18554-1-ludo@gnu.org> References: <20201227143809.18554-1-ludo@gnu.org> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches * guix/utils.scm (lzip-port): Return a single value. (zstd-port): New procedure. (decompressed-port, compressed-output-port): Add 'zstd' case. * tests/utils.scm (test-compression/decompression): Test 'zstd' when the (zstd) module is available. --- guix/utils.scm | 12 +++++++++++- tests/utils.scm | 6 ++++-- 2 files changed, 15 insertions(+), 3 deletions(-) diff --git a/guix/utils.scm b/guix/utils.scm index e3c78959ed..09eee0ecfb 100644 --- a/guix/utils.scm +++ b/guix/utils.scm @@ -209,7 +209,13 @@ buffered data is lost." "Return the lzip port produced by calling PROC (a symbol) on PORT and ARGS. Raise an error if lzlib support is missing." (let ((make-port (module-ref (resolve-interface '(lzlib)) proc))) - (values (make-port port) '()))) + (make-port port))) + +(define (zstd-port proc port . args) + "Return the zstd port produced by calling PROC (a symbol) on PORT and ARGS. +Raise an error if zstd support is missing." + (let ((make-port (module-ref (resolve-interface '(zstd)) proc))) + (make-port port))) (define (decompressed-port compression input) "Return an input port where INPUT is decompressed according to COMPRESSION, @@ -221,6 +227,8 @@ a symbol such as 'xz." ('gzip (filtered-port `(,%gzip "-dc") input)) ('lzip (values (lzip-port 'make-lzip-input-port input) '())) + ('zstd (values (zstd-port 'make-zstd-input-port input) + '())) (_ (error "unsupported compression scheme" compression)))) (define (call-with-decompressed-port compression port proc) @@ -280,6 +288,8 @@ program--e.g., '(\"--fast\")." ('gzip (filtered-output-port `(,%gzip "-c" ,@options) output)) ('lzip (values (lzip-port 'make-lzip-output-port output) '())) + ('zstd (values (zstd-port 'make-zstd-output-port output) + '())) (_ (error "unsupported compression scheme" compression)))) (define* (call-with-compressed-output-port compression port proc diff --git a/tests/utils.scm b/tests/utils.scm index c278b2a277..9bce446d98 100644 --- a/tests/utils.scm +++ b/tests/utils.scm @@ -228,8 +228,10 @@ skip these tests." get-bytevector-all))))) (for-each test-compression/decompression - '(gzip xz lzip) - (list (const #t) (const #t) (const #t))) + `(gzip xz lzip zstd) + (list (const #t) (const #t) (const #t) + (lambda () + (resolve-module '(zstd) #t #f #:ensure #f)))) ;; This is actually in (guix store). (test-equal "store-path-package-name" From patchwork Sun Dec 27 14:38:07 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Ludovic_Court=C3=A8s?= X-Patchwork-Id: 25967 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 178F327BC05; Sun, 27 Dec 2020 14:39:13 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_MSPIKE_H4,RCVD_IN_MSPIKE_WL,SPF_HELO_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id DC22B17A86 for ; Sun, 27 Dec 2020 14:39:11 +0000 (GMT) Received: from localhost ([::1]:34394 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ktXCR-0003hl-1k for patchwork@mira.cbaines.net; Sun, 27 Dec 2020 09:39:11 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:55168) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ktXCJ-0003g4-BL for guix-patches@gnu.org; Sun, 27 Dec 2020 09:39:03 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:48960) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1ktXCJ-0003On-42 for guix-patches@gnu.org; Sun, 27 Dec 2020 09:39:03 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1ktXCJ-0000Jl-27 for guix-patches@gnu.org; Sun, 27 Dec 2020 09:39:03 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#45460] [PATCH 3/5] publish: Add support for zstd compression. Resent-From: Ludovic =?utf-8?q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sun, 27 Dec 2020 14:39:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 45460 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 45460@debbugs.gnu.org Received: via spool by 45460-submit@debbugs.gnu.org id=B45460.16090799091142 (code B ref 45460); Sun, 27 Dec 2020 14:39:03 +0000 Received: (at 45460) by debbugs.gnu.org; 27 Dec 2020 14:38:29 +0000 Received: from localhost ([127.0.0.1]:60500 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ktXBk-0000IL-Dp for submit@debbugs.gnu.org; Sun, 27 Dec 2020 09:38:28 -0500 Received: from eggs.gnu.org ([209.51.188.92]:33742) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ktXBd-0000HV-Vs for 45460@debbugs.gnu.org; Sun, 27 Dec 2020 09:38:22 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]:49702) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ktXBY-0003AQ-PB; Sun, 27 Dec 2020 09:38:16 -0500 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=51504 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1ktXBY-0001Rq-9y; Sun, 27 Dec 2020 09:38:16 -0500 From: Ludovic =?utf-8?q?Court=C3=A8s?= Date: Sun, 27 Dec 2020 15:38:07 +0100 Message-Id: <20201227143809.18554-3-ludo@gnu.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201227143809.18554-1-ludo@gnu.org> References: <20201227143809.18554-1-ludo@gnu.org> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches * guix/scripts/publish.scm (compress-nar)[write-compressed-file]: New procedure. Use it for 'gzip' and 'lzip'. Add 'zstd. (nar-response-port, string->compression-type): Add case for 'zstd'. * tests/publish.scm (zstd-supported?): New procedure. ("/nar/zstd/*"): New test. * doc/guix.texi (Invoking guix publish): Document zstd compression. --- doc/guix.texi | 18 ++++++++++++------ guix/scripts/publish.scm | 31 ++++++++++++++++++------------- tests/publish.scm | 16 ++++++++++++++++ 3 files changed, 46 insertions(+), 19 deletions(-) diff --git a/doc/guix.texi b/doc/guix.texi index b12cb11bdf..ed38f2e37b 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -12329,17 +12329,23 @@ server socket is open and the signing key has been read. @item --compression[=@var{method}[:@var{level}]] @itemx -C [@var{method}[:@var{level}]] Compress data using the given @var{method} and @var{level}. @var{method} is -one of @code{lzip} and @code{gzip}; when @var{method} is omitted, @code{gzip} -is used. +one of @code{lzip}, @code{zstd}, and @code{gzip}; when @var{method} is +omitted, @code{gzip} is used. When @var{level} is zero, disable compression. The range 1 to 9 corresponds to different compression levels: 1 is the fastest, and 9 is the best (CPU-intensive). The default is 3. -Usually, @code{lzip} compresses noticeably better than @code{gzip} for a small -increase in CPU usage; see -@uref{https://nongnu.org/lzip/lzip_benchmark.html,benchmarks on the lzip Web -page}. +Usually, @code{lzip} compresses noticeably better than @code{gzip} for a +small increase in CPU usage; see +@uref{https://nongnu.org/lzip/lzip_benchmark.html,benchmarks on the lzip +Web page}. However, @code{lzip} achieves low decompression throughput +(on the order of 50@tie{}MiB/s on modern hardware), which can be a +bottleneck for someone who downloads over a fast network connection. + +The compression ratio of @code{zstd} is between that of @code{lzip} and +that of @code{gzip}; its main advantage is a +@uref{https://facebook.github.io/zstd/,high decompression speed}. Unless @option{--cache} is used, compression occurs on the fly and the compressed streams are not diff --git a/guix/scripts/publish.scm b/guix/scripts/publish.scm index 5a865c838d..fa85088ed0 100644 --- a/guix/scripts/publish.scm +++ b/guix/scripts/publish.scm @@ -56,6 +56,8 @@ #:use-module (zlib) #:autoload (lzlib) (call-with-lzip-output-port make-lzip-output-port) + #:autoload (zstd) (call-with-zstd-output-port + make-zstd-output-port) #:use-module (guix cache) #:use-module (guix ui) #:use-module (guix scripts) @@ -588,23 +590,22 @@ requested using POOL." (define nar (nar-cache-file cache item #:compression compression)) + (define (write-compressed-file call-with-compressed-output-port) + ;; Note: the file port gets closed along with the compressed port. + (call-with-compressed-output-port (open-output-file (string-append nar ".tmp")) + (lambda (port) + (write-file item port)) + #:level (compression-level compression)) + (rename-file (string-append nar ".tmp") nar)) + (mkdir-p (dirname nar)) (match (compression-type compression) ('gzip - ;; Note: the file port gets closed along with the gzip port. - (call-with-gzip-output-port (open-output-file (string-append nar ".tmp")) - (lambda (port) - (write-file item port)) - #:level (compression-level compression) - #:buffer-size %default-buffer-size) - (rename-file (string-append nar ".tmp") nar)) + (write-compressed-file call-with-gzip-output-port)) ('lzip - ;; Note: the file port gets closed along with the lzip port. - (call-with-lzip-output-port (open-output-file (string-append nar ".tmp")) - (lambda (port) - (write-file item port)) - #:level (compression-level compression)) - (rename-file (string-append nar ".tmp") nar)) + (write-compressed-file call-with-lzip-output-port)) + ('zstd + (write-compressed-file call-with-zstd-output-port)) ('none ;; Cache nars even when compression is disabled so that we can ;; guarantee the TTL (see .) @@ -871,6 +872,9 @@ example: \"/foo/bar\" yields '(\"foo\" \"bar\")." (($ 'lzip level) (make-lzip-output-port (response-port response) #:level level)) + (($ 'zstd level) + (make-zstd-output-port (response-port response) + #:level level)) (($ 'none) (response-port response)) (#f @@ -953,6 +957,7 @@ blocking." (match string ("gzip" 'gzip) ("lzip" 'lzip) + ("zstd" 'zstd) (_ #f))) (define (effective-compression requested-type compressions) diff --git a/tests/publish.scm b/tests/publish.scm index cafd0f13a2..52101876b5 100644 --- a/tests/publish.scm +++ b/tests/publish.scm @@ -38,6 +38,7 @@ #:use-module ((guix pki) #:select (%public-key-file %private-key-file)) #:use-module (zlib) #:use-module (lzlib) + #:autoload (zstd) (call-with-zstd-input-port) #:use-module (web uri) #:use-module (web client) #:use-module (web response) @@ -54,6 +55,9 @@ (define %store (open-connection-for-tests)) +(define (zstd-supported?) + (resolve-module '(zstd) #t #f #:ensure #f)) + (define %reference (add-text-to-store %store "ref" "foo")) (define %item (add-text-to-store %store "item" "bar" (list %reference))) @@ -237,6 +241,18 @@ References: ~%" (cut restore-file <> temp))) (call-with-input-file temp read-string)))) +(unless (zstd-supported?) (test-skip 1)) +(test-equal "/nar/zstd/*" + "bar" + (call-with-temporary-output-file + (lambda (temp port) + (let ((nar (http-get-port + (publish-uri + (string-append "/nar/zstd/" (basename %item)))))) + (call-with-zstd-input-port nar + (cut restore-file <> temp))) + (call-with-input-file temp read-string)))) + (test-equal "/*.narinfo with compression" `(("StorePath" . ,%item) ("URL" . ,(string-append "nar/gzip/" (basename %item))) From patchwork Sun Dec 27 14:38:08 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Ludovic_Court=C3=A8s?= X-Patchwork-Id: 25969 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id B213917AAC; Sun, 27 Dec 2020 14:39:23 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_MSPIKE_H4,RCVD_IN_MSPIKE_WL,SPF_HELO_PASS autolearn=unavailable autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 4E1291794D for ; Sun, 27 Dec 2020 14:39:23 +0000 (GMT) Received: from localhost ([::1]:35196 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ktXCc-00042R-HF for patchwork@mira.cbaines.net; Sun, 27 Dec 2020 09:39:22 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:55178) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ktXCJ-0003gL-Oj for guix-patches@gnu.org; Sun, 27 Dec 2020 09:39:03 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:48961) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1ktXCJ-0003Ot-H6 for guix-patches@gnu.org; Sun, 27 Dec 2020 09:39:03 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1ktXCJ-0000Js-F4 for guix-patches@gnu.org; Sun, 27 Dec 2020 09:39:03 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#45460] [PATCH 4/5] substitute: Add zstd support. Resent-From: Ludovic =?utf-8?q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sun, 27 Dec 2020 14:39:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 45460 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 45460@debbugs.gnu.org Received: via spool by 45460-submit@debbugs.gnu.org id=B45460.16090799091150 (code B ref 45460); Sun, 27 Dec 2020 14:39:03 +0000 Received: (at 45460) by debbugs.gnu.org; 27 Dec 2020 14:38:29 +0000 Received: from localhost ([127.0.0.1]:60502 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ktXBk-0000IN-Vt for submit@debbugs.gnu.org; Sun, 27 Dec 2020 09:38:29 -0500 Received: from eggs.gnu.org ([209.51.188.92]:33746) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ktXBf-0000HY-LL for 45460@debbugs.gnu.org; Sun, 27 Dec 2020 09:38:23 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]:49703) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ktXBZ-0003At-IM; Sun, 27 Dec 2020 09:38:17 -0500 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=51504 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1ktXBZ-0001Rq-2y; Sun, 27 Dec 2020 09:38:17 -0500 From: Ludovic =?utf-8?q?Court=C3=A8s?= Date: Sun, 27 Dec 2020 15:38:08 +0100 Message-Id: <20201227143809.18554-4-ludo@gnu.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201227143809.18554-1-ludo@gnu.org> References: <20201227143809.18554-1-ludo@gnu.org> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches * guix/scripts/substitute.scm (%compression-methods): Add zstd. (compresses-better?): "lzip" always wins. --- guix/scripts/substitute.scm | 3 +++ 1 file changed, 3 insertions(+) diff --git a/guix/scripts/substitute.scm b/guix/scripts/substitute.scm index 8084c89ae5..32ebcf1bb9 100755 --- a/guix/scripts/substitute.scm +++ b/guix/scripts/substitute.scm @@ -944,6 +944,8 @@ authorized substitutes." ;; supported. See 'decompressed-port' in (guix utils). `(("gzip" . ,(const #t)) ("lzip" . ,(const #t)) + ("zstd" . ,(lambda () + (resolve-module '(zstd) #t #f #:ensure #f))) ("xz" . ,(const #t)) ("bzip2" . ,(const #t)) ("none" . ,(const #t)))) @@ -961,6 +963,7 @@ this is a rough approximation." (match compression1 ("none" #f) ("gzip" (string=? compression2 "none")) + ("lzip" #t) (_ (or (string=? compression2 "none") (string=? compression2 "gzip"))))) From patchwork Sun Dec 27 14:38:09 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Ludovic_Court=C3=A8s?= X-Patchwork-Id: 25968 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 6FC3317AAC; Sun, 27 Dec 2020 14:39:13 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_MSPIKE_H4,RCVD_IN_MSPIKE_WL,SPF_HELO_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 7FF6D1794D for ; Sun, 27 Dec 2020 14:39:12 +0000 (GMT) Received: from localhost ([::1]:34426 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ktXCR-0003ih-L4 for patchwork@mira.cbaines.net; Sun, 27 Dec 2020 09:39:11 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:55184) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ktXCK-0003gh-7d for guix-patches@gnu.org; Sun, 27 Dec 2020 09:39:04 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:48962) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1ktXCJ-0003P0-VR for guix-patches@gnu.org; Sun, 27 Dec 2020 09:39:03 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1ktXCJ-0000K0-Su for guix-patches@gnu.org; Sun, 27 Dec 2020 09:39:03 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#45460] [PATCH 5/5] doc: Mention optional dependency on Guile-zstd. Resent-From: Ludovic =?utf-8?q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sun, 27 Dec 2020 14:39:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 45460 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 45460@debbugs.gnu.org Received: via spool by 45460-submit@debbugs.gnu.org id=B45460.16090799091156 (code B ref 45460); Sun, 27 Dec 2020 14:39:03 +0000 Received: (at 45460) by debbugs.gnu.org; 27 Dec 2020 14:38:29 +0000 Received: from localhost ([127.0.0.1]:60504 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ktXBl-0000IV-88 for submit@debbugs.gnu.org; Sun, 27 Dec 2020 09:38:29 -0500 Received: from eggs.gnu.org ([209.51.188.92]:33750) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ktXBf-0000HZ-Nq for 45460@debbugs.gnu.org; Sun, 27 Dec 2020 09:38:24 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]:49705) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ktXBa-0003BQ-Es; Sun, 27 Dec 2020 09:38:18 -0500 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=51504 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1ktXBZ-0001Rq-SC; Sun, 27 Dec 2020 09:38:18 -0500 From: Ludovic =?utf-8?q?Court=C3=A8s?= Date: Sun, 27 Dec 2020 15:38:09 +0100 Message-Id: <20201227143809.18554-5-ludo@gnu.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201227143809.18554-1-ludo@gnu.org> References: <20201227143809.18554-1-ludo@gnu.org> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches * doc/guix.texi (Requirements): Add Guile-zstd. --- doc/guix.texi | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/doc/guix.texi b/doc/guix.texi index ed38f2e37b..dafa1ffc56 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -850,6 +850,11 @@ Support for build offloading (@pxref{Daemon Offload Setup}) and @uref{https://github.com/artyom-poptsov/guile-ssh, Guile-SSH}, version 0.13.0 or later. +@item +@uref{https://notabug.org/guile-zstd/guile-zstd, Guile-zstd}, for zstd +compression and decompression in @command{guix publish} and for +substitutes (@pxref{Invoking guix publish}). + @item @uref{https://ngyro.com/software/guile-semver.html, Guile-Semver} for the @code{crate} importer (@pxref{Invoking guix import}).