From patchwork Tue Jan 25 19:21:57 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: pukkamustard X-Patchwork-Id: 36809 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 020A827BBEA; Tue, 25 Jan 2022 19:26:06 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL, SPF_HELO_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id B55CD27BBE9 for ; Tue, 25 Jan 2022 19:26:05 +0000 (GMT) Received: from localhost ([::1]:48498 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nCRS7-0004mO-Pj for patchwork@mira.cbaines.net; Tue, 25 Jan 2022 14:26:04 -0500 Received: from eggs.gnu.org ([209.51.188.92]:37988) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nCRPE-00048O-CX for guix-patches@gnu.org; Tue, 25 Jan 2022 14:23:04 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:57255) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1nCRPE-0000BO-2S for guix-patches@gnu.org; Tue, 25 Jan 2022 14:23:04 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1nCRPD-0001F3-Vn for guix-patches@gnu.org; Tue, 25 Jan 2022 14:23:03 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#52555] [RFC PATCH v2 1/5] WIP: gnu: guile-eris: Update to unreleased git version. Resent-From: pukkamustard Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Tue, 25 Jan 2022 19:23:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 52555 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 52555@debbugs.gnu.org Cc: pukkamustard , ~pukkamustard/eris@lists.sr.ht Received: via spool by 52555-submit@debbugs.gnu.org id=B52555.16431385634693 (code B ref 52555); Tue, 25 Jan 2022 19:23:03 +0000 Received: (at 52555) by debbugs.gnu.org; 25 Jan 2022 19:22:43 +0000 Received: from localhost ([127.0.0.1]:50152 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nCROt-0001DX-HI for submit@debbugs.gnu.org; Tue, 25 Jan 2022 14:22:43 -0500 Received: from mout01.posteo.de ([185.67.36.65]:41619) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nCROY-0001C7-7H for 52555@debbugs.gnu.org; Tue, 25 Jan 2022 14:22:29 -0500 Received: from submission (posteo.de [89.146.220.130]) by mout01.posteo.de (Postfix) with ESMTPS id 4A0B4240027 for <52555@debbugs.gnu.org>; Tue, 25 Jan 2022 20:22:15 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1643138536; bh=qhTWe7UWmXECMl9CT46acau1cZgMduxF1qX8x4pCwgw=; h=From:To:Cc:Subject:Date:From; b=ldwCPusN2dX6HRhDxeqpR67P0+ZOgUpSB0vczXGwylfKlK6bM4WMcgh8+P9NV/wUz Yxb77kEu3qNCbdnD10I6CB+FCmC2Bq9h1msGrCJKg/HgJGIYh1KTspaL65/aGoz9KA ENP7peFHSqTXJSGVQgYoBiTZ80smHY7/DBCO8Ob1UYpj/NfKy9lfUaHeMvXwXE5pUo RnN5pCYNCGA6h0RwRs3s4wJrIjP2Zkid/y88cKWSfOGfAaCJHFULBWDOtDyo5ge6NB d0WEgNqeReTsz/j19C6ZYWjLLOx7fx5Q8y27nvWgL01L6MuFhu5L60FpKD1PMw0Tkc /7FG12IHHEH0Q== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4JjxYz10tFz9rxM; Tue, 25 Jan 2022 20:22:15 +0100 (CET) From: pukkamustard Date: Tue, 25 Jan 2022 19:21:57 +0000 Message-Id: <20220125192201.7582-2-pukkamustard@posteo.net> In-Reply-To: <20220125192201.7582-1-pukkamustard@posteo.net> References: <20220125192201.7582-1-pukkamustard@posteo.net> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches * gnu/packages/guile-xyz.schm (guile-eris): Update to unreleased git version. [source]: Update source URI. [propagated-inputs]: Add guile-json-4 and guile-gdbm-ffi. --- gnu/packages/guile-xyz.scm | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/gnu/packages/guile-xyz.scm b/gnu/packages/guile-xyz.scm index e2cf793acc..66ac486a74 100644 --- a/gnu/packages/guile-xyz.scm +++ b/gnu/packages/guile-xyz.scm @@ -4374,15 +4374,15 @@ (define-public guile-sodium (define-public guile-eris (package (name "guile-eris") - (version "0.2.0") + (version "bcbbcbc88f3ec1f2fafcd034ce5b620516bff105") (source (origin (method git-fetch) (uri (git-reference - (url "https://inqlab.net/git/eris.git") - (commit (string-append "v" version)))) + (url "https://codeberg.org/eris/guile-eris") + (commit version))) (file-name (git-file-name name version)) - (sha256 (base32 "1ijglmwkdy1l87gj429qfjis0v8b1zlxhbyfhx5za8664h68nqka")))) + (sha256 (base32 "17v3h2hqx080739rl57nfradp5vlmy24fgqdxry1zal5z9d3i8sr")))) (build-system gnu-build-system) (arguments '()) (native-inputs @@ -4394,7 +4394,7 @@ (define-public guile-eris guile-srfi-180)) (inputs (list guile-3.0)) (propagated-inputs - (list guile-sodium)) + (list guile-sodium guile-json-4 guile-gdbm-ffi)) (synopsis "Guile implementation of the Encoding for Robust Immutable Storage (ERIS)") (description "Guile-ERIS is the reference implementation of the Encoding for Robust From patchwork Tue Jan 25 19:21:58 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: pukkamustard X-Patchwork-Id: 36807 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 55D3827BBEB; Tue, 25 Jan 2022 19:23:32 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL, SPF_HELO_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 330C127BBEA for ; Tue, 25 Jan 2022 19:23:31 +0000 (GMT) Received: from localhost ([::1]:47066 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nCRPe-00048t-8v for patchwork@mira.cbaines.net; Tue, 25 Jan 2022 14:23:30 -0500 Received: from eggs.gnu.org ([209.51.188.92]:37968) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nCRPD-00046Q-9L for guix-patches@gnu.org; Tue, 25 Jan 2022 14:23:03 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:57252) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1nCRPC-0000Av-TE for guix-patches@gnu.org; Tue, 25 Jan 2022 14:23:02 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1nCRPC-0001Eg-QL for guix-patches@gnu.org; Tue, 25 Jan 2022 14:23:02 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#52555] [RFC PATCH v2 2/5] publish: Add ERIS URN to narinfo Resent-From: pukkamustard Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Tue, 25 Jan 2022 19:23:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 52555 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 52555@debbugs.gnu.org Cc: pukkamustard , ~pukkamustard/eris@lists.sr.ht Received: via spool by 52555-submit@debbugs.gnu.org id=B52555.16431385624671 (code B ref 52555); Tue, 25 Jan 2022 19:23:02 +0000 Received: (at 52555) by debbugs.gnu.org; 25 Jan 2022 19:22:42 +0000 Received: from localhost ([127.0.0.1]:50144 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nCROe-0001Cs-Gx for submit@debbugs.gnu.org; Tue, 25 Jan 2022 14:22:42 -0500 Received: from mout01.posteo.de ([185.67.36.65]:38629) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nCROZ-0001CA-Dn for 52555@debbugs.gnu.org; Tue, 25 Jan 2022 14:22:25 -0500 Received: from submission (posteo.de [89.146.220.130]) by mout01.posteo.de (Postfix) with ESMTPS id D5D42240027 for <52555@debbugs.gnu.org>; Tue, 25 Jan 2022 20:22:17 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1643138537; bh=W1gZghgCR8earSsN5aBLBtmdrhJyAYyFNtxvyJcaNcI=; h=From:To:Cc:Subject:Date:From; b=KnTlWy6rbh0pmW2mOQtgouzkXQte7i7DPY25YXtv8koUv2u8uy0it7lO9fJYq8SQ8 LR6lkHAxPOEtOgJsVTHI2ZFrpXR4VZ0pAP5zP6ew7D1620U8aJWlRFMHrjsQQkAkaQ VM0lMMrPbquYDs1aMH0L94GQZ3WDiLc2T9pzeKN5xPqtMPzIRG6mnurbQfYzRcPAqz O7gzMbbb1EpRbWbJD3aKqXr0yUf8Jgu/DaKvU70sqUYkcfVDfDktZHQEZM8k6cG9OG Nj2JkNENB60IbyoYOyWuf+DlS0Tp9txBTEzbKaxNnmRLq0Mbj64MLYntaBVO4ZKbyT 8zLPcoEz+Fohw== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4JjxZ110YGz9rxM; Tue, 25 Jan 2022 20:22:17 +0100 (CET) From: pukkamustard Date: Tue, 25 Jan 2022 19:21:58 +0000 Message-Id: <20220125192201.7582-3-pukkamustard@posteo.net> In-Reply-To: <20220125192201.7582-1-pukkamustard@posteo.net> References: <20220125192201.7582-1-pukkamustard@posteo.net> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches * guix/scripts/publish.scm: (bake-narinfo+nar): Compute ERIS URN of compressed nars. (narinfo-string): Add #:eris-urn parameter and honor it. * guix/scripts/narinfo.scm: ()[eris-format,eris-urn]: New fields. (narinfo-maker): Handle ERIS URN and ERIS format. * configure.ac: (HAVE_GUILE_ERIS): New conditional. * gnu/packages/package-management.scm: (guix)[native-inputs]: Add guile-eris. --- configure.ac | 5 +++++ gnu/packages/package-management.scm | 1 + guix/narinfo.scm | 14 ++++++++++---- guix/scripts/publish.scm | 24 ++++++++++++++++++++---- 4 files changed, 36 insertions(+), 8 deletions(-) diff --git a/configure.ac b/configure.ac index 341cff8fbd..72396be8aa 100644 --- a/configure.ac +++ b/configure.ac @@ -170,6 +170,11 @@ GUILE_MODULE_AVAILABLE([have_guile_avahi], [(avahi)]) AM_CONDITIONAL([HAVE_GUILE_AVAHI], [test "x$have_guile_avahi" = "xyes"]) +dnl Check for Guile-eris. +GUILE_MODULE_AVAILABLE([have_guile_eris], [(eris)]) +AM_CONDITIONAL([HAVE_GUILE_ERIS], + [test "x$have_guile_eris" = "xyes"]) + dnl Guile-newt is used by the graphical installer. GUILE_MODULE_AVAILABLE([have_guile_newt], [(newt)]) diff --git a/gnu/packages/package-management.scm b/gnu/packages/package-management.scm index 05795824b5..a9094b8b7f 100644 --- a/gnu/packages/package-management.scm +++ b/gnu/packages/package-management.scm @@ -404,6 +404,7 @@ (define code ("guile-zstd" ,guile-zstd) ("guile-ssh" ,guile-ssh) ("guile-git" ,guile-git) + ("guile-eris" ,guile-eris) ;; XXX: Keep the development inputs here even though ;; they're unnecessary, just so that 'guix environment diff --git a/guix/narinfo.scm b/guix/narinfo.scm index 4fc550aa6c..a6a5d3b84b 100644 --- a/guix/narinfo.scm +++ b/guix/narinfo.scm @@ -45,6 +45,8 @@ (define-module (guix narinfo) narinfo-file-sizes narinfo-hash narinfo-size + narinfo-eris-format + narinfo-eris-urn narinfo-references narinfo-deriver narinfo-system @@ -68,8 +70,8 @@ (define-module (guix narinfo) (define-record-type (%make-narinfo path uri-base uris compressions file-sizes file-hashes - nar-hash nar-size references deriver system - signature contents) + nar-hash nar-size eris-format eris-urn references deriver + system signature contents) narinfo? (path narinfo-path) (uri-base narinfo-uri-base) ;URI of the cache it originates from @@ -79,6 +81,8 @@ (define-record-type (file-hashes narinfo-file-hashes) (nar-hash narinfo-hash) (nar-size narinfo-size) + (eris-format narinfo-eris-format) + (eris-urn narinfo-eris-urn) (references narinfo-references) (deriver narinfo-deriver) (system narinfo-system) @@ -135,7 +139,7 @@ (define (narinfo-maker str cache-url) "Return a narinfo constructor for narinfos originating from CACHE-URL. STR must contain the original contents of a narinfo file." (lambda (path urls compressions file-hashes file-sizes - nar-hash nar-size references deriver system + nar-hash nar-size eris-format eris-urn references deriver system signature) "Return a new object." (define len (length urls)) @@ -157,6 +161,8 @@ (define len (length urls)) ((lst ...) (map string->number lst))) nar-hash (and=> nar-size string->number) + eris-format + (if eris-urn (string->uri eris-urn) #f) (string-tokenize references) (match deriver ((or #f "") #f) @@ -184,7 +190,7 @@ (define* (read-narinfo port #:optional url (narinfo-maker str url) '("StorePath" "URL" "Compression" "FileHash" "FileSize" "NarHash" "NarSize" - "References" "Deriver" "System" + "ERISFormat" "ERIS" "References" "Deriver" "System" "Signature") '("URL" "Compression" "FileSize" "FileHash")))) diff --git a/guix/scripts/publish.scm b/guix/scripts/publish.scm index 6e2b4368da..9c83f5183d 100644 --- a/guix/scripts/publish.scm +++ b/guix/scripts/publish.scm @@ -58,6 +58,7 @@ (define-module (guix scripts publish) #:use-module (guix workers) #:use-module (guix store) #:use-module ((guix serialization) #:select (write-file)) + #:use-module (eris) #:use-module (zlib) #:autoload (lzlib) (call-with-lzip-output-port make-lzip-output-port) @@ -146,6 +147,9 @@ (define %default-gzip-compression ;; Since we compress on the fly, default to fast compression. (compression 'gzip 3)) +(define %eris-zstd-compression + (compression 'zstd 19)) + (define (default-compression type) (compression type 3)) @@ -324,7 +328,8 @@ (define* (store-item->recutils store-item (define* (narinfo-string store store-path #:key (compressions (list %no-compression)) - (nar-path "nar") (file-sizes '())) + (nar-path "nar") (file-sizes '()) + eris-urn) "Generate a narinfo key/value string for STORE-PATH; an exception is raised if STORE-PATH is invalid. Produce a URL that corresponds to COMPRESSION. The narinfo is signed with KEY. NAR-PATH specifies the prefix for nar URLs. @@ -347,7 +352,7 @@ (define* (narinfo-string store store-path StorePath: ~a ~{~a~}\ NarHash: sha256:~a -NarSize: ~d +NarSize: ~d~@[~%ERISFormat: application/x-nix-archive+zstd-19~%ERIS: ~a~] References: ~a~%" store-path (map (lambda (compression) @@ -359,7 +364,7 @@ (define* (narinfo-string store store-path #:compression compression))) compressions) - hash size references)) + hash size eris-urn references)) ;; Do not render a "Deriver" line if we are rendering info for a ;; derivation. Also do not render a "System" line that would be ;; expensive to compute and is currently unused. @@ -632,6 +637,16 @@ (define (compressed-nar-size compression) (and stat (cons compression (stat:size stat))))) + (define (eris-encode-nar compressions) + (and (member %eris-zstd-compression compressions) + (let* ((nar (nar-cache-file cache item + #:compression %eris-zstd-compression)) + (stat (stat nar #f))) + (and stat + (call-with-input-file nar + (cut eris-encode->string <> + #:block-size %eris-block-size-large)))))) + (let ((compression (actual-compressions item compressions))) (for-each (cut compress-nar cache item <>) compressions) @@ -650,7 +665,8 @@ (define (compressed-nar-size compression) (display (narinfo-string store item #:nar-path nar-path #:compressions compressions - #:file-sizes sizes) + #:file-sizes sizes + #:eris-urn (eris-encode-nar compression)) port))) ;; Make the cached narinfo world-readable, contrary to what From patchwork Tue Jan 25 19:21:59 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: pukkamustard X-Patchwork-Id: 36810 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 510D827BBEA; Tue, 25 Jan 2022 19:26:10 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL, SPF_HELO_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 0887F27BBE9 for ; Tue, 25 Jan 2022 19:26:10 +0000 (GMT) Received: from localhost ([::1]:48536 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nCRSC-0004nL-UE for patchwork@mira.cbaines.net; Tue, 25 Jan 2022 14:26:09 -0500 Received: from eggs.gnu.org ([209.51.188.92]:37980) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nCRPD-000475-Jz for guix-patches@gnu.org; Tue, 25 Jan 2022 14:23:03 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:57253) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1nCRPD-0000B6-9i for guix-patches@gnu.org; Tue, 25 Jan 2022 14:23:03 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1nCRPD-0001Eo-6o for guix-patches@gnu.org; Tue, 25 Jan 2022 14:23:03 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#52555] [RFC PATCH v2 3/5] Add (guix eris). Resent-From: pukkamustard Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Tue, 25 Jan 2022 19:23:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 52555 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 52555@debbugs.gnu.org Cc: pukkamustard , ~pukkamustard/eris@lists.sr.ht Received: via spool by 52555-submit@debbugs.gnu.org id=B52555.16431385634679 (code B ref 52555); Tue, 25 Jan 2022 19:23:03 +0000 Received: (at 52555) by debbugs.gnu.org; 25 Jan 2022 19:22:43 +0000 Received: from localhost ([127.0.0.1]:50148 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nCROs-0001DI-BI for submit@debbugs.gnu.org; Tue, 25 Jan 2022 14:22:43 -0500 Received: from mout01.posteo.de ([185.67.36.65]:36957) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nCROb-0001CE-9T for 52555@debbugs.gnu.org; Tue, 25 Jan 2022 14:22:26 -0500 Received: from submission (posteo.de [89.146.220.130]) by mout01.posteo.de (Postfix) with ESMTPS id B5880240027 for <52555@debbugs.gnu.org>; Tue, 25 Jan 2022 20:22:19 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1643138539; bh=fVwvh/6zrIV6B6hs8ZwDshihybQDuGkAoOjNHeNPWh0=; h=From:To:Cc:Subject:Date:From; b=fTYsV4CtQohwGGN+e0QsRfHniRoyA+0SWQRTT2GBxL7YZcRXo45y1ErrC2zwJvy2O PGflmXpSURrMEQnNriXAl/8Fe66gU++bDKN2+VC6DAoPKaStR8NStByTW+aOSYcv9M 1RhQ0Dlcw1kCPXpJe5qJQPaaTSkGLhFm+xPuNLbXFDxvatMP82Jpmr3917yYvi91eF ClknkBsjLiwHbFnA8oALpQ9uES/59k4i08LYdruI8+2X31Vvoyfx+Wq0DXtlDy8BHk fDTxGjJKyQ+J0yTlRGW+3TvbKzVlwtKrvFq8oB+7h4iYVwfFXgu16tYk8114pkHRo3 lRS+RrCvPAw3Q== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4JjxZ25J44z9rxF; Tue, 25 Jan 2022 20:22:18 +0100 (CET) From: pukkamustard Date: Tue, 25 Jan 2022 19:21:59 +0000 Message-Id: <20220125192201.7582-4-pukkamustard@posteo.net> In-Reply-To: <20220125192201.7582-1-pukkamustard@posteo.net> References: <20220125192201.7582-1-pukkamustard@posteo.net> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches * guix/ipfs.scm: New file. * Makefile.am (MODULES): Add it. --- Makefile.am | 1 + guix/eris.scm | 55 +++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 56 insertions(+) create mode 100644 guix/eris.scm diff --git a/Makefile.am b/Makefile.am index a10aeb817b..7219386361 100644 --- a/Makefile.am +++ b/Makefile.am @@ -131,6 +131,7 @@ MODULES = \ guix/cve.scm \ guix/workers.scm \ guix/ipfs.scm \ + guix/eris.scm \ guix/build-system.scm \ guix/build-system/android-ndk.scm \ guix/build-system/ant.scm \ diff --git a/guix/eris.scm b/guix/eris.scm new file mode 100644 index 0000000000..163bbe05ac --- /dev/null +++ b/guix/eris.scm @@ -0,0 +1,55 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright © 2022 pukkamustard +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see . + +(define-module (guix eris) + #:use-module (eris) + #:use-module (eris blocks ipfs) + #:use-module (web client) + #:use-module (web response) + #:use-module (srfi srfi-71) + + #:export (guix-eris-block-reducer)) + +(define (ipfs-daemon-alive?) + "Attempt to connect to the IPFS daemon. Returns #t if the daemon is alive +and #f else." + (with-exception-handler + (const #f) + (lambda _ + (let ((response _ + (http-post (string-append (%ipfs-base-url) + "/api/v0/version")))) + (equal? 200 (response-code response)))) + #:unwind? #t)) + +(define guix-eris-block-reducer + (case-lambda + + ;; Check if IPFS Daemon is running. + (() (if (ipfs-daemon-alive?) + (eris-blocks-ipfs-reducer) + #f)) + + ;; Completion. Nothing to do. + ((_) #t) + + ((ipfs ref-block) + ;; If IPFS has been initialized store block there + (if ipfs + (eris-blocks-ipfs-reducer ipfs ref-block) + ipfs)))) From patchwork Tue Jan 25 19:22:00 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: pukkamustard X-Patchwork-Id: 36811 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 5040027BBEA; Tue, 25 Jan 2022 19:26:45 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL, SPF_HELO_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id CD46927BBE9 for ; Tue, 25 Jan 2022 19:26:44 +0000 (GMT) Received: from localhost ([::1]:49762 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nCRSm-0005XT-0U for patchwork@mira.cbaines.net; Tue, 25 Jan 2022 14:26:44 -0500 Received: from eggs.gnu.org ([209.51.188.92]:37986) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nCRPD-00047a-VQ for guix-patches@gnu.org; Tue, 25 Jan 2022 14:23:03 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:57254) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1nCRPD-0000BE-Lv for guix-patches@gnu.org; Tue, 25 Jan 2022 14:23:03 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1nCRPD-0001Ew-JG for guix-patches@gnu.org; Tue, 25 Jan 2022 14:23:03 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#52555] [RFC PATCH v2 4/5] publish: Add support for storing ERIS encoded blocks to IPFS. Resent-From: pukkamustard Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Tue, 25 Jan 2022 19:23:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 52555 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 52555@debbugs.gnu.org Cc: pukkamustard , ~pukkamustard/eris@lists.sr.ht Received: via spool by 52555-submit@debbugs.gnu.org id=B52555.16431385634686 (code B ref 52555); Tue, 25 Jan 2022 19:23:03 +0000 Received: (at 52555) by debbugs.gnu.org; 25 Jan 2022 19:22:43 +0000 Received: from localhost ([127.0.0.1]:50150 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nCROt-0001DQ-8v for submit@debbugs.gnu.org; Tue, 25 Jan 2022 14:22:43 -0500 Received: from mout02.posteo.de ([185.67.36.66]:59993) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nCROd-0001CH-SK for 52555@debbugs.gnu.org; Tue, 25 Jan 2022 14:22:28 -0500 Received: from submission (posteo.de [89.146.220.130]) by mout02.posteo.de (Postfix) with ESMTPS id 4C66F240103 for <52555@debbugs.gnu.org>; Tue, 25 Jan 2022 20:22:22 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1643138542; bh=WjdsN5qtP47bgM3Qv/+NX+Fy8ZzMEmAQi9X/KFRZPLo=; h=From:To:Cc:Subject:Date:From; b=QlZsIs6HZgRLJycSuQJN9og4N1nW8ttJnDB8/C1w9CfdkTH18W0i6o1cj1enMolVg psxeNyntX7MBIcBMHLjeFdKe0d8JYP+pmvcLNn2FKoiS5XP+Xnl+TYjQYfylPHUo65 iX+hB9mXQVP1XmxFngAb4CwoYji/B3RQ/p/FrlkBjlsGgjM8noz0jfq8g5nBg2rfmN WvkKaLA8lTM9oHN6OOPU4JibOgz5IqF+d0OWx8G3+5w4OlVG44iC70tsYYVinRHFl3 tjmeTxbM6FT8TMZ4tZj8YXOaTo/Gz/btZ3qd8srheuo5Wr4No9TYT8reev2mVywF+r XbIa2SwyoD0Cg== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4JjxZ54K7cz9rxS; Tue, 25 Jan 2022 20:22:21 +0100 (CET) From: pukkamustard Date: Tue, 25 Jan 2022 19:22:00 +0000 Message-Id: <20220125192201.7582-5-pukkamustard@posteo.net> In-Reply-To: <20220125192201.7582-1-pukkamustard@posteo.net> References: <20220125192201.7582-1-pukkamustard@posteo.net> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches * guix/scripts/publish.scm (bake-narinfo+nar): Use guix-eris-block-reducer from (guix eris). --- guix/scripts/publish.scm | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/guix/scripts/publish.scm b/guix/scripts/publish.scm index 9c83f5183d..556107ab7d 100644 --- a/guix/scripts/publish.scm +++ b/guix/scripts/publish.scm @@ -41,6 +41,7 @@ (define-module (guix scripts publish) #:use-module (srfi srfi-26) #:use-module (srfi srfi-34) #:use-module (srfi srfi-37) + #:use-module (srfi srfi-71) #:use-module (web http) #:use-module (web request) #:use-module (web response) @@ -58,6 +59,7 @@ (define-module (guix scripts publish) #:use-module (guix workers) #:use-module (guix store) #:use-module ((guix serialization) #:select (write-file)) + #:use-module (guix eris) #:use-module (eris) #:use-module (zlib) #:autoload (lzlib) (call-with-lzip-output-port @@ -644,8 +646,14 @@ (define (eris-encode-nar compressions) (stat (stat nar #f))) (and stat (call-with-input-file nar - (cut eris-encode->string <> - #:block-size %eris-block-size-large)))))) + (lambda (port) + (let ((eris-urn _ + (eris-encode port + #:block-size + %eris-block-size-large + #:block-reducer + guix-eris-block-reducer))) + (uri->string eris-urn)))))))) (let ((compression (actual-compressions item compressions))) From patchwork Tue Jan 25 19:22:01 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: pukkamustard X-Patchwork-Id: 36808 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 1F48727BBEA; Tue, 25 Jan 2022 19:23:33 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL, SPF_HELO_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id D125C27BBE9 for ; Tue, 25 Jan 2022 19:23:32 +0000 (GMT) Received: from localhost ([::1]:47254 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nCRPf-0004EA-V2 for patchwork@mira.cbaines.net; Tue, 25 Jan 2022 14:23:31 -0500 Received: from eggs.gnu.org ([209.51.188.92]:37992) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nCRPE-000496-PQ for guix-patches@gnu.org; Tue, 25 Jan 2022 14:23:04 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:57256) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1nCRPE-0000BV-E7 for guix-patches@gnu.org; Tue, 25 Jan 2022 14:23:04 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1nCRPE-0001FA-BX for guix-patches@gnu.org; Tue, 25 Jan 2022 14:23:04 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#52555] [RFC PATCH v2 5/5] substitute: Fetch substitutes using ERIS. Resent-From: pukkamustard Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Tue, 25 Jan 2022 19:23:04 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 52555 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 52555@debbugs.gnu.org Cc: pukkamustard , ~pukkamustard/eris@lists.sr.ht Received: via spool by 52555-submit@debbugs.gnu.org id=B52555.16431385644699 (code B ref 52555); Tue, 25 Jan 2022 19:23:04 +0000 Received: (at 52555) by debbugs.gnu.org; 25 Jan 2022 19:22:44 +0000 Received: from localhost ([127.0.0.1]:50154 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nCROt-0001De-So for submit@debbugs.gnu.org; Tue, 25 Jan 2022 14:22:44 -0500 Received: from mout01.posteo.de ([185.67.36.65]:41807) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nCROf-0001CR-O1 for 52555@debbugs.gnu.org; Tue, 25 Jan 2022 14:22:30 -0500 Received: from submission (posteo.de [89.146.220.130]) by mout01.posteo.de (Postfix) with ESMTPS id 2BFA2240029 for <52555@debbugs.gnu.org>; Tue, 25 Jan 2022 20:22:24 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1643138544; bh=j7T0v7hpPHZH+YN4wChkVCZNNIqTEpqBrEMRdms1Xc4=; h=From:To:Cc:Subject:Date:From; b=aCsMaZzdU2NgT+wR3r3pQ6awfvPTBY/ROM+4GQsQrm0YW0BD+bgZsQiRm7MdfRV2/ vx/b76cm1NwC+84qbrS2xAyMHdLiC1nNu/YHZ6oRKLoPL6akpybgp3MITSUU99pMXf IJHqO7EuONC5H0GwBXjD9Jz+PfX6nIXkl1a5huc1bKbNF0w7r1NN4pfgVUsARHdVgz 3jw2phmXQCDJ1Ta6NCAHztahZKj9mkP7lsROG9+FjJnDcRjZPThEuE6iSYk8oS7Mzp yAcXjYxeajUzQssVumj3m1pTiaguZYKhdeFVZbwU/OFD/me0V5RKu/Op0zzZeHLP1L 8bmk5mH9vFRxQ== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4JjxZ72hWgz9rxV; Tue, 25 Jan 2022 20:22:23 +0100 (CET) From: pukkamustard Date: Tue, 25 Jan 2022 19:22:01 +0000 Message-Id: <20220125192201.7582-6-pukkamustard@posteo.net> In-Reply-To: <20220125192201.7582-1-pukkamustard@posteo.net> References: <20220125192201.7582-1-pukkamustard@posteo.net> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches * guix/scripts/substitute.scm (process-substitution): Fetch substitutes using ERIS. * guix/eris.scm (guix-eris-block-ref): New procedure. --- guix/eris.scm | 7 ++++++- guix/scripts/substitute.scm | 21 ++++++++++++++++----- 2 files changed, 22 insertions(+), 6 deletions(-) diff --git a/guix/eris.scm b/guix/eris.scm index 163bbe05ac..0999564c1f 100644 --- a/guix/eris.scm +++ b/guix/eris.scm @@ -23,7 +23,8 @@ (define-module (guix eris) #:use-module (web response) #:use-module (srfi srfi-71) - #:export (guix-eris-block-reducer)) + #:export (guix-eris-block-reducer + guix-eris-block-ref)) (define (ipfs-daemon-alive?) "Attempt to connect to the IPFS daemon. Returns #t if the daemon is alive @@ -53,3 +54,7 @@ (define guix-eris-block-reducer (if ipfs (eris-blocks-ipfs-reducer ipfs ref-block) ipfs)))) + +(define (guix-eris-block-ref ref) + "Dereference a block for decoding ERIS content" + (eris-blocks-ipfs-ref ref)) diff --git a/guix/scripts/substitute.scm b/guix/scripts/substitute.scm index 908a8334a8..852264976e 100755 --- a/guix/scripts/substitute.scm +++ b/guix/scripts/substitute.scm @@ -62,6 +62,8 @@ (define-module (guix scripts substitute) #:use-module (srfi srfi-35) #:use-module (web uri) #:use-module (guix http-client) + #:use-module (guix eris) + #:use-module (eris) #:export (%allow-unauthenticated-substitutes? %reply-file-descriptor @@ -486,18 +488,27 @@ (define (fetch uri) #:port port #:keep-alive? #t #:buffered? #f))))) + (else - (leave (G_ "unsupported substitute URI scheme: ~a~%") - (uri->string uri))))) + (if (and (eris-read-capability? uri)) + (values (eris-decode->port uri + #:block-ref + guix-eris-block-ref) #f) + (leave (G_ "unsupported substitute URI scheme: ~a~%") + (uri->string uri)))))) + + (define* (best-uri narinfo #:key (eris? #f)) + (if (and eris? (narinfo-eris-urn narinfo)) + (values (narinfo-eris-urn narinfo) "zstd" #f) + (narinfo-best-uri narinfo #:fast-decompression? + %prefer-fast-decompression?))) (unless narinfo (leave (G_ "no valid substitute for '~a'~%") store-item)) (let-values (((uri compression file-size) - (narinfo-best-uri narinfo - #:fast-decompression? - %prefer-fast-decompression?))) + (best-uri narinfo #:eris? #t))) (unless print-build-trace? (format (current-error-port) (G_ "Downloading ~a...~%") (uri->string uri)))