| Message ID | 20220125192201.7582-6-pukkamustard@posteo.net |
|---|---|
| State | New |
| Headers | show |
| Series | Decentralized substitute distribution with ERIS | expand |
| Context | Check | Description |
|---|---|---|
| cbaines/comparison | success | View comparision |
| cbaines/git branch | success | View Git branch |
| cbaines/applying patch | success | View Laminar job |
| cbaines/issue | success | View issue |
pukkamustard schreef op di 25-01-2022 om 19:22 [+0000]: > +(define (guix-eris-block-ref ref) > + "Dereference a block for decoding ERIS content" > + (eris-blocks-ipfs-ref ref)) 'guix-eris-block-ref' just calls 'eris-blocks-ipfs-ref', so I'm not seeing the point of this procedure. Greetings, Maxime.
pukkamustard schreef op di 25-01-2022 om 19:22 [+0000]: > + Superfluous new empty line? > (else > - (leave (G_ "unsupported substitute URI scheme: ~a~%") > - (uri->string uri))))) > + (if (and (eris-read-capability? uri)) guile-eris (which has the eris-read-capability? procedure) is an optional dependency, so you have to check here is Guix was compiled with guile-eris. Greetings, Maxime.
pukkamustard schreef op di 25-01-2022 om 19:22 [+0000]: > + (define* (best-uri narinfo #:key (eris? #f)) > + (if (and eris? (narinfo-eris-urn narinfo)) > + (values (narinfo-eris-urn narinfo) "zstd" #f) > + (narinfo-best-uri narinfo #:fast-decompression? > + %prefer-fast-decompression?))) When Guix is compiled without guile-eris support, '(and eris? (narinfo-eris-urn narinfo))' is the worst, not the best.
pukkamustard schreef op di 25-01-2022 om 19:22 [+0000]: > + (define* (best-uri narinfo #:key (eris? #f)) > + (if (and eris? (narinfo-eris-urn narinfo)) > + (values (narinfo-eris-urn narinfo) "zstd" #f) > + (narinfo-best-uri narinfo #:fast-decompression? > + %prefer-fast-decompression?))) Why is ERIS the best here? Fast download speeds, decentralisation, less network I/O, less heat production, more pronouncable than HTTPS? I would add a comment here. Greetings, Maxime.
Maxime Devos schreef op za 29-01-2022 om 22:38 [+0100]: > pukkamustard schreef op di 25-01-2022 om 19:22 [+0000]: > > + (define* (best-uri narinfo #:key (eris? #f)) > > + (if (and eris? (narinfo-eris-urn narinfo)) > > + (values (narinfo-eris-urn narinfo) "zstd" #f) > > + (narinfo-best-uri narinfo #:fast-decompression? > > + %prefer-fast-decompression?))) > > When Guix is compiled without guile-eris support, > '(and eris? (narinfo-eris-urn narinfo))' is the worst, not the best. Nevermind, that's what (eris? #f) is for, I presume? Greetings, Maxime.
Maxime Devos <maximedevos@telenet.be> writes: > [[PGP Signed Part:Undecided]] > pukkamustard schreef op di 25-01-2022 om 19:22 [+0000]: >> +(define (guix-eris-block-ref ref) >> + "Dereference a block for decoding ERIS content" >> + (eris-blocks-ipfs-ref ref)) > > 'guix-eris-block-ref' just calls 'eris-blocks-ipfs-ref', > so I'm not seeing the point of this procedure. Yes, currently it is an unnecessary level of abstraction. The idea is that when there are multiple backends/transports they are multiplexed here. E.g. guix-eris-block-ref would attempt to use HTTP/IPFS or whatever to get the block. Whatever calls guix-eris-block-ref does not need to know from where the blocks come. I hope to make this more clear in a V3 that will add HTTP transport. -pukkamustard
Maxime Devos <maximedevos@telenet.be> writes: > [[PGP Signed Part:Undecided]] > pukkamustard schreef op di 25-01-2022 om 19:22 [+0000]: >> + (define* (best-uri narinfo #:key (eris? #f)) >> + (if (and eris? (narinfo-eris-urn narinfo)) >> + (values (narinfo-eris-urn narinfo) "zstd" #f) >> + (narinfo-best-uri narinfo #:fast-decompression? >> + %prefer-fast-decompression?))) > > Why is ERIS the best here? Fast download speeds, decentralisation, > less network I/O, less heat production, more pronouncable than HTTPS? > I would add a comment here. Those are all possible reasons. I think we first need to do some experiments to see if any of those claims can be justified. In general, the logic for when to use ERIS transports needs more thought. For one, I think it should be user configurable. I can imagine that certain users do not want to use decentralized substitutes at all. Users should be able to deactivate the entire ERIS thing. For the default, I personally think it would be ok to try and use ERIS. The only thing we absolutely need is a clean fallback logic that transparently falls back to getting the entire NAR by HTTP. Currently such a fallback is not yet implemented. -pukkamustard
diff --git a/guix/eris.scm b/guix/eris.scm index 163bbe05ac..0999564c1f 100644 --- a/guix/eris.scm +++ b/guix/eris.scm @@ -23,7 +23,8 @@ (define-module (guix eris) #:use-module (web response) #:use-module (srfi srfi-71) - #:export (guix-eris-block-reducer)) + #:export (guix-eris-block-reducer + guix-eris-block-ref)) (define (ipfs-daemon-alive?) "Attempt to connect to the IPFS daemon. Returns #t if the daemon is alive @@ -53,3 +54,7 @@ (define guix-eris-block-reducer (if ipfs (eris-blocks-ipfs-reducer ipfs ref-block) ipfs)))) + +(define (guix-eris-block-ref ref) + "Dereference a block for decoding ERIS content" + (eris-blocks-ipfs-ref ref)) diff --git a/guix/scripts/substitute.scm b/guix/scripts/substitute.scm index 908a8334a8..852264976e 100755 --- a/guix/scripts/substitute.scm +++ b/guix/scripts/substitute.scm @@ -62,6 +62,8 @@ (define-module (guix scripts substitute) #:use-module (srfi srfi-35) #:use-module (web uri) #:use-module (guix http-client) + #:use-module (guix eris) + #:use-module (eris) #:export (%allow-unauthenticated-substitutes? %reply-file-descriptor @@ -486,18 +488,27 @@ (define (fetch uri) #:port port #:keep-alive? #t #:buffered? #f))))) + (else - (leave (G_ "unsupported substitute URI scheme: ~a~%") - (uri->string uri))))) + (if (and (eris-read-capability? uri)) + (values (eris-decode->port uri + #:block-ref + guix-eris-block-ref) #f) + (leave (G_ "unsupported substitute URI scheme: ~a~%") + (uri->string uri)))))) + + (define* (best-uri narinfo #:key (eris? #f)) + (if (and eris? (narinfo-eris-urn narinfo)) + (values (narinfo-eris-urn narinfo) "zstd" #f) + (narinfo-best-uri narinfo #:fast-decompression? + %prefer-fast-decompression?))) (unless narinfo (leave (G_ "no valid substitute for '~a'~%") store-item)) (let-values (((uri compression file-size) - (narinfo-best-uri narinfo - #:fast-decompression? - %prefer-fast-decompression?))) + (best-uri narinfo #:eris? #t))) (unless print-build-trace? (format (current-error-port) (G_ "Downloading ~a...~%") (uri->string uri)))