mbox series

[bug#67072,0/4] Helping diagnose substitute setup issues

Message ID cover.1699700049.git.ludo@gnu.org
Headers show
Series Helping diagnose substitute setup issues | expand

Message

Ludovic Courtès Nov. 11, 2023, 11:03 a.m. UTC
Hello Guix!

While discussing at the Reproducible Software Environments Workshop
yesterday, Emmanuel Agullo and Simon Tournier suggested adding
tools to help diagnose substitute setup issues: to see which
substitutes URLs are being used and whether one of them is unauthorized.

This is a step in that direction.  First ‘guix weather’ and ‘guix
challenge’ now default to the same substitute URLs as guix-daemon
(this was not the case until now because there was no way to get
that information from the daemon).  Second ‘guix weather’ reports
about unauthorized servers, like so:

--8<---------------cut here---------------start------------->8---
$ guix weather coreutils
computing 1 package derivations for x86_64-linux...
looking for 2 store items on https://ci.guix.gnu.org...
guix weather: warning: substitutes from 'https://ci.guix.gnu.org' are unauthorized
hint: To authorize substitute download from `https://ci.guix.gnu.org', the following command
needs to be run as root:

     guix archive --authorize <<EOF
     (public-key 
      (ecc 
       (curve Ed25519)
       (q #8D156F295D24B0D9A86FA5741A840FF2D24F60F7B6C4134814AD55625971B394#)
       )
      )
     
     EOF

Alternatively, on Guix System, you can add the signing key above to the
`authorized-keys' field of `guix-configuration'.

See "Getting Substitutes from Other Servers" in the manual for more information.

https://ci.guix.gnu.org ☀
  100.0% substitutes available (2 out of 2)
  at least 19.3 MiB of nars (compressed)
  25.3 MiB on disk (uncompressed)
[…]
--8<---------------cut here---------------end--------------->8---

It turned out to be a low-hanging fruit!

Thoughts?

Ludo’.

Ludovic Courtès (4):
  daemon: Implement ‘substitute-urls’ RPC.
  challenge: Use the same substitute URLs as guix-daemon.
  weather: Use the same substitute URLs as guix-daemon.
  weather: Report unauthorized substitute servers.

 doc/guix.texi                   | 26 ++++++++++++++++---
 guix/scripts/challenge.scm      | 11 +++++---
 guix/scripts/weather.scm        | 46 ++++++++++++++++++++++++++++++---
 guix/store.scm                  | 18 ++++++++++---
 nix/libstore/worker-protocol.hh |  5 ++--
 nix/nix-daemon/nix-daemon.cc    | 17 ++++++++++++
 tests/store.scm                 | 25 ++++++++++++++++--
 7 files changed, 132 insertions(+), 16 deletions(-)


base-commit: 08d94fe20eca47b69678b3eced8749dd02c700a4

Comments

Ludovic Courtès Nov. 27, 2023, 5:21 p.m. UTC | #1
Hello!

Comments or suggestions regarding this change?

  https://issues.guix.gnu.org/67072

If not I’d like to push it soon.

TIA.  :-)

Ludo’.

Ludovic Courtès <ludo@gnu.org> skribis:

> Hello Guix!
>
> While discussing at the Reproducible Software Environments Workshop
> yesterday, Emmanuel Agullo and Simon Tournier suggested adding
> tools to help diagnose substitute setup issues: to see which
> substitutes URLs are being used and whether one of them is unauthorized.
>
> This is a step in that direction.  First ‘guix weather’ and ‘guix
> challenge’ now default to the same substitute URLs as guix-daemon
> (this was not the case until now because there was no way to get
> that information from the daemon).  Second ‘guix weather’ reports
> about unauthorized servers, like so:
>
> $ guix weather coreutils
> computing 1 package derivations for x86_64-linux...
> looking for 2 store items on https://ci.guix.gnu.org...
> guix weather: warning: substitutes from 'https://ci.guix.gnu.org' are unauthorized
> hint: To authorize substitute download from `https://ci.guix.gnu.org', the following command
> needs to be run as root:
>
>      guix archive --authorize <<EOF
>      (public-key 
>       (ecc 
>        (curve Ed25519)
>        (q #8D156F295D24B0D9A86FA5741A840FF2D24F60F7B6C4134814AD55625971B394#)
>        )
>       )
>      
>      EOF
>
> Alternatively, on Guix System, you can add the signing key above to the
> `authorized-keys' field of `guix-configuration'.
>
> See "Getting Substitutes from Other Servers" in the manual for more information.
>
> https://ci.guix.gnu.org ☀
>   100.0% substitutes available (2 out of 2)
>   at least 19.3 MiB of nars (compressed)
>   25.3 MiB on disk (uncompressed)
> […]
>
> It turned out to be a low-hanging fruit!
>
> Thoughts?
>
> Ludo’.
>
> Ludovic Courtès (4):
>   daemon: Implement ‘substitute-urls’ RPC.
>   challenge: Use the same substitute URLs as guix-daemon.
>   weather: Use the same substitute URLs as guix-daemon.
>   weather: Report unauthorized substitute servers.
>
>  doc/guix.texi                   | 26 ++++++++++++++++---
>  guix/scripts/challenge.scm      | 11 +++++---
>  guix/scripts/weather.scm        | 46 ++++++++++++++++++++++++++++++---
>  guix/store.scm                  | 18 ++++++++++---
>  nix/libstore/worker-protocol.hh |  5 ++--
>  nix/nix-daemon/nix-daemon.cc    | 17 ++++++++++++
>  tests/store.scm                 | 25 ++++++++++++++++--
>  7 files changed, 132 insertions(+), 16 deletions(-)
>
>
> base-commit: 08d94fe20eca47b69678b3eced8749dd02c700a4
Simon Tournier Nov. 28, 2023, 1:17 p.m. UTC | #2
Hi Ludo,

Sorry for the delay.

On Mon, 27 Nov 2023 at 18:21, Ludovic Courtès <ludovic.courtes@inria.fr> wrote:

> Comments or suggestions regarding this change?
>
>   https://issues.guix.gnu.org/67072

Cool!

Well, I did some minor comments about “guix weather”.  I have not
checked the “guix challenge” counter-part.  Maybe later this week. :-)

Cheers,
simon
Emmanuel Agullo Nov. 30, 2023, 10:11 a.m. UTC | #3
Hello Ludo, hello Simon, hello Guix,

First of all thank you Ludo for the patch.

Indeed, it would be great one can check the substitutes are set up
as expected.

>> First ‘guix weather’ and ‘guix
>> challenge’ now default to the same substitute URLs as guix-daemon
>> (this was not the case until now because there was no way to get
>> that information from the daemon).

This is excellent. I was not aware of the difference.

>>  Second ‘guix weather’ reports about unauthorized servers

I guess it should help a lot!

>> If not I’d like to push it soon.

As best as I can read, green light for me. I'll be pleased to test
it once pushed.

Thanks again!

Best,

Manu
Ludovic Courtès Nov. 30, 2023, 10:28 a.m. UTC | #4
Hi Emmanuel,

Emmanuel Agullo <emmanuel.agullo@inria.fr> skribis:

>>> First ‘guix weather’ and ‘guix
>>> challenge’ now default to the same substitute URLs as guix-daemon
>>> (this was not the case until now because there was no way to get
>>> that information from the daemon).
>
> This is excellent. I was not aware of the difference.
>
>>>  Second ‘guix weather’ reports about unauthorized servers
>
> I guess it should help a lot!
>
>>> If not I’d like to push it soon.
>
> As best as I can read, green light for me. I'll be pleased to test
> it once pushed.

Awesome, thanks for your feedback!

Ludo’.