mbox

[bug#44800,0/2] publish: Add Avahi support.

Message ID 20201122155659.67235-1-othacehe@gnu.org
Headers show

Message

Mathieu Othacehe Nov. 22, 2020, 3:56 p.m. UTC 
Hello,

Here are two patches adding Avahi support to "guix publish". When the
"--enable-avahi" option is passed to "guix publish", the server is advertised
on the local network.

Future patches will use this option to automatically add all the publish
servers on the local network to the daemon "substitute-urls" list.

Thanks,

Mathieu

Mathieu Othacehe (2):
  Add Avahi support.
  publish: Add avahi support.

 Makefile.am                         |   1 +
 configure.ac                        |   6 ++
 doc/guix.texi                       |   5 +
 gnu/packages/package-management.scm |   2 +
 guix/avahi.scm                      | 162 ++++++++++++++++++++++++++++
 guix/scripts/publish.scm            |  23 ++++
 guix/self.scm                       |   9 +-
 7 files changed, 206 insertions(+), 2 deletions(-)
 create mode 100644 guix/avahi.scm

Comments

Simon Tournier Nov. 23, 2020, 10:04 p.m. UTC | #1 
Hi Mathieu,

On Sun, 22 Nov 2020 at 16:56, Mathieu Othacehe <othacehe@gnu.org> wrote:

> Here are two patches adding Avahi support to "guix publish". When the
> "--enable-avahi" option is passed to "guix publish", the server is advertised
> on the local network.

Is this part of your new design?

> Future patches will use this option to automatically add all the publish
> servers on the local network to the daemon "substitute-urls" list.

What happen if the option is passed but the servers are not reliable
(the connection is unstable)?


All the best,
simon
Mathieu Othacehe Nov. 24, 2020, 1:21 p.m. UTC | #2 
Hello,

Here's a v2 that now provides a complete chain. "guix publish" services that
are started with "--enable-avahi" will be discovered by the "guix-daemon" when
the "--use-local-publish" option is passed.

A new "guix discover" script is used to discover publish services in the
background. I have tested this setup by starting a guix-publish server in a
VM.

As suggested here:
https://guix.gnu.org/blog/2017/reproducible-builds-a-status-update/ the local
publish servers are prefixed to the substitute-url list, so that the daemon
will try to use substitute from the local network first, before falling back
to the CI build farm.

Thanks,

Mathieu

Mathieu Othacehe (3):
  Add Avahi support.
  publish: Add avahi support.
  Use substitute servers on the local network.

 Makefile.am                         |   2 +
 configure.ac                        |   6 +
 doc/guix.texi                       |  13 +++
 gnu/packages/package-management.scm |   5 +-
 gnu/services/base.scm               |  17 ++-
 guix/avahi.scm                      | 170 ++++++++++++++++++++++++++++
 guix/scripts/discover.scm           | 158 ++++++++++++++++++++++++++
 guix/scripts/publish.scm            |  24 ++++
 guix/scripts/substitute.scm         |  11 +-
 guix/self.scm                       |   9 +-
 nix/libstore/globals.cc             |   1 +
 nix/libstore/globals.hh             |   4 +
 nix/nix-daemon/guix-daemon.cc       |  20 ++++
 13 files changed, 433 insertions(+), 7 deletions(-)
 create mode 100644 guix/avahi.scm
 create mode 100644 guix/scripts/discover.scm
Mathieu Othacehe Nov. 24, 2020, 1:35 p.m. UTC | #3 
Hey zimoun,

> Is this part of your new design?

Somehow because it allows the machines on a local network to share
substitutes between each other transparently.

It not strictly needed however because the offloading workers directly
add the offloading server to their substitute list in the design I'm
proposing.

> What happen if the option is passed but the servers are not reliable
> (the connection is unstable)?

The build fails with the following message:

--8<---------------cut here---------------start------------->8---
guix substitute: error: connect: No route to host
substitution of /gnu/store/pbaihd2k2dbv02s7aq3aybk29r00lg9h-abduco-0.6 failed
guix build: error: some substitutes for the outputs of derivation `/gnu/store/xnfyilqfhk05rgxbq7hmx81d65bsnprh-abduco-0.6.drv' failed (usually happens due to networking issues); try `--fallback' to build derivation from source 
--8<---------------cut here---------------end--------------->8---

Thanks,

Mathieu