Message ID | cover.1695386493.git.ludo@gnu.org |
---|---|
Headers |
Return-Path: <guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org> X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 72D3827BBEA; Fri, 22 Sep 2023 13:54:19 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,SPF_HELO_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 5BB8527BBE2 for <patchwork@mira.cbaines.net>; Fri, 22 Sep 2023 13:54:18 +0100 (BST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from <guix-patches-bounces@gnu.org>) id 1qjffN-0000rA-Q9; Fri, 22 Sep 2023 08:53:53 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <Debian-debbugs@debbugs.gnu.org>) id 1qjffL-0000lu-L5 for guix-patches@gnu.org; Fri, 22 Sep 2023 08:53:51 -0400 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from <Debian-debbugs@debbugs.gnu.org>) id 1qjffL-0007bL-BZ for guix-patches@gnu.org; Fri, 22 Sep 2023 08:53:51 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from <Debian-debbugs@debbugs.gnu.org>) id 1qjffV-0005ce-Sv for guix-patches@gnu.org; Fri, 22 Sep 2023 08:54:01 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#66156] [PATCH 00/12] Introducing Smart Hurdloading Resent-From: Ludovic =?utf-8?q?Court=C3=A8s?= <ludo@gnu.org> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces@debbugs.gnu.org> Resent-CC: guix-patches@gnu.org Resent-Date: Fri, 22 Sep 2023 12:54:01 +0000 Resent-Message-ID: <handler.66156.B.169538718621537@debbugs.gnu.org> Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 66156 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 66156@debbugs.gnu.org Cc: Ludovic =?utf-8?q?Court=C3=A8s?= <ludo@gnu.org>, Josselin Poiret <dev@jpoiret.xyz>, Janneke Nieuwenhuizen <janneke@gnu.org> X-Debbugs-Original-To: guix-patches@gnu.org Received: via spool by submit@debbugs.gnu.org id=B.169538718621537 (code B ref -1); Fri, 22 Sep 2023 12:54:01 +0000 Received: (at submit) by debbugs.gnu.org; 22 Sep 2023 12:53:06 +0000 Received: from localhost ([127.0.0.1]:35546 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces@debbugs.gnu.org>) id 1qjfec-0005bJ-6i for submit@debbugs.gnu.org; Fri, 22 Sep 2023 08:53:06 -0400 Received: from lists.gnu.org ([2001:470:142::17]:58300) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <ludo@gnu.org>) id 1qjfea-0005ah-JI for submit@debbugs.gnu.org; Fri, 22 Sep 2023 08:53:04 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <ludo@gnu.org>) id 1qjfeI-0008Vk-Va for guix-patches@gnu.org; Fri, 22 Sep 2023 08:52:47 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <ludo@gnu.org>) id 1qjfeE-0007UY-AQ; Fri, 22 Sep 2023 08:52:43 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:Date:Subject:To:From:in-reply-to: references; bh=lq8K8dMi+KVea48r7S7IdT1m3buQEnjwhZ0EqyNcQVo=; b=hBotmSNAEchxMV zKdUtoo3OcLfWJkMCAOPlZUjp8u4LN3rAfewEpNNHarOytYu97Oz2b0akhLF/UrIufvdBPJDQ5DVH pm4fU8bZnt9ptd+vFKWRYb4EmtPBWS7EbJDpY6tTVrpef14qnvlkyMQmZ5dA6/bKEcGbnTPayyqgn xfvID5yypkOkJb2sY+I7H1dtuUyFSUcdacdCb3TcssXQetBJcSUPSwHAlXxVfofQ2SBHh5MCh/9up p3cDW61B/XqhMyGzmdSgROCfdHGJ4eAuASMFK/m5JZMzW7kPIE5tK2960olvLH2so6CaaZZsZwOn6 EqGZ6GDm1ZGSFS6IHMNw==; From: Ludovic =?utf-8?q?Court=C3=A8s?= <ludo@gnu.org> Date: Fri, 22 Sep 2023 14:52:27 +0200 Message-ID: <cover.1695386493.git.ludo@gnu.org> X-Mailer: git-send-email 2.41.0 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: <guix-patches.gnu.org> List-Unsubscribe: <https://lists.gnu.org/mailman/options/guix-patches>, <mailto:guix-patches-request@gnu.org?subject=unsubscribe> List-Archive: <https://lists.gnu.org/archive/html/guix-patches> List-Post: <mailto:guix-patches@gnu.org> List-Help: <mailto:guix-patches-request@gnu.org?subject=help> List-Subscribe: <https://lists.gnu.org/mailman/listinfo/guix-patches>, <mailto:guix-patches-request@gnu.org?subject=subscribe> Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org X-getmail-retrieved-from-mailbox: Patches |
Series |
Introducing Smart Hurdloading
|
|
Message
Ludovic Courtès
Sept. 22, 2023, 12:52 p.m. UTC
This patch series, my friends, sets up Smart Hurdloading™, sometimes referred to as “Hurd auto-offloading”. (Yeah I worked hard on branding…) Concretely, if you add (service hurd-vm-service-type), you can now run: guix build libreoffice -s i586-gnu on your machine, and the childhurd will take care of keeping one CPU core busy for a while. There’s no configuration involved beyond this one (service hurd-vm-service-type) line: no need to exchange SSH keys, substitute keys, write /etc/guix/machines.scm, and all that. It’s all taken care off automatically. One key element here is support for declarative and extensible offloading configuration—meaning that /etc/guix/machines.scm is now optionally generated straight from your ‘operating-system’ config. The rest is about generating keys, authorizing them, etc. The series is a bit of a potpourri: I fixed Hurd issues here and there (currently offloading to a childhurd doesn’t work on ‘master’, because of the locale issue, for instance), improved documentation, etc. I also took a couple of long detours not shown here that might lead to further improvements in the future. My goal is to extend this mechanism beyond the Hurd, to have a generic mechanism to spin up Guix System VMs we can easily offload to. Thoughts? Ludo’. Ludovic Courtès (12): system: vm: Remove unused variable. secret-service: Increase default handshake timeout. services: hurd-vm: Use the default SSH port number. gnu: glibc-utf8-locales: Reintroduce input labels. services: guix: Use the right locale package on GNU/Hurd. services: guix: Support declarative offloading setup. services: childhurd: Authorize the childhurd’s key on the host. services: hurd-vm: ‘image’ field has to be an <image> record. tests: hurd-vm: Remove custom disk image configuration. services: hurd-vm: Disable password-based authentication for root. doc: Give an example showing how to add an account in the childhurd. services: hurd-vm: Implement zero-configuration offloading. doc/guix.texi | 167 ++++++++++++++++++++++++++------ gnu/build/secret-service.scm | 2 +- gnu/packages/base.scm | 4 +- gnu/services/base.scm | 58 ++++++++++- gnu/services/virtualization.scm | 153 ++++++++++++++++++++++++++--- gnu/system/vm.scm | 41 ++------ gnu/tests/virtualization.scm | 68 ++++++++++--- 7 files changed, 394 insertions(+), 99 deletions(-) base-commit: 3d8d67ef6928f5d81118c97f03372cd341eab8b0
Comments
Ludovic Courtès writes: Hi Ludo, > This patch series, my friends, sets up Smart Hurdloading™, sometimes > referred to as “Hurd auto-offloading”. (Yeah I worked hard on branding…) > > Concretely, if you add (service hurd-vm-service-type), you can now run: > > guix build libreoffice -s i586-gnu Beautiful! I'm wondering how this works wrt disk size; the default is still 'guess? In my childhurds I have (hurd-vm-configuration .. (disk-size (* 16 1024 (expt 2 20))) ;16GiB which is pretty cheap now that we have qcow2. I'm not sure how expensive it is to set memory-size if you don't use it? > The series is a bit of a potpourri: I fixed Hurd issues here and there > (currently offloading to a childhurd doesn’t work on ‘master’, because > of the locale issue, for instance), improved documentation, etc. I also > took a couple of long detours not shown here that might lead to further > improvements in the future. That's great, very nice. And quite understandable afaic, the hurd-team branch has also been quite a mixed set of work. > My goal is to extend this mechanism beyond the Hurd, to have a generic > mechanism to spin up Guix System VMs we can easily offload to. > > Thoughts? I'm wondering if there is a way to/if we need a way to set the default priority of the childhurd. I'm usually offloading to another machine's childhurd and will probably want to prioritize that when I'm at home. (Making a copy of /etc/guix/machines.scm and editing that as required would probably mork for me.) I've added this patch set to the hurd-team branch and am testing it now. The only thing I found without testing: > Subject: [bug#66156] [PATCH 06/12] services: guix: Support declarative offloading setup. > To: 66156@debbugs.gnu.org > Cc: Ludovic Courtès <ludo@gnu.org> > Date: Fri, 22 Sep 2023 14:54:07 +0200 (44 minutes, 21 seconds ago) > Resent-From: Ludovic Courtès <ludo@gnu.org> > > * gnu/services/base.scm (guix-machines-files-installation): New > procedure. > (<guix-configuration>)[build-machines]: New field. > (guix-activation): Call ‘ guix-machines-files-installation’. ^ Remove space. Thanks for this amazing piece of work and bug-fixing! Greetings, Janneke
Hello! Janneke Nieuwenhuizen <janneke@gnu.org> skribis: > Beautiful! I'm wondering how this works wrt disk size; the default is > still 'guess? In my childhurds I have > > (hurd-vm-configuration > .. > (disk-size (* 16 1024 (expt 2 20))) ;16GiB > > which is pretty cheap now that we have qcow2. I'm not sure how > expensive it is to set memory-size if you don't use it? I guess setting ‘disk-size’ should still work (and as you write: with QCOW2, it doesn’t cost much to ask for extra space, until you actually use it); nothing has changed here. > I'm wondering if there is a way to/if we need a way to set the default > priority of the childhurd. I'm usually offloading to another machine's > childhurd and will probably want to prioritize that when I'm at home. Good question. Perhaps you could declare that other machine with a higher ‘speed’ value? Thanks for your feedback! Ludo’.
Ludovic Courtès writes: Hello! Okay, after overcoming completely unreleated troubles I finally managed to reconfigure a laptop to hurd-team to test it. It works great! > Janneke Nieuwenhuizen <janneke@gnu.org> skribis: > >> Beautiful! I'm wondering how this works wrt disk size; the default is >> still 'guess? In my childhurds I have >> >> (hurd-vm-configuration >> .. >> (disk-size (* 16 1024 (expt 2 20))) ;16GiB >> >> which is pretty cheap now that we have qcow2. I'm not sure how >> expensive it is to set memory-size if you don't use it? > > I guess setting ‘disk-size’ should still work (and as you write: with > QCOW2, it doesn’t cost much to ask for extra space, until you actually > use it); nothing has changed here. Yes, works beautifully. >> I'm wondering if there is a way to/if we need a way to set the default >> priority of the childhurd. I'm usually offloading to another machine's >> childhurd and will probably want to prioritize that when I'm at home. > > Good question. Perhaps you could declare that other machine with a > higher ‘speed’ value? That also works nicely. I found that by default, only root can offload. Is that intentional, how would that work? I changed permissions on `/etc/guix/offload/ssh/childhurd' like so --8<---------------cut here---------------start------------->8--- $ l /etc/guix/offload/ssh total 8 -rw-r--r-- 1 root root 93 Sep 23 14:47 childhurd.pub -rw-r----- 1 root wheel 399 Sep 23 14:47 childhurd --8<---------------cut here---------------end--------------->8--- to "fix" that. WDYT? Found another two commit message nitpicks > Subject: Re: [bug#66156] [PATCH 12/12] services: hurd-vm: Implement zero-configuration offloading. [..] > > This allows for zero-configuration offloading to a childhurd. > > * gnu/services/virtualization.scm (operating-system-with-offloading-account): > New procedure. > (<hurd-vm-configuration>)[offloading?]: New field. > (hurd-vm-disk-image): Define ‘transform’ and use it. > (hurd-vm-activation): Generate SSH key for user ‘offloading’ and add > authorize it via /etc/childhurd/etc/ssh/authorized_keys.d. Remove "add". > (hurd-vm-configuration-offloading-ssh-key) > (hurd-vm-guix-extension): New procedures. Rather use (hurd-vm-configuration-offloading-ssh-key, hurd-vm-guix-extension): New procedures. ? Greetings, Janneke
Hello, Janneke Nieuwenhuizen <janneke@gnu.org> skribis: > Okay, after overcoming completely unreleated troubles I finally managed > to reconfigure a laptop to hurd-team to test it. It works great! Yay, thanks for testing! > I found that by default, only root can offload. Is that intentional, > how would that work? It’s is intentional: ‘guix offload’ is only ever invoked by guix-daemon¹, as root. ¹ Except for ‘guix offload test’, which is meant to be spawned by users, as root too. > I changed permissions on `/etc/guix/offload/ssh/childhurd' like so > > $ l /etc/guix/offload/ssh > total 8 > -rw-r--r-- 1 root root 93 Sep 23 14:47 childhurd.pub > -rw-r----- 1 root wheel 399 Sep 23 14:47 childhurd > > to "fix" that. WDYT? What would be the use case? > Found another two commit message nitpicks Noted! I’d like to see if there are other opinions in particular about /etc/guix/machines.scm generation, then I guess we can merge? Thanks, Ludo’.
Ludovic Courtès writes: Hi! > Janneke Nieuwenhuizen <janneke@gnu.org> skribis: > >> Okay, after overcoming completely unreleated troubles I finally managed >> to reconfigure a laptop to hurd-team to test it. It works great! > > Yay, thanks for testing! Happy to :) >> I found that by default, only root can offload. Is that intentional, >> how would that work? > > It’s is intentional: ‘guix offload’ is only ever invoked by > guix-daemon¹, as root. ...ah. I missed... > ¹ Except for ‘guix offload test’, which is meant to be spawned by users, > as root too. ...this bit. >> I changed permissions on `/etc/guix/offload/ssh/childhurd' like so >> >> $ l /etc/guix/offload/ssh >> total 8 >> -rw-r--r-- 1 root root 93 Sep 23 14:47 childhurd.pub >> -rw-r----- 1 root wheel 399 Sep 23 14:47 childhurd >> >> to "fix" that. WDYT? > > What would be the use case? Well, running `guix offload test' as "janneke" :) I didn't "dare" (should I say "proceed"?) to test a real offload before seeing "guix offload test" work. That's how I usually go about offloading. Reading the manual, especially if you know it, I guess that it does say so in a subtle way --8<---------------cut here---------------start------------->8--- To test whether your setup is operational, run this command on the master node: # guix offload test --8<---------------cut here---------------end--------------->8--- And sure enough, offloading does work when I revert permessions on the private key, so yeah... >> Found another two commit message nitpicks > > Noted! > > I’d like to see if there are other opinions in particular about > /etc/guix/machines.scm generation, then I guess we can merge? Sure! Greetings, Janneke
Hello, I pushed these as b9fae146d6cc4a6968a8eb18beef29aa1414a31e. I forgot to amend the commit logs as you had suggested; apologies! I also forgot to submit a news entry with the initial patch set, but then thought it’d be nice to have, so I pushed the following as b9fae146d6cc4a6968a8eb18beef29aa1414a31e (with French translation): (entry (commit "953c65ffdd43c02c934518fb7a1c68542584b223") (title (en "Declarative offloading on Guix System and childhurds")) (body (en "When configuring Guix System, it is now possible to declare builds machines to offload to directly from the @code{operating-system} declaration by specifying the @code{build-machines} field of @code{guix-configuration}. When you do this, @command{guix system} generates a @file{/etc/guix/machines.scm} file by concatenating the @code{(build-machine @dots{})} expressions you specified. This mechanism is used by @code{hurd-vm-service-type}, also known as ``childhurd'', to create virtual machines running the GNU/Hurd operating system one can offload to, without additional configuration steps. Run @command{info \"(guix) Daemon Offload Setup\"} for more info on offloading; run @command{info \"(guix) Virtualization Services\"} for info on @code{hurd-vm-service-type}."))) Thanks, Ludo’.