From patchwork Thu Jul 4 11:50:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christopher Baines X-Patchwork-Id: 65915 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id ED8A927BBEA; Thu, 4 Jul 2024 12:51:23 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=BAYES_00,MAILING_LIST_MULTI, SPF_HELO_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 1773927BBE2 for ; Thu, 4 Jul 2024 12:51:23 +0100 (BST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sPKzX-0004M8-Uo; Thu, 04 Jul 2024 07:51:11 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sPKzS-0004H4-H7 for guix-patches@gnu.org; Thu, 04 Jul 2024 07:51:07 -0400 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sPKzR-0004tI-8T; Thu, 04 Jul 2024 07:51:05 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1sPKzO-0006qs-Ts; Thu, 04 Jul 2024 07:51:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#71038] [PATCH v3 1/2] guix: store: Enable specifying the builtin builders. References: <87y187b6xv.fsf@cbaines.net> In-Reply-To: <87y187b6xv.fsf@cbaines.net> Resent-From: Christopher Baines Original-Sender: "Debbugs-submit" Resent-CC: guix@cbaines.net, dev@jpoiret.xyz, ludo@gnu.org, othacehe@gnu.org, zimon.toutoune@gmail.com, me@tobias.gr, guix-patches@gnu.org Resent-Date: Thu, 04 Jul 2024 11:51:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 71038 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 71038@debbugs.gnu.org Cc: Christopher Baines , Josselin Poiret , Ludovic =?utf-8?q?Court=C3=A8s?= , Mathieu Othacehe , Simon Tournier , Tobias Geerinckx-Rice X-Debbugs-Original-Xcc: Christopher Baines , Josselin Poiret , Ludovic =?utf-8?q?Court=C3=A8s?= , Mathieu Othacehe , Simon Tournier , Tobias Geerinckx-Rice Received: via spool by 71038-submit@debbugs.gnu.org id=B71038.172009384826310 (code B ref 71038); Thu, 04 Jul 2024 11:51:02 +0000 Received: (at 71038) by debbugs.gnu.org; 4 Jul 2024 11:50:48 +0000 Received: from localhost ([127.0.0.1]:41687 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1sPKz9-0006qI-P2 for submit@debbugs.gnu.org; Thu, 04 Jul 2024 07:50:48 -0400 Received: from mira.cbaines.net ([212.71.252.8]:43904) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1sPKz7-0006qA-KQ for 71038@debbugs.gnu.org; Thu, 04 Jul 2024 07:50:46 -0400 Received: from localhost (nat-inria-guest-32-gw-01-bso.bordeaux.inria.fr [194.199.1.32]) by mira.cbaines.net (Postfix) with ESMTPSA id A8F7D27BBE2 for <71038@debbugs.gnu.org>; Thu, 4 Jul 2024 12:50:11 +0100 (BST) Received: from localhost (localhost [local]) by localhost (OpenSMTPD) with ESMTPA id 9a747b75 for <71038@debbugs.gnu.org>; Thu, 4 Jul 2024 11:50:10 +0000 (UTC) From: Christopher Baines Date: Thu, 4 Jul 2024 13:50:08 +0200 Message-ID: <14fc15bdb8b5fd615fb93e025e86ccf70cf81814.1720093809.git.mail@cbaines.net> X-Mailer: git-send-email 2.45.2 MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org X-getmail-retrieved-from-mailbox: Patches To open-connection and port->connection. This overrides the discovered builtin builders that the daemon says it provides. This is useful when you want to generate compatible derivations that can be run with a daemon that potentially doesn't support builtin builders that the daemon you're using to generate the derivations has. I'm looking at this in particular because I want to use this in the data service, since it provides substitutes for derivations, and since these can be built on other machines, it's useful to control which builtin builders they depend on. * guix/store.scm (open-connection, port->connection): Accept #:built-in-builders and use this instead of %built-in-builders. Fixes: . Change-Id: I45d58ab93b6d276d280552858fc81ebc2b58828a --- guix/store.scm | 28 +++++++++++++++++++--------- 1 file changed, 19 insertions(+), 9 deletions(-) base-commit: 5f1e4e4c0242af6bcba656aedf8b49afbe7247b7 diff --git a/guix/store.scm b/guix/store.scm index 4070b686cb..cf5848e580 100644 --- a/guix/store.scm +++ b/guix/store.scm @@ -566,7 +566,7 @@ (define* (connect-to-daemon uri-or-filename #:key non-blocking?) (define* (open-connection #:optional (uri (%daemon-socket-uri)) #:key port (reserve-space? #t) cpu-affinity - non-blocking?) + non-blocking? built-in-builders) "Connect to the daemon at URI (a string), or, if PORT is not #f, use it as the I/O port over which to communicate to a build daemon. @@ -575,8 +575,10 @@ (define* (open-connection #:optional (uri (%daemon-socket-uri)) should the disk become full. When CPU-AFFINITY is true, it must be an integer corresponding to an OS-level CPU number to which the daemon's worker process for this connection will be pinned. If NON-BLOCKING?, use a non-blocking -socket when using the file, unix or guix URI schemes. Return a server -object." +socket when using the file, unix or guix URI schemes. If +BUILT-IN-BUILDERS is provided, it should be a list of strings +and this will be used instead of the builtin builders provided by the build +daemon. Return a server object." (define (handshake-error) (raise (condition (&store-connection-error (file (or port uri)) @@ -610,8 +612,10 @@ (define* (open-connection #:optional (uri (%daemon-socket-uri)) (write-int cpu-affinity port))) (when (>= (protocol-minor v) 11) (write-int (if reserve-space? 1 0) port)) - (letrec* ((built-in-builders - (delay (%built-in-builders conn))) + (letrec* ((actual-built-in-builders + (if built-in-builders + (delay built-in-builders) + (delay (%built-in-builders conn)))) (caches (make-vector (atomic-box-ref %store-connection-caches) @@ -624,15 +628,19 @@ (define* (open-connection #:optional (uri (%daemon-socket-uri)) (make-hash-table 100) (make-hash-table 100) caches - built-in-builders))) + actual-built-in-builders))) (let loop ((done? (process-stderr conn))) (or done? (process-stderr conn))) conn)))))) (define* (port->connection port - #:key (version %protocol-version)) + #:key (version %protocol-version) + built-in-builders) "Assimilate PORT, an input/output port, and return a connection to the -daemon, assuming the given protocol VERSION. +daemon, assuming the given protocol VERSION. If +BUILT-IN-BUILDERS is provided, it should be a list of strings +and this will be used instead of the builtin builders provided by the build +daemon. Warning: this procedure assumes that the initial handshake with the daemon has already taken place on PORT and that we're just continuing on this established @@ -649,7 +657,9 @@ (define* (port->connection port (make-vector (atomic-box-ref %store-connection-caches) vlist-null) - (delay (%built-in-builders connection)))) + (if built-in-builders + (delay built-in-builders) + (delay (%built-in-builders connection))))) connection))