From patchwork Mon Oct 31 19:18:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: phodina X-Patchwork-Id: 44221 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id A4B9027BBEA; Mon, 31 Oct 2022 19:19:38 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-3.7 required=5.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H2, SPF_HELO_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 7686327BBE9 for ; Mon, 31 Oct 2022 19:19:36 +0000 (GMT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1opaJO-0003W5-4x; Mon, 31 Oct 2022 15:19:06 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1opaJK-0003V9-S9 for guix-patches@gnu.org; Mon, 31 Oct 2022 15:19:02 -0400 Received: from debbugs.gnu.org ([209.51.188.43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1opaJK-00051f-KZ for guix-patches@gnu.org; Mon, 31 Oct 2022 15:19:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1opaJK-0007JN-Ak for guix-patches@gnu.org; Mon, 31 Oct 2022 15:19:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#58903] Specify the build dir for Nix Resent-From: phodina Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Mon, 31 Oct 2022 19:19:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 58903 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Tobias Geerinckx-Rice Cc: "58903@debbugs.gnu.org" <58903@debbugs.gnu.org>, "go.wigust@gmail.com" , Ludovic =?utf-8?q?Court=C3=A8s?= , "i@pengmeiyu.com" Received: via spool by 58903-submit@debbugs.gnu.org id=B58903.166724392228071 (code B ref 58903); Mon, 31 Oct 2022 19:19:02 +0000 Received: (at 58903) by debbugs.gnu.org; 31 Oct 2022 19:18:42 +0000 Received: from localhost ([127.0.0.1]:41798 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1opaJ0-0007Ih-5x for submit@debbugs.gnu.org; Mon, 31 Oct 2022 15:18:42 -0400 Received: from mail-40133.protonmail.ch ([185.70.40.133]:40739) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1opaIy-0007IQ-2K for 58903@debbugs.gnu.org; Mon, 31 Oct 2022 15:18:40 -0400 Date: Mon, 31 Oct 2022 19:18:29 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=protonmail.com; s=protonmail3; t=1667243913; x=1667503113; bh=YknWCnXAfSZjwYHPAFlPkjjjBbzv4lAnNiIRvsGjKPA=; h=Date:To:From:Cc:Subject:Message-ID:In-Reply-To:References: Feedback-ID:From:To:Cc:Date:Subject:Reply-To:Feedback-ID: Message-ID:BIMI-Selector; b=ZKbfk8qOTYqDRsJBMjGAU8KzhjHg7Jo0vqDvPvUTEngFuBPECVqdKbqN4pb3zBfDp snB6FhBNfSaJF5E+uMPcUXy/ghynSwiJrWtTEmcIoHn+bpAclfrMle/R3gXNGRbelO IZPA3rO/O7vLXmoewzoE/NBMLsQ0puhI+Z2pPfOJmKdElvmn5dKn8d/XtEn7/rtdGn ixUsemRCPJKebmionHFx/HUJpImh8DcL7ytlWulqa2tWaQpAiux8fucpGoa59B1FWT C9G6kPh6O5J11+dqj2xecCr3+KJxCM8srYiPjJFNxavb6IkMAl5DJa4YNCGIMPL2PP nnQryPAzP9YFQ== Message-ID: In-Reply-To: <87sfj4551q.fsf@nckx> References: <89-rZ-T-EfdANIf7D2MDU2c9GRrDND1-wshyPcAx6X4dE6AJnN-N9N2r_93XsmOnKcrGShflEKyle5bRxtumYcHvcgwwF_3DZvZUPOSLr70=@protonmail.com> <87sfj4551q.fsf@nckx> Feedback-ID: 14143818:user:proton MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Guix-patches" Reply-to: phodina X-ACL-Warn: , phodina via Guix-patches X-Patchwork-Original-From: phodina via Guix-patches via From: phodina Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net+patchwork=mira.cbaines.net@gnu.org X-getmail-retrieved-from-mailbox: Patches Hi Tobias, thanks for feedback! > > > (build-dir nix-configuration-build-dir ;string > > (default "/tmp")) > > > ‘directory’. Plz, no unnec. abbrev. > > The concept LGTM. Calling it a ‘build directory’ might be editorialising too much. Does Nix use it for anything else? I've updated the patch so there is no abbreviation. I asked on Nix matrix channel and was told just for derivations. But I'm not expert so take it with grain of salt. Ludo', you have a lot insight into the build infrastructure and you also took the concepts from Nix. What do you think about this approach? It should not be premanently set to block device as you loose throughput and increase latency, but it's neccessary for some derivations as not everybody can have powerful machine at their disposal. Is there some way to share the approach with guix daemon? If I remeber correctly the derivation is offloaded to nix backend - the contents of `nix` directory (haven't yet had the chance to go through the C code). I guess the machines behind 'https://ci.guix.gnu.org' have enough resources. Or is there some script that handles large derivations? Also this might be interesting problem for embedded devices though offloading is definitely the best approach. ---- Petr From d71d48d0fd5d4803abbb50900e0d9aa3c374e7e1 Mon Sep 17 00:00:00 2001 From: Petr Hodina Date: Mon, 31 Oct 2022 16:12:38 +0100 Subject: [PATCH v2] services: nix: Add more configuration fields. * gnu/services/nix.scm ()[build-directory]: New field. (nix-service-etc, nix-shepherd-service): Take them into account. * doc/guix.texi (Nix): Update it. diff --git a/doc/guix.texi b/doc/guix.texi index 80fb3bc47f..825cf0831d 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -37111,6 +37111,26 @@ The Nix package to use. @item @code{sandbox} (default: @code{#t}) Specifies whether builds are sandboxed by default. +@item @code{build-directory} (default: @code{"/tmp"}) +Specifies build directory. This is useful to change if there is not enough RAM +on the machine (e.g. embedded targets) or if building big packages (e.g. +chromium). + +Normally the default location is @code{"/tmp"} which is mostly represented by +@code{tmpfs}. Therefore this solution is fast and does not perform any IO to the +nonvolatile block storage (unless swap is enabled). However, the user might +encounter build failure when building large packages (e.g. @code{chromium}) +which take a lot of space when unpacking, the build artifacts consume more space +and the linking stage might fail due to insufficient amount of RAM for LTO as +all the object files are loaded into memory. + +This can be overcome by storing the intermediate files of the derivation in +other locations such as @code{"/var/tmp/"} that are usually storaged on block +device. + +Be aware that on embedded targets this option will result in more flash wear out +due to large amount of writes. + @item @code{build-sandbox-items} (default: @code{'()}) This is a list of strings or objects appended to the @code{build-sandbox-items} field of the configuration file. diff --git a/gnu/services/nix.scm b/gnu/services/nix.scm index df04a85c22..dcf994b603 100644 --- a/gnu/services/nix.scm +++ b/gnu/services/nix.scm @@ -54,6 +54,8 @@ (define-record-type* (default nix)) (sandbox nix-configuration-sandbox ;boolean (default #t)) + (build-directory nix-configuration-build-directory ;string + (default "/tmp")) (build-sandbox-items nix-configuration-build-sandbox-items ;list of strings (default '())) (extra-config nix-configuration-extra-config ;list of strings @@ -106,7 +108,7 @@ (define (nix-activation _) (define nix-service-etc (match-lambda - (($ package sandbox build-sandbox-items extra-config) + (($ package sandbox build-directory build-sandbox-items extra-config) (let ((ref-file (references-file package))) `(("nix/nix.conf" ,(computed-file @@ -130,7 +132,7 @@ (define internal-sandbox-paths (define nix-shepherd-service ;; Return a for Nix. (match-lambda - (($ package _ _ _ extra-options) + (($ package _ build-directory _ _ extra-options) (list (shepherd-service (provision '(nix-daemon)) @@ -138,7 +140,10 @@ (define nix-shepherd-service (requirement '()) (start #~(make-forkexec-constructor (list (string-append #$package "/bin/nix-daemon") - #$@extra-options))) + #$@extra-options) + #:environment-variables + (list (string-append "TMPDIR=" build-directory) + "PATH=/run/current-system/profile/bin"))) (respawn? #f) (stop #~(make-kill-destructor))))))) -- 2.37.2