diff mbox series

[bug#58903] Specify the build dir for Nix

Message ID uW9hMR6BWryRTewYg5uaKR3Mk9wLNSpG0SFJPC14MGNIf0VfLMyEHTIub2kmn5GA1cbEPzdvvLSTzJeZGAAscr3Q1gxGlM49nhZrEWjcuD0=@protonmail.com
State New
Headers show
Series [bug#58903] Specify the build dir for Nix | expand

Checks

Context Check Description
cbaines/comparison success View comparision
cbaines/git-branch success View Git branch
cbaines/applying patch success
cbaines/issue success View issue

Commit Message

phodina Oct. 31, 2022, 7:18 p.m. UTC 
Hi Tobias,

thanks for feedback!

> 
> > (build-dir nix-configuration-build-dir ;string
> > (default "/tmp"))
> 
> 
> ‘directory’. Plz, no unnec. abbrev.
> 
> The concept LGTM. Calling it a ‘build directory’ might be editorialising too much. Does Nix use it for anything else?

I've updated the patch so there is no abbreviation. I asked on Nix matrix channel and was told just for derivations. But I'm not expert so take it with grain of salt.

Ludo',
you have a lot insight into the build infrastructure and you also took the concepts from Nix.

What do you think about this approach? It should not be premanently set to block device as you loose throughput and increase latency, but it's neccessary for some derivations as not everybody can have powerful machine at their disposal.

Is there some way to share the approach with guix daemon? If I remeber correctly the derivation is offloaded to nix backend - the contents of `nix` directory (haven't yet had the chance to go through the C code).

I guess the machines behind 'https://ci.guix.gnu.org' have enough resources. Or is there some script that handles large derivations? 

Also this might be interesting problem for embedded devices though offloading is definitely the best approach.

----
Petr

Comments

Ludovic Courtès Nov. 9, 2022, 10:45 p.m. UTC | #1 
Hi,

phodina <phodina@protonmail.com> skribis:

> I've updated the patch so there is no abbreviation. I asked on Nix matrix channel and was told just for derivations. But I'm not expert so take it with grain of salt.
>
> Ludo',
> you have a lot insight into the build infrastructure and you also took the concepts from Nix.

The manual documents it in the context of guix-daemon (info "(guix)
Build Environment Setup"):

     You can influence the directory where the daemon stores build trees
  via the ‘TMPDIR’ environment variable.  However, the build tree within
  the chroot is always called ‘/tmp/guix-build-NAME.drv-0’, where NAME is
  the derivation name—e.g., ‘coreutils-8.24’.  This way, the value of
  ‘TMPDIR’ does not leak inside build environments, which avoids
  discrepancies in cases where build processes capture the name of their
  build tree.

> I guess the machines behind 'https://ci.guix.gnu.org' have enough resources. Or is there some script that handles large derivations? 

On Guix System, /tmp is not on tmpfs by default (maybe something to
change?); that’s also the case on build machines.

> From d71d48d0fd5d4803abbb50900e0d9aa3c374e7e1 Mon Sep 17 00:00:00 2001
> From: Petr Hodina <phodina@protonmail.com>
> Date: Mon, 31 Oct 2022 16:12:38 +0100
> Subject: [PATCH v2] services: nix: Add more configuration fields.
>
> * gnu/services/nix.scm (<nix-configuration>)[build-directory]: New field.
> (nix-service-etc, nix-shepherd-service): Take them into account.
> * doc/guix.texi (Nix): Update it.

[...]

> +@item @code{build-directory} (default: @code{"/tmp"})
> +Specifies build directory.

Rather something like: “The directory where build directory are stored
during builds.”

> This is useful to change if there is not enough RAM
> +on the machine (e.g. embedded targets) or if building big packages (e.g.
> +chromium).

“This is useful to change if, for example, the default location does not
have enough space to hold build trees for big packages.”

(There’s no ‘chromium’ package in Guix.)

Then, instead of a long explanation, I’d just write something like:

  This is similar to setting the @env{TMPDIR} environment variable for
  @command{guix-daemon}.  @ref{Build Environment Setup, @env{TMPDIR}},
  for more info.

Could you send an updated patch?

Thanks,
Ludo’.
diff mbox series

Patch

From d71d48d0fd5d4803abbb50900e0d9aa3c374e7e1 Mon Sep 17 00:00:00 2001
From: Petr Hodina <phodina@protonmail.com>
Date: Mon, 31 Oct 2022 16:12:38 +0100
Subject: [PATCH v2] services: nix: Add more configuration fields.

* gnu/services/nix.scm (<nix-configuration>)[build-directory]: New field.
(nix-service-etc, nix-shepherd-service): Take them into account.
* doc/guix.texi (Nix): Update it.

diff --git a/doc/guix.texi b/doc/guix.texi
index 80fb3bc47f..825cf0831d 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -37111,6 +37111,26 @@  The Nix package to use.
 @item @code{sandbox} (default: @code{#t})
 Specifies whether builds are sandboxed by default.
 
+@item @code{build-directory} (default: @code{"/tmp"})
+Specifies build directory. This is useful to change if there is not enough RAM
+on the machine (e.g. embedded targets) or if building big packages (e.g.
+chromium).
+
+Normally the default location is @code{"/tmp"} which is mostly represented by
+@code{tmpfs}. Therefore this solution is fast and does not perform any IO to the
+nonvolatile block storage (unless swap is enabled). However, the user might
+encounter build failure when building large packages (e.g. @code{chromium})
+which take a lot of space when unpacking, the build artifacts consume more space
+and the linking stage might fail due to insufficient amount of RAM for LTO as
+all the object files are loaded into memory.
+
+This can be overcome by storing the intermediate files of the derivation in
+other locations such as @code{"/var/tmp/"} that are usually storaged on block
+device.
+
+Be aware that on embedded targets this option will result in more flash wear out
+due to large amount of writes.
+
 @item @code{build-sandbox-items} (default: @code{'()})
 This is a list of strings or objects appended to the
 @code{build-sandbox-items} field of the configuration file.
diff --git a/gnu/services/nix.scm b/gnu/services/nix.scm
index df04a85c22..dcf994b603 100644
--- a/gnu/services/nix.scm
+++ b/gnu/services/nix.scm
@@ -54,6 +54,8 @@  (define-record-type* <nix-configuration>
                        (default nix))
   (sandbox             nix-configuration-sandbox ;boolean
                        (default #t))
+  (build-directory     nix-configuration-build-directory ;string
+                       (default "/tmp"))
   (build-sandbox-items nix-configuration-build-sandbox-items ;list of strings
                        (default '()))
   (extra-config        nix-configuration-extra-config ;list of strings
@@ -106,7 +108,7 @@  (define (nix-activation _)
 
 (define nix-service-etc
   (match-lambda
-    (($ <nix-configuration> package sandbox build-sandbox-items extra-config)
+    (($ <nix-configuration> package sandbox build-directory build-sandbox-items extra-config)
      (let ((ref-file (references-file package)))
        `(("nix/nix.conf"
           ,(computed-file
@@ -130,7 +132,7 @@  (define internal-sandbox-paths
 (define nix-shepherd-service
   ;; Return a <shepherd-service> for Nix.
   (match-lambda
-    (($ <nix-configuration> package _ _ _ extra-options)
+    (($ <nix-configuration> package _ build-directory _ _ extra-options)
      (list
       (shepherd-service
        (provision '(nix-daemon))
@@ -138,7 +140,10 @@  (define nix-shepherd-service
        (requirement '())
        (start #~(make-forkexec-constructor
                  (list (string-append #$package "/bin/nix-daemon")
-                       #$@extra-options)))
+                       #$@extra-options)
+                 #:environment-variables
+                   (list (string-append "TMPDIR=" build-directory)
+                         "PATH=/run/current-system/profile/bin")))
        (respawn? #f)
        (stop #~(make-kill-destructor)))))))
 
-- 
2.37.2