diff mbox series

[bug#58035] sync-before-registering is false, possibly the cause of empty files in the store

Message ID 878rlel43z.fsf@gmail.com
State New
Headers show
Series [bug#58035] sync-before-registering is false, possibly the cause of empty files in the store | expand

Checks

Context Check Description
cbaines/comparison success View comparision
cbaines/git-branch success View Git branch
cbaines/applying patch success View Laminar job
cbaines/issue success View issue

Commit Message

Simon Tournier Oct. 17, 2022, 6:03 p.m. UTC 
Hi,

3x time slower with my hardware.  Closing?


On mar., 04 oct. 2022 at 23:49, Ludovic Courtès <ludo@gnu.org> wrote:

> Maybe a good test would be to run a daemon on an “average” HDD with
> sync-before-registering = true and to run ‘perf timechart record’ while
> it’s fetching a large number of substitutes.  That way we’d have
> concrete data to talk about.

+ Comparison using the manifest below.
+ guix-daemon built using fd4cbcbed7; two flavors.
+ Guix revision f43a783.
+ Before the experiment, complete ’guix gc’.

  # test-58035/guix-daemon --build-users-group=guixbuild
  # perf timechart record
  $ time guix build -m some-packages.scm
  # perf timechart --highlight guix

1/ sync-before-registering = false

real    6m24.215s
user    0m10.627s
sys     0m0.512s

[ perf record: Woken up 1984 times to write data ]
[ perf record: Captured and wrote 497.403 MB perf.data (5042727 samples)]


2/ sync-before-registering = true

real    19m46.470s
user    0m12.367s
sys     0m0.557s      

[ perf record: Woken up 2625 times to write data ]
[ perf record: Captured and wrote 658.066 MB perf.data (6712342 samples) ]


Well, the charts are really big.

    $ du -sh {1,2}-build-*.svg
    331M    1-build-false.svg
    599M    2-build-true.svg

Therefore, please find them using this link (alive for 15 days):

https://filesender.renater.fr/?s=download&token=0e861286-ec0c-4815-bb49-2bce91b14462


Cheers,
simon


--8<---------------cut here---------------start------------->8---
(use-modules (guix build-system haskell)
             (guix build-system julia)
             (guix packages)
             (guix profiles)
             (gnu packages))

(manifest
 (map package->manifest-entry
      (fold-packages
       (lambda (package result)
         (if (or
              ;;(eq? (package-build-system package) haskell-build-system)
              (eq? (package-build-system package) julia-build-system))
             (cons package result)
             result))
       '())))
--8<---------------cut here---------------end--------------->8---

--8<---------------cut here---------------start------------->8---
--8<---------------cut here---------------end--------------->8---

Comments

M Oct. 17, 2022, 6:51 p.m. UTC | #1 
On 17-10-2022 20:03, zimoun wrote:
> 3x time slower with my hardware.  Closing?

I had an alternative proposal to 'sync-before-registering = true' -- 
more work to implement, but potentially faster, so IMO no.

Greetings,
Maxime.
Ludovic Courtès Oct. 18, 2022, 3:34 p.m. UTC | #2 
Hi,

zimoun <zimon.toutoune@gmail.com> skribis:

> 3x time slower with my hardware.  Closing?

Ouch!

> On mar., 04 oct. 2022 at 23:49, Ludovic Courtès <ludo@gnu.org> wrote:
>
>> Maybe a good test would be to run a daemon on an “average” HDD with
>> sync-before-registering = true and to run ‘perf timechart record’ while
>> it’s fetching a large number of substitutes.  That way we’d have
>> concrete data to talk about.
>
> + Comparison using the manifest below.
> + guix-daemon built using fd4cbcbed7; two flavors.
> + Guix revision f43a783.
> + Before the experiment, complete ’guix gc’.
>
>   # test-58035/guix-daemon --build-users-group=guixbuild
>   # perf timechart record
>   $ time guix build -m some-packages.scm
>   # perf timechart --highlight guix

OK.

> 1/ sync-before-registering = false
>
> real    6m24.215s
> user    0m10.627s
> sys     0m0.512s
>
> [ perf record: Woken up 1984 times to write data ]
> [ perf record: Captured and wrote 497.403 MB perf.data (5042727 samples)]
>
>
> 2/ sync-before-registering = true
>
> real    19m46.470s
> user    0m12.367s
> sys     0m0.557s      
>
> [ perf record: Woken up 2625 times to write data ]
> [ perf record: Captured and wrote 658.066 MB perf.data (6712342 samples) ]

That speaks for itself.

Let’s keep the bug open in case Maxime or someone else comes up with a
more practical fix.

Thanks,
Ludo’.
diff mbox series

Patch

diff --git a/nix/libstore/globals.cc b/nix/libstore/globals.cc
index d4f9a46a74..5f8a3a3031 100644
--- a/nix/libstore/globals.cc
+++ b/nix/libstore/globals.cc
@@ -40,7 +40,7 @@  Settings::Settings()
     reservedSize = 8 * 1024 * 1024;
     fsyncMetadata = true;
     useSQLiteWAL = true;
-    syncBeforeRegistering = false;
+    syncBeforeRegistering = true;
     useSubstitutes = true;
     useChroot = false;
     impersonateLinux26 = false;