[bug#77680,4/4] guix: Avoid ‘fdatasync’ call for caches and regular files.
Commit Message
Fixes <https://issues.guix.gnu.org/77606>.
Calling ‘fdatasync’ for each and every narinfo file created by ‘guix
substitute’ proved to be too expensive on spinning HDDs and/or under
load (from 0.1s to 1.3s for the ‘fdatasync’ call alone).
* guix/git-authenticate.scm (cache-authenticated-commit): Pass #:sync? #f.
* guix/http-client.scm (http-fetch/cached): Likewise.
* guix/scripts/discover.scm (write-publish-file): Likewise.
* guix/scripts/style.scm (format-whole-file): Likewise.
* guix/substitutes.scm (cache-narinfo!): Likewise.
Reported-by: Christopher Baines <mail@cbaines.net>
Change-Id: I82297eae737bc5aae8a3f7604119e9f3d4b625bf
---
guix/git-authenticate.scm | 5 +++--
guix/http-client.scm | 5 +++--
guix/scripts/discover.scm | 3 ++-
guix/scripts/style.scm | 5 +++--
guix/substitutes.scm | 3 ++-
5 files changed, 13 insertions(+), 8 deletions(-)
Comments
Ludovic Courtès <ludo@gnu.org> writes:
> Fixes <https://issues.guix.gnu.org/77606>.
>
> Calling ‘fdatasync’ for each and every narinfo file created by ‘guix
> substitute’ proved to be too expensive on spinning HDDs and/or under
> load (from 0.1s to 1.3s for the ‘fdatasync’ call alone).
>
> * guix/git-authenticate.scm (cache-authenticated-commit): Pass #:sync? #f.
> * guix/http-client.scm (http-fetch/cached): Likewise.
> * guix/scripts/discover.scm (write-publish-file): Likewise.
> * guix/scripts/style.scm (format-whole-file): Likewise.
> * guix/substitutes.scm (cache-narinfo!): Likewise.
>
> Reported-by: Christopher Baines <mail@cbaines.net>
> Change-Id: I82297eae737bc5aae8a3f7604119e9f3d4b625bf
> ---
> guix/git-authenticate.scm | 5 +++--
> guix/http-client.scm | 5 +++--
> guix/scripts/discover.scm | 3 ++-
> guix/scripts/style.scm | 5 +++--
> guix/substitutes.scm | 3 ++-
> 5 files changed, 13 insertions(+), 8 deletions(-)
>
> diff --git a/guix/git-authenticate.scm b/guix/git-authenticate.scm
> index 37c69d0880..3cd1175c32 100644
> --- a/guix/git-authenticate.scm
> +++ b/guix/git-authenticate.scm
> @@ -1,5 +1,5 @@
> ;;; GNU Guix --- Functional package management for GNU
> -;;; Copyright © 2019, 2020, 2021, 2022 Ludovic Courtès <ludo@gnu.org>
> +;;; Copyright © 2019-2022, 2025 Ludovic Courtès <ludo@gnu.org>
> ;;;
> ;;; This file is part of GNU Guix.
> ;;;
> @@ -344,7 +344,8 @@ (define (cache-authenticated-commit key commit-id)
> (chmod port #o600)
> (display ";; List of previously-authenticated commits.\n\n"
> port)
> - (pretty-print lst port))))))
> + (pretty-print lst port)))
> + #:sync? #f)))
This introduces (or maybe increases) the risk that this file will be
empty, does this happen often enough to justify the change?
Christopher Baines <mail@cbaines.net> skribis:
>> @@ -344,7 +344,8 @@ (define (cache-authenticated-commit key commit-id)
>> (chmod port #o600)
>> (display ";; List of previously-authenticated commits.\n\n"
>> port)
>> - (pretty-print lst port))))))
>> + (pretty-print lst port)))
>> + #:sync? #f)))
>
> This introduces (or maybe increases) the risk that this file will be
> empty, does this happen often enough to justify the change?
My thinking is that it’s just a cache, so it shouldn’t need special
treatment as if it were a precious file (which is what the ‘fdatasync’
call was initially added for); it’s OK if we lose this file.
But it’s rarely modified so from that perspective keeping the
‘fdatasync’ call cannot hurt.
Ludo’.
@@ -1,5 +1,5 @@
;;; GNU Guix --- Functional package management for GNU
-;;; Copyright © 2019, 2020, 2021, 2022 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2019-2022, 2025 Ludovic Courtès <ludo@gnu.org>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -344,7 +344,8 @@ (define (cache-authenticated-commit key commit-id)
(chmod port #o600)
(display ";; List of previously-authenticated commits.\n\n"
port)
- (pretty-print lst port))))))
+ (pretty-print lst port)))
+ #:sync? #f)))
;;;
@@ -1,5 +1,5 @@
;;; GNU Guix --- Functional package management for GNU
-;;; Copyright © 2012-2018, 2020-2022 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2012-2018, 2020-2022, 2025 Ludovic Courtès <ludo@gnu.org>
;;; Copyright © 2015 Mark H Weaver <mhw@netris.org>
;;; Copyright © 2012, 2015 Free Software Foundation, Inc.
;;; Copyright © 2017 Tobias Geerinckx-Rice <me@tobias.gr>
@@ -346,7 +346,8 @@ (define* (http-fetch/cached uri #:key (ttl (%http-cache-ttl)) text?
(when cache-port
(close-port cache-port))
(with-atomic-file-output file
- (cut write-cache port <>))
+ (cut write-cache port <>)
+ #:sync? #f)
(close-port port)
(open-input-file file))))
@@ -87,7 +87,8 @@ (define* (write-publish-file #:key (file (%publish-file)))
(format port "http://~a:~a~%"
(avahi-service-address service)
(avahi-service-port service)))
- %publish-services)))
+ %publish-services))
+ #:sync? #f)
(chmod file #o644))
(define* (read-substitute-urls #:key (file (%publish-file)))
@@ -1,5 +1,5 @@
;;; GNU Guix --- Functional package management for GNU
-;;; Copyright © 2021-2024 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2021-2025 Ludovic Courtès <ludo@gnu.org>
;;; Copyright © 2024 Herman Rimm <herman@rimm.ee>
;;;
;;; This file is part of GNU Guix.
@@ -554,7 +554,8 @@ (define* (format-whole-file file order? #:rest rest)
(apply pretty-print-with-comments/splice port lst
#:format-comment canonicalize-comment
#:format-vertical-space canonicalize-vertical-space
- rest))))))
+ rest))
+ #:sync? #f))))
;;;
@@ -127,7 +127,8 @@ (define (cache-narinfo! cache-url path narinfo ttl)
(mkdir-p (dirname file))
(with-atomic-file-output file
(lambda (out)
- (write (cache-entry cache-url narinfo) out))))
+ (write (cache-entry cache-url narinfo) out))
+ #:sync? #f))
narinfo)