mbox series

[bug#72137,0/2] Avoid cache cleanup storms

Message ID cover.1721120884.git.ludo@gnu.org
Headers show
Series Avoid cache cleanup storms | expand

Message

Ludovic Courtès July 16, 2024, 9:10 a.m. UTC 
Hello!

This fixes “cache cleanup storms” reported by Chris Baines whereby
typically multiple ‘guix substitute’ processes would start cleaning
up /var/guix/substitute/cache concurrently, leading to poor performance
(in particular on build farms with many such processes running in
parallel, even worse when on spinning disks).

Thoughts?

Ludo’.

Ludovic Courtès (2):
  syscalls: Add ‘mode’ parameter to ‘lock-file’.
  cache: Avoid cache cleanup storms from concurrent processes.

 guix/build/syscalls.scm | 14 +++++++++-----
 guix/cache.scm          | 27 ++++++++++++++++++---------
 tests/cache.scm         | 30 +++++++++++++++++++++++++++++-
 tests/syscalls.scm      | 13 +++++++++++++
 4 files changed, 69 insertions(+), 15 deletions(-)


base-commit: eb508e32d2d359c94d2cabebfe90dc32ca5dcf4f

Comments

Ludovic Courtès Aug. 20, 2024, 10:54 p.m. UTC | #1 
Hi,

Ludovic Courtès <ludo@gnu.org> skribis:

> This fixes “cache cleanup storms” reported by Chris Baines whereby
> typically multiple ‘guix substitute’ processes would start cleaning
> up /var/guix/substitute/cache concurrently, leading to poor performance
> (in particular on build farms with many such processes running in
> parallel, even worse when on spinning disks).
>
> Thoughts?
>
> Ludo’.
>
> Ludovic Courtès (2):
>   syscalls: Add ‘mode’ parameter to ‘lock-file’.
>   cache: Avoid cache cleanup storms from concurrent processes.

I went ahead and pushed it as d921c742b774a9f0a016f3db6442d5c58a330c92.

Ludo’.