mbox series

[bug#48806,0/7] Generalized cache support and improved graft caching

Message ID 20210603072958.13424-1-ludo@gnu.org
Headers show
Series Generalized cache support and improved graft caching | expand

Message

Ludovic Courtès June 3, 2021, 7:29 a.m. UTC 
Hi!

This patch series allows us to dynamically allocate per-store-connection
caches, such that subsequently created <store-connection> records
can have the newly allocated caches.

So far there were a couple of per-connection caches; for anything else,
we’d resort to global hash tables.  That’s not great because those caches
are usually valid only for one connection to a store, and only for the
duration of that session.  This new feature addresses that.

The last patch uses it to have session-wide caches mapping derivations
to applicable grafts, which partly addresses the performance problems
described in <https://issues.guix.gnu.org/41702>.

Feedback welcome, including performance reports!

Thanks,
Ludo’.

Ludovic Courtès (7):
  store: Support dynamic allocation of per-connection caches.
  store: Generalize cache lookup recording.
  grafts: Record cache lookups for profiling.
  grafts: Use SRFI-71 instead of SRFI-11.
  store: Remove 'references/substitutes'.
  store: 'references/cached' now uses a per-session cache.
  grafts: Cache the derivation/graft mapping for the whole session.

 guix/grafts.scm |  56 ++++++++-----
 guix/store.scm  | 205 ++++++++++++++++++++++++++----------------------
 tests/store.scm |  36 ---------
 3 files changed, 150 insertions(+), 147 deletions(-)

Comments

Lars-Dominik Braun June 3, 2021, 11:59 a.m. UTC | #1 
Hi Ludo’!

> Feedback welcome, including performance reports!
I can confirm this further improves the situation around `guix
environment --ad-hoc r-learnr`. The additive behavior from my initial
report is also gone for r-learnr and r-stm, but not for r-learnr and
jupyter. I’m assuming the last two don’t share alot of grafts/packages
and thus don’t benefit from caching.

Thanks,
Lars
Ludovic Courtès June 3, 2021, 8:39 p.m. UTC | #2 
Hi!

Lars-Dominik Braun <lars@6xq.net> skribis:

>> Feedback welcome, including performance reports!
> I can confirm this further improves the situation around `guix
> environment --ad-hoc r-learnr`. The additive behavior from my initial
> report is also gone for r-learnr and r-stm, but not for r-learnr and
> jupyter. I’m assuming the last two don’t share alot of grafts/packages
> and thus don’t benefit from caching.

Yes, most likely.

Thanks for testing!

Ludo’.
Ludovic Courtès June 8, 2021, 7:34 a.m. UTC | #3 
Hi,

Lars-Dominik Braun <lars@6xq.net> skribis:

>> Feedback welcome, including performance reports!
> I can confirm this further improves the situation around `guix
> environment --ad-hoc r-learnr`. The additive behavior from my initial
> report is also gone for r-learnr and r-stm, but not for r-learnr and
> jupyter. I’m assuming the last two don’t share alot of grafts/packages
> and thus don’t benefit from caching.

Pushed as 0c109026093e6fa8730efe0d7454656275d6efe3!

Thanks,
Ludo’.