mbox

[bug#36029,0/2] 'publish' and 'substitute' support several compression methods

Message ID 87blzhzv1z.fsf@ambrevar.xyz
Headers show

Message

Pierre Neidhardt June 1, 2019, 6:19 a.m. UTC 
Hi!

> One idea we discussed there was to allow clients to pass an
> ‘X-Guix-Accepted-Encoding’ header in HTTP requests, and the server
> would return an lzip narinfo or a gzip narinfo depending on that.
> However, I thought that this was not very flexible, and that we
> were bound to mess up with caching.
>
> This patch implements a different solution: ‘guix publish’ can
> be passed multiple ‘-C’ options, in which case it compresses
> substitutes with all these compression methods.  The corresponding
> narinfo looks like this:
>
> --8<---------------cut here---------------start------------->8---
> StorePath: /gnu/store/9czlz7ss3187l2vi1hvrlkwlgrggdg5p-inkscape-0.92.4
> URL: nar/gzip/9czlz7ss3187l2vi1hvrlkwlgrggdg5p-inkscape-0.92.4
> Compression: gzip
> FileSize: 40308611
> URL: nar/lzip/9czlz7ss3187l2vi1hvrlkwlgrggdg5p-inkscape-0.92.4
> Compression: lzip
> FileSize: 19867767
> NarHash: sha256:1jv4nkq68a7zwqhi9inrnh340a4jxcpb91wq7d25hgw0nk8isbbk
> NarSize: 136499024
> References: …
> --8<---------------cut here---------------end--------------->8---

Huhu, inkscape's size is already halved ;)

> IOW, it’s like before, except that there are multiple
> URL/Compression/FileSize fields instead of just one of each.
>
> The trick is that old clients take the first occurrence of each
> of these fields and ignore subsequent occurrences.  In the example
> above, they’d just take gzip and ignore the rest.

Smart!  I like it!

I gave the patches a quick skim but I'm not very knowledgeable of those
parts, so there is little for me to comment about I'm afraid.

Other than that, excited to see .lz substitutes becoming a reality!

Comments

Ludovic Courtès June 2, 2019, 8:44 p.m. UTC | #1 
Hello!

Pierre Neidhardt <mail@ambrevar.xyz> skribis:

>> --8<---------------cut here---------------start------------->8---
>> StorePath: /gnu/store/9czlz7ss3187l2vi1hvrlkwlgrggdg5p-inkscape-0.92.4
>> URL: nar/gzip/9czlz7ss3187l2vi1hvrlkwlgrggdg5p-inkscape-0.92.4
>> Compression: gzip
>> FileSize: 40308611
>> URL: nar/lzip/9czlz7ss3187l2vi1hvrlkwlgrggdg5p-inkscape-0.92.4
>> Compression: lzip
>> FileSize: 19867767
>> NarHash: sha256:1jv4nkq68a7zwqhi9inrnh340a4jxcpb91wq7d25hgw0nk8isbbk
>> NarSize: 136499024
>> References: …
>> --8<---------------cut here---------------end--------------->8---
>
> Huhu, inkscape's size is already halved ;)

Yup, I’d be curious to gather more stats once it’s running!

>> IOW, it’s like before, except that there are multiple
>> URL/Compression/FileSize fields instead of just one of each.
>>
>> The trick is that old clients take the first occurrence of each
>> of these fields and ignore subsequent occurrences.  In the example
>> above, they’d just take gzip and ignore the rest.
>
> Smart!  I like it!

Cool!  I’ve pushed these:

  b90ae065b5 substitute: Select the best compression methods.
  b8fa86adfc publish: '--compression' can be repeated.

Now I’d like to run a test instance on berlin before we deploy it “for
real.”  To be continued…

Ludo’.