[bug#47027,2/2] gnu: Add disarchive.

* gnu/packages/archival.scm: New file.
* gnu/local.mk (GNU_SYSTEM_MODULES): Add it.
---
 gnu/local.mk              |  1 +
 gnu/packages/archival.scm | 60 +++++++++++++++++++++++++++++++++++++++
 2 files changed, 61 insertions(+)
 create mode 100644 gnu/packages/archival.scm

I've checked and the package seems to build fine with Guile 3.0.2.  I
think the bytecode mismatch happens, because Guix compiles stuff with
3.0.2 by default, but users have 3.0.5 in their system, which is not
bytecode-compatible.  (As an exception, Guix itself seems to be
compiled with Guile 3.0.5 for performance reasons).

I think it would be fine to add with Guile 3.0.2, perhaps adding a note
that Guile 3.0.5 will effectively be required to use Guix interop?  If
not, could you provide a script, that breaks in a way other than
recompiling the mismatching code?

As far as the location is concerned, I personally do not like adding
too many single-package files.  Would it make sense to add this to
compression.scm (like gzip) or backup.scm (like libarchive)?

Regards,
Leo

Hi!

Leo Prikler <leo.prikler@student.tugraz.at> skribis:

> I've checked and the package seems to build fine with Guile 3.0.2.  I
> think the bytecode mismatch happens, because Guix compiles stuff with
> 3.0.2 by default, but users have 3.0.5 in their system, which is not
> bytecode-compatible.  (As an exception, Guix itself seems to be
> compiled with Guile 3.0.5 for performance reasons).
>
> I think it would be fine to add with Guile 3.0.2, perhaps adding a note
> that Guile 3.0.5 will effectively be required to use Guix interop?  If
> not, could you provide a script, that breaks in a way other than
> recompiling the mismatching code?

I tend to agree here: I don’t think ‘guile-3.0-latest’ is needed in this
case.  The only case where you need it is if it depends on a library,
such as Guix, that is itself built with ‘guile-3.0-latest’.

> As far as the location is concerned, I personally do not like adding
> too many single-package files.  Would it make sense to add this to
> compression.scm (like gzip) or backup.scm (like libarchive)?

Maybe there’ll be other packages eventually in archival.scm, like the
SWH Python code?  It’s fine with me, but I don’t have a strong opinion.

Ludo’.

Ludovic Courtès 写道：
> Maybe there’ll be other packages eventually in archival.scm, 
> like the
> SWH Python code?

Yes, that's the intention.

Kind regards,

T G-R

Hi,

Thanks Leo for the review!  (And to Ludo and Tobias for follow ups.)

Ludovic Courtès <ludo@gnu.org> writes:

> Leo Prikler <leo.prikler@student.tugraz.at> skribis:
>
>> I've checked and the package seems to build fine with Guile 3.0.2.  I
>> think the bytecode mismatch happens, because Guix compiles stuff with
>> 3.0.2 by default, but users have 3.0.5 in their system, which is not
>> bytecode-compatible.  (As an exception, Guix itself seems to be
>> compiled with Guile 3.0.5 for performance reasons).
>>
>> I think it would be fine to add with Guile 3.0.2, perhaps adding a note
>> that Guile 3.0.5 will effectively be required to use Guix interop?  If
>> not, could you provide a script, that breaks in a way other than
>> recompiling the mismatching code?
>
> I tend to agree here: I don’t think ‘guile-3.0-latest’ is needed in this
> case.  The only case where you need it is if it depends on a library,
> such as Guix, that is itself built with ‘guile-3.0-latest’.

Well, now I’m second guessing myself.  :)

It is just the auto compilation notes and warnings that I’m worried
about.  The module closure of “swh.scm” works fine on Guile 3.0.2.

Eventually, the daemon will invoke Disarchive via “builtin:download” and
“perform-download.scm”.  I intend to use the Scheme interface there,
which means Disarchive will be runing on Guile 3.0.5.  For that, it
would be preferable to have a Guile 3.0.5 version of Disarchive, right?

On the other hand, when using Disarchive to extract metadata (e.g., with
Cuirass), the SWH code is not needed at all.

I will resurrect my patch for calling Disarchive from Guix, and come
back to this when I know exactly what kind of package I need for that to
work smoothly.

>> As far as the location is concerned, I personally do not like adding
>> too many single-package files.  Would it make sense to add this to
>> compression.scm (like gzip) or backup.scm (like libarchive)?
>
> Maybe there’ll be other packages eventually in archival.scm, like the
> SWH Python code?  It’s fine with me, but I don’t have a strong opinion.

I don’t feel strongly about it either.  There’s other software besides
Disarchive and SWH that could be called “archival”, and I think it’s
more accurate than the other options.


-- Tim

Hi,
Am Donnerstag, den 11.03.2021, 22:06 -0500 schrieb Timothy Sample:
> It is just the auto compilation notes and warnings that I’m worried
> about.  The module closure of “swh.scm” works fine on Guile 3.0.2.
> 
> Eventually, the daemon will invoke Disarchive via “builtin:download”
> and
> “perform-download.scm”.  I intend to use the Scheme interface there,
> which means Disarchive will be runing on Guile 3.0.5.  For that, it
> would be preferable to have a Guile 3.0.5 version of Disarchive,
> right?
Guix offers package transformations on a command line/code level. 
Having packaged disarchival for Guile 3.0.2, it should be trivial to
provide a variant for Guix, that uses Guile 3.0.5.

Looking at the package description for Guix itself, it does not seem as
though an exception would be needed, though.  All of its dependencies
seem to "build and link" fine with Guile 3.0.2 even though Guix uses
3.0.5.  Perhaps it's because you're calling into Guix and thus forming
a dependency cycle or perhaps Guix is really good at hiding such
messages from the user otherwise (not that I know of any particularly
large effort to do so). 

> > > As far as the location is concerned, I personally do not like
> > > adding
> > > too many single-package files.  Would it make sense to add this
> > > to
> > > compression.scm (like gzip) or backup.scm (like libarchive)?
> > 
> > Maybe there’ll be other packages eventually in archival.scm, like
> > the
> > SWH Python code?  It’s fine with me, but I don’t have a strong
> > opinion.
> 
> I don’t feel strongly about it either.  There’s other software
> besides
> Disarchive and SWH that could be called “archival”, and I think it’s
> more accurate than the other options.
I think in that case it'd be fine, but OTOH we group libraries by
programming language and prefix them as well.  For instance, a python
library, that communicates with SWH would probably go to python-web or
python-xyz and be prefixed with python-.  Likewise, if disarchive is
not supposed to be primarily used as a command line tool, I think
putting it into guile-xyz as "guile-disarchive" makes more sense. 
WDYT?

Regards,
Leo

Hello!

Timothy Sample <samplet@ngyro.com> skribis:

> Ludovic Courtès <ludo@gnu.org> writes:
>
>> Leo Prikler <leo.prikler@student.tugraz.at> skribis:
>>
>>> I've checked and the package seems to build fine with Guile 3.0.2.  I
>>> think the bytecode mismatch happens, because Guix compiles stuff with
>>> 3.0.2 by default, but users have 3.0.5 in their system, which is not
>>> bytecode-compatible.  (As an exception, Guix itself seems to be
>>> compiled with Guile 3.0.5 for performance reasons).
>>>
>>> I think it would be fine to add with Guile 3.0.2, perhaps adding a note
>>> that Guile 3.0.5 will effectively be required to use Guix interop?  If
>>> not, could you provide a script, that breaks in a way other than
>>> recompiling the mismatching code?
>>
>> I tend to agree here: I don’t think ‘guile-3.0-latest’ is needed in this
>> case.  The only case where you need it is if it depends on a library,
>> such as Guix, that is itself built with ‘guile-3.0-latest’.
>
> Well, now I’m second guessing myself.  :)
>
> It is just the auto compilation notes and warnings that I’m worried
> about.  The module closure of “swh.scm” works fine on Guile 3.0.2.
>
> Eventually, the daemon will invoke Disarchive via “builtin:download” and
> “perform-download.scm”.  I intend to use the Scheme interface there,
> which means Disarchive will be runing on Guile 3.0.5.  For that, it
> would be preferable to have a Guile 3.0.5 version of Disarchive, right?

No, that’s fine.  Guile 3.0.5 can run 3.0.2 bytecode without any
warnings; what yields warnings is doing it the other way around.
Anyway, we can always revisit this if problems come up.

> On the other hand, when using Disarchive to extract metadata (e.g., with
> Cuirass), the SWH code is not needed at all.
>
> I will resurrect my patch for calling Disarchive from Guix, and come
> back to this when I know exactly what kind of package I need for that to
> work smoothly.

Yay!

>>> As far as the location is concerned, I personally do not like adding
>>> too many single-package files.  Would it make sense to add this to
>>> compression.scm (like gzip) or backup.scm (like libarchive)?
>>
>> Maybe there’ll be other packages eventually in archival.scm, like the
>> SWH Python code?  It’s fine with me, but I don’t have a strong opinion.
>
> I don’t feel strongly about it either.  There’s other software besides
> Disarchive and SWH that could be called “archival”, and I think it’s
> more accurate than the other options.

Dunno maybe you can do as Leo suggests by putting it in guile-xyz.scm or
some such.

Thanks!

Ludo’.

Ping!  :-)

Ludovic Courtès <ludo@gnu.org> skribis:

> Hello!
>
> Timothy Sample <samplet@ngyro.com> skribis:
>
>> Ludovic Courtès <ludo@gnu.org> writes:
>>
>>> Leo Prikler <leo.prikler@student.tugraz.at> skribis:
>>>
>>>> I've checked and the package seems to build fine with Guile 3.0.2.  I
>>>> think the bytecode mismatch happens, because Guix compiles stuff with
>>>> 3.0.2 by default, but users have 3.0.5 in their system, which is not
>>>> bytecode-compatible.  (As an exception, Guix itself seems to be
>>>> compiled with Guile 3.0.5 for performance reasons).
>>>>
>>>> I think it would be fine to add with Guile 3.0.2, perhaps adding a note
>>>> that Guile 3.0.5 will effectively be required to use Guix interop?  If
>>>> not, could you provide a script, that breaks in a way other than
>>>> recompiling the mismatching code?
>>>
>>> I tend to agree here: I don’t think ‘guile-3.0-latest’ is needed in this
>>> case.  The only case where you need it is if it depends on a library,
>>> such as Guix, that is itself built with ‘guile-3.0-latest’.
>>
>> Well, now I’m second guessing myself.  :)
>>
>> It is just the auto compilation notes and warnings that I’m worried
>> about.  The module closure of “swh.scm” works fine on Guile 3.0.2.
>>
>> Eventually, the daemon will invoke Disarchive via “builtin:download” and
>> “perform-download.scm”.  I intend to use the Scheme interface there,
>> which means Disarchive will be runing on Guile 3.0.5.  For that, it
>> would be preferable to have a Guile 3.0.5 version of Disarchive, right?
>
> No, that’s fine.  Guile 3.0.5 can run 3.0.2 bytecode without any
> warnings; what yields warnings is doing it the other way around.
> Anyway, we can always revisit this if problems come up.
>
>> On the other hand, when using Disarchive to extract metadata (e.g., with
>> Cuirass), the SWH code is not needed at all.
>>
>> I will resurrect my patch for calling Disarchive from Guix, and come
>> back to this when I know exactly what kind of package I need for that to
>> work smoothly.
>
> Yay!
>
>>>> As far as the location is concerned, I personally do not like adding
>>>> too many single-package files.  Would it make sense to add this to
>>>> compression.scm (like gzip) or backup.scm (like libarchive)?
>>>
>>> Maybe there’ll be other packages eventually in archival.scm, like the
>>> SWH Python code?  It’s fine with me, but I don’t have a strong opinion.
>>
>> I don’t feel strongly about it either.  There’s other software besides
>> Disarchive and SWH that could be called “archival”, and I think it’s
>> more accurate than the other options.
>
> Dunno maybe you can do as Leo suggests by putting it in guile-xyz.scm or
> some such.
>
> Thanks!
>
> Ludo’.

Am Sonntag, den 21.03.2021, 11:29 +0100 schrieb Ludovic Courtès:
> Ping!  :-)
I've pushed guile-quickcheck as
4cd88522f233dcb9affa3d3b0eada154439487c1, so we now only need to
discuss what to do with (guile-)?disarchive.

> > Hello!
> > 
> > Timothy Sample <samplet@ngyro.com> skribis:
> > 
> > > Ludovic Courtès <ludo@gnu.org> writes:
> > > 
> > > > Leo Prikler <leo.prikler@student.tugraz.at> skribis:
> > > > 
> > > > > I've checked and the package seems to build fine with Guile
> > > > > 3.0.2.  I
> > > > > think the bytecode mismatch happens, because Guix compiles
> > > > > stuff with
> > > > > 3.0.2 by default, but users have 3.0.5 in their system, which
> > > > > is not
> > > > > bytecode-compatible.  (As an exception, Guix itself seems to
> > > > > be
> > > > > compiled with Guile 3.0.5 for performance reasons).
> > > > > 
> > > > > I think it would be fine to add with Guile 3.0.2, perhaps
> > > > > adding a note
> > > > > that Guile 3.0.5 will effectively be required to use Guix
> > > > > interop?  If
> > > > > not, could you provide a script, that breaks in a way other
> > > > > than
> > > > > recompiling the mismatching code?
> > > > 
> > > > I tend to agree here: I don’t think ‘guile-3.0-latest’ is
> > > > needed in this
> > > > case.  The only case where you need it is if it depends on a
> > > > library,
> > > > such as Guix, that is itself built with ‘guile-3.0-latest’.
> > > 
> > > Well, now I’m second guessing myself.  :)
> > > 
> > > It is just the auto compilation notes and warnings that I’m
> > > worried
> > > about.  The module closure of “swh.scm” works fine on Guile
> > > 3.0.2.
> > > 
> > > Eventually, the daemon will invoke Disarchive via
> > > “builtin:download” and
> > > “perform-download.scm”.  I intend to use the Scheme interface
> > > there,
> > > which means Disarchive will be runing on Guile 3.0.5.  For that,
> > > it
> > > would be preferable to have a Guile 3.0.5 version of Disarchive,
> > > right?
> > 
> > No, that’s fine.  Guile 3.0.5 can run 3.0.2 bytecode without any
> > warnings; what yields warnings is doing it the other way around.
> > Anyway, we can always revisit this if problems come up.
> > 
> > > On the other hand, when using Disarchive to extract metadata
> > > (e.g., with
> > > Cuirass), the SWH code is not needed at all.
> > > 
> > > I will resurrect my patch for calling Disarchive from Guix, and
> > > come
> > > back to this when I know exactly what kind of package I need for
> > > that to
> > > work smoothly.
> > 
> > Yay!
> > 
> > > > > As far as the location is concerned, I personally do not like
> > > > > adding
> > > > > too many single-package files.  Would it make sense to add
> > > > > this to
> > > > > compression.scm (like gzip) or backup.scm (like libarchive)?
> > > > 
> > > > Maybe there’ll be other packages eventually in archival.scm,
> > > > like the
> > > > SWH Python code?  It’s fine with me, but I don’t have a strong
> > > > opinion.
> > > 
> > > I don’t feel strongly about it either.  There’s other software
> > > besides
> > > Disarchive and SWH that could be called “archival”, and I think
> > > it’s
> > > more accurate than the other options.
> > 
> > Dunno maybe you can do as Leo suggests by putting it in guile-
> > xyz.scm or
> > some such.
> > 
> > Thanks!
> > 
> > Ludo’.

Hi,

Leo Prikler <leo.prikler@student.tugraz.at> writes:

> Am Sonntag, den 21.03.2021, 11:29 +0100 schrieb Ludovic Courtès:
>> Ping!  :-)
> I've pushed guile-quickcheck as
> 4cd88522f233dcb9affa3d3b0eada154439487c1, so we now only need to
> discuss what to do with (guile-)?disarchive.

Hey thanks Leo!

I will put Disarchive in “backup.scm” with Guile 3.0.2.  In my eyes, it
is a regular command-line program that happens to have a Scheme
interface.  (Thanks Ludo – I didn’t know Guile 3.0.5 was fine with 3.0.2
bytecode).

I need to release a new version though, hence the delay.  While working
on the download code, I wanted some changes to Disarchive.  Now, it lets
Guix figure out how to download the Disarchive specification file and
the directory from SWH.  This allows Guix to reuse all of its
specialized HTTP code, simplifies Disarchive, and makes it more suitable
for use outside of Guix (by Nix or SWH or whatever).  It all works here
with a database I set up at <https://disarchive.ngyro.com>.  Right now,
I am finishing one last UI tweak (stopping Disarchive from printing a
backtrace when the directory does not exist in the SWH archive).  Then,
I will release 0.2.0 and push this patch with the new version.  Then –
finally – we can talk about using it.

Thanks for your continued patience!  :)


-- Tim

On Sun, 2021-03-21 at 10:13 -0400, Timothy Sample wrote:
> Hi,
> 
> [...] It all works here
> with a database I set up at <https://disarchive.ngyro.com>;.

That web page is saying ‘403 Forbidden --- nginx’.

> Thanks for your continued patience!  :)

Thank you for your continued work,

Maxime.

Howdy!

Timothy Sample <samplet@ngyro.com> skribis:

> I will put Disarchive in “backup.scm” with Guile 3.0.2.  In my eyes, it
> is a regular command-line program that happens to have a Scheme
> interface.  (Thanks Ludo – I didn’t know Guile 3.0.5 was fine with 3.0.2
> bytecode).

Alright!

> I need to release a new version though, hence the delay.  While working
> on the download code, I wanted some changes to Disarchive.  Now, it lets
> Guix figure out how to download the Disarchive specification file and
> the directory from SWH.  This allows Guix to reuse all of its
> specialized HTTP code, simplifies Disarchive, and makes it more suitable
> for use outside of Guix (by Nix or SWH or whatever).  It all works here
> with a database I set up at <https://disarchive.ngyro.com>.  Right now,
> I am finishing one last UI tweak (stopping Disarchive from printing a
> backtrace when the directory does not exist in the SWH archive).  Then,
> I will release 0.2.0 and push this patch with the new version.  Then –
> finally – we can talk about using it.

That sounds great—always improving.

> Thanks for your continued patience!  :)

Did you mean my continued impatience?  :-)

Thanks for the great quality work, as always!

Ludo’.

Hey,

Timothy Sample <samplet@ngyro.com> writes:

> Right now, I am finishing one last UI tweak (stopping Disarchive from
> printing a backtrace when the directory does not exist in the SWH
> archive).  Then, I will release 0.2.0 and push this patch with the
> new version.

Done and pushed!

> Then – finally – we can talk about using it.

See <https://issues.guix.gnu.org/47336>.


-- Tim

Hi Timothy,

Timothy Sample <samplet@ngyro.com> skribis:

> Timothy Sample <samplet@ngyro.com> writes:
>
>> Right now, I am finishing one last UI tweak (stopping Disarchive from
>> printing a backtrace when the directory does not exist in the SWH
>> archive).  Then, I will release 0.2.0 and push this patch with the
>> new version.
>
> Done and pushed!
>
>> Then – finally – we can talk about using it.
>
> See <https://issues.guix.gnu.org/47336>.

Woohoo, congrats on the new release and thank you for all this!

Ludo’.