[bug#47126,0/7] Add 'generic-html' updater

Message ID 20210313214326.28052-1-ludo@gnu.org
Headers show
Series Add 'generic-html' updater | expand

Message

Ludovic Courtès March 13, 2021, 9:43 p.m. UTC
Hi!

These patches allow ‘guix refresh’ coverage to go from 78% to 88%
as reported by ‘guix refresh --list-updaters’ (both are probably
slightly overestimated) by adding a new ‘generic-html’ updater.

The updater crawls the web page where the package’s source tarball
is stored, using Guile-Lib’s (htmlprag), which we depend on since
commit 02e2e093e858e8a0ca7bd66c1f1f6fd0a1705edb.  Among other things,
it handles freedesktop.org packages.

Feedback welcome!

Thanks,
Ludo’.

Ludovic Courtès (7):
  gnu-maintenance: Use (htmlprag) for 'latest-html-release'.
  gnu-maintenance: 'latest-html-release' considers non-relative URLs.
  gnu-maintenance: 'release-file?' rejects checksum files.
  gnu-maintenance: 'latest-html-release' can determine signature file
    name.
  gnu-maintenance: 'latest-html-release' better computes version number.
  gnu-maintenance: Add 'generic-html' updater.
  gnu: hwloc: Add 'release-monitoring-url' property.

 doc/guix.texi            |   6 +-
 gnu/packages/mpi.scm     |   6 ++
 guix/gnu-maintenance.scm | 136 ++++++++++++++++++++++++++++-----------
 3 files changed, 108 insertions(+), 40 deletions(-)

Comments

Léo Le Bouter March 17, 2021, 10:18 a.m. UTC | #1
Hello!

That's awesome thanks a lot Ludo!!

I am wondering, does this handle cases where there's a subfolder with
version and then another tarball with version as well?

Like GNOME for example: 
https://download.gnome.org/sources/NetworkManager/1.31/NetworkManager-1.31.1.tar.xz

I see this is a generic solution, I see you made available some options
to customize per-package as needed but can we get as precise/reliable
as Debian's watch/uscan with that?

Or if I understand correctly, we should always point it to a page where
the link for the latest release is always published? That last thing
really sounds nice!

Léo
Ludovic Courtès March 17, 2021, 1:52 p.m. UTC | #2
Hi Léo,

Léo Le Bouter <lle-bout@zaclys.net> skribis:

> That's awesome thanks a lot Ludo!!

Just pushed this series as fe96f64110676f28b948f0d31a1726501abdae0e.
Unleash your update powers, comrades!  :-)

> I am wondering, does this handle cases where there's a subfolder with
> version and then another tarball with version as well?
>
> Like GNOME for example: 
> https://download.gnome.org/sources/NetworkManager/1.31/NetworkManager-1.31.1.tar.xz
>
> I see this is a generic solution, I see you made available some options
> to customize per-package as needed but can we get as precise/reliable
> as Debian's watch/uscan with that?

There’s a ‘gnome’ updater for GNOME:

  https://guix.gnu.org/manual/en/html_node/Invoking-guix-refresh.html

And yes, it actually works.  :-)

In the case of NetworkManager, there’s a bug right now:

--8<---------------cut here---------------start------------->8---
$ guix refresh network-manager
ni sekvas la redirektigon al 'https://download.gnome.org/sources/NetworkManager/cache.json'...
ni sekvas la redirektigon al 'https://fr2.rpmfind.net/linux/gnome.org/sources/NetworkManager/cache.json'...
gnu/packages/gnome.scm:7648:13: network-manager would be upgraded from 1.24.0 to rc2
--8<---------------cut here---------------end--------------->8---

I’ll see what’s up.  But otherwise ‘guix refresh -t gnome’ produces
sensible results.

At any rate, updaters sometimes bitrot, produce buggy results as in the
example above.  Please do use ‘guix refresh’ and report any issues!

Also, there are still ~12% of packages for which none of the updaters
apply.  We should investigate and see how we can bring that down to
zero.

Thanks for your feedback!

Ludo’.