[bug#36919,1/2] gnu-maintenance: KDE updater no longer relies on FTP access.

* guix/gnu-maintenance.scm (%kde-file-list-uri): New variable.
  (download.kde.org-files): New procedure.
  (latest-kde-release): Change to use DOWNLOAD.KDE.ORG-FILES and search
  for files in this list.
---
 guix/gnu-maintenance.scm | 77 ++++++++++++++++++++++++++++++++++++----
 1 file changed, 70 insertions(+), 7 deletions(-)

Hi Hartmut,

Hartmut Goebel <h.goebel@crazy-compilers.com> skribis:

> * guix/gnu-maintenance.scm (%kde-file-list-uri): New variable.
>   (download.kde.org-files): New procedure.
>   (latest-kde-release): Change to use DOWNLOAD.KDE.ORG-FILES and search
>   for files in this list.

Nice!

How about moving this code to (guix import kde) as was done for (guix
import gnome) when we discussed it back then?  (See
<https://issues.guix.gnu.org/issue/28159>.)

> +(define download.kde.org-files
> +  (mlambda ()
> +    "Return the list of files available at download.kde.org."
> +    ;; XXX: Memoize the whole procedure to work around the fact that
> +    ;; 'http-fetch/cached' caches the bzip2-compressed version.
> +
> +    (define (canonicalize-path path)
> +      (if (string-prefix? "/srv/archives/ftp/" path)
> +          (set! path (string-drop path 17)))
> +      (if (string-suffix? ":" path)
> +          (set! path (string-drop-right path 1)))
> +      (if (not (string-suffix? "/" path))
> +          (set! path (string-append path "/")))
> +      path)

As a rule of thumb we don’t use ‘set!’ in Guix, except in special
circumstances.  In this case you can write:

  (define (canonicalize-path path)
    (cond ((string-prefix? …)
           (string-drop path 17))
          ((string-suffix? …)
           (string-drop-right path 1))
          …))

> +    (define (ls-lR-line->filename path line)
> +      ;; remove mode, blocks, user, group, size, date, time and one space
> +      (regexp-substitute
> +       #f (string-match "^(\\S+\\s+){6}\\S+\\s" line) path 'post))
> +
> +    (let ((entries `())
> +          (port (decompressed-port
> +                 'bzip2
> +                 (http-fetch/cached %kde-file-list-uri #:ttl 3600))))

What about passing ‘http-fetch/cached’ a custom #:write-cache, as is
done in (guix cve)?  That would allow us to store the cached file list
in a pre-processed (and possibly decompressed) format, speeding up
operation on cache hits.

> +      (do ((path (read-line port) (read-line port)))
> +          ((or (eof-object? path) (string= path "")))
> +        (set! path (canonicalize-path path))

I also recommend against ‘do’.  You can use a “named let” loop instead,
as in:

  (let loop ((files '()))
    (match (read-line port)
      ((? eof-object?)
       (reverse files))
      (line
       (loop (cons … files)))))

That’s about it.

Thanks!

Ludo’.

Am 17.08.19 um 23:01 schrieb Ludovic Courtès:
> Nice!

Thansk :-)


> How about moving this code to (guix import kde) as was done for (guix
> import gnome) when we discussed it back then?  (See
> <https://issues.guix.gnu.org/issue/28159>.)

I'll be fine with this.

I just wonder whether we/I should refactor the new code to be more
flexible for other ls-lR cases and keep the common parts in
gnu-maintenance.scm. OTOH currently there is no other use-case

Hi Ludo,

thanks for the coding advice. This was what I've been asking for :-)
Just one point:

Am 17.08.19 um 23:01 schrieb Ludovic Courtès:
> As a rule of thumb we don’t use ‘set!’ in Guix, except in special
> circumstances.  In this case you can write:
>
>   (define (canonicalize-path path)
>     (cond ((string-prefix? …)
>            (string-drop path 17))
>           ((string-suffix? …)
>            (string-drop-right path 1))
>           …))

AFAIK, `cond` only processes the first expression where `test ` is true.
In this case, we need to process *all* cases where the test is true.
This means we need to nest the evaluation, which is ugly and hard to
read IMHO. Is there some more "linear" syntax?

(BTW: The manual [1] is not quite precise on `cond`, so I needed to test
it. Maybe I did it wrong.)

[1] https://www.gnu.org/software/guile/manual/html_node/Conditionals.html

Hi,

Hartmut Goebel <h.goebel@crazy-compilers.com> skribis:

>> How about moving this code to (guix import kde) as was done for (guix
>> import gnome) when we discussed it back then?  (See
>> <https://issues.guix.gnu.org/issue/28159>.)
>
> I'll be fine with this.
>
> I just wonder whether we/I should refactor the new code to be more
> flexible for other ls-lR cases and keep the common parts in
> gnu-maintenance.scm. OTOH currently there is no other use-case

Yeah, we’d have to identify what common parts exist.  On IRC we discussed
utility procedures like ‘file-sans-extension’, which would be worth
factorizing.

Other things may not be good candidates—for instance, the GNU thing is
probably close to what you’d write for KDE, but it’s still not exactly
the same.  Since there’s usually fine-tuning to be done, it may be best
to keep them separate.

> Am 17.08.19 um 23:01 schrieb Ludovic Courtès:
>> As a rule of thumb we don’t use ‘set!’ in Guix, except in special
>> circumstances.  In this case you can write:
>>
>>   (define (canonicalize-path path)
>>     (cond ((string-prefix? …)
>>            (string-drop path 17))
>>           ((string-suffix? …)
>>            (string-drop-right path 1))
>>           …))
>
> AFAIK, `cond` only processes the first expression where `test ` is true.
> In this case, we need to process *all* cases where the test is true.
> This means we need to nest the evaluation, which is ugly and hard to
> read IMHO. Is there some more "linear" syntax?

Oh I see.  You could roughly have one procedure for each clause and
chain them.  A macro might help make that more readable (Clojure has
‘->’).

HTH!

Ludo’.