| Message ID | cover.1708697539.git.ludo@gnu.org |
|---|---|
| Headers | show |
| Series | Better source code recovery from SWH | expand |
I forgot to Cc: you Timothy, but you may have useful feedback to give on this series: <https://issues.guix.gnu.org/69328>. (Should we create a ‘source-code-archival’ team?)
Ludovic Courtès <ludo@gnu.org> writes: > Hello Guix! > > This patch series improves source code recovery from SWH, as a followup > to <https://issues.guix.gnu.org/68741>. > > It does several things: > > • ‘guix lint -c archival’ now emits save requests for VCSes other > than Git. > > • Fix <https://issues.guix.gnu.org/69070>. > > • Allow content-addressed recovery of Mercurial and Subversion > checkouts. > > • Allow Bazaar recovery using ‘download-nar’ (I didn’t bother with SWH). > > • Have all these things honor the ‘GUIX_DOWNLOAD_SEQUENCE’ environment > variable. Very nice! I like the design of ‘GUIX_DOWNLOAD_SEQUENCE’ compared to ‘GUIX_DOWNLOAD_FALLBACK_TEST’, but I’m not sure about the name (sorry for bike shedding!). In particular, the “sequences” ‘(nar swh)’ and ‘(swh nar)’ will both try ‘nar’ first and then ‘swh’. What about “methods” or “strategies” or something? > You can try the various methods like this: > > GUIX_DOWNLOAD_SEQUENCE=nar ./pre-inst-env guix build -S apl --check > GUIX_DOWNLOAD_SEQUENCE=swh ./pre-inst-env guix build -S guile-wisp --check > GUIX_DOWNLOAD_SEQUENCE=swh ./pre-inst-env guix build -S guile-gcrypt --check I tried GUIX_DOWNLOAD_SEQUENCE=disarchive ./pre-inst-env guix build -S mes --check and it worked like a charm. > Feedback welcome! Other than the name and the little separate comment on the last patch, this all LGTM. -- Tim
Hi, Timothy Sample <samplet@ngyro.com> skribis: > Ludovic Courtès <ludo@gnu.org> writes: [...] >> • Have all these things honor the ‘GUIX_DOWNLOAD_SEQUENCE’ environment >> variable. > > Very nice! I like the design of ‘GUIX_DOWNLOAD_SEQUENCE’ compared to > ‘GUIX_DOWNLOAD_FALLBACK_TEST’, but I’m not sure about the name (sorry > for bike shedding!). In particular, the “sequences” ‘(nar swh)’ and > ‘(swh nar)’ will both try ‘nar’ first and then ‘swh’. What about > “methods” or “strategies” or something? Good point; I like “methods”. > Other than the name and the little separate comment on the last patch, > this all LGTM. Awesome; I’ll send an updated version and merge by the end of the week if nobody objects. Ludo’.
Hello Guix! This patch series improves source code recovery from SWH, as a followup to <https://issues.guix.gnu.org/68741>. It does several things: • ‘guix lint -c archival’ now emits save requests for VCSes other than Git. • Fix <https://issues.guix.gnu.org/69070>. • Allow content-addressed recovery of Mercurial and Subversion checkouts. • Allow Bazaar recovery using ‘download-nar’ (I didn’t bother with SWH). • Have all these things honor the ‘GUIX_DOWNLOAD_SEQUENCE’ environment variable. You can try the various methods like this: GUIX_DOWNLOAD_SEQUENCE=nar ./pre-inst-env guix build -S apl --check GUIX_DOWNLOAD_SEQUENCE=swh ./pre-inst-env guix build -S guile-wisp --check GUIX_DOWNLOAD_SEQUENCE=swh ./pre-inst-env guix build -S guile-gcrypt --check In the last case, note that you must be running guix-daemon for the checkout since that uses “builtin:git-download”, which is implemented on the server side. There’s a few caveats: • Mercurial SWH fallback almost works, but not quite, due to this SWH bug: <https://gitlab.softwareheritage.org/swh/infra/sysadm-environment/-/issues/5256>. • Right now, no Subversion checkout has the nar-sha256 ExtID at SWH for unclear reasons, so retrieving the source of ‘apl’ (say) from SWH doesn’t work yet. • Multi-directory Subversion downloads (‘svn-multi-fetch’) is not supported yet. For that we’ll need to arrange with our SWH friends so they compute nar-sha256 ExtIDs for combined directories (and we’ll have to include that info in ‘sources.json’). Feedback welcome! Ludo’. Ludovic Courtès (12): lint: Switch to SRFI-71. lint: archival: Fix crash in non-Git case. lint: archival: Trigger “Save Code Now” for VCSes other than Git. swh: Add ‘type’ field to <visit>. swh: ‘origin-visits’ takes an optional ‘max’ parameter. swh: ‘lookup-origin-revision’ handles branches pointing to directories. hg-download: Use ‘swh-download-directory-by-nar-hash’. svn-download: Use ‘swh-download-directory-by-nar-hash’. bzr-download: Implement nar fallback. download-nar: Distinguish ‘output’ and ‘item’ parameter. perform-download: Allow use of ‘download-nar’ for ‘--check’ builds. download: Honor ‘GUIX_DOWNLOAD_SEQUENCE’ environment variable. guix/build/bzr.scm | 3 +- guix/build/download-nar.scm | 12 +-- guix/build/download.scm | 50 +++++++--- guix/build/git.scm | 27 ++++-- guix/bzr-download.scm | 57 ++++++++--- guix/cvs-download.scm | 24 +++-- guix/download.scm | 53 ++++------- guix/git-download.scm | 20 ++-- guix/hg-download.scm | 36 ++++--- guix/lint.scm | 151 +++++++++++++++++++----------- guix/scripts/perform-download.scm | 65 +++++++------ guix/svn-download.scm | 84 +++++++++++------ guix/swh.scm | 71 ++++++++------ tests/lint.scm | 20 ++++ tests/swh.scm | 74 +++++++++++++++ 15 files changed, 501 insertions(+), 246 deletions(-) base-commit: ffcce77ec488e3c89401ad77fafa65fcd9e9f5be