| Message ID | 20200926204819.30602-1-kyle@kyleam.com |
|---|---|
| State | Accepted |
| Headers | show |
| Series | [bug#43637] gnu: packages: Add grokmirror. | expand |
| Context | Check | Description |
|---|---|---|
| cbaines/comparison | success | View comparision |
| cbaines/git branch | success | View Git branch |
| cbaines/applying patch | success | View Laminar job |
A recent message on bug-guix [1] mentioned cloning the archives at
<https://yhetil.org>. public-inbox archives, including those at
yhetil.org, publish a grokmirror-compatible manifest. So, if you're
interested in mirroring the Guix-related archives at yhetil.org for any
reason [2], you can use grokmirror to do so in a way that's nicer than
periodically hitting yhetil.org with a git-fetch call for each
Guix-related inbox.
You can clone (and then later update) Guix-related inboxes with
$ grok-pull -vc /path/to/config
and a config file that looks something like this:
--8<---------------cut here---------------start------------->8---
[core]
toplevel = /path/to/inboxes
manifest = ${toplevel}/manifest.js.gz
log = ${toplevel}/grok.log
loglevel = info
objstore = ${toplevel}/objstore
[pull]
pull_threads = 2
include = /guix-*
[remote]
site = https://yhetil.org
manifest = ${site}/manifest.js.gz
--8<---------------cut here---------------end--------------->8---
[1] https://issues.guix.gnu.org/43528#4
CAJ3okZ0g25+-wsL2OepE+xMR5QWShmS73hSHsjEmEgYBiOWwAg@mail.gmail.com
[2] to have a fast local copy, to publish your own mirror, to feed the
archives into some other tool...
Hi, Kyle Meyer <kyle@kyleam.com> skribis: > * gnu/packages/version-control.scm (grokmirror): New variable. Applied, thanks! > A recent message on bug-guix [1] mentioned cloning the archives at > <https://yhetil.org>. public-inbox archives, including those at > yhetil.org, publish a grokmirror-compatible manifest. So, if you're > interested in mirroring the Guix-related archives at yhetil.org for any > reason [2], you can use grokmirror to do so in a way that's nicer than > periodically hitting yhetil.org with a git-fetch call for each > Guix-related inbox. So yhetil.org stores mailing list archives as a Git repo? Sounds interesting. Thanks, Ludo’.
Ludovic Courtès writes: > Hi, > > Kyle Meyer <kyle@kyleam.com> skribis: > >> * gnu/packages/version-control.scm (grokmirror): New variable. > > Applied, thanks! Thanks for reviewing/applying! >> A recent message on bug-guix [1] mentioned cloning the archives at >> <https://yhetil.org>. public-inbox archives, including those at >> yhetil.org, publish a grokmirror-compatible manifest. So, if you're >> interested in mirroring the Guix-related archives at yhetil.org for any >> reason [2], you can use grokmirror to do so in a way that's nicer than >> periodically hitting yhetil.org with a git-fetch call for each >> Guix-related inbox. > > So yhetil.org stores mailing list archives as a Git repo? > > Sounds interesting. Yep, messages are stored in a Git repo or, for archives greater than around one gigabyte, a collection of repos [*]. (A freshly gc'd guix-devel weighs in at ~140M.) [*] documentation on public-inbox's v2 format: https://public-inbox.org/public-inbox-v2-format.html
diff --git a/gnu/packages/version-control.scm b/gnu/packages/version-control.scm index 2ee8247071..68971c60f9 100644 --- a/gnu/packages/version-control.scm +++ b/gnu/packages/version-control.scm @@ -7,7 +7,7 @@ ;;; Copyright © 2014, 2015, 2016 Mark H Weaver <mhw@netris.org> ;;; Copyright © 2014, 2016, 2019 Eric Bavier <bavier@member.fsf.org> ;;; Copyright © 2015, 2016, 2017, 2018, 2019, 2020 Efraim Flashner <efraim@flashner.co.il> -;;; Copyright © 2015, 2018 Kyle Meyer <kyle@kyleam.com> +;;; Copyright © 2015, 2018, 2020 Kyle Meyer <kyle@kyleam.com> ;;; Copyright © 2015, 2017, 2018, 2020 Ricardo Wurmus <rekado@elephly.net> ;;; Copyright © 2016, 2017 Leo Famulari <leo@famulari.name> ;;; Copyright © 2016, 2017, 2018 Nikita <nikita@n0.is> @@ -2269,6 +2269,43 @@ (define-public myrepos Mercurial, Bazaar, Darcs, CVS, Fossil, and Veracity.") (license license:gpl2+))) +(define-public grokmirror + (package + (name "grokmirror") + (version "2.0.0") + (source + (origin + (method git-fetch) + (uri (git-reference + (url (string-append "https://git.kernel.org/pub/scm/" + "utils/grokmirror/grokmirror.git")) + (commit (string-append "v" version)))) + (file-name (string-append name "-" version "-checkout")) + (sha256 + (base32 "1cs43vf87x8x5k5ncgiwiclc92a1dvxpg2z6lh6psaiip808gylp")))) + (build-system python-build-system) + (arguments + `(#:tests? #f ; no test suite + #:phases + (modify-phases %standard-phases + (add-after 'install 'install-manpages + (lambda* (#:key outputs #:allow-other-keys) + (let* ((man (string-append (assoc-ref outputs "out") + "/man/man1/"))) + (mkdir-p man) + (for-each (lambda (file) (install-file file man)) + (find-files "." "\\.1$"))) + #t))))) + (propagated-inputs + `(("python-requests" ,python-requests))) + (home-page + "https://git.kernel.org/pub/scm/utils/grokmirror/grokmirror.git") + (synopsis "Framework to smartly mirror git repositories") + (description "Grokmirror enables replicating large git repository +collections efficiently. Mirrors decide to clone and update repositories +based on a manifest file published by servers.") + (license license:gpl3+))) + (define-public git-annex-remote-rclone (package (name "git-annex-remote-rclone")