Message ID | 20210419000958.49968055@tachikoma.lepiller.eu |
---|---|
State | Accepted |
Headers | show |
Series | [bug#47882] Makefile: Reimplement download-po target | expand |
Context | Check | Description |
---|---|---|
cbaines/comparison | success | View comparision |
cbaines/git branch | success | View Git branch |
cbaines/applying patch | fail | View Laminar job |
cbaines/issue | success | View issue |
Hi! Julien Lepiller <julien@lepiller.eu> skribis: > The attached patch reimplements the download-po target in Makefile.am. > The previous implementation was based on the implementation for the TP > and would use the API of Weblate to get files. > > However, the rate-limit is very close to the number of files we need to > download, which was a big limitation, and this target would only > download existing files, ignoring new translations. OK. > To document the new process: > > make download-po > for any new file in po/guix and po/packages, add the language in a > single line in po/guix/LINGUAS or po/packages/LINGUAS. > for any new file in po/doc, add the file name to po/doc/local.mk, in > DOC_PO_FILES or DOC_COOKBOOK_PO_FILES depending on the translation > type. Add the texi file name to doc/local.mk too, in info_TEXINFOS. Alright. This looks like a nice improvement! Some nitpicking follows, but it LGTM overall. >>From 5c14506d6af24b5307e03604cabf8ca10af56067 Mon Sep 17 00:00:00 2001 > From: Julien Lepiller <julien@lepiller.eu> > Date: Sun, 18 Apr 2021 23:56:48 +0200 > Subject: [PATCH] Makefile: Reimplement `download-po` target. > > The weblate API rate limit is very close to the number of files we need > to download. The previous implementation did not add new translations. > > * Makefile.am (download-po): Update target. > (make-download-po, make-check-po): Remove functions. Should be: (make-download-po-rule, make-check-po-rule) > - rm "$(top_srcdir)/$(2)/$(3)$$$$lang.po.tmp" ; \ > - mv "$(top_srcdir)/$(2)/$(3)$$$$lang.po"{.tmp2,} ; \ That was kinda crazy, I’m glad you divided the number of dollars by two. :-) > +WEBLATE_REPO="https://framagit.org/tyreunom/guix-translations" Since it’s not shell, rather: WEBLATE_REPO = https://framagit.org/tyreunom/guix-translations BTW, I vaguely remember we asked for a Savannah repo but I’m not sure what the outcome was; did it eventually stall for some obscure reason? > +# shallow clone the git repository behind weblate and copy files from it if > +# they contain at least one translation, and they are well-formed (scheme format > +# only), warn otherwise. Copied files are converted to a canonical form. Please capitalize sentences and proper names. > +download-po: > + dir=$$(mktemp -d); \ > + git clone --depth 1 "$(WEBLATE_REPO)" "$$dir/translations"; \ > + for domain in po/doc po/guix po/packages; do \ > + for po in "$$dir/translations/$$domain"/*.po; do \ > + translated=$$(LANG=en_US.UTF-8 msgfmt --statistics "$$po" 2>&1 | cut -f1 -d' '); \ > + target=$$(basename "$$po"); \ > + target="$$domain/$$target"; \ > + if msgfmt -c "$$po" && [ "$$translated" != "0" ]; then \ > + msgfilter --no-wrap -i "$$po" cat > "$$target"; \ Maybe write to $$target.tmp and then do “mv $$target.tmp $$target” so it’s atomic and we don’t end up with corrupt files if ‘msgfilter’ fails. I would use spaces rather than tabs to indent the ‘for’ loops. Thanks! Ludo’.
Le Fri, 30 Apr 2021 21:57:05 +0200, Ludovic Courtès <ludo@gnu.org> a écrit : > Hi! > > Alright. This looks like a nice improvement! Some nitpicking > follows, but it LGTM overall. Thank you! I fixed your comments and pushed to master as 45549e9f31e462d0d1519fd678dd782da6ed5cb9. Thank you! > BTW, I vaguely remember we asked for a Savannah repo but I’m not sure > what the outcome was; did it eventually stall for some obscure reason? Yeah, there were still some issues related to copyright. To be honest I lost my energy to push on this front...
Julien Lepiller <julien@lepiller.eu> skribis: > Le Fri, 30 Apr 2021 21:57:05 +0200, [...] >> Alright. This looks like a nice improvement! Some nitpicking >> follows, but it LGTM overall. > > Thank you! I fixed your comments and pushed to master as > 45549e9f31e462d0d1519fd678dd782da6ed5cb9. Thank you! Cool! >> BTW, I vaguely remember we asked for a Savannah repo but I’m not sure >> what the outcome was; did it eventually stall for some obscure reason? > > Yeah, there were still some issues related to copyright. To be honest I > lost my energy to push on this front... Yeah, I sympathize. It’s fine and more productive to use the repo you set up. Thank you! Ludo’.
From 5c14506d6af24b5307e03604cabf8ca10af56067 Mon Sep 17 00:00:00 2001 From: Julien Lepiller <julien@lepiller.eu> Date: Sun, 18 Apr 2021 23:56:48 +0200 Subject: [PATCH] Makefile: Reimplement `download-po` target. The weblate API rate limit is very close to the number of files we need to download. The previous implementation did not add new translations. * Makefile.am (download-po): Update target. (make-download-po, make-check-po): Remove functions. --- Makefile.am | 87 ++++++++++++++--------------------------------------- 1 file changed, 22 insertions(+), 65 deletions(-) diff --git a/Makefile.am b/Makefile.am index 17ad236655..811e47dbfc 100644 --- a/Makefile.am +++ b/Makefile.am @@ -969,71 +969,28 @@ cuirass-jobs: $(GOBJECTS) # Downloading up-to-date PO files. -# make-download-po-rule DOMAIN DIRECTORY [FILE-NAME-PREFIX] -define make-download-po-rule - -download-po.$(1): - if [ -f "$(top_srcdir)/$(2)/LINGUAS" ]; then \ - LINGUAS="`grep -v '^[[:blank:]]*#' < $(top_srcdir)/$(2)/LINGUAS`" ; \ - else \ - LINGUAS="`(cd $(top_srcdir)/$(2); \ - for i in *.po; do echo $$$$i; done) | cut -d . -f 2`" ; \ - fi ; \ - for lang in $$$$LINGUAS; do \ - if wget -nv -O "$(top_srcdir)/$(2)/$(3)$$$$lang.po.tmp" \ - "https://translate.fedoraproject.org/api/translations/guix/$(1)/$$$$lang/file/" ; \ - then \ - msgfilter --no-wrap -i "$(top_srcdir)/$(2)/$(3)$$$$lang.po.tmp" \ - cat > "$(top_srcdir)/$(2)/$(3)$$$$lang.po.tmp2" ; \ - rm "$(top_srcdir)/$(2)/$(3)$$$$lang.po.tmp" ; \ - mv "$(top_srcdir)/$(2)/$(3)$$$$lang.po"{.tmp2,} ; \ - else \ - rm "$(top_srcdir)/$(2)/$(3)$$$$lang.po.tmp" ; \ - fi ; \ - done - -.PHONY: download-po.$(1) - -endef - -# Checking po files for issues. This is useful to run after downloading new -# po files. - -# make-check-po-rule DOMAIN DIRECTORY [FILE-NAME-PREFIX] -define make-check-po-rule - -check-po.$(1): - if [ -f "$(top_srcdir)/$(2)/LINGUAS" ]; then \ - LINGUAS="`grep -v '^[[:blank:]]*#' < $(top_srcdir)/$(2)/LINGUAS`" ; \ - else \ - LINGUAS="`(cd $(top_srcdir)/$(2); \ - for i in *.po; do echo $$$$i; done) | cut -d . -f 2`" ; \ - fi ; \ - for lang in $$$$LINGUAS; do \ - if [ -f "$(top_srcdir)/$(2)/$(3)$$$$lang.po" ]; \ - then \ - if ! msgfmt -c "$(top_srcdir)/$(2)/$(3)$$$$lang.po" ; \ - then \ - exit 1 ; \ - fi ; \ - fi ; \ - done - -.PHONY: check-po.$(1) - -endef - -$(eval $(call make-download-po-rule,documentation-cookbook,po/doc,guix-cookbook.)) -$(eval $(call make-download-po-rule,documentation-manual,po/doc,guix-manual.)) -$(eval $(call make-download-po-rule,guix,po/guix)) -$(eval $(call make-download-po-rule,packages,po/packages)) - -$(eval $(call make-check-po-rule,documentation-cookbook,po/doc,guix-cookbook.)) -$(eval $(call make-check-po-rule,documentation-manual,po/doc,guix-manual.)) -$(eval $(call make-check-po-rule,guix,po/guix)) -$(eval $(call make-check-po-rule,packages,po/packages)) - -download-po: $(foreach domain,guix packages documentation-manual documentation-cookbook,download-po.$(domain)) +WEBLATE_REPO="https://framagit.org/tyreunom/guix-translations" + +# shallow clone the git repository behind weblate and copy files from it if +# they contain at least one translation, and they are well-formed (scheme format +# only), warn otherwise. Copied files are converted to a canonical form. +download-po: + dir=$$(mktemp -d); \ + git clone --depth 1 "$(WEBLATE_REPO)" "$$dir/translations"; \ + for domain in po/doc po/guix po/packages; do \ + for po in "$$dir/translations/$$domain"/*.po; do \ + translated=$$(LANG=en_US.UTF-8 msgfmt --statistics "$$po" 2>&1 | cut -f1 -d' '); \ + target=$$(basename "$$po"); \ + target="$$domain/$$target"; \ + if msgfmt -c "$$po" && [ "$$translated" != "0" ]; then \ + msgfilter --no-wrap -i "$$po" cat > "$$target"; \ + echo "copied $$target."; \ + else \ + echo "WARN: $$target ($$translated translated messages) was not added/updated."; \ + fi; \ + done; \ + done; \ + rm -rf "$$dir" .PHONY: download-po check-po: $(foreach domain,guix packages documentation-manual documentation-cookbook,check-po.$(domain)) -- 2.31.1