mbox

[bug#37413,0/9] Channel news distribution mechanism

Message ID 20190915221021.4165-1-ludo@gnu.org
Headers show

Message

Ludovic Courtès Sept. 15, 2019, 10:10 p.m. UTC
Hello Guix!

This patch series adds a mechanism for channel authors to distribute
“news entries” alongside their channels.  The goal is to provide a
way for channel authors (including authors of the 'guix channel!)
to inform their users about important changes.

For example, you’d tell people about package upgrades that require
manual intervention, about important new features, bug fixes, etc.

This is done by adding a “news file” to the channel.  Each entry in
that file is associated with a commit, meaning that the entry
describes changes related to that commit.  This information is what
allows ‘guix pull --news’ to determine whether a news entry is
relevant or not for the user: only entries corresponding to commits
that were just pulled are displayed.

It’s a bit weird to refer to commits from within a file in the repo
because it almost forces you to push so that you know the commit ID
that your news entry should refer to.  News entries become obsolete
if you rebase, too.  I think it’s acceptable, though.

Regarding i18n, the title and body of an entry can be provided in
several languages.  I wonder what workflow we should adopt;
obviously the TP wouldn’t be appropriate, and there’s no gettext
involved.

Anyway, this is it.  Feedback welcome!

Ludo’.

Ludovic Courtès (9):
  pull: '--news' shows the list of channels added or removed.
  git: 'update-cached-checkout' avoids network access when unnecessary.
  git: Add 'commit-difference'.
  channels: Add support for a news file.
  ui: Add 'current-message-language'.
  pull: Display channel news.
  pull: '-l' displays channel news.
  Add '.guix-channel' file.
  DRAFT etc: Add channel news file.

 .dir-locals.el        |   1 +
 .guix-channel         |   5 ++
 Makefile.am           |   8 ++-
 doc/guix.texi         |  62 +++++++++++++++++++--
 etc/news.scm          |  13 +++++
 guix/channels.scm     | 121 ++++++++++++++++++++++++++++++++++++++---
 guix/git.scm          |  58 +++++++++++++++++++-
 guix/scripts/pull.scm | 122 +++++++++++++++++++++++++++++++++++++++---
 guix/tests/git.scm    | 102 +++++++++++++++++++++++++++++++++++
 guix/ui.scm           |  18 +++++++
 tests/channels.scm    |  96 +++++++++++++++++++++++++++++++++
 tests/git.scm         |  99 ++++++++++++++++++++++++++++++++++
 12 files changed, 685 insertions(+), 20 deletions(-)
 create mode 100644 .guix-channel
 create mode 100644 etc/news.scm
 create mode 100644 guix/tests/git.scm
 create mode 100644 tests/git.scm

Comments

Ricardo Wurmus Sept. 16, 2019, 9:31 a.m. UTC | #1
Ludovic Courtès <ludo@gnu.org> writes:

> It’s a bit weird to refer to commits from within a file in the repo
> because it almost forces you to push so that you know the commit ID
> that your news entry should refer to.  News entries become obsolete
> if you rebase, too.  I think it’s acceptable, though.

Would tags be acceptable instead of commit IDs?  I think version tags
would be easier to deal with than commit IDs and it wouldn’t require
time travel as you can just make up a new version and refer to that in
the news file.

--
Ricardo
Ludovic Courtès Sept. 16, 2019, 12:59 p.m. UTC | #2
Hello,

Ricardo Wurmus <rekado@elephly.net> skribis:

> Ludovic Courtès <ludo@gnu.org> writes:
>
>> It’s a bit weird to refer to commits from within a file in the repo
>> because it almost forces you to push so that you know the commit ID
>> that your news entry should refer to.  News entries become obsolete
>> if you rebase, too.  I think it’s acceptable, though.
>
> Would tags be acceptable instead of commit IDs?  I think version tags
> would be easier to deal with than commit IDs and it wouldn’t require
> time travel as you can just make up a new version and refer to that in
> the news file.

So you would add a Git tag like “news-xyz” on the commit of interest and
then refer to it in the news entry?

I guess that would work.  Compared to this patch set, we’d just need to
check whether the entry’s commit is a commit or a tag, and in the latter
case resolve it.  How does that sound?

I tend to use tags sparsely, almost exclusively for releases, so I would
not naturally propose to use them for this purpose, but maybe it’s not a
problem to have many tags in the repo after all?

Thanks,
Ludo’.
Ricardo Wurmus Sept. 16, 2019, 1:16 p.m. UTC | #3
Hi Ludo,

> Ricardo Wurmus <rekado@elephly.net> skribis:
>
>> Ludovic Courtès <ludo@gnu.org> writes:
>>
>>> It’s a bit weird to refer to commits from within a file in the repo
>>> because it almost forces you to push so that you know the commit ID
>>> that your news entry should refer to.  News entries become obsolete
>>> if you rebase, too.  I think it’s acceptable, though.
>>
>> Would tags be acceptable instead of commit IDs?  I think version tags
>> would be easier to deal with than commit IDs and it wouldn’t require
>> time travel as you can just make up a new version and refer to that in
>> the news file.
>
> So you would add a Git tag like “news-xyz” on the commit of interest and
> then refer to it in the news entry?

No, I actually meant for releases.  For a channel it seems to me that a
release would make sense whenever information must be displayed to the
users (presumably to announce breaking changes).

--
Ricardo
Ludovic Courtès Sept. 16, 2019, 3:10 p.m. UTC | #4
Hi,

Ricardo Wurmus <rekado@elephly.net> skribis:

>> Ricardo Wurmus <rekado@elephly.net> skribis:
>>
>>> Ludovic Courtès <ludo@gnu.org> writes:
>>>
>>>> It’s a bit weird to refer to commits from within a file in the repo
>>>> because it almost forces you to push so that you know the commit ID
>>>> that your news entry should refer to.  News entries become obsolete
>>>> if you rebase, too.  I think it’s acceptable, though.
>>>
>>> Would tags be acceptable instead of commit IDs?  I think version tags
>>> would be easier to deal with than commit IDs and it wouldn’t require
>>> time travel as you can just make up a new version and refer to that in
>>> the news file.
>>
>> So you would add a Git tag like “news-xyz” on the commit of interest and
>> then refer to it in the news entry?
>
> No, I actually meant for releases.  For a channel it seems to me that a
> release would make sense whenever information must be displayed to the
> users (presumably to announce breaking changes).

The way I see it, we could use it in between release.  I’d have used it
to announce lzip support (and then explain how users can migrate, which
I think many haven’t done because they’re unaware), improvements to
‘guix pack’, the addition of ‘guix deploy’, things like that.

Does that make sense?

Authors of third-party channels may also want to use it anytime.

Thanks,
Ludo’.
Ludovic Courtès Sept. 17, 2019, 12:44 p.m. UTC | #5
Hi Florian,

"pelzflorian (Florian Pelz)" <pelzflorian@pelzflorian.de> skribis:

> On Mon, Sep 16, 2019 at 11:49:50PM +0200, Julien Lepiller wrote:
>> Le Mon, 16 Sep 2019 23:25:55 +0200,
>> Ludovic Courtès <ludo@gnu.org> a écrit :
>> > I’m thinking we could have an alias with volunteer translators who
>> > would provide text fragments by email to the developer who wrote the
>> > news?  Does that sound reasonable?
>> > 
>> > Thanks,
>> > Ludo’.
>> 
>> As long as the volume is low, I think it's reasonable, but it should be
>> documented.

For Guix proper, I think the volume would be quite low (once per
month?).  Translators may have to react within a couple of days though,
basically while the change in question is under review.  It’s OK to
translate later, after the initial news has been pushed, but fewer
people will see it.

> Yes, I want to provide such a service, but package names and
> descriptions should be translated too.

<https://translationproject.org/domain/guix-packages.html> is about
translating packages and descriptions (but not package names), but I
don’t see how this relates to news entries.  What do you have in mind?

The reasons I think the TP is inadequate for news entries is that the TP
is geared towards translating releases of software packages, and those
tiny snippets could come anytime and there would be no PO file for them.

Thanks,
Ludo’.
pelzflorian Sept. 17, 2019, 1:33 p.m. UTC | #6
On Tue, Sep 17, 2019 at 02:44:41PM +0200, Ludovic Courtès wrote:
> "pelzflorian (Florian Pelz)" <pelzflorian@pelzflorian.de> skribis:
> > Yes, I want to provide such a service, but package names and
> > descriptions should be translated too.
> 
> <https://translationproject.org/domain/guix-packages.html> is about
> translating packages and descriptions (but not package names),

Yes, I was confused. :)


> but I
> don’t see how this relates to news entries.  What do you have in mind?
>

I thought everything about translating the channel should be in a
single location, so translators can ensure the same terminology is
used in news file and package information.


> The reasons I think the TP is inadequate for news entries is that the TP
> is geared towards translating releases of software packages, and those
> tiny snippets could come anytime and there would be no PO file for them.
> 
> Thanks,
> Ludo’.

You mean it takes too long for the TP coordinator to accept new
versions of PO files?

Regards,
Florian
Ludovic Courtès Sept. 17, 2019, 1:39 p.m. UTC | #7
Hi,

"pelzflorian (Florian Pelz)" <pelzflorian@pelzflorian.de> skribis:

>> but I
>> don’t see how this relates to news entries.  What do you have in mind?
>>
>
> I thought everything about translating the channel should be in a
> single location, so translators can ensure the same terminology is
> used in news file and package information.

What do you mean by “everything about translating the channel”?  You
mean instructions on how to do it?

I agree that using consistent terminology is important, but I think it’s
up to translators to do the right thing here.

>> The reasons I think the TP is inadequate for news entries is that the TP
>> is geared towards translating releases of software packages, and those
>> tiny snippets could come anytime and there would be no PO file for them.
>> 
>> Thanks,
>> Ludo’.
>
> You mean it takes too long for the TP coordinator to accept new
> versions of PO files?

I mean two things: (1) there will not be PO files (see the format of
news entries proposed in this issue), and (2) I think this is outside
the scope of the TP.

Does that clarify?

Ludo’.
pelzflorian Sept. 17, 2019, 2:28 p.m. UTC | #8
On Tue, Sep 17, 2019 at 03:39:38PM +0200, Ludovic Courtès wrote:
> "pelzflorian (Florian Pelz)" <pelzflorian@pelzflorian.de> skribis:
> > You mean it takes too long for the TP coordinator to accept new
> > versions of PO files?
> 
> I mean two things: (1) there will not be PO files (see the format of
> news entries proposed in this issue),

It is easy to write a Guile script that prints a PO file with the news
file content inserted as the msgid of a single entry, if xgettext does
not support this already.  Then there would be two PO files, one for
the news file and one for the package info, that could be concatenated
by msgcat to a single file.  Guix could ship with this tooling for
channel authors and an explanation for how to submit the resulting POT
file to the Translation Project.

I think a single file is easier for translators to manage than two
files and typically we’d want news files and packages to be translated
by the same (group of) translator(s).


> and (2) I think this is outside
> the scope of the TP.
> 
> Does that clarify?
> 
> Ludo’.

I do not see why such translation should be outside the scope of the
TP?  I can only think of the speed of POT file acceptance and PO file
translators.  I am unsure if setting up Guix’ own translation team
would attract quicker translators than relying on the TP.

Regards,
Florian
Ludovic Courtès Sept. 17, 2019, 3:27 p.m. UTC | #9
Hello,

"pelzflorian (Florian Pelz)" <pelzflorian@pelzflorian.de> skribis:

> On Tue, Sep 17, 2019 at 03:39:38PM +0200, Ludovic Courtès wrote:
>> "pelzflorian (Florian Pelz)" <pelzflorian@pelzflorian.de> skribis:
>> > You mean it takes too long for the TP coordinator to accept new
>> > versions of PO files?
>> 
>> I mean two things: (1) there will not be PO files (see the format of
>> news entries proposed in this issue),
>
> It is easy to write a Guile script that prints a PO file with the news
> file content inserted as the msgid of a single entry, if xgettext does
> not support this already.  Then there would be two PO files, one for
> the news file and one for the package info, that could be concatenated
> by msgcat to a single file.  Guix could ship with this tooling for
> channel authors and an explanation for how to submit the resulting POT
> file to the Translation Project.
>
> I think a single file is easier for translators to manage than two
> files and typically we’d want news files and packages to be translated
> by the same (group of) translator(s).

As you’ve seen, the format I proposed does not rely on PO files and
gettext at all:

  https://issues.guix.gnu.org/issue/37413#5

Are you suggesting that it should rely on PO files?  We can discuss it,
but that would be a significant change with technical challenges
compared to what I propose.  (Also keep in mind that the news mechanism
aims to be available to third-party channels as well.)

Also, why do you mention “the package info”?  There’s no notion of a
package here, so I wonder if there’s a misunderstanding.

>> and (2) I think this is outside
>> the scope of the TP.
>> 
>> Does that clarify?
>> 
>> Ludo’.
>
> I do not see why such translation should be outside the scope of the
> TP?  I can only think of the speed of POT file acceptance and PO file
> translators.

Speed would be a problem (POT files have to be manually accepted by
Benno.)  But also, like I wrote earlier, (1) the TP is geared towards
translating releases of software packages, and (2) and those news
snippets could arrive anytime, not in sync with a “release.”

> I am unsure if setting up Guix’ own translation team would attract
> quicker translators than relying on the TP.

Yeah, having a real translation team is best, but in this case I don’t
see how that could work.

Julien mentioned some time ago that we could run our own Pootle
instance.  Maybe that could be helpful in this case.

Thank you,
Ludo’.
pelzflorian Sept. 17, 2019, 5:41 p.m. UTC | #10
On Tue, Sep 17, 2019 at 05:27:37PM +0200, Ludovic Courtès wrote:
> As you’ve seen, the format I proposed does not rely on PO files and
> gettext at all:
> 
>   https://issues.guix.gnu.org/issue/37413#5
>

I did not read carefully before, but even with this format, a Guile
script could turn it into a POT file with multiple entries and another
Guile script could add translations from a PO file into such a news
file.




> Are you suggesting that it should rely on PO files?


Yes.  The format need not be changed.  PO file conversion scripts can
be added later, but should be a goal.




> We can discuss it,
> but that would be a significant change with technical challenges
> compared to what I propose.
> (Also keep in mind that the news mechanism
> aims to be available to third-party channels as well.)
> 

3rd parties would not need to use the PO file conversion scripts.



> Also, why do you mention “the package info”?  There’s no notion of a
> package here, so I wonder if there’s a misunderstanding.
> 

Channels contain packages whose summaries and descriptions need
translations as well, or do I misunderstand?  Admittedly, it will take
some time until guix-packages is translated, but it should be a goal.




> >> and (2) I think this is outside
> >> the scope of the TP.
> >> 
> >> Does that clarify?
> >> 
> >> Ludo’.
> >
> > I do not see why such translation should be outside the scope of the
> > TP?  I can only think of the speed of POT file acceptance and PO file
> > translators.
> 
> Speed would be a problem (POT files have to be manually accepted by
> Benno.)  But also, like I wrote earlier, (1) the TP is geared towards
> translating releases of software packages, and (2) and those news
> snippets could arrive anytime, not in sync with a “release.”
>
> > I am unsure if setting up Guix’ own translation team would attract
> > quicker translators than relying on the TP.
> 
> Yeah, having a real translation team is best, but in this case I don’t
> see how that could work.
> 
> Julien mentioned some time ago that we could run our own Pootle
> instance.  Maybe that could be helpful in this case.
> 
> Thank you,
> Ludo’.

I do not know what Benno thinks about this.  I would prefer this to be
handled by some Translation Project eventually.

Regards,
Florian
Julien Lepiller Sept. 17, 2019, 6:21 p.m. UTC | #11
Le Tue, 17 Sep 2019 19:41:02 +0200,
"pelzflorian (Florian Pelz)" <pelzflorian@pelzflorian.de> a écrit :

> On Tue, Sep 17, 2019 at 05:27:37PM +0200, Ludovic Courtès wrote:
> > As you’ve seen, the format I proposed does not rely on PO files and
> > gettext at all:
> > 
> >   https://issues.guix.gnu.org/issue/37413#5
> >  
> 
> I did not read carefully before, but even with this format, a Guile
> script could turn it into a POT file with multiple entries and another
> Guile script could add translations from a PO file into such a news
> file.
> 
> 
> 
> 
> > Are you suggesting that it should rely on PO files?  
> 
> 
> Yes.  The format need not be changed.  PO file conversion scripts can
> be added later, but should be a goal.
> 
> 
> 
> 
> > We can discuss it,
> > but that would be a significant change with technical challenges
> > compared to what I propose.
> > (Also keep in mind that the news mechanism
> > aims to be available to third-party channels as well.)
> >   
> 
> 3rd parties would not need to use the PO file conversion scripts.
> 
> 
> 
> > Also, why do you mention “the package info”?  There’s no notion of a
> > package here, so I wonder if there’s a misunderstanding.
> >   
> 
> Channels contain packages whose summaries and descriptions need
> translations as well, or do I misunderstand?  Admittedly, it will take
> some time until guix-packages is translated, but it should be a goal.
> 
> 
> 
> 
> > >> and (2) I think this is outside
> > >> the scope of the TP.
> > >> 
> > >> Does that clarify?
> > >> 
> > >> Ludo’.  
> > >
> > > I do not see why such translation should be outside the scope of
> > > the TP?  I can only think of the speed of POT file acceptance and
> > > PO file translators.  
> > 
> > Speed would be a problem (POT files have to be manually accepted by
> > Benno.)  But also, like I wrote earlier, (1) the TP is geared
> > towards translating releases of software packages, and (2) and
> > those news snippets could arrive anytime, not in sync with a
> > “release.” 
> > > I am unsure if setting up Guix’ own translation team would attract
> > > quicker translators than relying on the TP.  
> > 
> > Yeah, having a real translation team is best, but in this case I
> > don’t see how that could work.
> > 
> > Julien mentioned some time ago that we could run our own Pootle
> > instance.  Maybe that could be helpful in this case.
> > 
> > Thank you,
> > Ludo’.  
> 
> I do not know what Benno thinks about this.  I would prefer this to be
> handled by some Translation Project eventually.

I partially agree: on the one hand, having our own translation platform
only increases fragmentation of the free software translation teams,
and on the other hand, it means we can have a more reactive and
customized translation process. I also agree with Ludo that the TP is
probably not the right place for news translations: even if Benno
accepts the new translations super fast (and Benno is usually fast,
I just keep making mistakes when submitting a new version), it's
going to update all translations (guix, guix-packages and guix-manual),
and translators might focus on these instead of the news.

Using a PO file for news items seem a bit drastic too: a news item is
not supposed to change, except maybe for a typo.

Channel translation is another subject, but they could provide their
own PO file and have them translated separately from the official Guix
translation process. Guix pull would then concatenate po files from
every channel and install that. Does it sound good/feasible?

> 
> Regards,
> Florian
pelzflorian Sept. 17, 2019, 7:44 p.m. UTC | #12
I did not want to slow down these useful patches; the news file and
its translations can remain as proposed by Ludo.  How channel authors
translate their news file is their choice.

I thought we can suggest channel authors to use one gettext PO file
containing translations for both the news file and package information
and submit it to the TP.  We can provide scripts to convert to and
from a news file.  Writing such scripts is not hard because PO files
use a simple format.  Guix itself should use this process too.


On Tue, Sep 17, 2019 at 08:21:18PM +0200, Julien Lepiller wrote:
> […]
> Channel translation is another subject, but they could provide their
> own PO file and have them translated separately from the official Guix
> translation process. Guix pull would then concatenate po files from
> every channel and install that. Does it sound good/feasible?
>

I did not think about this before, but Julien’s idea how to process a
channel’s PO files with translations for package descriptions seems
good.  Just msgcat guix-packages and the other channels’ PO files to
the PO file for guix-packages and then run msgfmt on the result.


On Tue, Sep 17, 2019 at 08:21:18PM +0200, Julien Lepiller wrote:
> I partially agree: on the one hand, having our own translation platform
> only increases fragmentation of the free software translation teams,
> and on the other hand, it means we can have a more reactive and
> customized translation process. I also agree with Ludo that the TP is
> probably not the right place for news translations: even if Benno
> accepts the new translations super fast (and Benno is usually fast,
> I just keep making mistakes when submitting a new version), it's
> going to update all translations (guix, guix-packages and guix-manual),
> and translators might focus on these instead of the news.
>

How about adding a comment to the POT file that we wish for
translators to prioritize and urgently add translations for news?  Is
it really necessary to submit the entire Guix tarball to the TP for
updating news; I thought the tarball is just for convenience of
translators?  I wonder what Benno or others from the TP think.  I
think Guix’ goals align well with those of the TP.



> Using a PO file for news items seem a bit drastic too: a news item is
> not supposed to change, except maybe for a typo.
>

PO files are simple and what the Translation Project’s translators are
used to.  I do not understand how another format could be more
appropriate.

Regards,
Florian
pelzflorian Sept. 17, 2019, 10:02 p.m. UTC | #13
On Tue, Sep 17, 2019 at 09:44:16PM +0200, pelzflorian (Florian Pelz) wrote:
> On Tue, Sep 17, 2019 at 08:21:18PM +0200, Julien Lepiller wrote:
> > […]
> > Channel translation is another subject, but they could provide their
> > own PO file and have them translated separately from the official Guix
> > translation process. Guix pull would then concatenate po files from
> > every channel and install that. Does it sound good/feasible?
> >
> 
> I did not think about this before, but Julien’s idea how to process a
> channel’s PO files with translations for package descriptions seems
> good.  Just msgcat guix-packages and the other channels’ PO files to
> the PO file for guix-packages and then run msgfmt on the result.
> 

No, concatenation via msgcat is not feasible.  It could fail if
translations clash, e.g. when channel A translates "Hello" as
"Bonjour" and channel B translates "Hello" as "Salut".  To handle this
edge case, PO files would need to remain separate and the textdomain
procedure would need to be called by Guix before looking up package
descriptions from this channel, I think.

Regards,
Florian






> 
> On Tue, Sep 17, 2019 at 08:21:18PM +0200, Julien Lepiller wrote:
> > I partially agree: on the one hand, having our own translation platform
> > only increases fragmentation of the free software translation teams,
> > and on the other hand, it means we can have a more reactive and
> > customized translation process. I also agree with Ludo that the TP is
> > probably not the right place for news translations: even if Benno
> > accepts the new translations super fast (and Benno is usually fast,
> > I just keep making mistakes when submitting a new version), it's
> > going to update all translations (guix, guix-packages and guix-manual),
> > and translators might focus on these instead of the news.
> >
> 
> How about adding a comment to the POT file that we wish for
> translators to prioritize and urgently add translations for news?  Is
> it really necessary to submit the entire Guix tarball to the TP for
> updating news; I thought the tarball is just for convenience of
> translators?  I wonder what Benno or others from the TP think.  I
> think Guix’ goals align well with those of the TP.
> 
> 
> 
> > Using a PO file for news items seem a bit drastic too: a news item is
> > not supposed to change, except maybe for a typo.
> >
> 
> PO files are simple and what the Translation Project’s translators are
> used to.  I do not understand how another format could be more
> appropriate.
> 
> Regards,
> Florian
> 
> 
>
Ludovic Courtès Sept. 18, 2019, 12:33 p.m. UTC | #14
"pelzflorian (Florian Pelz)" <pelzflorian@pelzflorian.de> skribis:

>> My feeling is that this use case doesn’t correspond to what the TP is
>> about, but yeah, you could ask them.
>>
>
> I will ask them, but later after Guix website translation is online.

Oops, thanks for the reminder.  :-)

>> Also, “urgently” won’t work because they would have to be manual
>> intervention by at least two people before the info reaches translators:
>> someone on the Guix side who’d submit a tarball (?) to the TP, and the
>
> The channel author would submit the tarball.  No Guix team involved.

If it’s the 'guix channel, then the Guix team is definitely involved,
and typically there’s only a couple of people in the group “entitled” to
do that (so far it’s been Julien or myself.)

Sending a whole tarball seems overkill to me; currently we don’t even
send them tarballs in between releases, even though that’d be useful.

Anyway, let’s see what the TP folks think!

Ludo’.