[bug#53144,01/13] doc: Give some tips on Minetest packaging.

* doc/contributing.texi (Minetest Packages): New section.
* doc/guix.texi: Copyright update.
---
 doc/contributing.texi | 42 ++++++++++++++++++++++++++++++++++++++++++
 doc/guix.texi         |  2 +-
 2 files changed, 43 insertions(+), 1 deletion(-)


base-commit: 9fd4f4b09cc0495d6b1418f171ff738a1086cc00

Am Sonntag, dem 09.01.2022 um 19:10 +0000 schrieb Maxime Devos:
> * doc/contributing.texi (Minetest Packages): New section.
> * doc/guix.texi: Copyright update.
> ---
>  doc/contributing.texi | 42
> ++++++++++++++++++++++++++++++++++++++++++
>  doc/guix.texi         |  2 +-
>  2 files changed, 43 insertions(+), 1 deletion(-)
> 
> diff --git a/doc/contributing.texi b/doc/contributing.texi
> index 72f5ce1e0e..5b91fc7867 100644
> --- a/doc/contributing.texi
> +++ b/doc/contributing.texi
> @@ -394,6 +394,7 @@ needed is to review and apply the patch.
>  * Synopses and Descriptions::   Helping users find the right
> package.
>  * Snippets versus Phases::      Whether to use a snippet, or a build
> phase.
>  * Emacs Packages::              Your Elisp fix.
> +* Minetest Packages::           Building blocks.
>  * Python Modules::              A touch of British comedy.
>  * Perl Modules::                Little pearls.
>  * Java Packages::               Coffee break.
> @@ -703,6 +704,47 @@ When encountering problems, it is wise to check
> for the presence of the
>  file, and whether any dependencies and their versions listed therein
> are
>  satisfied.
>  
> +@node Minetest Packages
> +@subsection Minetest Packages
> +@cindex minetest, packaging
> +
> +A Minetest mod @code{foo} is named @code{minetest-foo} -- the author
> +name from ContentDB is not included, unless required to resolve a
> name
> +collision.
> +
> +Sometimes, it might be unclear what the version of a Minetest mod
> is.
> +For example, ContentDB and the importer reports 2020-01-01, but
> +according to the forums the version is 2.1.  Usually, in these cases
> the
> +version on ContentDB is the newest and intended for distribution. As
> +such, you can use the version from ContentDB without any special
> +comments.
We might want to quote an authoritative resource on that, perhaps in
the footnote?

> +@c Currently it's always checked out from git, but in principle
> +@c tarballs could be used.
> +
> +Even though the source code is often checked out from version
> control,
> +it is not necessary to use @code{git-version} or @code{hg-version}:
> the
> +releases on ContentDB are formal releases; in fact they are
> upstream's
> +official source of Minetest packages and they are not mutated in-
> place.
> +
> +@c Example (zip): mods by TenPlus1
> +@c Example (git): basic_materials, ethereal
> +While ContentDB provides the source code of packages in zip form, it
> is
> +recommended not to use these, because users can and do delete old
> +versions.  Likewise, sometimes the maintainer initially did tag
> versions
> +but later stops doing so, breaking @command{guix refresh -u}.  As
> such,
> +it is recommended not to use git tags in @code{origin} records and
> +instead refer to the commit directly.
This combination of version+commit is something I'd generally
discourage (my reasoning for doing so already explained elsewhere), so
to me it might make sense to still explicitly point attention to it. 
Perhaps setting a package-property such as (upstream . contentdb),
which would also make it clear why we don't e.g. want the latest-git
updater to apply?


Otherwise LGTM.

Liliana Marie Prikler schreef op zo 09-01-2022 om 22:15 [+0100]:
> [...]
> > +
> > +Sometimes, it might be unclear what the version of a Minetest mod
> > is.
> > +For example, ContentDB and the importer reports 2020-01-01, but
> > +according to the forums the version is 2.1.  Usually, in these cases
> > the
> > +version on ContentDB is the newest and intended for distribution. As
> > +such, you can use the version from ContentDB without any special
> > +comments.
> We might want to quote an authoritative resource on that, perhaps in
> the footnote?

Yes, quoting sources seems good.

About newest version: I don't have an authoritive source at hand; this
is more speaking of my own experience of what seems to be the case.  I
could compute some statistics w.r.t. how often is the latest version on
ContentDB >= latest tagged version in the repository (where > is the
is-(indirect)-child-commit-of relation, as the versions strings aren't
always directly comparable -- see discussion about CalVer vs. SemVer
and ContentDB sometimes using CalVer and sometimes SemVer).

This also seems a consequence of ContentDB being the official source
of mods and used by Minetest's built-in installer.

About ‘intended for distribution’: I'll look for a forum post or wiki
page targetting new mod authors that want to distribute their mod.

What do you think of me gathering this information and (if not
falsified) presenting it to upstream in some public location,
asking if it seems about right, and (presuming they answer ‘yes’),
quoting their response?

> > +@c Currently it's always checked out from git, but in principle
> > +@c tarballs could be used.
> > +
> > +Even though the source code is often checked out from version
> > control,
> > +it is not necessary to use @code{git-version} or @code{hg-version}:
> > the
> > +releases on ContentDB are formal releases; in fact they are
> > upstream's
> > +official source of Minetest packages and they are not mutated in-
> > place.
> > +
> > +@c Example (zip): mods by TenPlus1
> > +@c Example (git): basic_materials, ethereal
> > +While ContentDB provides the source code of packages in zip form, it
> > is
> > +recommended not to use these, because users can and do delete old
> > +versions.  Likewise, sometimes the maintainer initially did tag
> > versions
> > +but later stops doing so, breaking @command{guix refresh -u}.  As
> > such,
> > +it is recommended not to use git tags in @code{origin} records and
> > +instead refer to the commit directly.
> This combination of version+commit is something I'd generally
> discourage (my reasoning for doing so already explained elsewhere), so
> to me it might make sense to still explicitly point attention to it. 

That discussion I had in mind while writing this documentation.

I happen to disagree though, being mostly neutral about
version+commit with a slight preference towards including the commit
itself in the commit field of 'git-reference' (and not a tag pointing
to the commit), because it is more explicit and it fits slightly
better in some nebulous plans for decentralising source code
downloading/storage.  There are ‘tricking peer review’ issues
with this though, hence neutral.

(See <https://notabug.org/maximed/rehash> if interested)

Hence, adding something like ‘This is specific to Minetest packages,
for other packages it is advised to use git tags, see [...]’ doesn't
make much sense to me, though I understand why it would make sense
to you.  Furthermore, the reference [...] currently doesn't exist, so I
cannot point the reader to an explanation why for other packages we
want git tags and not commits.

If there was consensus (one way or the other) and some section in the
manual explaining what should usually be used (tags or commits)
(preferably also explaining the reasons and not simply stating things),
I could link to it though.  Or if there's no consensus, and the section
said something like

‘There are two options: tags or commits.  Currently there's no
consensus about what's best.  Here are some pros and cons of each: ...
Due to a lack of consensus, the patch submitter can make the choice.
When in doubt, throw a dice.’ (*) (to be reworded!),

then I could work with that as well.

Perhaps I could write out (*) a bit more, as a separate documentation
patch?  I'd have to ask on guix-devel@ if I'm
understanding/misunderstanding the lack of consensus, and see if
someone has already summarised things, maybe see if Nix people
have thought about this, etc.

> Perhaps setting a package-property such as (upstream . contentdb),
> which would also make it clear why we don't e.g. want the latest-git
> updater to apply?

More generally, being able to explicitely choose the updater
(minetest/github/elpa/...) seems useful!

However, in the context of this documentation section and the changes
to the ContentDB importer, I don't think latest-git is relevant here
(except for the infrequent edge cases like minetest-throwing-arrows and
emacs-next): we almost never want the latest-git updater to apply
(because formal releases etc.).  And when we do want it to apply, we
set

  (with-latest-git-commit . #true/"refs/heads/master"/...)

otherwise latest-git doesn't run.  Well, we don't do that yet except
for minetest-throwing-arrows, but that's the idea.

To summarise, I don't see the value that adding (upstream . contentdb)
would bring, it seems to me that it would only make the package
definitions longer.  Would this package property be pure documentation,
or would it be interpreted by the updater code in some way?


> Otherwise LGTM.

I'm not complaining, but from
<https://lists.gnu.org/archive/html/guix-devel/2021-12/msg00259.html>
and resulting discussion I was kind of expecting that you'd
want me to do

(let ((commit ...)
      (revision ...))
  (package ...
    (version (git-version "contentdb-version" revision commit)) ...)),

though you did write ‘I'd want a comment like the ones I find in
minetest.scm to [...]’ I suppose, and this new documentation is
explaining the reasons for using commits (raw commits in your
terminology) in some detail and for all Minetest packages at once.

Unrelated: git mirrors seem a cool idea!

Greetings,
Maxime

Hi,

Am Montag, dem 10.01.2022 um 00:29 +0100 schrieb Maxime Devos:
> Liliana Marie Prikler schreef op zo 09-01-2022 om 22:15 [+0100]:
> > [...]
> > > +
> > > +Sometimes, it might be unclear what the version of a Minetest
> > > mod
> > > is.
> > > +For example, ContentDB and the importer reports 2020-01-01, but
> > > +according to the forums the version is 2.1.  Usually, in these
> > > cases
> > > the
> > > +version on ContentDB is the newest and intended for
> > > distribution. As
> > > +such, you can use the version from ContentDB without any special
> > > +comments.
> > We might want to quote an authoritative resource on that, perhaps
> > in the footnote?
> 
> Yes, quoting sources seems good.
> 
> About newest version: I don't have an authoritive source at hand;
> this is more speaking of my own experience of what seems to be the
> case.  I could compute some statistics w.r.t. how often is the latest
> version on ContentDB >= latest tagged version in the repository
> (where > is the is-(indirect)-child-commit-of relation, as the
> versions strings aren't always directly comparable -- see discussion
> about CalVer vs. SemVer and ContentDB sometimes using CalVer and
> sometimes SemVer).
> 
> This also seems a consequence of ContentDB being the official source
> of mods and used by Minetest's built-in installer.
> 
> About ‘intended for distribution’: I'll look for a forum post or wiki
> page targetting new mod authors that want to distribute their mod.
> 
> What do you think of me gathering this information and (if not
> falsified) presenting it to upstream in some public location,
> asking if it seems about right, and (presuming they answer ‘yes’),
> quoting their response?
In my opinion the fact that Minetest's built-in installer serves as
this ‘M-x package-install’-like gateway would already be authoritative
enough, but it still deserves being spelled out in a footnote that this
is the basis of our assumptions :)

Communicating with upstreams is also important, but a poll or similar
would be significantly harder to quote (perhaps we could insert it in a
comment), and that's not even talking about what we'd have to do to
account for input selection.

> > 
> I happen to disagree though, being mostly neutral about
> version+commit with a slight preference towards including the commit
> itself in the commit field of 'git-reference' (and not a tag pointing
> to the commit), because it is more explicit and it fits slightly
> better in some nebulous plans for decentralising source code
> downloading/storage.  There are ‘tricking peer review’ issues
> with this though, hence neutral.
> 
> (See <https://notabug.org/maximed/rehash> if interested)
> 
> Hence, adding something like ‘This is specific to Minetest packages,
> for other packages it is advised to use git tags, see [...]’ doesn't
> make much sense to me, though I understand why it would make sense
> to you.  Furthermore, the reference [...] currently doesn't exist, so
> I cannot point the reader to an explanation why for other packages we
> want git tags and not commits.
Again note that you're not only competing against version+tag, but also
against git-version+hash, with git-version embedding the hash.  Among
Guix packages, Minetest *is* rather unique in having a separate source
(ContentDB) pointing to the bespoke commits that serve as release
basis.

> If there was consensus (one way or the other) and some section in the
> manual explaining what should usually be used (tags or commits)
> (preferably also explaining the reasons and not simply stating
> things), I could link to it though.  Or if there's no consensus, and
> the section said something like
> 
> ‘There are two options: tags or commits.  Currently there's no
> consensus about what's best.  Here are some pros and cons of each:
> ...
> Due to a lack of consensus, the patch submitter can make the choice.
> When in doubt, throw a dice.’ (*) (to be reworded!),
> 
> then I could work with that as well.
> 
> Perhaps I could write out (*) a bit more, as a separate documentation
> patch?  I'd have to ask on guix-devel@ if I'm
> understanding/misunderstanding the lack of consensus, and see if
> someone has already summarised things, maybe see if Nix people
> have thought about this, etc.
One question whose answer is not quite clear to me is whether you want
to generally follow the commit vs. tag guidelines of the rest of Guix
(and therefore seek to formalize them in a similar way, though perhaps
with different reasoning from what I do), or if you want ContentDB
packages to follow their own style largely independent from the rest,
at least for the time being.

> > Perhaps setting a package-property such as (upstream . contentdb),
> > which would also make it clear why we don't e.g. want the latest-
> > git updater to apply?
> 
> More generally, being able to explicitely choose the updater
> (minetest/github/elpa/...) seems useful!
> 
> However, in the context of this documentation section and the changes
> to the ContentDB importer, I don't think latest-git is relevant here
> (except for the infrequent edge cases like minetest-throwing-arrows
> and emacs-next): we almost never want the latest-git updater to apply
> (because formal releases etc.).  And when we do want it to apply, we
> set
> 
>   (with-latest-git-commit . #true/"refs/heads/master"/...)
> 
> otherwise latest-git doesn't run.  Well, we don't do that yet except
> for minetest-throwing-arrows, but that's the idea.
I think the generic git updater would still be relevant here, hence why
I'm asking.  Yes, latest-git is restricted in that it needs opting in,
but the property combination

  ((upstream . latest-git) (latest-git-head "master"))

would also work, for example.

> To summarise, I don't see the value that adding (upstream .
> contentdb) would bring, it seems to me that it would only make the
> package definitions longer.  Would this package property be pure
> documentation, or would it be interpreted by the updater code in some
> way?
Ideally it'd serve as a better way of making the minetest-updater say
"Now this looks like a job for me" than relying on the minetest-
package? procedure.  Alternatively, we could make it so that the
updater has to have a contentdb URL as homepage.  You are correct in
that there's no immediate benefit, but given that we would be
shortening a four-line comment to a single-line property while
essentially keeping all the information is still a win in my opinion.

> > Otherwise LGTM.
> 
> I'm not complaining, but from
> <https://lists.gnu.org/archive/html/guix-devel/2021-12/msg00259.html>
> and resulting discussion I was kind of expecting that you'd
> want me to do
> 
> (let ((commit ...)
>       (revision ...))
>   (package ...
>     (version (git-version "contentdb-version" revision commit))
> ...)),
> 
> though you did write ‘I'd want a comment like the ones I find in
> minetest.scm to [...]’ I suppose, and this new documentation is
> explaining the reasons for using commits (raw commits in your
> terminology) in some detail and for all Minetest packages at once.
It does, but in a different location that requires a little bit of work
to do the mapping.  I think a comment line 
  ;; version from contentdb
or similar would still be appropriate even with said documentation
changes.  Alternatively, as pointed out above, we could have home-page
point to ContentDB, which IIRC should also present that information to
the user.

I think "Can we use ContentDB as a home-page for minetest-blub?" also
serves as a good leading question on what source is the most
authoritative one.  If the ContentDB page looks ill-maintained, but the
forum is active, the forum should probably be preferred (while
"generally, use ContentDB" still applies otherwise).

The distinction between "raw commits" and just commits is also an
important one that I want to keep.  ContentDB provides both release
titles and commits in a usable format, so that's why raw commits are in
this context permissible in my opinion.  There's no ContentDB for
guile-aiscm or node-something-that-loses-its-git-tag.

Cheers

Hi!

Maxime Devos <maximedevos@telenet.be> skribis:

> * doc/contributing.texi (Minetest Packages): New section.
> * doc/guix.texi: Copyright update.

[...]

> +@node Minetest Packages
> +@subsection Minetest Packages
> +@cindex minetest, packaging
> +
> +A Minetest mod @code{foo} is named @code{minetest-foo} -- the author
> +name from ContentDB is not included, unless required to resolve a name
> +collision.

The section looks great to me, but I’d suggest starting it with a couple
of sentences giving context, like “The @uref{…, Minetest} game can be
extended with so-called @dfn{mods}, which are …”

It might also be worth defining ContentDB or giving a @uref.

Last, parenthetical phrases should be enclosed in em dashes, represented
by three hyphens in Texinfo (two hyphens is for en dashes, used to
represent ranges.)

Ludo’.