[bug#34613] gnu: Add prometheus.

* gnu/packages/monitoring.scm
(go-github-com-prometheus-prometheus-cmd-prometheus): New variable.
---
 gnu/packages/monitoring.scm | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)

Hello,

I have packaged prometheus.

There are some problems with it however:

The tarball has a vendor directory, containing sources of third party
go stuff. How do we deal with that?
(Should we package the stuff there, and link the sources into the tree?)

I have noticed a few possible problems:
https://github.com/prometheus/prometheus/blob/master/vendor/github.com/influxdata/influxdb/LICENSE_OF_DEPENDENCIES.md
indicates that glyphicons is possibly non-free, needs further
investigation.
https://github.com/prometheus/prometheus/tree/master/vendor/github.com/shurcooL/httpfs
does not copy the original LICENSE file from upstream, it is MIT
licensed. Should report a bug against prometheus.

Other preliminary license checks does not show further issues.

I would like to know if anyone could help me get this package into
shape, and point me towards a way to use inputs instead of linking the
source into the build tree. It would be nice, if we didn't have to
build these stuffs twice.
I am willing to do the work, but I need some pointer, as I am not into go.

Unbundling would also help to get a more exact license line.

Any help welcome.

Additionally:
1. this package misses a service,
2. only prometheus is built, the repository contains some other tools,
like promtool, those are not provided.

Best regards,
g_bor

Hi Gábor!

Gábor Boskovits <boskovits@gmail.com> skribis:

> There are some problems with it however:
>
> The tarball has a vendor directory, containing sources of third party
> go stuff. How do we deal with that?
> (Should we package the stuff there, and link the sources into the tree?)

I think we generally do that yes.  Pierre, Leo?

> I have noticed a few possible problems:
> https://github.com/prometheus/prometheus/blob/master/vendor/github.com/influxdata/influxdb/LICENSE_OF_DEPENDENCIES.md
> indicates that glyphicons is possibly non-free, needs further
> investigation.

Glyphicons appears to be non-free, indeed.  Could you check what Debian
does, for instance?  It may be that we can just remove or replace it
with something else.

> https://github.com/prometheus/prometheus/tree/master/vendor/github.com/shurcooL/httpfs
> does not copy the original LICENSE file from upstream, it is MIT
> licensed. Should report a bug against prometheus.

OK.

> I would like to know if anyone could help me get this package into
> shape, and point me towards a way to use inputs instead of linking the
> source into the build tree. It would be nice, if we didn't have to
> build these stuffs twice.
> I am willing to do the work, but I need some pointer, as I am not into go.

I’m ignorant about Go, but hopefully Pierre, Leo, or someone else with
experience with Go packaging can provide some guidance!

It’d be great to have Prometheus!

Thank you
Ludo’.

> > (Should we package the stuff there, and link the sources into the tree?)
>
> I think we generally do that yes.  Pierre, Leo?

We used to, but since Go 1.10 there is a bug in the compiler which keeps tracks
of _all_ recursive dependencies in the resulting _static_ binary.  This can
result in a ridiculously large closure size.

As for now, I'd recommend the following:

- If there are many dependencies (check the vendor/ folder), use it, it will be
  easier and produce a package with an ideal closure size.

- If it's only a few dependencies, it's probably worth it to package all of them
  independently.

> > I am willing to do the work, but I need some pointer, as I am not into go.
>
> I’m ignorant about Go, but hopefully Pierre, Leo, or someone else with
> experience with Go packaging can provide some guidance!

For a Go package with unvendored deps, I suggest you look at "demlo", it's
reasonably easy to understand.  For vendored deps, look at go-ipfs.

The Go build system does most things right.  A few pointers:

- Prometheus might use its own build system, e.g. with a Makefile.  Then you'll
  have to read the makefile to know what it's doing.

- In Go, it's possible to include multiple packages in the same repository.  Our
  Go build system does not handle this very well and it's not possible to use
  multiple such packages as dependencies: you need to create a union.  See
  go-golang-org-x-crypto-union.

Let me know if you need more help, I can look into it.

Good luck!

On Tue, Mar 12, 2019 at 08:03:55AM +0100, Pierre Neidhardt wrote:
> We used to, but since Go 1.10 there is a bug in the compiler which keeps tracks
> of _all_ recursive dependencies in the resulting _static_ binary.  This can
> result in a ridiculously large closure size.

I am about to push a fix for this, probably to the staging branch. But I
think you should feel free to use the vendor tree for now and unbundle
things later.

> - In Go, it's possible to include multiple packages in the same repository.  Our
>   Go build system does not handle this very well and it's not possible to use
>   multiple such packages as dependencies: you need to create a union.  See
>   go-golang-org-x-crypto-union.

This becomes uneccesary with the work I am about to push.

Hello,

Pierre Neidhardt <mail@ambrevar.xyz> ezt írta (időpont: 2019. márc.
12., K, 8:04):

>
> - If there are many dependencies (check the vendor/ folder), use it, it will be
>   easier and produce a package with an ideal closure size.

There are quite a lot.

> The Go build system does most things right.  A few pointers:
>
> - Prometheus might use its own build system, e.g. with a Makefile.  Then you'll
>   have to read the makefile to know what it's doing.

Yes, it does, but it turns out it is not actually needed.

>
> - In Go, it's possible to include multiple packages in the same repository.  Our
>   Go build system does not handle this very well and it's not possible to use
>   multiple such packages as dependencies: you need to create a union.  See
>   go-golang-org-x-crypto-union.
>
> Let me know if you need more help, I can look into it.
>
> Good luck!

Thanks!

> Pierre Neidhardt
> https://ambrevar.xyz/


Best regards,
g_bor

Hello,

Ludovic Courtès <ludo@gnu.org> ezt írta (időpont: 2019. márc. 11., H, 23:20):
>
> Hi Gábor!
>
> Gábor Boskovits <boskovits@gmail.com> skribis:

>
> Glyphicons appears to be non-free, indeed.  Could you check what Debian
> does, for instance?  It may be that we can just remove or replace it
> with something else.

I've checked this, this is not actually included in the prometheus
source anywhere,
only copied from the influxdb license file. The only other appearance
of glyphicons
in the code is for the web interface, where it is part of bootstrap,
and as such it is under
apl2.0 there.

I don't know if it is a problem to allow this influxdb module stub to
live in the source,
but I don't think so... (I would not consider this as a case of
promoting use of non-free
software.) Wdyt?

> Thank you
> Ludo’.

Best regards,
g_bor

Hi,

Gábor Boskovits <boskovits@gmail.com> skribis:

> Ludovic Courtès <ludo@gnu.org> ezt írta (időpont: 2019. márc. 11., H, 23:20):
>>
>> Hi Gábor!
>>
>> Gábor Boskovits <boskovits@gmail.com> skribis:
>
>>
>> Glyphicons appears to be non-free, indeed.  Could you check what Debian
>> does, for instance?  It may be that we can just remove or replace it
>> with something else.
>
> I've checked this, this is not actually included in the prometheus
> source anywhere,
> only copied from the influxdb license file. The only other appearance
> of glyphicons
> in the code is for the web interface, where it is part of bootstrap,
> and as such it is under
> apl2.0 there.
>
> I don't know if it is a problem to allow this influxdb module stub to
> live in the source,
> but I don't think so... (I would not consider this as a case of
> promoting use of non-free
> software.) Wdyt?

What’s influxdb?  I’m afraid I don’t understand this sufficiently well
to provide useful comments, but I trust you to make the right decision.
:-)

The thing is, Glyphicons does seem to be non-free, so if it ends up
being included one way or another, we have a problem.  I would have
expected Debian to provide replacement icons or something like that.

Thanks,
Ludo’.

Hello,

2019. márc. 13., Sze 11:06 dátummal Ludovic Courtès <ludo@gnu.org> ezt írta:

> Hi,
>
> Gábor Boskovits <boskovits@gmail.com> skribis:
>
> > Ludovic Courtès <ludo@gnu.org> ezt írta (időpont: 2019. márc. 11., H,
> 23:20):
> >>
> >> Hi Gábor!
> >>
> >> Gábor Boskovits <boskovits@gmail.com> skribis:
> >
> >>
> >> Glyphicons appears to be non-free, indeed.  Could you check what Debian
> >> does, for instance?  It may be that we can just remove or replace it
> >> with something else.
> >
> > I've checked this, this is not actually included in the prometheus
> > source anywhere,
> > only copied from the influxdb license file. The only other appearance
> > of glyphicons
> > in the code is for the web interface, where it is part of bootstrap,
> > and as such it is under
> > apl2.0 there.
> >
> > I don't know if it is a problem to allow this influxdb module stub to
> > live in the source,
> > but I don't think so... (I would not consider this as a case of
> > promoting use of non-free
> > software.) Wdyt?
>
> What’s influxdb?  I’m afraid I don’t understand this sufficiently well
> to provide useful comments, but I trust you to make the right decision.
> :-)
>
Influxdb is an optional dependency, you can configure prometheus to use it
as storage. It is not included in any ways. What is included is a module
that can connect to it.

>
> The thing is, Glyphicons does seem to be non-free, so if it ends up
> being included one way or another, we have a problem.  I would have
> expected Debian to provide replacement icons or something like that.
>
In the case it is included, as part of bootstrap, it is under the apache
license, because of a special agreement, that when it is part of bootstrap
it is dual licensed to this effect. My current evaluation is that it is ok
to include the package as is.

>
> Thanks,
> Ludo’.
>

Hi,

Gábor Boskovits <boskovits@gmail.com> skribis:

> 2019. márc. 13., Sze 11:06 dátummal Ludovic Courtès <ludo@gnu.org> ezt írta:

[...]

>> The thing is, Glyphicons does seem to be non-free, so if it ends up
>> being included one way or another, we have a problem.  I would have
>> expected Debian to provide replacement icons or something like that.
>>
> In the case it is included, as part of bootstrap, it is under the apache
> license, because of a special agreement, that when it is part of bootstrap
> it is dual licensed to this effect. My current evaluation is that it is ok
> to include the package as is.

OK, sounds good.

Thank you for explaining!

Ludo’.

Hi,

What is the status of this old patch#34613:

    <http://issues.guix.gnu.org/issue/34613>


On Fri, 22 Feb 2019 at 12:45, Gábor Boskovits <boskovits@gmail.com> wrote:
> * gnu/packages/monitoring.scm
> (go-github-com-prometheus-prometheus-cmd-prometheus): New variable.
> ---
>  gnu/packages/monitoring.scm | 24 ++++++++++++++++++++++++
>  1 file changed, 24 insertions(+)
>
> diff --git a/gnu/packages/monitoring.scm b/gnu/packages/monitoring.scm
> index 4384406227..0fd6b9e49b 100644
> --- a/gnu/packages/monitoring.scm
> +++ b/gnu/packages/monitoring.scm
> @@ -447,3 +447,27 @@ written in Go with pluggable metric collectors.")
>      (description "This package provides a file system monitor.")
>      (home-page "https://github.com/emcrisostomo/fswatch")
>      (license license:gpl3+)))
> +
> +(define-public go-github-com-prometheus-prometheud-cmd-prometheus
> +  (package
> +    (name "go-github-com-prometheus-prometheus-cmd-prometheus")
> +    (version "2.7.1")
> +    (source (origin
> +              (method git-fetch)
> +              (uri (git-reference
> +                    (url "https://github.com/prometheus/prometheus.git")
> +                    (commit (string-append "v" version))))
> +              (file-name (git-file-name name version))
> +              (sha256
> +               (base32
> +                "08nd88m162bw5612cvw5fl028l2n9jy1v4w2a8cyd0dj4lxs5d98"))))
> +    (build-system go-build-system)
> +    (arguments
> +     '(#:import-path "github.com/prometheus/prometheus/cmd/prometheus"
> +       #:unpack-path "github.com/prometheus/prometheus"))
> +    (synopsis "Monitoring system & time series database")
> +    (description "Prometheus is an open-source monitoring system with a
> +dimensional data model, flexible query language, efficient time series database
> +and modern alerting approach.")
> +    (home-page "https://github.com/prometheus/node_exporter")
> +    (license license:asl2.0)))

Because the Go ecosystem changed a bit since the submission, do we merge
this patch?  Especially in the light of:

        There are some problems with it however:

        The tarball has a vendor directory, containing sources of third party
        go stuff. How do we deal with that?
        (Should we package the stuff there, and link the sources into the tree?)


        <https://issues.guix.gnu.org/issue/34613#1>

Well, I propose to drop this submission since the last interaction is
from 3 years, 2 weeks, 6 days ago and then start a fresh one.  WDYT?


Cheers,
simon

On Thu, 07 Apr 2022 at 14:29, zimoun <zimon.toutoune@gmail.com> wrote:

> What is the status of this old patch#34613:
>
>     <http://issues.guix.gnu.org/issue/34613>

> Because the Go ecosystem changed a bit since the submission, do we merge
> this patch?  Especially in the light of:
>
>         There are some problems with it however:
>
>         The tarball has a vendor directory, containing sources of third party
>         go stuff. How do we deal with that?
>         (Should we package the stuff there, and link the sources into the tree?)
>
>
>         <https://issues.guix.gnu.org/issue/34613#1>
>
> Well, I propose to drop this submission since the last interaction is
> from 3 years, 2 weeks, 6 days ago and then start a fresh one.  WDYT?

The patch is marked moreinfo since 23 Jun 2022  (15 weeks, 2 days, 6
hours ago) and dormant since years.

If no objection, I will close it soon.


Cheers,
simon

Hi,

On Sat, 08 Oct 2022 at 16:42, zimoun <zimon.toutoune@gmail.com> wrote:
> On Thu, 07 Apr 2022 at 14:29, zimoun <zimon.toutoune@gmail.com> wrote:
>
>> What is the status of this old patch#34613:
>>
>>     <http://issues.guix.gnu.org/issue/34613>
>
>> Because the Go ecosystem changed a bit since the submission, do we merge
>> this patch?  Especially in the light of:
>>
>>         There are some problems with it however:
>>
>>         The tarball has a vendor directory, containing sources of third party
>>         go stuff. How do we deal with that?
>>         (Should we package the stuff there, and link the sources into the tree?)
>>
>>
>>         <https://issues.guix.gnu.org/issue/34613#1>
>>
>> Well, I propose to drop this submission since the last interaction is
>> from 3 years, 2 weeks, 6 days ago and then start a fresh one.  WDYT?
>
> The patch is marked moreinfo since 23 Jun 2022  (15 weeks, 2 days, 6
> hours ago) and dormant since years.
>
> If no objection, I will close it soon.

Gábor sent me offlist that they agree to close because the problematic
part is non-free ui components.  And since no one seems aware of a fix,
closing.

Feel free to reopen if I am missing a point or if someone is currently
revisiting the issue.

Cheers,
simon