mbox series

[bug#49221,00/13] Add go-1.16 and build with gccgo

Message ID cover.1624602942.git.iskarian@mgsn.dev
Headers show
Series Add go-1.16 and build with gccgo | expand

Message

Sarah Morgensen June 25, 2021, 6:52 a.m. UTC
Hello Guix,

This patch adds go 1.16, but does *not* make it the default for other packages
which use go (as suggested in [2]). There have been a lot of changes between
1.14 and 1.16, and while I have fixed many issues, I am sure continued testing
will uncover many more.

This patch is based on [0], which adds gccgo-10.

With this patch, go automatically bootstraps from gccgo[0][1] on platforms not
supported by go 1.4, but you can do so on any platform with:

  guix build go --with-input=go@1.4-bootstrap-20171003=gccgo@10

Additionally, to build packages with 1.16:

  guix build <package> --with-input=go@1.14=go@1.16

I would appreciate testing and feedback on the new bootstrap process,
especially from powerpc64le users, as I have only been able to test on x86-64.
(Note that it took about two hours for gccgo to build and 30 minutes for gccgo
to build go.)

Changes from 1.14:

  * Bootstrap with gccgo on platforms which do not support go 1.4 (see above)
  * Use GO_LDSO to set the interpreter rather than patching files
  * Fix typo in cgoldflags patch ("-rpath=" > "-Wl,-rpath=")
  * Fix and re-enable some cgo/gccgo tests
  * Remove references to perl (used for tests) to reduce closure size by 10%
  * Set GOCACHE so go doesn't complain that it can't access it $HOME
  * Break out tests into "check" phase so builders have more control

I have successfully built and tested most go packages on x86-64, including
restic, syncthing, chezmoi, and exercism.

However... docker does not pass tests. It should probably be updated to 20.10,
but it is a very complicated package and I do not have the skill to do so. I
attempted to backport some of the fixes, but that was also beyond my abilities.

Perhaps someone else will rise to the challenge? :)

Sarah

[0] <https://issues.guix.gnu.org/49019> [PATCH 0/1] gnu: Add gccgo-10.
[1] <https://issues.guix.gnu.org/28226> Build go with gccgo
[2] <https://issues.guix.gnu.org/45161> [PATCH] Added Go 1.16.2

Sarah Morgensen (13):
  gnu: go-github-com-puerkitobio-goquery: Update to 1.7.0.
  gnu: go-go-uber-org-atomic: Update to 1.8.0.
  gnu: go-github-com-sergi-go-diff: Update to 1.2.0.
  gnu: go-github-com-pelletier-go-toml: Update to 1.9.3.
  gnu: go-github-com-masterminds-goutils: Update to 1.1.1.
  gnu: go-github-com-magiconair-properties: Update to 1.8.5.
  gnu: go-github-com-dlclark-regexp2: Update to 1.4.0.
  gnu: earlyoom: Patch tests for go-1.16.
  gnu: go-github-com-bmatcuk-doublestar: Remove test incompatible with
    go-1.16.
  gnu: restic: Patch tests for go-1.16.
  gnu: go-gopkg-in-check-v1: Update to latest.
  gnu: go-etcd-io-bbolt: Update to 1.3.6.
  gnu: Add go-1.16.

 gnu/local.mk                                  |   4 +
 gnu/packages/backup.scm                       |   5 +-
 gnu/packages/golang.scm                       | 243 +++++++++++++++---
 gnu/packages/linux.scm                        |   4 +-
 .../earlyoom-1.6.2-disable-go-module.patch    |  17 ++
 .../patches/go-fix-script-tests.patch         |  18 ++
 ...m-bmatcuk-doublestar-1.3-remove-test.patch |  15 ++
 .../restic-0.9.6-fix-tests-for-go1.15.patch   |  51 ++++
 8 files changed, 314 insertions(+), 43 deletions(-)
 create mode 100644 gnu/packages/patches/earlyoom-1.6.2-disable-go-module.patch
 create mode 100644 gnu/packages/patches/go-fix-script-tests.patch
 create mode 100644 gnu/packages/patches/go-github-com-bmatcuk-doublestar-1.3-remove-test.patch
 create mode 100644 gnu/packages/patches/restic-0.9.6-fix-tests-for-go1.15.patch


base-commit: c7804cd97b28ef012acc20c1d861904e9592382b
prerequisite-patch-id: edb56de17ac3cce58402c83a9671b00381dfee76

Comments

Efraim Flashner June 28, 2021, 7:22 p.m. UTC | #1
On Thu, Jun 24, 2021 at 11:52:55PM -0700, Sarah Morgensen via Guix-patches via wrote:
> Hello Guix,
> 
> This patch adds go 1.16, but does *not* make it the default for other packages
> which use go (as suggested in [2]). There have been a lot of changes between
> 1.14 and 1.16, and while I have fixed many issues, I am sure continued testing
> will uncover many more.
> 
> This patch is based on [0], which adds gccgo-10.

I've applied the patch, so it should now be available in the main repo :)

> With this patch, go automatically bootstraps from gccgo[0][1] on platforms not
> supported by go 1.4, but you can do so on any platform with:
> 
>   guix build go --with-input=go@1.4-bootstrap-20171003=gccgo@10
> 
> Additionally, to build packages with 1.16:
> 
>   guix build <package> --with-input=go@1.14=go@1.16
> 
> I would appreciate testing and feedback on the new bootstrap process,
> especially from powerpc64le users, as I have only been able to test on x86-64.
> (Note that it took about two hours for gccgo to build and 30 minutes for gccgo
> to build go.)

I was able to build go@1.16 as-is and with the conditional switched, so
it used gccgo to bootstrap go@1.16. Additionally on my x86_64 machine
using emulation I was able to build go@1.16 for powerpc64le. On the
powerpc64le box I have access to I was able to build gccgo and I'm still
building out to go@1.16.

> Changes from 1.14:
> 
>   * Bootstrap with gccgo on platforms which do not support go 1.4 (see above)
>   * Use GO_LDSO to set the interpreter rather than patching files
>   * Fix typo in cgoldflags patch ("-rpath=" > "-Wl,-rpath=")
>   * Fix and re-enable some cgo/gccgo tests
>   * Remove references to perl (used for tests) to reduce closure size by 10%
>   * Set GOCACHE so go doesn't complain that it can't access it $HOME
>   * Break out tests into "check" phase so builders have more control
> 
> I have successfully built and tested most go packages on x86-64, including
> restic, syncthing, chezmoi, and exercism.
> 
> However... docker does not pass tests. It should probably be updated to 20.10,
> but it is a very complicated package and I do not have the skill to do so. I
> attempted to backport some of the fixes, but that was also beyond my abilities.
> 
> Perhaps someone else will rise to the challenge? :)

My only concern about the patches is that I've been burned before about
pushing updates to go packages, where some seem to have insufficient
testing and then they'll build and fail. So in the meantime I've pushed
the go@1.16 patch.

> Sarah
> 
> [0] <https://issues.guix.gnu.org/49019> [PATCH 0/1] gnu: Add gccgo-10.
> [1] <https://issues.guix.gnu.org/28226> Build go with gccgo
> [2] <https://issues.guix.gnu.org/45161> [PATCH] Added Go 1.16.2
> 
> Sarah Morgensen (13):
>   gnu: go-github-com-puerkitobio-goquery: Update to 1.7.0.
>   gnu: go-go-uber-org-atomic: Update to 1.8.0.
>   gnu: go-github-com-sergi-go-diff: Update to 1.2.0.
>   gnu: go-github-com-pelletier-go-toml: Update to 1.9.3.
>   gnu: go-github-com-masterminds-goutils: Update to 1.1.1.
>   gnu: go-github-com-magiconair-properties: Update to 1.8.5.
>   gnu: go-github-com-dlclark-regexp2: Update to 1.4.0.
>   gnu: earlyoom: Patch tests for go-1.16.
>   gnu: go-github-com-bmatcuk-doublestar: Remove test incompatible with
>     go-1.16.
>   gnu: restic: Patch tests for go-1.16.
>   gnu: go-gopkg-in-check-v1: Update to latest.
>   gnu: go-etcd-io-bbolt: Update to 1.3.6.
>   gnu: Add go-1.16.
> 
>  gnu/local.mk                                  |   4 +
>  gnu/packages/backup.scm                       |   5 +-
>  gnu/packages/golang.scm                       | 243 +++++++++++++++---
>  gnu/packages/linux.scm                        |   4 +-
>  .../earlyoom-1.6.2-disable-go-module.patch    |  17 ++
>  .../patches/go-fix-script-tests.patch         |  18 ++
>  ...m-bmatcuk-doublestar-1.3-remove-test.patch |  15 ++
>  .../restic-0.9.6-fix-tests-for-go1.15.patch   |  51 ++++
>  8 files changed, 314 insertions(+), 43 deletions(-)
>  create mode 100644 gnu/packages/patches/earlyoom-1.6.2-disable-go-module.patch
>  create mode 100644 gnu/packages/patches/go-fix-script-tests.patch
>  create mode 100644 gnu/packages/patches/go-github-com-bmatcuk-doublestar-1.3-remove-test.patch
>  create mode 100644 gnu/packages/patches/restic-0.9.6-fix-tests-for-go1.15.patch
> 
> 
> base-commit: c7804cd97b28ef012acc20c1d861904e9592382b
> prerequisite-patch-id: edb56de17ac3cce58402c83a9671b00381dfee76
> -- 
> 2.31.1
> 
> 
> 
>
Sarah Morgensen July 1, 2021, 3:24 a.m. UTC | #2
Hello,

Thanks for the push!

Efraim Flashner <efraim@flashner.co.il> writes:

> On Thu, Jun 24, 2021 at 11:52:55PM -0700, Sarah Morgensen via Guix-patches via wrote:
> 
>> I would appreciate testing and feedback on the new bootstrap process,
>> especially from powerpc64le users, as I have only been able to test on x86-64.
>> (Note that it took about two hours for gccgo to build and 30 minutes for gccgo
>> to build go.)
>
> I was able to build go@1.16 as-is and with the conditional switched, so
> it used gccgo to bootstrap go@1.16. Additionally on my x86_64 machine
> using emulation I was able to build go@1.16 for powerpc64le. On the
> powerpc64le box I have access to I was able to build gccgo and I'm still
> building out to go@1.16.

Excellent! Thanks for testing. Fingers crossed :)

> My only concern about the patches is that I've been burned before about
> pushing updates to go packages, where some seem to have insufficient
> testing and then they'll build and fail. So in the meantime I've pushed
> the go@1.16 patch.

Fair enough. We'll have to wait & see if any issues surface. On that
note, I've found the tests in etcd-io-bbolt and go-gopkg-in-check-v1 to
be performance-based, and they would sometimes fail on my VM. Updating
them did help, but did not completely eliminate the issue.

--
Sarah
Sarah Morgensen July 25, 2021, 2:17 a.m. UTC | #3
Hi Efraim,

Just following up on this.

Efraim Flashner <efraim@flashner.co.il> writes:

> On Thu, Jun 24, 2021 at 11:52:55PM -0700, Sarah Morgensen via Guix-patches via wrote:
>> Hello Guix,
>> 
>> This patch adds go 1.16, but does *not* make it the default for other packages
>> which use go (as suggested in [2]). There have been a lot of changes between
>> 1.14 and 1.16, and while I have fixed many issues, I am sure continued testing
>> will uncover many more.
>> 
>> This patch is based on [0], which adds gccgo-10.
>
> I've applied the patch, so it should now be available in the main repo :)

Should the following related issues be closed now?

[0] <https://issues.guix.gnu.org/28226> Build go with gccgo
[1] <https://issues.guix.gnu.org/46380> gnu: gccgo: Add version 10.
[2] <https://issues.guix.gnu.org/47434> Added Go 1.16.2
[3] <https://issues.guix.gnu.org/45161> Update go?

[...]

> My only concern about the patches is that I've been burned before about
> pushing updates to go packages, where some seem to have insufficient
> testing and then they'll build and fail. So in the meantime I've pushed
> the go@1.16 patch.

Is there something specific to be done to sufficiently test them?  I'm
not sure what else to do other than manually testing dependents.

Thanks,
Sarah
Efraim Flashner Aug. 5, 2021, 2:03 p.m. UTC | #4
On Sat, Jul 24, 2021 at 07:17:28PM -0700, Sarah Morgensen wrote:
> Hi Efraim,
> 
> Just following up on this.

Sorry for taking so long to get back to you.

> Efraim Flashner <efraim@flashner.co.il> writes:
> 
> > On Thu, Jun 24, 2021 at 11:52:55PM -0700, Sarah Morgensen via Guix-patches via wrote:
> >> Hello Guix,
> >> 
> >> This patch adds go 1.16, but does *not* make it the default for other packages
> >> which use go (as suggested in [2]). There have been a lot of changes between
> >> 1.14 and 1.16, and while I have fixed many issues, I am sure continued testing
> >> will uncover many more.
> >> 
> >> This patch is based on [0], which adds gccgo-10.
> >
> > I've applied the patch, so it should now be available in the main repo :)
> 
> Should the following related issues be closed now?
> 
> [0] <https://issues.guix.gnu.org/28226> Build go with gccgo
> [1] <https://issues.guix.gnu.org/46380> gnu: gccgo: Add version 10.
> [2] <https://issues.guix.gnu.org/47434> Added Go 1.16.2
> [3] <https://issues.guix.gnu.org/45161> Update go?

Looks like they should be safe to close.

> > My only concern about the patches is that I've been burned before about
> > pushing updates to go packages, where some seem to have insufficient
> > testing and then they'll build and fail. So in the meantime I've pushed
> > the go@1.16 patch.
> 
> Is there something specific to be done to sufficiently test them?  I'm
> not sure what else to do other than manually testing dependents.

I thought more about it and I went ahead and applied most of them. I
wasn't able to apply the go-github-com-bmatcuk-doublestar one or the
restic patch. Can you resend those?

> Thanks,
> Sarah

Thank you for doing all the work.