mbox series

[bug#78404,0/2] Go: Module aware build system

Message ID 20250513094023.348947-1-j@lambda.is
Headers
Series Go: Module aware build system |

Message

Jørgen Kvalsvik May 13, 2025, 9:40 a.m. UTC 
  Hi,

This is a new build system for go, based on go modules. I'm submitting
this now to get an issue number, collect some feedback, and iron out the
last few wrinkles, but we have used an internal version of it for a
while and it works well for us. I have tweaked it a bit in preparation
for this patch series, and I do expect it to take a couple of
iterations.

The new build system can use packages built with go-build-system as
inputs, which I think is a prerequisite. There is already a large
collection of go libraries in guix, and we want to use that work
well. That being said, this build system should graudually replace the
go-build-system. Because they are (largely) compatible as inputs,
packages can be migrated gradually. The library or program being built does
not have to be "module aware" (have a go.mod file) to use this build system, a
new go.mod will be created unconditionally.

It addresses most of the shortcomings of the current go build
system. Here's the list from go.scm and how this system is different:

* Avoid copying dependencies into the build environment and / or avoid
  using a tmpdir when creating the inputs union.
    We still copy build dependencies into the build dir, so that go
    build can "fetch" dependencies as it likes. There is a path where
    this can be avoided by using replace directives in go.mod, but this
    is a significant complication and requires build inputs to be proper
    modules, which is not guaranteed.

* Use Go modules
    Yes.

* Re-use compiled packages
    Yes. We install the go build cache under $out/var, and seed
    downstream builds with this cache.

* Avoid the go-inputs hack
    Sort-of? We look at package name, but also the "shape" of the
    package, namely the presence of the src/ directory. This seems to
    work ok.

* Remove module packages, only offering the full Git repos? This is more
  idiomatic, I think, because Go downloads Git repos, not modules.
    There is the go-mod-fetch which downloads modules (as zips). The
    build system does not particularly care about the source, and works
    well both with git clone, hg clone, and url fetch.

* Figurie out how to passthrough --verbosity option to "build" and
  "check"
    Not addressed yet.

* Implement test-backend option, which would be similar to pyproject's
  one, allowing to provide custom test runner.
    Not really, but most go projects are just tested with `go test
    ./...` or `go test ./dir1 ./dir2 ...`. There are options for both
    test flags and test targets. Anything else probably warrants a
    custom check phase.

I have a prototype of guix import go-module ..., too, but it's not quite
ready yet. I will add onto this series a few packages to demonstrate the
build system, and port a few packages from go-build-system to
go-module-build-system.

Jørgen Kvalsvik (2):
  guix: Add downloader for go modules from GOPROXY
  guix: Add module-aware build system for go

 Makefile.am                           |   3 +
 guix/build-system/go-module.scm       | 268 +++++++++++++++
 guix/build/go-module-build-system.scm | 457 ++++++++++++++++++++++++++
 guix/go-mod-download.scm              | 126 +++++++
 4 files changed, 854 insertions(+)
 create mode 100644 guix/build-system/go-module.scm
 create mode 100644 guix/build/go-module-build-system.scm
 create mode 100644 guix/go-mod-download.scm