mbox series

[bug#49919,core-updates,0/2] build-system/go: Enable parallelism.

Message ID cover.1628309588.git.iskarian@mgsn.dev
Headers show
Series build-system/go: Enable parallelism. | expand

Message

Sarah Morgensen Aug. 7, 2021, 4:45 a.m. UTC 
Hello Guix,

These patches give the Go build system the standard parallelism levers. I would
have thought that Go was already detecting the correct number for GOMAXPROCS,
but after I made this same change for the bootstrapping Go compiler, Efraim
found that it cut the build time nearly in half [0].

[0] https://issues.guix.gnu.org/49616

--
Sarah Morgensen (2):
  build-system/go: Honor #:parallel-build?.
  build-system/go: Honor #:parallel-tests?.

 guix/build-system/go.scm       |  5 +++++
 guix/build/go-build-system.scm | 12 ++++++++++--
 2 files changed, 15 insertions(+), 2 deletions(-)


base-commit: 693d75e859150601145b7f7303f61d4f48e76927

Comments

Mathieu Othacehe Aug. 19, 2021, 3:13 p.m. UTC | #1 
Hello Sarah,

> These patches give the Go build system the standard parallelism levers. I would
> have thought that Go was already detecting the correct number for GOMAXPROCS,
> but after I made this same change for the bootstrapping Go compiler, Efraim
> found that it cut the build time nearly in half [0].

This looks good, but I'd rather have Cuirass fixed before pushing this
series so that we can evaluate the impact on all Go packages and
potentially find some regressions.

Thanks,

Mathieu
Simon Tournier Aug. 31, 2021, 10:03 a.m. UTC | #2 
Hi,

On Fri, 06 Aug 2021 at 21:45, Sarah Morgensen <iskarian@mgsn.dev> wrote:

> These patches give the Go build system the standard parallelism levers. I would
> have thought that Go was already detecting the correct number for GOMAXPROCS,
> but after I made this same change for the bootstrapping Go compiler, Efraim
> found that it cut the build time nearly in half [0].
>
> [0] https://issues.guix.gnu.org/49616

Honoring the parallelism, do the Go packages still build
deterministically?  If not, the default should be still non parallel, as
it is for Haskell packages, IIRC.  However, maybe there is room for a
package transformation ’--with-parallelism’ to easily turn on when speed
matters more than reproducibility.

All the best,
simon
Sarah Morgensen Aug. 31, 2021, 4:06 p.m. UTC | #3 
Hello,

zimoun <zimon.toutoune@gmail.com> writes:

> Hi,
>
> On Fri, 06 Aug 2021 at 21:45, Sarah Morgensen <iskarian@mgsn.dev> wrote:
>
>> These patches give the Go build system the standard parallelism levers. I would
>> have thought that Go was already detecting the correct number for GOMAXPROCS,
>> but after I made this same change for the bootstrapping Go compiler, Efraim
>> found that it cut the build time nearly in half [0].
>>
>> [0] https://issues.guix.gnu.org/49616
>
> Honoring the parallelism, do the Go packages still build
> deterministically?  If not, the default should be still non parallel, as
> it is for Haskell packages, IIRC.  However, maybe there is room for a
> package transformation ’--with-parallelism’ to easily turn on when speed
> matters more than reproducibility.

I haven't personally tested this aside from a few casual runs with
--check, but from everything I understand about Go, parallelism does not
affect build determinism.  Each build unit should still be compiled by a
single worker sequentially, but parallelism enables separate build units
to be compiled in parallel.

--
Sarah
Efraim Flashner Dec. 15, 2021, 10:35 a.m. UTC | #4 
On Thu, Aug 19, 2021 at 05:13:52PM +0200, Mathieu Othacehe wrote:
> 
> Hello Sarah,
> 
> > These patches give the Go build system the standard parallelism levers. I would
> > have thought that Go was already detecting the correct number for GOMAXPROCS,
> > but after I made this same change for the bootstrapping Go compiler, Efraim
> > found that it cut the build time nearly in half [0].
> 
> This looks good, but I'd rather have Cuirass fixed before pushing this
> series so that we can evaluate the impact on all Go packages and
> potentially find some regressions.
> 

Friendly ping to see if Cuirass is ready for these patches to be pushed.
Mathieu Othacehe Dec. 15, 2021, 10:43 a.m. UTC | #5 
Hey,

> Friendly ping to see if Cuirass is ready for these patches to be pushed.

Not really, I need to reconfigure Berlin and I'm hitting multiple
issues, such as a php build failure :(.

Mathieu
Maxim Cournoyer Jan. 20, 2024, 8:49 p.m. UTC | #6 
Hello,

Mathieu Othacehe <othacehe@gnu.org> writes:

> Hey,
>
>> Friendly ping to see if Cuirass is ready for these patches to be pushed.
>
> Not really, I need to reconfigure Berlin and I'm hitting multiple
> issues, such as a php build failure :(.

Queued for applying locally, at last.  Closing.