| Message ID | 20240108080048.25026-1-ngraves@ngraves.fr |
|---|---|
| Headers | show |
| Series | Extend bag-build to gexps. | expand |
-------------------- Start of forwarded message -------------------- From: Nicolas Graves <ngraves@ngraves.fr> To: guix-devel@gnu.org Cc: Andrew Tropin <andrew@trop.in> Subject: Re: [PATCH 00/48] Extend bag-build to gexps. Date: Wed, 10 Jan 2024 22:50:05 +0100 Here's a more complete proof of concept: The attached file guix.scm, when run a checkout of emacs branch emacs-29 with guix build -f guix.scm will : - compile everything as if run locally but with the patches provided by guix sources. - a local edit and the rerun of guix build -f guix.scm will only compile new / changed code. Now I can try developping emacs without worring about huge compilation times and without the developping issues that would happen if I ran the build-system by hand without the patches. We could even imagine adding an option to guix shell -D that could drop such a file for any package, since most of the code is reproducible (although applying a derivation this way is still quite hacky). Cheers! Nicolas On 2024-01-08 08:51, Nicolas Graves wrote: > Rationale: > Almost all build-systems are defined with gexpressions in functions > that return derivations. Derivations are not easily extensible while > gexps are. An example usage is given below. > > This is a pretty big rewrite that should recompile almost all packages, > but a lot of grafting happens such as I could rebuild my system quickly. > > I was trying to get the build-phases of an existing package to apply to > a local repository, because guix as a development tool for heavy packages > (emacs, ungoogled-chromium) is tedious, and there are precious info in > build-phases that can be applied in a local repository. I'm not aware of > prior work on this particular issue. > > These patches allow to do extensions such as: > > (build-system > (name 'local-gnu) > (description "GNU Build System applied in the current directory") > (lower > (lambda* args > (let ((old-bag (apply > (build-system-lower > (package-build-system emacs-pgtk)) > args))) > (bag > (inherit old-bag) > (build > (lambda* build-args > (mlet %store-monad > ((builder (apply (bag-build old-bag) build-args))) > (return (with-imported-modules '((guix build utils)) > #~(begin > (use-modules (guix build utils)) > (with-directory-excursion #$(getcwd) > #$builder)))))))))))) > > Of course this type of build-system isn't directly applicable because of > the chroot of the builder, but this other trick makes it happen : > > ;; We can't use package->derivation directly because we want the user rather > ;; than the daemon to build the derivation. > (with-store store > (run-with-store store > (mlet* %store-monad ((bag -> (package->bag pkg)) > (drv (bag->derivation bag pkg))) > ;; ensure inputs are in the store. > (built-derivations (derivation-inputs drv)) > (with-environment-excursion > (apply invoke (derivation-builder (pk 'd drv)) > (derivation-builder-arguments drv)))))) > > This isn't polished yet, but could serve as an handy way to develop > heavy packages locally while taking advantage of the code that's > already in guix build phases. > > > Nicolas Graves (48): > guix: packages: Extend bag-build to support gexp. > build-system: gnu: Improve gnu-cross-build style. > build-system: gnu: Redefine gnu-build and gnu-cross-build. > build-system: agda: Redefine agda-build. > build-system: android-ndk: Redefine gnu-build. > build-system: ant: Redefine ant-build. > build-system: asdf: Redefine asdf-build. > build-system: cargo: Redefine cargo-build and cargo-cross-build. > build-system: chicken: Redefine chicken-build. > build-system: clojure: Redefine clojure-build. > build-system: cmake: Redefine cmake-build and cmake-cross-build. > build-system: composer: Redefine composer-build. > build-system: copy: Redefine copy-build. > build-system: dub: Redefine dub-build. > build-system: dune: Redefine dune-build. > build-system: elm: Redefine elm-build. > build-system: emacs: Redefine emacs-build. > build-system: font: Redefine font-build. > build-system: glib-or-gtk: Improve glib-or-gtk-cross-build style. > build-system: glib-or-gtk: Redefine glib-or-gtk-build functions. > build-system: go: Redefine go-build and go-cross-build. > build-system: guile: Redefine guile-build and guile-cross-build. > build-system: haskell: Redefine haskell-build. > build-system: julia: Redefine julia-build. > build-system: linux-module: Redefine linux-module-build functions. > build-system: maven: Redefine maven-build. > build-system: meson: Redefine meson-build and meson-cross-build. > build-system: minify: Redefine minify-build. > build-system: mix: Redefine mix-build. > build-system: node: Redefine node-build. > build-system: ocaml: Redefine ocaml-build. > build-system: perl: Redefine perl-build and perl-cross-build. > build-system: pyproject: Redefine pyproject-build. > build-system: python: Redefine python-build. > build-system: qt: Redefine qt-build and qt-cross-build. > build-system: r: Redefine r-build. > build-system: rakudo: Redefine rakudo-build. > build-system: rebar: Redefine rebar-build. > build-system: renpy: Redefine renpy-build. > build-system: ruby: Improve ruby-cross-build style. > build-system: ruby: Redefine ruby-build. > build-system: scons: Redefine scons-build. > build-system: texlive: Redefine texlive-build. > build-system: tree-sitter: Redefine tree-sitter-build functions. > build-system: vim: Redefine vim-build. > build-system: waf: Improve waf-build style. > build-system: zig: Redefine zig-build. > build-system: trivial: Redefine trivial-build functions. > > guix/build-system.scm | 2 +- > guix/build-system/agda.scm | 8 +- > guix/build-system/android-ndk.scm | 8 +- > guix/build-system/ant.scm | 8 +- > guix/build-system/asdf.scm | 8 +- > guix/build-system/cargo.scm | 19 ++--- > guix/build-system/chicken.scm | 8 +- > guix/build-system/clojure.scm | 8 +- > guix/build-system/cmake.scm | 24 ++---- > guix/build-system/composer.scm | 9 +-- > guix/build-system/copy.scm | 11 +-- > guix/build-system/dub.scm | 8 +- > guix/build-system/dune.scm | 9 +-- > guix/build-system/elm.scm | 8 +- > guix/build-system/emacs.scm | 8 +- > guix/build-system/font.scm | 10 +-- > guix/build-system/glib-or-gtk.scm | 115 ++++++++++++---------------- > guix/build-system/gnu.scm | 119 +++++++++++++---------------- > guix/build-system/go.scm | 20 ++--- > guix/build-system/guile.scm | 21 ++--- > guix/build-system/haskell.scm | 8 +- > guix/build-system/julia.scm | 8 +- > guix/build-system/linux-module.scm | 17 ++--- > guix/build-system/maven.scm | 8 +- > guix/build-system/meson.scm | 25 ++---- > guix/build-system/minify.scm | 8 +- > guix/build-system/mix.scm | 12 +-- > guix/build-system/node.scm | 8 +- > guix/build-system/ocaml.scm | 9 +-- > guix/build-system/perl.scm | 22 ++---- > guix/build-system/pyproject.scm | 13 +--- > guix/build-system/python.scm | 12 +-- > guix/build-system/qt.scm | 17 ++--- > guix/build-system/r.scm | 9 +-- > guix/build-system/rakudo.scm | 8 +- > guix/build-system/rebar.scm | 12 +-- > guix/build-system/renpy.scm | 8 +- > guix/build-system/ruby.scm | 48 ++++++------ > guix/build-system/scons.scm | 9 +-- > guix/build-system/texlive.scm | 11 +-- > guix/build-system/tree-sitter.scm | 16 ++-- > guix/build-system/trivial.scm | 41 ++++------ > guix/build-system/vim.scm | 15 +--- > guix/build-system/waf.scm | 32 ++++---- > guix/build-system/zig.scm | 8 +- > guix/packages.scm | 53 ++++++++++--- > 46 files changed, 348 insertions(+), 520 deletions(-)
Rationale: Almost all build-systems are defined with gexpressions in functions that return derivations. Derivations are not easily extensible while gexps are. An example usage is given below. This is a pretty big rewrite that should recompile almost all packages, but a lot of grafting happens such as I could rebuild my system quickly. I was trying to get the build-phases of an existing package to apply to a local repository, because guix as a development tool for heavy packages (emacs, ungoogled-chromium) is tedious, and there are precious info in build-phases that can be applied in a local repository. I'm not aware of prior work on this particular issue. These patches allow to do extensions such as: (build-system (name 'local-gnu) (description "GNU Build System applied in the current directory") (lower (lambda* args (let ((old-bag (apply (build-system-lower (package-build-system emacs-pgtk)) args))) (bag (inherit old-bag) (build (lambda* build-args (mlet %store-monad ((builder (apply (bag-build old-bag) build-args))) (return (with-imported-modules '((guix build utils)) #~(begin (use-modules (guix build utils)) (with-directory-excursion #$(getcwd) #$builder)))))))))))) Of course this type of build-system isn't directly applicable because of the chroot of the builder, but this other trick makes it happen : ;; We can't use package->derivation directly because we want the user rather ;; than the daemon to build the derivation. (with-store store (run-with-store store (mlet* %store-monad ((bag -> (package->bag pkg)) (drv (bag->derivation bag pkg))) ;; ensure inputs are in the store. (built-derivations (derivation-inputs drv)) (with-environment-excursion (apply invoke (derivation-builder (pk 'd drv)) (derivation-builder-arguments drv)))))) This isn't polished yet, but could serve as an handy way to develop heavy packages locally while taking advantage of the code that's already in guix build phases. Nicolas Graves (48): guix: packages: Extend bag-build to support gexp. build-system: gnu: Improve gnu-cross-build style. build-system: gnu: Redefine gnu-build and gnu-cross-build. build-system: agda: Redefine agda-build. build-system: android-ndk: Redefine gnu-build. build-system: ant: Redefine ant-build. build-system: asdf: Redefine asdf-build. build-system: cargo: Redefine cargo-build and cargo-cross-build. build-system: chicken: Redefine chicken-build. build-system: clojure: Redefine clojure-build. build-system: cmake: Redefine cmake-build and cmake-cross-build. build-system: composer: Redefine composer-build. build-system: copy: Redefine copy-build. build-system: dub: Redefine dub-build. build-system: dune: Redefine dune-build. build-system: elm: Redefine elm-build. build-system: emacs: Redefine emacs-build. build-system: font: Redefine font-build. build-system: glib-or-gtk: Improve glib-or-gtk-cross-build style. build-system: glib-or-gtk: Redefine glib-or-gtk-build functions. build-system: go: Redefine go-build and go-cross-build. build-system: guile: Redefine guile-build and guile-cross-build. build-system: haskell: Redefine haskell-build. build-system: julia: Redefine julia-build. build-system: linux-module: Redefine linux-module-build functions. build-system: maven: Redefine maven-build. build-system: meson: Redefine meson-build and meson-cross-build. build-system: minify: Redefine minify-build. build-system: mix: Redefine mix-build. build-system: node: Redefine node-build. build-system: ocaml: Redefine ocaml-build. build-system: perl: Redefine perl-build and perl-cross-build. build-system: pyproject: Redefine pyproject-build. build-system: python: Redefine python-build. build-system: qt: Redefine qt-build and qt-cross-build. build-system: r: Redefine r-build. build-system: rakudo: Redefine rakudo-build. build-system: rebar: Redefine rebar-build. build-system: renpy: Redefine renpy-build. build-system: ruby: Improve ruby-cross-build style. build-system: ruby: Redefine ruby-build. build-system: scons: Redefine scons-build. build-system: texlive: Redefine texlive-build. build-system: tree-sitter: Redefine tree-sitter-build functions. build-system: vim: Redefine vim-build. build-system: waf: Improve waf-build style. build-system: zig: Redefine zig-build. build-system: trivial: Redefine trivial-build functions. guix/build-system.scm | 2 +- guix/build-system/agda.scm | 8 +- guix/build-system/android-ndk.scm | 8 +- guix/build-system/ant.scm | 8 +- guix/build-system/asdf.scm | 8 +- guix/build-system/cargo.scm | 19 ++--- guix/build-system/chicken.scm | 8 +- guix/build-system/clojure.scm | 8 +- guix/build-system/cmake.scm | 24 ++---- guix/build-system/composer.scm | 9 +-- guix/build-system/copy.scm | 11 +-- guix/build-system/dub.scm | 8 +- guix/build-system/dune.scm | 9 +-- guix/build-system/elm.scm | 8 +- guix/build-system/emacs.scm | 8 +- guix/build-system/font.scm | 10 +-- guix/build-system/glib-or-gtk.scm | 115 ++++++++++++---------------- guix/build-system/gnu.scm | 119 +++++++++++++---------------- guix/build-system/go.scm | 20 ++--- guix/build-system/guile.scm | 21 ++--- guix/build-system/haskell.scm | 8 +- guix/build-system/julia.scm | 8 +- guix/build-system/linux-module.scm | 17 ++--- guix/build-system/maven.scm | 8 +- guix/build-system/meson.scm | 25 ++---- guix/build-system/minify.scm | 8 +- guix/build-system/mix.scm | 12 +-- guix/build-system/node.scm | 8 +- guix/build-system/ocaml.scm | 9 +-- guix/build-system/perl.scm | 22 ++---- guix/build-system/pyproject.scm | 13 +--- guix/build-system/python.scm | 12 +-- guix/build-system/qt.scm | 17 ++--- guix/build-system/r.scm | 9 +-- guix/build-system/rakudo.scm | 8 +- guix/build-system/rebar.scm | 12 +-- guix/build-system/renpy.scm | 8 +- guix/build-system/ruby.scm | 48 ++++++------ guix/build-system/scons.scm | 9 +-- guix/build-system/texlive.scm | 11 +-- guix/build-system/tree-sitter.scm | 16 ++-- guix/build-system/trivial.scm | 41 ++++------ guix/build-system/vim.scm | 15 +--- guix/build-system/waf.scm | 32 ++++---- guix/build-system/zig.scm | 8 +- guix/packages.scm | 53 ++++++++++--- 46 files changed, 348 insertions(+), 520 deletions(-)