| Message ID | bd7d5613d104b8bb364e12686574a4f8f275911e.camel@planete-kraus.eu |
|---|---|
| State | New |
| Headers | show |
| Series | [bug#60358] gnu: Add gnulib. | expand |
Hello guix, I’m trying to provide gnulib for guix. However, when I try to use a git-checkout as the source, to always have the latest commit, then guix fails with: updating checkout of 'https://git.savannah.gnu.org/git/gnulib.git'... guix build: error: Git failure while fetching https://git.savannah.gnu.org/git/gnulib.git: reference 'refs/remotes/origin/HEAD' not found I don’t fully understand what happens. Do you have an idea? Also, when using gnulib, the whole point is to have GNULIB_SRCDIR point to a directory where the gnulib files are, so that bootstrap srcipts can use that instead of a git submodule (which is not possible with guix). I tried to set it as a search-path-specification, which seems to work, but I’m not sure that it is the correct approach. What do you think? Best regards, Vivien
Hi. Many packages needs a specific checkout of gnulib to work reliably, via --gnulib-refdir= (rather than --gnulib-srcdir=), would you consider installing the entire gnulib git archive instead of just the latest checkout? Since so many packages these days require different versions of gnulib, it may make sense to mak the gnulib git archive available as an installed Guix package, so that other packages may bootstrap build from it and get the intended gnulib git checkout. If a checked out gnulib git archive is available as a Guix package, other packages could build-depend on it and use both --gnulib-refdir and --gnulib-srcdir. If I understand your approach now, only --gnulib-srcdir is possible. You could put the git archive under /share/gnulib-git or merely /share/gnulib/ and set both GNULIB_REFDIR and GNULIB_SRCDIR. /Simon
Am Dienstag, dem 27.12.2022 um 17:23 +0100 schrieb Vivien Kraus: > * gnu/packages/build-tools.scm (gnulib-latest): New variable. This > one always > tracks the latest upstream commit, so don’t use it as an input. > (gnulib): New variable. This is a "recent" snapshot. The preferred package:patch ratio is 1:1. > --- > gnu/packages/build-tools.scm | 71 > ++++++++++++++++++++++++++++++++++++ > 1 file changed, 71 insertions(+) > > diff --git a/gnu/packages/build-tools.scm b/gnu/packages/build- > tools.scm > index 6c1350c44f..062956f685 100644 > --- a/gnu/packages/build-tools.scm > +++ b/gnu/packages/build-tools.scm > @@ -38,6 +38,7 @@ (define-module (gnu packages build-tools) > #:use-module (guix gexp) > #:use-module (guix download) > #:use-module (guix git-download) > + #:use-module (guix git) > #:use-module (guix build-system cmake) > #:use-module (gnu packages) > #:use-module (gnu packages adns) > @@ -48,11 +49,13 @@ (define-module (gnu packages build-tools) > #:use-module (gnu packages cpp) > #:use-module (gnu packages elf) > #:use-module (gnu packages linux) > + #:use-module (gnu packages lisp) > #:use-module (gnu packages logging) > #:use-module (gnu packages lua) > #:use-module (gnu packages ninja) > #:use-module (gnu packages package-management) > #:use-module (gnu packages pcre) > + #:use-module (gnu packages perl) > #:use-module (gnu packages pkg-config) > #:use-module (gnu packages pretty-print) > #:use-module (gnu packages protobuf) > @@ -803,3 +806,71 @@ (define-public genie > same settings to multiple projects. It supports generating projects > using GNU > Makefiles, JSON Compilation Database, and experimentally Ninja.") > (license license:bsd-3)))) > + > +;; Gnulib is a source code library. The developers encourage you to > develop > +;; your programs with the latest commit of gnulib. However, gnulib > is also > +;; used to bootstrap some GNU programs. To avoid rebuilding the > world when a > +;; new gnulib commit is pushed, gnulib as a native input must use a > known > +;; snapshot of the library. > + > +(define-public gnulib-latest > + ;; Use this to develop. > + (package > + (name "gnulib-latest") > + (version "latest") > + (source > + (git-checkout > + (url "https://git.savannah.gnu.org/git/gnulib.git"))) Don't. If you need to check out a particular commit, use the --with- commit CLI option or a transformer. > + (build-system gnu-build-system) > + (arguments > + (list > + #:phases > + #~(modify-phases %standard-phases > + (delete 'configure) > + (replace 'install > + (lambda _ > + (install-file "gnulib-tool" > + (string-append #$output "/bin")) > + (copy-recursively "." > + (string-append #$output > "/share/gnulib-srcdir"))))) > + #:tests? #f)) ;; Tests are syntax and indentation checks for > the > + ;; maintainer. > + (inputs ;; Shebangs for some auxiliary build files. > + (list python perl clisp)) > + (home-page "https://www.gnu.org/software/gnulib/") > + (synopsis "GNU portability library") > + (description > + "Gnulib is a library that provides common functions from the > glibc to your > +programs, in order to enhance portability across operating systems. > It also > +provides common maintainer tools for gnu packages. This package > always tracks > +the latest commit in gnulib. @strong{For packages dependencies, > please use the > +@code{gnulib} package instead.} The latter provides a recent enough > snapshot.") > + (native-search-paths > + (list > + (search-path-specification > + (variable "GNULIB_SRCDIR") > + (files (list "share/gnulib-srcdir"))))) > + (license > + (list license:lgpl2.0+ license:gpl3+)))) > + > +(define-public gnulib > + ;; Use this as a native input. > + (package > + (inherit gnulib-latest) > + (name "gnulib") > + (version "2022-12-27") > + (source > + (origin > + (method git-fetch) > + (uri (git-reference > + (url "https://git.savannah.gnu.org/git/gnulib.git") > + (commit "fde75446490e18d2539817ca418ab8adf73b02d3"))) Don't throw together raw commits and commit-less versions. Use git- version instead. > + (sha256 > + (base32 > + "0fjbdhwi9025wyq39rwc2j6aazfmagx056kkbvxx6bs97i80dcim")))) > + (synopsis "GNU portability library") > + (description > + "Gnulib is a library that provides common functions from the > glibc to your > +programs, in order to enhance portability across operating systems. > It also > +provides common maintainer tools for gnu packages. This package > provides a > +recent snapshot of gnulib."))) Cheers
Hello! Le mardi 27 décembre 2022 à 19:44 +0100, Simon Josefsson a écrit : > Hi. Many packages needs a specific checkout of gnulib to work > reliably, > via --gnulib-refdir= (rather than --gnulib-srcdir=), would you > consider > installing the entire gnulib git archive instead of just the latest > checkout? I did not know about gnulib-refdir. Providing the entire gnulib archive is tempting, but there might be downsides. The contents of the .git directory depends on how trees and objects have been packed by git. As far as I understand, it is possible that pushing a commit in gnulib results in commit objects from previous commits to be re-packed. Thus, if I clone gnulib from the initial commit to a specific commit, the result might depend on other, unrelated commits. So, the content of .git is not reproducible. Maybe I could get around that by deleting all the refs, doing an aggressive garbage collection and then re-packing, but I’m not sure I would get a reproducible result. Now, I don’t know much about gnulib-refdir. How does gnulib-refdir work? Do you have an example of a package that uses this feature? Can I convince the package to use a checkout instead of the gnulib repository, if I swear I have the exact checkout it wants, maybe by tweaking bootstrap.conf or something? Vivien
Vivien Kraus via Guix-patches via <guix-patches@gnu.org> writes: > Hello! > > Le mardi 27 décembre 2022 à 19:44 +0100, Simon Josefsson a écrit : >> Hi. Many packages needs a specific checkout of gnulib to work >> reliably, >> via --gnulib-refdir= (rather than --gnulib-srcdir=), would you >> consider >> installing the entire gnulib git archive instead of just the latest >> checkout? > > I did not know about gnulib-refdir. It is newer than --gnulib-srcdir, but came about because --gnulib-srcdir is often a fragile solution: you have no idea which gnulib version the person building the package supplied. Since gnulib is rolling, compatibility becomes difficult. The --gnulib-srcdir approach works if you make sure to use the same gnulib git checkout as the project you wish to build uses. But then it becomes difficult to package gnulib: no two projects are likely to rely on the same gnulib git commit. So which gnulib git version to package? That's the motivation for packaging the gnulib git repository instead. This may sound strange, but compare how gettext/autopoint ships a CVS repository and checks out the particular files that are needed. Yeah, I can agree that this approach is not ideal, and there are many concerns with it. I'm not convinced gnulib's idea of "source-level library" is something that is viable long-term. But it is what exists today. > Providing the entire gnulib archive is tempting, but there might be > downsides. The contents of the .git directory depends on how trees and > objects have been packed by git. As far as I understand, it is > possible that pushing a commit in gnulib results in commit objects > from previous commits to be re-packed. Thus, if I clone gnulib from > the initial commit to a specific commit, the result might depend on > other, unrelated commits. So, the content of .git is not reproducible. > > Maybe I could get around that by deleting all the refs, doing an > aggressive garbage collection and then re-packing, but I’m not sure I > would get a reproducible result. Interesting -- I think researching this more would be useful. It should be possible to come up with a safe approach to produce a reproducible checkout of a git repository. Doesn't 'git archive' produce a reproducible output from a git repository for a particular branch and commit? > Now, I don’t know much about gnulib-refdir. How does gnulib-refdir > work? You supply --gnulib-refdir pointing to a local gnulib git repository clone when you run ./bootstrap. This avoid checking out the gnulib git submodule from Savannah, and instead ./bootstrap will use the local git repository instead. I believe it should automatically extract the intended gnulib git commit from the gnulib/ git submodule, and extract that version from the local copy (please test -- may be bugs). > Do you have an example of a package that uses this feature? Packages wouldn't use it, but if they use gnulib's ./bootstrap script the support this approach. The idea is that people building projects that use gnulib doesn't always have to fetch the gnulib git submodule, but have a local copy for security or performance reasons. > Can I convince the package to use a checkout instead of the gnulib > repository, if I swear I have the exact checkout it wants, maybe by > tweaking bootstrap.conf or something? You can force a package to use another gnulib version by using --gnulib-srcdir and point that to some other version of gnulib. But the project may not build. The point of --gnulib-refdir is to actually get the same version of gnulib that the project uses, why would you not want that? There is the GNULIB_REVISION environment variable that you can set to something else, but this is probably not very well tested and sounds like a bad idea (but I may be missing something). /Simon
Simon Josefsson via Guix-patches via <guix-patches@gnu.org> writes: > Doesn't 'git archive' produce a reproducible output from a git > repository for a particular branch and commit? Sorry, git-archive that doesn't include the .git/ sub-directory, so that's not the command you want. I don't know what command you would use. I reckon it should be a git clone, check out a particular commit, reset the branch to that commit, filtering away all other branches, and then do 'git gc --aggresive --prune=now'. If that produces unreproducable results for other unrelated changes to the repository, it would be nice to know how to get rid of them. /Simon
Dear guix, gnulib is a useful tool to recreate distribution tarballs from git on many GNU projects. In the future, it may be a native input for many. Should I move it to its own module, (gnu packages gnulib) for instance? That would certainly help with forseeable mess with circular guile module dependencies. What do you think? Best regards, Vivien
diff --git a/gnu/packages/build-tools.scm b/gnu/packages/build-tools.scm index 6c1350c44f..062956f685 100644 --- a/gnu/packages/build-tools.scm +++ b/gnu/packages/build-tools.scm @@ -38,6 +38,7 @@ (define-module (gnu packages build-tools) #:use-module (guix gexp) #:use-module (guix download) #:use-module (guix git-download) + #:use-module (guix git) #:use-module (guix build-system cmake) #:use-module (gnu packages) #:use-module (gnu packages adns) @@ -48,11 +49,13 @@ (define-module (gnu packages build-tools) #:use-module (gnu packages cpp) #:use-module (gnu packages elf) #:use-module (gnu packages linux) + #:use-module (gnu packages lisp) #:use-module (gnu packages logging) #:use-module (gnu packages lua) #:use-module (gnu packages ninja) #:use-module (gnu packages package-management) #:use-module (gnu packages pcre) + #:use-module (gnu packages perl) #:use-module (gnu packages pkg-config) #:use-module (gnu packages pretty-print) #:use-module (gnu packages protobuf) @@ -803,3 +806,71 @@ (define-public genie same settings to multiple projects. It supports generating projects using GNU Makefiles, JSON Compilation Database, and experimentally Ninja.") (license license:bsd-3)))) + +;; Gnulib is a source code library. The developers encourage you to develop +;; your programs with the latest commit of gnulib. However, gnulib is also +;; used to bootstrap some GNU programs. To avoid rebuilding the world when a +;; new gnulib commit is pushed, gnulib as a native input must use a known +;; snapshot of the library. + +(define-public gnulib-latest + ;; Use this to develop. + (package + (name "gnulib-latest") + (version "latest") + (source + (git-checkout + (url "https://git.savannah.gnu.org/git/gnulib.git"))) + (build-system gnu-build-system) + (arguments + (list + #:phases + #~(modify-phases %standard-phases + (delete 'configure) + (replace 'install + (lambda _ + (install-file "gnulib-tool" + (string-append #$output "/bin")) + (copy-recursively "." + (string-append #$output "/share/gnulib-srcdir"))))) + #:tests? #f)) ;; Tests are syntax and indentation checks for the + ;; maintainer. + (inputs ;; Shebangs for some auxiliary build files. + (list python perl clisp)) + (home-page "https://www.gnu.org/software/gnulib/") + (synopsis "GNU portability library") + (description + "Gnulib is a library that provides common functions from the glibc to your +programs, in order to enhance portability across operating systems. It also +provides common maintainer tools for gnu packages. This package always tracks +the latest commit in gnulib. @strong{For packages dependencies, please use the +@code{gnulib} package instead.} The latter provides a recent enough snapshot.") + (native-search-paths + (list + (search-path-specification + (variable "GNULIB_SRCDIR") + (files (list "share/gnulib-srcdir"))))) + (license + (list license:lgpl2.0+ license:gpl3+)))) + +(define-public gnulib + ;; Use this as a native input. + (package + (inherit gnulib-latest) + (name "gnulib") + (version "2022-12-27") + (source + (origin + (method git-fetch) + (uri (git-reference + (url "https://git.savannah.gnu.org/git/gnulib.git") + (commit "fde75446490e18d2539817ca418ab8adf73b02d3"))) + (sha256 + (base32 + "0fjbdhwi9025wyq39rwc2j6aazfmagx056kkbvxx6bs97i80dcim")))) + (synopsis "GNU portability library") + (description + "Gnulib is a library that provides common functions from the glibc to your +programs, in order to enhance portability across operating systems. It also +provides common maintainer tools for gnu packages. This package provides a +recent snapshot of gnulib.")))