| Message ID | 20240522145956.31218-2-ian@retrospec.tv |
|---|---|
| State | New |
| Headers | show |
| Series | Update LibreWolf to 126.0-1 [security fixes] | expand |
Hi Ian, Ian Eure <ian@retrospec.tv> writes: > * gnu/packages/librewolf.scm (librewolf): This patch removes an intermediate > step in the build chain. The upstream source tarball is created with an > automated build process, where Firefox sources are fetched, patched, and > repacked. Rather than download the output of that process, as the package has > been, it’s now replicated within the build process, similar to how IceCat > works. I think I'd rather keep using a human-prepared and vetted tarball, to avoid anything going stale in our local recipe of how it's meant to be prepared. It's also simpler and less maintenance, and arguably shields the users better against non-free source code (although I don't think there's anything non-free in the Firefox tree, so that point is more moot than say, for linux) to use a tarball. What do you or others think?
Hi Maxim, Maxim Cournoyer <maxim.cournoyer@gmail.com> writes: > Hi Ian, > > Ian Eure <ian@retrospec.tv> writes: > >> * gnu/packages/librewolf.scm (librewolf): This patch removes an >> intermediate >> step in the build chain. The upstream source tarball is >> created with an >> automated build process, where Firefox sources are fetched, >> patched, and >> repacked. Rather than download the output of that process, as >> the package has >> been, it’s now replicated within the build process, similar to >> how IceCat >> works. > > I think I'd rather keep using a human-prepared and vetted > tarball, to > avoid anything going stale in our local recipe of how it's meant > to be > prepared. > The upstream tarball is built by scripts run under a CI system which triggers when changes are pushed[1], and aren’t human-prepared or vetted in the same way that many release tarballs have tradionally been. This patchset uses the same script as upstream, with modifications to make it reproduceable, as the upstream process isn’t. As noted in the commit messages, IceCat also builds this way[2], including patching the upstream build script[3][4], so this seems like a reasonable & accepted way to build. Though perhaps there’s dissatisfaction with the IceCat build which I wasn’t aware of, being a fairly new contributor. > It's also simpler and less maintenance, and arguably shields > the users better against non-free source code (although I don't > think > there's anything non-free in the Firefox tree, so that point is > more > moot than say, for linux) to use a tarball. > > What do you or others think? > It’s definitely simpler to use the upstream tarball in most cases, which is why I went that direction when I initially packaged LibreWolf. But, since IceCat builds this way, and the xz backdoor was discovered hiding in the non-reproduceable build process, I’ve been intending to update the package to control the full build, rather than trusting an unreproducable external process. I understand that if the build scripts are backdoored, it doesn’t matter whether upstream runs them or Guix does, but I believe that aligning with IceCat and having a reproduceable build directly from the upstream source repo are worthwhile. In the specific case of the 126.0-1 release, owning the whole build process made things easier. Upstream backported a very large Firefox change[5] which updates a bundled dependency to a new version; that dependency doesn’t work with Rust 1.75, which is what’s in Guix. With the Guix build process controlling what patches get applied, I was able to solve the problem by removing one line from the manifest of patches to apply to the Firefox source. If the package builds from the 126.0-1 tarball, it’ll need to ship a 22,000-line patch(!) to back out that change. That may still be necessary, depending on the timing of the rust-team branch merging and the next Firefox release, but at least for now, things are simpler. Ideally, this wolud be solved by unbundling that (and the other) vendored Rust libraries (and that’s something I intend to look into), but I didn’t want to block security fixes on work with unknown-but-probably-large scope -- there will almost definitely be Rust libraries currently not packaged in Guix which need to be addressed. As far as maintenance burden or things getting stale, the risk is that upstream alters their scripts, which requires updates to the Guix patches for them. This doesn’t seem like a major drawback to me, and I’m the one doing the maintenance. :) Overall, I think it’s a reasonable tradeoff for the reproducability we gain. If this approach to building LibreWolf in this patchset is acepted, I’d like to work with upstream to make their build process more flexible, ideally running it unmodified in the Guix build, which would eliminate the risk. Lastly: I noticed that I neglected to update %librewolf-build-id when I sent this patchset in. If my arguments are compelling enough for you, I think it’d make sense to update that when the changes are pushed (it’s a one-line change & the command to print an ID are in the comment above the variable). But, if you’d like a v2 patchset, either just to update that, or to back out the build process change and replace it with a 22kloc patch, I’d be happy to handle it instead. Thank you very much for your thoughts and the time you took to respond. — Ian [1]: https://codeberg.org/librewolf/source/actions/runs/168/jobs/0 [2]: https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/gnuzilla.scm?id=898b5f30f3d485d48275c920da172863da9524c6#n530 [3]: https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/gnuzilla.scm?id=898b5f30f3d485d48275c920da172863da9524c6#n571 [4]: https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/patches/icecat-makeicecat.patch [5]: https://codeberg.org/librewolf/source/commit/d292bdd2213a22e5b364339dfee68a27670f1b72
Hi Ian, Ian Eure <ian@retrospec.tv> writes: > Hi Maxim, > > Maxim Cournoyer <maxim.cournoyer@gmail.com> writes: > >> Hi Ian, >> >> Ian Eure <ian@retrospec.tv> writes: >> >>> * gnu/packages/librewolf.scm (librewolf): This patch removes an >>> intermediate >>> step in the build chain. The upstream source tarball is created >>> with an >>> automated build process, where Firefox sources are fetched, >>> patched, and >>> repacked. Rather than download the output of that process, as the >>> package has >>> been, it’s now replicated within the build process, similar to how >>> IceCat >>> works. >> >> I think I'd rather keep using a human-prepared and vetted tarball, >> to >> avoid anything going stale in our local recipe of how it's meant to >> be >> prepared. >> > > The upstream tarball is built by scripts run under a CI system which > triggers when changes are pushed[1], and aren’t human-prepared or > vetted in the same way that many release tarballs have tradionally > been. This patchset uses the same script as upstream, with > modifications to make it reproduceable, as the upstream process isn’t. Perhaps the modifications to make it reproducible could be shared to upstream? We'd benefit all thee users of librewolf this way, not only Guix ones. > As noted in the commit messages, IceCat also builds this way[2], > including patching the upstream build script[3][4], so this seems like > a reasonable & accepted way to build. Though perhaps there’s > dissatisfaction with the IceCat build which I wasn’t aware of, being a > fairly new contributor. The "dissatisfaction", if we can call it that, was about Linux-libre, and voiced by some a few years ago, including the project maintainers, if I recall correctly. The idea of linux-libre is to shield users from blobs. In this sense it is valuable that they don't even have to touch the pristine blobbed (there are a few array-defined firmwares in the tree still, at least one old Apple one IIRC) Linux source, which is considered problematic for some from a GNU FSDG perspective. >> It's also simpler and less maintenance, and arguably shields >> the users better against non-free source code (although I don't >> think >> there's anything non-free in the Firefox tree, so that point is more >> moot than say, for linux) to use a tarball. >> >> What do you or others think? >> > > It’s definitely simpler to use the upstream tarball in most cases, > which is why I went that direction when I initially packaged > LibreWolf. But, since IceCat builds this way, and the xz backdoor was > discovered hiding in the non-reproduceable build process, I’ve been > intending to update the package to control the full build, rather than > trusting an unreproducable external process. I understand that if the > build scripts are backdoored, it doesn’t matter whether upstream runs > them or Guix does, but I believe that aligning with IceCat and having > a reproduceable build directly from the upstream source repo are > worthwhile. Right. > In the specific case of the 126.0-1 release, owning the whole build > process made things easier. Upstream backported a very large Firefox > change[5] which updates a bundled dependency to a new version; that > dependency doesn’t work with Rust 1.75, which is what’s in Guix. With > the Guix build process controlling what patches get applied, I was > able to solve the problem by removing one line from the manifest of > patches to apply to the Firefox source. If the package builds from > the 126.0-1 tarball, it’ll need to ship a 22,000-line patch(!) to back > out that change. That may still be necessary, depending on the timing > of the rust-team branch merging and the next Firefox release, but at > least for now, things are simpler. Ideally, this wolud be solved by > unbundling that (and the other) vendored Rust libraries (and that’s > something I intend to look into), but I didn’t want to block security > fixes on work with unknown-but-probably-large scope -- there will > almost definitely be Rust libraries currently not packaged in Guix > which need to be addressed. OK, this flexibility seems indeed useful here. > As far as maintenance burden or things getting stale, the risk is that > upstream alters their scripts, which requires updates to the Guix > patches for them. This doesn’t seem like a major drawback to me, and > I’m the one doing the maintenance. :) Overall, I think it’s a > reasonable tradeoff for the reproducability we gain. If this approach > to building LibreWolf in this patchset is acepted, I’d like to work > with upstream to make their build process more flexible, ideally > running it unmodified in the Guix build, which would eliminate the > risk. Yes, you are the one doing it (thank you!) until you won't :-) (life...). Then someone else would have to pick it up and understand it. The simpler the better. > Lastly: I noticed that I neglected to update %librewolf-build-id when > I sent this patchset in. If my arguments are compelling enough for > you, I think it’d make sense to update that when the changes are > pushed (it’s a one-line change & the command to print an ID are in the > comment above the variable). But, if you’d like a v2 patchset, either > just to update that, or to back out the build process change and > replace it with a 22kloc patch, I’d be happy to handle it instead. The 22kloc patch doesn't sound too good... I guess we can stick with the self-made tarball for now. > Thank you very much for your thoughts and the time you took to > respond. Sorry for the delay handling this security-sensitive issue (still better than our ungoogled-chromium package which appears untouched for a full year, though! We should probably open a security issue about that). If you could send v2 with the build id thing, I'll try to apply it quickly.
Maxim Cournoyer <maxim.cournoyer@gmail.com> writes: > Hi Ian, > > Ian Eure <ian@retrospec.tv> writes: > >> Hi Maxim, >> >> Maxim Cournoyer <maxim.cournoyer@gmail.com> writes: >> >>> Hi Ian, >>> >>> Ian Eure <ian@retrospec.tv> writes: >>> >>>> * gnu/packages/librewolf.scm (librewolf): This patch removes >>>> an >>>> intermediate >>>> step in the build chain. The upstream source tarball is >>>> created >>>> with an >>>> automated build process, where Firefox sources are fetched, >>>> patched, and >>>> repacked. Rather than download the output of that process, >>>> as the >>>> package has >>>> been, it’s now replicated within the build process, similar >>>> to how >>>> IceCat >>>> works. >>> >>> I think I'd rather keep using a human-prepared and vetted >>> tarball, >>> to >>> avoid anything going stale in our local recipe of how it's >>> meant to >>> be >>> prepared. >>> >> >> The upstream tarball is built by scripts run under a CI system >> which >> triggers when changes are pushed[1], and aren’t human-prepared >> or >> vetted in the same way that many release tarballs have >> tradionally >> been. This patchset uses the same script as upstream, with >> modifications to make it reproduceable, as the upstream process >> isn’t. > > Perhaps the modifications to make it reproducible could be > shared to > upstream? We'd benefit all thee users of librewolf this way, > not only > Guix ones. > Yes, I plan to work with upstream on this. The current modifications are Guix-specific, but I believe a mechanism which allows for both better upstream reproducability and less hacky Guix packaging is possible. >> As noted in the commit messages, IceCat also builds this >> way[2], >> including patching the upstream build script[3][4], so this >> seems like >> a reasonable & accepted way to build. Though perhaps there’s >> dissatisfaction with the IceCat build which I wasn’t aware of, >> being a >> fairly new contributor. > > The "dissatisfaction", if we can call it that, was about > Linux-libre, > and voiced by some a few years ago, including the project > maintainers, > if I recall correctly. The idea of linux-libre is to shield > users from > blobs. In this sense it is valuable that they don't even have > to touch > the pristine blobbed (there are a few array-defined firmwares in > the > tree still, at least one old Apple one IIRC) Linux source, which > is > considered problematic for some from a GNU FSDG perspective. > Gotcha. I agree that these are unlikely to apply here. Thank you for pushing this, and I’ll try to get commit messages closer to the convention in the future. — Ian
diff --git a/gnu/packages/librewolf.scm b/gnu/packages/librewolf.scm index fa83857c96..bb8bc8a283 100644 --- a/gnu/packages/librewolf.scm +++ b/gnu/packages/librewolf.scm @@ -40,10 +40,12 @@ (define-module (gnu packages librewolf) + #:use-module ((srfi srfi-1) #:hide (zip)) #:use-module (guix build-system gnu) #:use-module (guix build-system cargo) #:use-module (guix build-system trivial) #:use-module (guix download) + #:use-module (guix git-download) #:use-module ((guix licenses) #:prefix license:) #:use-module (guix gexp) #:use-module (guix packages) @@ -62,6 +64,7 @@ (define-module (gnu packages librewolf) #:use-module (gnu packages gl) #:use-module (gnu packages glib) #:use-module (gnu packages gnome) + #:use-module (gnu packages gnuzilla) #:use-module (gnu packages gtk) #:use-module (gnu packages hunspell) #:use-module (gnu packages icu4c) @@ -81,6 +84,7 @@ (define-module (gnu packages librewolf) #:use-module (gnu packages pkg-config) #:use-module (gnu packages pulseaudio) #:use-module (gnu packages python) + #:use-module (gnu packages python-xyz) #:use-module (gnu packages rust) #:use-module (gnu packages rust-apps) #:use-module (gnu packages speech) @@ -89,6 +93,109 @@ (define-module (gnu packages librewolf) #:use-module (gnu packages xdisorg) #:use-module (gnu packages xorg)) +(define (firefox-source-origin version hash) + (origin + (method url-fetch) + (uri (string-append + "https://ftp.mozilla.org/pub/firefox/releases/" + version "/source/" "firefox-" version + ".source.tar.xz")) + (sha256 (base32 hash)))) + +(define (librewolf-source-origin version hash) + (origin + (method git-fetch) + (uri (git-reference + (url "https://codeberg.org/librewolf/source.git") + (commit version) + (recursive? #t))) + (file-name (git-file-name "librewolf-source" version)) + (sha256 (base32 hash)))) + +(define computed-origin-method (@@ (guix packages) computed-origin-method)) + +(define librewolf-source + (let* ((ff-src (firefox-source-origin "125.0.2" "16gpd6n52lshvkkha41z7xicggj64dw0qhr5gd07bcxsc4rmdl39")) + (version "125.0.2-1") + (lw-src (librewolf-source-origin version "17i36s2ny1pv3cz44w0gz48fy4vjfw6vp9jk21j62f5d3dl726x8"))) + + (origin + (method computed-origin-method) + (file-name (string-append "librewolf-" version ".source.tar.gz")) + (sha256 #f) + (uri + (delay + (with-imported-modules '((guix build utils)) + #~(begin + (use-modules (guix build utils)) + (set-path-environment-variable + "PATH" '("bin") + (list #+python + #+(canonical-package bash) + #+(canonical-package gnu-make) + #+(canonical-package coreutils) + #+(canonical-package findutils) + #+(canonical-package patch) + #+(canonical-package xz) + #+(canonical-package sed) + #+(canonical-package grep) + #+(canonical-package gzip) + #+(canonical-package tar))) + (set-path-environment-variable + "PYTHONPATH" + (list #+(format #f "lib/python~a/site-packages" + (version-major+minor + (package-version python)))) + '#+(cons python-jsonschema + (map second + (package-transitive-propagated-inputs + python-jsonschema)))) + + ;; Copy LibreWolf source into the build directory and make + ;; everything writable. + (copy-recursively #+lw-src ".") + (for-each make-file-writable (find-files ".")) + + ;; Patch Makefile to use the upstream source instead of downloading. + (substitute* '("Makefile") + (("^ff_source_tarball:=.*") + (string-append "ff_source_tarball:=" #+ff-src))) + + ;; Stage locales + (begin + (format #t "Staging locales...~%") + (force-output) + (mkdir "l10n-staging") + (with-directory-excursion "l10n-staging" + (for-each + (lambda (locale-dir) + (let ((locale + (string-drop (basename locale-dir) + (+ 32 ; length of hash + (string-length "-mozilla-locale-"))))) + (format #t " ~a~%" locale) + (force-output) + (copy-recursively locale-dir locale + #:log (%make-void-port "w")) + (for-each make-file-writable (find-files locale)) + (with-directory-excursion locale + (when (file-exists? ".hgtags") + (delete-file ".hgtags"))))) + '#+all-mozilla-locales))) + + ;; Patch build script to use staged locales. + (begin + (substitute* '("scripts/generate-locales.sh") + (("wget") "# wget") + (("unzip") "# unzip") + (("mv browser/locales/l10n/\\$1-\\*/") + "mv ../l10n-staging/$1/"))) + + ;; Run the build script + (invoke "make" "all") + (copy-file (string-append "librewolf-" #$version ".source.tar.gz") + #$output)))))))) + ;; Define the versions of rust needed to build librewolf, trying to match ;; upstream. See the file taskcluster/ci/toolchain/rust.yml at ;; https://searchfox.org under the particular firefox release, like @@ -104,18 +211,7 @@ (define-public librewolf (package (name "librewolf") (version "125.0.2-1") - (source - (origin - (method url-fetch) - - (uri (string-append "https://gitlab.com/api/v4/projects/32320088/" - "packages/generic/librewolf-source/" - version - "/librewolf-" - version - ".source.tar.gz")) - (sha256 - (base32 "09qzdaq9l01in9h4q14vyinjvvffycha2iyjqj5p4dd5jh6q5zma")))) + (source librewolf-source) (build-system gnu-build-system) (arguments (list