| Message ID | AS8P250MB07186E45F45ABAC876FE11629B39A@AS8P250MB0718.EURP250.PROD.OUTLOOK.COM |
|---|---|
| State | New |
| Headers | show |
| Series | [bug#64738] gnu: Make Mesa use zstd compression for shader cache instead of zlib. | expand |
Hi, On Thu, Jul 20, 2023 at 06:58 PM, 宋文武 wrote: >> >> gnu: mesa: Enable zstd compression for shader cache. >> >> * gnu/packages/gl.scm (mesa)[inputs]: Add zstd:lib. >> [arguments]: Add '-Dzstd=enabled' to configure-flags. > > Pushed to the mesa-updates branch, merging progress tracked in > https://issues.guix.gnu.org/64369. > > Thanks. I just saw this as the message only went to the bug number. I would not have merged this yet without checking about build status after the big texlive updates. I was hoping to avoid a full rebuild so we could merge to master with substitutes already, after checking post-texlive and the graft of mesa on master. However, doesn't seem like Cuirass has picked up this change and hasn't built yet. Any idea why? My plan right now was going to be to merge master into mesa-updates, see what the build status looks like, and then decide from there. If there were not too many rebuilds (say less than 1000) from the updates on master, we should be able to go ahead soon. And then start on the next round of patches/updates for mesa-updates. On the other hand, if there were lots of rebuilds there are other patches waiting that should be used also (libdrm, libva if I remember) as well as ungrafting the change on master. So, preventing waiting and doing another big rebuild with the changes that have been waiting since the first build of this branch. What do people think? My thought right now was to revert this change (since it rebuilds all of mesa dependents), merge master in, and check the status. But I'm confused why Cuirass wasn't building.. John
John Kehayias <john.kehayias@protonmail.com> writes: > Hi, > > On Thu, Jul 20, 2023 at 06:58 PM, 宋文武 wrote: > >>> >>> gnu: mesa: Enable zstd compression for shader cache. >>> >>> * gnu/packages/gl.scm (mesa)[inputs]: Add zstd:lib. >>> [arguments]: Add '-Dzstd=enabled' to configure-flags. >> >> Pushed to the mesa-updates branch, merging progress tracked in >> https://issues.guix.gnu.org/64369. >> >> Thanks. > > I just saw this as the message only went to the bug number. I would not > have merged this yet without checking about build status after the big > texlive updates. I was hoping to avoid a full rebuild so we could merge > to master with substitutes already, after checking post-texlive and the > graft of mesa on master. > > However, doesn't seem like Cuirass has picked up this change and hasn't > built yet. Any idea why? Cuirass at ci.guix.gnu.org seems to be having problems noticing new revisions. At least it's about a day behind for master branch revisions. > My plan right now was going to be to merge master into mesa-updates, see > what the build status looks like, and then decide from there. If there > were not too many rebuilds (say less than 1000) from the updates on > master, we should be able to go ahead soon. And then start on the next > round of patches/updates for mesa-updates. > > On the other hand, if there were lots of rebuilds there are other > patches waiting that should be used also (libdrm, libva if I remember) > as well as ungrafting the change on master. So, preventing waiting and > doing another big rebuild with the changes that have been waiting since > the first build of this branch. > > What do people think? My thought right now was to revert this change > (since it rebuilds all of mesa dependents), merge master in, and check > the status. But I'm confused why Cuirass wasn't building. The mesa package wasn't affected by tex-team-next, so unless there are other inputs to most/all off the packages to which mesa is an input, there shouldn't be many rebuilds. I'd still rebase or merge master in to the branch and see what QA says though. Whether you want to revert this latest commit and try and merge sooner, or wait for things to be built and merge later is up to you though.
Hi Chris, On Thu, Jul 20, 2023 at 05:03 PM, Christopher Baines wrote: > > John Kehayias <john.kehayias@protonmail.com> writes: >> However, doesn't seem like Cuirass has picked up this change and hasn't >> built yet. Any idea why? > > Cuirass at ci.guix.gnu.org seems to be having problems noticing new > revisions. At least it's about a day behind for master branch revisions. > Okay, thanks for the info. So either way will let things build through the weekend probably. >> My plan right now was going to be to merge master into mesa-updates, see >> what the build status looks like, and then decide from there. If there >> were not too many rebuilds (say less than 1000) from the updates on >> master, we should be able to go ahead soon. And then start on the next >> round of patches/updates for mesa-updates. >> >> On the other hand, if there were lots of rebuilds there are other >> patches waiting that should be used also (libdrm, libva if I remember) >> as well as ungrafting the change on master. So, preventing waiting and >> doing another big rebuild with the changes that have been waiting since >> the first build of this branch. >> >> What do people think? My thought right now was to revert this change >> (since it rebuilds all of mesa dependents), merge master in, and check >> the status. But I'm confused why Cuirass wasn't building. > > The mesa package wasn't affected by tex-team-next, so unless there are > other inputs to most/all off the packages to which mesa is an input, > there shouldn't be many rebuilds. I'd still rebase or merge master in to > the branch and see what QA says though. > Right, wasn't sure what the number of rebuilds the texlive updates caused beyond tex-related stuff, so I didn't want to assume. Are we able to rebase and then force push on savannah? I thought not, or is that just for the master branch? If I can force push (I suppose I could just try later), I would leave off 64738 (zstd in mesa) and rebase on master. Otherwise I guess revert and merge master to mesa-updates. > Whether you want to revert this latest commit and try and merge sooner, > or wait for things to be built and merge later is up to you though. > Then either way see what rebuilding looks like. If there isn't much I'd opt for going sooner while we have substitutes already and then gather these other patches for the next merge. Likely waiting for a new mesa update which is probably due very soon. But if things will be rebuilding a lot anyway, I might as well get all these other related changes in to just build once. Thanks Chris, I'll see what to do and keep an eye on the builds. John
John Kehayias <john.kehayias@protonmail.com> writes: > Are we able to rebase and then force push on savannah? I thought not, > or is that just for the master branch? If I can force push (I suppose > I could just try later), I would leave off 64738 (zstd in mesa) and > rebase on master. Otherwise I guess revert and merge master to > mesa-updates. Not directly, but you can delete the branch and then just push as if it is a new branch to effectively force push.
On Thu, Jul 20, 2023 at 06:04 PM, Christopher Baines wrote: > > John Kehayias <john.kehayias@protonmail.com> writes: > >> Are we able to rebase and then force push on savannah? I thought not, >> or is that just for the master branch? If I can force push (I suppose >> I could just try later), I would leave off 64738 (zstd in mesa) and >> rebase on master. Otherwise I guess revert and merge master to >> mesa-updates. > > Not directly, but you can delete the branch and then just push as if it > is a new branch to effectively force push. > Gotcha. That's what I did to rebase on master as of now. Let's see what happens with the build and go from there. Thanks!
On Thu, Jul 20, 2023 at 06:58 PM, 宋文武 wrote: >> >> gnu: mesa: Enable zstd compression for shader cache. >> >> * gnu/packages/gl.scm (mesa)[inputs]: Add zstd:lib. >> [arguments]: Add '-Dzstd=enabled' to configure-flags. > > Pushed to the mesa-updates branch, merging progress tracked in > https://issues.guix.gnu.org/64369. > > Thanks. An update here: I reset mesa-updates to just be the 2 commits from before this patch (that were already built on Cuirass) but rebased on master. Based on how many builds may be new, we can see what to do. I hope it is just a small amount of rebuilds so mesa-updates can be merged as is, and then include this and other patches for the next round. Hope that is okay, especially as I think this is a minor change to mesa, rather than a bug or missing hardware support. John
John Kehayias <john.kehayias@protonmail.com> writes: > On Thu, Jul 20, 2023 at 06:58 PM, 宋文武 wrote: > >>> >>> gnu: mesa: Enable zstd compression for shader cache. >>> >>> * gnu/packages/gl.scm (mesa)[inputs]: Add zstd:lib. >>> [arguments]: Add '-Dzstd=enabled' to configure-flags. >> >> Pushed to the mesa-updates branch, merging progress tracked in >> https://issues.guix.gnu.org/64369. >> >> Thanks. > > An update here: I reset mesa-updates to just be the 2 commits from > before this patch (that were already built on Cuirass) but rebased on > master. Based on how many builds may be new, we can see what to do. I > hope it is just a small amount of rebuilds so mesa-updates can be merged > as is, and then include this and other patches for the next round. > > Hope that is okay, especially as I think this is a minor change to mesa, > rather than a bug or missing hardware support. > That's fine, thank you for take care of it.
On Thu, Jul 20, 2023 at 03:23 PM, John Kehayias wrote: > On Thu, Jul 20, 2023 at 06:04 PM, Christopher Baines wrote: > >> >> John Kehayias <john.kehayias@protonmail.com> writes: >> >>> Are we able to rebase and then force push on savannah? I thought not, >>> or is that just for the master branch? If I can force push (I suppose >>> I could just try later), I would leave off 64738 (zstd in mesa) and >>> rebase on master. Otherwise I guess revert and merge master to >>> mesa-updates. >> >> Not directly, but you can delete the branch and then just push as if it >> is a new branch to effectively force push. >> > > Gotcha. That's what I did to rebase on master as of now. Let's see > what happens with the build and go from there. > > Thanks! I'm not sure what is going on with the CI and QA. <https://ci.guix.gnu.org/> hasn't shown any new evaluations in several days (the workers were stuck at some point too, but nckx gave it a poke and now I don't see any builds left). But when I check with guix weather I do get substitutes. Any ideas what the actual status is? Thanks, John
Hello, On Fri, Jul 21, 2023 at 06:15 PM, 宋文武 wrote: > John Kehayias <john.kehayias@protonmail.com> writes: > >> On Thu, Jul 20, 2023 at 06:58 PM, 宋文武 wrote: >> >>>> >>>> gnu: mesa: Enable zstd compression for shader cache. >>>> >>>> * gnu/packages/gl.scm (mesa)[inputs]: Add zstd:lib. >>>> [arguments]: Add '-Dzstd=enabled' to configure-flags. >>> >>> Pushed to the mesa-updates branch, merging progress tracked in >>> <https://issues.guix.gnu.org/64369>. >>> >>> Thanks. >> >> An update here: I reset mesa-updates to just be the 2 commits from >> before this patch (that were already built on Cuirass) but rebased on >> master. Based on how many builds may be new, we can see what to do. I >> hope it is just a small amount of rebuilds so mesa-updates can be merged >> as is, and then include this and other patches for the next round. >> >> Hope that is okay, especially as I think this is a minor change to mesa, >> rather than a bug or missing hardware support. >> > > That's fine, thank you for take care of it. I ended up adding this (and other changes) to mesa-updates along with a mesa version bump and some other patches. Hopefully merging to master in the next few days once substitutes look good. Thanks! John
Hello, On Mon, Jul 24, 2023 at 03:14 PM, John Kehayias wrote: > I'm not sure what is going on with the CI and QA. > <https://ci.guix.gnu.org/> hasn't shown any new evaluations in several > days (the workers were stuck at some point too, but nckx gave it a > poke and now I don't see any builds left). But when I check with guix > weather I do get substitutes. Any ideas what the actual status is? > Things got unstuck recently at ci.guix.gnu.org was churning away. I saw some failed builds due to what looks like some build farm download issue, e.g. <https://ci.guix.gnu.org/build/1671444/details> which build fine locally. I think this is what blocked a bunch of KDE-related packages as well. A bunch of java packages also failed but the ones I tried also built locally (some had long build logs but I didn't see the cause). So I think x86 is okay, while e.g. aarch64 is slowly chipping away. If that's okay then I'll cherry pick the commits to master today or tomorrow. Thanks for your help Chris!
On Fri, Jul 28, 2023 at 03:29 PM, John Kehayias wrote: > Hello, > > On Mon, Jul 24, 2023 at 03:14 PM, John Kehayias wrote: > >> I'm not sure what is going on with the CI and QA. >> <https://ci.guix.gnu.org/> hasn't shown any new evaluations in several >> days (the workers were stuck at some point too, but nckx gave it a >> poke and now I don't see any builds left). But when I check with guix >> weather I do get substitutes. Any ideas what the actual status is? >> > > Things got unstuck recently at ci.guix.gnu.org was churning away. I saw > some failed builds due to what looks like some build farm download > issue, e.g. <https://ci.guix.gnu.org/build/1671444/details> which build > fine locally. I think this is what blocked a bunch of KDE-related > packages as well. A bunch of java packages also failed but the ones I > tried also built locally (some had long build logs but I didn't see the > cause). > > So I think x86 is okay, while e.g. aarch64 is slowly chipping away. > > If that's okay then I'll cherry pick the commits to master today or > tomorrow. > > Thanks for your help Chris! Pushed to master with 4f0ce65b74a3d28bf6ecbe4c15052dd0de22b284 the final commit from that branch. Thanks! (I'll delete the remote branch for now, until needed again.)
On Tue, Jul 25, 2023 at 09:27 PM, John Kehayias wrote: > Hello, > > On Fri, Jul 21, 2023 at 06:15 PM, 宋文武 wrote: > >> John Kehayias <john.kehayias@protonmail.com> writes: >> >>> On Thu, Jul 20, 2023 at 06:58 PM, 宋文武 wrote: >>> >>>>> >>>>> gnu: mesa: Enable zstd compression for shader cache. >>>>> >>>>> * gnu/packages/gl.scm (mesa)[inputs]: Add zstd:lib. >>>>> [arguments]: Add '-Dzstd=enabled' to configure-flags. >>>> >>>> Pushed to the mesa-updates branch, merging progress tracked in >>>> <https://issues.guix.gnu.org/64369>. >>>> >>>> Thanks. >>> >>> An update here: I reset mesa-updates to just be the 2 commits from >>> before this patch (that were already built on Cuirass) but rebased on >>> master. Based on how many builds may be new, we can see what to do. I >>> hope it is just a small amount of rebuilds so mesa-updates can be merged >>> as is, and then include this and other patches for the next round. >>> >>> Hope that is okay, especially as I think this is a minor change to mesa, >>> rather than a bug or missing hardware support. >>> >> >> That's fine, thank you for take care of it. > > I ended up adding this (and other changes) to mesa-updates along with > a mesa version bump and some other patches. Hopefully merging to > master in the next few days once substitutes look good. > > Thanks! > John Pushed as 090c254fe7713db330636dae2c204c8282207cc8. Thanks!
diff --git a/gnu/packages/gl.scm b/gnu/packages/gl.scm index 1691086e1a..2f54d8553d 100644 --- a/gnu/packages/gl.scm +++ b/gnu/packages/gl.scm @@ -299,7 +299,8 @@ (define-public mesa libxvmc llvm-for-mesa wayland - wayland-protocols)) + wayland-protocols + `(,zstd "lib"))) (native-inputs (list bison flex @@ -362,6 +363,9 @@ (define-public mesa ;; Enable the codecs that were built by default as part of the ;; 21.3.x releases to avoid functionality regressions. "-Dvideo-codecs=vc1dec,h264dec,h264enc,h265dec,h265enc" + + ;; Enable ZSTD compression for shader cache + "-Dzstd=enabled" ;; Also enable the tests. "-Dbuild-tests=true"