| Message ID | cover.1709899566.git.aurtzy@gmail.com |
|---|---|
| Headers | show |
| Series | gnu: mesa: Update to 24.0.2. | expand |
On Fri, Mar 08, 2024 at 07:32:39AM -0500, aurtzy wrote: > Hello! > > This series of patches updates Mesa to the latest stable version with the NVK > Vulkan driver for NVIDIA GPUs enabled for x86_64 systems. > > Mesa 24.0.2 depends on a newer version of libdrm (>= 2.4.119), which is > pending in another issue (https://issues.guix.gnu.org/68845). > > Some packages (rust-syn-2.0.39, rust-proc-macro2-1.0.70, rust-quote-1.0.33) > were added specifically to be used as sources that satisfy explicit versions > of dependencies required by Mesa (for NVK). Would it be better to inline > origins in the change-subproject-sources phase? The same question also > applies for the rust-unicode-ident-1 source. Are there other architectures which have rust based drivers? x86_64 isn't the only architecture which has rust building on it. > Currently, the Rust dependencies are found in the build by substituting the > fallback https://* URLs in the relevant ./subprojects/*.wrap files with > file://*, then relying on the fallback to "download" from the store. It > seemed a bit hacky doing it this way, but I'm not aware of a better solution. The crates are also available in %output/share/cargo/registry/name-version.crate, although I can't think of a good way to address them by name without using find-files. I would personally replace the versions requested by mesa with whatever version we happen to have in guix so that we don't have to add special versions just for mesa. > A newer Meson version is also required for NVK (>= 1.3.0), which this patch > series adds a new variable for. > > x86_64-linux and i686-linux have been tested to successfully build. > > Cheers, > > aurtzy > > aurtzy (6): > gnu: Add meson-1.3. > gnu: Add rust-syn-2.0.39. > gnu: Add rust-proc-macro2-1.0.70. > gnu: Add rust-quote-1.0.33. > gnu: Add rust-bindgen-cli-0.69. > gnu: mesa: Update to 24.0.2. > > gnu/packages/build-tools.scm | 14 ++++++ > gnu/packages/crates-io.scm | 95 ++++++++++++++++++++++++++++++++++++ > gnu/packages/gl.scm | 68 +++++++++++++++++++------- > 3 files changed, 160 insertions(+), 17 deletions(-) > > > base-commit: 0b5cbca689b2f91949e191ca256ffc9bb48c1141 > -- > 2.41.0 > > > >
On 3/12/24 04:11, Efraim Flashner wrote: > Are there other architectures which have rust based drivers? x86_64 > isn't the only architecture which has rust building on it. There doesn't appear to be any official documentation stating architecture requirements for Mesa with NVK/Rust, however I have added few new inputs other than rust to get NVK working. I've only done extensive testing with x86_64 so I was unsure of potential issues with including this on other architectures (other than i686 not building rust). > The crates are also available in > %output/share/cargo/registry/name-version.crate, although I can't think > of a good way to address them by name without using find-files. > > I would personally replace the versions requested by mesa with whatever > version we happen to have in guix so that we don't have to add special > versions just for mesa. I have/had tried a few approaches to use the crates already available in Guix with no success so far. I've outlined the approaches below; still looking into solutions, but perhaps there's something I'm missing or haven't tried yet? - Simply including crates as (native-)inputs does not make them discoverable by meson. - Mesa uses these *.wrap files which specify the rust dependency versions, source URLs, and tar hashes. I currently get the build working by relying on meson to fall back to "downloading" from a patched source URL (pointing to store), although it still has to match the hash. - I recently discovered a way to disable the hash requirement so I could use a different input version (i.e. one from Guix), but doing it causes "File src/lib.rs does not exist" errors. I'm still looking into this right now, as it seems promising. - Old IRC logs point to projects like newsboat and librsvg which also mix cargo with with another build system, but these start with cargo-build-system with phases added/replaced from the second build system. Cargo.toml doesn't exist in Mesa either (which cargo-build-system seems to depend on), so experimenting with using cargo-build-system didn't yield much. I wanted to look more into the third bullet before responding, but I felt it would be unfortunate to have this information rot while trying to make time for hacking - hopefully it's still useful. Cheers, aurtzy
Hi aurtzy and Efraim, On Wed, Mar 20, 2024 at 09:52 PM, aurtzy wrote: > On 3/12/24 04:11, Efraim Flashner wrote: > > Are there other architectures which have rust based drivers? x86_64 > isn't the only architecture which has rust building on it. > > There doesn't appear to be any official documentation stating architecture requirements for Mesa with > NVK/Rust, however I have added few new inputs other than rust to get NVK working. I've only done extensive > testing with x86_64 so I was unsure of potential issues with including this on other architectures (other than > i686 not building rust). > I guess this is the main question, if we will try to enable the NVK driver for other architectures, if that is supported. We could also leave it for the known working x86_64 to start. I would say we could provide a mesa variant package for testing, but that might be difficult as far as I know (e.g. trying to get Xorg to use a different mesa package looked difficult from what I saw others try). We could try just checking for where rust is available (is that just supported-systems for the rust package, or do we have other logic?) and building with NVK to see what fails... Though with how long it can take us to build on other architectures, that might take a while to find out and then correct. > The crates are also available in > %output/share/cargo/registry/name-version.crate, although I can't think > of a good way to address them by name without using find-files. > > I would personally replace the versions requested by mesa with whatever > version we happen to have in guix so that we don't have to add special > versions just for mesa. > > I have/had tried a few approaches to use the crates already available in Guix with no success so far. I've > outlined the approaches below; still looking into solutions, but perhaps there's something I'm missing or > haven't tried yet? > > - Simply including crates as (native-)inputs does not make them discoverable by meson. > > - Mesa uses these *.wrap files which specify the rust dependency versions, source URLs, and tar hashes. I > currently get the build working by relying on meson to fall back to "downloading" from a patched source URL > (pointing to store), although it still has to match the hash. > > - I recently discovered a way to disable the hash requirement so I could use a different input version (i.e. one > from Guix), but doing it causes "File src/lib.rs does not exist" errors. I'm still looking into this right now, as it > seems promising. > > - Old IRC logs point to projects like newsboat and librsvg which also mix cargo with with another build > system, but these start with cargo-build-system with phases added/replaced from the second build system. > Cargo.toml doesn't exist in Mesa either (which cargo-build-system seems to depend on), so experimenting > with using cargo-build-system didn't yield much. > > I wanted to look more into the third bullet before responding, but I felt it would be unfortunate to have this > information rot while trying to make time for hacking - hopefully it's still useful. > > Cheers, > > aurtzy Thanks for this additional info, aurtzy, and your work on this! Efraim, any thoughts on the rust related stuff based on these other attempts? I'm not familiar enough with rust, rust packaging, or what mesa is doing in the meson builds to comment right now. I would like to get the build farm cranking on the updates I have queued for mesa-updates (cairo, libdrm, mesa, vulkan). We could also do just the version update of mesa to start, or just NVK on x86_64, leaving future changes for the next round. I don't have a preference myself, other than wanting to get this branch moving with these updates. Thoughts? And thanks both of you again! John
On Thu, Mar 21, 2024 at 04:39:29AM +0000, John Kehayias wrote: > Hi aurtzy and Efraim, > > On Wed, Mar 20, 2024 at 09:52 PM, aurtzy wrote: > > > On 3/12/24 04:11, Efraim Flashner wrote: > > > > Are there other architectures which have rust based drivers? x86_64 > > isn't the only architecture which has rust building on it. > > > > There doesn't appear to be any official documentation stating architecture requirements for Mesa with > > NVK/Rust, however I have added few new inputs other than rust to get NVK working. I've only done extensive > > testing with x86_64 so I was unsure of potential issues with including this on other architectures (other than > > i686 not building rust). > > > > I guess this is the main question, if we will try to enable the NVK > driver for other architectures, if that is supported. We could also > leave it for the known working x86_64 to start. I would say we could > provide a mesa variant package for testing, but that might be > difficult as far as I know (e.g. trying to get Xorg to use a different > mesa package looked difficult from what I saw others try). > > We could try just checking for where rust is available (is that just > supported-systems for the rust package, or do we have other logic?) > and building with NVK to see what fails... Though with how long it can > take us to build on other architectures, that might take a while to > find out and then correct. We have (supported-package? package) to check if a package is supported. Then we can wrap the phase in #$@(if (assoc-ref inputs "rust") ...) and it should all just work. > > The crates are also available in > > %output/share/cargo/registry/name-version.crate, although I can't think > > of a good way to address them by name without using find-files. > > > > I would personally replace the versions requested by mesa with whatever > > version we happen to have in guix so that we don't have to add special > > versions just for mesa. > > > > I have/had tried a few approaches to use the crates already available in Guix with no success so far. I've > > outlined the approaches below; still looking into solutions, but perhaps there's something I'm missing or > > haven't tried yet? > > > > - Simply including crates as (native-)inputs does not make them discoverable by meson. > > > > - Mesa uses these *.wrap files which specify the rust dependency versions, source URLs, and tar hashes. I > > currently get the build working by relying on meson to fall back to "downloading" from a patched source URL > > (pointing to store), although it still has to match the hash. > > > > - I recently discovered a way to disable the hash requirement so I could use a different input version (i.e. one > > from Guix), but doing it causes "File src/lib.rs does not exist" errors. I'm still looking into this right now, as it > > seems promising. > > > > - Old IRC logs point to projects like newsboat and librsvg which also mix cargo with with another build > > system, but these start with cargo-build-system with phases added/replaced from the second build system. > > Cargo.toml doesn't exist in Mesa either (which cargo-build-system seems to depend on), so experimenting > > with using cargo-build-system didn't yield much. > > > > I wanted to look more into the third bullet before responding, but I felt it would be unfortunate to have this > > information rot while trying to make time for hacking - hopefully it's still useful. I also tried a couple of different options. The one that I most want involved using with-output-to-file to rewrite the wrap file and replacing all the fields. I borrowed the file-sha256 function from guix/build/cargo-utils.scm to get the source_hash. In the end I wasn't able to get the gexp and un-gexp bits working to actually get the file written. When I kept a failed build I saw that the 'directory' field is the directory into which meson writes the meson.build file, which is why using a different version of the rust crate caused problems with src/lib.rs not existing. I suppose we could start from your patch and then, after running substitute, extract the tarball into either a hardcoded path (determined after manually reading the sources) or we can extract the 'directory' field by reading the sources and then untar the source there. > > Cheers, > > > > aurtzy > > Thanks for this additional info, aurtzy, and your work on this! > > Efraim, any thoughts on the rust related stuff based on these other > attempts? I'm not familiar enough with rust, rust packaging, or what > mesa is doing in the meson builds to comment right now. > > I would like to get the build farm cranking on the updates I have > queued for mesa-updates (cairo, libdrm, mesa, vulkan). We could also > do just the version update of mesa to start, or just NVK on x86_64, > leaving future changes for the next round. I don't have a preference > myself, other than wanting to get this branch moving with these > updates. > > Thoughts? Looking at qa.guix.gnu.org I believe that gnome-team is going to merge soon, and then the emacs-team is up next. I would prefer to use the already packaged crates but we can go ahead and use the ones from the patchset for now and work out swapping them later. As far as which architectures, I think I'd start with x86_64 only first. And sprinkle a couple of TODOs around. > And thanks both of you again! > John >