[bug#73071,mesa-updates] gnu: mesa: Enable NVK driver for x86_64 architecture.

* gnu/packages/gl.scm (mesa): Enable NVK vulkan driver for x86_64
architecture.
[native-inputs]: Add rust, rust-bindgen-cli, rust-cbindgen-0.26 for x86_64.
[arguments]<#:meson>: Use newer version of meson to satisfy NVK build
requirements.
<#:configure-flags>: Use "auto" for 'vulkan-drivers' flag when target arch is
x86_64.
<#:phases>: Add phase to patch rust subproject sources when target arch is
x86_64.
* gnu/packages/crates-io.scm (rust-paste-1, rust-proc-macro2-1, rust-quote-1,
rust-syn-2, rust-unicode-ident-1): Add a comment about being dependencies of
mesa.

Change-Id: I76e3db5072ca3c718155b188ca64b52766d55f36
---

Hi John,

Thanks for the CC, and sorry for not submitting a patch sooner!

Since last, I have managed to find a better solution to the issue with
substituting rust dependency sources.  Briefly on what I have so far: this
makes use of writing wrap files with just the "directory" property, having
meson fall back to using a local directory that sources are copied to.  Doing
this resulted in an odd behavior with the overlay directories (i.e. important
extra files in subprojects/packagefiles) where they don't seem to be
automatically copied as expected when falling back to local files; however, a
simple copy resolves it, although I thought it would still be good to note and
add a comment about.

I initially made the wrap-file-writing a build utility procedure
'patch-wrap-file' (which can be seen my personal config [1]) as it seemed like
it could be useful for other meson packages, but for this patch I've inlined
it since I'm not confident in its applicability across different projects.
I'd be happy to hear thoughts about it if it might be useful, though.

[1] https://github.com/aurtzy/guix-config/blob/5cf4c9e277f4ab3a8f1ef87ec5ac47a686da0ad2/my-guix/build/utils.scm#L27

Cheers,

aurtzy

 gnu/packages/crates-io.scm |  5 +++
 gnu/packages/gl.scm        | 64 ++++++++++++++++++++++++++++++++++++--
 2 files changed, 67 insertions(+), 2 deletions(-)


base-commit: c1ca70a98024a1e9dd6fedded1d31ad763b8b282

Hi aurtzy!

On Mon, Sep 09, 2024 at 12:12 PM, aurtzy wrote:

> * gnu/packages/gl.scm (mesa): Enable NVK vulkan driver for x86_64
> architecture.
> [native-inputs]: Add rust, rust-bindgen-cli, rust-cbindgen-0.26 for x86_64.
> [arguments]<#:meson>: Use newer version of meson to satisfy NVK build
> requirements.
> <#:configure-flags>: Use "auto" for 'vulkan-drivers' flag when target arch is
> x86_64.
> <#:phases>: Add phase to patch rust subproject sources when target arch is
> x86_64.
> * gnu/packages/crates-io.scm (rust-paste-1, rust-proc-macro2-1, rust-quote-1,
> rust-syn-2, rust-unicode-ident-1): Add a comment about being dependencies of
> mesa.
>
> Change-Id: I76e3db5072ca3c718155b188ca64b52766d55f36
> ---
>
> Hi John,
>
> Thanks for the CC, and sorry for not submitting a patch sooner!
>

No problem, I let things go for a bit too, thanks for this work!

> Since last, I have managed to find a better solution to the issue with
> substituting rust dependency sources.  Briefly on what I have so far: this
> makes use of writing wrap files with just the "directory" property, having
> meson fall back to using a local directory that sources are copied to.  Doing
> this resulted in an odd behavior with the overlay directories (i.e. important
> extra files in subprojects/packagefiles) where they don't seem to be
> automatically copied as expected when falling back to local files; however, a
> simple copy resolves it, although I thought it would still be good to note and
> add a comment about.
>
> I initially made the wrap-file-writing a build utility procedure
> 'patch-wrap-file' (which can be seen my personal config [1]) as it seemed like
> it could be useful for other meson packages, but for this patch I've inlined
> it since I'm not confident in its applicability across different projects.
> I'd be happy to hear thoughts about it if it might be useful, though.
>
> [1] <https://github.com/aurtzy/guix-config/blob/5cf4c9e277f4ab3a8f1ef87ec5ac47a686da0ad2/my-guix/build/utils.scm#L27>
>

Very nice! Yes, more utilities to help with unbundling essentially
will be helpful. I don't know if I ever wrote a wrap file or just
patched the directory bit into the meson file directly. I would have
to look this over, but a utility function (or via keyword argument in
meson-build-system?) would be great.

As for the main patch: I've built locally and see nouveau as a vulkan
driver on x86_64 but not i686. That's about all I can test though. The
change to #:meson means other archs will rebuild too, but I don't know
if they got far anyway (so we can cancel old derivations). Perhaps a
new mesa point release will be out this week? Then I could bump the
version, rebase, and rebuild with this patch. Thoughts?

Other than that, I made a minor tweak to the changelog (remove first
line, name the phase) locally, but have not done a rebase/push on
mesa-updates. Let me give it a few days for other comments and if
there are any other rebuilds to go with it.

Thanks again!
John

> Cheers,
>
> aurtzy
>
>  gnu/packages/crates-io.scm |  5 +++
>  gnu/packages/gl.scm        | 64 ++++++++++++++++++++++++++++++++++++--
>  2 files changed, 67 insertions(+), 2 deletions(-)
>
> diff --git a/gnu/packages/crates-io.scm b/gnu/packages/crates-io.scm
> index 0cdb071ea0..2a40062500 100644
> --- a/gnu/packages/crates-io.scm
> +++ b/gnu/packages/crates-io.scm
> @@ -51869,6 +51869,7 @@ (define-public rust-password-hash-0.2
>                              ("rust-subtle" ,rust-subtle-2)
>                              ("rust-rand-core" ,rust-rand-core-0.6))))))
>
> +;; WARNING: This package is a dependency of mesa.
>  (define-public rust-paste-1
>    (package
>      (name "rust-paste")
> @@ -56703,6 +56704,7 @@ (define-public rust-proc-macro-nested-0.1
>       "Support for nested proc-macro-hack invocations.")
>      (license (list license:expat license:asl2.0))))
>
> +;; WARNING: This package is a dependency of mesa.
>  (define-public rust-proc-macro2-1
>    (package
>      (name "rust-proc-macro2")
> @@ -60144,6 +60146,7 @@ (define-public rust-quinn-udp-0.1
>          ("rust-tokio" ,rust-tokio-1)
>          ("rust-tracing" ,rust-tracing-0.1))))))
>
> +;; WARNING: This package is a dependency of mesa.
>  (define-public rust-quote-1
>    (package
>      (name "rust-quote")
> @@ -76939,6 +76942,7 @@ (define-public rust-symphonia-utils-xiph-0.4
>  codecs and formats.")
>      (license license:mpl2.0)))
>
> +;; WARNING: This package is a dependency of mesa.
>  (define-public rust-syn-2
>    (package
>      (name "rust-syn")
> @@ -86599,6 +86603,7 @@ (define-public rust-unicode-id-0.3
>  according to Unicode Standard Annex #31.")
>      (license (list license:expat license:asl2.0))))
>
> +;; WARNING: This package is a dependency of mesa.
>  (define-public rust-unicode-ident-1
>    (package
>      (name "rust-unicode-ident")
> diff --git a/gnu/packages/gl.scm b/gnu/packages/gl.scm
> index c417f11571..3939f1ba1f 100644
> --- a/gnu/packages/gl.scm
> +++ b/gnu/packages/gl.scm
> @@ -21,6 +21,7 @@
>  ;;; Copyright © 2023, 2024 Zheng Junjie <873216071@qq.com>
>  ;;; Copyright © 2024 Liliana Marie Prikler <liliana.prikler@gmail.com>
>  ;;; Copyright © 2024 Artyom V. Poptsov <poptsov.artyom@gmail.com>
> +;;; Copyright © 2024 aurtzy <aurtzy@gmail.com>
>  ;;;
>  ;;; This file is part of GNU Guix.
>  ;;;
> @@ -41,6 +42,7 @@ (define-module (gnu packages gl)
>    #:use-module (gnu packages)
>    #:use-module (gnu packages autotools)
>    #:use-module (gnu packages bison)
> +  #:use-module (gnu packages build-tools)
>    #:use-module (gnu packages check)
>    #:use-module (gnu packages compression)
>    #:use-module (gnu packages documentation)
> @@ -56,6 +58,7 @@ (define-module (gnu packages gl)
>    #:use-module (gnu packages pkg-config)
>    #:use-module (gnu packages python)
>    #:use-module (gnu packages python-xyz)
> +  #:use-module (gnu packages rust)
>    #:use-module (gnu packages tls)
>    #:use-module (gnu packages video)
>    #:use-module (gnu packages vulkan)
> @@ -75,7 +78,8 @@ (define-module (gnu packages gl)
>    #:use-module (guix gexp)
>    #:use-module (guix utils)
>    #:use-module (ice-9 match)
> -  #:use-module ((srfi srfi-1) #:hide (zip)))
> +  #:use-module ((srfi srfi-1) #:hide (zip))
> +  #:use-module (srfi srfi-26))
>
>  (define-public glu
>    (package
> @@ -352,10 +356,19 @@ (define-public mesa
>                  pkg-config-for-build
>                  wayland
>                  wayland-protocols)
> +          '())
> +      (if (target-x86-64?)
> +          ;; NVK dependencies
> +          (list rust
> +                (module-ref (resolve-interface '(gnu packages rust-apps))
> +                            'rust-bindgen-cli)
> +                (module-ref (resolve-interface '(gnu packages rust-apps))
> +                            'rust-cbindgen-0.26))
>            '())))
>      (outputs '("out" "bin"))
>      (arguments
>       (list
> +      #:meson meson-1.5
>        #:configure-flags
>        #~(list
>           #$@(cond
> @@ -387,7 +400,7 @@ (define-public mesa
>           "-Dshared-glapi=enabled"
>
>           #$@(cond
> -             ((or (target-x86-32?) (target-x86-64?))
> +             ((target-x86-32?)
>                ;; This doesn't include nouveau (which is in "auto") as it needs
>                ;; rust.
>                ;; TODO: Enable nouveau/NVK.
> @@ -481,6 +494,53 @@ (define-public mesa
>                        (("'lp_test_arit', ") ""))))
>                   (_
>                    '((display "No tests to disable on this architecture.\n"))))))
> +         #$@(if (target-x86-64?)
> +                #~((add-after 'unpack 'patch-subproject-sources
> +                     (lambda _
> +                       ;; Patch each relevant subproject source URL in wrapfiles to
> +                       ;; use the store, which avoids an attempt to download them
> +                       ;; mid-build.
> +                       (for-each
> +                        (match-lambda
> +                          ((name source)
> +                           (let ((wrap-file (string-append
> +                                             "subprojects/" name ".wrap"))
> +                                 (subproject-dest (string-append
> +                                                   "subprojects/" name))
> +                                 (overlay-dir (string-append
> +                                               "subprojects/packagefiles/" name)))
> +                             (copy-recursively source subproject-dest)
> +                             ;; Normally when the patch_directory wrap file property
> +                             ;; is specified, meson automatically copies from
> +                             ;; packagefiles, but this is not the case here (only
> +                             ;; happens when downloading source?) so we manually copy
> +                             ;; overlay-dir to subproject-dest.
> +                             (when (file-exists? overlay-dir)
> +                               (copy-recursively overlay-dir subproject-dest))
> +                             (call-with-output-file wrap-file
> +                               (lambda (port)
> +                                 (format port "[wrap-file]
> +directory = ~a
> +"
> +                                         name))))))
> +                        '#+(map (lambda (pkg)
> +                                  (let ((name (package-upstream-name* pkg))
> +                                        (version (package-version pkg)))
> +                                    (list (package-upstream-name* pkg)
> +                                          (file-append pkg
> +                                                       "/share/cargo/src/"
> +                                                       name "-" version))))
> +                                (let ((from-crates-io
> +                                       (cut module-ref
> +                                            (resolve-interface
> +                                             '(gnu packages crates-io))
> +                                            <>)))
> +                                  (list (from-crates-io 'rust-syn-2)
> +                                        (from-crates-io 'rust-unicode-ident-1)
> +                                        (from-crates-io 'rust-quote-1)
> +                                        (from-crates-io 'rust-proc-macro2-1)
> +                                        (from-crates-io 'rust-paste-1))))))))
> +                #~())
>           (add-before 'configure 'fix-dlopen-libnames
>             (lambda* (#:key inputs #:allow-other-keys)
>               (let ((out #$output))
>
> base-commit: c1ca70a98024a1e9dd6fedded1d31ad763b8b282

Hi,

I have an additional question/concern that's come up, but to address 
your comments first:

On 9/15/24 22:59, John Kehayias wrote:

>> Since last, I have managed to find a better solution to the issue with
>> substituting rust dependency sources.  Briefly on what I have so far: this
>> makes use of writing wrap files with just the "directory" property, having
>> meson fall back to using a local directory that sources are copied to.  Doing
>> this resulted in an odd behavior with the overlay directories (i.e. important
>> extra files in subprojects/packagefiles) where they don't seem to be
>> automatically copied as expected when falling back to local files; however, a
>> simple copy resolves it, although I thought it would still be good to note and
>> add a comment about.
>>
>> I initially made the wrap-file-writing a build utility procedure
>> 'patch-wrap-file' (which can be seen my personal config [1]) as it seemed like
>> it could be useful for other meson packages, but for this patch I've inlined
>> it since I'm not confident in its applicability across different projects.
>> I'd be happy to hear thoughts about it if it might be useful, though.
>>
>> [1]<https://github.com/aurtzy/guix-config/blob/5cf4c9e277f4ab3a8f1ef87ec5ac47a686da0ad2/my-guix/build/utils.scm#L27>
>>
> Very nice! Yes, more utilities to help with unbundling essentially
> will be helpful. I don't know if I ever wrote a wrap file or just
> patched the directory bit into the meson file directly. I would have
> to look this over, but a utility function (or via keyword argument in
> meson-build-system?) would be great.
Okay! I'll have a look at the keyword argument approach as well and see 
if I can make a proposal for changes and further comments sometime soon.
> As for the main patch: I've built locally and see nouveau as a vulkan
> driver on x86_64 but not i686. That's about all I can test though. The
> change to #:meson means other archs will rebuild too, but I don't know
> if they got far anyway (so we can cancel old derivations). Perhaps a
> new mesa point release will be out this week? Then I could bump the
> version, rebase, and rebuild with this patch. Thoughts?

I didn't consider this when writing the patch, but would it be a good 
idea to only use meson-1.5 for x86_64? We could avoid rebuilds that way.

Looks like a new mesa release is planned for the 18th [1].

> Other than that, I made a minor tweak to the changelog (remove first
> line, name the phase) locally, but have not done a rebase/push on
> mesa-updates. Let me give it a few days for other comments and if
> there are any other rebuilds to go with it.

Regarding my concern (unrelated to NVK): I've had issues with a nonfree 
application segfaulting which I narrowed down to a mesa commit from a 
few months ago [2] [3] and hit a dead end with (until today). It's only 
this single program which I had ever noticed the issue with so I pushed 
it aside, but it just occurred to me that this seems like a change that 
could affect other cases my daily driving has not covered.

Further investigation today has revealed that adding libglvnd as an 
input to mesa fixes this, although from what I can tell reading past 
discussions on adding it, it seems like a complicated issue [4]. 
Thoughts about this situation?

[1] https://docs.mesa3d.org/release-calendar.html#calendar

[2] https://gitlab.freedesktop.org/mesa/mesa/-/issues/11666

[3] 
https://gitlab.freedesktop.org/mesa/mesa/-/commit/50fc7cc290db49288b318c116c6fdc3daaf801e9

[4] https://issues.guix.gnu.org/49339#3

Cheers,

aurtzy