[bug#48325,v5,5/5] DRAFT gnu: julia: Update to 1.6.1.

* gnu/packages/julia.scm (julia): Update to 1.6.1.
[arguments]<#:phases>: Fixed paths to shared objects in
'shared-objects-paths phase, Fix tests that did not propagate
GUIX_LOCPATH in 'fix-setenv, Add symlinks to libs in 'symlink-libraries
[arguments]<:make-flags>: Added generic,-cx16 target for better compatibility
[inputs]: Add nghttp2:lib, Add gfortran:lib, Add curl-ssh, Remove arpack-ng.
[native-inputs]: Add nss-certs.

* gnu/packages/julia.scm (curl-ssh): New variable.

* gnu/packages/julia.scm (llvm-julia): Update to 11.0.0
[source]: Move julia specific patches to a julia-patches phase.
[arguments]<#:phases>: Apply julia specific patches in 'julia-patches phase

Co-Authored-By: Nicolò Balzarotti <nicolo@nixo.xyz>.
Co-Authored-By: zimoun <zimon.toutoune@gmail.com>.
---
 gnu/packages/julia.scm | 353 ++++++++++++++++++++++++++++-------------
 1 file changed, 239 insertions(+), 114 deletions(-)

--
2.32.0

Jean-Baptiste Volatier schreef op za 03-07-2021 om 17:41 [+0000]:
> +               (invoke "ln" "-s" (string-append (assoc-ref inputs "p7zip") "/bin/7z") "usr/bin/")

No need to shell out here. You can use the 'symlink' procedure,
see info '(guile)File System'.  Likewise for other uses of (invoke "ln" "-s" ...)

Greetings,
Maxime.

Jean-Baptiste Volatier schreef op za 03-07-2021 om 17:41 [+0000]:
> +       ("nss-certs" ,nss-certs)

Preferably not if it can be avoided.

Users should be able to curate their own bundle of certificates
instead of relying on Mozilla, though in practice people will
just let Mozilla handle that.  Or, on multi-user systems, just
rely on the administrator to choose a bundle of certificates,
install it globally and update it regularily.

Also, it should be possible to update the certificate bundle
(e.g. if it turns out some root was or became evil or something,
or was compromised) quickly, without going through a world rebuild.
So dependencies on "nss-certs" should be avoided.

What are the reasons for adding "nss-certs" here?

Greetings,
Maxime

On Saturday, July 3rd, 2021 at 9:54 PM, Maxime Devos <maximedevos@telenet.be> wrote:

> What are the reasons for adding "nss-certs" here?

nss-certs is added to address this: https://github.com/JuliaLang/julia/issues/40185
Quoting Sacha0 from github:

> IIRC generate_precompile builds a mock registry to avoid going to network, but
> the cert issue that nico202 mentioned breaks that mock setup, and downstream
> operations subsequently try to go to network.

It is only needed to build the julia package.
Cheers,
JB.

Jean-Baptiste Volatier schreef op zo 04-07-2021 om 07:17 [+0000]:
> On Saturday, July 3rd, 2021 at 9:54 PM, Maxime Devos <maximedevos@telenet.be> wrote:
> 
> > What are the reasons for adding "nss-certs" here?
> 
> nss-certs is added to address this: https://github.com/JuliaLang/julia/issues/40185
> Quoting Sacha0 from github:
> 
> > IIRC generate_precompile builds a mock registry to avoid going to network, but
> > the cert issue that nico202 mentioned breaks that mock setup, and downstream
> > operations subsequently try to go to network.
> 
> It is only needed to build the julia package.
> Cheers,
> JB.

I see. The following still applies though:

‘Also, it should be possible to update the certificate bundle
(e.g. if it turns out some root was or became evil or something,
or was compromised) quickly, without going through a world rebuild.
So dependencies on "nss-certs" should be avoided.’

So perhaps you could define a "nss-certs-for-tests" variable
(currently just pointing to nss-certs), and add "nss-certs-for-tests" to
#:disallowed-references to prevent "nss-certs-for-tests" being used at
run-time?

I don't quite see _why_ Julia needs certificates at build time though ...
Also, I wonder if this causes reproducibility issues --- if we try to rebuild
julia after, say, 50 years, e.g. by using "guix time-machine", wouldn't the
certificates become invalid, and would therefore the build of Julia fail?

Greetings,
Maxime.

On Sunday, July 4th, 2021 at 1:43 PM, Maxime Devos <maximedevos@telenet.be> wrote:

> So perhaps you could define a "nss-certs-for-tests" variable
> (currently just pointing to nss-certs), and add "nss-certs-for-tests" to
> #:disallowed-references to prevent "nss-certs-for-tests" being used at
> run-time?

Ok, I did not know about #:disallowed-references.

> I don't quite see why Julia needs certificates at build time though ...

I can only speculate, but my guess is: julia is compiled just-in-time, functions
are compiled first time they are run. For some common function the build
procedure runs some sample code in order to compile them ahead of time. If
certificates are not present there are some functions that do not get compiled
ahead of time and then the build procedure fails.

> Also, I wonder if this causes reproducibility issues --- if we try to rebuild
> julia after, say, 50 years, e.g. by using "guix time-machine", wouldn't the
> certificates become invalid, and would therefore the build of Julia fail?

I thought that guix build was running with time set 1st January 1970 so that
should not happen? Or am I mistaken?

Also current julia package is also not reproducible it seems. [1]

[1] https://issues.guix.gnu.org/22304

Cheers,
JB.

Jean-Baptiste Volatier schreef op zo 04-07-2021 om 17:47 [+0000]:
> 
[...]
> > Also, I wonder if this causes reproducibility issues --- if we try to rebuild
> > julia after, say, 50 years, e.g. by using "guix time-machine", wouldn't the
> > certificates become invalid, and would therefore the build of Julia fail?
> 
> I thought that guix build was running with time set 1st January 1970 so that
> should not happen? Or am I mistaken?

There is an issue on https://issues.guix.gnu.org on using time namespaces
to always run with with time set to 1970 (or some other date).  I can't
find it anymore though.  The problem was that the ‘real time’ clock could
not be adjusted.

Maybe we could rewind the other clocks anyway, even if the ‘real time clock’
cannot be adjusted?  Or maybe the inconsistency would cause problems ...

SOURCE_DATE_EPOCH is set to 1 jan 1970 and likewise for the 'mtime'
and 'atime' of files in /gnu/store (or was it some other date?) though.

Greetings,
Maxime.

On Sat, Jul 03, 2021 at 05:41:58PM +0000, Jean-Baptiste Volatier via Guix-patches via wrote:
> * gnu/packages/julia.scm (julia): Update to 1.6.1.
> [arguments]<#:phases>: Fixed paths to shared objects in
> 'shared-objects-paths phase, Fix tests that did not propagate
> GUIX_LOCPATH in 'fix-setenv, Add symlinks to libs in 'symlink-libraries
> [arguments]<:make-flags>: Added generic,-cx16 target for better compatibility
> [inputs]: Add nghttp2:lib, Add gfortran:lib, Add curl-ssh, Remove arpack-ng.
> [native-inputs]: Add nss-certs.
> 
> * gnu/packages/julia.scm (curl-ssh): New variable.
> 
> * gnu/packages/julia.scm (llvm-julia): Update to 11.0.0
> [source]: Move julia specific patches to a julia-patches phase.
> [arguments]<#:phases>: Apply julia specific patches in 'julia-patches phase
> 
> Co-Authored-By: Nicolò Balzarotti <nicolo@nixo.xyz>.
> Co-Authored-By: zimoun <zimon.toutoune@gmail.com>.
> ---
>  gnu/packages/julia.scm | 353 ++++++++++++++++++++++++++++-------------
>  1 file changed, 239 insertions(+), 114 deletions(-)
> 
> diff --git a/gnu/packages/julia.scm b/gnu/packages/julia.scm
> index 13c9f7baf1..9903bb009b 100644
> --- a/gnu/packages/julia.scm
> +++ b/gnu/packages/julia.scm
> @@ -1,9 +1,11 @@
>  ;;; GNU Guix --- Functional package management for GNU
>  ;;; Copyright © 2015, 2016, 2017 Ricardo Wurmus <rekado@elephly.net>
>  ;;; Copyright © 2016, 2020 Efraim Flashner <efraim@flashner.co.il>
> -;;; Copyright © 2020 Nicolò Balzarotti <nicolo@nixo.xyz>
> +;;; Copyright © 2020, 2021 Nicolò Balzarotti <nicolo@nixo.xyz>
>  ;;; Copyright © 2020 Tim Howes <timhowes@lavabit.com>
>  ;;; Copyright © 2020 Tobias Geerinckx-Rice <me@tobias.gr>
> +;;; Copyright © 2021 Jean-Baptiste Volatier <jbv@pm.me>
> +;;; Copyright © 2021 Simon Tournier <zimon.toutoune@gmail.com>
>  ;;;
>  ;;; This file is part of GNU Guix.
>  ;;;
> @@ -30,6 +32,7 @@
>    #:use-module (gnu packages)
>    #:use-module (gnu packages algebra)
>    #:use-module (gnu packages base)
> +  #:use-module (gnu packages certs)
>    #:use-module (gnu packages compression)
>    #:use-module (gnu packages curl)
>    #:use-module (gnu packages elf)
> @@ -48,14 +51,15 @@
>    #:use-module (gnu packages ssh)
>    #:use-module (gnu packages tls)
>    #:use-module (gnu packages version-control)
> +  #:use-module (gnu packages web)
>    #:use-module (gnu packages wget)
>    #:use-module (ice-9 match))
> 
>  (define libuv-julia
> -  (let ((commit "1fcc6d66f9df74189c74d3d390f02202bb7db953")
> -        (revision "2"))
> +  (let ((commit "fb3e3364c33ae48c827f6b103e05c3f0e78b79a9")
> +        (revision "3"))
>      ;; When upgrading Julia, also upgrade this.  Get the commit from
> -    ;; https://github.com/JuliaLang/julia/blob/v1.5.2/deps/libuv.version
> +    ;; https://github.com/JuliaLang/julia/blob/v1.6.0/deps/libuv.version
>      (package
>        (inherit libuv)
>        (name "libuv-julia")
> @@ -68,7 +72,7 @@
>                  (file-name (string-append name "-" version "-checkout"))
>                  (sha256
>                   (base32
> -                  "040l7f1hk7xyza11sry5cj4fhw05na949865axqqhxnifdvnmfji"))))
> +                  "1kqpn19d20aka30h6q5h8lnzyp0vw0xzgx0wm4w2r5j6yf76m2hr"))))
>        (build-system gnu-build-system)
>        (arguments
>         (substitute-keyword-arguments (package-arguments libuv)
> @@ -97,7 +101,9 @@
>           (julia-patch "libunwind-prefer-extbl"
>                        "0lr4dafw8qyfh8sw8hhbwkql1dlhqv8px7k81y2l20hhxfgnh2m1")
>           (julia-patch "libunwind-static-arm"
> -                      "1jk3bmiw61ypcchqkk1fyg5wh8wpggk574wxyfyaic870zh3lhgq")))))
> +                      "1jk3bmiw61ypcchqkk1fyg5wh8wpggk574wxyfyaic870zh3lhgq")
> +         (julia-patch "libunwind-cfa-rsp"
> +                      "1aswjhvysahhldbzh1afbf0hsjxrvs6xidsz2i7s1cjkjbdiia1z")))))
>      (home-page "https://github.com/JuliaLang/tree/master/deps/")))
> 
>  (define (julia-patch-url version name)
> @@ -105,7 +111,7 @@
>                   "/deps/patches/" name ".patch"))
> 
>  (define (julia-patch name sha)
> -  (let ((version "1.5.3"))
> +  (let ((version "1.6.1"))
>      (origin (method url-fetch)
>              (uri (julia-patch-url version name))
>              (sha256 (base32 sha))
> @@ -113,52 +119,10 @@
> 
>  (define llvm-julia
>    (package
> -    (inherit llvm-9)
> +    (inherit llvm-11)
>      (name "llvm-julia")
> -    (source (origin
> -              (inherit (package-source llvm-9))
> -              ;; Those patches are inside the Julia source repo.
> -              ;; They are _not_ Julia specific (https://github.com/julialang/julia#llvm)
> -              ;; but they are required to build Julia.
> -              ;; Discussion: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=919628
> -              (patches
> -               (map (match-lambda
> -                      ((name hash)
> -                       (julia-patch name hash)))
> -                    (list
> -                     '("llvm-D27629-AArch64-large_model_6.0.1"
> -                       "1qrshmlqvnasdyc158vfn3hnbigqph3lsq7acb9w8lwkpnnm2j4z")
> -                     '("llvm8-D34078-vectorize-fdiv"
> -                       "19spqc3xsazn1xs9gpcgv9ldadfkv49rmc5khl7sf1dlmhgi4602")
> -                     '("llvm-7.0-D44650"
> -                       "1h55kkmkiisfj6sk956if2bcj9s0v6n5czn8dxb870vp5nccj3ir")
> -                     '("llvm9-D50010-VNCoercion-ni"
> -                       "1s1d3sjsiq4vxg7ncy5cz56zgy5vcq6ls3iqaiqkvr23wyryqmdx")
> -                     '("llvm-exegesis-mingw"
> -                       "0ph1cj1j7arvf1xq2xcr7qf9g0cpdl14fincgr67vpi520zvd3vp")
> -                     '("llvm-test-plugin-mingw"
> -                       "12z738cnahbf6n381im7i0hxp1m6k9hrnfjlmq9sac46nxly9gnj")
> -                     '("llvm7-revert-D44485"
> -                       "0f59kq3p3mpwsbmskypbi4zn01l6ig0x7v2rjp08k2r8z8m6fa8n")
> -                     '("llvm-8.0-D66657-codegen-degenerate"
> -                       "1n1ddx19h90bbpimdyd9dh8fsm6gb93xxyqm4ljkxa1k3cx2vm72")
> -                     '("llvm-8.0-D71495-vectorize-freduce"
> -                       "1zff08wvji9lnpskk4b3p5zyjsy5hhy23ynxjqlj9dw7jvvfrf0p")
> -                     '("llvm-D75072-SCEV-add-type"
> -                       "029a3fywsm233vf48mscina24idd50dc75wr70lmimrhwnw27p0z")
> -                     '("llvm-9.0-D65174-limit-merge-stores"
> -                       "04bff1mnblfj9mxfdwr1qdnw3i3szmp60gnhxwas5y68qg33z6j0")
> -                     '("llvm9-D71443-PPC-MC-redef-symbol"
> -                       "1c93nv7rgc9jg5mqrnvv08xib1789qvlql94fwggh18mp3b9hbgy")
> -                     '("llvm-9.0-D78196"
> -                       "08a43hyg7yyqjq2vmfsmppf34xcz60wq6y9zw5fdyhw2h1mcnmns")
> -                     '("llvm-julia-tsan-custom-as"
> -                       "0awh40kf6lm4wn1nsjd1bmhfwq7rqj811szanp2xkpspykw9hg9s")
> -                     '("llvm-9.0-D85499"
> -                       "0vxlr35srvbvihlgrxq15v6dylp90vgi0qahj22j01jgqmdasjkm"))))
> -              (patch-flags '("-p1"))))
>      (arguments
> -     (substitute-keyword-arguments (package-arguments llvm-9)
> +     (substitute-keyword-arguments (package-arguments llvm-11)
>         ((#:configure-flags flags)
>          `(list ;; Taken from NixOS. Only way I could get libLLVM-6.0.so
>             "-DCMAKE_BUILD_TYPE=Release"
> @@ -177,7 +141,61 @@
>             ;; "-DLLVM_DEFAULT_TARGET_TRIPLE=${stdenv.hostPlatform.config}"
>             ;; "-DLLVM_EXPERIMENTAL_TARGETS_TO_BUILD=WebAssembly"
>             "-DLLVM_ENABLE_DUMP=ON"
> -           "-DLLVM_LINK_LLVM_DYLIB=ON"))))))
> +           "-DLLVM_LINK_LLVM_DYLIB=ON"))
> +       ((#:phases phases)
> +        `(modify-phases ,phases
> +           ;; applying patches from julia
> +           ;; list of patches can be found in deps/llvm.mk in julia source
> +           (add-after 'unpack 'julia-patches
> +             (lambda* (#:key inputs outputs #:allow-other-keys)
> +               (let ((patch
> +                      (lambda (patchname flag)
> +                        (invoke "patch" flag "-i"
> +                                (string-append
> +                                 "julia-src/deps/patches/"
> +                                 patchname
> +                                 ".patch")))))
> +                 (mkdir-p "julia-src")
> +                 (invoke "tar" "xf"
> +                         (assoc-ref inputs "julia-source")
> +                         "-C" "julia-src" "--strip-components=1")
> +                 (map (lambda (patchname)
> +                        (patch patchname "-p1"))
> +                      (list "llvm-D27629-AArch64-large_model_6.0.1"
> +                            "llvm8-D34078-vectorize-fdiv"
> +                            "llvm-7.0-D44650"
> +                            "llvm-6.0-DISABLE_ABI_CHECKS"
> +                            "llvm9-D50010-VNCoercion-ni"
> +                            "llvm7-revert-D44485"
> +                            "llvm-11-D75072-SCEV-add-type"
> +                            "llvm-julia-tsan-custom-as"
> +                            "llvm-D80101"
> +                            "llvm-D84031"
> +                            "llvm-10-D85553"
> +                            "llvm-10-unique_function_clang-sa"
> +                            "llvm-11-D85313-debuginfo-empty-arange"
> +                            "llvm-11-D90722-rtdyld-absolute-relocs"
> +                            "llvm-invalid-addrspacecast-sink"
> +                            "llvm-11-D92906-ppc-setjmp"
> +                            "llvm-11-PR48458-X86ISelDAGToDAG"
> +                            "llvm-11-D93092-ppc-knownbits"
> +                            "llvm-11-D93154-globalisel-as"
> +                            "llvm-11-ppc-half-ctr"
> +                            "llvm-11-ppc-sp-from-bp"
> +                            "llvm-rGb498303066a6-gcc11-header-fix"
> +                            "llvm-11-D94813-mergeicmps"
> +                            "llvm-11-D94980-CTR-half"
> +                            "llvm-11-D94058-sext-atomic-ops"
> +                            "llvm-11-D96283-dagcombine-half"))
> +                 (map (lambda (patchname)
> +                        (patch patchname "-p2"))
> +                      (list "llvm-11-AArch64-FastIsel-bug"
> +                            "llvm-11-D97435-AArch64-movaddrreg"
> +                            "llvm-11-D97571-AArch64-loh"
> +                            "llvm-11-aarch64-addrspace")))))))))
> +    (inputs
> +     `(("julia-source" ,(package-source julia))
> +       ,@(package-inputs llvm-11)))))
> 
>  (define-public libwhich
>    (package
> @@ -220,10 +238,22 @@
>  libraries.  It is also a bit like @code{ldd} and @code{otool -L}.")
>      (license license:expat)))
> 

IMO since this package inherits from curl it should go next to curl, but
be a hidden package.

> +(define curl-ssh
> +  (package
> +    (inherit curl)
> +    (arguments
> +     (substitute-keyword-arguments (package-arguments curl)
> +       ((#:configure-flags flags)
> +        `(cons "--with-libssh2" ,flags))))
> +    (inputs
> +     `(("libssh2" ,libssh2)
> +       ,@(package-inputs curl)))
> +    ))
> +
>  (define-public julia
>    (package
>      (name "julia")
> -    (version "1.5.3")
> +    (version "1.6.1")
>      (source (origin
>                (method url-fetch)
>                (uri (string-append
> @@ -231,9 +261,7 @@ libraries.  It is also a bit like @code{ldd} and @code{otool -L}.")
>                      version "/julia-" version ".tar.gz"))
>                (sha256
>                 (base32
> -                "1zmim82x9kkdcgn0cdi01hmzi59zbszy1sqlygb86xq4hc1n66dy"))
> -              (patches
> -               (search-patches "julia-SOURCE_DATE_EPOCH-mtime.patch"))))
> +                "1mfzbjyqcmx7wb1sa7qab5fl78yzd7ap088krqbphbwvpn880srn"))))
>      (build-system gnu-build-system)
>      (arguments
>       `(#:test-target "test"
> @@ -259,14 +287,13 @@ libraries.  It is also a bit like @code{ldd} and @code{otool -L}.")
>                       (string-join (map (lambda (pkg)
>                                           (string-append (assoc-ref inputs pkg)
>                                                          "/lib"))
> -                                       '("arpack-ng" "curl" "dsfmt"
> +                                       '("curl" "dsfmt"
>                                           "gmp" "lapack"
> -                                         "libssh2" "libgit2"
> +                                         "libssh2" "libnghttp2" "libgit2"
>                                           "mbedtls" "mpfr"
>                                           "openblas" "openlibm" "pcre2"
> -                                         "suitesparse"))
> -                                  ":"))
> -             #t))
> +                                         "suitesparse" "libfortran"))
> +                                  ":"))))
>           ;; FIXME: Building the documentation requires Julia packages that
>           ;; would be downloaded from the Internet.  We should build them in a
>           ;; separate build phase.
> @@ -304,7 +331,7 @@ libraries.  It is also a bit like @code{ldd} and @code{otool -L}.")
>                  "$(BUILDDIR)/$(EXENAME)-debug\\$\\(EXE\\): $(DOBJS) $(LLT_debug)"))
> 
>               ;; The REPL must be linked with libuv.
> -             (substitute* "ui/Makefile"
> +             (substitute* "cli/Makefile"
>                 (("JLDFLAGS \\+= ")
>                  (string-append "JLDFLAGS += "
>                                 (assoc-ref %build-inputs "libuv")
> @@ -320,27 +347,82 @@ libraries.  It is also a bit like @code{ldd} and @code{otool -L}.")
>               (substitute* "base/client.jl"
>                 (("/bin/sh") (which "sh")))
>               #t))
> -         (add-before 'build 'fix-precompile
> -           (lambda _
> -             (substitute* "base/loading.jl"
> -               (("something(Base.active_project(), \"\")") "\"\""))
> +         (add-before 'build 'shared-objects-paths
> +           (lambda* (#:key inputs #:allow-other-keys)
> +             (let ((jlpath
> +                    (lambda (pkgname)
> +                      (string-append
> +                       "stdlib/" pkgname "_jll/src/" pkgname "_jll.jl")))
> +                   (from
> +                    (lambda (libname)
> +                      (string-append "const " libname " = .*\\.so")))
> +                   (to
> +                    (lambda* (pkg libname #:optional libname_jl)
> +                      (string-append
> +                       "const " (or libname_jl libname)  "= \""
> +                       (assoc-ref inputs pkg) "/lib/" libname ".so"))))
> +               (substitute* (jlpath "dSFMT")
> +                 (((from "libdSFMT")) (to "dsfmt" "libdSFMT")))
> +               (substitute* (jlpath "GMP")
> +                 (((from "libgmp")) (to "gmp" "libgmp"))
> +                 (((from "libgmpxx")) (to "gmp" "libgmpxx")))
> +               (substitute* (jlpath "libLLVM")
> +                 (((from "libLLVM")) (to "llvm" "libLLVM")))
> +               (substitute* (jlpath "LibCURL")
> +                 (((from "libcurl")) (to "curl" "libcurl")))
> +               (substitute* (jlpath "LibGit2")
> +                 (((from "libgit2")) (to "libgit2" "libgit2")))
> +               (substitute* (jlpath "LibSSH2")
> +                 (((from "libssh2")) (to "libssh2" "libssh2")))
> +               (substitute* (jlpath "LibUV")
> +                 (((from "libuv")) (to "libuv" "libuv")))
> +               (substitute* (jlpath "LibUnwind")
> +                 (((from "libunwind")) (to "libunwind" "libunwind")))
> +               (substitute* (jlpath "MPFR")
> +                 (((from "libmpfr")) (to "mpfr" "libmpfr")))
> +               (substitute* (jlpath "MbedTLS")

After updating mbedtls-apache I added the following two lines here:
;; For the newer version of mbedtls-apache:
(("libmbedcrypto.so.5") "libmbedcrypto.so.6")

> +                 (((from "libmbedcrypto")) (to "mbedtls" "libmbedcrypto"))
> +                 (((from "libmbedtls")) (to "mbedtls" "libmbedtls"))
> +                 (((from "libmbedx509")) (to "mbedtls" "libmbedx509")))
> +               (substitute* (jlpath "nghttp2")
> +                 (((from "libnghttp2")) (to "libnghttp2" "libnghttp2")))
> +               (substitute* (jlpath "OpenBLAS")
> +                 (((from "libopenblas")) (to "openblas" "libopenblas")))
> +               (substitute* (jlpath "OpenLibm")
> +                 (((from "libopenlibm")) (to "openlibm" "libopenlibm")))
> +               (substitute* (jlpath "PCRE2")
> +                 (((from "libpcre2")) (to "pcre2" "libpcre2" "libpcre2_8")))
> +               (substitute* (jlpath "SuiteSparse")
> +                 (((from "libamd")) (to "suitesparse" "libamd"))
> +                 (((from "libbtf")) (to "suitesparse" "libbtf"))
> +                 (((from "libcamd")) (to "suitesparse" "libcamd"))
> +                 (((from "libccolamd")) (to "suitesparse" "libccolamd"))
> +                 (((from "libcholmod")) (to "suitesparse" "libcholmod"))
> +                 (((from "libcolamd")) (to "suitesparse" "libcolamd"))
> +                 (((from "libklu")) (to "suitesparse" "libklu"))
> +                 (((from "libldl")) (to "suitesparse" "libldl"))
> +                 (((from "librbio")) (to "suitesparse" "librbio"))
> +                 (((from "libspqr")) (to "suitesparse" "libspqr"))
> +                 (((from "libsuitesparse")) (to "suitesparse" "libsuitesparse"))
> +                 (((from "libsuitesparseconfig")) (to "suitesparse" "libsuitesparseconfig"))
> +                 (((from "libumfpack")) (to "suitesparse" "libumfpack")))
> +               (substitute* (jlpath "Zlib")
> +                 (((from "libz")) (to "zlib" "libz"))))
>               #t))
> +         (add-before 'check 'fix-setenv
> +           (lambda* (#:key inputs #:allow-other-keys)
> +             ;; some tests execute julia in an environment that needs to propagate GUIX_LOCPATH
> +             (substitute* "test/cmdlineargs.jl"
> +               (("\"HOME\"\\s=>\\shomedir\\(\\)")
> +                "\"HOME\" => homedir(), \"GUIX_LOCPATH\" => ENV[\"GUIX_LOCPATH\"]"))))
>           (add-before 'check 'disable-broken-tests
>             (lambda _
>               (substitute* "test/choosetests.jl"
> -               (("tests = testnames")
> -                ;; Those failings are not deterministic.  They depends on the
> -                ;; running order.  I think it depends on the number of
> -                ;; runners, disabling it for now
> -                ;; https://github.com/JuliaLang/julia/issues/34330
> -                "tests = filter(e->!in(e,[\"backtrace\",\"exceptions\",\"precompile\",
> -                                           \"client\",\"stacktraces\"]),
> -                                       testnames)"))
> -             ;; Marking the test as broken as it's a known bug:
> -             ;; https://github.com/JuliaLang/julia/issues/32377
> -             (substitute* "stdlib/REPL/test/replcompletions.jl"
> -               (("@test count") "@test_broken count"))
> -             ;; Dates has a similar bug:
> +               (("skip_tests = \\[\\]")
> +                ;; disabling REPL tests because they require a stdin
> +                ;; disabling tests for MPFR, MbedTLS and SuiteSparse because they only check for hardcoded versions
> +                "skip_tests = [\"REPL\", \"MPFR_jll\", \"MbedTLS_jll\", \"SuiteSparse_jll\"]"))
> +             ;; Dates/io tests fail on master when networking is unavailable
>               ;; https://github.com/JuliaLang/julia/issues/34655
>               (substitute* "stdlib/Dates/test/io.jl"
>                 (("\"Dates.Date") "\"Date")
> @@ -349,11 +431,43 @@ libraries.  It is also a bit like @code{ldd} and @code{otool -L}.")
>               ;; https://github.com/JuliaLang/julia/issues/35785
>               (substitute* "test/file.jl"
>                 (("@test dirname\\(t\\) == d") "@test_broken dirname(t) == d"))
> -             ;; Deprecation test fails with --depwarn=no
> -             ;; https://github.com/JuliaLang/julia/issues/37673
> -             (substitute* "test/Makefile"
> -               (("./runtests.jl") "--depwarn=error ./runtests.jl"))
> +             ;; julia embeds a certificate, we are not doing that
> +             (substitute* "stdlib/MozillaCACerts_jll/test/runtests.jl"
> +               (("@test isfile\\(MozillaCACerts_jll.cacert\\)") "@test_broken isfile(MozillaCACerts_jll.cacert)"))
> +             ;; since certificate is not present some tests are failing in network option
> +             (substitute* "usr/share/julia/stdlib/v1.6/NetworkOptions/test/runtests.jl"
> +               (("@test isfile\\(bundled_ca_roots\\(\\)\\)") "@test_broken isfile(bundled_ca_roots())")
> +               (("@test ispath\\(ca_roots_path\\(\\)\\)") "@test_broken ispath(ca_roots_path())")
> +               (("@test ca_roots_path\\(\\) \\!= bundled_ca_roots\\(\\)") "@test_broken ca_roots_path() != bundled_ca_roots()"))
>               #t))
> +         (add-before 'install 'symlink-libraries
> +           (lambda* (#:key inputs outputs #:allow-other-keys)
> +             (let ((link
> +                    (lambda (pkgname dir pred)
> +                      (map (lambda (file)
> +                             (invoke "ln" "-s" file dir))
> +                           (find-files (string-append (assoc-ref inputs pkgname) "/lib") pred)))))
> +               (link "curl" "usr/lib/" "\\.so") ;; missing libpthreads libLLVM-11jl
> +               (link "suitesparse" "usr/lib/julia/" "libbtf\\.so")
> +               (link "suitesparse" "usr/lib/julia/" "libklu\\.so")
> +               (link "suitesparse" "usr/lib/julia/" "libldl\\.so")
> +               (link "suitesparse" "usr/lib/julia/" "librbio\\.so")
> +               (link "gmp" "usr/lib/julia/" "libgmpxx\\.so")
> +               (link "libuv" "usr/lib/julia/" "libuv\\.so")
> +               (link "zlib" "usr/lib/julia/" "libz\\.so")
> +               (link "libunwind" "usr/lib/julia/" "libunwind\\.so")
> +               (invoke "ln" "-s" (string-append (assoc-ref inputs "p7zip") "/bin/7z") "usr/bin/")
> +               #t)))
> +         (add-after 'install 'symlink-llvm-utf8proc
> +           (lambda* (#:key inputs outputs #:allow-other-keys)
> +             (let ((link
> +                    (lambda (pkgname pred)
> +                      (map (lambda (file)

can we use the 'symlink' function here rather than shelling out to 'ln'?
Also for the other instances of it in this patch.

> +                             (invoke "ln" "-s" file (string-append (assoc-ref outputs "out") "/lib/julia/")))
> +                           (find-files (string-append (assoc-ref inputs pkgname) "/lib") pred)))))
> +               (link "llvm" "libLLVM-11\\.so")
> +               (link "utf8proc" "libutf8proc\\.so")
> +               #t)))
>           (add-after 'install 'make-wrapper
>             (lambda* (#:key inputs outputs #:allow-other-keys)
>               (let* ((out (assoc-ref outputs "out"))
> @@ -370,7 +484,10 @@ libraries.  It is also a bit like @code{ldd} and @code{otool -L}.")
>         #:make-flags
>         (list
>          (string-append "prefix=" (assoc-ref %outputs "out"))
> -
> +         ;; Adding generic,-cx16 target to the list of defaults target for better compatibility
> +         ;; clone_all is required otherwise julia complains during compilation
> +         ;; see https://docs.julialang.org/en/v1/devdocs/sysimg/#Specifying-multiple-system-image-targets
> +         "JULIA_CPU_TARGET=generic;generic,-cx16,clone_all;sandybridge,-xsaveopt,clone_all;haswell,-rdrnd,base(1)"

Passing JULIA_CPU_TARGET unconditionally here will likely break julia on
i686. On the other architectures it will be overridden. It would be best
to put it with the x86_64 MARCH architecture and adapt the comment there
to mention setting the MARCH and optionally the JULIA_CPU_TARGET on a
per-architecture basis.

>           ;; Passing the MARCH flag is necessary to build binary substitutes for
>           ;; the supported architectures.
>           ,(match (or (%current-target-system)
> @@ -382,69 +499,76 @@ libraries.  It is also a bit like @code{ldd} and @code{otool -L}.")
>                   ;; platforms, e.g. when running "guix package --search="
>                   (_ "MARCH=UNSUPPORTED"))
> 
> -         "CONFIG_SHELL=bash"     ;needed to build bundled libraries
> +         "CONFIG_SHELL=bash -x"     ;needed to build bundled libraries
> +         "USE_BINARYBUILDER=0"
>           ;; list of "USE_SYSTEM_*" is here:
> -         ;; https://github.com/JuliaLang/julia/blob/v1.3.1/Make.inc
> +         ;; https://github.com/JuliaLang/julia/blob/v1.6.0/Make.inc
> +         ;;

As long as these all got moved around I'd keep them sorted either
quasi-alphabetically or in the order they appear in the Make.inc file. I
didn't check the file to see if that was the case.

> +         "USE_SYSTEM_CSL=1"
> +         "USE_SYSTEM_LLVM=1"
> +         "USE_SYSTEM_LIBUNWIND=1"
> +         "USE_SYSTEM_PCRE=1"
> +         "USE_SYSTEM_OPENLIBM=1"
> +         "USE_SYSTEM_DSFMT=1"
> +         "USE_SYSTEM_BLAS=1"
> +         "USE_SYSTEM_LAPACK=1"
> +         "USE_SYSTEM_GMP=1"
> +         "USE_SYSTEM_MPFR=1"
> +         "USE_SYSTEM_SUITESPARSE=1"
> +         "USE_SYSTEM_LIBUV=1"
> +         "USE_SYSTEM_UTF8PROC=1"
> +         "USE_SYSTEM_MBEDTLS=1"
> +         "USE_SYSTEM_LIBSSH2=1"
> +         "USE_SYSTEM_NGHTTP2=1"
> +         "USE_SYSTEM_CURL=1"
> +         "USE_SYSTEM_LIBGIT2=1"
> +         "USE_SYSTEM_PATCHELF=1"
> +         "USE_SYSTEM_ZLIB=1"
> +         "USE_SYSTEM_P7ZIP=1"
>           "USE_SYSTEM_DSFMT=1"
>           "USE_SYSTEM_P7ZIP=1"
>           "USE_SYSTEM_LAPACK=1"
> -         "USE_SYSTEM_BLAS=1"
> +         "USE_SYSTEM_CURL=1"
>           "USE_BLAS64=0"          ;needed when USE_SYSTEM_BLAS=1
>           "LIBBLAS=-lopenblas"
>           "LIBBLASNAME=libopenblas"
> 
> -         "USE_SYSTEM_SUITESPARSE=1"
>           (string-append "SUITESPARSE_INC=-I "
>                          (assoc-ref %build-inputs "suitesparse")
>                          "/include")
> +
>           "USE_GPL_LIBS=1"        ;proudly
> -         "USE_SYSTEM_UTF8PROC=1"
>           (string-append "UTF8PROC_INC="
>                          (assoc-ref %build-inputs "utf8proc")
>                          "/include")
> -         "USE_SYSTEM_LLVM=1"
> -         "LLVM_VER=9.0.1"
> +         "LLVM_VER=11.0.0"
> 
>           "USE_LLVM_SHLIB=1"
> -         "USE_SYSTEM_LIBUNWIND=1"
> -         "USE_SYSTEM_LIBUV=1"
>           (string-append "LIBUV="
>                          (assoc-ref %build-inputs "libuv")
>                          "/lib/libuv.so")
>           (string-append "LIBUV_INC="
>                          (assoc-ref %build-inputs "libuv")
> -                        "/include")
> -         "USE_SYSTEM_PATCHELF=1"
> -         "USE_SYSTEM_PCRE=1"
> -         "USE_SYSTEM_OPENLIBM=1"
> -         "USE_SYSTEM_MBEDTLS=1"
> -         "USE_SYSTEM_LIBSSH2=1"
> -         "USE_SYSTEM_GMP=1"
> -         "USE_SYSTEM_MPFR=1"
> -         "USE_SYSTEM_ARPACK=1"
> -         "USE_SYSTEM_LIBGIT2=1"
> -         "USE_SYSTEM_ZLIB=1")))
> +                        "/include"))))
>      (inputs
>       `(("llvm" ,llvm-julia)
>         ("p7zip" ,p7zip)
> -       ;; The bundled version is 3.3.0 so stick to that version.  With other
> -       ;; versions, we get test failures in 'linalg/arnoldi' as described in
> -       ;; <https://bugs.gnu.org/30282>.
> -       ("arpack-ng" ,arpack-ng-3.3.0)
> -
>         ("coreutils" ,coreutils) ;for bindings to "mkdir" and the like
>         ("lapack" ,lapack)
>         ("openblas" ,openblas) ;Julia does not build with Atlas
>         ("libunwind" ,libunwind-julia)
>         ("openlibm" ,openlibm)
>         ("mbedtls" ,mbedtls-apache)
> -       ("curl" ,curl)
> -       ("libgit2" ,libgit2-0.28)
> +       ("curl" ,curl-ssh)
> +       ("libnghttp2" ,nghttp2 "lib")
> +       ("libgit2" ,libgit2)
>         ("libssh2" ,libssh2)
>         ("fortran" ,gfortran)
> +       ;; required for libgcc_s.so
> +       ("libfortran" ,gfortran "lib")
>         ("libuv" ,libuv-julia)
> -       ("pcre2" ,pcre2)
> -       ("utf8proc" ,utf8proc)
> +       ("pcre2" ,pcre2-10.36)
> +       ("utf8proc" ,utf8proc-2.6.1)
>         ("mpfr" ,mpfr)
>         ("wget" ,wget)
>         ("which" ,which)
> @@ -452,7 +576,7 @@ libraries.  It is also a bit like @code{ldd} and @code{otool -L}.")
>         ("gmp" ,gmp)
>         ("suitesparse" ,suitesparse)
>         ;; Find dependencies versions here:
> -       ;; https://raw.githubusercontent.com/JuliaLang/julia/v1.3.0/deps/Versions.make
> +       ;; https://raw.githubusercontent.com/JuliaLang/julia/v1.6.0/deps/Versions.make
>         ("libwhich" ,libwhich)
>         ("dsfmt" ,dsfmt)))
>      (native-inputs
> @@ -460,6 +584,7 @@ libraries.  It is also a bit like @code{ldd} and @code{otool -L}.")
>         ("perl" ,perl)
>         ("patchelf" ,patchelf)
>         ("pkg-config" ,pkg-config)
> +       ("nss-certs" ,nss-certs)

try to keep the libraries sorted alphabetically, it reduces the chance
of duplication.

>         ("python" ,python-2)))
>      (native-search-paths
>        (list (search-path-specification
> --
> 2.32.0
> 
> 
> 
> 
>