diff mbox series

[bug#48325] julia-1.6 guix

Message ID kNqwtYdhnfFKZLGBH6tL0YPqkGs7pncwgvX6u2YQbW6ATIyet9HpY1O4_hksoo0AQaH5zslseW0FbOskAAxnpA==@pm.me
State Accepted
Headers show
Series [bug#48325] julia-1.6 guix | expand

Checks

Context Check Description
cbaines/comparison success View comparision
cbaines/git branch success View Git branch
cbaines/applying patch fail View Laminar job
cbaines/issue success View issue

Commit Message

Jean-Baptiste Volatier May 10, 2021, 11:29 a.m. UTC
Hi,
Find attached the updated patch. I applied your suggestions.

Nicolò Balzarotti wrote:
>>> -            ;; "-DLLVM_HOST_TRIPLE=${stdenv.hostPlatform.config}"
>>> -            ;; "-DLLVM_DEFAULT_TARGET_TRIPLE=${stdenv.hostPlatform.config}"
>>> -            ;; "-DLLVM_EXPERIMENTAL_TARGETS_TO_BUILD=WebAssembly"
>> Those seems to be taken from nixos, but is there a reason why are those
>> commented?

No clue, this was already there.

>>> build to install
>>>
>>> What does this comment refer to?

Some leftover I forgot to remove.

Cheers,
jB

Comments

Simon Tournier May 11, 2021, 12:07 a.m. UTC | #1
Hi Jean-Baptiste,

Thanks for the patch.  Here some minor comments.

On Mon, 10 May 2021 at 11:29, Jean-Baptiste Volatier <jbv@pm.me> wrote:

> From e610dacab669ce84fe8f263a01aefff1fe49b6aa Mon Sep 17 00:00:00 2001
> From: Jean-Baptiste Volatier <jbv@pm.me>
> Date: Mon, 10 May 2021 09:57:23 +0200
> Subject: [PATCH] gnu: julia: update to 1.6.1
>
> gnu: openlibm: update to 0.7.4
> gnu: pcre2: update to 10.56
> gnu: utf8proc: update to 2.6.1
> gnu: julia-benchmarktools: update to 0.7.0

Please, split this patch.  One per update, i.e., 5 patches I guess.

> diff --git a/gnu/packages/julia.scm b/gnu/packages/julia.scm
> index 13c9f7baf1..39627eeed0 100644
> --- a/gnu/packages/julia.scm
> +++ b/gnu/packages/julia.scm
> @@ -1,9 +1,10 @@
>  ;;; GNU Guix --- Functional package management for GNU
>  ;;; Copyright © 2015, 2016, 2017 Ricardo Wurmus <rekado@elephly.net>
>  ;;; Copyright © 2016, 2020 Efraim Flashner <efraim@flashner.co.il>
> -;;; Copyright © 2020 Nicolò Balzarotti <nicolo@nixo.xyz>
> +;;; Copyright © 2020, 2021 Nicolò Balzarotti <nicolo@nixo.xyz>

Just to be sure, if Nicoló is co-author, it should be worth to add them
in the commit message, something like:

Co-Authored-By: Nicolò Balzarotti <nicolo@nixo.xyz>.

> -    (source (origin
> -              (inherit (package-source llvm-9))
> -              ;; Those patches are inside the Julia source repo.
> -              ;; They are _not_ Julia specific (https://github.com/julialang/julia#llvm)
> -              ;; but they are required to build Julia.
> -              ;; Discussion: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=919628
> -              (patches
> -               (map (match-lambda
> -                      ((name hash)
> -                       (julia-patch name hash)))
> -                    (list
> -                     '("llvm-D27629-AArch64-large_model_6.0.1"
> -                       "1qrshmlqvnasdyc158vfn3hnbigqph3lsq7acb9w8lwkpnnm2j4z")
> -                     '("llvm8-D34078-vectorize-fdiv"
> -                       "19spqc3xsazn1xs9gpcgv9ldadfkv49rmc5khl7sf1dlmhgi4602")
> -                     '("llvm-7.0-D44650"
> -                       "1h55kkmkiisfj6sk956if2bcj9s0v6n5czn8dxb870vp5nccj3ir")
> -                     '("llvm9-D50010-VNCoercion-ni"
> -                       "1s1d3sjsiq4vxg7ncy5cz56zgy5vcq6ls3iqaiqkvr23wyryqmdx")
> -                     '("llvm-exegesis-mingw"
> -                       "0ph1cj1j7arvf1xq2xcr7qf9g0cpdl14fincgr67vpi520zvd3vp")
> -                     '("llvm-test-plugin-mingw"
> -                       "12z738cnahbf6n381im7i0hxp1m6k9hrnfjlmq9sac46nxly9gnj")
> -                     '("llvm7-revert-D44485"
> -                       "0f59kq3p3mpwsbmskypbi4zn01l6ig0x7v2rjp08k2r8z8m6fa8n")
> -                     '("llvm-8.0-D66657-codegen-degenerate"
> -                       "1n1ddx19h90bbpimdyd9dh8fsm6gb93xxyqm4ljkxa1k3cx2vm72")
> -                     '("llvm-8.0-D71495-vectorize-freduce"
> -                       "1zff08wvji9lnpskk4b3p5zyjsy5hhy23ynxjqlj9dw7jvvfrf0p")
> -                     '("llvm-D75072-SCEV-add-type"
> -                       "029a3fywsm233vf48mscina24idd50dc75wr70lmimrhwnw27p0z")
> -                     '("llvm-9.0-D65174-limit-merge-stores"
> -                       "04bff1mnblfj9mxfdwr1qdnw3i3szmp60gnhxwas5y68qg33z6j0")
> -                     '("llvm9-D71443-PPC-MC-redef-symbol"
> -                       "1c93nv7rgc9jg5mqrnvv08xib1789qvlql94fwggh18mp3b9hbgy")
> -                     '("llvm-9.0-D78196"
> -                       "08a43hyg7yyqjq2vmfsmppf34xcz60wq6y9zw5fdyhw2h1mcnmns")
> -                     '("llvm-julia-tsan-custom-as"
> -                       "0awh40kf6lm4wn1nsjd1bmhfwq7rqj811szanp2xkpspykw9hg9s")
> -                     '("llvm-9.0-D85499"
> -                       "0vxlr35srvbvihlgrxq15v6dylp90vgi0qahj22j01jgqmdasjkm"))))
> -              (patch-flags '("-p1"))))
>      (arguments
> -     (substitute-keyword-arguments (package-arguments llvm-9)
> +     (substitute-keyword-arguments (package-arguments llvm-11)
>         ((#:configure-flags flags)
>          `(list ;; Taken from NixOS. Only way I could get libLLVM-6.0.so
>             "-DCMAKE_BUILD_TYPE=Release"
> @@ -177,7 +140,61 @@
>             ;; "-DLLVM_DEFAULT_TARGET_TRIPLE=${stdenv.hostPlatform.config}"
>             ;; "-DLLVM_EXPERIMENTAL_TARGETS_TO_BUILD=WebAssembly"
>             "-DLLVM_ENABLE_DUMP=ON"
> -           "-DLLVM_LINK_LLVM_DYLIB=ON"))))))
> +           "-DLLVM_LINK_LLVM_DYLIB=ON"))
> +       ((#:phases phases)
> +        `(modify-phases ,phases
> +           ;; applying patches from julia
> +           ;; list of patches can be found in deps/llvm.mk in julia source
> +           (add-after 'unpack 'julia-patches
> +             (lambda* (#:key inputs outputs #:allow-other-keys)
> +               (let ((patch
> +                      (lambda (patchname flag)
> +                        (invoke "patch" flag "-i"
> +                                (string-append
> +                                 "julia-src/deps/patches/"
> +                                 patchname
> +                                 ".patch")))))
> +                 (mkdir-p "julia-src")
> +                 (invoke "tar" "xf"
> +                         (assoc-ref inputs "julia-source")
> +                         "-C" "julia-src" "--strip-components=1")
> +                 (map (lambda (patchname)
> +                        (patch patchname "-p1"))
> +                      (list "llvm-D27629-AArch64-large_model_6.0.1"
> +                            "llvm8-D34078-vectorize-fdiv"
> +                            "llvm-7.0-D44650"
> +                            "llvm-6.0-DISABLE_ABI_CHECKS"
> +                            "llvm9-D50010-VNCoercion-ni"
> +                            "llvm7-revert-D44485"
> +                            "llvm-11-D75072-SCEV-add-type"
> +                            "llvm-julia-tsan-custom-as"
> +                            "llvm-D80101"
> +                            "llvm-D84031"
> +                            "llvm-10-D85553"
> +                            "llvm-10-unique_function_clang-sa"
> +                            "llvm-11-D85313-debuginfo-empty-arange"
> +                            "llvm-11-D90722-rtdyld-absolute-relocs"
> +                            "llvm-invalid-addrspacecast-sink"
> +                            "llvm-11-D92906-ppc-setjmp"
> +                            "llvm-11-PR48458-X86ISelDAGToDAG"
> +                            "llvm-11-D93092-ppc-knownbits"
> +                            "llvm-11-D93154-globalisel-as"
> +                            "llvm-11-ppc-half-ctr"
> +                            "llvm-11-ppc-sp-from-bp"
> +                            "llvm-rGb498303066a6-gcc11-header-fix"
> +                            "llvm-11-D94813-mergeicmps"
> +                            "llvm-11-D94980-CTR-half"
> +                            "llvm-11-D94058-sext-atomic-ops"
> +                            "llvm-11-D96283-dagcombine-half"))
> +                 (map (lambda (patchname)
> +                        (patch patchname "-p2"))
> +                      (list "llvm-11-AArch64-FastIsel-bug"
> +                            "llvm-11-D97435-AArch64-movaddrreg"
> +                            "llvm-11-D97571-AArch64-loh"
> +                            "llvm-11-aarch64-addrspace")))))))))

I am not convinced by this move of patches from ’source’ to ’phases’.
My understanding about the usual way is to let the patch in the source
field.  Is this move motivated by something special?

> -                                       '("arpack-ng" "curl" "dsfmt"

I have not read the Julia ChangeLog.  Do they remove Arpack?  This
should be mentioned in the commit message.

> +                                       '("curl" "dsfmt"
>                                           "gmp" "lapack"
> -                                         "libssh2" "libgit2"
> +                                         "libssh2" "libnghttp2" "libgit2"

Idem for libnghttp2.

>                                           "mbedtls" "mpfr"
>                                           "openblas" "openlibm" "pcre2"
> -                                         "suitesparse"))
> -                                  ":"))
> -             #t))
> +                                         "suitesparse" "libfortran"))

Idem for libfortran.

> -         (add-before 'build 'fix-precompile
> -           (lambda _
> -             (substitute* "base/loading.jl"
> -               (("something(Base.active_project(), \"\")") "\"\""))
> +         (add-before 'build 'shared-objects-paths
> +           (lambda* (#:key inputs #:allow-other-keys)

[...]

> +               ;; FAILING: OpenBLAS

What does it mean?


> +         (add-before 'install 'symlink-libraries

[...]

> +               (link "zlib" "usr/lib/julia/" "libz\\.so")

Does this fix

   <http://issues.guix.gnu.org/48238>

?  If yes, cool and thank you! :-)  So it should be mentioned in the
commit message, something like:

--8<---------------cut here---------------start------------->8---
* gnu: julia: Update to 1.6.1.

Fixes <https://bug.gnu.org/48238>.

* gnu/packages/julia.scm (julia): Update to 1.6.1.
[arguments]: …stuff that changed…
[inputs]: Add foo, Remove bar.

Co-Authored-By: Nicolò Balzarotti <nicolo@nixo.xyz>.
--8<---------------cut here---------------end--------------->8---

Does it make sense?

> -         "USE_SYSTEM_ARPACK=1"

What is the motivation for removing Arpack?  Sorry if my question is naive.

>           "USE_SYSTEM_LIBGIT2=1"
>           "USE_SYSTEM_ZLIB=1")))
>      (inputs
>       `(("llvm" ,llvm-julia)
>         ("p7zip" ,p7zip)
> -       ;; The bundled version is 3.3.0 so stick to that version.  With other
> -       ;; versions, we get test failures in 'linalg/arnoldi' as described in
> -       ;; <https://bugs.gnu.org/30282>.
> -       ("arpack-ng" ,arpack-ng-3.3.0)
> -
> -       ("coreutils" ,coreutils) ;for bindings to "mkdir" and the like
> +       ("coreutils" ,coreutils)         ;for bindings to "mkdir" and the like

This is not a change.  Even if the new indentation is correct, please
let avoid cosmetic change in the same commit updating a complex package.
Because then digging in the history becomes more complex. :-)

>         ("lapack" ,lapack)
> -       ("openblas" ,openblas) ;Julia does not build with Atlas
> +       ("openblas" ,openblas)           ;Julia does not build with Atlas

Idem.

>         ("libunwind" ,libunwind-julia)
>         ("openlibm" ,openlibm)
>         ("mbedtls" ,mbedtls-apache)
>         ("curl" ,curl)
> -       ("libgit2" ,libgit2-0.28)
> +       ("libnghttp2" ,nghttp2 "lib")
> +       ("libgit2" ,libgit2)
>         ("libssh2" ,libssh2)
>         ("fortran" ,gfortran)
> +       ;; required for libgcc_s.so
> +       ("libfortran" ,gfortran "lib")
>         ("libuv" ,libuv-julia)
>         ("pcre2" ,pcre2)
>         ("utf8proc" ,utf8proc)
>         ("mpfr" ,mpfr)
> +       ("nss-certs" ,nss-certs)         ; required to precompile

Hum?  Is it really necessary?

> +       ("glibc-locales" ,glibc-locales)

Idem.  Is it really necessary?  Because it is a “big“ packages which
drastically increases the closure size of the Julia package.



Thanks again for the patch.


Cheers,
simon
Nicolò Balzarotti May 11, 2021, 9:18 a.m. UTC | #2
zimoun <zimon.toutoune@gmail.com> writes:

> Hi Jean-Baptiste,
>
> Thanks for the patch.  Here some minor comments.
>
> Please, split this patch.  One per update, i.e., 5 patches I guess.
>

Some changes should probably be committed together since current julia
will fail to build with newer deps versions, but at least for utf8proc
I'm sure this can be splitted.
 
> Just to be sure, if Nicoló is co-author, it should be worth to add them
> in the commit message, something like:
>
> Co-Authored-By: Nicolò Balzarotti <nicolo@nixo.xyz>.
>

Yep I helped a bit

>> -                                       '("arpack-ng" "curl" "dsfmt"
>
> I have not read the Julia ChangeLog.  Do they remove Arpack?  This
> should be mentioned in the commit message.

Yep, see for example:
https://github.com/JuliaLang/julia/commit/0196cf1ff549ac120d8c556d610927d76d679722

>
>> +                                       '("curl" "dsfmt"
>>                                           "gmp" "lapack"
>> -                                         "libssh2" "libgit2"
>> +                                         "libssh2" "libnghttp2" "libgit2"
>
> Idem for libnghttp2.
>
Yep they added libnghttp2
https://github.com/JuliaLang/julia/blob/bb5b98e72a151c41471d8cc14cacb495d647fb7f/stdlib/nghttp2_jll/src/nghttp2_jll.jl

>>                                           "mbedtls" "mpfr"
>>                                           "openblas" "openlibm" "pcre2"
>> -                                         "suitesparse"))
>> -                                  ":"))
>> -             #t))
>> +                                         "suitesparse" "libfortran"))
>
> Idem for libfortran.

I just realized this is not sorted, would you mind fixing it?  Also,
there are still a few #t which can be removed (like this one in
'prepare-deps)

>
>> -         (add-before 'build 'fix-precompile
>> -           (lambda _
>> -             (substitute* "base/loading.jl"
>> -               (("something(Base.active_project(), \"\")") "\"\""))
>> +         (add-before 'build 'shared-objects-paths
>> +           (lambda* (#:key inputs #:allow-other-keys)
>
> [...]
>
>> +               ;; FAILING: OpenBLAS
>
> What does it mean?
>
I think it's a leftover which can be removed

>
>> +         (add-before 'install 'symlink-libraries
>
> [...]
>
>> +               (link "zlib" "usr/lib/julia/" "libz\\.so")
>
> Does this fix
>
>    <http://issues.guix.gnu.org/48238>
>
> ?
> Does it make sense?
>
I just tried
running
> cd /gnu/store/fp5by15552ihyppxppigmbxs981rd08h-julia-1.6.1
> find -type f -exec grep '/tmp/guix-build-julia-1.5.3.drv-0' {} \;

and it returns nothing.  Also, your Pkg.add("GZip"); using GZip worked

(but it printed
┌ Error: curl_easy_setopt: 48
└ @ Downloads.Curl
~/guix-build-julia-1.6.1.drv-0/julia-1.6.1/usr/share/julia/stdlib/v1.6/Downloads/src/Curl/utils.jl:36
which might be worth investigating)


>> -         "USE_SYSTEM_ARPACK=1"
>
> What is the motivation for removing Arpack?  Sorry if my question is
> naive.
As before, it's no longer used by Julia

>
>>           "USE_SYSTEM_LIBGIT2=1"
>>           "USE_SYSTEM_ZLIB=1")))
>>      (inputs
>>       `(("llvm" ,llvm-julia)
>>         ("p7zip" ,p7zip)
>> -       ;; The bundled version is 3.3.0 so stick to that version.  With other
>> -       ;; versions, we get test failures in 'linalg/arnoldi' as described in
>> -       ;; <https://bugs.gnu.org/30282>.
>> -       ("arpack-ng" ,arpack-ng-3.3.0)
>> -
>> -       ("coreutils" ,coreutils) ;for bindings to "mkdir" and the like
>> +       ("coreutils" ,coreutils)         ;for bindings to "mkdir" and the like
>
> This is not a change.  Even if the new indentation is correct, please
> let avoid cosmetic change in the same commit updating a complex package.
> Because then digging in the history becomes more complex. :-)
>

Agreed
>> +       ("nss-certs" ,nss-certs)         ; required to precompile
>
> Hum?  Is it really necessary?
>
I think that the alternative is to disable a few tests.  I opened a bug
report a few months ago here:
https://github.com/JuliaLang/julia/issues/40185

>> +       ("glibc-locales" ,glibc-locales)
>
> Idem.  Is it really necessary?  Because it is a “big“ packages which
> drastically increases the closure size of the Julia package.
>

I think it's the same as before (fixes a few tests) but I cannot verify
this, Jean-Baptiste do you know what breaks when this is removed?

>
> Thanks again for the patch.
>
Yep, thanks Jean-Baptiste and Simon for your review


Nicolò
Nicolò Balzarotti May 11, 2021, 9:53 a.m. UTC | #3
>> cd /gnu/store/fp5by15552ihyppxppigmbxs981rd08h-julia-1.6.1
>> find -type f -exec grep '/tmp/guix-build-julia-1.5.3.drv-0' {} \;
>
I just realized I should have run

> find -type f -exec grep '/tmp/guix-build-julia-1.6.1.drv-0' {} \;

Binary file ./bin/.julia-real matches
Binary file ./lib/julia/sys.so matches
Binary file ./lib/julia/libccalltest.so.debug matches
Binary file ./lib/julia/libjulia-internal.so.1.6 matches
Binary file ./lib/julia/libllvmcalltest.so matches
Binary file ./lib/libjulia.so.1.6 matches
Binary file ./share/julia/base.cache matches
Simon Tournier May 11, 2021, 9:55 a.m. UTC | #4
Hi,

On Tue, 11 May 2021 at 11:18, Nicolò Balzarotti <anothersms@gmail.com> wrote:
> zimoun <zimon.toutoune@gmail.com> writes:

>> Please, split this patch.  One per update, i.e., 5 patches I guess.
>
> Some changes should probably be committed together since current julia
> will fail to build with newer deps versions, but at least for utf8proc
> I'm sure this can be splitted.

The issue is that the update of pcre2 is a ’core-updates’ change, IIUC:

  guix refresh -l pcre2

so the package cannot be simply updated.  This one also should be
splitted, IMHO.

Is the current julia@1.5.3 ’will fail’ for sure or probably ’would fail’
with openlibm@0.7.4 and julia-benchmarktools@0.7.0?  I have not tried
yet. :-)


>>> +         (add-before 'install 'symlink-libraries
>>
>> [...]
>>
>>> +               (link "zlib" "usr/lib/julia/" "libz\\.so")
>>
>> Does this fix
>>
>>    <http://issues.guix.gnu.org/48238>
>>
>> ?
>> Does it make sense?
>>
> I just tried running
>> cd /gnu/store/fp5by15552ihyppxppigmbxs981rd08h-julia-1.6.1
>> find -type f -exec grep '/tmp/guix-build-julia-1.5.3.drv-0' {} \;
>
> and it returns nothing.  Also, your Pkg.add("GZip"); using GZip worked

Cool!

> (but it printed
>
> ┌ Error: curl_easy_setopt: 48
> └ @ Downloads.Curl
> ~/guix-build-julia-1.6.1.drv-0/julia-1.6.1/usr/share/julia/stdlib/v1.6/Downloads/src/Curl/utils.jl:36
>
> which might be worth investigating)

Thanks for trying and reporting.  Did you use a --pure or --container
environment?

>>> +       ("nss-certs" ,nss-certs)         ; required to precompile
>>
>> Hum?  Is it really necessary?
>>
> I think that the alternative is to disable a few tests.  I opened a bug
> report a few months ago here:
> https://github.com/JuliaLang/julia/issues/40185
>
>>> +       ("glibc-locales" ,glibc-locales)
>>
>> Idem.  Is it really necessary?  Because it is a “big“ packages which
>> drastically increases the closure size of the Julia package.
>>
>
> I think it's the same as before (fixes a few tests) but I cannot verify
> this, Jean-Baptiste do you know what breaks when this is removed?

Well, ‘guix size glibc-locales’ reports ~922MiB.  Although this package
is probably already on the store, I would prefer to avoid to add it as
an input.  Currently, ’guix size julia’ says ~1080MiB.  How much with
the glibc-locales as inputs?

Thanks for the patch and the explanations.

Cheers,
simon
Nicolò Balzarotti May 11, 2021, 10:18 a.m. UTC | #5
zimoun <zimon.toutoune@gmail.com> writes:

> Hi,
>
> On Tue, 11 May 2021 at 11:18, Nicolò Balzarotti <anothersms@gmail.com> wrote:
>> zimoun <zimon.toutoune@gmail.com> writes:
>
>>> Please, split this patch.  One per update, i.e., 5 patches I guess.
>>
>> Some changes should probably be committed together since current julia
>> will fail to build with newer deps versions, but at least for utf8proc
>> I'm sure this can be splitted.
>
> The issue is that the update of pcre2 is a ’core-updates’ change, IIUC:
>
>   guix refresh -l pcre2
>
> so the package cannot be simply updated.  This one also should be
> splitted, IMHO.
>
> Is the current julia@1.5.3 ’will fail’ for sure or probably ’would fail’
> with openlibm@0.7.4 and julia-benchmarktools@0.7.0?  I have not tried
> yet. :-)
>
>
>>>> +         (add-before 'install 'symlink-libraries
>>>
>>> [...]
>>>
>>>> +               (link "zlib" "usr/lib/julia/" "libz\\.so")
>>>
>>> Does this fix
>>>
>>>    <http://issues.guix.gnu.org/48238>
>>>
>>> ?
>>> Does it make sense?
>>>
>> I just tried running
>>> cd /gnu/store/fp5by15552ihyppxppigmbxs981rd08h-julia-1.6.1
>>> find -type f -exec grep '/tmp/guix-build-julia-1.5.3.drv-0' {} \;
>>
>> and it returns nothing.  Also, your Pkg.add("GZip"); using GZip worked
>
> Cool!
>
>> (but it printed
>>
>> ┌ Error: curl_easy_setopt: 48
>> └ @ Downloads.Curl
>> ~/guix-build-julia-1.6.1.drv-0/julia-1.6.1/usr/share/julia/stdlib/v1.6/Downloads/src/Curl/utils.jl:36
>>
>> which might be worth investigating)
>
> Thanks for trying and reporting.  Did you use a --pure or --container
> environment?

Nope, I just changed home with
> HOME=/tmp julia
>
I tried with a container now:

./pre-inst-env guix environment --ad-hoc julia nss-certs --container
export SSL_CERT_DIR=$GUIX_ENVIRONMENT/etc/ssl/certs/
export SSL_CERT_FILE=$GUIX_ENVIRONMENT/etc/ssl/certs/ca-certificates.crt
export GIT_SSL_CAINFO="$SSL_CERT_FILE"
julia  -e 'using Pkg; Pkg.add("GZip"); using GZip'

Same error
┌ Error: curl_easy_setopt: 48
└ @ Downloads.Curl /tmp/guix-build-julia-1.6.1.drv-0/julia-1.6.1/usr/share/julia/stdlib/v1.6/Downloads/src/Curl/utils.jl:36

But GZip works (I tried the example on the homepage:
import GZip

fh = GZip.open("infile.gz")
s = readline(fh)

close(fh))

> Well, ‘guix size glibc-locales’ reports ~922MiB.  Although this package
> is probably already on the store, I would prefer to avoid to add it as
> an input.  Currently, ’guix size julia’ says ~1080MiB.  How much with
> the glibc-locales as inputs?

./pre-inst-env guix size julia
1205.6 MiB

glibc-locales does not appear in the closure size (I attached the
output).

The biggest change seams to lie in Julia itself (~+130Mb)
Simon Tournier May 11, 2021, 1:13 p.m. UTC | #6
Hi Nicolò,

On mar., 11 mai 2021 at 12:18, Nicolò Balzarotti <anothersms@gmail.com> wrote:

>>>>    <http://issues.guix.gnu.org/48238>

[...]

>>> (but it printed
>>>
>>> ┌ Error: curl_easy_setopt: 48
>>> └ @ Downloads.Curl
>>> ~/guix-build-julia-1.6.1.drv-0/julia-1.6.1/usr/share/julia/stdlib/v1.6/Downloads/src/Curl/utils.jl:36
>>>
>>> which might be worth investigating)

[...]

> Same error
> ┌ Error: curl_easy_setopt: 48
> └ @ Downloads.Curl /tmp/guix-build-julia-1.6.1.drv-0/julia-1.6.1/usr/share/julia/stdlib/v1.6/Downloads/src/Curl/utils.jl:36

Soemthing appears to me wrong somewhere.  Why does the build directory
appear in the output directory?  This reference should not appear, IMHO.

Applying the last 5 patches, I get (without-tests, just to speedup the
feedback :-)):

  ./pre-inst-env guix build --no-grafts julia --without-tests=julia
  /gnu/store/xbg7kwpylal63gqqb84185j54ib52aji-julia-1.6.1

  find $(./pre-inst-env guix build --no-grafts julia --without-tests=julia) \
       -type f -exec grep '/tmp/guix-build-julia-1.6.1' {} \;
  Fichier binaire /gnu/store/xbg7kwpylal63gqqb84185j54ib52aji-julia-1.6.1/bin/.julia-real correspondant
  Fichier binaire /gnu/store/xbg7kwpylal63gqqb84185j54ib52aji-julia-1.6.1/lib/julia/libllvmcalltest.so correspondant
  Fichier binaire /gnu/store/xbg7kwpylal63gqqb84185j54ib52aji-julia-1.6.1/lib/julia/libjulia-internal.so.1.6 correspondant
  Fichier binaire /gnu/store/xbg7kwpylal63gqqb84185j54ib52aji-julia-1.6.1/lib/julia/libccalltest.so.debug correspondant
  Fichier binaire /gnu/store/xbg7kwpylal63gqqb84185j54ib52aji-julia-1.6.1/lib/julia/sys.so correspondant
  Fichier binaire /gnu/store/xbg7kwpylal63gqqb84185j54ib52aji-julia-1.6.1/lib/libjulia.so.1.6 correspondant
  Fichier binaire /gnu/store/xbg7kwpylal63gqqb84185j54ib52aji-julia-1.6.1/share/julia/base.cache correspondant

Well, let first converge to a patch set updating Julia then let discuss
if this very same patch set fixes the bug or if the fix will be done in
another patch.  WDYT?


Cheers,
simon
Jean-Baptiste Volatier May 11, 2021, 1:46 p.m. UTC | #7
zimoun <zimon.toutoune@gmail.com> wrote:

>> ┌ Error: curl_easy_setopt: 48
>> └ @ Downloads.Curl /tmp/guix-build-julia-1.6.1.drv-0/julia-1.6.1/usr/share/julia/stdlib/v1.6/Downloads/src/Curl/utils.jl:36
>
> Soemthing appears to me wrong somewhere. Why does the build directory
> appear in the output directory? This reference should not appear, IMHO.

I think this is Julia related, during the build process the code from
the stdlib is precompiled in the sysimage and the paths are stored then.
I sometimes create custom sysimage to package some julia based apps
and I have the same behavior on a non-Guix system.
Some info about sysimages [1] from the doc of PackageCompiler, a project
that makes it easy to create custom sysimages.
That could also be a potential future improvement, make it easy for users
to create variants of the julia guix package with different sysimages.
That way we can make it much faster to load julia packages that are not part
of stdlib, Plots.jl for example.

[1] https://julialang.github.io/PackageCompiler.jl/dev/sysimages/#What-is-a-sysimage-1

> Well, let first converge to a patch set updating Julia then let discuss
> if this very same patch set fixes the bug or if the fix will be done in
> another patch. WDYT?

Agreed.

Cheers,
JB
diff mbox series

Patch

From e610dacab669ce84fe8f263a01aefff1fe49b6aa Mon Sep 17 00:00:00 2001
From: Jean-Baptiste Volatier <jbv@pm.me>
Date: Mon, 10 May 2021 09:57:23 +0200
Subject: [PATCH] gnu: julia: update to 1.6.1

gnu: openlibm: update to 0.7.4
gnu: pcre2: update to 10.56
gnu: utf8proc: update to 2.6.1
gnu: julia-benchmarktools: update to 0.7.0
---
 gnu/packages/julia-xyz.scm |   4 +-
 gnu/packages/julia.scm     | 293 +++++++++++++++++++++++++------------
 gnu/packages/maths.scm     |   6 +-
 gnu/packages/pcre.scm      |   4 +-
 gnu/packages/textutils.scm |  21 ++-
 5 files changed, 219 insertions(+), 109 deletions(-)

diff --git a/gnu/packages/julia-xyz.scm b/gnu/packages/julia-xyz.scm
index db3e3b846d..b6272c0b4d 100644
--- a/gnu/packages/julia-xyz.scm
+++ b/gnu/packages/julia-xyz.scm
@@ -100,7 +100,7 @@  be GPU compatible without throwing away the wrapper.")
 (define-public julia-benchmarktools
   (package
     (name "julia-benchmarktools")
-    (version "0.5.0")
+    (version "0.7.0")
     (source
      (origin
        (method git-fetch)
@@ -109,7 +109,7 @@  be GPU compatible without throwing away the wrapper.")
              (commit (string-append "v" version))))
        (file-name (git-file-name name version))
        (sha256
-        (base32 "0nsx21m3i5h22lkgyrmfj6r085va6ag40khwssqs8y7l0wz98lvp"))))
+        (base32 "000izw9pj7cbh9r35nnwg2ijkb9dpfd5nkl2889b8b2dpsh4fi63"))))
     (build-system julia-build-system)
     (propagated-inputs `(("julia-json" ,julia-json)))
     (home-page "https://github.com/JuliaCI/BenchmarkTools.jl")
diff --git a/gnu/packages/julia.scm b/gnu/packages/julia.scm
index 13c9f7baf1..39627eeed0 100644
--- a/gnu/packages/julia.scm
+++ b/gnu/packages/julia.scm
@@ -1,9 +1,10 @@ 
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2015, 2016, 2017 Ricardo Wurmus <rekado@elephly.net>
 ;;; Copyright © 2016, 2020 Efraim Flashner <efraim@flashner.co.il>
-;;; Copyright © 2020 Nicolò Balzarotti <nicolo@nixo.xyz>
+;;; Copyright © 2020, 2021 Nicolò Balzarotti <nicolo@nixo.xyz>
 ;;; Copyright © 2020 Tim Howes <timhowes@lavabit.com>
 ;;; Copyright © 2020 Tobias Geerinckx-Rice <me@tobias.gr>
+;;; Copyright © 2021 Jean-Baptiste Volatier <jbv@pm.me>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -30,6 +31,7 @@ 
   #:use-module (gnu packages)
   #:use-module (gnu packages algebra)
   #:use-module (gnu packages base)
+  #:use-module (gnu packages certs)
   #:use-module (gnu packages compression)
   #:use-module (gnu packages curl)
   #:use-module (gnu packages elf)
@@ -48,14 +50,15 @@ 
   #:use-module (gnu packages ssh)
   #:use-module (gnu packages tls)
   #:use-module (gnu packages version-control)
+  #:use-module (gnu packages web)
   #:use-module (gnu packages wget)
   #:use-module (ice-9 match))
 
 (define libuv-julia
-  (let ((commit "1fcc6d66f9df74189c74d3d390f02202bb7db953")
-        (revision "2"))
+  (let ((commit "fb3e3364c33ae48c827f6b103e05c3f0e78b79a9")
+        (revision "3"))
     ;; When upgrading Julia, also upgrade this.  Get the commit from
-    ;; https://github.com/JuliaLang/julia/blob/v1.5.2/deps/libuv.version
+    ;; https://github.com/JuliaLang/julia/blob/v1.6.0/deps/libuv.version
     (package
       (inherit libuv)
       (name "libuv-julia")
@@ -68,7 +71,7 @@ 
                 (file-name (string-append name "-" version "-checkout"))
                 (sha256
                  (base32
-                  "040l7f1hk7xyza11sry5cj4fhw05na949865axqqhxnifdvnmfji"))))
+                  "1kqpn19d20aka30h6q5h8lnzyp0vw0xzgx0wm4w2r5j6yf76m2hr"))))
       (build-system gnu-build-system)
       (arguments
        (substitute-keyword-arguments (package-arguments libuv)
@@ -97,7 +100,9 @@ 
          (julia-patch "libunwind-prefer-extbl"
                       "0lr4dafw8qyfh8sw8hhbwkql1dlhqv8px7k81y2l20hhxfgnh2m1")
          (julia-patch "libunwind-static-arm"
-                      "1jk3bmiw61ypcchqkk1fyg5wh8wpggk574wxyfyaic870zh3lhgq")))))
+                      "1jk3bmiw61ypcchqkk1fyg5wh8wpggk574wxyfyaic870zh3lhgq")
+         (julia-patch "libunwind-cfa-rsp"
+                      "1aswjhvysahhldbzh1afbf0hsjxrvs6xidsz2i7s1cjkjbdiia1z")))))
     (home-page "https://github.com/JuliaLang/tree/master/deps/")))
 
 (define (julia-patch-url version name)
@@ -105,7 +110,7 @@ 
                  "/deps/patches/" name ".patch"))
 
 (define (julia-patch name sha)
-  (let ((version "1.5.3"))
+  (let ((version "1.6.1"))
     (origin (method url-fetch)
             (uri (julia-patch-url version name))
             (sha256 (base32 sha))
@@ -113,52 +118,10 @@ 
 
 (define llvm-julia
   (package
-    (inherit llvm-9)
+    (inherit llvm-11)
     (name "llvm-julia")
-    (source (origin
-              (inherit (package-source llvm-9))
-              ;; Those patches are inside the Julia source repo.
-              ;; They are _not_ Julia specific (https://github.com/julialang/julia#llvm)
-              ;; but they are required to build Julia.
-              ;; Discussion: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=919628
-              (patches
-               (map (match-lambda
-                      ((name hash)
-                       (julia-patch name hash)))
-                    (list
-                     '("llvm-D27629-AArch64-large_model_6.0.1"
-                       "1qrshmlqvnasdyc158vfn3hnbigqph3lsq7acb9w8lwkpnnm2j4z")
-                     '("llvm8-D34078-vectorize-fdiv"
-                       "19spqc3xsazn1xs9gpcgv9ldadfkv49rmc5khl7sf1dlmhgi4602")
-                     '("llvm-7.0-D44650"
-                       "1h55kkmkiisfj6sk956if2bcj9s0v6n5czn8dxb870vp5nccj3ir")
-                     '("llvm9-D50010-VNCoercion-ni"
-                       "1s1d3sjsiq4vxg7ncy5cz56zgy5vcq6ls3iqaiqkvr23wyryqmdx")
-                     '("llvm-exegesis-mingw"
-                       "0ph1cj1j7arvf1xq2xcr7qf9g0cpdl14fincgr67vpi520zvd3vp")
-                     '("llvm-test-plugin-mingw"
-                       "12z738cnahbf6n381im7i0hxp1m6k9hrnfjlmq9sac46nxly9gnj")
-                     '("llvm7-revert-D44485"
-                       "0f59kq3p3mpwsbmskypbi4zn01l6ig0x7v2rjp08k2r8z8m6fa8n")
-                     '("llvm-8.0-D66657-codegen-degenerate"
-                       "1n1ddx19h90bbpimdyd9dh8fsm6gb93xxyqm4ljkxa1k3cx2vm72")
-                     '("llvm-8.0-D71495-vectorize-freduce"
-                       "1zff08wvji9lnpskk4b3p5zyjsy5hhy23ynxjqlj9dw7jvvfrf0p")
-                     '("llvm-D75072-SCEV-add-type"
-                       "029a3fywsm233vf48mscina24idd50dc75wr70lmimrhwnw27p0z")
-                     '("llvm-9.0-D65174-limit-merge-stores"
-                       "04bff1mnblfj9mxfdwr1qdnw3i3szmp60gnhxwas5y68qg33z6j0")
-                     '("llvm9-D71443-PPC-MC-redef-symbol"
-                       "1c93nv7rgc9jg5mqrnvv08xib1789qvlql94fwggh18mp3b9hbgy")
-                     '("llvm-9.0-D78196"
-                       "08a43hyg7yyqjq2vmfsmppf34xcz60wq6y9zw5fdyhw2h1mcnmns")
-                     '("llvm-julia-tsan-custom-as"
-                       "0awh40kf6lm4wn1nsjd1bmhfwq7rqj811szanp2xkpspykw9hg9s")
-                     '("llvm-9.0-D85499"
-                       "0vxlr35srvbvihlgrxq15v6dylp90vgi0qahj22j01jgqmdasjkm"))))
-              (patch-flags '("-p1"))))
     (arguments
-     (substitute-keyword-arguments (package-arguments llvm-9)
+     (substitute-keyword-arguments (package-arguments llvm-11)
        ((#:configure-flags flags)
         `(list ;; Taken from NixOS. Only way I could get libLLVM-6.0.so
            "-DCMAKE_BUILD_TYPE=Release"
@@ -177,7 +140,61 @@ 
            ;; "-DLLVM_DEFAULT_TARGET_TRIPLE=${stdenv.hostPlatform.config}"
            ;; "-DLLVM_EXPERIMENTAL_TARGETS_TO_BUILD=WebAssembly"
            "-DLLVM_ENABLE_DUMP=ON"
-           "-DLLVM_LINK_LLVM_DYLIB=ON"))))))
+           "-DLLVM_LINK_LLVM_DYLIB=ON"))
+       ((#:phases phases)
+        `(modify-phases ,phases
+           ;; applying patches from julia
+           ;; list of patches can be found in deps/llvm.mk in julia source
+           (add-after 'unpack 'julia-patches
+             (lambda* (#:key inputs outputs #:allow-other-keys)
+               (let ((patch
+                      (lambda (patchname flag)
+                        (invoke "patch" flag "-i"
+                                (string-append
+                                 "julia-src/deps/patches/"
+                                 patchname
+                                 ".patch")))))
+                 (mkdir-p "julia-src")
+                 (invoke "tar" "xf"
+                         (assoc-ref inputs "julia-source")
+                         "-C" "julia-src" "--strip-components=1")
+                 (map (lambda (patchname)
+                        (patch patchname "-p1"))
+                      (list "llvm-D27629-AArch64-large_model_6.0.1"
+                            "llvm8-D34078-vectorize-fdiv"
+                            "llvm-7.0-D44650"
+                            "llvm-6.0-DISABLE_ABI_CHECKS"
+                            "llvm9-D50010-VNCoercion-ni"
+                            "llvm7-revert-D44485"
+                            "llvm-11-D75072-SCEV-add-type"
+                            "llvm-julia-tsan-custom-as"
+                            "llvm-D80101"
+                            "llvm-D84031"
+                            "llvm-10-D85553"
+                            "llvm-10-unique_function_clang-sa"
+                            "llvm-11-D85313-debuginfo-empty-arange"
+                            "llvm-11-D90722-rtdyld-absolute-relocs"
+                            "llvm-invalid-addrspacecast-sink"
+                            "llvm-11-D92906-ppc-setjmp"
+                            "llvm-11-PR48458-X86ISelDAGToDAG"
+                            "llvm-11-D93092-ppc-knownbits"
+                            "llvm-11-D93154-globalisel-as"
+                            "llvm-11-ppc-half-ctr"
+                            "llvm-11-ppc-sp-from-bp"
+                            "llvm-rGb498303066a6-gcc11-header-fix"
+                            "llvm-11-D94813-mergeicmps"
+                            "llvm-11-D94980-CTR-half"
+                            "llvm-11-D94058-sext-atomic-ops"
+                            "llvm-11-D96283-dagcombine-half"))
+                 (map (lambda (patchname)
+                        (patch patchname "-p2"))
+                      (list "llvm-11-AArch64-FastIsel-bug"
+                            "llvm-11-D97435-AArch64-movaddrreg"
+                            "llvm-11-D97571-AArch64-loh"
+                            "llvm-11-aarch64-addrspace")))))))))
+    (inputs
+     `(("julia-source" ,(package-source julia))
+       ,@(package-inputs llvm-11)))))
 
 (define-public libwhich
   (package
@@ -223,7 +240,7 @@  libraries.  It is also a bit like @code{ldd} and @code{otool -L}.")
 (define-public julia
   (package
     (name "julia")
-    (version "1.5.3")
+    (version "1.6.1")
     (source (origin
               (method url-fetch)
               (uri (string-append
@@ -231,9 +248,7 @@  libraries.  It is also a bit like @code{ldd} and @code{otool -L}.")
                     version "/julia-" version ".tar.gz"))
               (sha256
                (base32
-                "1zmim82x9kkdcgn0cdi01hmzi59zbszy1sqlygb86xq4hc1n66dy"))
-              (patches
-               (search-patches "julia-SOURCE_DATE_EPOCH-mtime.patch"))))
+                "1mfzbjyqcmx7wb1sa7qab5fl78yzd7ap088krqbphbwvpn880srn"))))
     (build-system gnu-build-system)
     (arguments
      `(#:test-target "test"
@@ -259,14 +274,13 @@  libraries.  It is also a bit like @code{ldd} and @code{otool -L}.")
                      (string-join (map (lambda (pkg)
                                          (string-append (assoc-ref inputs pkg)
                                                         "/lib"))
-                                       '("arpack-ng" "curl" "dsfmt"
+                                       '("curl" "dsfmt"
                                          "gmp" "lapack"
-                                         "libssh2" "libgit2"
+                                         "libssh2" "libnghttp2" "libgit2"
                                          "mbedtls" "mpfr"
                                          "openblas" "openlibm" "pcre2"
-                                         "suitesparse"))
-                                  ":"))
-             #t))
+                                         "suitesparse" "libfortran"))
+                                  ":"))))
          ;; FIXME: Building the documentation requires Julia packages that
          ;; would be downloaded from the Internet.  We should build them in a
          ;; separate build phase.
@@ -304,7 +318,7 @@  libraries.  It is also a bit like @code{ldd} and @code{otool -L}.")
                 "$(BUILDDIR)/$(EXENAME)-debug\\$\\(EXE\\): $(DOBJS) $(LLT_debug)"))
 
              ;; The REPL must be linked with libuv.
-             (substitute* "ui/Makefile"
+             (substitute* "cli/Makefile"
                (("JLDFLAGS \\+= ")
                 (string-append "JLDFLAGS += "
                                (assoc-ref %build-inputs "libuv")
@@ -320,27 +334,83 @@  libraries.  It is also a bit like @code{ldd} and @code{otool -L}.")
              (substitute* "base/client.jl"
                (("/bin/sh") (which "sh")))
              #t))
-         (add-before 'build 'fix-precompile
-           (lambda _
-             (substitute* "base/loading.jl"
-               (("something(Base.active_project(), \"\")") "\"\""))
+         (add-before 'build 'shared-objects-paths
+           (lambda* (#:key inputs #:allow-other-keys)
+             (let ((jlpath
+                    (lambda (pkgname)
+                      (string-append
+                       "stdlib/" pkgname "_jll/src/" pkgname "_jll.jl")))
+                   (from
+                    (lambda (libname)
+                      (string-append "const " libname " = .*\\.so")))
+                   (to
+                    (lambda* (pkg libname #:optional libname_jl)
+                      (string-append
+                       "const " (or libname_jl libname)  "= \""
+                       (assoc-ref inputs pkg) "/lib/" libname ".so"))))
+               (substitute* (jlpath "dSFMT")
+                 (((from "libdSFMT")) (to "dsfmt" "libdSFMT")))
+               (substitute* (jlpath "GMP")
+                 (((from "libgmp")) (to "gmp" "libgmp"))
+                 (((from "libgmpxx")) (to "gmp" "libgmpxx")))
+               (substitute* (jlpath "libLLVM")
+                 (((from "libLLVM")) (to "llvm" "libLLVM")))
+               (substitute* (jlpath "LibCURL")
+                 (((from "libcurl")) (to "curl" "libcurl")))
+               (substitute* (jlpath "LibGit2")
+                 (((from "libgit2")) (to "libgit2" "libgit2")))
+               (substitute* (jlpath "LibSSH2")
+                 (((from "libssh2")) (to "libssh2" "libssh2")))
+               (substitute* (jlpath "LibUV")
+                 (((from "libuv")) (to "libuv" "libuv")))
+               (substitute* (jlpath "LibUnwind")
+                 (((from "libunwind")) (to "libunwind" "libunwind")))
+               (substitute* (jlpath "MPFR")
+                 (((from "libmpfr")) (to "mpfr" "libmpfr")))
+               (substitute* (jlpath "MbedTLS")
+                 (((from "libmbedcrypto")) (to "mbedtls" "libmbedcrypto"))
+                 (((from "libmbedtls")) (to "mbedtls" "libmbedtls"))
+                 (((from "libmbedx509")) (to "mbedtls" "libmbedx509")))
+               (substitute* (jlpath "nghttp2")
+                 (((from "libnghttp2")) (to "libnghttp2" "libnghttp2")))
+               ;; FAILING: OpenBLAS
+               (substitute* (jlpath "OpenBLAS")
+                 (((from "libopenblas")) (to "openblas" "libopenblas")))
+               (substitute* (jlpath "OpenLibm")
+                 (((from "libopenlibm")) (to "openlibm" "libopenlibm")))
+               (substitute* (jlpath "PCRE2")
+                 (((from "libpcre2")) (to "pcre2" "libpcre2" "libpcre2_8")))
+               (substitute* (jlpath "SuiteSparse")
+                 (((from "libamd")) (to "suitesparse" "libamd"))
+                 (((from "libbtf")) (to "suitesparse" "libbtf"))
+                 (((from "libcamd")) (to "suitesparse" "libcamd"))
+                 (((from "libccolamd")) (to "suitesparse" "libccolamd"))
+                 (((from "libcholmod")) (to "suitesparse" "libcholmod"))
+                 (((from "libcolamd")) (to "suitesparse" "libcolamd"))
+                 (((from "libklu")) (to "suitesparse" "libklu"))
+                 (((from "libldl")) (to "suitesparse" "libldl"))
+                 (((from "librbio")) (to "suitesparse" "librbio"))
+                 (((from "libspqr")) (to "suitesparse" "libspqr"))
+                 (((from "libsuitesparse")) (to "suitesparse" "libsuitesparse"))
+                 (((from "libsuitesparseconfig")) (to "suitesparse" "libsuitesparseconfig"))
+                 (((from "libumfpack")) (to "suitesparse" "libumfpack")))
+               (substitute* (jlpath "Zlib")
+                 (((from "libz")) (to "zlib" "libz"))))
              #t))
+         (add-before 'check 'fix-setenv
+           (lambda* (#:key inputs #:allow-other-keys)
+             ;; some tests execute julia in an environment that needs to propagate GUIX_LOCPATH
+             (substitute* "test/cmdlineargs.jl"
+               (("\"HOME\"\\s=>\\shomedir\\(\\)")
+                "\"HOME\" => homedir(), \"GUIX_LOCPATH\" => ENV[\"GUIX_LOCPATH\"]"))))
          (add-before 'check 'disable-broken-tests
            (lambda _
              (substitute* "test/choosetests.jl"
-               (("tests = testnames")
-                ;; Those failings are not deterministic.  They depends on the
-                ;; running order.  I think it depends on the number of
-                ;; runners, disabling it for now
-                ;; https://github.com/JuliaLang/julia/issues/34330
-                "tests = filter(e->!in(e,[\"backtrace\",\"exceptions\",\"precompile\",
-                                           \"client\",\"stacktraces\"]),
-                                       testnames)"))
-             ;; Marking the test as broken as it's a known bug:
-             ;; https://github.com/JuliaLang/julia/issues/32377
-             (substitute* "stdlib/REPL/test/replcompletions.jl"
-               (("@test count") "@test_broken count"))
-             ;; Dates has a similar bug:
+               (("skip_tests = \\[\\]")
+                ;; disabling REPL tests because they require a stdin
+                ;; disabling tests for MPFR, MbedTLS and SuiteSparse because they only check for hardcoded versions
+                "skip_tests = [\"REPL\", \"MPFR_jll\", \"MbedTLS_jll\", \"SuiteSparse_jll\"]"))
+             ;; Dates/io tests fail on master when networking is unavailable
              ;; https://github.com/JuliaLang/julia/issues/34655
              (substitute* "stdlib/Dates/test/io.jl"
                (("\"Dates.Date") "\"Date")
@@ -349,11 +419,43 @@  libraries.  It is also a bit like @code{ldd} and @code{otool -L}.")
              ;; https://github.com/JuliaLang/julia/issues/35785
              (substitute* "test/file.jl"
                (("@test dirname\\(t\\) == d") "@test_broken dirname(t) == d"))
-             ;; Deprecation test fails with --depwarn=no
-             ;; https://github.com/JuliaLang/julia/issues/37673
-             (substitute* "test/Makefile"
-               (("./runtests.jl") "--depwarn=error ./runtests.jl"))
+             ;; julia embeds a certificate, we are not doing that
+             (substitute* "stdlib/MozillaCACerts_jll/test/runtests.jl"
+               (("@test isfile\\(MozillaCACerts_jll.cacert\\)") "@test_broken isfile(MozillaCACerts_jll.cacert)"))
+             ;; since certificate is not present some tests are failing in network option
+             (substitute* "usr/share/julia/stdlib/v1.6/NetworkOptions/test/runtests.jl"
+               (("@test isfile\\(bundled_ca_roots\\(\\)\\)") "@test_broken isfile(bundled_ca_roots())")
+               (("@test ispath\\(ca_roots_path\\(\\)\\)") "@test_broken ispath(ca_roots_path())")
+               (("@test ca_roots_path\\(\\) \\!= bundled_ca_roots\\(\\)") "@test_broken ca_roots_path() != bundled_ca_roots()"))
              #t))
+         (add-before 'install 'symlink-libraries
+           (lambda* (#:key inputs outputs #:allow-other-keys)
+             (let ((link
+                    (lambda (pkgname dir pred)
+                      (map (lambda (file)
+                             (invoke "ln" "-s" file dir))
+                           (find-files (string-append (assoc-ref inputs pkgname) "/lib") pred)))))
+               (link "curl" "usr/lib/" "\\.so") ;; missing libpthreads libLLVM-11jl
+               (link "suitesparse" "usr/lib/julia/" "libbtf\\.so")
+               (link "suitesparse" "usr/lib/julia/" "libklu\\.so")
+               (link "suitesparse" "usr/lib/julia/" "libldl\\.so")
+               (link "suitesparse" "usr/lib/julia/" "librbio\\.so")
+               (link "gmp" "usr/lib/julia/" "libgmpxx\\.so")
+               (link "libuv" "usr/lib/julia/" "libuv\\.so")
+               (link "zlib" "usr/lib/julia/" "libz\\.so")
+               (link "libunwind" "usr/lib/julia/" "libunwind\\.so")
+               (invoke "ln" "-s" (string-append (assoc-ref inputs "p7zip") "/bin/7z") "usr/bin/")
+               #t)))
+         (add-after 'install 'symlink-llvm-utf8proc
+           (lambda* (#:key inputs outputs #:allow-other-keys)
+             (let ((link
+                    (lambda (pkgname pred)
+                      (map (lambda (file)
+                             (invoke "ln" "-s" file (string-append (assoc-ref outputs "out") "/lib/julia/")))
+                           (find-files (string-append (assoc-ref inputs pkgname) "/lib") pred)))))
+               (link "llvm" "libLLVM-11\\.so")
+               (link "utf8proc" "libutf8proc\\.so")
+               #t)))
          (add-after 'install 'make-wrapper
            (lambda* (#:key inputs outputs #:allow-other-keys)
              (let* ((out (assoc-ref outputs "out"))
@@ -384,7 +486,6 @@  libraries.  It is also a bit like @code{ldd} and @code{otool -L}.")
 
          "CONFIG_SHELL=bash"     ;needed to build bundled libraries
          ;; list of "USE_SYSTEM_*" is here:
-         ;; https://github.com/JuliaLang/julia/blob/v1.3.1/Make.inc
          "USE_SYSTEM_DSFMT=1"
          "USE_SYSTEM_P7ZIP=1"
          "USE_SYSTEM_LAPACK=1"
@@ -392,6 +493,8 @@  libraries.  It is also a bit like @code{ldd} and @code{otool -L}.")
          "USE_BLAS64=0"          ;needed when USE_SYSTEM_BLAS=1
          "LIBBLAS=-lopenblas"
          "LIBBLASNAME=libopenblas"
+         ;; https://github.com/JuliaLang/julia/blob/v1.6.0/Make.inc
+         "USE_SYSTEM_CSL=1"
 
          "USE_SYSTEM_SUITESPARSE=1"
          (string-append "SUITESPARSE_INC=-I "
@@ -403,7 +506,7 @@  libraries.  It is also a bit like @code{ldd} and @code{otool -L}.")
                         (assoc-ref %build-inputs "utf8proc")
                         "/include")
          "USE_SYSTEM_LLVM=1"
-         "LLVM_VER=9.0.1"
+         "LLVM_VER=11.0.0"
 
          "USE_LLVM_SHLIB=1"
          "USE_SYSTEM_LIBUNWIND=1"
@@ -416,43 +519,43 @@  libraries.  It is also a bit like @code{ldd} and @code{otool -L}.")
                         "/include")
          "USE_SYSTEM_PATCHELF=1"
          "USE_SYSTEM_PCRE=1"
+         "USE_SYSTEM_NGHTTP2=1"
          "USE_SYSTEM_OPENLIBM=1"
          "USE_SYSTEM_MBEDTLS=1"
          "USE_SYSTEM_LIBSSH2=1"
          "USE_SYSTEM_GMP=1"
          "USE_SYSTEM_MPFR=1"
-         "USE_SYSTEM_ARPACK=1"
          "USE_SYSTEM_LIBGIT2=1"
          "USE_SYSTEM_ZLIB=1")))
     (inputs
      `(("llvm" ,llvm-julia)
        ("p7zip" ,p7zip)
-       ;; The bundled version is 3.3.0 so stick to that version.  With other
-       ;; versions, we get test failures in 'linalg/arnoldi' as described in
-       ;; <https://bugs.gnu.org/30282>.
-       ("arpack-ng" ,arpack-ng-3.3.0)
-
-       ("coreutils" ,coreutils) ;for bindings to "mkdir" and the like
+       ("coreutils" ,coreutils)         ;for bindings to "mkdir" and the like
        ("lapack" ,lapack)
-       ("openblas" ,openblas) ;Julia does not build with Atlas
+       ("openblas" ,openblas)           ;Julia does not build with Atlas
        ("libunwind" ,libunwind-julia)
        ("openlibm" ,openlibm)
        ("mbedtls" ,mbedtls-apache)
        ("curl" ,curl)
-       ("libgit2" ,libgit2-0.28)
+       ("libnghttp2" ,nghttp2 "lib")
+       ("libgit2" ,libgit2)
        ("libssh2" ,libssh2)
        ("fortran" ,gfortran)
+       ;; required for libgcc_s.so
+       ("libfortran" ,gfortran "lib")
        ("libuv" ,libuv-julia)
        ("pcre2" ,pcre2)
        ("utf8proc" ,utf8proc)
        ("mpfr" ,mpfr)
+       ("nss-certs" ,nss-certs)         ; required to precompile
+       ("glibc-locales" ,glibc-locales)
        ("wget" ,wget)
        ("which" ,which)
        ("zlib" ,zlib)
        ("gmp" ,gmp)
        ("suitesparse" ,suitesparse)
        ;; Find dependencies versions here:
-       ;; https://raw.githubusercontent.com/JuliaLang/julia/v1.3.0/deps/Versions.make
+       ;; https://raw.githubusercontent.com/JuliaLang/julia/v1.6.0/deps/Versions.make
        ("libwhich" ,libwhich)
        ("dsfmt" ,dsfmt)))
     (native-inputs
diff --git a/gnu/packages/maths.scm b/gnu/packages/maths.scm
index 9f7bdb9bb1..7be5d418e7 100644
--- a/gnu/packages/maths.scm
+++ b/gnu/packages/maths.scm
@@ -3959,7 +3959,7 @@  access to BLIS implementations via traditional BLAS routine calls.")
 (define-public openlibm
   (package
     (name "openlibm")
-    (version "0.6.0")
+    (version "0.7.4")
     (source
      (origin
        (method git-fetch)
@@ -3968,11 +3968,11 @@  access to BLIS implementations via traditional BLAS routine calls.")
              (commit (string-append "v" version))))
        (file-name (git-file-name name version))
        (sha256
-        (base32 "08wfchmmr5200fvmn1kwq9byc1fhsq46hn0y5k8scdl74771c7gh"))))
+        (base32 "1azms0lpxb7vxb3bln5lyz0wpwx6jnzbffkclclpq2v5aiw8d14i"))))
     (build-system gnu-build-system)
     (arguments
      `(#:make-flags
-       (list (string-append "prefix=" (assoc-ref %outputs "out")))
+       (list (string-append "prefix=" (assoc-ref %outputs "out")) "CC=gcc")
        #:phases
        ;; no configure script
        (modify-phases %standard-phases (delete 'configure))
diff --git a/gnu/packages/pcre.scm b/gnu/packages/pcre.scm
index ab48168567..42b2052598 100644
--- a/gnu/packages/pcre.scm
+++ b/gnu/packages/pcre.scm
@@ -89,14 +89,14 @@  POSIX regular expression API.")
 (define-public pcre2
   (package
     (name "pcre2")
-    (version "10.35")
+    (version "10.36")
     (source (origin
               (method url-fetch)
               (uri (string-append "mirror://sourceforge/pcre/pcre2/"
                                   version "/pcre2-" version ".tar.bz2"))
               (sha256
                (base32
-                "04s6kmk9qdd4rjz477h547j4bx7hfz0yalpvrm381rqc5ghaijww"))))
+                "0p3699msps07p40g9426lvxa3b41rg7k2fn7qxl2jm0kh4kkkvx9"))))
    (build-system gnu-build-system)
    (inputs `(("bzip2" ,bzip2)
              ("readline" ,readline)
diff --git a/gnu/packages/textutils.scm b/gnu/packages/textutils.scm
index ab34373705..d82675bf79 100644
--- a/gnu/packages/textutils.scm
+++ b/gnu/packages/textutils.scm
@@ -63,6 +63,7 @@ 
   #:use-module (gnu packages python-build)
   #:use-module (gnu packages python-xyz)
   #:use-module (gnu packages readline)
+  #:use-module (gnu packages ruby)
   #:use-module (gnu packages slang)
   #:use-module (gnu packages web))
 
@@ -146,7 +147,7 @@  libenca and several charset conversion libraries and tools.")
 (define-public utf8proc
   (package
     (name "utf8proc")
-    (version "2.5.0")
+    (version "2.6.1")
     (source
      (origin
        (method git-fetch)
@@ -155,7 +156,7 @@  libenca and several charset conversion libraries and tools.")
              (commit (string-append "v" version))))
        (file-name (git-file-name name version))
        (sha256
-        (base32 "1xlkazhdnja4lksn5c9nf4bln5gjqa35a8gwlam5r0728w0h83qq"))))
+        (base32 "1zqc6airkzkssbjxanx5v8blfk90180gc9id0dx8ncs54f1ib8w7"))))
     (build-system gnu-build-system)
     (native-inputs
      (let ((UNICODE_VERSION "13.0.0"))  ; defined in data/Makefile
@@ -167,6 +168,13 @@  libenca and several charset conversion libraries and tools.")
                                  UNICODE_VERSION "/ucd/NormalizationTest.txt"))
              (sha256
               (base32 "07g0ya4f6zfzvpp24ccxkb2yq568kh83gls85rjl950nv5fya3nn"))))
+         ("DerivedCoreProperties.txt"
+          ,(origin
+             (method url-fetch)
+             (uri (string-append "https://www.unicode.org/Public/"
+                                 UNICODE_VERSION "/ucd/DerivedCoreProperties.txt"))
+             (sha256
+              (base32 "0j12x112cd8fpgazkc8izxnhhpia44p1m36ff8yapslxndcmzm55"))))
          ("GraphemeBreakTest.txt"
           ,(origin
              (method url-fetch)
@@ -177,10 +185,12 @@  libenca and several charset conversion libraries and tools.")
               (base32 "07f8rrvcsq4pibdz6zxggxy8w7zjjqyw2ggclqlhalyv45yv7prj"))))
 
          ;; For tests.
-         ("perl" ,perl))))
+         ("perl" ,perl)
+         ("ruby" ,ruby))))
     (arguments
      '(#:make-flags (list "CC=gcc"
                           (string-append "prefix=" (assoc-ref %outputs "out")))
+       #:tests? #t
        #:phases
        (modify-phases %standard-phases
          (delete 'configure)
@@ -189,10 +199,7 @@  libenca and several charset conversion libraries and tools.")
              (for-each (lambda (i)
                          (copy-file (assoc-ref inputs i)
                                     (string-append "data/" i)))
-                       '("NormalizationTest.txt" "GraphemeBreakTest.txt"))
-             (substitute* "data/GraphemeBreakTest.txt"
-               (("÷") "/")
-               (("×") "+"))
+                       '("NormalizationTest.txt" "DerivedCoreProperties.txt" "GraphemeBreakTest.txt"))
              #t)))))
     (home-page "https://juliastrings.github.io/utf8proc/")
     (synopsis "C library for processing UTF-8 Unicode data")
-- 
2.31.1