| Message ID | 87v9qpvbyr.fsf@guixSD.i-did-not-set--mail-host-address--so-tickle-me |
|---|---|
| Headers | show |
Hi! nixo <anothersms@gmail.com> skribis: > This set of patches improve the state of julia-build-system. > > 1. set path JULIA_LOAD_PATH to enable installing packages, and > JULIA_DEPOT_PATH to allow guix-precompiled cache to be recognized by julia. To These two environment variables are supported upstream, right? > do this, a small "hack" with libfaketime was required (julia decides if the > cache is valid by comparing source ".jl" mtime with cache ".ji" file, in which > julia stores a timestamp. We need to force this timestamp to be equal to the > mtime of guix-store files). OK. This hack shouldn’t break “normal” Julia uses (outside of the store), right? Ludo’.
Ok I had time to look at this again. 1. Package installation can be done both the julia way and the guix way 2. SOURCE_DATE_EPOCH is respected in package precompile timestamp. Other fixes are needed to get reproducible builds, but for now that means that package cache is used. 3. This patch adds some first julia package (HTTP.jl and dependencies) 4. Updated julia to 1.3.1 (latest release). It's working great and the build recipe is simplified a lot. However, some test is failing. The "interesting" part is that when running tests alone, they are working. It seems like the problem arise when tests are run on the _same core_ and not on different cores. I opened a issue upstream for this (https://github.com/JuliaLang/julia/issues/34330), I'll have to look further on it, but for now if it's fine for me this is ready to be merged. I'm sending the patch series here. Thanks, Nicolò Ludovic Courtès <ludo@gnu.org> writes: > Hi, > > (+Cc: bug report.) > > Nicolò Balzarotti <anothersms@gmail.com> skribis: > >> Ludovic Courtès <ludo@gnu.org> writes: >> >>> Hi! >>> >>> nixo <anothersms@gmail.com> skribis: >>> >>>> This set of patches improve the state of julia-build-system. >>>> >>>> 1. set path JULIA_LOAD_PATH to enable installing packages, and >>>> JULIA_DEPOT_PATH to allow guix-precompiled cache to be recognized by julia. To >>> >>> These two environment variables are supported upstream, right? >> >> Yes, those variables are documented here: >> https://docs.julialang.org/en/v1/manual/environment-variables/index.html > > Great. > >> Should user be allowed to manually install packages in his own private >> home folder manually? (I just tried with R: R recognize the store is RO >> and ask for an alternative path). I'll try to allow "standard" package >> installation and submit the patch again (also, I just read you other >> comments, so I'll fix everything!) > > I think users should have the option to install packages in the normal > Julia way, without Guix. > >>>> do this, a small "hack" with libfaketime was required (julia decides if the >>>> cache is valid by comparing source ".jl" mtime with cache ".ji" file, in which >>>> julia stores a timestamp. We need to force this timestamp to be equal to the >>>> mtime of guix-store files). >>> >>> OK. This hack shouldn’t break “normal” Julia uses (outside of the >>> store), right? >> You are right, the issue is that julia finds its src (.jl) file to have >> a different (although older) timestamp (because guix fixes the >> mtime). This applies only to cache files placed in the store. Other >> cache files (non-guix installed files) have the right timestamp, so are >> not affected (I don't know if this explanation is clear. The TLDR is: >> with this hack store works without double-precompilation, non-store >> works normally, too). >> However, I'll check if I can make it work with SOURCE_DATE_EPOCH. I >> already read the code responsible for precompilation, so hopefully it >> won't take too much time > > Awesome, thanks! > > Ludo’.
Ok I had time to look at this again. 1. Package installation can be done both the julia way and the guix way 2. SOURCE_DATE_EPOCH is respected in package precompile timestamp. Other fixes are needed to get reproducible builds, but for now that means that package cache is used. 3. This patch adds some first julia package (HTTP.jl and dependencies) 4. Updated julia to 1.3.1 (latest release). It's working great and the build recipe is simplified a lot. However, some test is failing. The "interesting" part is that when running tests alone, they are working. It seems like the problem arise when tests are run on the _same core_ and not on different cores. I opened a issue upstream for this (https://github.com/JuliaLang/julia/issues/34330), I'll have to look further on it, but for now if it's fine for me this is ready to be merged. I'm sending the patch series here. Thanks, Nicolò Ludovic Courtès <ludo@gnu.org> writes: > Hi, > > (+Cc: bug report.) > > Nicolò Balzarotti <anothersms@gmail.com> skribis: > >> Ludovic Courtès <ludo@gnu.org> writes: >> >>> Hi! >>> >>> nixo <anothersms@gmail.com> skribis: >>> >>>> This set of patches improve the state of julia-build-system. >>>> >>>> 1. set path JULIA_LOAD_PATH to enable installing packages, and >>>> JULIA_DEPOT_PATH to allow guix-precompiled cache to be recognized by julia. To >>> >>> These two environment variables are supported upstream, right? >> >> Yes, those variables are documented here: >> https://docs.julialang.org/en/v1/manual/environment-variables/index.html > > Great. > >> Should user be allowed to manually install packages in his own private >> home folder manually? (I just tried with R: R recognize the store is RO >> and ask for an alternative path). I'll try to allow "standard" package >> installation and submit the patch again (also, I just read you other >> comments, so I'll fix everything!) > > I think users should have the option to install packages in the normal > Julia way, without Guix. > >>>> do this, a small "hack" with libfaketime was required (julia decides if the >>>> cache is valid by comparing source ".jl" mtime with cache ".ji" file, in which >>>> julia stores a timestamp. We need to force this timestamp to be equal to the >>>> mtime of guix-store files). >>> >>> OK. This hack shouldn’t break “normal” Julia uses (outside of the >>> store), right? >> You are right, the issue is that julia finds its src (.jl) file to have >> a different (although older) timestamp (because guix fixes the >> mtime). This applies only to cache files placed in the store. Other >> cache files (non-guix installed files) have the right timestamp, so are >> not affected (I don't know if this explanation is clear. The TLDR is: >> with this hack store works without double-precompilation, non-store >> works normally, too). >> However, I'll check if I can make it work with SOURCE_DATE_EPOCH. I >> already read the code responsible for precompilation, so hopefully it >> won't take too much time > > Awesome, thanks! > > Ludo’.
Hello Guix! Changes: This set of patches improve the state of julia-build-system. 1. set path JULIA_LOAD_PATH to enable installing packages, and JULIA_DEPOT_PATH to allow guix-precompiled cache to be recognized by julia. To do this, a small "hack" with libfaketime was required (julia decides if the cache is valid by comparing source ".jl" mtime with cache ".ji" file, in which julia stores a timestamp. We need to force this timestamp to be equal to the mtime of guix-store files). 2. Enable package tests 3. Add a first package (julia-compat) definition. If this is fine, I'll start pushing many many more (I have them ready, but I need to update many of them) Thanks! :) One note: with this PATH changes, julia will not be able to find packages installed by Julia Pkg3.jl anymore. This is expected (it should happen for R, python, emacs... too) but since we still need to merge the packages, this patch renders julia pretty unusable. The workaround is for users to add to JULIA_LOAD_PATH and JULIA_LOAD_PATH local user-writable folders, activate a profile, and then install packages manually: #+begin_src bash JULIA_LOAD_PATH=/tmp/julia:$JULIA_LOAD_PATH JULIA_DEPOT_PATH=/tmp/julia:$JULIA_DEPOT_PATH julia #+end_src nixo (3): gnu: julia: Set to JULIA_LOAD_PATH and JULIA_DEPOT_PATH variables. gnu: julia-build-system: Enable tests. gnu: julia-xyz: Add julia-compat. #+begin_src julia using Pkg Pkg.activate("/tmp/julia") Pkg.add("PACKAGE_NAME") #+end_src