From patchwork Tue Jan 21 13:45:20 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Nicol=C3=B2_Balzarotti?= X-Patchwork-Id: 19933 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id E26D017AE5; Tue, 21 Jan 2020 14:27:25 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,FREEMAIL_FROM, T_DKIM_INVALID,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTP id C00B217AB2 for ; Tue, 21 Jan 2020 14:27:22 +0000 (GMT) Received: from localhost ([::1]:55122 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ituUy-0008Jy-Vm for patchwork@mira.cbaines.net; Tue, 21 Jan 2020 09:27:22 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:35209) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ittr4-0002Uo-DO for guix-patches@gnu.org; Tue, 21 Jan 2020 08:46:10 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ittr0-0005Rc-Rv for guix-patches@gnu.org; Tue, 21 Jan 2020 08:46:06 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:40652) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1ittr0-0005RR-L6 for guix-patches@gnu.org; Tue, 21 Jan 2020 08:46:02 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1ittr0-0000RA-IB for guix-patches@gnu.org; Tue, 21 Jan 2020 08:46:02 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#38546] [00/11] Update Julia, Fix precompilation, add HTTP.jl Resent-From: =?utf-8?q?Nicol=C3=B2?= Balzarotti Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Tue, 21 Jan 2020 13:46:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 38546 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: zimoun Cc: Ludovic =?utf-8?q?Court=C3=A8s?= , 38546@debbugs.gnu.org Received: via spool by 38546-submit@debbugs.gnu.org id=B38546.15796143491016 (code B ref 38546); Tue, 21 Jan 2020 13:46:02 +0000 Received: (at 38546) by debbugs.gnu.org; 21 Jan 2020 13:45:49 +0000 Received: from localhost ([127.0.0.1]:46625 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ittqd-00007v-Gd for submit@debbugs.gnu.org; Tue, 21 Jan 2020 08:45:49 -0500 Received: from mail-wm1-f42.google.com ([209.85.128.42]:51197) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ittqT-0008KR-QX for 38546@debbugs.gnu.org; Tue, 21 Jan 2020 08:45:38 -0500 Received: by mail-wm1-f42.google.com with SMTP id a5so3026242wmb.0 for <38546@debbugs.gnu.org>; Tue, 21 Jan 2020 05:45:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:in-reply-to:references:date:message-id :mime-version; bh=d8kcpDpD3Y3T8CG5PiGasloqS2xL1IuVNgTGtcgq/GU=; b=vKkQA8XIRptzjOX9Y0ahKXyxjLB05ODtvTAfS97PITrN9Xwt9axxXkMQPNURrJMzo6 2PrKjS/Fc4o6iiT0rr5SjVN6YWwos3UajBAEtHXVwHtjv2nC0BndjKGEmOwi4yhD1IXJ P1CrYfg/2KOQHaNqqJOEachzM6sXp2+c4qnhNSefjCN9Ab5IpKGf+gtpNGN7VL99ohFc pXbeaNZY11yzftlxGmM6ImYQwoWtFlH+iHGpsT16Vk7PFqvSFWdSsDsNlvWF5/XYpPz8 hIH0wltpfUITI8wDW19eU/nr+gmVEPBU8CiOtgz6IhYpXNG2kZyrJchC81bxMGf/XCs1 FB2g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:in-reply-to:references:date :message-id:mime-version; bh=d8kcpDpD3Y3T8CG5PiGasloqS2xL1IuVNgTGtcgq/GU=; b=pomHLwfCUhUroxtrFMieAJcm6UfJIDQJ0CpepVsBx1QoEmau4DYQJ8CJCrFPf9dtbU mkd4WBO8pJ/GPZXD9TEKhwHU4bTO6DEmdpnZj+ovZ/E4BaUtrMPFAcisIcn78RmgL7/d CQVM6kDtnOiy8AGF+bdjzTlMxeD6cDEDSh2RWsqkzNpUz8sJ4v0msQbXXhZuGtDYccTC bULYku5Iw5f+3p3q9huJ7gcdtF4Y1DPIdMPh2nBhfEhTnvTY0iCbAIfg01vl1VEvVZ1c gujrnzyivNV1XZiVS1N7JBsCXW/7jjFmPaXnSgKTWUwU6sSittuvjCwpCwUXu3VqB7nl PqOg== X-Gm-Message-State: APjAAAX+3W4Io1w+1t4a1R+NCdW7xgrrZK2IHsLHKwbRAZ3hXytus4yt wfrPv7g2ftLouD2HImHnUQcPXc2V2bU= X-Google-Smtp-Source: APXvYqwVnfNmBF62+Z3k0hvaI9oV2s0gNrVk0QTXBmENEEoO3rv+zUffgNhW+StV3GKsk9ZTRN57oA== X-Received: by 2002:a1c:6605:: with SMTP id a5mr4274492wmc.112.1579614323527; Tue, 21 Jan 2020 05:45:23 -0800 (PST) Received: from guixSD ([192.167.192.190]) by smtp.gmail.com with ESMTPSA id 60sm53474838wrn.86.2020.01.21.05.45.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 21 Jan 2020 05:45:22 -0800 (PST) From: =?utf-8?q?Nicol=C3=B2?= Balzarotti In-Reply-To: References: <87v9qpvbyr.fsf@guixSD.i-did-not-set--mail-host-address--so-tickle-me> <87wo9ouaq5.fsf@guixSD.i-did-not-set--mail-host-address--so-tickle-me> Date: Tue, 21 Jan 2020 14:45:20 +0100 Message-ID: <87d0bcly8f.fsf@guixSD.i-did-not-set--mail-host-address--so-tickle-me> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.51.188.43 X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches Hi Simon, Sorry, I forgot to send the dsfmt patch. Also, julia's SOURCE_DATE_EPOCH patch was named differently. I've fixed this in theattached patches. You need to apply Add-dsfmt.patch, Update-to-1.3.1 and then julia-use-SOURCE_DATE_EPOCH. About reproducibility: if I'm not wrong, sys.so contains Base library precompiled ([1]). Precompilation is still non deterministic (here's [2] an issue on github). Something I did to check precompilation: Since I'm tired of this, I've started investigating better. Loaded a guix container #+begin_src bash mkdir -p /tmp/jlprecompiled/ ./pre-inst-env guix environment -u nixbld --no-cwd --container --network --expose=$(realpath determinism) --share=/tmp/jlprecompiled/ --ad-hoc julia coreutils nss-certs curl tar gzip libfaketime #+end_src from this container, run some time this: #+begin_src bash export SSL_CERT_DIR="$GUIX_ENVIRONMENT/etc/ssl/certs" export SSL_CERT_FILE="$SSL_CERT_DIR/ca-certificates.crt" export GIT_SSL_CAINFO="$SSL_CERT_FILE" SOURCE_DATE_EPOCH=1 julia --startup-file=no -q -e 'using Pkg; Pkg.instantiate()' SOURCE_DATE_EPOCH=1 faketime '2008-12-24 08:15:42' julia --startup-file=no -q -e 'using Random; using Pkg; Pkg.add("Compat"); Random.seed!(0); using Compat' cp -r ~/.julia/compiled/v1.3/Compat/ /tmp/jlprecompiled/$RANDOM sha256sum ~/.julia/compiled/v1.3/Compat/* # 6c10d65f6cd9cdd5b98e4d5e2b82b7bd00508f0daccef0967dec92cb6468052e /home/nixbld/.julia/compiled/v1.3/Compat/GSFWK_Pua4I.ji ls -lah ~/.julia/compiled/v1.3/Compat/* # -rw------- 1 nixbld users 102K Jan 21 07:39 /home/nixbld/.julia/compiled/v1.3/Compat/GSFWK_Pua4I.ji #+end_src I could not get the same results twice (also, size differs). I'll work on this on some spare time (for example, there are other places where SOURCE_DATE_EPOCH can be used, but this [3] is a problem I need to solve first). Maybe 1.3.1 (when reviewed) can be merged, since we have the same problem with julia 1.1, but we can wait for the source-date-epoch and julia-xyz patches until we solve this. What do you think? Thanks, Nicolò [1] https://docs.julialang.org/en/v1/devdocs/sysimg/ [2] https://github.com/JuliaLang/julia/issues/25900 [3] https://github.com/JuliaLang/julia/issues/34115 zimoun writes: > Hi Nicolò, > > Thank you for working on this. > Julia rocks! ;-) > > > However, I am lost with your patch set. > > I have tried only the first patch you attached. I hit this error: > > --8<---------------cut here---------------start------------->8--- > error: dsfmt: unbound variable > --8<---------------cut here---------------end--------------->8--- > > So something is missing. :-) > > Then I notice that you change the list of 'patches' and you "forgot" > to send them too. :-) > > > Almost related, does this update fix the reproducibility bug [1]? For example, > > --8<---------------cut here---------------start------------->8--- > ./pre-inst-env guix build julia > ./pre-inst-env guix build julia --check --no-grafts -K > --8<---------------cut here---------------end--------------->8--- > > > [1] https://debbugs.gnu.org/cgi/bugreport.cgi?bug=22304 > > > All the best, > simon From f33153f5f185562d01cfbfbfed43732483132f84 Mon Sep 17 00:00:00 2001 From: nixo Date: Sat, 18 Jan 2020 13:58:37 +0100 Subject: [PATCH] gnu: julia: use SOURCE_DATE_EPOCH for precompilation timestamp * gnu/packages/patches/julia-fake-mtime-with-SOURCE_DATE_EPOCH.patch: new file --- gnu/local.mk | 1 + gnu/packages/julia.scm | 4 ++- .../julia-SOURCE_DATE_EPOCH-mtime.patch | 29 +++++++++++++++++++ 3 files changed, 33 insertions(+), 1 deletion(-) create mode 100644 gnu/packages/patches/julia-SOURCE_DATE_EPOCH-mtime.patch diff --git a/gnu/local.mk b/gnu/local.mk index 00ff3b8d61..fe8634b44d 100644 --- a/gnu/local.mk +++ b/gnu/local.mk @@ -1010,6 +1010,7 @@ dist_patch_DATA = \ %D%/packages/patches/java-xerces-bootclasspath.patch \ %D%/packages/patches/java-xerces-build_dont_unzip.patch \ %D%/packages/patches/java-xerces-xjavac_taskdef.patch \ + %D%/packages/patches/julia-SOURCE_DATE_EPOCH-mtime.patch \ %D%/packages/patches/jbig2dec-ignore-testtest.patch \ %D%/packages/patches/kdbusaddons-kinit-file-name.patch \ %D%/packages/patches/libvirt-create-machine-cgroup.patch \ diff --git a/gnu/packages/julia.scm b/gnu/packages/julia.scm index 3455dc6239..794980c12f 100644 --- a/gnu/packages/julia.scm +++ b/gnu/packages/julia.scm @@ -191,7 +191,9 @@ version "/julia-" version ".tar.gz")) (sha256 (base32 - "1nwkmr9j55g1zkxdchnid1h022s0is52vx23niksshgvh793g41x")))) + "1nwkmr9j55g1zkxdchnid1h022s0is52vx23niksshgvh793g41x")) + (patches + (search-patches "julia-SOURCE_DATE_EPOCH-mtime.patch")))) (build-system gnu-build-system) (arguments `(#:test-target "test" diff --git a/gnu/packages/patches/julia-SOURCE_DATE_EPOCH-mtime.patch b/gnu/packages/patches/julia-SOURCE_DATE_EPOCH-mtime.patch new file mode 100644 index 0000000000..467e6d68d1 --- /dev/null +++ b/gnu/packages/patches/julia-SOURCE_DATE_EPOCH-mtime.patch @@ -0,0 +1,29 @@ +From e4dc28db1d70819505fd1a68fd9d2bfc4fb61a7d Mon Sep 17 00:00:00 2001 +From: nixo +Date: Fri, 17 Jan 2020 11:28:30 +0100 +Subject: [PATCH] base: loading: support fake mtime with SOURCE_DATE_EPCOCH + +Do this when loading libraries +--- + base/loading.jl | 5 ++++- + 1 file changed, 4 insertions(+), 1 deletion(-) + +diff --git a/base/loading.jl b/base/loading.jl +index 7f11a2d4fc..1b4686d1dc 100644 +--- a/base/loading.jl ++++ b/base/loading.jl +@@ -807,7 +807,10 @@ function _include_dependency(mod::Module, _path::AbstractString) + path = normpath(joinpath(dirname(prev), _path)) + end + if _track_dependencies[] +- push!(_require_dependencies, (mod, path, mtime(path))) ++ push!(_require_dependencies, ++ (mod, path, ++ haskey(ENV, "SOURCE_DATE_EPOCH") ? ++ parse(Float64, ENV["SOURCE_DATE_EPOCH"]) : mtime(path))) + end + return path, prev + end +-- +2.24.1 + -- 2.25.0