From patchwork Thu Oct 21 13:09:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jean-Baptiste Volatier X-Patchwork-Id: 34000 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 5E77127BBE3; Thu, 21 Oct 2021 14:14:16 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,T_DKIM_INVALID, URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 4CC7B27BBE1 for ; Thu, 21 Oct 2021 14:14:15 +0100 (BST) Received: from localhost ([::1]:55830 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mdXte-0000KS-Dw for patchwork@mira.cbaines.net; Thu, 21 Oct 2021 09:14:14 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39640) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mdXqZ-000580-Mt for guix-patches@gnu.org; Thu, 21 Oct 2021 09:11:03 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:44963) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mdXqZ-0007s2-C9 for guix-patches@gnu.org; Thu, 21 Oct 2021 09:11:03 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1mdXqY-0008Oa-Vq for guix-patches@gnu.org; Thu, 21 Oct 2021 09:11:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#51319] [PATCH 1/2] build-system/julia: Enable Julia Pkg to find installed packages Resent-From: Jean-Baptiste Volatier Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Thu, 21 Oct 2021 13:11:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 51319 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 51319@debbugs.gnu.org Cc: Jean-Baptiste Volatier Received: via spool by 51319-submit@debbugs.gnu.org id=B51319.163482182032193 (code B ref 51319); Thu, 21 Oct 2021 13:11:02 +0000 Received: (at 51319) by debbugs.gnu.org; 21 Oct 2021 13:10:20 +0000 Received: from localhost ([127.0.0.1]:56507 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mdXpr-0008NB-Qv for submit@debbugs.gnu.org; Thu, 21 Oct 2021 09:10:20 -0400 Received: from mail-40133.protonmail.ch ([185.70.40.133]:18355) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mdXpn-0008MV-3U for 51319@debbugs.gnu.org; Thu, 21 Oct 2021 09:10:16 -0400 Date: Thu, 21 Oct 2021 13:09:58 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pm.me; s=protonmail; t=1634821804; bh=cEdR2dCvWdAvcCyYVXz5VXAYrVBsAQxdqJl8j64rjac=; h=Date:To:From:Cc:Reply-To:Subject:In-Reply-To:References:From; b=pbz02UYIMykj7eEOIQzUbz+DbjbnkIAGXmXD3yNq3DoFxmPFw4OJ0ju4RWWenUxtL eNOz+UdADI+2imGB3jMG6Kwjo3RARkiibRjBck0/924MN/uK90S1WlFOzkp69i/+Ga ztfxbef9t8TsjFKiTfv4ybv6CO0SAiOoCMzlCNvW0o1EAIPT+7D42wTICmnnCbm5LV l8gB117HAyhxZnjLXQFdfMpevO7CLBppgwLM7oXcmDcb2/EdSIxFwz/Dwy7MHVEXT7 BVhqeqvXQUHj34JHle/DQukzO8FChI2YMF7vMp24mOasoLeE8YSP6SnHk1GGkNlRKZ QCfdsDGaDjLgQ== Message-ID: <20211021130952.14696-2-jbv@pm.me> In-Reply-To: <20211021130952.14696-1-jbv@pm.me> References: <20211021130952.14696-1-jbv@pm.me> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" Reply-to: Jean-Baptiste Volatier X-ACL-Warn: , Jean-Baptiste Volatier via Guix-patches X-Patchwork-Original-From: Jean-Baptiste Volatier via Guix-patches via From: Jean-Baptiste Volatier X-getmail-retrieved-from-mailbox: Patches * guix/build/julia-build-system.scm (link-depot): New phase. Julia built-in package manager (Pkg) looks for packages in JULIA_DEPOT_PATH/packages/PACKAGENAME/XXXX where XXXX is a string encoding package UUID and SHA1 of files. The link-depot phase creates a link at the correct location to allow Pkg to find packages that were already installed by Guix. (%package-path): modified package path from packages/ to loadpath/ (project.toml->uuid): retrive package uuid from TOML file * gnu/packages/julia-xyz.scm: correct paths to reflect change of %package-path * gnu/packages/julia.scm: correct paths to reflect change of %package-path --- gnu/packages/julia-xyz.scm | 2 +- gnu/packages/julia.scm | 2 +- guix/build/julia-build-system.scm | 46 ++++++++++++++++++++++++++++--- 3 files changed, 44 insertions(+), 6 deletions(-) -- 2.33.1 diff --git a/gnu/packages/julia-xyz.scm b/gnu/packages/julia-xyz.scm index c1ebc0fba1..1e407a2277 100644 --- a/gnu/packages/julia-xyz.scm +++ b/gnu/packages/julia-xyz.scm @@ -4610,7 +4610,7 @@ (define-public julia-uris (lambda* (#:key source outputs #:allow-other-keys) (let ((out (assoc-ref outputs "out"))) (chdir - (string-append out "/share/julia/packages/URIs/test"))) + (string-append out "/share/julia/loadpath/URIs/test"))) #t))))) ;; required for tests (inputs `(("julia-json" ,julia-json))) diff --git a/gnu/packages/julia.scm b/gnu/packages/julia.scm index 411f2e2e10..5b9bf91ed6 100644 --- a/gnu/packages/julia.scm +++ b/gnu/packages/julia.scm @@ -673,7 +673,7 @@ (define-public julia (native-search-paths (list (search-path-specification (variable "JULIA_LOAD_PATH") - (files (list "share/julia/packages/"))) + (files (list "share/julia/loadpath/"))) (search-path-specification (variable "JULIA_DEPOT_PATH") (files (list "share/julia/"))))) diff --git a/guix/build/julia-build-system.scm b/guix/build/julia-build-system.scm index d74acf2a05..c7f2119974 100644 --- a/guix/build/julia-build-system.scm +++ b/guix/build/julia-build-system.scm @@ -1,5 +1,6 @@ ;;; GNU Guix --- Functional package management for GNU ;;; Copyright © 2019, 2020 Nicolò Balzarotti +;;; Copyright © 2021 Jean-Baptiste Volatier ;;; ;;; This file is part of GNU Guix. ;;; @@ -20,10 +21,13 @@ (define-module (guix build julia-build-system) #:use-module ((guix build gnu-build-system) #:prefix gnu:) #:use-module (guix build utils) + #:use-module (rnrs io ports) #:use-module (ice-9 match) #:use-module (ice-9 regex) #:use-module (ice-9 rdelim) + #:use-module (ice-9 popen) #:export (%standard-phases + %package-path julia-create-package-toml julia-build)) @@ -37,7 +41,7 @@ (define (invoke-julia code) (invoke "julia" "-e" code)) ;; subpath where we store the package content -(define %package-path "/share/julia/packages/") +(define %package-path "/share/julia/loadpath/") (define (project.toml->name file) "Look for Julia package name in the TOML file FILE (usually named @@ -51,6 +55,18 @@ (define (project.toml->name file) (if m (match:substring m 1) (loop (read-line in 'concat))))))))) +(define (project.toml->uuid file) + "Look for Julia package uuid in the TOML file FILE (usually named +Project.toml)." + (call-with-input-file file + (lambda (in) + (let loop ((line (read-line in 'concat))) + (if (eof-object? line) + #f + (let ((m (string-match "uuid\\s*=\\s*\"(.*)\"" line))) + (if m (match:substring m 1) + (loop (read-line in 'concat))))))))) + (define* (install #:key source inputs outputs julia-package-name #:allow-other-keys) (let* ((out (assoc-ref outputs "out")) @@ -73,7 +89,7 @@ (define* (precompile #:key source inputs outputs julia-package-name (setenv "JULIA_DEPOT_PATH" builddir) ;; Add new package dir to the load path. (setenv "JULIA_LOAD_PATH" - (string-append builddir "packages/" ":" + (string-append builddir "loadpath/" ":" (or (getenv "JULIA_LOAD_PATH") ""))) ;; Actual precompilation: @@ -97,15 +113,36 @@ (define* (check #:key tests? source inputs outputs julia-package-name (setenv "SOURCE_DATE_EPOCH" "1") (setenv "JULIA_DEPOT_PATH" builddir) (setenv "JULIA_LOAD_PATH" - (string-append builddir "packages/" ":" + (string-append builddir "loadpath/" ":" (or (getenv "JULIA_LOAD_PATH") ""))) (setenv "HOME" "/tmp") (invoke "julia" "--depwarn=yes" - (string-append builddir "packages/" + (string-append builddir "loadpath/" package "/test/runtests.jl")))) #t) +(define* (link-depot #:key source inputs outputs julia-package-name + #:allow-other-keys) + (let* ((out (assoc-ref outputs "out")) + (package-name (or + julia-package-name + (project.toml->name "Project.toml"))) + (package-dir (string-append out %package-path package-name)) + (uuid (project.toml->uuid "Project.toml")) + (pipe (open-pipe* OPEN_READ "julia" "-e" + (format #f "using Pkg; +println(Base.version_slug(Base.UUID(\"~a\"), + Base.SHA1(Pkg.GitTools.tree_hash(\".\"))))" uuid package-dir))) + (slug (string-trim-right (get-string-all pipe)))) + ;; When installing a package, julia looks first at in the JULIA_DEPOT_PATH + ;; for a path like packages/PACKAGE/XXXX + ;; Where XXXX is a slug encoding the package UUID and SHA1 of the files + ;; Here we create a link with the correct path to enable julia to find the package + (mkdir-p (string-append out "/share/julia/packages/" package-name)) + (symlink package-dir (string-append out "/share/julia/packages/" package-name "/" slug))) + #t) + (define (julia-create-package-toml outputs source name uuid version deps) @@ -138,6 +175,7 @@ (define %standard-phases (delete 'check) ; tests must be run after installation (replace 'install install) (add-after 'install 'precompile precompile) + (add-after 'unpack 'link-depot link-depot) (add-after 'install 'check check) ;; TODO: In the future we could add a "system-image-generation" phase ;; where we use PackageCompiler.jl to speed up package loading times From patchwork Thu Oct 21 13:10:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Baptiste Volatier X-Patchwork-Id: 33999 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id C6E6027BBE3; Thu, 21 Oct 2021 14:11:18 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,T_DKIM_INVALID, URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id A2E0A27BBE1 for ; Thu, 21 Oct 2021 14:11:17 +0100 (BST) Received: from localhost ([::1]:50576 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mdXqm-0005Ar-Il for patchwork@mira.cbaines.net; Thu, 21 Oct 2021 09:11:16 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39632) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mdXqZ-00056X-2O for guix-patches@gnu.org; Thu, 21 Oct 2021 09:11:03 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:44962) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mdXqY-0007rT-N2 for guix-patches@gnu.org; Thu, 21 Oct 2021 09:11:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1mdXqY-0008OT-Hu for guix-patches@gnu.org; Thu, 21 Oct 2021 09:11:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#51319] [PATCH 2/2] gnu: Add julia-sundials-jull Resent-From: Jean-Baptiste Volatier Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Thu, 21 Oct 2021 13:11:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 51319 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 51319@debbugs.gnu.org Cc: Jean-Baptiste Volatier Received: via spool by 51319-submit@debbugs.gnu.org id=B51319.163482181632183 (code B ref 51319); Thu, 21 Oct 2021 13:11:02 +0000 Received: (at 51319) by debbugs.gnu.org; 21 Oct 2021 13:10:16 +0000 Received: from localhost ([127.0.0.1]:56505 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mdXpo-0008N1-HC for submit@debbugs.gnu.org; Thu, 21 Oct 2021 09:10:16 -0400 Received: from mail-4316.protonmail.ch ([185.70.43.16]:32801) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mdXpm-0008MM-Hx for 51319@debbugs.gnu.org; Thu, 21 Oct 2021 09:10:15 -0400 Date: Thu, 21 Oct 2021 13:10:06 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pm.me; s=protonmail; t=1634821808; bh=xVUwNeSlSC1rt0/YxNCICUoTlJ7mwUEqqw//JzAtab8=; h=Date:To:From:Cc:Reply-To:Subject:In-Reply-To:References:From; b=mD0odhERT5+nI3Kk41NTb3nEhQMjz5rtGhRiittUmb+Y7XaUwO3XNnRLg2rKhsN1F /CFuU1BEbgXwNimf3YKnBTFnSYM41DRBSVc6BKZYY16YUHL8t2rzncwNeL72eNpiVI tx0oInOeZH0AjTR0Nprqk//68OPHruADV6p75nGJi4l6PgZg3qEZ/MdixPiz5hk6U5 fioSOAiFNCx+FmwsioVVZpPu4ybsvIWK5dcPCVJjbL2UNFa1CKVJxcryccFUlcPjjS SMm+TqxmkdsSnbld3DvRW/TU95XDytQmmsUa7+Jwey1+B9dte+LJdDmrsVZWLPgom7 yV6W4dyPtbHuQ== Message-ID: <20211021130952.14696-3-jbv@pm.me> In-Reply-To: <20211021130952.14696-1-jbv@pm.me> References: <20211021130952.14696-1-jbv@pm.me> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" Reply-to: Jean-Baptiste Volatier X-ACL-Warn: , Jean-Baptiste Volatier via Guix-patches X-Patchwork-Original-From: Jean-Baptiste Volatier via Guix-patches via From: Jean-Baptiste Volatier X-getmail-retrieved-from-mailbox: Patches * gnu/packages/julia-jll.scm (julia-sundials-jll): New variable. * gnu/packages/maths.scm (sundials-julia): New variable. --- gnu/packages/julia-jll.scm | 38 ++++++++++++++++++++++++++++++++++++++ gnu/packages/maths.scm | 24 ++++++++++++++++++++++++ 2 files changed, 62 insertions(+) -- 2.33.1 diff --git a/gnu/packages/julia-jll.scm b/gnu/packages/julia-jll.scm index d3677d0a33..e1c33903dd 100644 --- a/gnu/packages/julia-jll.scm +++ b/gnu/packages/julia-jll.scm @@ -2709,3 +2709,41 @@ (define-public julia-zstd-jll (synopsis "Zstd library wrappers") (description "This package provides a wrapper for the zstd library.") (license license:expat))) + +(define-public julia-sundials-jll + (package + (name "julia-sundials-jll") + (version "5.2.1+0") + (source + (origin + (method git-fetch) + (uri (git-reference + (url "https://github.com/JuliaBinaryWrappers/Sundials_jll.jl") + (commit (string-append "Sundials-v" version)))) + (file-name (git-file-name name version)) + (sha256 + (base32 "0cijb9frq8gj8bjpqf2lr5d0jxlj262y6h6xi4z3536dingrvffc")))) + (build-system julia-build-system) + (arguments + '(#:tests? #f ; no runtests + #:phases + (modify-phases %standard-phases + (add-after 'link-depot 'override-binary-path + (lambda* (#:key inputs #:allow-other-keys) + (map + (lambda (wrapper) + (substitute* wrapper + (("global artifact_dir.*") + (string-append + "global artifact_dir = \"" + (assoc-ref inputs "sundials") "\"\n")))) + ;; There's a Julia file for each platform, override them all + (find-files "src/wrappers/" "\\.jl$"))))))) + (inputs + `(("sundials" ,sundials-julia))) + (propagated-inputs + `(("julia-jllwrappers" ,julia-jllwrappers))) + (home-page "https://github.com/JuliaBinaryWrappers/Sundials_jll.jl") + (synopsis "SUndials library wrappers") + (description "This package provides a wrapper for the sundials library.") + (license license:expat))) diff --git a/gnu/packages/maths.scm b/gnu/packages/maths.scm index 038ca5518c..54f44a3b55 100644 --- a/gnu/packages/maths.scm +++ b/gnu/packages/maths.scm @@ -5943,6 +5943,30 @@ (define-public sundials-openmpi ,%openmpi-setup))))) (synopsis "SUNDIALS with OpenMPI support"))) +(define-public sundials-julia + (package + (inherit sundials) + (name "sundials-julia") + (version "5.2.0") + (source + (origin + (method git-fetch) + (uri (git-reference + (url "https://github.com/LLNL/sundials.git") + (commit (string-append "v" version)))) + (sha256 + (base32 + "08q9svjdl6fmqrwyd9p12m98n0wd2lws38yh223wyp6590zi3i49")))) + (inputs + `(("lapack" ,lapack) + ,@(package-inputs sundials))) + (arguments + (substitute-keyword-arguments (package-arguments sundials) + ((#:configure-flags flags '()) + `(cons* "-DLAPACK_ENABLE:BOOL=ON" + ,flags)))) + (synopsis "SUNDIALS with lapack support as required by julia-sundials-jll"))) + (define-public combinatorial-blas (package (name "combinatorial-blas")