From patchwork Mon Jan 17 21:12:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Liliana Marie Prikler X-Patchwork-Id: 36576 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id E55C227BBEA; Mon, 17 Jan 2022 21:16:18 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FROM,MAILING_LIST_MULTI, RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 65B5227BBE9 for ; Mon, 17 Jan 2022 21:16:17 +0000 (GMT) Received: from localhost ([::1]:41794 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1n9ZMO-0006rP-Gt for patchwork@mira.cbaines.net; Mon, 17 Jan 2022 16:16:16 -0500 Received: from eggs.gnu.org ([209.51.188.92]:37070) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1n9ZKE-0005QS-DW for guix-patches@gnu.org; Mon, 17 Jan 2022 16:14:04 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:55383) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1n9ZKE-0003pn-35 for guix-patches@gnu.org; Mon, 17 Jan 2022 16:14:02 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1n9ZKD-0004Zq-Vr for guix-patches@gnu.org; Mon, 17 Jan 2022 16:14:01 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#48463] gnu: Add j. Resent-From: Liliana Marie Prikler Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Mon, 17 Jan 2022 21:14:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 48463 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: elaexuotee@wilsonb.com Cc: Maxime Devos , 48463@debbugs.gnu.org Received: via spool by 48463-submit@debbugs.gnu.org id=B48463.164245399517532 (code B ref 48463); Mon, 17 Jan 2022 21:14:01 +0000 Received: (at 48463) by debbugs.gnu.org; 17 Jan 2022 21:13:15 +0000 Received: from localhost ([127.0.0.1]:48286 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1n9ZJH-0004YT-Ju for submit@debbugs.gnu.org; Mon, 17 Jan 2022 16:13:15 -0500 Received: from mail-wm1-f66.google.com ([209.85.128.66]:41881) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1n9ZJF-0004Xx-49 for 48463@debbugs.gnu.org; Mon, 17 Jan 2022 16:13:02 -0500 Received: by mail-wm1-f66.google.com with SMTP id q141-20020a1ca793000000b00347b48dfb53so2444863wme.0 for <48463@debbugs.gnu.org>; Mon, 17 Jan 2022 13:13:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:subject:from:to:cc:date:in-reply-to:references :user-agent:mime-version; bh=VzGdXAIs28AZDVu8HKTY1nj6yUIe7RnLRGGTnW9OrE0=; b=INL6VsVMuzCJAxgQ99X6kJ5VoFG9Zq2DIwc66eaM5droezmV4rMpDJjfb14nJVZwpw ADdIXVQQqGl4fozRSHVG5G25/c7tVeAAyFUdi84QgmUH3h8DQtzngmi0f7tzlIyvbaE6 PA5tmXrfvkNNRL0PiO0XxszJMJ+1S9koQUuPFDvv4vgI+c6xILkXtAZThtHfd/qkno2H qUOY0wJxjgh6STeILauk80q4DL36B9/t5SNRF7OxOpwwMx1blzj3oF+57fLayNQ5v8L/ /6w4iJPLd61gysWaovX3Iy7lIgAyh1gu4eHtSF0S7X2ZaGszq0Kl4eDg+f9ilaztcp88 fOWg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:subject:from:to:cc:date:in-reply-to :references:user-agent:mime-version; bh=VzGdXAIs28AZDVu8HKTY1nj6yUIe7RnLRGGTnW9OrE0=; b=YJlxDwlOJ9/nuI3oL9rVps8NVuCsriXgJVBK6zjxoSAZPpAxTpkhMyXx5kiQ7CzrsI G/AetW2kZLMozXDbSie354WjtCiUF/PXQtKRgt5UlgD/j4Hqx5I0dfD6vohWFQoehrbB b9jEQdV1ZmglcMGg4C0wL5LXy3HH+anYfOyuOSy7+2XCdd66m4yPMs0NQfYvd5P0pFHb X0sPDV/KYO54ah1ba6Zoq8AUHfRLwPMr8p7AlgYq0lZoMt2XV1ORUodfCweQ2vPIjDYU 1CXQOgSM/s01LirfoAT64/Ds5NQMGlploxkiW4H4lGoZwYCh1IrZpZCCpIyCiSygJNn1 5W9w== X-Gm-Message-State: AOAM533hoKna0GPgR5WP5Ox4JIET432y9jbhMxS0/ZgP57kg1EuWrYzW 4GPt3KoAneaoVwErW49nyCk= X-Google-Smtp-Source: ABdhPJzEpE4x7EeP6Cyu5jQohhMFCKXHB/aTsb941ggA/acqd449ziguUaZkaLdBFELI+IwThV16YQ== X-Received: by 2002:a5d:64c4:: with SMTP id f4mr6146978wri.429.1642453975455; Mon, 17 Jan 2022 13:12:55 -0800 (PST) Received: from nijino.fritz.box (85-127-52-93.dsl.dynamic.surfer.at. [85.127.52.93]) by smtp.gmail.com with ESMTPSA id l8sm459951wme.5.2022.01.17.13.12.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 17 Jan 2022 13:12:53 -0800 (PST) Message-ID: From: Liliana Marie Prikler Date: Mon, 17 Jan 2022 22:12:52 +0100 In-Reply-To: <2EZU214MJAIBY.3EXSPSUMS5WW5@wilsonb.com> References: <3LOAUDT0FLL4U.2SOD925YP915T@wilsonb.com> <8b853d0585505ce29c9afc638b644fa34805e6c0.camel@student.tugraz.at> <293L8YPQS4CLB.3VK1B1A36XNAY@wilsonb.com> <5d30160bd2a4592459cd407f99cbd3edadb1db1b.camel@student.tugraz.at> <27DCD25Y68ZWJ.2HRC4G65PWIA7@wilsonb.com> <94f4625dcb0479d873cf60449631527e841fd457.camel@gmail.com> <2JQJMV0O718S1.31FZE8GKCTLPF@wilsonb.com> <90704c2259f576a14fb1268219e8c0dc2b3bf289.camel@telenet.be> <2P322C327XW0Q.21O5A4IFGMNDI@wilsonb.com> <72aff035c93f9f91afa54ef5b51c7381b0b02ccb.camel@gmail.com> <3MMTDZQJQ8IR6.334ZWY8AD0487@wilsonb.com> <62d37956f16c08bc4ce26e44da16dce704ddd0f8.camel@gmail.com> <24ZUUMG4QYSHN.2OS7YAMCKREUA@wilsonb.com> <83aba994536bec60f79900d551d4801c967742bd.camel@gmail.com> <25Z6NGGGNJYD1.3S7A1QLFX7I9Y@wilsonb.com> <24FU0VP6N4ZZ7.3PE5LG30BSNUQ@wilsonb.com> <2EZU214MJAIBY.3EXSPSUMS5WW5@wilsonb.com> User-Agent: Evolution 3.42.1 MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches Hi, I tried to get J ready for packaging, but thanks to my CPU not supporting AVX2 and some helpful discussion in IRC, I was able to avoid an error here. The (guix cpu) code is not meant to be used at runtime as the comment to ijconsole implies it wants it to be used. Rather, you should use it at compile time to bake in the correct tuning parameters, and it should ideally also support "cross-tuning" (though in that case you probably need to disable the tests because your CPU won't make sense of the code). ijconsole not only fails that, but it also fails at an even more basic task in Guix' launcher scripts, which is actually knowing the thing you launch. (We hardcode everything, period.) Now I could just disable everything AVX-related in J and push something that works on x86 and amd64, but since you do claim that AVX is important to J, there are also other options one could take here. One is to implement tuning for this package the correct way, though since it reinvents build systems, that's probably going to be a hard one. The other would be to define package variants (e.g. jsoftware-j-with- avx) and use those -- there ought to be an old blog post on Guix HPC detailing the rationale behind doing that. WDYT? Is AVX worth the trouble? From 6b15deda2ecc09b73545e3feccfac7f3f695c9e8 Mon Sep 17 00:00:00 2001 From: "B. Wilson" Date: Wed, 12 Jan 2022 18:44:36 +0900 Subject: [PATCH] gnu: Add j. * gnu/packages/jsoftware.scm: New file. * gnu/packages/patches/jsoftware-j901-f-fixes.patch: New file. * gnu/local.mk [GNU_SYSTEM_MODULES]: Add jsoftware.scm. [dist_patch_DATA]: Add jsoftware-j901-f-fixes.patch. * gnu/packages/aux-files/jsoftware/profilex.ijs: New file. *Makefile.am [AUX_FILES]: Add it here. --- Makefile.am | 1 + gnu/local.mk | 2 + gnu/packages/aux-files/jsoftware/profilex.ijs | 14 + gnu/packages/jsoftware.scm | 421 ++++++++++++++++++ .../patches/jsoftware-j901-f-fixes.patch | 80 ++++ 5 files changed, 518 insertions(+) create mode 100644 gnu/packages/aux-files/jsoftware/profilex.ijs create mode 100644 gnu/packages/jsoftware.scm create mode 100644 gnu/packages/patches/jsoftware-j901-f-fixes.patch diff --git a/Makefile.am b/Makefile.am index a10aeb817b..1efd8a9b26 100644 --- a/Makefile.am +++ b/Makefile.am @@ -379,6 +379,7 @@ AUX_FILES = \ gnu/packages/aux-files/chromium/master-preferences.json \ gnu/packages/aux-files/emacs/guix-emacs.el \ gnu/packages/aux-files/guix.vim \ + gnu/packages/aux-files/jsoftware/profilex.ijs \ gnu/packages/aux-files/linux-libre/5.15-arm.conf \ gnu/packages/aux-files/linux-libre/5.15-arm64.conf \ gnu/packages/aux-files/linux-libre/5.15-i686.conf \ diff --git a/gnu/local.mk b/gnu/local.mk index 7e044d4a2b..eb1309c9d7 100644 --- a/gnu/local.mk +++ b/gnu/local.mk @@ -325,6 +325,7 @@ GNU_SYSTEM_MODULES = \ %D%/packages/jemalloc.scm \ %D%/packages/jrnl.scm \ %D%/packages/jose.scm \ + %D%/packages/jsoftware.scm \ %D%/packages/julia.scm \ %D%/packages/julia-jll.scm \ %D%/packages/julia-xyz.scm \ @@ -1295,6 +1296,7 @@ dist_patch_DATA = \ %D%/packages/patches/irrlicht-use-system-libs.patch \ %D%/packages/patches/isc-dhcp-gcc-compat.patch \ %D%/packages/patches/isl-0.11.1-aarch64-support.patch \ + %D%/packages/patches/jsoftware-j901-f-fixes.patch \ %D%/packages/patches/json-c-0.13-CVE-2020-12762.patch \ %D%/packages/patches/json-c-0.12-CVE-2020-12762.patch \ %D%/packages/patches/jsoncpp-pkg-config-version.patch \ diff --git a/gnu/packages/aux-files/jsoftware/profilex.ijs b/gnu/packages/aux-files/jsoftware/profilex.ijs new file mode 100644 index 0000000000..30e0d229e2 --- /dev/null +++ b/gnu/packages/aux-files/jsoftware/profilex.ijs @@ -0,0 +1,14 @@ +'jtype jversion'=. (3&{,{.) <;._2 ,&'/' 9!:14'' +basedir=. ({.~ _2 { I.@:=&'/') BINPATH + +share=. basedir,'/share/j' +system=. share,'/system' +tools=. share,'/tools' + +user=. home,'/.config/j/',jversion +addons=. user,'/addons' +break=. user,'/break' +config=. user,'/config' +install=. user,'/install' +snap=. user,'/snap' +temp=. user,'/temp' diff --git a/gnu/packages/jsoftware.scm b/gnu/packages/jsoftware.scm new file mode 100644 index 0000000000..c7d5c4d7b5 --- /dev/null +++ b/gnu/packages/jsoftware.scm @@ -0,0 +1,421 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright © 2022 B. Wilson +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see . + +(define-module (gnu packages jsoftware) + #:use-module (guix build utils) + #:use-module (guix build-system gnu) + #:use-module (guix build-system trivial) + #:use-module (guix git-download) + #:use-module ((guix licenses) #:prefix license:) + #:use-module (guix packages) + #:use-module (guix utils) + #:use-module (gnu packages) + #:use-module (gnu packages libedit) + #:use-module (gnu packages llvm) + #:use-module (gnu packages maths) + #:use-module (guix gexp) + #:use-module (ice-9 ftw) + #:use-module (ice-9 match) + #:use-module (ice-9 regex) + #:use-module (ice-9 rdelim) + #:use-module (srfi srfi-1) + #:use-module (srfi srfi-26) + #:use-module (srfi srfi-71)) + + +;;; TODO: Make importer and packages for J addons: +;;; http://www.jsoftware.com/jal/ + +;;; TODO: Package up j80x series + + +(define (jname prefix release-type) + "Return a package name for J, including RELEASE-TYPE only if not 'release." + (match release-type + ('release prefix) + (_ (string-append prefix "-" (symbol->string release-type))))) + +(define (version-major+minor* version) + "Like version-major+minor, but returning two string values." + (let ((parts (string-split version #\.))) + (values (first parts) (second parts)))) + +(define* (jrelease-string release-type #:optional version-minor) + "Construct J release identifier string." + (let ((release-type (symbol->string release-type))) + (if version-minor + (string-append release-type "-" version-minor) + release-type))) + +(define* (jinfo->git-tag version release-type) + "Given version parameters, construct a git tag for upstream releases." + (let ((major minor (version-major+minor* version))) + (string-append "j" major "-" (jrelease-string release-type minor)))) + +;; G-exp script that detects AVX/AVX2 support at runtime and executes jconsole +;; with the appropriate libj.so and profile.ijs." +;; NOTE: This should be baked in at compile time into `jsoftware-j'. +(define ijconsole + (with-imported-modules '((guix cpu) + (guix memoization) + (guix profiling) + (guix sets) + (srfi srfi-26)) + (program-file "ijconsole" + #~(begin + (use-modules ((guix cpu) #:select (cpu-flags current-cpu)) + ((guix sets) #:select (set-contains?)) + ((srfi srfi-26) #:select (cute))) + + ;; Assume that this script will be installed under bin/. + (define %basedir (dirname (dirname (current-filename)))) + + (let* ((jconsole (string-append %basedir "/libexec/j/jconsole")) + (cpu-has-flag? + (cute set-contains? (cpu-flags (current-cpu)) <>)) + (libj (format #f "~a/lib/j/libj~a.so" %basedir + ""#; + (cond ((cpu-has-flag? "avx2") "-avx2") + ((cpu-has-flag? "avx") "-avx") + (else "")))) + (jprofile (string-append %basedir "/etc/j/profile.ijs"))) + (apply execl jconsole "ijconsole" "-lib" libj "-jprofile" jprofile + (cdr (command-line)))))))) + +(define* (make-j base-version hash + #:key + revision + commit + tag + (release-type 'release) + (patches '()) + (modules '()) + (snippet #f) + (extra-inputs '()) + (extra-envars '()) + (builder "guix.gnu.org")) + (let ((major minor (version-major+minor* base-version))) + (package + (name (jname "jsoftware-j" release-type)) + (version + (if commit (git-version base-version revision commit) base-version)) + (source + (origin + (method git-fetch) + (uri (git-reference + (url "https://github.com/jsoftware/jsource") + (commit (or commit tag + (jinfo->git-tag base-version release-type))))) + (sha256 (base32 hash)) + (file-name (git-file-name name version)) + (patches patches) + (modules modules) + (snippet snippet))) + (build-system gnu-build-system) + (native-inputs (list clang-toolchain)) + (inputs + ;; ijconsole and profile.ijs still require labels + `(("ijconsole" ,ijconsole) + ("profilex.ijs" ,(search-auxiliary-file "jsoftware/profilex.ijs")) + ("libedit" ,libedit) + ("libomp" ,libomp) + ,@extra-inputs)) + (arguments + `(#:modules (((ice-9 ftw) #:select (scandir)) + ((ice-9 popen) #:select (open-pipe* close-pipe)) + ((ice-9 regex) #:select (match:substring string-match)) + ((ice-9 threads) #:select (parallel par-for-each)) + ((srfi srfi-26) #:select (cut)) + ((srfi srfi-1) #:select (fold)) + ,@%gnu-build-system-modules) + #:phases + ;; Upstream's build system consists of ad-hoc scripts that build up + ;; (very complicated) environment variables to pass to make. + ;; The basic build process looks like this: + ;; + ;; 1) Copy jsrc/jversion-x.h to jsrc/jversion.h and edit values; + ;; 2) Set jplatform and j64x environment variables; + ;; 3) Run make2/build_jconsole.sh and make2/build_libj.sh; + ;; + ;; However, upstream expects users to run J directly from the source + ;; directory; they do not supply a make `install' target. Thus it + ;; takes some massaging to install files in FHS-style directories. + (modify-phases %standard-phases + ;; In particular, we have to set up + ;; + ;; 1) jsrc/jversion.h as in a typical build; + ;; 2) jlibrary/bin/profilex.ijs to point to writable directories; + ;; 3) make2/build_*.sh to respect standard build conventions; + ;; 4) jsrc/jconsole.c to fix libedit dlopen; and + ;; 5) Hard coded references to addons directory. + (replace 'configure + (lambda* (#:key target inputs outputs #:allow-other-keys) + (let* ((clang-toolchain (assoc-ref inputs "clang-toolchain")) + (clang (string-append clang-toolchain "/bin/clang")) + (libedit (assoc-ref inputs "libedit")) + (out (assoc-ref outputs "out"))) + ;; Set up build constants + (copy-file "jsrc/jversion-x.h" "jsrc/jversion.h") + (substitute* "jsrc/jversion.h" + (("^#define jversion.*$") + (format #f "#define jversion ~s\n" ,major)) + (("^#define jtype.*$") + (format #f "#define jtype ~s\n" + ,(jrelease-string release-type minor))) + (("^#define jbuilder.*$") + (format #f "#define jbuilder ~s\n" ,builder))) + ;; Munge the build scripts into reason: + ;; 1. Short-circuit the fragile compiler detection; + ;; 2. Make sure to include our CFLAGS and LFLAGS; and + ;; 3. Propagate script errors to top level. + (for-each + (lambda (file) + (with-directory-excursion "make2" + (substitute* file + ;; The `compiler' variable doesn't point to the actual + ;; compiler. It is just a switch to tell the build + ;; scripts whether to use gcc- or clang-specific flags. + (("^compiler=.*$") "compiler=clang\n") + (("^LDFLAGS=\"" def) (string-append def "$LDFLAGS ")) + (("^(common=\")(\\$USETHREAD.*)$" _ def rest) + (string-append def "$CFLAGS " rest)) + (("^#!.*" shebang) + (string-append shebang "set -o errexit\n"))))) + '("build_jconsole.sh" "build_libj.sh")) + ;; The jconsole manually loads libedit with dlopen. The path + ;; must be absolute to correctly point to our input. + (substitute* "jsrc/jconsole.c" + (("libedit\\.so\\.[0-9]" so-file) + (format #f "~a/lib/~a" libedit so-file))) + ;; The ~addons/dev directory supplies tentative J-script + ;; definitions of new J engine functionality. Since we point + ;; ~addons under the ~user directory, we move it under ~system + ;; instead, which sits as-is in the output. + (with-directory-excursion "jsrc" + (for-each + (lambda (file) + (substitute* file (("~addons/dev") "~system/dev"))) + (scandir "." + (lambda (f) (eq? (stat:type (stat f)) 'regular))))) + ;; Implementation of 9!:14 records build time which breaks + ;; reproducibility. Note that upstream code depends on the + ;; exact format of these strings, so we need to mimic the + ;; standard. + (substitute* "jsrc/j.c" + (("__DATE__") "\"Jan 01 1970\"") + (("__TIME__") "\"00:00:00\"")) + ;; Upstream recommends using clang, with GCC support being + ;; second-class, often resulting in build failures. + (setenv "CC" clang)))) + + ;; The build output depends primarily on the values of the + ;; `jplatform' and `j64x' environment variables. If the target is + ;; ARM, then `jplatform' is "raspberry", otherwise it is `linux'. + ;; In addition to 32- and 64- bit versions, `j64x' controlls + ;; whether AVX or AVX2 variants of libj are built. + ;; + ;; However, build targets are not fine-grained enough to distinguish + ;; between CPU features. Thus we build and install all variants of + ;; libj, expecting jconsole to be called with a wrapper script that + ;; detects AVX features and loads the appropriate libj at runtime. + (replace 'build + (lambda _ + (setenv "USE_OPENMP" "1") + (setenv "USE_THREAD" "1") + (for-each (lambda (var-val) (apply setenv var-val)) + (quote ,extra-envars)) + ;; The build scripts assume that PWD is make2. + (with-directory-excursion "make2" + (let* ((platform ,(if (target-arm?) "raspberry" "linux")) + (target-bit ,(if (target-64bit?) "64" "32")) + (run (lambda* (script #:key (avx "")) + (invoke "env" + (string-append "jplatform=" platform) + (string-append "j64x=j" target-bit avx) + script)))) + (parallel + ;; Since jconsole doesn't depend on AVX features, we just + ;; build it once. + (run "./build_jconsole.sh") + (run "./build_libj.sh") + (when #false ;; ,(target-64bit?) + (run "./build_libj.sh" #:avx "avx") + (run "./build_libj.sh" #:avx "avx2"))))))) + ;; The test suite is expected to be run as follows for each variant + ;; of libj that we build: + ;; + ;; $ echo 'RUN ddall' | jconsole test/tsu.ijs + ;; + ;; This requires a working jconsole with accessible jlibrary files. + ;; We simply place these all under test/bin. + (replace 'check + (lambda* (#:key tests? #:allow-other-keys) + (when tests? + (let ((platform ,(if (target-arm?) "raspberry" "linux"))) + (mkdir-p "test/bin") + (for-each + (lambda (dir) + (let ((source (string-append "jlibrary/" dir)) + (dest (string-append "test/bin/" dir))) + (begin + (mkdir-p dest) + (copy-recursively source dest)))) + '("system" "tools" "addons")) + ;; The jlibrary/dev directory only exists sometimes, but + ;; when it does, it needs to be in ~system. + (for-each + (lambda (dev-dir) + (if (file-exists? dev-dir) + (copy-recursively dev-dir "test/bin/system/dev"))) + '("jlibrary/dev" "jlibrary/addons/dev")) + (par-for-each + (lambda (dir) + (let* ((bin (string-append "bin/" platform)) + (jbit ,(if (target-64bit?) "j64" "j32")) + (jconsole (string-append bin "/" jbit + "/jconsole")) + (source (string-append bin "/" dir)) + (dest (string-append "test/bin/" dir))) + (begin + (mkdir-p dest) + (copy-recursively source dest) + (install-file "jlibrary/bin/profile.ijs" dest) + (install-file jconsole dest) + (let* ((jconsole (string-append dest "/jconsole")) + (tests "test/tsu.ijs") + (port (open-pipe* OPEN_WRITE jconsole tests))) + (display "RUN ddall\n" port) + (unless (zero? (status:exit-val (close-pipe port))) + (error "Some J build tests failed.")))))) + (scandir (string-append "bin/" platform) + (negate (cut member <> '("." ".."))))) + #t)))) + ;; Now that everything is built, installation is fairly + ;; straightforward, following FHS conventions. The only quirk is + ;; that we install jconsole under /libexec to make room for the + ;; wrapper replacement under /bin. + (replace 'install + (lambda* (#:key outputs inputs #:allow-other-keys) + (let* ((platform ,(if (target-arm?) "raspberry" "linux")) + (jbit ,(if (target-64bit?) "j64" "j32")) + (out (assoc-ref outputs "out")) + (bin (string-append out "/bin")) + (etc (string-append out "/etc/j")) + (lib (string-append out "/lib/j")) + (libexec (string-append out "/libexec/j")) + (share (string-append out "/share/j")) + (system (string-append share "/system")) + (dev (string-append system "/dev"))) + (mkdir-p bin) + (copy-file (assoc-ref inputs "ijconsole") + (string-append bin "/ijconsole-" ,major)) + (mkdir-p lib) + (for-each + (lambda (jarch) + (let* ((jbin (string-join `("bin" ,platform ,jarch) "/")) + (javx-match (string-match "avx.*" jarch)) + (javx (if (not javx-match) "" + (match:substring javx-match))) + (sep (if javx-match "-" "")) + (source (string-append jbin "/libj.so")) + (dest (format #f "~a/libj~a~a.so" lib sep javx))) + (copy-file source dest))) + (scandir (string-append "bin/" platform) + (negate (cut member <> '("." ".."))))) + (install-file + (string-append "bin/" platform "/" jbit "/jconsole") + libexec) + (copy-recursively "jlibrary/system" system) + (for-each + (lambda (source-dev) + (if (access? source-dev R_OK) + (copy-recursively source-dev dev))) + '("jlibrary/dev" "jlibrary/addons/dev")) + (install-file "jlibrary/bin/profile.ijs" etc) + (copy-file (assoc-ref inputs "profilex.ijs") + (string-append etc "/profilex.ijs")))))))) + (home-page "https://www.jsoftware.com/") + (synopsis "Ascii-only, array programming language in the APL family") + (description + "J is a high-level, general-purpose programming language that is +particularly suited to the mathematical, statistical, and logical analysis of +data. It is a powerful tool for developing algorithms and exploring problems +that are not already well understood.") + (license license:gpl3+)))) + + +(define-public jsoftware-j-901 + (make-j "901.f" + "1776021m0j1aanzwg60by83n53pw7i6afd5wplfzczwk8bywax4p" + #:patches (search-patches "jsoftware-j901-f-fixes.patch"))) + + +(define j-build-configuration-with-sleef + ;; XXX: label required because of ijconsole and profile.ijs in make-j. + ;; if labels are dropped in make-j, drop them here too. + ;; XXX: sleef is still being bundled in j and unbundling it causes build + ;; errors... investigate them and unbundle + `(#:extra-inputs (("sleef" ,sleef)) + #:extra-envars (("USE_SLEEF_SRC" "0") + ("LDFLAGS" "-lsleef")))) + +(define-public jsoftware-j-902 + (apply make-j "902.b" + "0j67vgikqflwjqacsdicasvyv1k54s2c8vjgwmf0ix7l41p4xqz0" + j-build-configuration-with-sleef)) + +(define-public jsoftware-j-903 + (apply make-j "903.a" + "1fcfl7q7c2vj4fmnqqc8c6hwgsjm20ff93v8xxfniasss1b2fmc4" + #:tag "903-release-a" + j-build-configuration-with-sleef)) + +(define-public (jsoftware-ijconsole-symlink jpkg) + "Provide bin/ijconsole symlink that points to pkg's +bin/ijconsole-." + (package + (name "jsoftware-ijconsole") + (version (package-version jpkg)) + (source #f) + (build-system trivial-build-system) + (propagated-inputs `(("jpkg" ,jpkg))) + (arguments + `(#:modules ((guix build utils) + (srfi srfi-26)) + #:builder + (begin + (use-modules ((guix build utils) #:select (mkdir-p)) + ((ice-9 regex) #:select (string-match)) + ((ice-9 ftw) #:select (scandir)) + ((srfi srfi-26) #:select (cut))) + (let* ((out (assoc-ref %outputs "out")) + (jpkg (assoc-ref %build-inputs "jpkg")) + (ijconsole (car (scandir (string-append jpkg "/bin") + (cut string-match "ijconsole-.*" <>)))) + (source (string-append jpkg "/bin/" ijconsole)) + (dest (string-append out "/bin/ijconsole"))) + (mkdir-p (dirname dest)) + (symlink source dest))))) + (home-page (package-home-page jpkg)) + (synopsis "Provide `ijconsole' symlink to default interpreter version") + (description + "The interpreter provided by the J package has a filename like +ijconsole-, which provides support for having multiple, concurrent +versions installed. This package provides a version-agnostic `ijconsole' +symlink to interpreter version indicated and build time.") + (license license:gpl3+))) diff --git a/gnu/packages/patches/jsoftware-j901-f-fixes.patch b/gnu/packages/patches/jsoftware-j901-f-fixes.patch new file mode 100644 index 0000000000..0ac7e94de4 --- /dev/null +++ b/gnu/packages/patches/jsoftware-j901-f-fixes.patch @@ -0,0 +1,80 @@ +This patch fixes two separate issues with ustream sources: + +* Normalize import paths in jsrc/cip.c + +Upstream claims to have some build requirements that force them to use strange +import paths. However, these paths do not exist inside our build chroot. + +* Fix unititialized variable warning + +Clang 9 issues some warnings which cause the build to fail since upstream +compiles with -Werror. + + +diff --git a/jsrc/cip.c b/jsrc/cip.c +index 61da4088..fb3c03b6 100644 +--- a/jsrc/cip.c ++++ b/jsrc/cip.c +@@ -3,9 +3,9 @@ + /* */ + /* Conjunctions: Inner Product */ + +-#include "../../jsource/jsrc/j.h" +-#include "../../jsource/jsrc/vasm.h" +-#include "../../jsource/jsrc/gemm.h" ++#include "j.h" ++#include "vasm.h" ++#include "gemm.h" + + #define MAXAROWS 384 // max rows of a that we can process to stay in L2 cache a strip is m*CACHEHEIGHT, z strip is m*CACHEWIDTH this is wired to 128*3 - check if you chage + +@@ -1057,15 +1057,15 @@ static A jtipbx(J jt,A a,A w,C c,C d){A g=0,x0,x1,z;B*av,*av0,b,*v0,*v1,*zv;C c0 + switch(c){ + case CPLUSDOT: + #define F |= +-#include "../../jsource/jsrc/cip_t.h" ++#include "cip_t.h" + break; + case CSTARDOT: + #define F &= +-#include "../../jsource/jsrc/cip_t.h" ++#include "cip_t.h" + break; + case CNE: + #define F ^= +-#include "../../jsource/jsrc/cip_t.h" ++#include "cip_t.h" + break; + } + R z; +diff --git a/jsrc/gemm.c b/jsrc/gemm.c +index 51fe306e..b105dfc1 100644 +--- a/jsrc/gemm.c ++++ b/jsrc/gemm.c +@@ -318,7 +318,7 @@ dgemm_nn (I m, + _B); + + // loop 3 +- I i; ++ I i=0; + #pragma omp parallel for default(none),private(i),shared(j,l,A,C,mb,nc,kc,alpha,_beta,_mc,_B,rs_a,cs_a,rs_c,cs_c) + for (i=0; i