diff mbox

[bug#38606,WIP,SMLnj,0/1] Add SMLnj.

Message ID PU1PR01MB21558D0068012C681046F2038D909@PU1PR01MB2155.apcprd01.prod.exchangelabs.com
State Accepted
Headers show

Commit Message

Foo Chuan Wei Nov. 7, 2021, 3 p.m. UTC 
I managed to build a working version of SML/NJ 110.99.2 (released on
2021-09-22). The patch is included below (use `git am --scissors` to
apply).

Major caveats:
* This patch only deals with x86-64. For 32-bit, some changes need to be
  made:
  * Download "boot.x86-unix.tgz" instead of "boot.amd64-unix.tgz".
  * Patch "base/runtime/objs/mk.x86-linux" instead of
    "base/runtime/objs/mk.amd64-linux".
  * Invoke the install script using `-default 32` instead of `-default 64`.
* This does not install the SML/NJ library. I used the `-nolib` option
  to disable the building of the library. When I try to build without
  the `-nolib` option, the build always fails with this error:

----- start of snippet -----
/Users/jhr/Work/smlnj/sml-legacy/base/system/smlnj/installer.cm:29.7-29.32 Error: link-time exception in library code $smlnj/installer.cm@1570(installer/nix-install.sml)
  SysErr: No such file or directory [noent]<exec.c>


/tmp/guix-build-smlnj-110.99.2.drv-0/bin/sml: Fatal error -- Uncaught exception Link with 0
 raised at ../cm/compile/link.sml:357.28-357.35

FAILURE: unpacking failed
./config/install.sh: !!! Installation of libraries and programs failed.
command "./config/install.sh" "-default" "64" failed with status 1
----- end of snippet -----

  I don't know what this error means.

  Note that "jhr" is *not* my username. "jhr" appears to be the initials
  of John H. Reppy, one of the authors of SML/NJ.

  Surprisingly, when I keep the directory of the failed build (`guix
  build --keep-failed smlnj`), and then build inside the build directory
  by hand (i.e. `cd /tmp/guix-build-smlnj-110.99.2.drv-1 &&
  ./config/install.sh -default 64), all SML/NJ libraries are
  successfully built. I was not able to determine why the build fails in
  Guix but succeeds manually.


Credits:
* This patch is based on Brett Gilio's smlnj patches.
* I also looked at how smlnj is packaged in NixOS:
  * https://github.com/NixOS/nixpkgs/blob/21.05/pkgs/development/compilers/smlnj/default.nix
  * https://web.archive.org/web/20140704103009/http://lists.science.uu.nl/pipermail/nix-dev/attachments/20101215/e1beb921/attachment.obj


-- >8 --
Subject: [PATCH] gnu: Add smlnj.

* gnu/packages/sml.scm (smlnj): New variable.
---
 gnu/packages/sml.scm | 187 ++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 186 insertions(+), 1 deletion(-)


base-commit: a0f1476e8fa20e80c71f83342b82338ddedbf7fd

Comments

Foo Chuan Wei Nov. 8, 2021, 9:40 p.m. UTC | #1 
> /Users/jhr/Work/smlnj/sml-legacy/base/system/smlnj/installer.cm:29.7-29.32 Error: link-time exception in library code $smlnj/installer.cm@1570(installer/nix-install.sml)
>   SysErr: No such file or directory [noent]<exec.c>
> 
> 
> /tmp/guix-build-smlnj-110.99.2.drv-0/bin/sml: Fatal error -- Uncaught exception Link with 0
>  raised at ../cm/compile/link.sml:357.28-357.35
> 
> FAILURE: unpacking failed
> ./config/install.sh: !!! Installation of libraries and programs failed.
> command "./config/install.sh" "-default" "64" failed with status 1

The error above occurs because the build process uses nix-install.sml,
and nix-install.sml uses `OS.Process.system` (in the `unpack` function).
`OS.Process.system` relies on /bin/sh, but /bin/sh is not present in the
Guix build environment. "No such file or directory" presumably refers to
the absence of /bin/sh.

Unfortunately, the fix might require some binary patching. /bin/sh is
hardcoded in sml.boot.amd64-unix/SMLNJ-BASIS/.cm/amd64-unix/basis-common.cm
(a binary file from boot.amd64-unix.tgz).
Josselin Poiret Nov. 9, 2021, 10:03 a.m. UTC | #2 
Foo Chuan Wei <chuanwei.foo@hotmail.com> writes:

> The error above occurs because the build process uses nix-install.sml,
> and nix-install.sml uses `OS.Process.system` (in the `unpack` function).
> `OS.Process.system` relies on /bin/sh, but /bin/sh is not present in the
> Guix build environment. "No such file or directory" presumably refers to
> the absence of /bin/sh.
>
> Unfortunately, the fix might require some binary patching. /bin/sh is
> hardcoded in sml.boot.amd64-unix/SMLNJ-BASIS/.cm/amd64-unix/basis-common.cm
> (a binary file from boot.amd64-unix.tgz).

Linking [1] for our IRC discussion about this.

You proposed patching /bin/sh to /tmp/sh, and symlinking /tmp/sh
yourself, did that work?

If not, I can still write a SMLNJ binfile patcher if you feel the need
for it (literals should be pretty easy to patch, see
sml/gc/build-literals.c for their format).

Best,
Josslin Poiret

[1] https://logs.guix.gnu.org/guix/2021-11-08.log#135443
Foo Chuan Wei Nov. 10, 2021, 9:21 a.m. UTC | #3 
On 2021-11-09, Josselin Poiret wrote:
> You proposed patching /bin/sh to /tmp/sh, and symlinking /tmp/sh
> yourself, did that work?

Yes, it did work. I managed to build smlnj (including libraries). Thank
you for helping on IRC. Here's the patch that uses the /tmp/sh method:
https://issues.guix.gnu.org/51745
diff mbox

Patch

diff --git a/gnu/packages/sml.scm b/gnu/packages/sml.scm
index b7e96f028c..7447fa1627 100644
--- a/gnu/packages/sml.scm
+++ b/gnu/packages/sml.scm
@@ -28,7 +28,8 @@ 
   #:use-module (guix download)
   #:use-module (guix git-download)
   #:use-module ((guix licenses) #:prefix license:)
-  #:use-module (guix packages))
+  #:use-module (guix packages)
+  #:use-module (guix utils))
 
 (define-public polyml
   (package
@@ -75,3 +76,187 @@  function interface, and a symbolic debugger.")
     (license
      (list license:lgpl2.1
            license:lgpl2.1+))))
+
+(define (smlnj-file version filename hash)
+  (origin
+    (method url-fetch)
+    (uri (string-append "http://smlnj.cs.uchicago.edu/dist/working/"
+                        version "/" filename))
+    (sha256 (base32 hash))))
+
+(define-public smlnj
+  (package
+    (name "smlnj")
+    (version "110.99.2")
+    (source #f)  ; Sources are passed as native-inputs.
+    (build-system gnu-build-system)
+    (arguments
+     `(#:tests? #f
+       #:phases
+       (modify-phases %standard-phases
+         (replace 'unpack
+           (lambda* (#:key inputs #:allow-other-keys)
+             (for-each
+               (lambda (file)
+                 (invoke "tar" "xvf" (assoc-ref inputs file)))
+               (list "boot.amd64-unix"
+                     "config"
+                     "cm"
+                     "compiler"
+                     "runtime"
+                     "system"
+                     "MLRISC"
+                     "smlnj-lib"
+                     "old-basis"
+                     "ckit"
+                     "nlffi"
+                     "cml"
+                     "eXene"
+                     "ml-lpt"
+                     "ml-lex"
+                     "ml-yacc"
+                     "ml-burg"
+                     "pgraph"
+                     "trace-debug-profile"
+                     "heap2asm"
+                     "smlnj-c"
+                     "doc"
+                     "asdl"))
+             (mkdir "base")
+             ;; Same directory structure as what the config/unpack script
+             ;; would produce.
+             (rename-file "runtime" "base/runtime")
+             (rename-file "compiler" "base/compiler")
+             (rename-file "cm" "base/cm")
+             (rename-file "old-basis" "base/old-basis")
+             (rename-file "system" "base/system")))
+         (delete 'configure)
+         (replace 'patch-source-shebangs
+           (lambda _
+             (substitute* (list "config/install.sh"
+                                "base/runtime/objs/mk.amd64-linux")
+               (("SHELL.*=.*/bin/sh")
+                (string-append "SHELL=" (which "sh"))))
+             #t))
+         (replace 'build
+           (lambda* (#:key inputs #:allow-other-keys)
+             (substitute* "config/chk-global-names.sh"
+               (("CC=.*")
+                (string-append "CC=" ,(cc-for-target))))
+             (substitute* "config/_arch-n-opsys"
+               (("export PATH") "")
+               (("PATH=\"/bin:/usr/bin\"") "")
+               (("uname") (which "uname")))
+             (substitute* "base/runtime/config/gen-posix-names.sh"
+               (("PATH=/bin:/usr/bin") ""))
+             (invoke "./config/install.sh" "-default" "64" "-nolib")))
+        (replace 'install
+           (lambda* (#:key outputs #:allow-other-keys)
+             (let* ((out (assoc-ref outputs "out"))
+                    (out-lib (string-append out "/lib/"))
+                    (out-bin (string-append out "/bin/")))
+               (copy-recursively "lib" out-lib)
+               (copy-recursively "bin" out-bin))
+             #t)))))
+    (native-inputs
+     `(("boot.amd64-unix"
+        ,(smlnj-file version
+                     "boot.amd64-unix.tgz"
+                     "0cbaz8sxp30y2a4blm0pzk9aa1g9jj65d6d7kydvn0w7m22rjjff"))
+       ("config"
+        ,(smlnj-file version
+                     "config.tgz"
+                     "1sq60frd66kv807bahsyxda355qq67zvvb4sr1d72cv6capv5nsg"))
+       ("cm"
+        ,(smlnj-file version
+                     "cm.tgz"
+                     "087yy9k9vyyvhb24rni6js7s8iqbs8vjm9kjd9s311swjd66qhjc"))
+       ("compiler"
+        ,(smlnj-file version
+                     "compiler.tgz"
+                     "149vrmiba0dqggr15axqqzzfmd0kx7kchlr49gii6hw5dal6vqnn"))
+       ("runtime"
+        ,(smlnj-file version
+                     "runtime.tgz"
+                     "0c3q7qr2i3r91sx9p3z1ar7gvyk6qsm5gcbpbyj6l67qzn192yc3"))
+       ("system"
+        ,(smlnj-file version
+                     "system.tgz"
+                     "0aflnqh9zi9f8vs9pjlxysxplrwl98aiaxx4n41sba4m1r30n0za"))
+       ("MLRISC"
+        ,(smlnj-file version
+                     "MLRISC.tgz"
+                     "1kkga7r7qnw09s8yhqbhrq8gxf6c8x0fiwsbwkr8ls6xwv48sp74"))
+       ("smlnj-lib"
+        ,(smlnj-file version
+                     "smlnj-lib.tgz"
+                     "14fvqgn580k3ylvfhbkwv4cw87ipggq548r7jzp8fzfn2h6wdl01"))
+       ("old-basis"
+        ,(smlnj-file version
+                     "old-basis.tgz"
+                     "11j7a0sxyd1kwxjvxnarab5vc3x43gl90m07wxm37fd4jbfd1fn1"))
+       ("ckit"
+        ,(smlnj-file version
+                     "ckit.tgz"
+                     "1fbfjb2fhr6zkcz5jhqh3888zska6vffndyqwvk6rpbcl7an8niq"))
+       ("nlffi"
+        ,(smlnj-file version
+                     "nlffi.tgz"
+                     "0p5z77x295xfh71481kbd5pwis52kv03vxpad4pzkpk0l6smcgmj"))
+       ("cml"
+        ,(smlnj-file version
+                     "cml.tgz"
+                     "13xchaamwanxhwklsgkn1rmkr044h9qsj0rbr9c7pm903yivdwcn"))
+       ("eXene"
+        ,(smlnj-file version
+                     "eXene.tgz"
+                     "0p4snql0a1a952h98ma9ybmp7z1q305mz859b0mxhsg3jdrzl9wb"))
+       ("ml-lpt"
+        ,(smlnj-file version
+                     "ml-lpt.tgz"
+                     "0m00vglg95apdpzsy6qv88izj5ai4ibylxni01an75xpnxyy5qbg"))
+       ("ml-lex"
+        ,(smlnj-file version
+                     "ml-lex.tgz"
+                     "1pmi5qwjcf1h5nfi7d4vvm7cf90g6dlk2mqikj0y9c464ia1l2jc"))
+       ("ml-yacc"
+        ,(smlnj-file version
+                     "ml-yacc.tgz"
+                     "1zla2m1rn8r8k85ps9r2mw38xkh276j7aqv9f69v55102hchx13p"))
+       ("ml-burg"
+        ,(smlnj-file version
+                     "ml-burg.tgz"
+                     "14cqasasa273x09phykzjgk1wl6vrkdcwrdi39hnacp443cilz7x"))
+       ("pgraph"
+        ,(smlnj-file version
+                     "pgraph.tgz"
+                     "183fv61xlac5kpxn5m4iqgdvc2xb1chlxy5ip4i25x589bh4b5k9"))
+       ("trace-debug-profile"
+        ,(smlnj-file version
+                     "trace-debug-profile.tgz"
+                     "1k0w581kr43mpjzm7778xgx1rpz45aq1h80jdr6jls5vz3k8ia18"))
+       ("heap2asm"
+        ,(smlnj-file version
+                     "heap2asm.tgz"
+                     "0p9s42acngxh0401wm6fqs3im3rzzw9sh448x38zhdi47h8h1m9n"))
+       ("smlnj-c"
+        ,(smlnj-file version
+                     "smlnj-c.tgz"
+                     "054b1nhg5yk2jj01p11k08qzq8zc9jzg4mbgkcmcqaq7axp1rnxm"))
+       ("doc"
+        ,(smlnj-file version
+                     "doc.tgz"
+                     "0s35zrxdj76wzdz7c1i8ij00n6lfll4vjnypsy2j17q1maw7fq8j"))
+       ("asdl"
+        ,(smlnj-file version
+                     "asdl.tgz"
+                     "0mad2df5pmkdsb69gflxma6m6i3gla6hdmjjnkzk76pagpr8zb0m"))))
+    (home-page "http://www.smlnj.org")
+    (synopsis "Standard ML of New Jersey interactive compiler")
+    (description
+      "SML/NJ is an implementation of the Standard ML programming language.
+Standard ML has many features, including type safety, polymorphism, algebraic
+data types with pattern matching, higher-order functions, and a sophisticated
+module system.  It is especially well-suited for writing compilers and other
+language processors.")
+    (license license:bsd-2)))