[bug#68554,v3,3/6] gnu: Add dmd.

* gnu/packages/dlang.scm (dmd-bootstrap, dmd): New variables.

Change-Id: I53e00a429e2084a392341ef1bc7ea63f0fcaaab4
---

Changes in v3:
 - Add packages built from dmd and D demangling support to qt-creator

Changes in v2:
 - Build make flags via inheritance...
 - Fixing the missing SYSCONFDIR make flag

 gnu/packages/dlang.scm | 165 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 165 insertions(+)
  

On Thu, Jan 18, 2024 at 11:40:41PM -0500, Maxim Cournoyer wrote:
> * gnu/packages/dlang.scm (dmd-bootstrap, dmd): New variables.
> 
> Change-Id: I53e00a429e2084a392341ef1bc7ea63f0fcaaab4
> ---
> 
> Changes in v3:
>  - Add packages built from dmd and D demangling support to qt-creator
> 
> Changes in v2:
>  - Build make flags via inheritance...
>  - Fixing the missing SYSCONFDIR make flag
> 
>  gnu/packages/dlang.scm | 165 +++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 165 insertions(+)
> 
> diff --git a/gnu/packages/dlang.scm b/gnu/packages/dlang.scm
> index 5c9766110d..f8060fa0c3 100644
> --- a/gnu/packages/dlang.scm
> +++ b/gnu/packages/dlang.scm
> @@ -358,6 +358,171 @@ (define-public ldc
>           `(("clang" ,clang-14)          ;propagates llvm and clang-runtime
>             ("python-lit" ,python-lit))))))
>  
> +;;; Bootstrap version of phobos that is built with GDC, using GDC's standard
> +;;; library.
> +(define dmd-bootstrap
> +  (package
> +    ;; This package is purposefully named just "dmd" and not "dmd-bootstrap",
> +    ;; as the final dmd package rewrites references from this one to itself,
> +    ;; and their names must have the same length to avoid corrupting the
> +    ;; binary.
> +    (name "dmd")
> +    (version "2.106.1")
> +    (source (origin
> +              (method git-fetch)
> +              (uri (git-reference
> +                    (url "https://github.com/dlang/dmd")
> +                    (commit (string-append "v" version))))
> +              (file-name (git-file-name "dmd" version))
> +              (sha256
> +               (base32
> +                "1bq4jws1vns2jjzfz7biyngrx9y5pvvgklymhrvb5kvbzky1ldmy"))))
> +    (build-system gnu-build-system)
> +    (arguments
> +     (list
> +      #:disallowed-references (list (gexp-input (canonical-package gcc)
> +                                                "lib"))
> +      ;; Disable tests, as gdmd cannot cope with some arguments used such as
> +      ;; '-conf'.
> +      #:tests? #f
> +      #:test-target "test"
> +      #:make-flags
> +      #~(list (string-append "CC=" #$(cc-for-target))
> +              ;; XXX: Proceed despite conflicts from symbols provided by both
> +              ;; the source built and GDC.
> +              "DFLAGS=-L--allow-multiple-definition"
> +              "ENABLE_RELEASE=1"
> +              (string-append "HOST_CXX=" #$(cxx-for-target))
> +              "HOST_DMD=gdmd"
> +              (string-append "INSTALL_DIR=" #$output)
> +              ;; Do not build the shared libphobos2.so library, to avoid
> +              ;; retaining a reference to gcc:lib.
> +              "SHARED=0"
> +              (string-append "SYSCONFDIR=" #$output "/etc")
> +              "VERBOSE=1"
> +              "-f" "posix.mak")
> +      #:phases
> +      #~(modify-phases %standard-phases
> +          (add-after 'unpack 'copy-phobos-source-and-chdir
> +            ;; Start with building phobos, which in turns will automatically
> +            ;; build druntime and dmd.  A minimal dmd command is still
> +            ;; required to do so, which is why we need dmd-bootstrap-0.
> +            (lambda _
> +              (symlink "." "dmd")  ;to please the build system expected layout
> +              (copy-recursively
> +               #$(origin
> +                   (method git-fetch)
> +                   (uri (git-reference
> +                         (url "https://github.com/dlang/phobos")
> +                         (commit (string-append "v" version))))
> +                   (file-name (git-file-name name version))

you should probably switch name to "phobos", otherwise I assume it'll be
added as dmd. Also, is this the right way to add phobos as an input?

> +                   (sha256
> +                    (base32
> +                     "1yw7nb5d78cx9m7sfibv7rfc7wj3w0dw9mfk3d269qpfpnwzs4n9")))
> +               "phobos")
> +              (chdir "phobos")))
> +          (add-after 'copy-phobos-source-and-chdir 'adjust-phobos-install-dirs
> +            (lambda _
> +              (substitute* "posix.mak"
> +                ;; Install to lib directory, not to e.g. 'linux/lib64'.
> +                (("\\$\\(INSTALL_DIR)/\\$\\(OS)/\\$\\(lib_dir)")
> +                 (string-append #$output "/lib"))
> +                ;; Do not install license file, already done by the gnu build
> +                ;; system.
> +                ((".*\\$\\(INSTALL_DIR)/phobos-LICENSE.txt.*") ""))))
> +          (delete 'configure)
> +          (add-after 'install 'install-druntime
> +            (lambda args
> +              (chdir "../druntime")
> +              (apply (assoc-ref %standard-phases 'install) args)
> +              (chdir "..")))

Can this one be:
(with-directory-excursion "../druntime"
  (apply (assoc-ref %standard-phases 'install) args))


> +          (add-after 'install-druntime 'install-includes
> +            (lambda _
> +              ;; Normalize the include files prefix to include/dmd.
> +              (let ((include-dir (string-append #$output "/include/dmd")))
> +                (mkdir-p include-dir)
> +                (rename-file (string-append #$output "/src/phobos")
> +                             (string-append include-dir))
> +                (copy-recursively "druntime/import" include-dir))
> +              (delete-file-recursively (string-append #$output "/src"))))
> +          (add-after 'install-druntime 'install-dmd
> +            (assoc-ref %standard-phases 'install))

Is this different than the regular install phase?

> +          (add-after 'install-license-files 'refine-install-layout
> +            (lambda _
> +              (let* ((docdir (string-append #$output "/share/doc/"
> +                                            (strip-store-file-name #$output)))

is strip-store-file-name better than #$name "-" #$version ?

> +                     ;; The dmd binary gets installed to
> +                     ;; e.g. /linux/bin64/dmd.
> +                     (dmd (car (find-files #$output "^dmd$")))
> +                     (dmd.conf (car (find-files #$output "^dmd.conf$")))
> +                     (os-dir (dirname (dirname dmd))))
> +                ;; Move samples from root to the doc directory.
> +                (rename-file (string-append #$output "/samples")
> +                             (string-append docdir "/samples"))
> +                ;; Remove duplicate license file.
> +                (delete-file (string-append #$output
> +                                            "/dmd-boostlicense.txt"))
> +                ;; Move dmd binary and dmd.conf.
> +                (install-file dmd (string-append #$output "/bin"))
> +                (install-file dmd.conf (string-append #$output "/etc"))
> +                (delete-file-recursively os-dir))))
> +          (add-after 'refine-install-layout 'patch-dmd.conf
> +            (lambda* (#:key outputs #:allow-other-keys)
> +              (substitute* (search-input-file outputs "etc/dmd.conf")

I only see 1 output, so this should probably be
(string-append #$output "/etc/dmd.conf")

> +                (("lib(32|64)")
> +                 "lib")
> +                (("\\.\\./src/(phobos|druntime/import)")
> +                 "include/dmd")))))))
> +    (native-inputs (list gdmd which))
> +    (home-page "https://github.com/dlang/dmd")
> +    (synopsis "Reference D Programming Language compiler")
> +    (description "@acronym{DMD, Digital Mars D compiler} is the reference
> +compiler for the D programming language.")
> +    (license license:boost1.0)))
> +
> +;;; Second bootstrap of DMD, built using dmd-bootstrap, with its shared
> +;;; libraries preserved.
> +(define-public dmd
> +  (package
> +    (inherit dmd-bootstrap)
> +    (arguments
> +     (substitute-keyword-arguments
> +         (strip-keyword-arguments
> +          '(#:tests?)                   ;reinstate tests
> +          (package-arguments dmd-bootstrap))
> +       ((#:disallowed-references  _ ''())
> +        (list dmd-bootstrap))
> +       ((#:modules _ ''())
> +        '((guix build gnu-build-system)
> +          (guix build utils)
> +          (srfi srfi-1)))               ;for fold
> +       ((#:make-flags flags ''())
> +        #~(fold delete #$flags '("DFLAGS=-L--allow-multiple-definition"
> +                                 "HOST_DMD=gdmd"
> +                                 "SHARED=0")))
> +       ((#:phases phases '%standard-phases)
> +        #~(modify-phases #$phases
> +            (add-after 'patch-dmd.conf 'rewrite-references-to-bootstrap
> +              ;; DMD keeps references to include files used to build a
> +              ;; binary.  Rewrite those of dmd-bootstrap to itself, to reduce
> +              ;; its closure size.
> +              (lambda* (#:key native-inputs inputs outputs
> +                        #:allow-other-keys)
> +                (let ((dmd (search-input-file outputs "bin/dmd"))
> +                      (dmd-bootstrap (dirname
> +                                      (dirname
> +                                       (search-input-file
> +                                        (or native-inputs inputs)
> +                                        "bin/dmd")))))
> +                  ;; XXX: Use sed, as replace-store-references wouldn't
> +                  ;; replace the references, while substitute* throws an
> +                  ;; error.
> +                  (invoke "sed" "-i"
> +                          (format #f "s,~a,~a,g" dmd-bootstrap #$output)
> +                          dmd))))))))
> +    (native-inputs (modify-inputs (package-native-inputs dmd-bootstrap)
> +                     (replace "gdmd" dmd-bootstrap)))))
> +
>  (define-public dub
>    (package
>      (name "dub")
> -- 
> 2.41.0
> 
> 
> 
>
  

Hello!

Efraim Flashner <efraim@flashner.co.il> writes:

> On Thu, Jan 18, 2024 at 11:40:41PM -0500, Maxim Cournoyer wrote:
>> * gnu/packages/dlang.scm (dmd-bootstrap, dmd): New variables.
>> 
>> Change-Id: I53e00a429e2084a392341ef1bc7ea63f0fcaaab4

[...]

>> +;;; Bootstrap version of phobos that is built with GDC, using GDC's standard
>> +;;; library.
>> +(define dmd-bootstrap
>> +  (package
>> +    ;; This package is purposefully named just "dmd" and not "dmd-bootstrap",
>> +    ;; as the final dmd package rewrites references from this one to itself,
>> +    ;; and their names must have the same length to avoid corrupting the
>> +    ;; binary.
>> +    (name "dmd")
>> +    (version "2.106.1")
>> +    (source (origin
>> +              (method git-fetch)
>> +              (uri (git-reference
>> +                    (url "https://github.com/dlang/dmd")
>> +                    (commit (string-append "v" version))))
>> +              (file-name (git-file-name "dmd" version))
>> +              (sha256
>> +               (base32
>> +                "1bq4jws1vns2jjzfz7biyngrx9y5pvvgklymhrvb5kvbzky1ldmy"))))
>> +    (build-system gnu-build-system)
>> +    (arguments
>> +     (list
>> +      #:disallowed-references (list (gexp-input (canonical-package gcc)
>> +                                                "lib"))
>> +      ;; Disable tests, as gdmd cannot cope with some arguments used such as
>> +      ;; '-conf'.
>> +      #:tests? #f
>> +      #:test-target "test"
>> +      #:make-flags
>> +      #~(list (string-append "CC=" #$(cc-for-target))
>> +              ;; XXX: Proceed despite conflicts from symbols provided by both
>> +              ;; the source built and GDC.
>> +              "DFLAGS=-L--allow-multiple-definition"
>> +              "ENABLE_RELEASE=1"
>> +              (string-append "HOST_CXX=" #$(cxx-for-target))
>> +              "HOST_DMD=gdmd"
>> +              (string-append "INSTALL_DIR=" #$output)
>> +              ;; Do not build the shared libphobos2.so library, to avoid
>> +              ;; retaining a reference to gcc:lib.
>> +              "SHARED=0"
>> +              (string-append "SYSCONFDIR=" #$output "/etc")
>> +              "VERBOSE=1"
>> +              "-f" "posix.mak")
>> +      #:phases
>> +      #~(modify-phases %standard-phases
>> +          (add-after 'unpack 'copy-phobos-source-and-chdir
>> +            ;; Start with building phobos, which in turns will automatically
>> +            ;; build druntime and dmd.  A minimal dmd command is still
>> +            ;; required to do so, which is why we need dmd-bootstrap-0.
>> +            (lambda _
>> +              (symlink "." "dmd")  ;to please the build system expected layout
>> +              (copy-recursively
>> +               #$(origin
>> +                   (method git-fetch)
>> +                   (uri (git-reference
>> +                         (url "https://github.com/dlang/phobos")
>> +                         (commit (string-append "v" version))))
>> +                   (file-name (git-file-name name version))
>
> you should probably switch name to "phobos", otherwise I assume it'll be
> added as dmd.

Good catch, done.

> Also, is this the right way to add phobos as an input?

It's the gexp way to add extra source files to a package workspace.  We
used to do this via labeled origins, but this is not possible in the
label-less new world.  I think it'd be cool to have a procedure that
would take multiple source and merge them into one, possible using a
copy-build-system like "plan" for the layout.

Note: I tried building phobos separately at first, but that was a world
of hurt, and the end result was a seg faulting dmd :-(.

>> +                   (sha256
>> +                    (base32
>> +                     "1yw7nb5d78cx9m7sfibv7rfc7wj3w0dw9mfk3d269qpfpnwzs4n9")))
>> +               "phobos")
>> +              (chdir "phobos")))
>> +          (add-after 'copy-phobos-source-and-chdir 'adjust-phobos-install-dirs
>> +            (lambda _
>> +              (substitute* "posix.mak"
>> +                ;; Install to lib directory, not to e.g. 'linux/lib64'.
>> +                (("\\$\\(INSTALL_DIR)/\\$\\(OS)/\\$\\(lib_dir)")
>> +                 (string-append #$output "/lib"))
>> +                ;; Do not install license file, already done by the gnu build
>> +                ;; system.
>> +                ((".*\\$\\(INSTALL_DIR)/phobos-LICENSE.txt.*") ""))))
>> +          (delete 'configure)
>> +          (add-after 'install 'install-druntime
>> +            (lambda args
>> +              (chdir "../druntime")
>> +              (apply (assoc-ref %standard-phases 'install) args)
>> +              (chdir "..")))
>
> Can this one be:
> (with-directory-excursion "../druntime"
>   (apply (assoc-ref %standard-phases 'install) args))

Confusingly, no, as what I'm doing here is moving from ./phobos to
./druntime and finally to "."; i.e. the starting directory for the
'install-druntime phase is './phobos', not '.'

Does that make some sense (writing it I'm doubtful, eh) ?

>
>> +          (add-after 'install-druntime 'install-includes
>> +            (lambda _
>> +              ;; Normalize the include files prefix to include/dmd.
>> +              (let ((include-dir (string-append #$output "/include/dmd")))
>> +                (mkdir-p include-dir)
>> +                (rename-file (string-append #$output "/src/phobos")
>> +                             (string-append include-dir))
>> +                (copy-recursively "druntime/import" include-dir))
>> +              (delete-file-recursively (string-append #$output "/src"))))
>> +          (add-after 'install-druntime 'install-dmd
>> +            (assoc-ref %standard-phases 'install))
>
> Is this different than the regular install phase?

I the install phase of druntime doesn't install its import includes, for
some reason, and the install phase of phobos puts them under src/phobos
instead of include/dmd.

>> +          (add-after 'install-license-files 'refine-install-layout
>> +            (lambda _
>> +              (let* ((docdir (string-append #$output "/share/doc/"
>> +                                            (strip-store-file-name #$output)))
>
> is strip-store-file-name better than #$name "-" #$version ?

I got bit by gexps being expanded in-place and not being rewritten in
the inherited package, so to reuse that phase I had to use dynamic
alternatives.

>> +                     ;; The dmd binary gets installed to
>> +                     ;; e.g. /linux/bin64/dmd.
>> +                     (dmd (car (find-files #$output "^dmd$")))
>> +                     (dmd.conf (car (find-files #$output "^dmd.conf$")))
>> +                     (os-dir (dirname (dirname dmd))))
>> +                ;; Move samples from root to the doc directory.
>> +                (rename-file (string-append #$output "/samples")
>> +                             (string-append docdir "/samples"))
>> +                ;; Remove duplicate license file.
>> +                (delete-file (string-append #$output
>> +                                            "/dmd-boostlicense.txt"))
>> +                ;; Move dmd binary and dmd.conf.
>> +                (install-file dmd (string-append #$output "/bin"))
>> +                (install-file dmd.conf (string-append #$output "/etc"))
>> +                (delete-file-recursively os-dir))))
>> +          (add-after 'refine-install-layout 'patch-dmd.conf
>> +            (lambda* (#:key outputs #:allow-other-keys)
>> +              (substitute* (search-input-file outputs "etc/dmd.conf")
>
> I only see 1 output, so this should probably be
> (string-append #$output "/etc/dmd.conf")

The search-input-file on outputs is a defensive trick: it fails with a
clear error message when the file isn't found (I can't say that much
about substitute* ^^').  So it's used here to test the file got
installed there as expected.

Thanks for taking a look!