Message ID | CAKf5CqUPbRg1BL1WgR9K7Y-1pZz4fFKQVX6=xc8gXOfn8mGJrg@mail.gmail.com |
---|---|
State | Under Review |
Headers | show |
Series | [bug#39309,WIP] gnu: add stack. | expand |
Context | Check | Description |
---|---|---|
cbaines/comparison | success | View comparision |
cbaines/git branch | success | View Git branch |
cbaines/applying patch | fail | View Laminar job |
Hi John, John Soo <jsoo1@asu.edu> writes: > I tried using this patch and it still fails. Could I be missing > something? I’m not sure, but I got the shadowing to work. In fact, you don’t need shadowing at all. You can just pass “--with-hsc2hs=/gnu/store/...” as a configure flag: (arguments `(#:configure-flags (let ((hsc2hs (assoc-ref %build-inputs "ghc-hsc2hs"))) (list (string-append "--with-hsc2hs=" hsc2hs "/bin/hsc2hs"))))) Unfortunately, it doesn’t help. Looking at the source code, “hsc2hs” *is* calling GCC with a response file. AIUI, GCC is supposed to continue to use response files for all of its subprograms. Hence, everything should work. I’m stumped. :/ I’ll take another look with a fresh mind sometime, and see if I can make any progress. -- Tim
Hello again, Timothy Sample <samplet@ngyro.com> writes: > [...] > > I’ll take another look with a fresh mind sometime, and see if I can make > any progress. I don’t have a solution, but I have a bit more info. Cabal generates extremely long argument lists for GCC. For whatever reason, it repeats the same flags many times (maybe once for each Haskell dependency). When building Stack, it passes Gawk as an include path 164 times. In fact, there are only a handful of distinct include paths, each passed 164 times. The linker flags are similar but much worse. This should be fine since everything is using a response file, right? Nope. Like the bug you linked discusses, GCC writes all of its arguments into an environment variable, the length of which is limited by “MAX_ARG_STRLEN” (which is 128K). It would be nice to fix GCC here, and it’s almost a very small fix, but there is minor complication. The normal response file handling in GCC expects “argc” and “argv” parameters, but the environment variable doesn’t come with an “argc”. Rather, it has to be parsed (in the same way the shell would do it). It’s still doable, but a it would be a larger change. Not to mention that patching GCC is even less fun than patching GHC. There was some work to deduplicate the flags that Cabal generates: <https://github.com/haskell/cabal/pull/5356/>. This would actually fix the build, but the call to “hsc2hs” doesn’t do the same deduplication (see “ppHsc2hs” in “Cabal/Distribution/Simple/PreProcess.hs”). At this point, it might be worth joining the Nix folks on that issue, and suggesting a patch to Cabal to call “hsc2hs” with deduplicated flags. It will help that the other deduplication code was written by the same person who filed the bug you linked to earlier. Once the patch lands, we can apply it to our Cabal and then Stack will build. Yay! -- Tim
Hi Tim, > At this point, it might be worth joining the Nix folks on that issue, > and suggesting a patch to Cabal to call “hsc2hs” with deduplicated > flags. It will help that the other deduplication code was written by > the same person who filed the bug you linked to earlier. Once the patch > lands, we can apply it to our Cabal and then Stack will build. Yay! Sorry I'm a bit confused by all this. I guess if we can rely on a patch into cabal that would be the least effort, right? Is there already a patch to deduplicate flags passed to hsc2hs? Do we need to comment anywhere or express support? - John
Hi John, John Soo <jsoo1@asu.edu> writes: >> At this point, it might be worth joining the Nix folks on that issue, >> and suggesting a patch to Cabal to call “hsc2hs” with deduplicated >> flags. It will help that the other deduplication code was written by >> the same person who filed the bug you linked to earlier. Once the patch >> lands, we can apply it to our Cabal and then Stack will build. Yay! > > Sorry I'm a bit confused by all this. I guess if we can rely on a > patch into cabal that would be the least effort, right? Exactly. Also, sorry if I’m being a little unclear. I’ve been kind of thinking out loud with these messages, as you’ll see below. ;) > Is there already a patch to deduplicate flags passed to hsc2hs? Do we > need to comment anywhere or express support? I couldn’t find a patch. Expressing support was definitely what I was suggesting. I managed to find a note saying what I just said: https://github.com/NixOS/nixpkgs/issues/49206#issuecomment-470324743 I don’t think anything has become of it since then, though. There is one other thing we could do, but I’m still thinking it through. The reason we hit the limit is because of the way we use the “extra-lib-dirs” and “extra-include-dirs” flags when configuring. Every time we pass Cabal a list of directories via these flags, Cabal notes the directories in the package DB, and uses them when calling GCC. It doesn’t just use the directories specified when configuring a given package. It uses all the directories specified for all of that package’s dependencies, too. Right now, we pass in every “lib” and “include” directory from every input, including the “standard-packages” from the GNU build system. This means that Gawk’s “lib” directory (for example) is included every time we configure any package. In turn, when GHC calls GCC as part of compiling a Haskell package, Gawk gets included on the command line once for every node on that package’s dependency tree. Packages with a large dependency tree hit the 128K limit imposed by the kernel Linux. What’s funny is that not a single Haskell package even uses shared libraries from Gawk! The other thing that’s funny is that most Haskell packages build just fine without “extra-lib-dirs” and “extra-include-dirs”. Even those that call out to C code work, because Guix sets “LIBRARY_PATH” for us. Unfortunately, packages that depend on packages that call out to C code fail. This is because Guix doesn’t set “LIBRARY_PATH” for the transitive input, but GHC still wants to make use of the shared library. This can be solved by adding back the “extra-lib-dirs” and “extra-include-dirs” flags as-needed. With this approach, I was able to compile Stack (the tests failed, though). The part that I’m still thinking through is how to make this whole system work. I would really like something that works automatically, but I don’t see a way to do it. This means we would have to add the flags manually to the handful of packages that need them. The part that I really don’t like is that we don’t know a package needs the flags until a package that depends on it fails to build. Having everything one step removed like that is not ideal. On the other hand, there are probably less than a dozen Haskell packages that need flags – it’s not that common to mix Haskell and C code. I will make a patch that does this and see what the damages are. This feels like a step in the right direction, because we end up with a cleaner package graph as a result. The added work and delayed feedback will be tricky, but it should only come up infrequently. What do you think? -- Tim
From 200b3387c5466bc2cd96772400f5244fe9b78907 Mon Sep 17 00:00:00 2001 From: John Soo <jsoo1@asu.edu> Date: Sun, 26 Jan 2020 11:19:47 -0800 Subject: [PATCH] gnu: Add stack. * gnu/packages/haskell-xyz.scm (stack): New variable. --- gnu/packages/haskell-apps.scm | 129 ++++++++++++++++++++++++++++++++++ 1 file changed, 129 insertions(+) diff --git a/gnu/packages/haskell-apps.scm b/gnu/packages/haskell-apps.scm index 0a4a4f62cd..cbe7de3b0f 100644 --- a/gnu/packages/haskell-apps.scm +++ b/gnu/packages/haskell-apps.scm @@ -35,6 +35,7 @@ #:use-module (guix packages) #:use-module ((guix licenses) #:prefix license:) #:use-module (guix build-system haskell) + #:use-module (guix utils) #:use-module (gnu packages base) #:use-module (gnu packages curl) #:use-module (gnu packages gl) @@ -691,6 +692,134 @@ advanced user's otherwise working script to fail under future circumstances. @end enumerate") (license license:gpl3+))) +(define-public stack + (package + (name "stack") + (version "2.1.3.1") + (source + (origin + (method url-fetch) + (uri (string-append + "https://hackage.haskell.org/package/stack/stack-" + version + ".tar.gz")) + (sha256 + (base32 + "1q2nagnc24fvyj3hwnpgyp3rqhxswhscyw4pw2dazqx34ad3d0zr")))) + (build-system haskell-build-system) + (arguments + ;; Substitute ghc's bundled version of hsc2hs to a version that uses + ;; response files. + ;; See https://github.com/haskell/hsc2hs/issues/22 + `(#:haskell + ,(package + (inherit ghc-8.6) + (native-inputs + (cons `("ghc-hsc2hs" ,(package-source ghc-hsc2hs)) + (package-native-inputs ghc-8.6))) + (arguments + (substitute-keyword-arguments (package-arguments ghc-8.6) + ((#:phases ghc-8.6-phases) + `(modify-phases ,ghc-8.6-phases + (add-after 'unpack 'de-vendor-hsc2hs + (lambda* (#:key inputs #:allow-other-keys) + (invoke + "tar" "-xvf" (assoc-ref inputs "ghc-hsc2hs") + "-C" "utils/hsc2hs" "--overwrite")))))))))) + (inputs + `(("ghc-aeson" ,ghc-aeson) + ("ghc-annotated-wl-pprint" ,ghc-annotated-wl-pprint) + ("ghc-ansi-terminal" ,ghc-ansi-terminal) + ("ghc-async" ,ghc-async) + ("ghc-attoparsec" ,ghc-attoparsec) + ("ghc-base64-bytestring" ,ghc-base64-bytestring) + ("ghc-colour" ,ghc-colour) + ("ghc-conduit" ,ghc-conduit) + ("ghc-conduit-extra" ,ghc-conduit-extra) + ("ghc-cryptonite" ,ghc-cryptonite) + ("ghc-cryptonite-conduit" ,ghc-cryptonite-conduit) + ("ghc-echo" ,ghc-echo) + ("ghc-exceptions" ,ghc-exceptions) + ("ghc-extra" ,ghc-extra) + ("ghc-file-embed" ,ghc-file-embed) + ("ghc-filelock" ,ghc-filelock) + ("ghc-fsnotify" ,ghc-fsnotify) + ("ghc-generic-deriving" ,ghc-generic-deriving) + ("ghc-hackage-security" ,ghc-hackage-security) + ("ghc-hashable" ,ghc-hashable) + ("ghc-hi-file-parser" ,ghc-hi-file-parser) + ("ghc-hpack" ,ghc-hpack) + ("ghc-http-client" ,ghc-http-client) + ("ghc-http-client-tls" ,ghc-http-client-tls) + ("ghc-http-conduit" ,ghc-http-conduit) + ("ghc-http-download" ,ghc-http-download) + ("ghc-http-types" ,ghc-http-types) + ("ghc-memory" ,ghc-memory) + ("ghc-microlens" ,ghc-microlens) + ("ghc-mintty" ,ghc-mintty) + ("ghc-mono-traversable" ,ghc-mono-traversable) + ("ghc-mustache" ,ghc-mustache) + ("ghc-neat-interpolation" ,ghc-neat-interpolation) + ("ghc-network-uri" ,ghc-network-uri) + ("ghc-open-browser" ,ghc-open-browser) + ("ghc-optparse-applicative" ,ghc-optparse-applicative) + ("ghc-pantry" ,ghc-pantry) + ("ghc-path" ,ghc-path) + ("ghc-path-io" ,ghc-path-io) + ("ghc-persistent" ,ghc-persistent) + ("ghc-persistent-sqlite" ,ghc-persistent-sqlite) + ("ghc-persistent-template" ,ghc-persistent-template) + ("ghc-primitive" ,ghc-primitive) + ("ghc-project-template" ,ghc-project-template) + ("ghc-regex-applicative-text" ,ghc-regex-applicative-text) + ("ghc-resource-pool" ,ghc-resource-pool) + ("ghc-resourcet" ,ghc-resourcet) + ("ghc-retry" ,ghc-retry) + ("ghc-rio" ,ghc-rio) + ("ghc-rio-prettyprint" ,ghc-rio-prettyprint) + ("ghc-semigroups" ,ghc-semigroups) + ("ghc-split" ,ghc-split) + ("ghc-streaming-commons" ,ghc-streaming-commons) + ("ghc-tar" ,ghc-tar) + ("ghc-temporary" ,ghc-temporary) + ("ghc-terminal-size" ,ghc-terminal-size) + ("ghc-text-metrics" ,ghc-text-metrics) + ("ghc-th-reify-many" ,ghc-th-reify-many) + ("ghc-tls" ,ghc-tls) + ("ghc-typed-process" ,ghc-typed-process) + ("ghc-unicode-transforms" ,ghc-unicode-transforms) + ("ghc-unix-compat" ,ghc-unix-compat) + ("ghc-unliftio" ,ghc-unliftio) + ("ghc-unordered-containers" ,ghc-unordered-containers) + ("ghc-vector" ,ghc-vector) + ("ghc-yaml" ,ghc-yaml) + ("ghc-zip-archive" ,ghc-zip-archive) + ("ghc-zlib" ,ghc-zlib) + ("ghc-githash" ,ghc-githash) + ("ghc-optparse-simple" ,ghc-optparse-simple) + ("ghc-hspec" ,ghc-hspec) + ("ghc-optparse-generic" ,ghc-optparse-generic))) + (native-inputs + `(("ghc-quickcheck" ,ghc-quickcheck) + ("ghc-raw-strings-qq" ,ghc-raw-strings-qq) + ("ghc-smallcheck" ,ghc-smallcheck))) + (home-page "http://haskellstack.org") + (synopsis "Haskell Tool Stack") + (description + "Stack is a cross-platform program for developing Haskell projects. It +is aimed at Haskellers both new and experienced. + +It features: + +@itemize +@item Installing GHC automatically, in an isolated location. +@item Installing packages needed for your project. +@item Building your project. +@item Testing your project. +@item Benchmarking your project. +@end itemize") + (license license:bsd-3))) + (define-public stylish-haskell (package (name "stylish-haskell") -- 2.25.0