From patchwork Sat Oct 2 11:30:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pierre Langlois X-Patchwork-Id: 33550 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 1060227BBE3; Sat, 2 Oct 2021 12:41:33 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS, T_DKIM_INVALID,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 1BB0627BBE1 for ; Sat, 2 Oct 2021 12:41:32 +0100 (BST) Received: from localhost ([::1]:33638 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mWdOV-00036S-6J for patchwork@mira.cbaines.net; Sat, 02 Oct 2021 07:41:31 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51322) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mWdO2-00034t-EK for guix-patches@gnu.org; Sat, 02 Oct 2021 07:41:02 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:46548) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mWdO2-0001KE-2C for guix-patches@gnu.org; Sat, 02 Oct 2021 07:41:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1mWdO1-0007tT-Sm for guix-patches@gnu.org; Sat, 02 Oct 2021 07:41:01 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#49946] [PATCH 0/3] guix: node-build-system: Support compiling addons with node-gyp. Resent-From: Pierre Langlois Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sat, 02 Oct 2021 11:41:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 49946 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Philip McGrath Cc: Pierre Langlois , 49946@debbugs.gnu.org, Maxime Devos Received: via spool by 49946-submit@debbugs.gnu.org id=B49946.163317480930259 (code B ref 49946); Sat, 02 Oct 2021 11:41:01 +0000 Received: (at 49946) by debbugs.gnu.org; 2 Oct 2021 11:40:09 +0000 Received: from localhost ([127.0.0.1]:58092 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mWdNA-0007ry-GU for submit@debbugs.gnu.org; Sat, 02 Oct 2021 07:40:09 -0400 Received: from mout.gmx.net ([212.227.17.21]:53149) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mWdN7-0007rA-Ur for 49946@debbugs.gnu.org; Sat, 02 Oct 2021 07:40:07 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1633174798; bh=HR+bdLtpPgf30J4B1Qr3YvIPDUGQXn7pA5Z9UkN4pgk=; h=X-UI-Sender-Class:References:From:To:Cc:Subject:Date:In-reply-to; b=lkLFhD17LFyMiSOU6qtn2ihD+j8IQgy/Zz5q83o70BrWIUTFkHjx15X9kr7muTACQ 7FAG/LShpKmuUZuyDRzlWiUXdM/ItOKx/BdmrpiusKA91wcyR0gvrjV78V60GxCPmO AWsQMLCg1Et7y3Ba54YjyC/kC8Mew4P1cYFs+F/s= X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Received: from labiere ([82.69.64.142]) by mail.gmx.net (mrgmx104 [212.227.17.174]) with ESMTPSA (Nemesis) id 1MAONX-1me6QX1ucV-00Bqb3; Sat, 02 Oct 2021 13:39:58 +0200 References: <682f2ed3-90bd-a9f8-1193-34bef399ca62@philipmcgrath.com> <87h7fztt60.fsf@gmx.com> <20210808233354.6745-1-pierre.langlois@gmx.com> <20210808233354.6745-8-pierre.langlois@gmx.com> <42e10baddb6afe308f67c3240bf5da8159e6f118.camel@telenet.be> <87o88gq5p5.fsf@gmx.com> <20210930225611.2143194-1-philip@philipmcgrath.com> User-agent: mu4e 1.6.6; emacs 27.2 From: Pierre Langlois Date: Sat, 02 Oct 2021 12:30:56 +0100 In-reply-to: <20210930225611.2143194-1-philip@philipmcgrath.com> Message-ID: <87r1d3venp.fsf@gmx.com> MIME-Version: 1.0 X-Provags-ID: V03:K1:5D+XveQupYM0PHuHvGYPVytq9aXFRRozO+mLGlsIJpL/r39KSrY Dkl353AprYFcs8ekwS2LsrMBSVRtmEO3kVZ9DAMRAvGFdUcklGDN6+JLjz4SzvF9awWCTIA mqZXUHzW+Czk3y/MxXs08yTYMpLhUNTSoa3eyQoUEpVionZLK+mhFJe059o5pQI+o+V4yso YVPQbYUMGuU+2v2rx0Kcw== X-UI-Out-Filterresults: notjunk:1;V03:K0:37bPWH7u9p4=:ofa0WkaUa950765Qu8i7h1 6DM8DpvLLeOihOnoDmNz/QiBjEGfEQSKH9rSSjitKFB0AZ5zNshjSyKyyjLG5j3yE5HoVm8Il gD8wYBL6Y/pJyXUyRMh15NrB42F9gJhT8X5T5Yh7QSEhwdYFA2d/lQjZFaVI5QWaey7Crsxd1 Uw/7nZD7WgWSvmFJMt6pH95IUfAy9J5lvPGZPfsi65d3kkp0aIkcLcJxrpLeJyEXHALwS2eeZ lRkZ1lQNBo1Lduon/QCixf/c7OvA4CmP8NgVf2Bq0kAUcDDRKmz7bBrqEdbacJhfvKhvyjU8h j+Vkkp3zBl7YeCu3pVSTkHX5ZU6YTtWhPvrBuV5WiHwLD7CQIEYy7I0FeeneqZuc7cAy8BWb+ Ljmduqfh+hWhYG4O+iHOQm3Paq0q4Zfo4bVVOw5IoDF+yfuyF6UTGHKeWnqpOML5NemB3riks 9CezkArRqTYpvOS1Wls4LDLZoC9FYd+XFZjFbyn/spiYhwGRdcfckUm8nWwdB0BxUSsoZXeKo 9DFPglTWCTkt3vjBkMeU7k+KSoqm0jiH0eLZ9FW6oLMcMHPPI7PoLH3TxQy0atrhODko5Nluk 4BecJOTnI2K0aE4oPvkg+vZa+XGpMBVGtJjKWkw2J6YgXW/v9036G5ODJ4ip14N+IA7hfSd0x Sb0TTwyakSpwsTlNgvugPvvO5KLTgPJgtpIz7GkYiAqJlgm7ZhuzviuXaqTk8zyGjX0IImu9T Fg3zz0eDfBlc8C9xlWkC7jQLEjLP7XhwG2V1zTZT4YcbmSzh+mV2Au2TxaqZxiAmLoSvBykGD mcX7J/ao84Ck7aluFlJHNxvfpzUMq5TSwjMWB1eZ4GTcOvvoAtgXmwFHASVSaLH/RNI7xx3tX VyBWegu0X8hlqKNgjxiK03VRbOxxRifrRR+3vQbNZRhTPUvKdUbkKV2DI551NZad0BkmKtPD+ oqbYtcd07RRTa31nYFw5kFNJuZNbPwuXW9R+kDBPoUcYJd8pmISZKtgBIeF8btn3cumQzQL28 /fAPwqZ46JxB5E8B/iqCcMFGFweA3NxtY2b6pjaieKnj4B1BymShD+W55PvvvYRN57nFZYTje 7iq6HCmSAjN7so= X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches Hi Philip, Philip McGrath writes: > Hi, > > I've reworked the part of the patch series dealing with node-gyp. Nice, thanks for working on this! I'll take a look at the patches, sorry I've not had too much time this week yet :-). > I'd like to find an NPM addon package to submit as part of this series, too, > basically as a test case. If I can find one that doesn't raise too many other > complications, I may send this in as a separate patch, but feel free to try it > with tree-sitter, too. Cool, I'll give it a test with the tree-sitter series. > > There are a few things I'm still not sure about. I haven't made > node-build-system add libuv as an implicit input, because I think some > node-gyp addons don't actually need libuv, but maybe it's common enough that > it should be done automatically. I see, I assumed libuv was needed for all addons. I'm thinking given node itself heavily depends on libuv anymays, it might be nice to add it by default in the build system, but I don't really mind either way. I'll check if I can just add libuv in the tree-sitter packages. > > Likewise, I haven't tried to change the issue of `node` referring to > `node-bootstrap`, but I still think it should be changed. I agree it would be good to change this. I started working on updating node last weekend but haven't had time to work on it after that. I think we should probably name node always as "node-": node -> node-10 node-lts -> node-14 Then keep the node and node-lts names as aliases: (define-public node node-14) ;; The latest, but move it to node-16 later (define-public node-lts node-14) However, moving the node variable to node-14 is likely to cause rebuilds, so we'll have to check that. Anyways, here's my WIP patch in case there's anything of interest there. Note that it still has the bug you pointed out where I'm refering to the host python instead of the target one (thanks for pointing it out!). Thanks, Pierre From b43db52b9a1b55b5756268b710907813f07a42ba Mon Sep 17 00:00:00 2001 From: Pierre Langlois Date: Sun, 26 Sep 2021 18:58:53 +0100 Subject: [PATCH] update node --- gnu/packages/adns.scm | 46 ++++++++++- gnu/packages/node.scm | 175 +++++++++++++++++++++++++++++++++++++----- 2 files changed, 198 insertions(+), 23 deletions(-) diff --git a/gnu/packages/adns.scm b/gnu/packages/adns.scm index b36ec18462..6fba9783e0 100644 --- a/gnu/packages/adns.scm +++ b/gnu/packages/adns.scm @@ -23,8 +23,10 @@ #:use-module (guix licenses) #:use-module (guix packages) #:use-module (guix download) + #:use-module (guix git-download) #:use-module (guix build-system cmake) #:use-module (guix build-system gnu) + #:use-module (gnu packages autotools) #:use-module (gnu packages m4) #:use-module (gnu packages pkg-config)) @@ -92,7 +94,7 @@ multiple clients and programs with graphical user interfaces.") (package (inherit c-ares) (name "c-ares") - (version "1.17.1") + (version "1.17.2") (source (origin (method url-fetch) (uri (string-append @@ -100,10 +102,46 @@ multiple clients and programs with graphical user interfaces.") ".tar.gz")) (sha256 (base32 - "0h7wjfnk2092glqcp9mqaax7xx0s13m501z1gi0gsjl2vvvd0gfp")))) + "0gcincjvpll2qmlc906jx6mfq97s87mgi0zby0753ki0rr2ch0s8")))) (arguments - `(;; FIXME: Some tests require network access - #:tests? #f)))) + `(#:phases + (modify-phases %standard-phases + (replace 'check + (lambda* (#:key tests? #:allow-other-keys) + (when tests? + (invoke "./test/arestest" + (string-append + ;; "Live" tests require network access. + "--gtest_filter=-*.Live*:" + ;; FIXME: This test fails in the build sandbox, but + ;; otherwise passes. + "AddressFamiliesAI/" + "MockChannelTestAI.FamilyV4ServiceName/0")))))))))) + +(define-public c-ares-for-node + (let ((commit "6299d7be383de62da831a3c48f4017b70c664de8") + (revision "1")) + (package + (inherit c-ares/fixed) + (name "c-ares") + (version (git-version "1.17.2" revision commit)) + (source + (origin + (method git-fetch) + (uri (git-reference + (url "https://github.com/c-ares/c-ares.git") + (commit commit))) + (file-name (git-file-name name version)) + (sha256 + (base32 "1rmk7m7lv5kmqxhb2dmq5fxk73iicg5rgsz2y855hk0a92xlrcsd")))) + (native-inputs + `(("pkg-config" ,pkg-config) + ("autoconf" ,autoconf) + ("automake" ,automake) + ("libtool" ,libtool)))))) + ;; (arguments + ;; `(;; FIXME: Some tests require network access + ;; #:tests? #f))))) ;; gRPC requires a c-ares built with CMake in order to get the .cmake modules. ;; We can not build c-ares itself with CMake because that would introduce a diff --git a/gnu/packages/node.scm b/gnu/packages/node.scm index 71da2aa18f..79ffa8fca9 100644 --- a/gnu/packages/node.scm +++ b/gnu/packages/node.scm @@ -53,7 +53,7 @@ #:use-module (ice-9 match) #:use-module (srfi srfi-26)) -(define-public node +(define-public node-10 (package (name "node") (version "10.24.0") @@ -284,7 +284,7 @@ devices.") ;; This should be the latest version of node that still builds without ;; depending on llhttp. (define-public node-bootstrap - (hidden-package node)) + (hidden-package node-10)) ;; Duplicate of node-semver (define-public node-semver-bootstrap @@ -524,17 +524,17 @@ Node.js and web browsers.") (define-public node-llparse-bootstrap (package (name "node-llparse") - (version "7.1.0") + (version "7.1.1") (source (origin (method git-fetch) (uri (git-reference - (url "https://github.com/indutny/llparse.git") + (url "https://github.com/nodejs/llparse.git") (commit (string-append "v" version)))) (file-name (git-file-name name version)) (sha256 (base32 - "10da273iy2if88hp79cwms6c8qpsl1fkgzll6gmqyx5yxv5mkyp6")) + "0gzsa4nwrhvm7gz817l5r6v7i8lmqpnrg25smqiq6x8xgs8dlmgl")) (modules '((guix build utils))) (snippet '(begin @@ -577,7 +577,7 @@ Node.js and web browsers.") parser definition into a C output.") (license license:expat))) -(define-public llhttp-bootstrap +(define-public llhttp-bootstrap-2 (package (name "llhttp") (version "2.1.3") @@ -648,17 +648,42 @@ parser definition into a C output.") source files.") (license license:expat))) -(define-public node-lts +(define-public llhttp-bootstrap + (package (inherit llhttp-bootstrap-2) + (name "llhttp") + (version "6.0.5") + (source (origin + (method git-fetch) + (uri (git-reference + (url "https://github.com/nodejs/llhttp.git") + (commit (string-append "v" version)))) + (file-name (git-file-name name version)) + (sha256 + (base32 + "1wc5xsa76my32k86cax540q0g4y85w7cikqvdjy3rkz2r7fjlyyk")) + (modules '((guix build utils))) + (snippet + '(begin + ;; Fix imports for esbuild. + ;; https://github.com/evanw/esbuild/issues/477 + (substitute* "src/llhttp/http.ts" + (("\\* as assert") "assert")) + (substitute* "Makefile" + (("npx ts-node bin/generate.ts") + "node bin/generate.js")) + #t)))))) + +(define-public node-14 (package - (inherit node) - (version "14.16.0") + (inherit node-10) + (version "14.17.6") (source (origin (method url-fetch) (uri (string-append "https://nodejs.org/dist/v" version "/node-v" version ".tar.xz")) (sha256 (base32 - "19nz2mhmn6ikahxqyna1dn25pb5v3z9vsz9zb2flb6zp2yk4hxjf")) + "0pmd0haav2ychhcsw44klx6wfn8c7j1rsw08rc8hcm5i3h5wsn7l")) (modules '((guix build utils))) (snippet `(begin @@ -675,7 +700,7 @@ source files.") (("deps/zlib/zlib.gyp") "")) #t)))) (arguments - (substitute-keyword-arguments (package-arguments node) + (substitute-keyword-arguments (package-arguments node-10) ((#:configure-flags configure-flags) ''("--shared-cares" "--shared-libuv" @@ -798,6 +823,8 @@ source files.") '("test/parallel/test-dns.js" "test/parallel/test-dns-lookupService-promises.js")) + (delete-file "test/parallel/test-https-agent-unref-socket.js") + ;; FIXME: This test fails randomly: ;; https://github.com/nodejs/node/issues/31213 (delete-file "test/parallel/test-net-listen-after-destroying-stdin.js") @@ -822,9 +849,10 @@ source files.") ;; TODO: Regenerate certs instead. (for-each delete-file '("test/parallel/test-tls-passphrase.js" - "test/parallel/test-tls-server-verify.js")) - - ;; Replace pre-generated llhttp sources + "test/parallel/test-tls-server-verify.js")))) + ;; Replace pre-generated llhttp sources + (add-after 'patch-files 'replace-llhttp + (lambda* (#:key inputs #:allow-other-keys) (let ((llhttp (assoc-ref inputs "llhttp"))) (copy-file (string-append llhttp "/src/llhttp.c") "deps/llhttp/src/llhttp.c") @@ -836,7 +864,7 @@ source files.") "deps/llhttp/include/llhttp.h")))))))) (native-inputs `(;; Runtime dependencies for binaries used as a bootstrap. - ("c-ares" ,c-ares) + ("c-ares" ,c-ares-for-node) ("google-brotli" ,google-brotli) ("icu4c" ,icu4c-67) ("libuv" ,libuv-for-node) @@ -852,20 +880,129 @@ source files.") (inputs `(("bash" ,bash) ("coreutils" ,coreutils) - ("c-ares" ,c-ares) + ("c-ares" ,c-ares-for-node) ("icu4c" ,icu4c-67) ("libuv" ,libuv-for-node) - ("llhttp" ,llhttp-bootstrap) + ("llhttp" ,llhttp-bootstrap-2) ("google-brotli" ,google-brotli) ("nghttp2" ,nghttp2 "lib") ("openssl" ,openssl) ("zlib" ,zlib))))) +(define-public node + (package + (inherit node-14) + (version "16.10.0") + (source (origin + (method url-fetch) + (uri (string-append "https://nodejs.org/dist/v" version + "/node-v" version ".tar.xz")) + (sha256 + (base32 + "04krpy0r8msv64rcf0vy2l2yzf0a401km8p5p7h12j9b4g51mp4p")) + (modules '((guix build utils))) + (snippet + `(begin + ;; Remove bundled software, where possible + (for-each delete-file-recursively + '("deps/cares" + "deps/icu-small" + "deps/nghttp2" + "deps/openssl" + "deps/zlib")) + (substitute* "Makefile" + ;; Remove references to bundled software. + (("deps/uv/uv.gyp") "") + (("deps/zlib/zlib.gyp") "")) + #t)))) + (arguments + (substitute-keyword-arguments (package-arguments node-14) + ((#:phases phases) + `(modify-phases ,phases + (replace 'patch-files + (lambda* (#:key native-inputs inputs #:allow-other-keys) + ;; Fix hardcoded /bin/sh references. + (substitute* '("lib/child_process.js" + "lib/internal/v8_prof_polyfill.js" + "test/parallel/test-child-process-spawnsync-shell.js" + "test/parallel/test-fs-write-sigxfsz.js" + "test/parallel/test-stdio-closed.js" + "test/parallel/test-stdin-from-file-spawn.js" + "test/sequential/test-child-process-emfile.js") + (("'/bin/sh'") + (string-append "'" (assoc-ref inputs "bash") "/bin/sh'"))) + + ;; Fix hardcoded /usr/bin/env references. + (substitute* '("test/parallel/test-child-process-default-options.js" + "test/parallel/test-child-process-env.js" + "test/parallel/test-child-process-exec-env.js") + (("'/usr/bin/env'") + (string-append "'" (assoc-ref inputs "coreutils") + "/bin/env'"))) + + ;; Fix /usr/bin/env shebang in node-gyp. + (substitute* "deps/npm/node_modules/node-gyp/bin/node-gyp.js" + (("#!/usr/bin/env") + (string-append "#!" (assoc-ref inputs "coreutils") "/bin/env"))) + + (substitute* "deps/npm/node_modules/node-gyp/gyp/gyp_main.py" + (("#!/usr/bin/env python") + (string-append "#!" (assoc-ref (or native-inputs inputs) + "python") + "/bin/python3"))) + + ;; FIXME: These tests fail in the build container, but they don't + ;; seem to be indicative of real problems in practice. + (for-each delete-file + '("test/parallel/test-cluster-primary-error.js" + "test/parallel/test-cluster-primary-kill.js")) + + ;; These require a DNS resolver. + (for-each delete-file + '("test/parallel/test-dns.js" + "test/parallel/test-dns-lookupService-promises.js")) + + ;; FIXME: This test fails randomly: + ;; https://github.com/nodejs/node/issues/31213 + (delete-file "test/parallel/test-net-listen-after-destroying-stdin.js") + + ;; FIXME: These tests fail on armhf-linux: + ;; https://github.com/nodejs/node/issues/31970 + ,@(if (target-arm32?) + '((for-each delete-file + '("test/parallel/test-zlib.js" + "test/parallel/test-zlib-brotli.js" + "test/parallel/test-zlib-brotli-flush.js" + "test/parallel/test-zlib-brotli-from-brotli.js" + "test/parallel/test-zlib-brotli-from-string.js" + "test/parallel/test-zlib-convenience-methods.js" + "test/parallel/test-zlib-random-byte-pipes.js" + "test/parallel/test-zlib-write-after-flush.js"))) + '()) + + ;; These tests have an expiry date: they depend on the validity of + ;; TLS certificates that are bundled with the source. We want this + ;; package to be reproducible forever, so remove those. + ;; TODO: Regenerate certs instead. + (for-each delete-file + '("test/parallel/test-tls-passphrase.js" + "test/parallel/test-tls-server-verify.js")))))))) + (native-inputs + (alist-replace "icu4c" (list icu4c-68) + (package-native-inputs node-14))) + (inputs + (alist-replace "icu4c" (list icu4c-68) + (alist-replace "llhttp" (list llhttp-bootstrap) + (package-inputs node-14)))))) + +;; LTS release used by the node build system. +(define-public node-lts node-14) + (define-public libnode - (package/inherit node + (package/inherit node-10 (name "libnode") (arguments - (substitute-keyword-arguments (package-arguments node) + (substitute-keyword-arguments (package-arguments node-10) ((#:configure-flags flags ''()) `(cons* "--shared" "--without-npm" ,flags)) ((#:phases phases '%standard-phases) -- 2.33.0