From patchwork Sun Aug 8 22:29:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pierre Langlois X-Patchwork-Id: 256 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id A469B27BC78; Mon, 9 Aug 2021 00:27:09 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS, T_DKIM_INVALID,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 1889C27BC6B for ; Mon, 9 Aug 2021 00:27:09 +0100 (BST) Received: from localhost ([::1]:38670 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mCsCC-0005jj-3I for patchwork@mira.cbaines.net; Sun, 08 Aug 2021 19:27:08 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:56110) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mCsC6-0005ja-EZ for guix-patches@gnu.org; Sun, 08 Aug 2021 19:27:02 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:43389) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mCsC6-0000XE-8A for guix-patches@gnu.org; Sun, 08 Aug 2021 19:27:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1mCsC6-0000dI-0h for guix-patches@gnu.org; Sun, 08 Aug 2021 19:27:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#49946] [PATCH 00/31] Tree-sitter, node-gyp addon support and emacs-tree-sitter Resent-From: Pierre Langlois Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sun, 08 Aug 2021 23:27:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 49946 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 49946@debbugs.gnu.org X-Debbugs-Original-To: guix-patches@gnu.org Received: via spool by submit@debbugs.gnu.org id=B.16284651852386 (code B ref -1); Sun, 08 Aug 2021 23:27:01 +0000 Received: (at submit) by debbugs.gnu.org; 8 Aug 2021 23:26:25 +0000 Received: from localhost ([127.0.0.1]:54935 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mCsBU-0000cQ-GQ for submit@debbugs.gnu.org; Sun, 08 Aug 2021 19:26:24 -0400 Received: from lists.gnu.org ([209.51.188.17]:54406) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mCsBS-0000cH-M2 for submit@debbugs.gnu.org; Sun, 08 Aug 2021 19:26:23 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:56108) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mCsBS-0005hc-BU for guix-patches@gnu.org; Sun, 08 Aug 2021 19:26:22 -0400 Received: from mout.gmx.net ([212.227.15.19]:42791) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mCsBQ-0008Qj-6Q for guix-patches@gnu.org; Sun, 08 Aug 2021 19:26:22 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1628465176; bh=+yntOmXptqFSwgFtpzEf1tsbxlIZN27mdaABHfTpEO0=; h=X-UI-Sender-Class:From:To:Subject:Date; b=RMp3hTRSaOs26fxCiI/ZVRklg5luXr6ddNe5jjrzt1uajVRV4bGAcPqZwi+oQpHqc 6EEId016qBLLZSmXYtU50/4K0m2SZvQz6j+/DvSrR+xMwUrtLey8rn1FX+3PXCQbG0 i8lfp1E1e6nzfO7ZJkrSNdbVfR5g77Ivf3up5di0= X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Received: from labiere ([82.69.64.142]) by mail.gmx.net (mrgmx004 [212.227.17.184]) with ESMTPSA (Nemesis) id 1Mn2aD-1mwuFj0or9-00k4Ke for ; Mon, 09 Aug 2021 01:26:16 +0200 User-agent: mu4e 1.6.1; emacs 27.2 From: Pierre Langlois Date: Sun, 08 Aug 2021 23:29:43 +0100 Message-ID: <87h7fztt60.fsf@gmx.com> MIME-Version: 1.0 X-Provags-ID: V03:K1:Y5BYsE5Kdb4srnwpPFVWNe/+I8pcxFIANPUb2OR2FrCZGNXNz0T I40eI/SJbewEh6iL83cOPbZwMI8JOQaT633/HMVcirNe2d4MwWOPmZ47oAaT4KQINBodyJ4 slgDaoLQcUgmM81YbkP888wroOXkAxXtFZTHoP1AZNPKjOTu3uyvYwKuCPmB8yOF71Ri4Ot pTxVBKboZSGuO6RINFfrw== X-UI-Out-Filterresults: notjunk:1;V03:K0:+X6p37ikJJI=:uJFdd2nh1S7prerLS1+nKo 2e2/AaqgrcY8IBEqo5CTFu/YtAtS9GHWIUbfDDuauETiIC9aoCfr2LZ5iuNDwmV0W/X8iB06f 6PgIh2sb95QPh71EzA5QLpp++F+PCs+4fBZf8Y6UayU1PzpLpoV+fLGU+tsJVJKoVhInR2fXZ NBGb/ww7PiW1uRp2jaEy6QrlmIheqAKxAqsPk4GsLMI1xTLk4AlQv3wIl/WQUBDMEwbiGt2nN NTfOpf+ZJ1kdubDQ2IYsNQqkC04ZNmqbSgjjSbQDhNWOLMzR9T461tvyG+46HJmQ3NXCHVP5k DwMc91a/T109LjKDqTc+XsbSm8EH4dbBE2021IA1s4DuBMSdRhxMwxKZqkZezZiLxFGUNmaCi UpSecUf/tXhCIE3bLyqXOvK2F9bngHqumuGSaHa3w3HI3xi1jVZIiwDj9B0aCG+OvzCw8YWsa 9CAJ9yVrTS6hpPWmCzfFaaM0sy7Udru/nsOCzjMATLvPzQ2ZRPxU2SRw/8qUe+VD6Diw8DOms yQ2sKnvC0G7JfXIQJYIxelAlheSJIT6ggDNz1NhGDbtWzVeQOqimF+819W78jfQNjo8RAKDf2 RvjPCmTbvQ38m75hoHzKZ3C+qe09nhC/7dCa5dLu5r3Dga0Uub/q2i4lDafWTjvwIr6qe6G30 NN/zHJ3C82lUdMChgBCi0pXJC5/JUTCutS071FMTou6bZ4Z3BlxsluQgxxQfrnkAN/SJbNOw3 c6GQVVO20GZqMzxN/cfnT1EbYj3gT6gOwLZKn4VNYcUMWMrRdpy87ECHCUcqZMfSD10JqJJlJ tpKjJfo+EqrV69tLhEdljUkT5HmvjeLHfYixrtp1hOQl2ykJbJBNAHp1RWz+nx7EnfyCM11he 5+iF/aCHthAWWJNU/qQKFVsNE4MktcJwua3A6a1naL/aZlm/RUwhC/wLD5wN5BRHNPCVE8bji hmXq5Lsv3/yV1pGaajS/ce4ZMryqciD/adbz/ZVsaeJRqELjxh74/SyMDwPABKpPnAWSKShmQ IsIG+jYv9uIwfv+vaNl5hab5G25vStqTllRcI9asLu7LEqhSx7RH+qR8Elk6bN41JC2boLCD8 nMGldSDGPdZKVa6L6H4Gg5Cg/D025YpopDTlvV0hmCZiyfYZhevTnmpEQ== Received-SPF: pass client-ip=212.227.15.19; envelope-from=pierre.langlois@gmx.com; helo=mout.gmx.net X-Spam_score_int: -25 X-Spam_score: -2.6 X-Spam_bar: -- X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches Hi Guix! Here's a patch series that adds support for the Tree-sitter library, command-line tool and its support in emacs: https://tree-sitter.github.io/tree-sitter The series in split into multiple parts: 1. Add the tree-sitter package, which is both a command-line tool for generating parsers and a runtime library. The runtime is written in C but the tooling in Rust. * gnu: rust-chunked-transfer: Update to 1.4.0. * gnu: Add rust-fdlimit. * gnu: Add rust-html-escape. * gnu: Add rust-smallbitvec. * gnu: rust-tiny-http: Update to 0.8. * gnu: rust-spin: Update to 0.7. * gnu: Add tree-sitter. 2. Add support for native addons to the node build-system. * gnu: node: Patch /usr/bin/env in node-gyp. * guix: node-build-system: Support compiling addons with node-gyp. I've not touched the node build system before so it'd be great to get some review here! The idea is that when a package has a `binding.gyp' file then there's native code that needs to be compiled against node. This uses node-gyp, which is a build system written in python. Normally, there shouldn't be anything to do, the package.json file should invoke `node-gyp` tp compile C code. However, `node-gyp' needs node headers to compile against, packaged as a tarball, which it tries to download. Instead, we can run a `node-gyp --tarball <> configure' step to manually provide the tarball, which we can package separately for any given node version. I could have added this into the packages themselves, but I thought it made more sense to be a part of the build system, let me know what you think! 3. Packaging grammars for multiple languages. Tree-sitter grammars are Nodejs packages, they consist of a grammar.js file which the tree-sitter command-line program uses to generate C code for the parser/scanner. Using npm as a packaging mechanism for this seems overkill, but it's true some grammars depend on each other (cpp on c, typescript and javascript). * gnu: Add node-nan. * gnu: Add tree-sitter-c. * gnu: Add tree-sitter-cpp. * gnu: Add tree-sitter-css. * gnu: Add tree-sitter-go. * gnu: Add tree-sitter-html. * gnu: Add tree-sitter-java. * gnu: Add tree-sitter-javascript. * gnu: Add tree-sitter-json. * gnu: Add tree-sitter-julia. * gnu: Add tree-sitter-php. * gnu: Add tree-sitter-python. * gnu: Add tree-sitter-rust. * gnu: Add tree-sitter-typescript. 4. Finally, emacs support! emacs-tree-sitter can be used for syntax highlighting, replacing font-lock. The tree-sitter runtime library is built as a Rust crate and exposed to emacs using a module thanks to the rust-emacs crate. Then emacs-tree-sitter-core provides just the core APIs as exposed by the rust module, then emacs-tree-sitter integrates it with emacs' syntax highlighting (and probably other things). Finally, we bundle together the various language grammars into a single emacs-tree-sitter-langs package so that they can get loaded along with their respective major modes. Note that there are grammars left to support which I've not yet managed to package, but what we have already is a good start I think :-). So far I've tested C++, Rust, Python and JavaScript and JSON. * gnu: Add rust-bindgen@0.56. * gnu: Add rust-tree-sitter. * gnu: rust-emacs-module: Update to 0.16. * gnu: rust-emacs-macros: Update to 0.17. * gnu: rust-emacs: Update to 0.17. * gnu: Add emacs-tree-sitter-core. * gnu: Add emacs-tree-sitter. * gnu: Add emacs-tree-sitter-langs. Let me know what you think! This rebuilds all node packages, but there aren't that many so hopefully that'll be ok on master. Thanks, Pierre Pierre Langlois (31): gnu: rust-chunked-transfer: Update to 1.4.0. gnu: Add rust-fdlimit. gnu: Add rust-html-escape. gnu: Add rust-smallbitvec. gnu: rust-tiny-http: Update to 0.8. gnu: rust-spin: Update to 0.7. gnu: Add tree-sitter. gnu: node: Patch /usr/bin/env in node-gyp. guix: node-build-system: Support compiling addons with node-gyp. gnu: Add node-nan. gnu: Add tree-sitter-c. gnu: Add tree-sitter-cpp. gnu: Add tree-sitter-css. gnu: Add tree-sitter-go. gnu: Add tree-sitter-html. gnu: Add tree-sitter-java. gnu: Add tree-sitter-javascript. gnu: Add tree-sitter-json. gnu: Add tree-sitter-julia. gnu: Add tree-sitter-php. gnu: Add tree-sitter-python. gnu: Add tree-sitter-rust. gnu: Add tree-sitter-typescript. gnu: Add rust-bindgen@0.56. gnu: Add rust-tree-sitter. gnu: rust-emacs-module: Update to 0.16. gnu: rust-emacs-macros: Update to 0.17. gnu: rust-emacs: Update to 0.17. gnu: Add emacs-tree-sitter-core. gnu: Add emacs-tree-sitter. gnu: Add emacs-tree-sitter-langs. gnu/local.mk | 1 + gnu/packages/crates-io.scm | 290 +++++++++++-- gnu/packages/node-xyz.scm | 30 ++ gnu/packages/node.scm | 29 +- gnu/packages/tree-sitter.scm | 716 +++++++++++++++++++++++++++++++ guix/build-system/node.scm | 16 + guix/build/node-build-system.scm | 15 + 7 files changed, 1064 insertions(+), 33 deletions(-) create mode 100644 gnu/packages/tree-sitter.scm