From patchwork Fri Feb 18 14:12:08 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Pierre Langlois X-Patchwork-Id: 625 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id B1F1327BBEA; Fri, 18 Feb 2022 14:39:22 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,FREEMAIL_FROM,MAILING_LIST_MULTI,SPF_HELO_PASS, URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 2A76327BBE9 for ; Fri, 18 Feb 2022 14:39:22 +0000 (GMT) Received: from localhost ([::1]:55198 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nL4Po-0003Uo-SP for patchwork@mira.cbaines.net; Fri, 18 Feb 2022 09:39:20 -0500 Received: from eggs.gnu.org ([209.51.188.92]:42842) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nL4PW-0003T9-Hz for guix-patches@gnu.org; Fri, 18 Feb 2022 09:39:02 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:59847) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1nL4PW-0004o0-7r for guix-patches@gnu.org; Fri, 18 Feb 2022 09:39:02 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1nL4PV-0008M8-LH for guix-patches@gnu.org; Fri, 18 Feb 2022 09:39:01 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#49946] [PATCH v4 00/31] gnu: Add tree-sitter for emacs. Resent-From: Pierre Langlois Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Fri, 18 Feb 2022 14:39:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 49946 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 49946@debbugs.gnu.org Cc: Pierre Langlois , Luis Henrique Gomes Higino Received: via spool by 49946-submit@debbugs.gnu.org id=B49946.164519513232105 (code B ref 49946); Fri, 18 Feb 2022 14:39:01 +0000 Received: (at 49946) by debbugs.gnu.org; 18 Feb 2022 14:38:52 +0000 Received: from localhost ([127.0.0.1]:53744 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nL4PM-0008Ll-Gp for submit@debbugs.gnu.org; Fri, 18 Feb 2022 09:38:52 -0500 Received: from mout.gmx.net ([212.227.17.20]:59887) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nL4PK-0008LW-Am for 49946@debbugs.gnu.org; Fri, 18 Feb 2022 09:38:51 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1645195123; bh=rj7nVDeJtppciAWh1id64olepOSqifEG9UQCv6q8R94=; h=X-UI-Sender-Class:References:From:To:Cc:Subject:Date:In-reply-to; b=JlD+BnUFqpont27cOu1c/eGcWxFXlyGjJ5FYele676xuUbTprHov67Mw0FFEjbCeF /Sq37edGq15w3i9hsSIhJRqHqTOFi7FuRcPALteg+x0MOLp3OZCtQDYPLyn7PvWS/b PBl3Cm6fO4t1TKK0uNIBjaO/AuGmmEDhX16Is+H4= X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Received: from labiere ([82.69.64.142]) by mail.gmx.net (mrgmx104 [212.227.17.174]) with ESMTPSA (Nemesis) id 1N7zFj-1oO9TN4C3f-014yGM; Fri, 18 Feb 2022 15:38:43 +0100 References: <87h7fztt60.fsf@gmx.com> <20210808233354.6745-1-pierre.langlois@gmx.com> <20210808233354.6745-7-pierre.langlois@gmx.com> <87czpwzg9e.fsf@gmx.com> <87leyinxly.fsf@gmx.com> User-agent: mu4e 1.6.10; emacs 27.2 From: Pierre Langlois Date: Fri, 18 Feb 2022 14:12:08 +0000 In-reply-to: <87leyinxly.fsf@gmx.com> Message-ID: <877d9s2qa9.fsf@gmx.com> MIME-Version: 1.0 X-Provags-ID: V03:K1:uUnUHVTf07CUrfuu4ahVDXwtSFw6gFEh76gm8mXqh5O1P5+Z358 SwKPKYuHwq6n5Lvc4q63MAIEWiDh9YFnAoDsVquIijWRbPB4G2yGMED6lyr+pTuLdTxTZz0 Xrr9EDv3IPKerEXB5aE2ELIoWMLYi1QNITyiKVr+Pwqv4wcT/e0yT7/vQV5eWEAUizz6WRs O9PzDPiHNhE8WUZfRh6sA== X-UI-Out-Filterresults: notjunk:1;V03:K0:g9p6lUDsfzE=:ZpU1E4JM5uh0vhOrrpf8Va Z5m1jO5Tom6PyqvvmjGKXSP1cb5g9wfRW73bIqhZfjTeqVJWaoAE2Cde96c+BOg/lnyvfEfKJ wyHLev1tL87ER88h6qr4yB10Ej613cqJOIWj7C50Uw97wbUjwBZOex3hvWwDNPqoBH/Ko3S6t wdmqWXW8p9e7iZkIiuE/tRmMBq2nbVfiNutbjygcvrsdmfpnJYMyuzx3GOLKzRpAa0mu0QbPp iCb+dI/1LjenRBrA/eAXVk3X3xaaSP+mBg3tValL6emIqbDQNPCgfbb+uJzd1akY2ewWTM5Ov VI8c4bLyrpL2NhwPNtO7Y6BEuTeIXlWOHuEPp5l87hfXSLCMgQoSpfdoKfScGbhDHt1W6LS53 UqmUgjFkP6laOySaS/15i21vdTrTcmOvZ1oCOWqFIQTLa1NOewJG+p4oMW9vyVUj8gbWVBTTk ne54oEblNhn/6NZ9L4C9YggW9E69wG+TlDMF1faURBeYMfbuD2VITUU5ixkjid6+TGBez5SbY XW46QMIkH4dP2ftwBpYScjoLvmDcLuO3MRf4im8msZ+6XeM+DaODtjSydO7Xum4BQYQ/FkJ78 89oFt2SM1gtZf1hQ2f043X0AAPhxqVHcwj9S7pBNmJR/jkLOdFnHitDJ1KFB2tT8ZYvuF7piw kALBgaZMIwe6Ha1lv8qMjsY6Brh3Dvw1hRa4CHrm6rYoj1imSczVDLSuLdBlu0Q8CyfEkX0VZ EptTDC2DNnwT12EY87iVhLYBXV4TG5eFuK/uIfGLjxZ5MZoKzS4YSoMubdLXqa0uIarjuTI57 17b9jWR+5Pvs7UuhNNiCm4XWA1OGwko4AW/+2gxhNhrt9H7qkpI419AXWX+2xe9IPz+8PBJbO pfXc8w9wMvy1JKj//5hcTjv3KeRFV0ZCgoTfGKCRML0lVmxwiZrIdbuoaZuCR8rI6ID+ry5TG Wd+tBuaxKn//UPoCyLrzfXxnI1ZtVO9jhzM8ktMPXz0nBzimAsZcLOu3m/n6/XppUUKQYZ9FJ iNq6XC+RuYaAGZHVz+fSoP+DKvk1CkXXEJ4gj9F1NqYY4CqPlHXSwhVW9M/i9m8Ukgnpn2tqM 0l3gWN3SSoNEhY= X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches Hi Guix! Here's a v4 that's in a pretty good shape IMO. Here's a summary of the changes compared to previous versions, there were lots actually! :-) - Removed bundled icu4c headers from `tree-sitter', as pointed out on https://issues.guix.gnu.org/53523. - Added a snippet that removes all files generated by the `tree-sitter generate' command, to be used in all grammar packages. - For each grammar package, provide two outputs: * "out": With just the compiled grammar. --8<---------------cut here---------------start------------->8--- /gnu/store/mb2hcaanm6ld8rx19p1ihk2fs1m7lcj8-tree-sitter-c-0.20.1 ├── lib │   └── tree-sitter │   └── c.so └── share └── doc └── tree-sitter-c-0.20.1 └── LICENSE --8<---------------cut here---------------end--------------->8--- * "js": With the JS package output, so that it may be used as a dependency. For example C++ imports the C grammar, and Typescript imports JavaScript. - Removed the dependency on node-gyp! By default, the `npm' build system builds JavaScript bindings for each grammar, so that node applications may use them as a library to parse a given language. We don't need this yet in Guix, we just want to generate C code from the JS grammar description and compile it. So when invoking `tree-sitter generate', we disable bindings generation and then we ignore all custom scripts when installing the JS package. This simplifies grammar packages a *lot*, given those custom script often try to download binaries and would have had to be patched. - Grammars are compiled manually. Previous versions relied on doing `tree-sitter test' so that a grammar would be loaded and compiled by `tree-sitter' into its own cache. We would then install the library from that cache location, which can change across versions. This felt like a hack, looking at the source code, we're just compiling a couple of files at most, we can do that ourselve. - I added support for a few extra languages, like ruby, C#, elixir and ocaml. - Last but very much not least, tests for emacs-tree-sitter-langs are enabled! This is important because this package doesn't just bundle all the grammars, it also provides custom highlighting rules, which depend on the grammar and can easily be out-of-sync and break. Thanks to enabling these tests, I realized the php grammar need updating and didn't work for example. Let me know if you have any feedback on this series! It's taking me quite some time to follow it through, it's not the easiest thing to package :-/. I do wonder if I'm going about it the right way for Guix though, for instance I'm wondering if I should rewrite the grammar packages using a new custom build system, do people think it would be worth it? That could be done as a follow-up of course. Thanks, Pierre Pierre Langlois (31): gnu: Add rust-smallbitvec. gnu: Add rust-html-escape. gnu: Add rust-spin@0.7. gnu: Add tree-sitter. gnu: Add tree-sitter-cli. gnu: Add tree-sitter-c. gnu: Add tree-sitter-cpp. gnu: Add tree-sitter-bash. gnu: Add tree-sitter-css. gnu: Add tree-sitter-c-sharp. gnu: Add tree-sitter-elixir. gnu: Add tree-sitter-elm. gnu: Add tree-sitter-go. gnu: Add tree-sitter-html. gnu: Add tree-sitter-java. gnu: Add tree-sitter-javascript. gnu: Add tree-sitter-json. gnu: Add tree-sitter-julia. gnu: Add tree-sitter-ocaml. gnu: Add tree-sitter-php. gnu: Add tree-sitter-python. gnu: Add tree-sitter-ruby. gnu: Add tree-sitter-rust. gnu: Add tree-sitter-typescript. gnu: Add rust-tree-sitter-for-emacs. gnu: rust-emacs-module: Update to 0.18. gnu: rust-emacs-macros: Update to 0.17. gnu: rust-emacs: Update to 0.18. gnu: Add emacs-tree-sitter-core. gnu: Add emacs-tree-sitter. gnu: Add emacs-tree-sitter-langs. gnu/local.mk | 2 + gnu/packages/crates-io.scm | 154 ++- .../rust-tree-sitter-text-provider-fix.patch | 98 ++ gnu/packages/tree-sitter.scm | 1088 +++++++++++++++++ 4 files changed, 1324 insertions(+), 18 deletions(-) create mode 100644 gnu/packages/patches/rust-tree-sitter-text-provider-fix.patch create mode 100644 gnu/packages/tree-sitter.scm