mbox series

[bug#49946,v4,00/31] gnu: Add tree-sitter for emacs.

Message ID 877d9s2qa9.fsf@gmx.com
Headers show
Series gnu: Add tree-sitter for emacs. | expand

Message

Pierre Langlois Feb. 18, 2022, 2:12 p.m. UTC 
Hi Guix!

Here's a v4 that's in a pretty good shape IMO. Here's a summary of the
changes compared to previous versions, there were lots actually! :-)

- Removed bundled icu4c headers from `tree-sitter', as pointed out on
  https://issues.guix.gnu.org/53523.

- Added a snippet that removes all files generated by the
  `tree-sitter generate' command, to be used in all grammar packages.

- For each grammar package, provide two outputs:

  * "out": With just the compiled grammar.

--8<---------------cut here---------------start------------->8---
/gnu/store/mb2hcaanm6ld8rx19p1ihk2fs1m7lcj8-tree-sitter-c-0.20.1
├── lib
│   └── tree-sitter
│       └── c.so
└── share
    └── doc
        └── tree-sitter-c-0.20.1
            └── LICENSE
--8<---------------cut here---------------end--------------->8---

  * "js": With the JS package output, so that it may be used as a
    dependency. For example C++ imports the C grammar, and Typescript
    imports JavaScript.

- Removed the dependency on node-gyp!

  By default, the `npm' build system builds JavaScript bindings for each
  grammar, so that node applications may use them as a library to parse
  a given language. We don't need this yet in Guix, we just want to
  generate C code from the JS grammar description and compile it.

  So when invoking `tree-sitter generate', we disable bindings
  generation and then we ignore all custom scripts when installing the
  JS package. This simplifies grammar packages a *lot*, given those
  custom script often try to download binaries and would have had to be
  patched.

- Grammars are compiled manually.

  Previous versions relied on doing `tree-sitter test' so that a grammar
  would be loaded and compiled by `tree-sitter' into its own cache. We
  would then install the library from that cache location, which can
  change across versions.

  This felt like a hack, looking at the source code, we're just
  compiling a couple of files at most, we can do that ourselve.

- I added support for a few extra languages, like ruby, C#, elixir and
  ocaml.

- Last but very much not least, tests for emacs-tree-sitter-langs are
  enabled!

  This is important because this package doesn't just bundle all the
  grammars, it also provides custom highlighting rules, which depend on
  the grammar and can easily be out-of-sync and break. Thanks to
  enabling these tests, I realized the php grammar need updating and
  didn't work for example.

Let me know if you have any feedback on this series! It's taking me
quite some time to follow it through, it's not the easiest thing to
package :-/.

I do wonder if I'm going about it the right way for Guix though, for
instance I'm wondering if I should rewrite the grammar packages using a
new custom build system, do people think it would be worth it? That
could be done as a follow-up of course.

Thanks,
Pierre

Pierre Langlois (31):
  gnu: Add rust-smallbitvec.
  gnu: Add rust-html-escape.
  gnu: Add rust-spin@0.7.
  gnu: Add tree-sitter.
  gnu: Add tree-sitter-cli.
  gnu: Add tree-sitter-c.
  gnu: Add tree-sitter-cpp.
  gnu: Add tree-sitter-bash.
  gnu: Add tree-sitter-css.
  gnu: Add tree-sitter-c-sharp.
  gnu: Add tree-sitter-elixir.
  gnu: Add tree-sitter-elm.
  gnu: Add tree-sitter-go.
  gnu: Add tree-sitter-html.
  gnu: Add tree-sitter-java.
  gnu: Add tree-sitter-javascript.
  gnu: Add tree-sitter-json.
  gnu: Add tree-sitter-julia.
  gnu: Add tree-sitter-ocaml.
  gnu: Add tree-sitter-php.
  gnu: Add tree-sitter-python.
  gnu: Add tree-sitter-ruby.
  gnu: Add tree-sitter-rust.
  gnu: Add tree-sitter-typescript.
  gnu: Add rust-tree-sitter-for-emacs.
  gnu: rust-emacs-module: Update to 0.18.
  gnu: rust-emacs-macros: Update to 0.17.
  gnu: rust-emacs: Update to 0.18.
  gnu: Add emacs-tree-sitter-core.
  gnu: Add emacs-tree-sitter.
  gnu: Add emacs-tree-sitter-langs.

 gnu/local.mk                                  |    2 +
 gnu/packages/crates-io.scm                    |  154 ++-
 .../rust-tree-sitter-text-provider-fix.patch  |   98 ++
 gnu/packages/tree-sitter.scm                  | 1088 +++++++++++++++++
 4 files changed, 1324 insertions(+), 18 deletions(-)
 create mode 100644 gnu/packages/patches/rust-tree-sitter-text-provider-fix.patch
 create mode 100644 gnu/packages/tree-sitter.scm