[PATCH 00/31] Tree-sitter, node-gyp addon support and emacs-tree-sitter

  • Open
  • quality assurance status badge
Details
9 participants
  • Andrew Tropin
  • Liliana Marie Prikler
  • Luis Henrique Gomes Higino
  • Maxime Devos
  • Mekeor Melire
  • (
  • Philip McGrath
  • Pierre Langlois
  • zimoun
Owner
unassigned
Submitted by
Pierre Langlois
Severity
normal
P
P
Pierre Langlois wrote on 9 Aug 2021 00:29
(address . guix-patches@gnu.org)
87h7fztt60.fsf@gmx.com
Hi Guix!

Here's a patch series that adds support for the Tree-sitter library,
command-line tool and its support in emacs:

The series in split into multiple parts:

1. Add the tree-sitter package, which is both a command-line tool for
generating parsers and a runtime library. The runtime is written in
C but the tooling in Rust.

* gnu: rust-chunked-transfer: Update to 1.4.0.
* gnu: Add rust-fdlimit.
* gnu: Add rust-html-escape.
* gnu: Add rust-smallbitvec.
* gnu: rust-tiny-http: Update to 0.8.
* gnu: rust-spin: Update to 0.7.
* gnu: Add tree-sitter.

2. Add support for native addons to the node build-system.

* gnu: node: Patch /usr/bin/env in node-gyp.
* guix: node-build-system: Support compiling addons with node-gyp.

I've not touched the node build system before so it'd be great to get
some review here!

The idea is that when a package has a `binding.gyp' file then there's
native code that needs to be compiled against node. This uses
node-gyp, which is a build system written in python.
Normally, there shouldn't be anything to do, the package.json file
should invoke `node-gyp` tp compile C code. However, `node-gyp' needs
node headers to compile against, packaged as a tarball, which it tries
to download. Instead, we can run a `node-gyp --tarball <> configure'
step to manually provide the tarball, which we can package separately
for any given node version.

I could have added this into the packages themselves, but I thought it
made more sense to be a part of the build system, let me know what you
think!

3. Packaging grammars for multiple languages.

Tree-sitter grammars are Nodejs packages, they consist of a grammar.js
file which the tree-sitter command-line program uses to generate C
code for the parser/scanner. Using npm as a packaging mechanism for
this seems overkill, but it's true some grammars depend on each other
(cpp on c, typescript and javascript).

* gnu: Add node-nan.
* gnu: Add tree-sitter-c.
* gnu: Add tree-sitter-cpp.
* gnu: Add tree-sitter-css.
* gnu: Add tree-sitter-go.
* gnu: Add tree-sitter-html.
* gnu: Add tree-sitter-java.
* gnu: Add tree-sitter-javascript.
* gnu: Add tree-sitter-json.
* gnu: Add tree-sitter-julia.
* gnu: Add tree-sitter-php.
* gnu: Add tree-sitter-python.
* gnu: Add tree-sitter-rust.
* gnu: Add tree-sitter-typescript.

4. Finally, emacs support!

emacs-tree-sitter can be used for syntax highlighting, replacing
font-lock. The tree-sitter runtime library is built as a Rust crate
and exposed to emacs using a module thanks to the rust-emacs crate.

Then emacs-tree-sitter-core provides just the core APIs as exposed by
the rust module, then emacs-tree-sitter integrates it with emacs'
syntax highlighting (and probably other things).

Finally, we bundle together the various language grammars into a
single emacs-tree-sitter-langs package so that they can get loaded
along with their respective major modes. Note that there are grammars
left to support which I've not yet managed to package, but what we
have already is a good start I think :-). So far I've tested C++,
Rust, Python and JavaScript and JSON.

* gnu: Add rust-bindgen@0.56.
* gnu: Add rust-tree-sitter.
* gnu: rust-emacs-module: Update to 0.16.
* gnu: rust-emacs-macros: Update to 0.17.
* gnu: rust-emacs: Update to 0.17.
* gnu: Add emacs-tree-sitter-core.
* gnu: Add emacs-tree-sitter.
* gnu: Add emacs-tree-sitter-langs.


Let me know what you think! This rebuilds all node packages, but there
aren't that many so hopefully that'll be ok on master.

Thanks,
Pierre

Pierre Langlois (31):
gnu: rust-chunked-transfer: Update to 1.4.0.
gnu: Add rust-fdlimit.
gnu: Add rust-html-escape.
gnu: Add rust-smallbitvec.
gnu: rust-tiny-http: Update to 0.8.
gnu: rust-spin: Update to 0.7.
gnu: Add tree-sitter.
gnu: node: Patch /usr/bin/env in node-gyp.
guix: node-build-system: Support compiling addons with node-gyp.
gnu: Add node-nan.
gnu: Add tree-sitter-c.
gnu: Add tree-sitter-cpp.
gnu: Add tree-sitter-css.
gnu: Add tree-sitter-go.
gnu: Add tree-sitter-html.
gnu: Add tree-sitter-java.
gnu: Add tree-sitter-javascript.
gnu: Add tree-sitter-json.
gnu: Add tree-sitter-julia.
gnu: Add tree-sitter-php.
gnu: Add tree-sitter-python.
gnu: Add tree-sitter-rust.
gnu: Add tree-sitter-typescript.
gnu: Add rust-bindgen@0.56.
gnu: Add rust-tree-sitter.
gnu: rust-emacs-module: Update to 0.16.
gnu: rust-emacs-macros: Update to 0.17.
gnu: rust-emacs: Update to 0.17.
gnu: Add emacs-tree-sitter-core.
gnu: Add emacs-tree-sitter.
gnu: Add emacs-tree-sitter-langs.

gnu/local.mk | 1 +
gnu/packages/crates-io.scm | 290 +++++++++++--
gnu/packages/node-xyz.scm | 30 ++
gnu/packages/node.scm | 29 +-
gnu/packages/tree-sitter.scm | 716 +++++++++++++++++++++++++++++++
guix/build-system/node.scm | 16 +
guix/build/node-build-system.scm | 15 +
7 files changed, 1064 insertions(+), 33 deletions(-)
create mode 100644 gnu/packages/tree-sitter.scm

--
2.32.0
-----BEGIN PGP SIGNATURE-----

iQFMBAEBCgA2FiEEctU9gYy29KFyWDdMqPyeRH9PfVQFAmEQaBcYHHBpZXJyZS5s
YW5nbG9pc0BnbXguY29tAAoJEKj8nkR/T31Uk64H/RTjvpOAq1dLgQVt69syGPZO
cx2KRhhKea+vAnjzkcau71H+ekqv2faeUCeRYJATBViOsavxbs0cMp4lqTS+7YQY
1K0rH/sR0Kc4D5Reb6OhUBkp1QjbrETydrAhp9vdX8KPwsMf8qApaxZT7KC49eYj
UeiCFytDQ0f7fcELUWMnIsy3ca1F9Q/FarUofITWGNiJ5biWxkr0xhdO4qe7roo/
xEElSYGbVp7qNeXcyku7kRWQES7F2ijsc354Hv1G6HAA76tkyDF6r2OOzKtGLFPg
rpfduwfzWo4DENtOvSjekIHtkVSWuW3g5s6ThN+jSoAZf0IUE4XoxSn5i5b/Keg=
=vVea
-----END PGP SIGNATURE-----

P
P
Pierre Langlois wrote on 9 Aug 2021 01:33
[PATCH 02/31] gnu: Add rust-fdlimit.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20210808233354.6745-2-pierre.langlois@gmx.com
* gnu/packages/crates-io.scm (rust-fdlimit-0.1): New variable.
---
gnu/packages/crates-io.scm | 22 ++++++++++++++++++++++
1 file changed, 22 insertions(+)

Toggle diff (33 lines)
diff --git a/gnu/packages/crates-io.scm b/gnu/packages/crates-io.scm
index 01a0dff724..d6b52a217f 100644
--- a/gnu/packages/crates-io.scm
+++ b/gnu/packages/crates-io.scm
@@ -15930,6 +15930,28 @@ floats.")
it.")
(license (list license:expat license:asl2.0))))

+(define-public rust-fdlimit-0.1
+ (package
+ (name "rust-fdlimit")
+ (version "0.1.4")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (crate-uri "fdlimit" version))
+ (file-name
+ (string-append name "-" version ".tar.gz"))
+ (sha256
+ (base32
+ "0g30d6gqkrwy8ylwdy7pqm443iq0p5dmnpz4ks41pirl7dclm98d"))))
+ (build-system cargo-build-system)
+ (arguments
+ `(#:cargo-inputs
+ (("rust-libc" ,rust-libc-0.2))))
+ (home-page "https://github.com/paritytech/fdlimit")
+ (synopsis "Raise the open file descriptors limit")
+ (description "Utility crate for raising file descriptors limit.")
+ (license license:asl2.0)))
+
(define-public rust-femme-2
(package
(name "rust-femme")
--
2.32.0
P
P
Pierre Langlois wrote on 9 Aug 2021 01:33
[PATCH 03/31] gnu: Add rust-html-escape.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20210808233354.6745-3-pierre.langlois@gmx.com
* gnu/packages/crates-io.scm (rust-html-escape-0.2): New variable.
---
gnu/packages/crates-io.scm | 26 ++++++++++++++++++++++++++
1 file changed, 26 insertions(+)

Toggle diff (37 lines)
diff --git a/gnu/packages/crates-io.scm b/gnu/packages/crates-io.scm
index d6b52a217f..4a9db4943b 100644
--- a/gnu/packages/crates-io.scm
+++ b/gnu/packages/crates-io.scm
@@ -20824,6 +20824,32 @@ Hash-based Message Authentication Code algorithm} for SHA1.")
"This package provides a library for HTML entity encoding and decoding.")
(license (list license:asl2.0 license:expat license:mpl2.0))))

+(define-public rust-html-escape-0.2
+ (package
+ (name "rust-html-escape")
+ (version "0.2.9")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (crate-uri "html-escape" version))
+ (file-name
+ (string-append name "-" version ".tar.gz"))
+ (sha256
+ (base32
+ "1dxw9lpckrqzzqgbkw64ckbajr4b7xxnjdn8adgzqf2mm40shvl1"))))
+ (build-system cargo-build-system)
+ (arguments
+ `(#:cargo-inputs
+ (("rust-utf8-width" ,rust-utf8-width-0.1))
+ #:cargo-development-inputs
+ (("rust-bencher" ,rust-bencher-0.1))))
+ (home-page "https://magiclen.org/html-escape")
+ (synopsis "Library for encoding and escaping special characters in HTML")
+ (description
+ "This package provides a library for encoding and escaping special
+characters in HTML, decoding and unescaping HTML entities as well.")
+ (license license:expat)))
+
(define-public rust-http-0.2
(package
(name "rust-http")
--
2.32.0
P
P
Pierre Langlois wrote on 9 Aug 2021 01:33
[PATCH 01/31] gnu: rust-chunked-transfer: Update to 1.4.0.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20210808233354.6745-1-pierre.langlois@gmx.com
* gnu/packages/crates-io.scm (rust-chunked-transfer-1): New variable.
(rust-chunked-transfer-0.3): Inherit from rust-chunked-transfer-1.
---
gnu/packages/crates-io.scm | 24 ++++++++++++++++++++----
1 file changed, 20 insertions(+), 4 deletions(-)

Toggle diff (58 lines)
diff --git a/gnu/packages/crates-io.scm b/gnu/packages/crates-io.scm
index 24cc772ba7..01a0dff724 100644
--- a/gnu/packages/crates-io.scm
+++ b/gnu/packages/crates-io.scm
@@ -18,6 +18,7 @@
;;; Copyright © 2021 Antero Mejr <antero@kodmin.com>
;;; Copyright © 2021 Maxim Cournoyer <maxim.cournoyer@gmail.com>
;;; Copyright © 2021 Vinicius Monego <monego@posteo.net>
+;;; Copyright © 2021 Pierre Langlois <pierre.langlois@gmx.com>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -7825,19 +7826,21 @@ Python arrow.humanize.")
TimeZone trait for @code{rust-chrono}.")
(license (list license:expat license:asl2.0))))

-(define-public rust-chunked-transfer-0.3
+(define-public rust-chunked-transfer-1
(package
(name "rust-chunked-transfer")
- (version "0.3.1")
+ (version "1.4.0")
(source
(origin
(method url-fetch)
(uri (crate-uri "chunked_transfer" version))
(file-name (string-append name "-" version ".tar.gz"))
(sha256
- (base32 "11yghnd24w0i9p8g368c3pg7qh9nfz7kgri6pywja9pnmakj13a9"))))
+ (base32 "0bkdlsrszfcscw3j6yhs7kj6jbp8id47jjk6h9k58px47na5gy7z"))))
(build-system cargo-build-system)
- (arguments `(#:skip-build? #t))
+ (arguments
+ `(#:cargo-development-inputs
+ (("rust-criterion" ,rust-criterion-0.3))))
(home-page "https://github.com/frewsxcv/rust-chunked-transfer")
(synopsis "Encoder and decoder for HTTP chunked transfer coding")
(description
@@ -7845,6 +7848,19 @@ TimeZone trait for @code{rust-chrono}.")
coding.")
(license license:asl2.0)))

+(define-public rust-chunked-transfer-0.3
+ (package (inherit rust-chunked-transfer-1)
+ (name "rust-chunked-transfer")
+ (version "0.3.1")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (crate-uri "chunked_transfer" version))
+ (file-name (string-append name "-" version ".tar.gz"))
+ (sha256
+ (base32 "11yghnd24w0i9p8g368c3pg7qh9nfz7kgri6pywja9pnmakj13a9"))))
+ (arguments `(#:skip-build? #t))))
+
(define-public rust-ci-info-0.3
(package
(name "rust-ci-info")
--
2.32.0
P
P
Pierre Langlois wrote on 9 Aug 2021 01:33
[PATCH 05/31] gnu: rust-tiny-http: Update to 0.8.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20210808233354.6745-5-pierre.langlois@gmx.com
* gnu/packages/crates-io.scm (rust-tiny-http-0.8): New variable.
(rust-tiny-http-0.6): Inherit from rust-tiny-http-0.8.
---
gnu/packages/crates-io.scm | 39 ++++++++++++++++++++++++++++++++------
1 file changed, 33 insertions(+), 6 deletions(-)

Toggle diff (58 lines)
diff --git a/gnu/packages/crates-io.scm b/gnu/packages/crates-io.scm
index 2274e4f12b..bb31356ceb 100644
--- a/gnu/packages/crates-io.scm
+++ b/gnu/packages/crates-io.scm
@@ -48810,8 +48810,39 @@ closures after a delay or at a given timestamp.")
manipulation in Rust.")
(license license:expat)))

-(define-public rust-tiny-http-0.6
+(define-public rust-tiny-http-0.8
(package
+ (name "rust-tiny-http")
+ (version "0.8.2")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (crate-uri "tiny_http" version))
+ (file-name
+ (string-append name "-" version ".tar.gz"))
+ (sha256
+ (base32
+ "0fcdwpb2ghk671qjjrk6048hs3yp7f681hxpr68gamk00181prcw"))))
+ (build-system cargo-build-system)
+ (arguments
+ `(#:cargo-inputs
+ (("rust-ascii" ,rust-ascii-1)
+ ("rust-chrono" ,rust-chrono-0.4)
+ ("rust-chunked-transfer" ,rust-chunked-transfer-1)
+ ("rust-log" ,rust-log-0.4)
+ ("rust-openssl" ,rust-openssl-0.10)
+ ("rust-url" ,rust-url-2))
+ #:cargo-development-inputs
+ (("rust-fdlimit" ,rust-fdlimit-0.1)
+ ("rust-rustc-serialize" ,rust-rustc-serialize-0.3)
+ ("rust-sha1" ,rust-sha1-0.6))))
+ (home-page "https://github.com/tiny-http/tiny-http")
+ (synopsis "Low level HTTP server library")
+ (description "This package provides a low level HTTP server library.")
+ (license license:asl2.0)))
+
+(define-public rust-tiny-http-0.6
+ (package (inherit rust-tiny-http-0.8)
(name "rust-tiny-http")
(version "0.6.2")
(source
@@ -48830,11 +48861,7 @@ manipulation in Rust.")
("rust-chunked-transfer" ,rust-chunked-transfer-0.3)
("rust-log" ,rust-log-0.4)
("rust-openssl" ,rust-openssl-0.10)
- ("rust-url" ,rust-url-1))))
- (home-page "https://github.com/tiny-http/tiny-http")
- (synopsis "Low level HTTP server library")
- (description "This package provides a low level HTTP server library.")
- (license license:asl2.0)))
+ ("rust-url" ,rust-url-1))))))

(define-public rust-tiny-keccak-2
(package
--
2.32.0
P
P
Pierre Langlois wrote on 9 Aug 2021 01:33
[PATCH 06/31] gnu: rust-spin: Update to 0.7.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20210808233354.6745-6-pierre.langlois@gmx.com
* gnu/packages/crates-io.scm (rust-spin-0.7): New variable.
(rust-spin-0.5): Inherit from rust-spin-0.7.
---
gnu/packages/crates-io.scm | 24 +++++++++++++++++++++---
1 file changed, 21 insertions(+), 3 deletions(-)

Toggle diff (52 lines)
diff --git a/gnu/packages/crates-io.scm b/gnu/packages/crates-io.scm
index bb31356ceb..b551e6921d 100644
--- a/gnu/packages/crates-io.scm
+++ b/gnu/packages/crates-io.scm
@@ -44496,10 +44496,10 @@ track of where each new file and line starts.")
"An RSpec inspired minimal testing framework for Rust.")
(license license:expat)))

-(define-public rust-spin-0.5
+(define-public rust-spin-0.7
(package
(name "rust-spin")
- (version "0.5.2")
+ (version "0.7.1")
(source
(origin
(method url-fetch)
@@ -44507,8 +44507,11 @@ track of where each new file and line starts.")
(file-name (string-append name "-" version ".crate"))
(sha256
(base32
- "0b84m6dbzrwf2kxylnw82d3dr8w06av7rfkr8s85fb5f43rwyqvf"))))
+ "0qjips9f6fsvkyd7wj3a4gzaqknn2q4kkb19957pl86im56pna0k"))))
(build-system cargo-build-system)
+ (arguments
+ `(#:cargo-inputs
+ (("rust-lock-api" ,rust-lock-api-0.4))))
(home-page "https://github.com/mvdnes/spin-rs")
(synopsis "Synchronization primitives based on spinning")
(description "This crate provides synchronization primitives based on
@@ -44516,6 +44519,21 @@ spinning. They may contain data, are usable without @code{std},and static
initializers are available.")
(license license:expat)))

+(define-public rust-spin-0.5
+ (package
+ (inherit rust-spin-0.7)
+ (name "rust-spin")
+ (version "0.5.2")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (crate-uri "spin" version))
+ (file-name (string-append name "-" version ".crate"))
+ (sha256
+ (base32
+ "0b84m6dbzrwf2kxylnw82d3dr8w06av7rfkr8s85fb5f43rwyqvf"))))
+ (arguments '(#:cargo-inputs ()))))
+
(define-public rust-spin-0.4
(package
(inherit rust-spin-0.5)
--
2.32.0
P
P
Pierre Langlois wrote on 9 Aug 2021 01:33
[PATCH 04/31] gnu: Add rust-smallbitvec.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20210808233354.6745-4-pierre.langlois@gmx.com
* gnu/packages/crates-io.scm (rust-smallbitvec-2): New variable.
---
gnu/packages/crates-io.scm | 23 +++++++++++++++++++++++
1 file changed, 23 insertions(+)

Toggle diff (34 lines)
diff --git a/gnu/packages/crates-io.scm b/gnu/packages/crates-io.scm
index 4a9db4943b..2274e4f12b 100644
--- a/gnu/packages/crates-io.scm
+++ b/gnu/packages/crates-io.scm
@@ -44087,6 +44087,29 @@ I/O programming.")
("rust-futures-core-preview" ,rust-futures-core-preview-0.3)
("rust-futures-io-preview" ,rust-futures-io-preview-0.3))))))

+(define-public rust-smallbitvec-2
+ (package
+ (name "rust-smallbitvec")
+ (version "2.5.1")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (crate-uri "smallbitvec" version))
+ (file-name (string-append name "-" version ".tar.gz"))
+ (sha256
+ (base32
+ "0plrbldsjpwip3afbzd8fgrnvdhizcg5z4ncfqs4q6x4qjflzkkm"))))
+ (build-system cargo-build-system)
+ (arguments
+ `(#:cargo-development-inputs
+ (("rust-bit-vec" ,rust-bit-vec-0.4)
+ ("rust-rand" ,rust-rand-0.4))))
+ (home-page "https://github.com/servo/smallbitvec")
+ (synopsis "A bit vector optimized for size and inline storage")
+ (description "This package provides a bit vector optimized for size and
+inline storage")
+ (license (list license:expat license:asl2.0))))
+
(define-public rust-smallvec-1
(package
(name "rust-smallvec")
--
2.32.0
P
P
Pierre Langlois wrote on 9 Aug 2021 01:33
[PATCH 08/31] gnu: node: Patch /usr/bin/env in node-gyp.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20210808233354.6745-8-pierre.langlois@gmx.com
* gnu/packages/node.scm (node)[arguments]: Fix /usr/bin/env shebang in
node-gyp.js.
(node-lts)[arguments]: Ditto.
---
gnu/packages/node.scm | 10 +++++++++-
1 file changed, 9 insertions(+), 1 deletion(-)

Toggle diff (35 lines)
diff --git a/gnu/packages/node.scm b/gnu/packages/node.scm
index 36c45e9c7a..522d4943d0 100644
--- a/gnu/packages/node.scm
+++ b/gnu/packages/node.scm
@@ -6,7 +6,7 @@
;;; Copyright © 2017 Mike Gerwitz <mtg@gnu.org>
;;; Copyright © 2018 Tobias Geerinckx-Rice <me@tobias.gr>
;;; Copyright © 2018, 2019, 2020, 2021 Marius Bakke <marius@gnu.org>
-;;; Copyright © 2020 Pierre Langlois <pierre.langlois@gmx.com>
+;;; Copyright © 2020, 2021 Pierre Langlois <pierre.langlois@gmx.com>
;;; Copyright © 2020 Ricardo Wurmus <rekado@elephly.net>
;;; Copyright © 2021 Simon Tournier <zimon.toutoune@gmail.com>
;;;
@@ -120,6 +120,10 @@
(("'/usr/bin/env'")
(string-append "'" (which "env") "'")))

+ ;; Fix /usr/bin/env shebang in node-gyp.
+ (substitute* "deps/npm/node_modules/node-gyp/bin/node-gyp.js"
+ (("#!/usr/bin/env") (string-append "#!" (which "env"))))
+
;; FIXME: These tests fail in the build container, but they don't
;; seem to be indicative of real problems in practice.
(for-each delete-file
@@ -661,6 +665,10 @@ source files.")
(("'/usr/bin/env'")
(string-append "'" (which "env") "'")))

+ ;; Fix /usr/bin/env shebang in node-gyp.
+ (substitute* "deps/npm/node_modules/node-gyp/bin/node-gyp.js"
+ (("#!/usr/bin/env") (string-append "#!" (which "env"))))
+
;; FIXME: These tests fail in the build container, but they don't
;; seem to be indicative of real problems in practice.
(for-each delete-file
--
2.32.0
P
P
Pierre Langlois wrote on 9 Aug 2021 01:33
[PATCH 07/31] gnu: Add tree-sitter.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20210808233354.6745-7-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm: New file.
* gnu/local.mk (GNU_SYSTEM_MODULES): Register it.
---
gnu/local.mk | 1 +
gnu/packages/tree-sitter.scm | 98 ++++++++++++++++++++++++++++++++++++
2 files changed, 99 insertions(+)
create mode 100644 gnu/packages/tree-sitter.scm

Toggle diff (116 lines)
diff --git a/gnu/local.mk b/gnu/local.mk
index 3f09b167b1..3763df0821 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -559,6 +559,7 @@ GNU_SYSTEM_MODULES = \
%D%/packages/tls.scm \
%D%/packages/tmux.scm \
%D%/packages/tor.scm \
+ %D%/packages/tree-sitter.scm \
%D%/packages/tv.scm \
%D%/packages/uglifyjs.scm \
%D%/packages/uml.scm \
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
new file mode 100644
index 0000000000..2c749dc0a8
--- /dev/null
+++ b/gnu/packages/tree-sitter.scm
@@ -0,0 +1,98 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2021 Pierre Langlois <pierre.langlois@gmx.com>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu packages tree-sitter)
+ #:use-module ((guix licenses) #:prefix license:)
+ #:use-module (guix build-system cargo)
+ #:use-module (guix git-download)
+ #:use-module (guix packages)
+ #:use-module (gnu packages crates-io)
+ #:use-module (gnu packages crates-graphics))
+
+(define-public tree-sitter
+ (package
+ (name "tree-sitter")
+ (version "0.19.5")
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "1qmb0sva28zv6r3c3j7xs9pc8bpwwhkb9vxxndw2zbdn9wkvmbmn"))))
+ (build-system cargo-build-system)
+ (arguments
+ `(;; Running test requires downloading fixtures, see the
+ ;; script/fetch-fixtures script.
+ #:tests? #f
+ ;; FIXME: Installing the sources for the tree-sitter Rust bindings
+ ;; doesn't work out of the box due to tree-sitter having multiple
+ ;; Rust packages in the same repository (bindings and CLI).
+ #:install-source? #f
+ #:cargo-inputs
+ (("rust-ansi-term" ,rust-ansi-term-0.12)
+ ("rust-atty" ,rust-atty-0.2)
+ ("rust-clap" ,rust-clap-2)
+ ("rust-difference" ,rust-difference-2)
+ ("rust-dirs" ,rust-dirs-3)
+ ("rust-html-escape" ,rust-html-escape-0.2)
+ ("rust-libloading" ,rust-libloading-0.7)
+ ("rust-smallbitvec" ,rust-smallbitvec-2)
+ ("rust-spin" ,rust-spin-0.7)
+ ("rust-tiny-http" ,rust-tiny-http-0.8)
+ ("rust-walkdir" ,rust-walkdir-2)
+ ("rust-webbrowser" ,rust-webbrowser-0.5)
+ ("rust-which" ,rust-which-4))
+ #:phases
+ (modify-phases %standard-phases
+ (add-after 'unpack 'delete-cargo.lock
+ (lambda _ (delete-file "Cargo.lock")))
+ (add-after 'build 'build-lib
+ (lambda _ (invoke "make")))
+ (replace 'install
+ (lambda* (#:key outputs #:allow-other-keys)
+ (let* ((out (assoc-ref outputs "out"))
+ (bin (string-append out "/bin"))
+ (lib (string-append out "/lib")))
+ (mkdir-p bin)
+ (install-file "target/release/tree-sitter" bin)
+ (setenv "PREFIX" out)
+ (invoke "make" "install")))))))
+ (home-page "https://tree-sitter.github.io/tree-sitter/")
+ (synopsis "Incremental parsing system for programming tools")
+ (description "Tree-sitter is a parser generator tool and an incremental
+parsing library. It can build a concrete syntax tree for a source file and
+efficiently update the syntax tree as the source file is edited.
+
+Tree-sitter aims to be:
+
+@enumerate
+@item General enough to parse any programming language.
+@item Fast enough to parse on every keystroke in a text editor.
+@item Robust enough to provide useful results even in the presence of syntax
+errors.
+@item Dependency-free so that the runtime library (which is written in pure C)
+can be embedded in any application.
+@end enumerate
+
+This package includes the @command{tree-sitter} tool as well as the runtime
+library.")
+ (license license:expat)))
--
2.32.0
P
P
Pierre Langlois wrote on 9 Aug 2021 01:33
[PATCH 10/31] gnu: Add node-nan.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20210808233354.6745-10-pierre.langlois@gmx.com
---
gnu/packages/node-xyz.scm | 30 ++++++++++++++++++++++++++++++
1 file changed, 30 insertions(+)

Toggle diff (48 lines)
diff --git a/gnu/packages/node-xyz.scm b/gnu/packages/node-xyz.scm
index cbb68f77d6..eb4b59d1ab 100644
--- a/gnu/packages/node-xyz.scm
+++ b/gnu/packages/node-xyz.scm
@@ -3,6 +3,7 @@
;;; Copyright © 2020 Giacomo Leidi <goodoldpaul@autistici.org>
;;; Copyright © 2021 Noisytoot <noisytoot@disroot.org>
;;; Copyright © 2021 Charles <charles.b.jackson@protonmail.com>
+;;; Copyright © 2021 Pierre Langlois <pierre.langlois@gmx.com>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -171,6 +172,35 @@ user-land JavaScript.")
random number generator.")
(license license:bsd-3)))

+(define-public node-nan
+ (package
+ (name "node-nan")
+ (version "2.15.0")
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/nodejs/nan")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "18xslh9va5ld872scrp5y4251ax9s3c6qh0lnl1200lpzbsxy7yd"))))
+ (build-system node-build-system)
+ (arguments
+ '(#:tests? #f ; FIXME: tests depend on node-tap
+ #:phases
+ (modify-phases %standard-phases
+ ;; The default configure phase fails due to tap being missing, as we do
+ ;; not have tap packaged yet. It is used only for tests. This package
+ ;; still works as a dependency of node-glob and node-inflight.
+ (delete 'configure))))
+ (home-page "https://github.com/nodejs/nan")
+ (synopsis "Native Abstractions for Node.js")
+ (description "This package provides header files filled with macros and
+utilities for making add-on development for Node.js easier across versions.")
+ (license license:bsd-3)))
+
(define-public node-oop
;; No releases, last commit was February 2013.
(let ((commit "f9d87cda0958886955c14a0a716e57021ed295dc")
--
2.32.0
P
P
Pierre Langlois wrote on 9 Aug 2021 01:33
[PATCH 12/31] gnu: Add tree-sitter-cpp.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20210808233354.6745-12-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-cpp): New variable.
---
gnu/packages/tree-sitter.scm | 24 ++++++++++++++++++++++++
1 file changed, 24 insertions(+)

Toggle diff (32 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index c855cba347..ac4fe5c578 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -157,3 +157,27 @@ library.")
(description
"This package provides a C grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-cpp
+ (package (inherit tree-sitter-c)
+ (name "tree-sitter-cpp")
+ (version "0.19.0")
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-cpp")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "08ywv6n80sa541rr08bqz4zyg7byvjcabp68lvxmcahjk8xzcgwk"))))
+ (native-inputs
+ `(("tree-sitter" ,tree-sitter)
+ ("tree-sitter-c" ,tree-sitter-c)
+ ("node-nan" ,node-nan)))
+ (home-page "https://github.com/tree-sitter/tree-sitter-cpp")
+ (synopsis "Tree-sitter C++ grammar")
+ (description
+ "This package provides a C++ grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.32.0
P
P
Pierre Langlois wrote on 9 Aug 2021 01:33
[PATCH 11/31] gnu: Add tree-sitter-c.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20210808233354.6745-11-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-c): New variable.
---
gnu/packages/tree-sitter.scm | 63 +++++++++++++++++++++++++++++++++++-
1 file changed, 62 insertions(+), 1 deletion(-)

Toggle diff (81 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 2c749dc0a8..c855cba347 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -19,10 +19,12 @@
(define-module (gnu packages tree-sitter)
#:use-module ((guix licenses) #:prefix license:)
#:use-module (guix build-system cargo)
+ #:use-module (guix build-system node)
#:use-module (guix git-download)
#:use-module (guix packages)
#:use-module (gnu packages crates-io)
- #:use-module (gnu packages crates-graphics))
+ #:use-module (gnu packages crates-graphics)
+ #:use-module (gnu packages node-xyz))

(define-public tree-sitter
(package
@@ -96,3 +98,62 @@ can be embedded in any application.
This package includes the @command{tree-sitter} tool as well as the runtime
library.")
(license license:expat)))
+
+(define-public tree-sitter-c
+ (package
+ (name "tree-sitter-c")
+ (version "0.19.0")
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-c")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "1diys8yigvhm4ppbmp3a473yxjg2d5lk11y0ay7qprcz7233lakv"))))
+ (build-system node-build-system)
+ (native-inputs
+ `(("tree-sitter" ,tree-sitter)
+ ("node-nan" ,node-nan)))
+ (arguments
+ `(#:phases
+ (modify-phases %standard-phases
+ ;; tree-sitter-cli is listed as a Node.js dependency, however the
+ ;; node tree-sitter-cli package is just a wrapper which downloads a
+ ;; tree-sitter binary, see
+ ;; https://github.com/tree-sitter/tree-sitter/tree/master/cli/npm
+ ;; Instead we remove it as a dependency so that we can use our own
+ ;; tree-sitter package.
+ (add-after 'unpack 'fix-configure
+ (lambda _
+ (with-atomic-file-replacement "package.json"
+ (lambda (in out)
+ (use-modules ((guix build json)))
+ (let ((package-meta (read-json in)))
+ (assoc-remove! (assoc-ref package-meta "devDependencies")
+ "tree-sitter-cli")
+ (write-json package-meta out))))))
+ (add-before 'build 'set-cc
+ (lambda _
+ (setenv "CC" "gcc")))
+ (add-before 'build 'make-files-writable
+ (lambda _
+ (for-each make-file-writable (find-files "src" ".*"))))
+ (add-after 'install 'install-native-lib
+ (lambda* (#:key outputs #:allow-other-keys)
+ (let ((lib (string-append (assoc-ref outputs "out")
+ "/lib/tree-sitter")))
+ ;; Invoking `tree-sitter test' loads the grammar which
+ ;; compiles it to a .so binary that we install.
+ (invoke "tree-sitter" "test")
+ (mkdir-p lib)
+ (copy-recursively (string-append (getenv "HOME")
+ "/.tree-sitter/bin")
+ lib)))))))
+ (home-page "https://github.com/tree-sitter/tree-sitter-c")
+ (synopsis "Tree-sitter C grammar")
+ (description
+ "This package provides a C grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.32.0
P
P
Pierre Langlois wrote on 9 Aug 2021 01:33
[PATCH 13/31] gnu: Add tree-sitter-css.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20210808233354.6745-13-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-css): New variable.
---
gnu/packages/tree-sitter.scm | 20 ++++++++++++++++++++
1 file changed, 20 insertions(+)

Toggle diff (28 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index ac4fe5c578..05929a9062 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -181,3 +181,23 @@ library.")
(description
"This package provides a C++ grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-css
+ (package (inherit tree-sitter-c)
+ (name "tree-sitter-css")
+ (version "0.19.0")
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-css")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "014jrlgi7zfza9g38hsr4vlbi8964i5p7iglaih6qmzaiml7bja2"))))
+ (home-page "https://github.com/tree-sitter/tree-sitter-css")
+ (synopsis "Tree-sitter CSS grammar")
+ (description
+ "This package provides a CSS grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.32.0
P
P
Pierre Langlois wrote on 9 Aug 2021 01:33
[PATCH 09/31] guix: node-build-system: Support compiling addons with node-gyp.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20210808233354.6745-9-pierre.langlois@gmx.com
* gnu/packages/node.scm (node-headers): New function.
* guix/build-system/node.scm (python): New function.
(node-headers): New function.
(lower): Add node-headers and python to build inputs.
* guix/build/node-build-system.scm (configure-gyp): New function.
(%standard-phases): Add 'configure-gyp after 'configure.
---
gnu/packages/node.scm | 19 ++++++++++++++++++-
guix/build-system/node.scm | 16 ++++++++++++++++
guix/build/node-build-system.scm | 15 +++++++++++++++
3 files changed, 49 insertions(+), 1 deletion(-)

Toggle diff (111 lines)
diff --git a/gnu/packages/node.scm b/gnu/packages/node.scm
index 522d4943d0..d0f7c5cdb7 100644
--- a/gnu/packages/node.scm
+++ b/gnu/packages/node.scm
@@ -48,7 +48,24 @@
#:use-module (gnu packages pkg-config)
#:use-module (gnu packages python)
#:use-module (gnu packages tls)
- #:use-module (gnu packages web))
+ #:use-module (gnu packages web)
+ #:use-module (ice-9 match)
+ #:export (node-headers))
+
+(define (node-headers node)
+ "Return an <origin> object for a tarball with headers for the given node
+package version."
+ (let* ((version (package-version node))
+ (hash (match version
+ ("10.24.0"
+ "0h37zjwcpxjdqcxqjfj5zp1n5zjxaa0g8lsy83955afg5cca8p0n")
+ ("14.16.0"
+ "1vpdgq7kcw1a0w90lpvbvxbrc0n3pwjrs3sm42pjj7560clvji2b"))))
+ (origin
+ (method url-fetch)
+ (uri (string-append "https://nodejs.org/dist/v" version
+ "/node-v" version "-headers.tar.gz"))
+ (sha256 (base32 hash)))))

(define-public node
(package
diff --git a/guix/build-system/node.scm b/guix/build-system/node.scm
index 98f63f87ef..7828582a9a 100644
--- a/guix/build-system/node.scm
+++ b/guix/build-system/node.scm
@@ -1,6 +1,7 @@
;;; GNU Guix --- Functional package management for GNU
;;; Copyright © 2016 Jelle Licht <jlicht@fsfe.org>
;;; Copyright © 2019 Timothy Sample <samplet@ngyro.com>
+;;; Copyright © 2021 Pierre Langlois <pierre.langlois@gmx.com>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -41,6 +42,19 @@
(let ((node (resolve-interface '(gnu packages node))))
(module-ref node 'node-lts)))

+(define (python)
+ "Return the python package."
+ ;; Lazily resolve the binding to avoid a circular dependency.
+ (let ((module (resolve-interface '(gnu packages python))))
+ (module-ref module 'python-wrapper)))
+
+(define (node-headers node)
+ "Return a tarball with headers for the given node, needed for packages that
+need to build native bindings using node-gyp."
+ ;; Lazily resolve the binding to avoid a circular dependency.
+ (let ((module (resolve-interface '(gnu packages node))))
+ ((module-ref module 'node-headers) node)))
+
(define* (lower name
#:key source inputs native-inputs outputs system target
(node (default-node))
@@ -62,6 +76,8 @@
;; Keep the standard inputs of 'gnu-build-system'.
,@(standard-packages)))
(build-inputs `(("node" ,node)
+ ("node-headers" ,(node-headers node))
+ ("python" ,(python))
,@native-inputs))
(outputs outputs)
(build node-build)
diff --git a/guix/build/node-build-system.scm b/guix/build/node-build-system.scm
index 70a367618e..2993c49b2b 100644
--- a/guix/build/node-build-system.scm
+++ b/guix/build/node-build-system.scm
@@ -2,6 +2,7 @@
;;; Copyright © 2015 David Thompson <davet@gnu.org>
;;; Copyright © 2016, 2020 Jelle Licht <jlicht@fsfe.org>
;;; Copyright © 2019, 2021 Timothy Sample <samplet@ngyro.com>
+;;; Copyright © 2021 Pierre Langlois <pierre.langlois@gmx.com>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -101,6 +102,19 @@
(invoke npm "--offline" "--ignore-scripts" "install")
#t))

+(define* (configure-gyp #:key inputs #:allow-other-keys)
+ "Run 'node-gyp configure' if we see a 'binding.gyp' file.
+
+By default, 'node-gyp' will try to download node headers from the internet, we
+prevent this with the '--tarball' flag."
+ (let ((node-gyp (string-append (assoc-ref inputs "node")
+ "/lib/node_modules/npm/node_modules/node-gyp"
+ "/bin/node-gyp.js")))
+ (if (file-exists? "binding.gyp")
+ (invoke node-gyp "--tarball" (assoc-ref inputs "node-headers")
+ "configure"))
+ #t))
+
(define* (build #:key inputs #:allow-other-keys)
(let ((package-meta (call-with-input-file "package.json" read-json)))
(if (and=> (assoc-ref package-meta "scripts")
@@ -147,6 +161,7 @@
(add-after 'unpack 'set-home set-home)
(add-before 'configure 'patch-dependencies patch-dependencies)
(replace 'configure configure)
+ (add-after 'configure 'configure-gyp configure-gyp)
(replace 'build build)
(replace 'check check)
(add-before 'install 'repack repack)
--
2.32.0
P
P
Pierre Langlois wrote on 9 Aug 2021 01:33
[PATCH 14/31] gnu: Add tree-sitter-go.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20210808233354.6745-14-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-go): New variable.
---
gnu/packages/tree-sitter.scm | 30 ++++++++++++++++++++++++++++++
1 file changed, 30 insertions(+)

Toggle diff (45 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 05929a9062..f163efd7a3 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -22,6 +22,7 @@
#:use-module (guix build-system node)
#:use-module (guix git-download)
#:use-module (guix packages)
+ #:use-module (guix utils)
#:use-module (gnu packages crates-io)
#:use-module (gnu packages crates-graphics)
#:use-module (gnu packages node-xyz))
@@ -201,3 +202,32 @@ library.")
(description
"This package provides a CSS grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-go
+ (package (inherit tree-sitter-c)
+ (name "tree-sitter-go")
+ (version "0.19.1")
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-go")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "0nxs47vd2fc2fr0qlxq496y852rwg39flhg334s7dlyq7d3lcx4x"))))
+ (arguments
+ (substitute-keyword-arguments (package-arguments tree-sitter-c)
+ ((#:phases phases)
+ `(modify-phases ,phases
+ ;; The parse-example script tries to clone git repositories.
+ (add-after 'unpack 'remove-parse-example
+ (lambda _
+ (substitute* "package.json"
+ ((" && script\\/parse-examples") ""))))))))
+ (home-page "https://github.com/tree-sitter/tree-sitter-go")
+ (synopsis "Tree-sitter Go grammar")
+ (description
+ "This package provides a Golang grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.32.0
P
P
Pierre Langlois wrote on 9 Aug 2021 01:33
[PATCH 15/31] gnu: Add tree-sitter-html.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20210808233354.6745-15-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-html): New variable.
---
gnu/packages/tree-sitter.scm | 20 ++++++++++++++++++++
1 file changed, 20 insertions(+)

Toggle diff (28 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index f163efd7a3..c2ca5a842a 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -231,3 +231,23 @@ library.")
(description
"This package provides a Golang grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-html
+ (package (inherit tree-sitter-c)
+ (name "tree-sitter-html")
+ (version "0.19.0")
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-html")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "1hg7vbcy7bir6b8x11v0a4x0glvqnsqc3i2ixiarbxmycbgl3axy"))))
+ (home-page "https://github.com/tree-sitter/tree-sitter-html")
+ (synopsis "Tree-sitter HTML grammar")
+ (description
+ "This package provides a HTML grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.32.0
P
P
Pierre Langlois wrote on 9 Aug 2021 01:33
[PATCH 16/31] gnu: Add tree-sitter-java.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20210808233354.6745-16-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-java): New variable.
---
gnu/packages/tree-sitter.scm | 29 +++++++++++++++++++++++++++++
1 file changed, 29 insertions(+)

Toggle diff (37 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index c2ca5a842a..e671c3fbf9 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -251,3 +251,32 @@ library.")
(description
"This package provides a HTML grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-java
+ (package (inherit tree-sitter-c)
+ (name "tree-sitter-java")
+ (version "0.19.1")
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-java")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "07zw9ygb45hnvlx9qlz7rlz8hc3byjy03d24v72i5iyhpiiwlhvl"))))
+ (arguments
+ (substitute-keyword-arguments (package-arguments tree-sitter-c)
+ ((#:phases phases)
+ `(modify-phases ,phases
+ ;; The parse-example script tries to clone git repositories.
+ (add-after 'unpack 'remove-parse-example
+ (lambda _
+ (substitute* "package.json"
+ ((" && script\\/parse-examples") ""))))))))
+ (home-page "https://github.com/tree-sitter/tree-sitter-java")
+ (synopsis "Tree-sitter Java grammar")
+ (description
+ "This package provides a Java grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.32.0
P
P
Pierre Langlois wrote on 9 Aug 2021 01:33
[PATCH 17/31] gnu: Add tree-sitter-javascript.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20210808233354.6745-17-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-javascript): New variable.
---
gnu/packages/tree-sitter.scm | 30 ++++++++++++++++++++++++++++++
1 file changed, 30 insertions(+)

Toggle diff (38 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index e671c3fbf9..e59a44c521 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -280,3 +280,33 @@ library.")
(description
"This package provides a Java grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-javascript
+ (package (inherit tree-sitter-c)
+ (name "tree-sitter-javascript")
+ (version "0.19.0")
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-javascript")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "063va0s727yzhy1cz04fipzwwbq4af6fhgka6g970hk9yf7ggmnj"))))
+ (arguments
+ (substitute-keyword-arguments (package-arguments tree-sitter-c)
+ ((#:phases phases)
+ `(modify-phases ,phases
+ ;; The parse-example script tries to clone git repositories.
+ (add-after 'unpack 'remove-parse-example
+ (lambda _
+ (substitute* "package.json"
+ ((" && script\\/parse-examples") ""))))))))
+ (home-page "https://github.com/tree-sitter/tree-sitter-javascript")
+ (synopsis "Tree-sitter Javascript grammar")
+ (description
+ "This package provides a Javascript grammar for the Tree-sitter
+library.")
+ (license license:expat)))
--
2.32.0
P
P
Pierre Langlois wrote on 9 Aug 2021 01:33
[PATCH 18/31] gnu: Add tree-sitter-json.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20210808233354.6745-18-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-json): New variable.
---
gnu/packages/tree-sitter.scm | 20 ++++++++++++++++++++
1 file changed, 20 insertions(+)

Toggle diff (28 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index e59a44c521..d6214de2ec 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -310,3 +310,23 @@ library.")
"This package provides a Javascript grammar for the Tree-sitter
library.")
(license license:expat)))
+
+(define-public tree-sitter-json
+ (package (inherit tree-sitter-c)
+ (name "tree-sitter-json")
+ (version "0.19.0")
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-json")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "06pjh31bv9ja9hlnykk257a6zh8bsxg2fqa54al7qk1r4n9ksnff"))))
+ (home-page "https://github.com/tree-sitter/tree-sitter-json")
+ (synopsis "Tree-sitter JSON grammar")
+ (description
+ "This package provides a JSON grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.32.0
P
P
Pierre Langlois wrote on 9 Aug 2021 01:33
[PATCH 19/31] gnu: Add tree-sitter-julia.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20210808233354.6745-19-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-julia): New variable.
---
gnu/packages/tree-sitter.scm | 29 +++++++++++++++++++++++++++++
1 file changed, 29 insertions(+)

Toggle diff (37 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index d6214de2ec..aa72348347 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -330,3 +330,32 @@ library.")
(description
"This package provides a JSON grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-julia
+ (package (inherit tree-sitter-c)
+ (name "tree-sitter-julia")
+ (version "0.19.0")
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-julia")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "1pbnmvhy2gq4vg1b0sjzmjm4s2gsgdjh7h01yj8qrrqbcl29c463"))))
+ (arguments
+ (substitute-keyword-arguments (package-arguments tree-sitter-c)
+ ((#:phases phases)
+ `(modify-phases ,phases
+ ;; The parse-example script tries to clone git repositories.
+ (add-after 'unpack 'remove-parse-example
+ (lambda _
+ (substitute* "package.json"
+ ((" && script\\/parse-examples") ""))))))))
+ (home-page "https://github.com/tree-sitter/tree-sitter-julia")
+ (synopsis "Tree-sitter Julia grammar")
+ (description
+ "This package provides a Julia grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.32.0
P
P
Pierre Langlois wrote on 9 Aug 2021 01:33
[PATCH 20/31] gnu: Add tree-sitter-php.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20210808233354.6745-20-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-php): New variable.
---
gnu/packages/tree-sitter.scm | 29 +++++++++++++++++++++++++++++
1 file changed, 29 insertions(+)

Toggle diff (37 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index aa72348347..9f1a30f0cc 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -359,3 +359,32 @@ library.")
(description
"This package provides a Julia grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-php
+ (package (inherit tree-sitter-c)
+ (name "tree-sitter-php")
+ (version "0.19.0")
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-php")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "17cmybgpprw7w9d2v7lmc6zmr90d70g0jqq279gzg0mpwfzla53s"))))
+ (arguments
+ (substitute-keyword-arguments (package-arguments tree-sitter-c)
+ ((#:phases phases)
+ `(modify-phases ,phases
+ ;; The parse-example script tries to clone git repositories.
+ (add-after 'unpack 'remove-parse-example
+ (lambda _
+ (substitute* "package.json"
+ ((" && script\\/parse-examples") ""))))))))
+ (home-page "https://github.com/tree-sitter/tree-sitter-php")
+ (synopsis "Tree-sitter PHP grammar")
+ (description
+ "This package provides a PHP grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.32.0
P
P
Pierre Langlois wrote on 9 Aug 2021 01:43
[PATCH 21/31] gnu: Add tree-sitter-python.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20210808234358.7241-1-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-python): New variable.
---
gnu/packages/tree-sitter.scm | 29 +++++++++++++++++++++++++++++
1 file changed, 29 insertions(+)

Toggle diff (37 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 9f1a30f0cc..d7c3468035 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -388,3 +388,32 @@ library.")
(description
"This package provides a PHP grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-python
+ (package (inherit tree-sitter-c)
+ (name "tree-sitter-python")
+ (version "0.19.0")
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-python")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "04b85qxqs64x6nhbpcgrzkbilxaiwvr9yd3h065rynv7rsdg0hii"))))
+ (arguments
+ (substitute-keyword-arguments (package-arguments tree-sitter-c)
+ ((#:phases phases)
+ `(modify-phases ,phases
+ ;; The parse-example script tries to clone git repositories.
+ (add-after 'unpack 'remove-parse-example
+ (lambda _
+ (substitute* "package.json"
+ ((" && script\\/parse-examples") ""))))))))
+ (home-page "https://github.com/tree-sitter/tree-sitter-python")
+ (synopsis "Tree-sitter Python grammar")
+ (description
+ "This package provides a Python grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.32.0
P
P
Pierre Langlois wrote on 9 Aug 2021 01:43
[PATCH 22/31] gnu: Add tree-sitter-rust.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20210808234358.7241-2-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-rust): New variable.
---
gnu/packages/tree-sitter.scm | 25 +++++++++++++++++++++++++
1 file changed, 25 insertions(+)

Toggle diff (40 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index d7c3468035..6c2b2e7577 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -23,6 +23,7 @@
#:use-module (guix git-download)
#:use-module (guix packages)
#:use-module (guix utils)
+ #:use-module (gnu packages algebra)
#:use-module (gnu packages crates-io)
#:use-module (gnu packages crates-graphics)
#:use-module (gnu packages node-xyz))
@@ -417,3 +418,27 @@ library.")
(description
"This package provides a Python grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-rust
+ (package (inherit tree-sitter-c)
+ (name "tree-sitter-rust")
+ (version "0.19.1")
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-rust")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "118vkhv7n3sw8y9pi0987cgdcd74sjqwviijw01mhnk3bkyczi3l"))))
+ (native-inputs
+ `(("bc" ,bc)
+ ("tree-sitter" ,tree-sitter)
+ ("node-nan" ,node-nan)))
+ (home-page "https://github.com/tree-sitter/tree-sitter-rust")
+ (synopsis "Tree-sitter Rust grammar")
+ (description
+ "This package provides a Rust grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.32.0
P
P
Pierre Langlois wrote on 9 Aug 2021 01:43
[PATCH 24/31] gnu: Add rust-bindgen@0.56.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20210808234358.7241-4-pierre.langlois@gmx.com
* gnu/packages/crates-io.scm (rust-bindgen-0.56): New variable.
---
gnu/packages/crates-io.scm | 37 +++++++++++++++++++++++++++++++++++++
1 file changed, 37 insertions(+)

Toggle diff (48 lines)
diff --git a/gnu/packages/crates-io.scm b/gnu/packages/crates-io.scm
index b551e6921d..44f8885adb 100644
--- a/gnu/packages/crates-io.scm
+++ b/gnu/packages/crates-io.scm
@@ -4482,6 +4482,43 @@ bindings to C and C++ libraries.")
("rust-shlex" ,rust-shlex-0.1)
("rust-which" ,rust-which-3))))))

+(define-public rust-bindgen-0.56
+ (package
+ (inherit rust-bindgen-0.57)
+ (name "rust-bindgen")
+ (version "0.56.0")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (crate-uri "bindgen" version))
+ (file-name (string-append name "-" version ".tar.gz"))
+ (sha256
+ (base32
+ "0fajmgk2064ca1z9iq1jjkji63qwwz38z3d67kv6xdy0xgdpk8rd"))))
+ (arguments
+ `(#:skip-build? #t
+ #:cargo-inputs
+ (("rust-bitflags" ,rust-bitflags-1)
+ ("rust-cexpr" ,rust-cexpr-0.4)
+ ("rust-cfg-if" ,rust-cfg-if-0.1)
+ ("rust-clang-sys" ,rust-clang-sys-1)
+ ("rust-clap" ,rust-clap-2)
+ ("rust-env-logger" ,rust-env-logger-0.8)
+ ("rust-lazy-static" ,rust-lazy-static-1)
+ ("rust-lazycell" ,rust-lazycell-1)
+ ("rust-log" ,rust-log-0.4)
+ ("rust-peeking-take-while" ,rust-peeking-take-while-0.1)
+ ("rust-proc-macro2" ,rust-proc-macro2-1)
+ ("rust-quote" ,rust-quote-1)
+ ("rust-regex" ,rust-regex-1)
+ ("rust-rustc-hash" ,rust-rustc-hash-1)
+ ("rust-shlex" ,rust-shlex-0.1)
+ ("rust-which" ,rust-which-3))
+ #:cargo-development-inputs
+ (("rust-clap" ,rust-clap-2)
+ ("rust-diff" ,rust-diff-0.1)
+ ("rust-shlex" ,rust-shlex-0.1))))))
+
(define-public rust-bindgen-0.55
(package
(inherit rust-bindgen-0.57)
--
2.32.0
P
P
Pierre Langlois wrote on 9 Aug 2021 01:43
[PATCH 23/31] gnu: Add tree-sitter-typescript.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20210808234358.7241-3-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-typescript): New variable.
---
gnu/packages/tree-sitter.scm | 60 ++++++++++++++++++++++++++++++++++++
1 file changed, 60 insertions(+)

Toggle diff (68 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 6c2b2e7577..0f1946a77e 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -442,3 +442,63 @@ library.")
(description
"This package provides a Rust grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-typescript
+ (package (inherit tree-sitter-c)
+ (name "tree-sitter-typescript")
+ (version "0.19.0")
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-typescript")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "01pkmwwmbv6kxda0n6g4cfg72ldmpi9gmp11a5gygn472vhrh2xw"))))
+ (native-inputs
+ `(("tree-sitter" ,tree-sitter)
+ ("tree-sitter-javascript" ,tree-sitter-javascript)
+ ("node-nan" ,node-nan)))
+ (arguments
+ (substitute-keyword-arguments (package-arguments tree-sitter-c)
+ ((#:phases phases)
+ `(modify-phases ,phases
+ ;; The parse-example script tries to clone git repositories.
+ (add-after 'unpack 'remove-parse-example
+ (lambda _
+ (substitute* "package.json"
+ ((" && script\\/parse-examples") ""))))
+ (add-after 'build 'build-gyp
+ (lambda* (#:key inputs #:allow-other-keys)
+ (let ((node (assoc-ref inputs "node")))
+ (invoke (string-append node "/lib/node_modules/npm/node_modules"
+ "/node-gyp/bin/node-gyp.js")
+ "build"))))
+ (replace 'make-files-writable
+ (lambda _
+ (for-each make-file-writable
+ (find-files "typescript" ".*"))
+ (for-each make-file-writable
+ (find-files "tsx" ".*"))))
+ (replace 'install-native-lib
+ (lambda* (#:key outputs #:allow-other-keys)
+ (let ((lib (string-append (assoc-ref outputs "out")
+ "/lib/tree-sitter")))
+ ;; Invoking `tree-sitter test' loads the grammar which
+ ;; compiles it to a .so binary that we install.
+ (with-directory-excursion "typescript"
+ (invoke "tree-sitter" "test"))
+ (with-directory-excursion "tsx"
+ (invoke "tree-sitter" "test"))
+ (mkdir-p lib)
+ (copy-recursively (string-append (getenv "HOME")
+ "/.tree-sitter/bin")
+ lib))))))))
+ (home-page "https://github.com/tree-sitter/tree-sitter-typescript")
+ (synopsis "Tree-sitter Typescript grammar")
+ (description
+ "This package provides Typescript and TSX grammars for the Tree-sitter
+library.")
+ (license license:expat)))
--
2.32.0
P
P
Pierre Langlois wrote on 9 Aug 2021 01:43
[PATCH 26/31] gnu: rust-emacs-module: Update to 0.16.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20210808234358.7241-6-pierre.langlois@gmx.com
* gnu/packages/crates-io.scm (rust-emacs-module-0.16): New variable.
(rust-emacs-module-0.10): Inherit from rust-emacs-module-0.10.
---
gnu/packages/crates-io.scm | 25 +++++++++++++++++++++----
1 file changed, 21 insertions(+), 4 deletions(-)

Toggle diff (55 lines)
diff --git a/gnu/packages/crates-io.scm b/gnu/packages/crates-io.scm
index 44f8885adb..57a287c5d2 100644
--- a/gnu/packages/crates-io.scm
+++ b/gnu/packages/crates-io.scm
@@ -14638,10 +14638,10 @@ Emacs' support for dynamic modules.")
(description "This package provides proc macros for Emacs modules.")
(license license:bsd-3)))

-(define-public rust-emacs-module-0.10
+(define-public rust-emacs-module-0.16
(package
(name "rust-emacs-module")
- (version "0.10.0")
+ (version "0.16.2")
(source
(origin
(method url-fetch)
@@ -14650,11 +14650,11 @@ Emacs' support for dynamic modules.")
(string-append name "-" version ".tar.gz"))
(sha256
(base32
- "1gf9lz735xbkyir53dyv362drfx3nin5an5cx39kd8q8kjjwix5g"))))
+ "0s8qrh0ggjmqr04zkcf7s4ijmpd44rjcag78npnq64jv10lxvsry"))))
(build-system cargo-build-system)
(arguments
`(#:cargo-inputs
- (("rust-bindgen" ,rust-bindgen-0.48))))
+ (("rust-bindgen" ,rust-bindgen-0.56))))
(inputs
`(("clang" ,clang)))
(home-page "https://github.com/ubolonton/emacs-module-rs")
@@ -14663,6 +14663,23 @@ Emacs' support for dynamic modules.")
Emacs' support for dynamic modules.")
(license license:bsd-3)))

+(define-public rust-emacs-module-0.10
+ (package (inherit rust-emacs-module-0.16)
+ (name "rust-emacs-module")
+ (version "0.10.0")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (crate-uri "emacs_module" version))
+ (file-name
+ (string-append name "-" version ".tar.gz"))
+ (sha256
+ (base32
+ "1gf9lz735xbkyir53dyv362drfx3nin5an5cx39kd8q8kjjwix5g"))))
+ (arguments
+ `(#:cargo-inputs
+ (("rust-bindgen" ,rust-bindgen-0.48))))))
+
(define-public rust-embed-resource-1
(package
(name "rust-embed-resource")
--
2.32.0
P
P
Pierre Langlois wrote on 9 Aug 2021 01:43
[PATCH 25/31] gnu: Add rust-tree-sitter.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20210808234358.7241-5-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (rust-tree-sitter): New variable.
---
gnu/packages/tree-sitter.scm | 27 +++++++++++++++++++++++++++
1 file changed, 27 insertions(+)

Toggle diff (45 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 0f1946a77e..4817b293ce 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -20,6 +20,7 @@
#:use-module ((guix licenses) #:prefix license:)
#:use-module (guix build-system cargo)
#:use-module (guix build-system node)
+ #:use-module (guix download)
#:use-module (guix git-download)
#:use-module (guix packages)
#:use-module (guix utils)
@@ -101,6 +102,32 @@ This package includes the @command{tree-sitter} tool as well as the runtime
library.")
(license license:expat)))

+(define-public rust-tree-sitter-0.19
+ (package
+ (name "rust-tree-sitter")
+ (version "0.19.5")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (crate-uri "tree-sitter" version))
+ (file-name (string-append name "-" version ".tar.gz"))
+ (sha256
+ (base32
+ "1h6adq5kqf4izzsklch5lfxx2aisxga463zz7w44rgwnck16wwmd"))))
+ (build-system cargo-build-system)
+ (arguments
+ `(#:tests? #f ;; Running tests misinterprets comments as doc-tests.
+ #:cargo-inputs
+ (("rust-cc" ,rust-cc-1)
+ ("rust-lazy-static" ,rust-lazy-static-1)
+ ("rust-regex" ,rust-regex-1)
+ ("rust-spin" ,rust-spin-0.7))))
+ (home-page "https://tree-sitter.github.io/tree-sitter/")
+ (synopsis "Rust bindings to the Tree-sitter parsing library")
+ (description "This package provides Rust bindings to the Tree-sitter
+parsing library.")
+ (license license:expat)))
+
(define-public tree-sitter-c
(package
(name "tree-sitter-c")
--
2.32.0
P
P
Pierre Langlois wrote on 9 Aug 2021 01:43
[PATCH 27/31] gnu: rust-emacs-macros: Update to 0.17.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20210808234358.7241-7-pierre.langlois@gmx.com
* gnu/packages/crates-io.scm (rust-emacs-macros-0.17): New variable.
(rust-emacs-macros-0.11): Inherit from rust-emacs-macros-0.17.
---
gnu/packages/crates-io.scm | 32 +++++++++++++++++++++++++-------
1 file changed, 25 insertions(+), 7 deletions(-)

Toggle diff (55 lines)
diff --git a/gnu/packages/crates-io.scm b/gnu/packages/crates-io.scm
index 57a287c5d2..365cd6dcb1 100644
--- a/gnu/packages/crates-io.scm
+++ b/gnu/packages/crates-io.scm
@@ -14614,8 +14614,31 @@ signing, and verification in pure Rust.")
Emacs' support for dynamic modules.")
(license license:bsd-3)))

-(define-public rust-emacs-macros-0.11
+(define-public rust-emacs-macros-0.17
(package
+ (name "rust-emacs-macros")
+ (version "0.17.0")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (crate-uri "emacs-macros" version))
+ (file-name (string-append name "-" version ".tar.gz"))
+ (sha256
+ (base32
+ "0qg1dcn5acbirq617qq2fgg9adswif2dnr292s3qnq62wzgnyrb9"))))
+ (build-system cargo-build-system)
+ (arguments
+ `(#:cargo-inputs
+ (("rust-darling" ,rust-darling-0.10)
+ ("rust-quote" ,rust-quote-1)
+ ("rust-syn" ,rust-syn-1))))
+ (home-page "https://github.com/ubolonton/emacs-module-rs")
+ (synopsis "Proc macros for Emacs modules")
+ (description "This package provides proc macros for Emacs modules.")
+ (license license:bsd-3)))
+
+(define-public rust-emacs-macros-0.11
+ (package (inherit rust-emacs-macros-0.17)
(name "rust-emacs-macros")
(version "0.11.0")
(source
@@ -14627,16 +14650,11 @@ Emacs' support for dynamic modules.")
(sha256
(base32
"0390y8vafxdi334hhgrzvcqjq3n5ckcmvilqcfp8vajjq8irrly6"))))
- (build-system cargo-build-system)
(arguments
`(#:cargo-inputs
(("rust-darling" ,rust-darling-0.9)
("rust-quote" ,rust-quote-0.6)
- ("rust-syn" ,rust-syn-0.15))))
- (home-page "https://github.com/ubolonton/emacs-module-rs")
- (synopsis "Proc macros for Emacs modules")
- (description "This package provides proc macros for Emacs modules.")
- (license license:bsd-3)))
+ ("rust-syn" ,rust-syn-0.15))))))

(define-public rust-emacs-module-0.16
(package
--
2.32.0
P
P
Pierre Langlois wrote on 9 Aug 2021 01:43
[PATCH 28/31] gnu: rust-emacs: Update to 0.17.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20210808234358.7241-8-pierre.langlois@gmx.com
* gnu/packages/crates-io.scm (rust-emacs-0.17): New variable.
(rust-emacs-0.11): Inherit from rust-emacs-0.17.
---
gnu/packages/crates-io.scm | 38 +++++++++++++++++++++++++++++++-------
1 file changed, 31 insertions(+), 7 deletions(-)

Toggle diff (70 lines)
diff --git a/gnu/packages/crates-io.scm b/gnu/packages/crates-io.scm
index 365cd6dcb1..6b7dbe689c 100644
--- a/gnu/packages/crates-io.scm
+++ b/gnu/packages/crates-io.scm
@@ -14583,10 +14583,10 @@ signing, and verification in pure Rust.")
@code{Right} is a general purpose sum type with two cases.")
(license (list license:expat license:asl2.0))))

-(define-public rust-emacs-0.11
+(define-public rust-emacs-0.17
(package
(name "rust-emacs")
- (version "0.11.0")
+ (version "0.17.0")
(source
(origin
(method url-fetch)
@@ -14595,16 +14595,18 @@ signing, and verification in pure Rust.")
(string-append name "-" version ".tar.gz"))
(sha256
(base32
- "1xn6nr8m3p24irlmv3i7c9n95yb10qc6aikwqv99qlmhbky5x0z7"))))
+ "02f31apvz03dr0dwn5wyirdmqkk2d635iki0kmsh291xr6dysy2q"))))
(build-system cargo-build-system)
(arguments
`(#:cargo-inputs
- (("rust-ctor" ,rust-ctor-0.1)
- ("rust-emacs-macros" ,rust-emacs-macros-0.11)
- ("rust-emacs-module" ,rust-emacs-module-0.10)
+ (("rust-anyhow" ,rust-anyhow-1)
+ ("rust-ctor" ,rust-ctor-0.1)
+ ("rust-emacs-macros" ,rust-emacs-macros-0.17)
+ ("rust-emacs-module" ,rust-emacs-module-0.16)
("rust-failure" ,rust-failure-0.1)
("rust-failure-derive" ,rust-failure-derive-0.1)
- ("rust-lazy-static" ,rust-lazy-static-1))))
+ ("rust-once-cell" ,rust-once-cell-1)
+ ("rust-thiserror" ,rust-thiserror-1))))
(inputs
`(("clang" ,clang)))
(home-page "https://github.com/ubolonton/emacs-module-rs")
@@ -14614,6 +14616,28 @@ signing, and verification in pure Rust.")
Emacs' support for dynamic modules.")
(license license:bsd-3)))

+(define-public rust-emacs-0.11
+ (package (inherit rust-emacs-0.17)
+ (name "rust-emacs")
+ (version "0.11.0")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (crate-uri "emacs" version))
+ (file-name
+ (string-append name "-" version ".tar.gz"))
+ (sha256
+ (base32
+ "1xn6nr8m3p24irlmv3i7c9n95yb10qc6aikwqv99qlmhbky5x0z7"))))
+ (arguments
+ `(#:cargo-inputs
+ (("rust-ctor" ,rust-ctor-0.1)
+ ("rust-emacs-macros" ,rust-emacs-macros-0.11)
+ ("rust-emacs-module" ,rust-emacs-module-0.10)
+ ("rust-failure" ,rust-failure-0.1)
+ ("rust-failure-derive" ,rust-failure-derive-0.1)
+ ("rust-lazy-static" ,rust-lazy-static-1))))))
+
(define-public rust-emacs-macros-0.17
(package
(name "rust-emacs-macros")
--
2.32.0
P
P
Pierre Langlois wrote on 9 Aug 2021 01:43
[PATCH 29/31] gnu: Add emacs-tree-sitter-core.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20210808234358.7241-9-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-emacs-module): New local variable.
(emacs-tree-sitter-core): New variable.
---
gnu/packages/tree-sitter.scm | 79 ++++++++++++++++++++++++++++++++++++
1 file changed, 79 insertions(+)

Toggle diff (101 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 4817b293ce..a9c1714936 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -19,6 +19,7 @@
(define-module (gnu packages tree-sitter)
#:use-module ((guix licenses) #:prefix license:)
#:use-module (guix build-system cargo)
+ #:use-module (guix build-system emacs)
#:use-module (guix build-system node)
#:use-module (guix download)
#:use-module (guix git-download)
@@ -27,6 +28,7 @@
#:use-module (gnu packages algebra)
#:use-module (gnu packages crates-io)
#:use-module (gnu packages crates-graphics)
+ #:use-module (gnu packages llvm)
#:use-module (gnu packages node-xyz))

(define-public tree-sitter
@@ -529,3 +531,80 @@ library.")
"This package provides Typescript and TSX grammars for the Tree-sitter
library.")
(license license:expat)))
+
+;; Local package definition solely for building the native emacs module
+;; written in Rust.
+(define tree-sitter-emacs-module
+ (package
+ (name "tree-sitter-emacs-module")
+ (version "0.15.1")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/emacs-tree-sitter/elisp-tree-sitter")
+ (commit version)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "15y0wjnck8rbfhl0xrl71ci7clbcp11lhqil5l8ykprsdjv0c2as"))))
+ (build-system cargo-build-system)
+ (inputs
+ `(("clang" ,clang)))
+ (arguments
+ `(#:cargo-inputs
+ (("rust-anyhow" ,rust-anyhow-1)
+ ("rust-emacs" ,rust-emacs-0.17)
+ ("rust-libloading" ,rust-libloading-0.7)
+ ("rust-once-cell" ,rust-once-cell-1)
+ ("rust-tree-sitter" ,rust-tree-sitter-0.19))
+ #:phases
+ (modify-phases %standard-phases
+ (add-after 'unpack 'delete-cargo.lock
+ (lambda _ (delete-file "Cargo.lock")))
+ (add-after 'delete-cargo.lock 'do-not-fetch-from-github
+ (lambda _
+ (substitute* "Cargo.toml"
+ (("\\[patch.*") "")
+ (("git = .*") ""))))
+ (add-after 'do-not-fetch-from-github 'chdir
+ (lambda _ (chdir "core")))
+ (replace 'install
+ (lambda* (#:key outputs #:allow-other-keys)
+ (let ((lib (string-append (assoc-ref outputs "out") "/lib")))
+ (install-file "../target/release/libtsc_dyn.so" lib)))))))
+ (home-page #f)
+ (synopsis #f)
+ (description #f)
+ (license license:expat)))
+
+(define-public emacs-tree-sitter-core
+ (package
+ (name "emacs-tree-sitter-core")
+ (version (package-version tree-sitter-emacs-module))
+ (source (package-source tree-sitter-emacs-module))
+ (build-system emacs-build-system)
+ (native-inputs
+ `(("tree-sitter-emacs-module" ,tree-sitter-emacs-module)))
+ (arguments
+ `(#:phases
+ (modify-phases %standard-phases
+ (add-after 'unpack 'chdir
+ (lambda _ (chdir "core")))
+ (add-after 'install 'install-module
+ (lambda* (#:key inputs outputs #:allow-other-keys)
+ (let ((elpa (elpa-directory (assoc-ref outputs "out")))
+ (module (string-append (assoc-ref inputs "tree-sitter-emacs-module")
+ "/lib/libtsc_dyn.so")))
+ ;; Writing "LOCAL" in this file prevents the package from
+ ;; trying to download the module from the internet.
+ (call-with-output-file (string-append elpa "/DYN-VERSION")
+ (lambda (port) (display "LOCAL" port)))
+ (substitute* "tsc-dyn-get.el"
+ (("defcustom tsc-dyn-dir tsc--dir")
+ (string-append "defcustom tsc-dyn-dir \"" elpa "\"")))
+ (copy-file module (string-append elpa "/tsc-dyn.so"))))))))
+ (home-page "https://github.com/emacs-tree-sitter/elisp-tree-sitter")
+ (synopsis "Tree-sitter bindings for Emacs Lisp, core library")
+ (description "This package provides core APIs of the Emacs binding for
+Tree-sitter, an incremental parsing system.")
+ (license license:expat)))
--
2.32.0
P
P
Pierre Langlois wrote on 9 Aug 2021 01:43
[PATCH 30/31] gnu: Add emacs-tree-sitter.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20210808234358.7241-10-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (emacs-tree-sitter): New variable.
---
gnu/packages/tree-sitter.scm | 27 +++++++++++++++++++++++++++
1 file changed, 27 insertions(+)

Toggle diff (35 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index a9c1714936..22551d4482 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -608,3 +608,30 @@ library.")
(description "This package provides core APIs of the Emacs binding for
Tree-sitter, an incremental parsing system.")
(license license:expat)))
+
+(define-public emacs-tree-sitter
+ (package
+ (name "emacs-tree-sitter")
+ (version (package-version emacs-tree-sitter-core))
+ (source (package-source emacs-tree-sitter-core))
+ (build-system emacs-build-system)
+ (propagated-inputs
+ `(("emacs-tree-sitter-core" ,emacs-tree-sitter-core)))
+ (arguments
+ `(#:phases
+ (modify-phases %standard-phases
+ (add-after 'unpack 'chdir
+ (lambda _ (chdir "lisp"))))))
+ (home-page "https://github.com/emacs-tree-sitter/elisp-tree-sitter")
+ (synopsis "Tree-sitter bindings for Emacs Lisp")
+ (description "This package provides Emacs bindings for Tree-sitter, an
+incremental parsing library. It aims to be the foundation for a new breed of
+Emacs packages that understand code structurally. For example:
+
+@enumerate
+@item Faster, fine-grained code highlighting.
+@item More flexible code folding.
+@item Structural editing (like Paredit, or even better) for non-Lisp code.
+@item More informative indexing for imenu.
+@end enumerate")
+ (license license:expat)))
--
2.32.0
P
P
Pierre Langlois wrote on 9 Aug 2021 01:43
[PATCH 31/31] gnu: Add emacs-tree-sitter-langs.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20210808234358.7241-11-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (emacs-tree-sitter-langs): New variable.
---
gnu/packages/tree-sitter.scm | 79 ++++++++++++++++++++++++++++++++++++
1 file changed, 79 insertions(+)

Toggle diff (87 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 22551d4482..bfdfacf122 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -635,3 +635,82 @@ Emacs packages that understand code structurally. For example:
@item More informative indexing for imenu.
@end enumerate")
(license license:expat)))
+
+(define-public emacs-tree-sitter-langs
+ (package
+ (name "emacs-tree-sitter-langs")
+ (version "0.10.2")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/emacs-tree-sitter/tree-sitter-langs")
+ (commit version)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "0lym09jnlb7kvqkwkjbrqpc24fkpv99f8r9139sw8g537yj5f8rq"))))
+ (build-system emacs-build-system)
+ ;; FIXME: Support for some languages, such as bash, is still missing from
+ ;; the bundle.
+ (inputs
+ `(("tree-sitter-c" ,tree-sitter-c)
+ ("tree-sitter-cpp" ,tree-sitter-cpp)
+ ("tree-sitter-css" ,tree-sitter-css)
+ ("tree-sitter-go" ,tree-sitter-go)
+ ("tree-sitter-html" ,tree-sitter-html)
+ ("tree-sitter-java" ,tree-sitter-java)
+ ("tree-sitter-javascript" ,tree-sitter-javascript)
+ ("tree-sitter-json" ,tree-sitter-json)
+ ("tree-sitter-julia" ,tree-sitter-julia)
+ ("tree-sitter-php" ,tree-sitter-php)
+ ("tree-sitter-python" ,tree-sitter-python)
+ ("tree-sitter-rust" ,tree-sitter-rust)
+ ("tree-sitter-typescript" ,tree-sitter-typescript)))
+ (propagated-inputs
+ `(("emacs-tree-sitter" ,emacs-tree-sitter)))
+ (arguments
+ `(#:phases
+ (modify-phases %standard-phases
+ (add-before 'build 'fake-bundle
+ (lambda _
+ ;; The BUNDLE-VERSION file prevents the package from
+ ;; downloading libraries at load time.
+ (call-with-output-file "bin/BUNDLE-VERSION"
+ (lambda (port) (display ,version port)))))
+ (add-after 'install 'install-libs
+ (lambda* (#:key outputs inputs #:allow-other-keys)
+ (let ((elpa (elpa-directory (assoc-ref outputs "out"))))
+ (define (install-lib lang)
+ (let ((input (assoc-ref inputs (string-append "tree-sitter-" lang))))
+ (copy-file (string-append input "/lib/tree-sitter/" lang ".so")
+ (string-append elpa "/bin/" lang ".so"))))
+ (mkdir-p (string-append elpa "/bin"))
+ ;; The BUNDLE-VERSION file prevents the package from
+ ;; downloading libraries at load time.
+ (call-with-output-file (string-append elpa "/bin/BUNDLE-VERSION")
+ (lambda (port) (display ,version port)))
+ (for-each (lambda (input)
+ (use-modules (ice-9 regex))
+ (cond
+ ((string-match "^tree-sitter-(.*)$" (car input))
+ => (lambda (m)
+ (install-lib (match:substring m 1))))))
+ inputs))))
+ (add-after 'install 'install-queries
+ (lambda* (#:key outputs #:allow-other-keys)
+ (let ((elpa (elpa-directory (assoc-ref outputs "out"))))
+ (copy-recursively "queries" (string-append elpa "/queries"))))))))
+ (home-page "https://ubolonton.github.io/emacs-tree-sitter/languages/")
+ (synopsis "Language support bundle for Tree-sitter")
+ (description "This package is a convenient language bundle for
+Tree-sitter. For each supported language, this package provides:
+
+@enumerate
+@item Pre-compiled grammar binaries.
+@item An optional highlights.scm file that provides highlighting patterns.
+This is mainly intended for major modes that are not aware of tree-sitter.
+@item Optional query patterns for other minor modes that provide high-level
+functionalities on top of tree-sitter, such as code folding, evil text
+objects, ...etc.
+@end enumerate")
+ (license license:expat)))
--
2.32.0
M
M
Maxime Devos wrote on 10 Aug 2021 20:28
Re: [bug#49946] [PATCH 08/31] gnu: node: Patch /usr/bin/env in node-gyp.
42e10baddb6afe308f67c3240bf5da8159e6f118.camel@telenet.be
Pierre Langlois schreef op ma 09-08-2021 om 00:33 [+0100]:
Toggle quote (8 lines)
> @@ -120,6 +120,10 @@
> (("'/usr/bin/env'")
> (string-append "'" (which "env") "'")))
>
> + ;; Fix /usr/bin/env shebang in node-gyp.
> + (substitute* "deps/npm/node_modules/node-gyp/bin/node-gyp.js"
> + (("#!/usr/bin/env") (string-append "#!" (which "env"))))

For cross-compilation, this should most likely be
(string-append (assoc-ref inputs "coreutils") "/bin/env")
or something like that instead. Likewise in other places.
The old code uses (which "env") in some cases, but those
are probably wrong (except where it is patched in tests).

Greetings,
Maxime.
-----BEGIN PGP SIGNATURE-----

iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYRLFaRccbWF4aW1lZGV2
b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7jcOAQCV9XqauiR9WpI550tltw8xH4Wv
lvGKYPnUb14G5U/W3gEA6sDJnEtt6s7Ag/t6AerN4qI28JQeCxQ0LkrLhz4nzQ0=
=80uq
-----END PGP SIGNATURE-----


M
M
Maxime Devos wrote on 10 Aug 2021 20:30
Re: [bug#49946] [PATCH 11/31] gnu: Add tree-sitter-c.
fd74360c5ce59c7800d1ee528036d43a4a9fec32.camel@telenet.be
Pierre Langlois schreef op ma 09-08-2021 om 00:33 [+0100]:
Toggle quote (2 lines)
> + (setenv "CC" "gcc")))

Use (setenv "CC" ,(cc-for-target)) instead,
such that the cross-compiler TARGET-gcc is used
when cross-compiling.

reetings,
Maxime.
-----BEGIN PGP SIGNATURE-----

iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYRLFxxccbWF4aW1lZGV2
b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7pqsAP93DYjd22Jnvq86nV9t/Fc0ePrv
JFHBVxHk0H3v68M7VAD9Fo4kKgqsuDWUqyiahLhV81pHRrSIMHEGpAIMVxqp8Q0=
=uvuY
-----END PGP SIGNATURE-----


P
P
Pierre Langlois wrote on 11 Aug 2021 17:36
Re: [bug#49946] [PATCH 08/31] gnu: node: Patch /usr/bin/env in node-gyp.
(name . Maxime Devos)(address . maximedevos@telenet.be)
87czqkng4k.fsf@gmx.com
Hi Maxime,

Maxime Devos <maximedevos@telenet.be> writes:

Toggle quote (16 lines)
> [[PGP Signed Part:Undecided]]
> Pierre Langlois schreef op ma 09-08-2021 om 00:33 [+0100]:
>> @@ -120,6 +120,10 @@
>> (("'/usr/bin/env'")
>> (string-append "'" (which "env") "'")))
>>
>> + ;; Fix /usr/bin/env shebang in node-gyp.
>> + (substitute* "deps/npm/node_modules/node-gyp/bin/node-gyp.js"
>> + (("#!/usr/bin/env") (string-append "#!" (which "env"))))
>
> For cross-compilation, this should most likely be
> (string-append (assoc-ref inputs "coreutils") "/bin/env")
> or something like that instead. Likewise in other places.
> The old code uses (which "env") in some cases, but those
> are probably wrong (except where it is patched in tests).

Good point, I didn't consider cross-compilation. Actually, trying it,
it looks like our node package doesn't currently cross-compile
correctly. I just managed to get it to cross-compile though, I'll
submit another patch for it! In the meantime, with this series I agree
it's better for the new code to be correct from the begining though.

Thanks for taking a look!
Pierre
-----BEGIN PGP SIGNATURE-----

iQFMBAEBCgA2FiEEctU9gYy29KFyWDdMqPyeRH9PfVQFAmET75sYHHBpZXJyZS5s
YW5nbG9pc0BnbXguY29tAAoJEKj8nkR/T31U65EH/1viouDCu3YEBTff3X725srp
+kf8oKFE4Vl/cE2uR8Gqolj4ldjYKYiYrf/Mhz5oM4+N2LqaJjaxcs+3dRXV8NrS
51CYHzWmhPu4kMf3LaetK3s/kjauDME80EHaOYDiH9aZn2h6D4QOmHbA/Xs7Oct+
2Ntoz356uMaD6jzluK1ljCmLQyew4TnlnaOzzF4MzJ6mE98noT6MAiMZyWmf7aXv
ax6Zr6f+hJc83TiUAN6hKX4hc/sIGv8c9MwCepNeZZGQHeJvAiw223bIfClXlIPJ
2NPTab5FQ0ypbpDhmmhoZ787yS3SRp18mJuC/PblI9l5WZHd+GR2Bue7kXdl1vY=
=YsOg
-----END PGP SIGNATURE-----

P
P
Pierre Langlois wrote on 13 Aug 2021 15:17
Re: [PATCH 07/31] gnu: Add tree-sitter.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
87zgtla2x0.fsf@gmx.com
Pierre Langlois <pierre.langlois@gmx.com> writes:

Toggle quote (68 lines)
> * gnu/packages/tree-sitter.scm: New file.
> * gnu/local.mk (GNU_SYSTEM_MODULES): Register it.
> ---
> gnu/local.mk | 1 +
> gnu/packages/tree-sitter.scm | 98 ++++++++++++++++++++++++++++++++++++
> 2 files changed, 99 insertions(+)
> create mode 100644 gnu/packages/tree-sitter.scm
>
> diff --git a/gnu/local.mk b/gnu/local.mk
> index 3f09b167b1..3763df0821 100644
> --- a/gnu/local.mk
> +++ b/gnu/local.mk
> @@ -559,6 +559,7 @@ GNU_SYSTEM_MODULES = \
> %D%/packages/tls.scm \
> %D%/packages/tmux.scm \
> %D%/packages/tor.scm \
> + %D%/packages/tree-sitter.scm \
> %D%/packages/tv.scm \
> %D%/packages/uglifyjs.scm \
> %D%/packages/uml.scm \
> diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
> new file mode 100644
> index 0000000000..2c749dc0a8
> --- /dev/null
> +++ b/gnu/packages/tree-sitter.scm
> @@ -0,0 +1,98 @@
> +;;; GNU Guix --- Functional package management for GNU
> +;;; Copyright © 2021 Pierre Langlois <pierre.langlois@gmx.com>
> +;;;
> +;;; This file is part of GNU Guix.
> +;;;
> +;;; GNU Guix is free software; you can redistribute it and/or modify it
> +;;; under the terms of the GNU General Public License as published by
> +;;; the Free Software Foundation; either version 3 of the License, or (at
> +;;; your option) any later version.
> +;;;
> +;;; GNU Guix is distributed in the hope that it will be useful, but
> +;;; WITHOUT ANY WARRANTY; without even the implied warranty of
> +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> +;;; GNU General Public License for more details.
> +;;;
> +;;; You should have received a copy of the GNU General Public License
> +;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
> +
> +(define-module (gnu packages tree-sitter)
> + #:use-module ((guix licenses) #:prefix license:)
> + #:use-module (guix build-system cargo)
> + #:use-module (guix git-download)
> + #:use-module (guix packages)
> + #:use-module (gnu packages crates-io)
> + #:use-module (gnu packages crates-graphics))
> +
> +(define-public tree-sitter
> + (package
> + (name "tree-sitter")
> + (version "0.19.5")
> + (source
> + (origin
> + (method git-fetch)
> + (uri (git-reference
> + (url "https://github.com/tree-sitter/tree-sitter")
> + (commit (string-append "v" version))))
> + (file-name (git-file-name name version))
> + (sha256
> + (base32
> + "1qmb0sva28zv6r3c3j7xs9pc8bpwwhkb9vxxndw2zbdn9wkvmbmn"))))
> + (build-system cargo-build-system)

The tree-sitter package here both ships the `tree-sitter' command-line
program, written in Rust, and the runtime library, written in C. I
think it would be better to split this into two packages, `tree-sitter'
for the runtime and `tree-sitter-cli` for the Rust program.

That way, a potential future Emacs will be able to link against
libtree-sitter *without* bringing in Rust as a hard dependency. I saw
that native tree-sitter support was being worked on on emacs-devel
list. One would still need Rust to actually use it, but at least it
remains an optional dependency.

Thanks,
Pierre
-----BEGIN PGP SIGNATURE-----

iQFMBAEBCgA2FiEEctU9gYy29KFyWDdMqPyeRH9PfVQFAmEWc7sYHHBpZXJyZS5s
YW5nbG9pc0BnbXguY29tAAoJEKj8nkR/T31ULTwH/R+uDT5j6mrUlYW6Q39UKilQ
WzzPPUVLns8eSZmvDX4hJOxO6+qG3X1lprUOVNgRC4DKncMVW36ntYLqJDin+eFo
Smh87p+1K0itjMuriiOU+foMfUxPlZ7oeHbrQD6y3cBmfUAy8J7G/UtZ+ZBb677b
jycusQGkWXaahi6Llh81g+pZuFAp0+pYo33fhlsabAgI1HzuEUsHDXEWUgfJM2Zx
G+vuyAmGyX3jpA/RXHtXXifB+dnhD4NbUsSb+edCLWUG2OcoW9PNakBhloo3O82O
eHibETllS3pAV66m7gFrZ0pM5V3LdW7o4x0qzQXtYgFAbtPiJjNOqf2Gc4y+Zig=
=hY4u
-----END PGP SIGNATURE-----

P
P
Pierre Langlois wrote on 29 Aug 2021 12:34
[PATCH v2 00/33] gnu: Add tree-sitter.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
87czpwzg9e.fsf@gmx.com
Hi Guix!

Here's a V2 of the tree-sitter patch series. Changes:

- Split tree-sitter into tree-sitter enacs tree-sitter-cli. This way
applications can link against the tree-sitter runtime without
depending on rust. Emacs may do that in the future :-).
- Fix the node build system changes to work with cross-compilation in
the future.
- Use cc-for-target in tree-sitter-c.
- Add tree-sitter-bash, figured out how to package that one.

Thanks,
Pierre

Pierre Langlois (33):
gnu: rust-chunked-transfer: Update to 1.4.0.
gnu: Add rust-fdlimit.
gnu: Add rust-html-escape.
gnu: Add rust-smallbitvec.
gnu: rust-tiny-http: Update to 0.8.
gnu: rust-spin: Update to 0.7.
gnu: Add tree-sitter.
gnu: Add tree-sitter-cli.
gnu: node: Patch /usr/bin/env in node-gyp.
guix: node-build-system: Support compiling addons with node-gyp.
gnu: Add node-nan.
gnu: Add tree-sitter-c.
gnu: Add tree-sitter-cpp.
gnu: Add tree-sitter-bash.
gnu: Add tree-sitter-css.
gnu: Add tree-sitter-go.
gnu: Add tree-sitter-html.
gnu: Add tree-sitter-java.
gnu: Add tree-sitter-javascript.
gnu: Add tree-sitter-json.
gnu: Add tree-sitter-julia.
gnu: Add tree-sitter-php.
gnu: Add tree-sitter-python.
gnu: Add tree-sitter-rust.
gnu: Add tree-sitter-typescript.
gnu: Add rust-bindgen@0.56.
gnu: Add rust-tree-sitter.
gnu: rust-emacs-module: Update to 0.16.
gnu: rust-emacs-macros: Update to 0.17.
gnu: rust-emacs: Update to 0.17.
gnu: Add emacs-tree-sitter-core.
gnu: Add emacs-tree-sitter.
gnu: Add emacs-tree-sitter-langs.

gnu/local.mk | 1 +
gnu/packages/crates-io.scm | 290 +++++++++--
gnu/packages/node-xyz.scm | 30 ++
gnu/packages/node.scm | 31 +-
gnu/packages/tree-sitter.scm | 791 +++++++++++++++++++++++++++++++
guix/build-system/node.scm | 16 +
guix/build/node-build-system.scm | 15 +
7 files changed, 1141 insertions(+), 33 deletions(-)
create mode 100644 gnu/packages/tree-sitter.scm

--
2.33.0
-----BEGIN PGP SIGNATURE-----

iQFMBAEBCgA2FiEEctU9gYy29KFyWDdMqPyeRH9PfVQFAmErY+0YHHBpZXJyZS5s
YW5nbG9pc0BnbXguY29tAAoJEKj8nkR/T31UtlUH/iGlSOaAWq3oU/Sx2zTpmC/5
cgzSwm0XmUaWSZNHTk30WbflF3HwqoqpBV5g92xpk4vfZbyDyXaqcr5YO6ueJhpc
ga60DiP+GD6edx053mzg+/A7/JdCoVQGag+QXhk3brrO95DNQwq7St+DjhKPvdEa
+9EE9/ePmIJ30cPxWk7zziVMyivD8YqcBc7z5bRpn/jjTh647+98aZ+yzgtTXHyF
+wsG7EBSUNRNoMT9cyqnER0werOVXluSkjvhn6RNKdRJHqSR90QBGYim/jxoIMZH
ZZUavwb4JzfwArS44c/j49h0OoaWIoX8cLZL3fQuPKSYkCX71+NMdB/GgXi6GvA=
=z5lI
-----END PGP SIGNATURE-----

P
P
Pierre Langlois wrote on 29 Aug 2021 12:45
[PATCH v2 02/33] gnu: Add rust-fdlimit.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20210829104608.3103-2-pierre.langlois@gmx.com
* gnu/packages/crates-io.scm (rust-fdlimit-0.1): New variable.
---
gnu/packages/crates-io.scm | 22 ++++++++++++++++++++++
1 file changed, 22 insertions(+)

Toggle diff (33 lines)
diff --git a/gnu/packages/crates-io.scm b/gnu/packages/crates-io.scm
index 3e93db1879..bfa14d5297 100644
--- a/gnu/packages/crates-io.scm
+++ b/gnu/packages/crates-io.scm
@@ -16008,6 +16008,28 @@ it.")
(("rust-libc" ,rust-libc-0.2)
("rust-winapi" ,rust-winapi-0.3))))))

+(define-public rust-fdlimit-0.1
+ (package
+ (name "rust-fdlimit")
+ (version "0.1.4")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (crate-uri "fdlimit" version))
+ (file-name
+ (string-append name "-" version ".tar.gz"))
+ (sha256
+ (base32
+ "0g30d6gqkrwy8ylwdy7pqm443iq0p5dmnpz4ks41pirl7dclm98d"))))
+ (build-system cargo-build-system)
+ (arguments
+ `(#:cargo-inputs
+ (("rust-libc" ,rust-libc-0.2))))
+ (home-page "https://github.com/paritytech/fdlimit")
+ (synopsis "Raise the open file descriptors limit")
+ (description "Utility crate for raising file descriptors limit.")
+ (license license:asl2.0)))
+
(define-public rust-femme-2
(package
(name "rust-femme")
--
2.33.0
P
P
Pierre Langlois wrote on 29 Aug 2021 12:45
[PATCH v2 01/33] gnu: rust-chunked-transfer: Update to 1.4.0.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20210829104608.3103-1-pierre.langlois@gmx.com
* gnu/packages/crates-io.scm (rust-chunked-transfer-1): New variable.
(rust-chunked-transfer-0.3): Inherit from rust-chunked-transfer-1.
---
gnu/packages/crates-io.scm | 24 ++++++++++++++++++++----
1 file changed, 20 insertions(+), 4 deletions(-)

Toggle diff (58 lines)
diff --git a/gnu/packages/crates-io.scm b/gnu/packages/crates-io.scm
index 087ffa1d27..3e93db1879 100644
--- a/gnu/packages/crates-io.scm
+++ b/gnu/packages/crates-io.scm
@@ -18,6 +18,7 @@
;;; Copyright © 2021 Antero Mejr <antero@kodmin.com>
;;; Copyright © 2021 Maxim Cournoyer <maxim.cournoyer@gmail.com>
;;; Copyright © 2021 Vinicius Monego <monego@posteo.net>
+;;; Copyright © 2021 Pierre Langlois <pierre.langlois@gmx.com>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -7884,19 +7885,21 @@ Python arrow.humanize.")
TimeZone trait for @code{rust-chrono}.")
(license (list license:expat license:asl2.0))))

-(define-public rust-chunked-transfer-0.3
+(define-public rust-chunked-transfer-1
(package
(name "rust-chunked-transfer")
- (version "0.3.1")
+ (version "1.4.0")
(source
(origin
(method url-fetch)
(uri (crate-uri "chunked_transfer" version))
(file-name (string-append name "-" version ".tar.gz"))
(sha256
- (base32 "11yghnd24w0i9p8g368c3pg7qh9nfz7kgri6pywja9pnmakj13a9"))))
+ (base32 "0bkdlsrszfcscw3j6yhs7kj6jbp8id47jjk6h9k58px47na5gy7z"))))
(build-system cargo-build-system)
- (arguments `(#:skip-build? #t))
+ (arguments
+ `(#:cargo-development-inputs
+ (("rust-criterion" ,rust-criterion-0.3))))
(home-page "https://github.com/frewsxcv/rust-chunked-transfer")
(synopsis "Encoder and decoder for HTTP chunked transfer coding")
(description
@@ -7904,6 +7907,19 @@ TimeZone trait for @code{rust-chrono}.")
coding.")
(license license:asl2.0)))

+(define-public rust-chunked-transfer-0.3
+ (package (inherit rust-chunked-transfer-1)
+ (name "rust-chunked-transfer")
+ (version "0.3.1")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (crate-uri "chunked_transfer" version))
+ (file-name (string-append name "-" version ".tar.gz"))
+ (sha256
+ (base32 "11yghnd24w0i9p8g368c3pg7qh9nfz7kgri6pywja9pnmakj13a9"))))
+ (arguments `(#:skip-build? #t))))
+
(define-public rust-ci-info-0.3
(package
(name "rust-ci-info")
--
2.33.0
P
P
Pierre Langlois wrote on 29 Aug 2021 12:45
[PATCH v2 03/33] gnu: Add rust-html-escape.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20210829104608.3103-3-pierre.langlois@gmx.com
* gnu/packages/crates-io.scm (rust-html-escape-0.2): New variable.
---
gnu/packages/crates-io.scm | 26 ++++++++++++++++++++++++++
1 file changed, 26 insertions(+)

Toggle diff (37 lines)
diff --git a/gnu/packages/crates-io.scm b/gnu/packages/crates-io.scm
index bfa14d5297..31136a88e3 100644
--- a/gnu/packages/crates-io.scm
+++ b/gnu/packages/crates-io.scm
@@ -20921,6 +20921,32 @@ Hash-based Message Authentication Code algorithm} for SHA1.")
"This package provides a library for HTML entity encoding and decoding.")
(license (list license:asl2.0 license:expat license:mpl2.0))))

+(define-public rust-html-escape-0.2
+ (package
+ (name "rust-html-escape")
+ (version "0.2.9")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (crate-uri "html-escape" version))
+ (file-name
+ (string-append name "-" version ".tar.gz"))
+ (sha256
+ (base32
+ "1dxw9lpckrqzzqgbkw64ckbajr4b7xxnjdn8adgzqf2mm40shvl1"))))
+ (build-system cargo-build-system)
+ (arguments
+ `(#:cargo-inputs
+ (("rust-utf8-width" ,rust-utf8-width-0.1))
+ #:cargo-development-inputs
+ (("rust-bencher" ,rust-bencher-0.1))))
+ (home-page "https://magiclen.org/html-escape")
+ (synopsis "Library for encoding and escaping special characters in HTML")
+ (description
+ "This package provides a library for encoding and escaping special
+characters in HTML, decoding and unescaping HTML entities as well.")
+ (license license:expat)))
+
(define-public rust-http-0.2
(package
(name "rust-http")
--
2.33.0
P
P
Pierre Langlois wrote on 29 Aug 2021 12:45
[PATCH v2 04/33] gnu: Add rust-smallbitvec.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20210829104608.3103-4-pierre.langlois@gmx.com
* gnu/packages/crates-io.scm (rust-smallbitvec-2): New variable.
---
gnu/packages/crates-io.scm | 23 +++++++++++++++++++++++
1 file changed, 23 insertions(+)

Toggle diff (34 lines)
diff --git a/gnu/packages/crates-io.scm b/gnu/packages/crates-io.scm
index 31136a88e3..de1b581cbd 100644
--- a/gnu/packages/crates-io.scm
+++ b/gnu/packages/crates-io.scm
@@ -44429,6 +44429,29 @@ I/O programming.")
("rust-futures-core-preview" ,rust-futures-core-preview-0.3)
("rust-futures-io-preview" ,rust-futures-io-preview-0.3))))))

+(define-public rust-smallbitvec-2
+ (package
+ (name "rust-smallbitvec")
+ (version "2.5.1")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (crate-uri "smallbitvec" version))
+ (file-name (string-append name "-" version ".tar.gz"))
+ (sha256
+ (base32
+ "0plrbldsjpwip3afbzd8fgrnvdhizcg5z4ncfqs4q6x4qjflzkkm"))))
+ (build-system cargo-build-system)
+ (arguments
+ `(#:cargo-development-inputs
+ (("rust-bit-vec" ,rust-bit-vec-0.4)
+ ("rust-rand" ,rust-rand-0.4))))
+ (home-page "https://github.com/servo/smallbitvec")
+ (synopsis "A bit vector optimized for size and inline storage")
+ (description "This package provides a bit vector optimized for size and
+inline storage")
+ (license (list license:expat license:asl2.0))))
+
(define-public rust-smallvec-1
(package
(name "rust-smallvec")
--
2.33.0
P
P
Pierre Langlois wrote on 29 Aug 2021 12:45
[PATCH v2 06/33] gnu: rust-spin: Update to 0.7.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20210829104608.3103-6-pierre.langlois@gmx.com
* gnu/packages/crates-io.scm (rust-spin-0.7): New variable.
(rust-spin-0.5): Inherit from rust-spin-0.7.
---
gnu/packages/crates-io.scm | 24 +++++++++++++++++++++---
1 file changed, 21 insertions(+), 3 deletions(-)

Toggle diff (52 lines)
diff --git a/gnu/packages/crates-io.scm b/gnu/packages/crates-io.scm
index 07d0a7594e..476d35c241 100644
--- a/gnu/packages/crates-io.scm
+++ b/gnu/packages/crates-io.scm
@@ -44838,10 +44838,10 @@ track of where each new file and line starts.")
"An RSpec inspired minimal testing framework for Rust.")
(license license:expat)))

-(define-public rust-spin-0.5
+(define-public rust-spin-0.7
(package
(name "rust-spin")
- (version "0.5.2")
+ (version "0.7.1")
(source
(origin
(method url-fetch)
@@ -44849,8 +44849,11 @@ track of where each new file and line starts.")
(file-name (string-append name "-" version ".crate"))
(sha256
(base32
- "0b84m6dbzrwf2kxylnw82d3dr8w06av7rfkr8s85fb5f43rwyqvf"))))
+ "0qjips9f6fsvkyd7wj3a4gzaqknn2q4kkb19957pl86im56pna0k"))))
(build-system cargo-build-system)
+ (arguments
+ `(#:cargo-inputs
+ (("rust-lock-api" ,rust-lock-api-0.4))))
(home-page "https://github.com/mvdnes/spin-rs")
(synopsis "Synchronization primitives based on spinning")
(description "This crate provides synchronization primitives based on
@@ -44858,6 +44861,21 @@ spinning. They may contain data, are usable without @code{std},and static
initializers are available.")
(license license:expat)))

+(define-public rust-spin-0.5
+ (package
+ (inherit rust-spin-0.7)
+ (name "rust-spin")
+ (version "0.5.2")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (crate-uri "spin" version))
+ (file-name (string-append name "-" version ".crate"))
+ (sha256
+ (base32
+ "0b84m6dbzrwf2kxylnw82d3dr8w06av7rfkr8s85fb5f43rwyqvf"))))
+ (arguments '(#:cargo-inputs ()))))
+
(define-public rust-spin-0.4
(package
(inherit rust-spin-0.5)
--
2.33.0
P
P
Pierre Langlois wrote on 29 Aug 2021 12:45
[PATCH v2 05/33] gnu: rust-tiny-http: Update to 0.8.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20210829104608.3103-5-pierre.langlois@gmx.com
* gnu/packages/crates-io.scm (rust-tiny-http-0.8): New variable.
(rust-tiny-http-0.6): Inherit from rust-tiny-http-0.8.
---
gnu/packages/crates-io.scm | 39 ++++++++++++++++++++++++++++++++------
1 file changed, 33 insertions(+), 6 deletions(-)

Toggle diff (58 lines)
diff --git a/gnu/packages/crates-io.scm b/gnu/packages/crates-io.scm
index de1b581cbd..07d0a7594e 100644
--- a/gnu/packages/crates-io.scm
+++ b/gnu/packages/crates-io.scm
@@ -49152,8 +49152,39 @@ closures after a delay or at a given timestamp.")
manipulation in Rust.")
(license license:expat)))

-(define-public rust-tiny-http-0.6
+(define-public rust-tiny-http-0.8
(package
+ (name "rust-tiny-http")
+ (version "0.8.2")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (crate-uri "tiny_http" version))
+ (file-name
+ (string-append name "-" version ".tar.gz"))
+ (sha256
+ (base32
+ "0fcdwpb2ghk671qjjrk6048hs3yp7f681hxpr68gamk00181prcw"))))
+ (build-system cargo-build-system)
+ (arguments
+ `(#:cargo-inputs
+ (("rust-ascii" ,rust-ascii-1)
+ ("rust-chrono" ,rust-chrono-0.4)
+ ("rust-chunked-transfer" ,rust-chunked-transfer-1)
+ ("rust-log" ,rust-log-0.4)
+ ("rust-openssl" ,rust-openssl-0.10)
+ ("rust-url" ,rust-url-2))
+ #:cargo-development-inputs
+ (("rust-fdlimit" ,rust-fdlimit-0.1)
+ ("rust-rustc-serialize" ,rust-rustc-serialize-0.3)
+ ("rust-sha1" ,rust-sha1-0.6))))
+ (home-page "https://github.com/tiny-http/tiny-http")
+ (synopsis "Low level HTTP server library")
+ (description "This package provides a low level HTTP server library.")
+ (license license:asl2.0)))
+
+(define-public rust-tiny-http-0.6
+ (package (inherit rust-tiny-http-0.8)
(name "rust-tiny-http")
(version "0.6.2")
(source
@@ -49172,11 +49203,7 @@ manipulation in Rust.")
("rust-chunked-transfer" ,rust-chunked-transfer-0.3)
("rust-log" ,rust-log-0.4)
("rust-openssl" ,rust-openssl-0.10)
- ("rust-url" ,rust-url-1))))
- (home-page "https://github.com/tiny-http/tiny-http")
- (synopsis "Low level HTTP server library")
- (description "This package provides a low level HTTP server library.")
- (license license:asl2.0)))
+ ("rust-url" ,rust-url-1))))))

(define-public rust-tiny-keccak-2
(package
--
2.33.0
P
P
Pierre Langlois wrote on 29 Aug 2021 12:45
[PATCH v2 08/33] gnu: Add tree-sitter-cli.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20210829104608.3103-8-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-cli): New variable.
---
gnu/packages/tree-sitter.scm | 61 +++++++++++++++++++++++++++++++++++-
1 file changed, 60 insertions(+), 1 deletion(-)

Toggle diff (79 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index e6e66cd447..d8d4a305da 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -18,10 +18,13 @@

(define-module (gnu packages tree-sitter)
#:use-module ((guix licenses) #:prefix license:)
+ #:use-module (guix build-system cargo)
#:use-module (guix build-system gnu)
#:use-module (guix git-download)
#:use-module (guix packages)
- #:use-module (guix utils))
+ #:use-module (guix utils)
+ #:use-module (gnu packages crates-graphics)
+ #:use-module (gnu packages crates-io))

(define-public tree-sitter
(package
@@ -71,3 +74,59 @@ can be embedded in any application.

This package includes the @code{libtree-sitter} runtime library.")
(license license:expat)))
+
+(define-public tree-sitter-cli
+ (package (inherit tree-sitter)
+ (name "tree-sitter-cli")
+ (build-system cargo-build-system)
+ (arguments
+ `(;; Running test requires downloading fixtures, see the
+ ;; script/fetch-fixtures script.
+ #:tests? #f
+ ;; FIXME: Installing the sources for the tree-sitter Rust bindings
+ ;; doesn't work out of the box due to tree-sitter having multiple
+ ;; Rust packages in the same repository (bindings and CLI).
+ #:install-source? #f
+ #:cargo-inputs
+ (("rust-ansi-term" ,rust-ansi-term-0.12)
+ ("rust-atty" ,rust-atty-0.2)
+ ("rust-clap" ,rust-clap-2)
+ ("rust-difference" ,rust-difference-2)
+ ("rust-dirs" ,rust-dirs-3)
+ ("rust-html-escape" ,rust-html-escape-0.2)
+ ("rust-libloading" ,rust-libloading-0.7)
+ ("rust-smallbitvec" ,rust-smallbitvec-2)
+ ("rust-spin" ,rust-spin-0.7)
+ ("rust-tiny-http" ,rust-tiny-http-0.8)
+ ("rust-walkdir" ,rust-walkdir-2)
+ ("rust-webbrowser" ,rust-webbrowser-0.5)
+ ("rust-which" ,rust-which-4))
+ #:phases
+ (modify-phases %standard-phases
+ (add-after 'unpack 'delete-cargo.lock
+ (lambda _ (delete-file "Cargo.lock")))
+ (replace 'install
+ (lambda* (#:key outputs #:allow-other-keys)
+ (let* ((out (assoc-ref outputs "out"))
+ (bin (string-append out "/bin"))
+ (lib (string-append out "/lib")))
+ (mkdir-p bin)
+ (install-file "target/release/tree-sitter" bin)))))))
+ (synopsis "Incremental parsing system for programming tools")
+ (description "Tree-sitter is a parser generator tool and an incremental
+parsing library. It can build a concrete syntax tree for a source file and
+efficiently update the syntax tree as the source file is edited.
+
+Tree-sitter aims to be:
+
+@enumerate
+@item General enough to parse any programming language.
+@item Fast enough to parse on every keystroke in a text editor.
+@item Robust enough to provide useful results even in the presence of syntax
+errors.
+@item Dependency-free so that the runtime library (which is written in pure C)
+can be embedded in any application.
+@end enumerate
+
+This package includes the @command{tree-sitter} command-line tool.")
+ (license license:expat)))
--
2.33.0
P
P
Pierre Langlois wrote on 29 Aug 2021 12:45
[PATCH v2 09/33] gnu: node: Patch /usr/bin/env in node-gyp.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20210829104608.3103-9-pierre.langlois@gmx.com
* gnu/packages/node.scm (node)[arguments]: Fix /usr/bin/env shebang in
node-gyp.js.
(node-lts)[arguments]: Ditto.
---
gnu/packages/node.scm | 12 +++++++++++-
1 file changed, 11 insertions(+), 1 deletion(-)

Toggle diff (37 lines)
diff --git a/gnu/packages/node.scm b/gnu/packages/node.scm
index 36c45e9c7a..530c04bc68 100644
--- a/gnu/packages/node.scm
+++ b/gnu/packages/node.scm
@@ -6,7 +6,7 @@
;;; Copyright © 2017 Mike Gerwitz <mtg@gnu.org>
;;; Copyright © 2018 Tobias Geerinckx-Rice <me@tobias.gr>
;;; Copyright © 2018, 2019, 2020, 2021 Marius Bakke <marius@gnu.org>
-;;; Copyright © 2020 Pierre Langlois <pierre.langlois@gmx.com>
+;;; Copyright © 2020, 2021 Pierre Langlois <pierre.langlois@gmx.com>
;;; Copyright © 2020 Ricardo Wurmus <rekado@elephly.net>
;;; Copyright © 2021 Simon Tournier <zimon.toutoune@gmail.com>
;;;
@@ -120,6 +120,11 @@
(("'/usr/bin/env'")
(string-append "'" (which "env") "'")))

+ ;; Fix /usr/bin/env shebang in node-gyp.
+ (substitute* "deps/npm/node_modules/node-gyp/bin/node-gyp.js"
+ (("#!/usr/bin/env")
+ (string-append "#!" (assoc-ref inputs "coreutils") "/bin/env")))
+
;; FIXME: These tests fail in the build container, but they don't
;; seem to be indicative of real problems in practice.
(for-each delete-file
@@ -661,6 +666,11 @@ source files.")
(("'/usr/bin/env'")
(string-append "'" (which "env") "'")))

+ ;; Fix /usr/bin/env shebang in node-gyp.
+ (substitute* "deps/npm/node_modules/node-gyp/bin/node-gyp.js"
+ (("#!/usr/bin/env")
+ (string-append "#!" (assoc-ref inputs "coreutils") "/bin/env")))
+
;; FIXME: These tests fail in the build container, but they don't
;; seem to be indicative of real problems in practice.
(for-each delete-file
--
2.33.0
P
P
Pierre Langlois wrote on 29 Aug 2021 12:45
[PATCH v2 11/33] gnu: Add node-nan.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20210829104608.3103-11-pierre.langlois@gmx.com
---
gnu/packages/node-xyz.scm | 30 ++++++++++++++++++++++++++++++
1 file changed, 30 insertions(+)

Toggle diff (48 lines)
diff --git a/gnu/packages/node-xyz.scm b/gnu/packages/node-xyz.scm
index cbb68f77d6..eb4b59d1ab 100644
--- a/gnu/packages/node-xyz.scm
+++ b/gnu/packages/node-xyz.scm
@@ -3,6 +3,7 @@
;;; Copyright © 2020 Giacomo Leidi <goodoldpaul@autistici.org>
;;; Copyright © 2021 Noisytoot <noisytoot@disroot.org>
;;; Copyright © 2021 Charles <charles.b.jackson@protonmail.com>
+;;; Copyright © 2021 Pierre Langlois <pierre.langlois@gmx.com>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -171,6 +172,35 @@ user-land JavaScript.")
random number generator.")
(license license:bsd-3)))

+(define-public node-nan
+ (package
+ (name "node-nan")
+ (version "2.15.0")
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/nodejs/nan")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "18xslh9va5ld872scrp5y4251ax9s3c6qh0lnl1200lpzbsxy7yd"))))
+ (build-system node-build-system)
+ (arguments
+ '(#:tests? #f ; FIXME: tests depend on node-tap
+ #:phases
+ (modify-phases %standard-phases
+ ;; The default configure phase fails due to tap being missing, as we do
+ ;; not have tap packaged yet. It is used only for tests. This package
+ ;; still works as a dependency of node-glob and node-inflight.
+ (delete 'configure))))
+ (home-page "https://github.com/nodejs/nan")
+ (synopsis "Native Abstractions for Node.js")
+ (description "This package provides header files filled with macros and
+utilities for making add-on development for Node.js easier across versions.")
+ (license license:bsd-3)))
+
(define-public node-oop
;; No releases, last commit was February 2013.
(let ((commit "f9d87cda0958886955c14a0a716e57021ed295dc")
--
2.33.0
P
P
Pierre Langlois wrote on 29 Aug 2021 12:45
[PATCH v2 10/33] guix: node-build-system: Support compiling addons with node-gyp.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20210829104608.3103-10-pierre.langlois@gmx.com
* gnu/packages/node.scm (node-headers): New function.
* guix/build-system/node.scm (python): New function.
(node-headers): New function.
(lower): Add node-headers and python to build inputs.
* guix/build/node-build-system.scm (configure-gyp): New function.
(%standard-phases): Add 'configure-gyp after 'configure.
---
gnu/packages/node.scm | 19 ++++++++++++++++++-
guix/build-system/node.scm | 16 ++++++++++++++++
guix/build/node-build-system.scm | 15 +++++++++++++++
3 files changed, 49 insertions(+), 1 deletion(-)

Toggle diff (111 lines)
diff --git a/gnu/packages/node.scm b/gnu/packages/node.scm
index 530c04bc68..a6dab75309 100644
--- a/gnu/packages/node.scm
+++ b/gnu/packages/node.scm
@@ -48,7 +48,24 @@
#:use-module (gnu packages pkg-config)
#:use-module (gnu packages python)
#:use-module (gnu packages tls)
- #:use-module (gnu packages web))
+ #:use-module (gnu packages web)
+ #:use-module (ice-9 match)
+ #:export (node-headers))
+
+(define (node-headers node)
+ "Return an <origin> object for a tarball with headers for the given node
+package version."
+ (let* ((version (package-version node))
+ (hash (match version
+ ("10.24.0"
+ "0h37zjwcpxjdqcxqjfj5zp1n5zjxaa0g8lsy83955afg5cca8p0n")
+ ("14.16.0"
+ "1vpdgq7kcw1a0w90lpvbvxbrc0n3pwjrs3sm42pjj7560clvji2b"))))
+ (origin
+ (method url-fetch)
+ (uri (string-append "https://nodejs.org/dist/v" version
+ "/node-v" version "-headers.tar.gz"))
+ (sha256 (base32 hash)))))

(define-public node
(package
diff --git a/guix/build-system/node.scm b/guix/build-system/node.scm
index 98f63f87ef..7828582a9a 100644
--- a/guix/build-system/node.scm
+++ b/guix/build-system/node.scm
@@ -1,6 +1,7 @@
;;; GNU Guix --- Functional package management for GNU
;;; Copyright © 2016 Jelle Licht <jlicht@fsfe.org>
;;; Copyright © 2019 Timothy Sample <samplet@ngyro.com>
+;;; Copyright © 2021 Pierre Langlois <pierre.langlois@gmx.com>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -41,6 +42,19 @@
(let ((node (resolve-interface '(gnu packages node))))
(module-ref node 'node-lts)))

+(define (python)
+ "Return the python package."
+ ;; Lazily resolve the binding to avoid a circular dependency.
+ (let ((module (resolve-interface '(gnu packages python))))
+ (module-ref module 'python-wrapper)))
+
+(define (node-headers node)
+ "Return a tarball with headers for the given node, needed for packages that
+need to build native bindings using node-gyp."
+ ;; Lazily resolve the binding to avoid a circular dependency.
+ (let ((module (resolve-interface '(gnu packages node))))
+ ((module-ref module 'node-headers) node)))
+
(define* (lower name
#:key source inputs native-inputs outputs system target
(node (default-node))
@@ -62,6 +76,8 @@
;; Keep the standard inputs of 'gnu-build-system'.
,@(standard-packages)))
(build-inputs `(("node" ,node)
+ ("node-headers" ,(node-headers node))
+ ("python" ,(python))
,@native-inputs))
(outputs outputs)
(build node-build)
diff --git a/guix/build/node-build-system.scm b/guix/build/node-build-system.scm
index 70a367618e..2993c49b2b 100644
--- a/guix/build/node-build-system.scm
+++ b/guix/build/node-build-system.scm
@@ -2,6 +2,7 @@
;;; Copyright © 2015 David Thompson <davet@gnu.org>
;;; Copyright © 2016, 2020 Jelle Licht <jlicht@fsfe.org>
;;; Copyright © 2019, 2021 Timothy Sample <samplet@ngyro.com>
+;;; Copyright © 2021 Pierre Langlois <pierre.langlois@gmx.com>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -101,6 +102,19 @@
(invoke npm "--offline" "--ignore-scripts" "install")
#t))

+(define* (configure-gyp #:key inputs #:allow-other-keys)
+ "Run 'node-gyp configure' if we see a 'binding.gyp' file.
+
+By default, 'node-gyp' will try to download node headers from the internet, we
+prevent this with the '--tarball' flag."
+ (let ((node-gyp (string-append (assoc-ref inputs "node")
+ "/lib/node_modules/npm/node_modules/node-gyp"
+ "/bin/node-gyp.js")))
+ (if (file-exists? "binding.gyp")
+ (invoke node-gyp "--tarball" (assoc-ref inputs "node-headers")
+ "configure"))
+ #t))
+
(define* (build #:key inputs #:allow-other-keys)
(let ((package-meta (call-with-input-file "package.json" read-json)))
(if (and=> (assoc-ref package-meta "scripts")
@@ -147,6 +161,7 @@
(add-after 'unpack 'set-home set-home)
(add-before 'configure 'patch-dependencies patch-dependencies)
(replace 'configure configure)
+ (add-after 'configure 'configure-gyp configure-gyp)
(replace 'build build)
(replace 'check check)
(add-before 'install 'repack repack)
--
2.33.0
P
P
Pierre Langlois wrote on 29 Aug 2021 12:45
[PATCH v2 13/33] gnu: Add tree-sitter-cpp.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20210829104608.3103-13-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-cpp): New variable.
---
gnu/packages/tree-sitter.scm | 24 ++++++++++++++++++++++++
1 file changed, 24 insertions(+)

Toggle diff (32 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index bc04a4a5a7..07bdb54627 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -191,3 +191,27 @@ This package includes the @command{tree-sitter} command-line tool.")
(description
"This package provides a C grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-cpp
+ (package (inherit tree-sitter-c)
+ (name "tree-sitter-cpp")
+ (version "0.19.0")
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-cpp")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "08ywv6n80sa541rr08bqz4zyg7byvjcabp68lvxmcahjk8xzcgwk"))))
+ (native-inputs
+ `(("tree-sitter-cli" ,tree-sitter-cli)
+ ("tree-sitter-c" ,tree-sitter-c)
+ ("node-nan" ,node-nan)))
+ (home-page "https://github.com/tree-sitter/tree-sitter-cpp")
+ (synopsis "Tree-sitter C++ grammar")
+ (description
+ "This package provides a C++ grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.33.0
P
P
Pierre Langlois wrote on 29 Aug 2021 12:45
[PATCH v2 14/33] gnu: Add tree-sitter-bash.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20210829104608.3103-14-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-bash): New variable.
---
gnu/packages/tree-sitter.scm | 41 ++++++++++++++++++++++++++++++++++++
1 file changed, 41 insertions(+)

Toggle diff (49 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 07bdb54627..f98a7d392f 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -215,3 +215,44 @@ This package includes the @command{tree-sitter} command-line tool.")
(description
"This package provides a C++ grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-bash
+ (package (inherit tree-sitter-c)
+ (name "tree-sitter-bash")
+ (version "0.19.0")
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-bash")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "18c030bb65r50i6z37iy7jb9z9i8i36y7b08dbc9bchdifqsijs5"))))
+ (native-inputs
+ `(("tree-sitter-cli" ,tree-sitter-cli)
+ ("node-nan" ,node-nan)))
+ (arguments
+ (substitute-keyword-arguments (package-arguments tree-sitter-c)
+ ((#:phases phases)
+ `(modify-phases ,phases
+ ;; The `prebuild' dependency is a way to intercept the build step
+ ;; and download a prebuilt binary instead. If we remove it then the
+ ;; buid system falls back to building from source.
+ (add-after 'unpack 'remove-prebuild-dependency
+ (lambda _
+ (with-atomic-file-replacement "package.json"
+ (lambda (in out)
+ (use-modules ((guix build json)))
+ (let ((package-meta (read-json in)))
+ (assoc-remove! (assoc-ref package-meta "dependencies")
+ "prebuild-install")
+ (assoc-remove! (assoc-ref package-meta "devDependencies")
+ "prebuild")
+ (write-json package-meta out))))))))))
+ (home-page "https://github.com/tree-sitter/tree-sitter-bash")
+ (synopsis "Tree-sitter Bash grammar")
+ (description
+ "This package provides a Bash grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.33.0
P
P
Pierre Langlois wrote on 29 Aug 2021 12:45
[PATCH v2 15/33] gnu: Add tree-sitter-css.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20210829104608.3103-15-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-css): New variable.
---
gnu/packages/tree-sitter.scm | 20 ++++++++++++++++++++
1 file changed, 20 insertions(+)

Toggle diff (28 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index f98a7d392f..592a61ed25 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -256,3 +256,23 @@ This package includes the @command{tree-sitter} command-line tool.")
(description
"This package provides a Bash grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-css
+ (package (inherit tree-sitter-c)
+ (name "tree-sitter-css")
+ (version "0.19.0")
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-css")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "014jrlgi7zfza9g38hsr4vlbi8964i5p7iglaih6qmzaiml7bja2"))))
+ (home-page "https://github.com/tree-sitter/tree-sitter-css")
+ (synopsis "Tree-sitter CSS grammar")
+ (description
+ "This package provides a CSS grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.33.0
P
P
Pierre Langlois wrote on 29 Aug 2021 12:45
[PATCH v2 16/33] gnu: Add tree-sitter-go.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20210829104608.3103-16-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-go): New variable.
---
gnu/packages/tree-sitter.scm | 29 +++++++++++++++++++++++++++++
1 file changed, 29 insertions(+)

Toggle diff (37 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 592a61ed25..5cb3a0e76b 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -276,3 +276,32 @@ This package includes the @command{tree-sitter} command-line tool.")
(description
"This package provides a CSS grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-go
+ (package (inherit tree-sitter-c)
+ (name "tree-sitter-go")
+ (version "0.19.1")
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-go")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "0nxs47vd2fc2fr0qlxq496y852rwg39flhg334s7dlyq7d3lcx4x"))))
+ (arguments
+ (substitute-keyword-arguments (package-arguments tree-sitter-c)
+ ((#:phases phases)
+ `(modify-phases ,phases
+ ;; The parse-example script tries to clone git repositories.
+ (add-after 'unpack 'remove-parse-example
+ (lambda _
+ (substitute* "package.json"
+ ((" && script\\/parse-examples") ""))))))))
+ (home-page "https://github.com/tree-sitter/tree-sitter-go")
+ (synopsis "Tree-sitter Go grammar")
+ (description
+ "This package provides a Golang grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.33.0
P
P
Pierre Langlois wrote on 29 Aug 2021 12:45
[PATCH v2 12/33] gnu: Add tree-sitter-c.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20210829104608.3103-12-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-c): New variable.
---
gnu/packages/tree-sitter.scm | 63 +++++++++++++++++++++++++++++++++++-
1 file changed, 62 insertions(+), 1 deletion(-)

Toggle diff (82 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index d8d4a305da..bc04a4a5a7 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -20,11 +20,13 @@
#:use-module ((guix licenses) #:prefix license:)
#:use-module (guix build-system cargo)
#:use-module (guix build-system gnu)
+ #:use-module (guix build-system node)
#:use-module (guix git-download)
#:use-module (guix packages)
#:use-module (guix utils)
#:use-module (gnu packages crates-graphics)
- #:use-module (gnu packages crates-io))
+ #:use-module (gnu packages crates-io)
+ #:use-module (gnu packages node-xyz))

(define-public tree-sitter
(package
@@ -130,3 +132,62 @@ can be embedded in any application.

This package includes the @command{tree-sitter} command-line tool.")
(license license:expat)))
+
+(define-public tree-sitter-c
+ (package
+ (name "tree-sitter-c")
+ (version "0.19.0")
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-c")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "1diys8yigvhm4ppbmp3a473yxjg2d5lk11y0ay7qprcz7233lakv"))))
+ (build-system node-build-system)
+ (native-inputs
+ `(("tree-sitter-cli" ,tree-sitter-cli)
+ ("node-nan" ,node-nan)))
+ (arguments
+ `(#:phases
+ (modify-phases %standard-phases
+ ;; tree-sitter-cli is listed as a Node.js dependency, however the
+ ;; node tree-sitter-cli package is just a wrapper which downloads a
+ ;; tree-sitter binary, see
+ ;; https://github.com/tree-sitter/tree-sitter/tree/master/cli/npm
+ ;; Instead we remove it as a dependency so that we can use our own
+ ;; tree-sitter package.
+ (add-after 'unpack 'fix-configure
+ (lambda _
+ (with-atomic-file-replacement "package.json"
+ (lambda (in out)
+ (use-modules ((guix build json)))
+ (let ((package-meta (read-json in)))
+ (assoc-remove! (assoc-ref package-meta "devDependencies")
+ "tree-sitter-cli")
+ (write-json package-meta out))))))
+ (add-before 'build 'set-cc
+ (lambda _
+ (setenv "CC" ,(cc-for-target))))
+ (add-before 'build 'make-files-writable
+ (lambda _
+ (for-each make-file-writable (find-files "src" ".*"))))
+ (add-after 'install 'install-native-lib
+ (lambda* (#:key outputs #:allow-other-keys)
+ (let ((lib (string-append (assoc-ref outputs "out")
+ "/lib/tree-sitter")))
+ ;; Invoking `tree-sitter test' loads the grammar which
+ ;; compiles it to a .so binary that we install.
+ (invoke "tree-sitter" "test")
+ (mkdir-p lib)
+ (copy-recursively (string-append (getenv "HOME")
+ "/.tree-sitter/bin")
+ lib)))))))
+ (home-page "https://github.com/tree-sitter/tree-sitter-c")
+ (synopsis "Tree-sitter C grammar")
+ (description
+ "This package provides a C grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.33.0
P
P
Pierre Langlois wrote on 29 Aug 2021 12:45
[PATCH v2 17/33] gnu: Add tree-sitter-html.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20210829104608.3103-17-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-html): New variable.
---
gnu/packages/tree-sitter.scm | 20 ++++++++++++++++++++
1 file changed, 20 insertions(+)

Toggle diff (28 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 5cb3a0e76b..e2a0757e30 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -305,3 +305,23 @@ This package includes the @command{tree-sitter} command-line tool.")
(description
"This package provides a Golang grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-html
+ (package (inherit tree-sitter-c)
+ (name "tree-sitter-html")
+ (version "0.19.0")
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-html")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "1hg7vbcy7bir6b8x11v0a4x0glvqnsqc3i2ixiarbxmycbgl3axy"))))
+ (home-page "https://github.com/tree-sitter/tree-sitter-html")
+ (synopsis "Tree-sitter HTML grammar")
+ (description
+ "This package provides a HTML grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.33.0
P
P
Pierre Langlois wrote on 29 Aug 2021 12:45
[PATCH v2 18/33] gnu: Add tree-sitter-java.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20210829104608.3103-18-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-java): New variable.
---
gnu/packages/tree-sitter.scm | 29 +++++++++++++++++++++++++++++
1 file changed, 29 insertions(+)

Toggle diff (37 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index e2a0757e30..174f0923f8 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -325,3 +325,32 @@ This package includes the @command{tree-sitter} command-line tool.")
(description
"This package provides a HTML grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-java
+ (package (inherit tree-sitter-c)
+ (name "tree-sitter-java")
+ (version "0.19.1")
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-java")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "07zw9ygb45hnvlx9qlz7rlz8hc3byjy03d24v72i5iyhpiiwlhvl"))))
+ (arguments
+ (substitute-keyword-arguments (package-arguments tree-sitter-c)
+ ((#:phases phases)
+ `(modify-phases ,phases
+ ;; The parse-example script tries to clone git repositories.
+ (add-after 'unpack 'remove-parse-example
+ (lambda _
+ (substitute* "package.json"
+ ((" && script\\/parse-examples") ""))))))))
+ (home-page "https://github.com/tree-sitter/tree-sitter-java")
+ (synopsis "Tree-sitter Java grammar")
+ (description
+ "This package provides a Java grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.33.0
P
P
Pierre Langlois wrote on 29 Aug 2021 12:45
[PATCH v2 19/33] gnu: Add tree-sitter-javascript.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20210829104608.3103-19-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-javascript): New variable.
---
gnu/packages/tree-sitter.scm | 30 ++++++++++++++++++++++++++++++
1 file changed, 30 insertions(+)

Toggle diff (38 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 174f0923f8..2be6b15cb0 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -354,3 +354,33 @@ This package includes the @command{tree-sitter} command-line tool.")
(description
"This package provides a Java grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-javascript
+ (package (inherit tree-sitter-c)
+ (name "tree-sitter-javascript")
+ (version "0.19.0")
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-javascript")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "063va0s727yzhy1cz04fipzwwbq4af6fhgka6g970hk9yf7ggmnj"))))
+ (arguments
+ (substitute-keyword-arguments (package-arguments tree-sitter-c)
+ ((#:phases phases)
+ `(modify-phases ,phases
+ ;; The parse-example script tries to clone git repositories.
+ (add-after 'unpack 'remove-parse-example
+ (lambda _
+ (substitute* "package.json"
+ ((" && script\\/parse-examples") ""))))))))
+ (home-page "https://github.com/tree-sitter/tree-sitter-javascript")
+ (synopsis "Tree-sitter Javascript grammar")
+ (description
+ "This package provides a Javascript grammar for the Tree-sitter
+library.")
+ (license license:expat)))
--
2.33.0
P
P
Pierre Langlois wrote on 29 Aug 2021 12:45
[PATCH v2 20/33] gnu: Add tree-sitter-json.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20210829104608.3103-20-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-json): New variable.
---
gnu/packages/tree-sitter.scm | 20 ++++++++++++++++++++
1 file changed, 20 insertions(+)

Toggle diff (28 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 2be6b15cb0..74249b2aca 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -384,3 +384,23 @@ This package includes the @command{tree-sitter} command-line tool.")
"This package provides a Javascript grammar for the Tree-sitter
library.")
(license license:expat)))
+
+(define-public tree-sitter-json
+ (package (inherit tree-sitter-c)
+ (name "tree-sitter-json")
+ (version "0.19.0")
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-json")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "06pjh31bv9ja9hlnykk257a6zh8bsxg2fqa54al7qk1r4n9ksnff"))))
+ (home-page "https://github.com/tree-sitter/tree-sitter-json")
+ (synopsis "Tree-sitter JSON grammar")
+ (description
+ "This package provides a JSON grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.33.0
P
P
Pierre Langlois wrote on 29 Aug 2021 12:46
[PATCH v2 27/33] gnu: Add rust-tree-sitter.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20210829104608.3103-27-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (rust-tree-sitter): New variable.
---
gnu/packages/tree-sitter.scm | 27 +++++++++++++++++++++++++++
1 file changed, 27 insertions(+)

Toggle diff (45 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 001322dd84..3357fce4a5 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -21,6 +21,7 @@
#:use-module (guix build-system cargo)
#:use-module (guix build-system gnu)
#:use-module (guix build-system node)
+ #:use-module (guix download)
#:use-module (guix git-download)
#:use-module (guix packages)
#:use-module (guix utils)
@@ -134,6 +135,32 @@ can be embedded in any application.
This package includes the @command{tree-sitter} command-line tool.")
(license license:expat)))

+(define-public rust-tree-sitter-0.19
+ (package
+ (name "rust-tree-sitter")
+ (version "0.19.5")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (crate-uri "tree-sitter" version))
+ (file-name (string-append name "-" version ".tar.gz"))
+ (sha256
+ (base32
+ "1h6adq5kqf4izzsklch5lfxx2aisxga463zz7w44rgwnck16wwmd"))))
+ (build-system cargo-build-system)
+ (arguments
+ `(#:tests? #f ;; Running tests misinterprets comments as doc-tests.
+ #:cargo-inputs
+ (("rust-cc" ,rust-cc-1)
+ ("rust-lazy-static" ,rust-lazy-static-1)
+ ("rust-regex" ,rust-regex-1)
+ ("rust-spin" ,rust-spin-0.7))))
+ (home-page "https://tree-sitter.github.io/tree-sitter/")
+ (synopsis "Rust bindings to the Tree-sitter parsing library")
+ (description "This package provides Rust bindings to the Tree-sitter
+parsing library.")
+ (license license:expat)))
+
(define-public tree-sitter-c
(package
(name "tree-sitter-c")
--
2.33.0
P
P
Pierre Langlois wrote on 29 Aug 2021 12:45
[PATCH v2 07/33] gnu: Add tree-sitter.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20210829104608.3103-7-pierre.langlois@gmx.com
* gnu/package/tree-sitter.scm: New file.
* gnu/local.mk (GNU_SYSTEM_MODULES): Register it.
---
gnu/local.mk | 1 +
gnu/packages/tree-sitter.scm | 73 ++++++++++++++++++++++++++++++++++++
2 files changed, 74 insertions(+)
create mode 100644 gnu/packages/tree-sitter.scm

Toggle diff (91 lines)
diff --git a/gnu/local.mk b/gnu/local.mk
index a0da1a96d8..0315106c30 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -559,6 +559,7 @@ GNU_SYSTEM_MODULES = \
%D%/packages/tls.scm \
%D%/packages/tmux.scm \
%D%/packages/tor.scm \
+ %D%/packages/tree-sitter.scm \
%D%/packages/tv.scm \
%D%/packages/uglifyjs.scm \
%D%/packages/uml.scm \
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
new file mode 100644
index 0000000000..e6e66cd447
--- /dev/null
+++ b/gnu/packages/tree-sitter.scm
@@ -0,0 +1,73 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2021 Pierre Langlois <pierre.langlois@gmx.com>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu packages tree-sitter)
+ #:use-module ((guix licenses) #:prefix license:)
+ #:use-module (guix build-system gnu)
+ #:use-module (guix git-download)
+ #:use-module (guix packages)
+ #:use-module (guix utils))
+
+(define-public tree-sitter
+ (package
+ (name "tree-sitter")
+ (version "0.19.5")
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "1qmb0sva28zv6r3c3j7xs9pc8bpwwhkb9vxxndw2zbdn9wkvmbmn"))))
+ (build-system gnu-build-system)
+ (arguments
+ `(#:tests? #f ;; No check target.
+ #:phases
+ (modify-phases %standard-phases
+ (delete 'configure)
+ (add-before 'build 'set-cc
+ (lambda _
+ (setenv "CC" ,(cc-for-target))))
+ (replace 'install
+ (lambda* (#:key outputs #:allow-other-keys)
+ (let* ((out (assoc-ref outputs "out"))
+ (lib (string-append out "/lib")))
+ (setenv "PREFIX" out)
+ (invoke "make" "install")))))))
+ (home-page "https://tree-sitter.github.io/tree-sitter/")
+ (synopsis "Incremental parsing system for programming tools")
+ (description "Tree-sitter is a parser generator tool and an incremental
+parsing library. It can build a concrete syntax tree for a source file and
+efficiently update the syntax tree as the source file is edited.
+
+Tree-sitter aims to be:
+
+@enumerate
+@item General enough to parse any programming language.
+@item Fast enough to parse on every keystroke in a text editor.
+@item Robust enough to provide useful results even in the presence of syntax
+errors.
+@item Dependency-free so that the runtime library (which is written in pure C)
+can be embedded in any application.
+@end enumerate
+
+This package includes the @code{libtree-sitter} runtime library.")
+ (license license:expat)))
--
2.33.0
P
P
Pierre Langlois wrote on 29 Aug 2021 12:46
[PATCH v2 28/33] gnu: rust-emacs-module: Update to 0.16.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20210829104608.3103-28-pierre.langlois@gmx.com
* gnu/packages/crates-io.scm (rust-emacs-module-0.16): New variable.
(rust-emacs-module-0.10): Inherit from rust-emacs-module-0.10.
---
gnu/packages/crates-io.scm | 25 +++++++++++++++++++++----
1 file changed, 21 insertions(+), 4 deletions(-)

Toggle diff (55 lines)
diff --git a/gnu/packages/crates-io.scm b/gnu/packages/crates-io.scm
index f769a8e9a5..e4892d5852 100644
--- a/gnu/packages/crates-io.scm
+++ b/gnu/packages/crates-io.scm
@@ -14697,10 +14697,10 @@ Emacs' support for dynamic modules.")
(description "This package provides proc macros for Emacs modules.")
(license license:bsd-3)))

-(define-public rust-emacs-module-0.10
+(define-public rust-emacs-module-0.16
(package
(name "rust-emacs-module")
- (version "0.10.0")
+ (version "0.16.2")
(source
(origin
(method url-fetch)
@@ -14709,11 +14709,11 @@ Emacs' support for dynamic modules.")
(string-append name "-" version ".tar.gz"))
(sha256
(base32
- "1gf9lz735xbkyir53dyv362drfx3nin5an5cx39kd8q8kjjwix5g"))))
+ "0s8qrh0ggjmqr04zkcf7s4ijmpd44rjcag78npnq64jv10lxvsry"))))
(build-system cargo-build-system)
(arguments
`(#:cargo-inputs
- (("rust-bindgen" ,rust-bindgen-0.48))))
+ (("rust-bindgen" ,rust-bindgen-0.56))))
(inputs
`(("clang" ,clang)))
(home-page "https://github.com/ubolonton/emacs-module-rs")
@@ -14722,6 +14722,23 @@ Emacs' support for dynamic modules.")
Emacs' support for dynamic modules.")
(license license:bsd-3)))

+(define-public rust-emacs-module-0.10
+ (package (inherit rust-emacs-module-0.16)
+ (name "rust-emacs-module")
+ (version "0.10.0")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (crate-uri "emacs_module" version))
+ (file-name
+ (string-append name "-" version ".tar.gz"))
+ (sha256
+ (base32
+ "1gf9lz735xbkyir53dyv362drfx3nin5an5cx39kd8q8kjjwix5g"))))
+ (arguments
+ `(#:cargo-inputs
+ (("rust-bindgen" ,rust-bindgen-0.48))))))
+
(define-public rust-embed-resource-1
(package
(name "rust-embed-resource")
--
2.33.0
P
P
Pierre Langlois wrote on 29 Aug 2021 12:46
[PATCH v2 31/33] gnu: Add emacs-tree-sitter-core.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20210829104608.3103-31-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-emacs-module): New local variable.
(emacs-tree-sitter-core): New variable.
---
gnu/packages/tree-sitter.scm | 79 ++++++++++++++++++++++++++++++++++++
1 file changed, 79 insertions(+)

Toggle diff (101 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 3357fce4a5..13f7a63acc 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -19,6 +19,7 @@
(define-module (gnu packages tree-sitter)
#:use-module ((guix licenses) #:prefix license:)
#:use-module (guix build-system cargo)
+ #:use-module (guix build-system emacs)
#:use-module (guix build-system gnu)
#:use-module (guix build-system node)
#:use-module (guix download)
@@ -28,6 +29,7 @@
#:use-module (gnu packages algebra)
#:use-module (gnu packages crates-graphics)
#:use-module (gnu packages crates-io)
+ #:use-module (gnu packages llvm)
#:use-module (gnu packages node-xyz))

(define-public tree-sitter
@@ -603,3 +605,80 @@ library.")
"This package provides Typescript and TSX grammars for the Tree-sitter
library.")
(license license:expat)))
+
+;; Local package definition solely for building the native emacs module
+;; written in Rust.
+(define tree-sitter-emacs-module
+ (package
+ (name "tree-sitter-emacs-module")
+ (version "0.15.1")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/emacs-tree-sitter/elisp-tree-sitter")
+ (commit version)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "15y0wjnck8rbfhl0xrl71ci7clbcp11lhqil5l8ykprsdjv0c2as"))))
+ (build-system cargo-build-system)
+ (inputs
+ `(("clang" ,clang)))
+ (arguments
+ `(#:cargo-inputs
+ (("rust-anyhow" ,rust-anyhow-1)
+ ("rust-emacs" ,rust-emacs-0.17)
+ ("rust-libloading" ,rust-libloading-0.7)
+ ("rust-once-cell" ,rust-once-cell-1)
+ ("rust-tree-sitter" ,rust-tree-sitter-0.19))
+ #:phases
+ (modify-phases %standard-phases
+ (add-after 'unpack 'delete-cargo.lock
+ (lambda _ (delete-file "Cargo.lock")))
+ (add-after 'delete-cargo.lock 'do-not-fetch-from-github
+ (lambda _
+ (substitute* "Cargo.toml"
+ (("\\[patch.*") "")
+ (("git = .*") ""))))
+ (add-after 'do-not-fetch-from-github 'chdir
+ (lambda _ (chdir "core")))
+ (replace 'install
+ (lambda* (#:key outputs #:allow-other-keys)
+ (let ((lib (string-append (assoc-ref outputs "out") "/lib")))
+ (install-file "../target/release/libtsc_dyn.so" lib)))))))
+ (home-page #f)
+ (synopsis #f)
+ (description #f)
+ (license license:expat)))
+
+(define-public emacs-tree-sitter-core
+ (package
+ (name "emacs-tree-sitter-core")
+ (version (package-version tree-sitter-emacs-module))
+ (source (package-source tree-sitter-emacs-module))
+ (build-system emacs-build-system)
+ (native-inputs
+ `(("tree-sitter-emacs-module" ,tree-sitter-emacs-module)))
+ (arguments
+ `(#:phases
+ (modify-phases %standard-phases
+ (add-after 'unpack 'chdir
+ (lambda _ (chdir "core")))
+ (add-after 'install 'install-module
+ (lambda* (#:key inputs outputs #:allow-other-keys)
+ (let ((elpa (elpa-directory (assoc-ref outputs "out")))
+ (module (string-append (assoc-ref inputs "tree-sitter-emacs-module")
+ "/lib/libtsc_dyn.so")))
+ ;; Writing "LOCAL" in this file prevents the package from
+ ;; trying to download the module from the internet.
+ (call-with-output-file (string-append elpa "/DYN-VERSION")
+ (lambda (port) (display "LOCAL" port)))
+ (substitute* "tsc-dyn-get.el"
+ (("defcustom tsc-dyn-dir tsc--dir")
+ (string-append "defcustom tsc-dyn-dir \"" elpa "\"")))
+ (copy-file module (string-append elpa "/tsc-dyn.so"))))))))
+ (home-page "https://github.com/emacs-tree-sitter/elisp-tree-sitter")
+ (synopsis "Tree-sitter bindings for Emacs Lisp, core library")
+ (description "This package provides core APIs of the Emacs binding for
+Tree-sitter, an incremental parsing system.")
+ (license license:expat)))
--
2.33.0
P
P
Pierre Langlois wrote on 29 Aug 2021 12:45
[PATCH v2 23/33] gnu: Add tree-sitter-python.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20210829104608.3103-23-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-python): New variable.
---
gnu/packages/tree-sitter.scm | 29 +++++++++++++++++++++++++++++
1 file changed, 29 insertions(+)

Toggle diff (37 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index d717bbf7b0..ae8bfcd9ff 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -462,3 +462,32 @@ library.")
(description
"This package provides a PHP grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-python
+ (package (inherit tree-sitter-c)
+ (name "tree-sitter-python")
+ (version "0.19.0")
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-python")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "04b85qxqs64x6nhbpcgrzkbilxaiwvr9yd3h065rynv7rsdg0hii"))))
+ (arguments
+ (substitute-keyword-arguments (package-arguments tree-sitter-c)
+ ((#:phases phases)
+ `(modify-phases ,phases
+ ;; The parse-example script tries to clone git repositories.
+ (add-after 'unpack 'remove-parse-example
+ (lambda _
+ (substitute* "package.json"
+ ((" && script\\/parse-examples") ""))))))))
+ (home-page "https://github.com/tree-sitter/tree-sitter-python")
+ (synopsis "Tree-sitter Python grammar")
+ (description
+ "This package provides a Python grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.33.0
P
P
Pierre Langlois wrote on 29 Aug 2021 12:45
[PATCH v2 21/33] gnu: Add tree-sitter-julia.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20210829104608.3103-21-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-julia): New variable.
---
gnu/packages/tree-sitter.scm | 29 +++++++++++++++++++++++++++++
1 file changed, 29 insertions(+)

Toggle diff (37 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 74249b2aca..36831b00b6 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -404,3 +404,32 @@ library.")
(description
"This package provides a JSON grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-julia
+ (package (inherit tree-sitter-c)
+ (name "tree-sitter-julia")
+ (version "0.19.0")
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-julia")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "1pbnmvhy2gq4vg1b0sjzmjm4s2gsgdjh7h01yj8qrrqbcl29c463"))))
+ (arguments
+ (substitute-keyword-arguments (package-arguments tree-sitter-c)
+ ((#:phases phases)
+ `(modify-phases ,phases
+ ;; The parse-example script tries to clone git repositories.
+ (add-after 'unpack 'remove-parse-example
+ (lambda _
+ (substitute* "package.json"
+ ((" && script\\/parse-examples") ""))))))))
+ (home-page "https://github.com/tree-sitter/tree-sitter-julia")
+ (synopsis "Tree-sitter Julia grammar")
+ (description
+ "This package provides a Julia grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.33.0
P
P
Pierre Langlois wrote on 29 Aug 2021 12:45
[PATCH v2 24/33] gnu: Add tree-sitter-rust.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20210829104608.3103-24-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-rust): New variable.
---
gnu/packages/tree-sitter.scm | 25 +++++++++++++++++++++++++
1 file changed, 25 insertions(+)

Toggle diff (40 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index ae8bfcd9ff..8456a15607 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -24,6 +24,7 @@
#:use-module (guix git-download)
#:use-module (guix packages)
#:use-module (guix utils)
+ #:use-module (gnu packages algebra)
#:use-module (gnu packages crates-graphics)
#:use-module (gnu packages crates-io)
#:use-module (gnu packages node-xyz))
@@ -491,3 +492,27 @@ library.")
(description
"This package provides a Python grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-rust
+ (package (inherit tree-sitter-c)
+ (name "tree-sitter-rust")
+ (version "0.19.1")
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-rust")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "118vkhv7n3sw8y9pi0987cgdcd74sjqwviijw01mhnk3bkyczi3l"))))
+ (native-inputs
+ `(("bc" ,bc)
+ ("tree-sitter-cli" ,tree-sitter-cli)
+ ("node-nan" ,node-nan)))
+ (home-page "https://github.com/tree-sitter/tree-sitter-rust")
+ (synopsis "Tree-sitter Rust grammar")
+ (description
+ "This package provides a Rust grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.33.0
P
P
Pierre Langlois wrote on 29 Aug 2021 12:45
[PATCH v2 22/33] gnu: Add tree-sitter-php.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20210829104608.3103-22-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-php): New variable.
---
gnu/packages/tree-sitter.scm | 29 +++++++++++++++++++++++++++++
1 file changed, 29 insertions(+)

Toggle diff (37 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 36831b00b6..d717bbf7b0 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -433,3 +433,32 @@ library.")
(description
"This package provides a Julia grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-php
+ (package (inherit tree-sitter-c)
+ (name "tree-sitter-php")
+ (version "0.19.0")
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-php")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "17cmybgpprw7w9d2v7lmc6zmr90d70g0jqq279gzg0mpwfzla53s"))))
+ (arguments
+ (substitute-keyword-arguments (package-arguments tree-sitter-c)
+ ((#:phases phases)
+ `(modify-phases ,phases
+ ;; The parse-example script tries to clone git repositories.
+ (add-after 'unpack 'remove-parse-example
+ (lambda _
+ (substitute* "package.json"
+ ((" && script\\/parse-examples") ""))))))))
+ (home-page "https://github.com/tree-sitter/tree-sitter-php")
+ (synopsis "Tree-sitter PHP grammar")
+ (description
+ "This package provides a PHP grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.33.0
P
P
Pierre Langlois wrote on 29 Aug 2021 12:46
[PATCH v2 25/33] gnu: Add tree-sitter-typescript.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20210829104608.3103-25-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-typescript): New variable.
---
gnu/packages/tree-sitter.scm | 60 ++++++++++++++++++++++++++++++++++++
1 file changed, 60 insertions(+)

Toggle diff (68 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 8456a15607..001322dd84 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -516,3 +516,63 @@ library.")
(description
"This package provides a Rust grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-typescript
+ (package (inherit tree-sitter-c)
+ (name "tree-sitter-typescript")
+ (version "0.19.0")
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-typescript")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "01pkmwwmbv6kxda0n6g4cfg72ldmpi9gmp11a5gygn472vhrh2xw"))))
+ (native-inputs
+ `(("tree-sitter-cli" ,tree-sitter-cli)
+ ("tree-sitter-javascript" ,tree-sitter-javascript)
+ ("node-nan" ,node-nan)))
+ (arguments
+ (substitute-keyword-arguments (package-arguments tree-sitter-c)
+ ((#:phases phases)
+ `(modify-phases ,phases
+ ;; The parse-example script tries to clone git repositories.
+ (add-after 'unpack 'remove-parse-example
+ (lambda _
+ (substitute* "package.json"
+ ((" && script\\/parse-examples") ""))))
+ (add-after 'build 'build-gyp
+ (lambda* (#:key inputs #:allow-other-keys)
+ (let ((node (assoc-ref inputs "node")))
+ (invoke (string-append node "/lib/node_modules/npm/node_modules"
+ "/node-gyp/bin/node-gyp.js")
+ "build"))))
+ (replace 'make-files-writable
+ (lambda _
+ (for-each make-file-writable
+ (find-files "typescript" ".*"))
+ (for-each make-file-writable
+ (find-files "tsx" ".*"))))
+ (replace 'install-native-lib
+ (lambda* (#:key outputs #:allow-other-keys)
+ (let ((lib (string-append (assoc-ref outputs "out")
+ "/lib/tree-sitter")))
+ ;; Invoking `tree-sitter test' loads the grammar which
+ ;; compiles it to a .so binary that we install.
+ (with-directory-excursion "typescript"
+ (invoke "tree-sitter" "test"))
+ (with-directory-excursion "tsx"
+ (invoke "tree-sitter" "test"))
+ (mkdir-p lib)
+ (copy-recursively (string-append (getenv "HOME")
+ "/.tree-sitter/bin")
+ lib))))))))
+ (home-page "https://github.com/tree-sitter/tree-sitter-typescript")
+ (synopsis "Tree-sitter Typescript grammar")
+ (description
+ "This package provides Typescript and TSX grammars for the Tree-sitter
+library.")
+ (license license:expat)))
--
2.33.0
P
P
Pierre Langlois wrote on 29 Aug 2021 12:46
[PATCH v2 26/33] gnu: Add rust-bindgen@0.56.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20210829104608.3103-26-pierre.langlois@gmx.com
* gnu/packages/crates-io.scm (rust-bindgen-0.56): New variable.
---
gnu/packages/crates-io.scm | 37 +++++++++++++++++++++++++++++++++++++
1 file changed, 37 insertions(+)

Toggle diff (48 lines)
diff --git a/gnu/packages/crates-io.scm b/gnu/packages/crates-io.scm
index 476d35c241..f769a8e9a5 100644
--- a/gnu/packages/crates-io.scm
+++ b/gnu/packages/crates-io.scm
@@ -4541,6 +4541,43 @@ bindings to C and C++ libraries.")
("rust-shlex" ,rust-shlex-0.1)
("rust-which" ,rust-which-3))))))

+(define-public rust-bindgen-0.56
+ (package
+ (inherit rust-bindgen-0.57)
+ (name "rust-bindgen")
+ (version "0.56.0")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (crate-uri "bindgen" version))
+ (file-name (string-append name "-" version ".tar.gz"))
+ (sha256
+ (base32
+ "0fajmgk2064ca1z9iq1jjkji63qwwz38z3d67kv6xdy0xgdpk8rd"))))
+ (arguments
+ `(#:skip-build? #t
+ #:cargo-inputs
+ (("rust-bitflags" ,rust-bitflags-1)
+ ("rust-cexpr" ,rust-cexpr-0.4)
+ ("rust-cfg-if" ,rust-cfg-if-0.1)
+ ("rust-clang-sys" ,rust-clang-sys-1)
+ ("rust-clap" ,rust-clap-2)
+ ("rust-env-logger" ,rust-env-logger-0.8)
+ ("rust-lazy-static" ,rust-lazy-static-1)
+ ("rust-lazycell" ,rust-lazycell-1)
+ ("rust-log" ,rust-log-0.4)
+ ("rust-peeking-take-while" ,rust-peeking-take-while-0.1)
+ ("rust-proc-macro2" ,rust-proc-macro2-1)
+ ("rust-quote" ,rust-quote-1)
+ ("rust-regex" ,rust-regex-1)
+ ("rust-rustc-hash" ,rust-rustc-hash-1)
+ ("rust-shlex" ,rust-shlex-0.1)
+ ("rust-which" ,rust-which-3))
+ #:cargo-development-inputs
+ (("rust-clap" ,rust-clap-2)
+ ("rust-diff" ,rust-diff-0.1)
+ ("rust-shlex" ,rust-shlex-0.1))))))
+
(define-public rust-bindgen-0.55
(package
(inherit rust-bindgen-0.57)
--
2.33.0
P
P
Pierre Langlois wrote on 29 Aug 2021 12:46
[PATCH v2 29/33] gnu: rust-emacs-macros: Update to 0.17.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20210829104608.3103-29-pierre.langlois@gmx.com
* gnu/packages/crates-io.scm (rust-emacs-macros-0.17): New variable.
(rust-emacs-macros-0.11): Inherit from rust-emacs-macros-0.17.
---
gnu/packages/crates-io.scm | 32 +++++++++++++++++++++++++-------
1 file changed, 25 insertions(+), 7 deletions(-)

Toggle diff (55 lines)
diff --git a/gnu/packages/crates-io.scm b/gnu/packages/crates-io.scm
index e4892d5852..d7375d7e56 100644
--- a/gnu/packages/crates-io.scm
+++ b/gnu/packages/crates-io.scm
@@ -14673,8 +14673,31 @@ signing, and verification in pure Rust.")
Emacs' support for dynamic modules.")
(license license:bsd-3)))

-(define-public rust-emacs-macros-0.11
+(define-public rust-emacs-macros-0.17
(package
+ (name "rust-emacs-macros")
+ (version "0.17.0")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (crate-uri "emacs-macros" version))
+ (file-name (string-append name "-" version ".tar.gz"))
+ (sha256
+ (base32
+ "0qg1dcn5acbirq617qq2fgg9adswif2dnr292s3qnq62wzgnyrb9"))))
+ (build-system cargo-build-system)
+ (arguments
+ `(#:cargo-inputs
+ (("rust-darling" ,rust-darling-0.10)
+ ("rust-quote" ,rust-quote-1)
+ ("rust-syn" ,rust-syn-1))))
+ (home-page "https://github.com/ubolonton/emacs-module-rs")
+ (synopsis "Proc macros for Emacs modules")
+ (description "This package provides proc macros for Emacs modules.")
+ (license license:bsd-3)))
+
+(define-public rust-emacs-macros-0.11
+ (package (inherit rust-emacs-macros-0.17)
(name "rust-emacs-macros")
(version "0.11.0")
(source
@@ -14686,16 +14709,11 @@ Emacs' support for dynamic modules.")
(sha256
(base32
"0390y8vafxdi334hhgrzvcqjq3n5ckcmvilqcfp8vajjq8irrly6"))))
- (build-system cargo-build-system)
(arguments
`(#:cargo-inputs
(("rust-darling" ,rust-darling-0.9)
("rust-quote" ,rust-quote-0.6)
- ("rust-syn" ,rust-syn-0.15))))
- (home-page "https://github.com/ubolonton/emacs-module-rs")
- (synopsis "Proc macros for Emacs modules")
- (description "This package provides proc macros for Emacs modules.")
- (license license:bsd-3)))
+ ("rust-syn" ,rust-syn-0.15))))))

(define-public rust-emacs-module-0.16
(package
--
2.33.0
P
P
Pierre Langlois wrote on 29 Aug 2021 12:46
[PATCH v2 32/33] gnu: Add emacs-tree-sitter.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20210829104608.3103-32-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (emacs-tree-sitter): New variable.
---
gnu/packages/tree-sitter.scm | 27 +++++++++++++++++++++++++++
1 file changed, 27 insertions(+)

Toggle diff (35 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 13f7a63acc..0454d75ad4 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -682,3 +682,30 @@ library.")
(description "This package provides core APIs of the Emacs binding for
Tree-sitter, an incremental parsing system.")
(license license:expat)))
+
+(define-public emacs-tree-sitter
+ (package
+ (name "emacs-tree-sitter")
+ (version (package-version emacs-tree-sitter-core))
+ (source (package-source emacs-tree-sitter-core))
+ (build-system emacs-build-system)
+ (propagated-inputs
+ `(("emacs-tree-sitter-core" ,emacs-tree-sitter-core)))
+ (arguments
+ `(#:phases
+ (modify-phases %standard-phases
+ (add-after 'unpack 'chdir
+ (lambda _ (chdir "lisp"))))))
+ (home-page "https://github.com/emacs-tree-sitter/elisp-tree-sitter")
+ (synopsis "Tree-sitter bindings for Emacs Lisp")
+ (description "This package provides Emacs bindings for Tree-sitter, an
+incremental parsing library. It aims to be the foundation for a new breed of
+Emacs packages that understand code structurally. For example:
+
+@enumerate
+@item Faster, fine-grained code highlighting.
+@item More flexible code folding.
+@item Structural editing (like Paredit, or even better) for non-Lisp code.
+@item More informative indexing for imenu.
+@end enumerate")
+ (license license:expat)))
--
2.33.0
P
P
Pierre Langlois wrote on 29 Aug 2021 12:46
[PATCH v2 33/33] gnu: Add emacs-tree-sitter-langs.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20210829104608.3103-33-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (emacs-tree-sitter-langs): New variable.
---
gnu/packages/tree-sitter.scm | 80 ++++++++++++++++++++++++++++++++++++
1 file changed, 80 insertions(+)

Toggle diff (88 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 0454d75ad4..6bb52b68ac 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -709,3 +709,83 @@ Emacs packages that understand code structurally. For example:
@item More informative indexing for imenu.
@end enumerate")
(license license:expat)))
+
+(define-public emacs-tree-sitter-langs
+ (package
+ (name "emacs-tree-sitter-langs")
+ (version "0.10.3")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/emacs-tree-sitter/tree-sitter-langs")
+ (commit version)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "0fa0xi8sky5h3rscnvcci9djdi71qj30vivzzvx7w8pqkmsy1vq7"))))
+ (build-system emacs-build-system)
+ ;; FIXME: Support for some languages, such as bash, is still missing from
+ ;; the bundle.
+ (inputs
+ `(("tree-sitter-bash" ,tree-sitter-bash)
+ ("tree-sitter-c" ,tree-sitter-c)
+ ("tree-sitter-cpp" ,tree-sitter-cpp)
+ ("tree-sitter-css" ,tree-sitter-css)
+ ("tree-sitter-go" ,tree-sitter-go)
+ ("tree-sitter-html" ,tree-sitter-html)
+ ("tree-sitter-java" ,tree-sitter-java)
+ ("tree-sitter-javascript" ,tree-sitter-javascript)
+ ("tree-sitter-json" ,tree-sitter-json)
+ ("tree-sitter-julia" ,tree-sitter-julia)
+ ("tree-sitter-php" ,tree-sitter-php)
+ ("tree-sitter-python" ,tree-sitter-python)
+ ("tree-sitter-rust" ,tree-sitter-rust)
+ ("tree-sitter-typescript" ,tree-sitter-typescript)))
+ (propagated-inputs
+ `(("emacs-tree-sitter" ,emacs-tree-sitter)))
+ (arguments
+ `(#:phases
+ (modify-phases %standard-phases
+ (add-before 'build 'fake-bundle
+ (lambda _
+ ;; The BUNDLE-VERSION file prevents the package from
+ ;; downloading libraries at load time.
+ (call-with-output-file "bin/BUNDLE-VERSION"
+ (lambda (port) (display ,version port)))))
+ (add-after 'install 'install-libs
+ (lambda* (#:key outputs inputs #:allow-other-keys)
+ (let ((elpa (elpa-directory (assoc-ref outputs "out"))))
+ (define (install-lib lang)
+ (let ((input (assoc-ref inputs (string-append "tree-sitter-" lang))))
+ (copy-file (string-append input "/lib/tree-sitter/" lang ".so")
+ (string-append elpa "/bin/" lang ".so"))))
+ (mkdir-p (string-append elpa "/bin"))
+ ;; The BUNDLE-VERSION file prevents the package from
+ ;; downloading libraries at load time.
+ (call-with-output-file (string-append elpa "/bin/BUNDLE-VERSION")
+ (lambda (port) (display ,version port)))
+ (for-each (lambda (input)
+ (use-modules (ice-9 regex))
+ (cond
+ ((string-match "^tree-sitter-(.*)$" (car input))
+ => (lambda (m)
+ (install-lib (match:substring m 1))))))
+ inputs))))
+ (add-after 'install 'install-queries
+ (lambda* (#:key outputs #:allow-other-keys)
+ (let ((elpa (elpa-directory (assoc-ref outputs "out"))))
+ (copy-recursively "queries" (string-append elpa "/queries"))))))))
+ (home-page "https://ubolonton.github.io/emacs-tree-sitter/languages/")
+ (synopsis "Language support bundle for Tree-sitter")
+ (description "This package is a convenient language bundle for
+Tree-sitter. For each supported language, this package provides:
+
+@enumerate
+@item Pre-compiled grammar binaries.
+@item An optional highlights.scm file that provides highlighting patterns.
+This is mainly intended for major modes that are not aware of tree-sitter.
+@item Optional query patterns for other minor modes that provide high-level
+functionalities on top of tree-sitter, such as code folding, evil text
+objects, ...etc.
+@end enumerate")
+ (license license:expat)))
--
2.33.0
P
P
Pierre Langlois wrote on 29 Aug 2021 12:46
[PATCH v2 30/33] gnu: rust-emacs: Update to 0.17.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20210829104608.3103-30-pierre.langlois@gmx.com
* gnu/packages/crates-io.scm (rust-emacs-0.17): New variable.
(rust-emacs-0.11): Inherit from rust-emacs-0.17.
---
gnu/packages/crates-io.scm | 38 +++++++++++++++++++++++++++++++-------
1 file changed, 31 insertions(+), 7 deletions(-)

Toggle diff (70 lines)
diff --git a/gnu/packages/crates-io.scm b/gnu/packages/crates-io.scm
index d7375d7e56..18538650f8 100644
--- a/gnu/packages/crates-io.scm
+++ b/gnu/packages/crates-io.scm
@@ -14642,10 +14642,10 @@ signing, and verification in pure Rust.")
@code{Right} is a general purpose sum type with two cases.")
(license (list license:expat license:asl2.0))))

-(define-public rust-emacs-0.11
+(define-public rust-emacs-0.17
(package
(name "rust-emacs")
- (version "0.11.0")
+ (version "0.17.0")
(source
(origin
(method url-fetch)
@@ -14654,16 +14654,18 @@ signing, and verification in pure Rust.")
(string-append name "-" version ".tar.gz"))
(sha256
(base32
- "1xn6nr8m3p24irlmv3i7c9n95yb10qc6aikwqv99qlmhbky5x0z7"))))
+ "02f31apvz03dr0dwn5wyirdmqkk2d635iki0kmsh291xr6dysy2q"))))
(build-system cargo-build-system)
(arguments
`(#:cargo-inputs
- (("rust-ctor" ,rust-ctor-0.1)
- ("rust-emacs-macros" ,rust-emacs-macros-0.11)
- ("rust-emacs-module" ,rust-emacs-module-0.10)
+ (("rust-anyhow" ,rust-anyhow-1)
+ ("rust-ctor" ,rust-ctor-0.1)
+ ("rust-emacs-macros" ,rust-emacs-macros-0.17)
+ ("rust-emacs-module" ,rust-emacs-module-0.16)
("rust-failure" ,rust-failure-0.1)
("rust-failure-derive" ,rust-failure-derive-0.1)
- ("rust-lazy-static" ,rust-lazy-static-1))))
+ ("rust-once-cell" ,rust-once-cell-1)
+ ("rust-thiserror" ,rust-thiserror-1))))
(inputs
`(("clang" ,clang)))
(home-page "https://github.com/ubolonton/emacs-module-rs")
@@ -14673,6 +14675,28 @@ signing, and verification in pure Rust.")
Emacs' support for dynamic modules.")
(license license:bsd-3)))

+(define-public rust-emacs-0.11
+ (package (inherit rust-emacs-0.17)
+ (name "rust-emacs")
+ (version "0.11.0")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (crate-uri "emacs" version))
+ (file-name
+ (string-append name "-" version ".tar.gz"))
+ (sha256
+ (base32
+ "1xn6nr8m3p24irlmv3i7c9n95yb10qc6aikwqv99qlmhbky5x0z7"))))
+ (arguments
+ `(#:cargo-inputs
+ (("rust-ctor" ,rust-ctor-0.1)
+ ("rust-emacs-macros" ,rust-emacs-macros-0.11)
+ ("rust-emacs-module" ,rust-emacs-module-0.10)
+ ("rust-failure" ,rust-failure-0.1)
+ ("rust-failure-derive" ,rust-failure-derive-0.1)
+ ("rust-lazy-static" ,rust-lazy-static-1))))))
+
(define-public rust-emacs-macros-0.17
(package
(name "rust-emacs-macros")
--
2.33.0
P
P
Pierre Langlois wrote on 29 Aug 2021 12:53
Re: [PATCH v2 33/33] gnu: Add emacs-tree-sitter-langs.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
87a6l0zfll.fsf@gmx.com
Pierre Langlois <pierre.langlois@gmx.com> writes:

Toggle quote (31 lines)
> * gnu/packages/tree-sitter.scm (emacs-tree-sitter-langs): New variable.
> ---
> gnu/packages/tree-sitter.scm | 80 ++++++++++++++++++++++++++++++++++++
> 1 file changed, 80 insertions(+)
>
> diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
> index 0454d75ad4..6bb52b68ac 100644
> --- a/gnu/packages/tree-sitter.scm
> +++ b/gnu/packages/tree-sitter.scm
> @@ -709,3 +709,83 @@ Emacs packages that understand code structurally. For example:
> @item More informative indexing for imenu.
> @end enumerate")
> (license license:expat)))
> +
> +(define-public emacs-tree-sitter-langs
> + (package
> + (name "emacs-tree-sitter-langs")
> + (version "0.10.3")
> + (source (origin
> + (method git-fetch)
> + (uri (git-reference
> + (url "https://github.com/emacs-tree-sitter/tree-sitter-langs")
> + (commit version)))
> + (file-name (git-file-name name version))
> + (sha256
> + (base32
> + "0fa0xi8sky5h3rscnvcci9djdi71qj30vivzzvx7w8pqkmsy1vq7"))))
> + (build-system emacs-build-system)
> + ;; FIXME: Support for some languages, such as bash, is still missing from
> + ;; the bundle.

Whoops, I forgot to update the comment after adding support for bash,
I'll fix that locally!

Thanks,
Pierre
-----BEGIN PGP SIGNATURE-----

iQFMBAEBCgA2FiEEctU9gYy29KFyWDdMqPyeRH9PfVQFAmErZ0YYHHBpZXJyZS5s
YW5nbG9pc0BnbXguY29tAAoJEKj8nkR/T31UCNEIAJ5g/QxVLsVCxc0mteI/GGH8
9JJgBK7Vm/I1YctpnEWyllBqARrPz5n1lz8S3fVTbq6ULO6jM+uf8FEIerZWz57R
ds9qK+0LzioOxCEKW5o1p0ag3uasxeJjb1B0bOK9xa7zE6dnpPRQqzl0Itz9lAPd
julUBUsMT9LjdLlZjbiLhA2W36ebocPo3DiOU4iaMbO2ajt3DD99mJ+ZNdRS//7T
gY9xNnfIxugg4r6TH/XGOAph5NVjKzcmyqIinLen8fdgUlNUxlV1xZrXGz5YoCDV
jWAzxUT10y/T6kdUgSICYRQP6X0IY/WlijGCPIFiWpX9DmrJigq9dg2gAnI+xfE=
=lgHC
-----END PGP SIGNATURE-----

P
P
Philip McGrath wrote on 23 Sep 2021 11:18
Re: [bug#49946] [PATCH 08/31] gnu: node: Patch /usr/bin/env in node-gyp.
b6be6177-c9f4-22ec-87dc-a54a06ec48cf@philipmcgrath.com
I'm interested in the node-gyp part of this, which has come up in some
other software I'm trying to package. These comments come with the
caveat that my experience with node.js and npm is fairly shallow.

On 8/10/21 2:28 PM, Maxime Devos wrote:
Toggle quote (13 lines)
> Pierre Langlois schreef op ma 09-08-2021 om 00:33 [+0100]:
>> @@ -120,6 +120,10 @@
>> (("'/usr/bin/env'")
>> (string-append "'" (which "env") "'")))
>>
>> + ;; Fix /usr/bin/env shebang in node-gyp.
>> + (substitute* "deps/npm/node_modules/node-gyp/bin/node-gyp.js"
>> + (("#!/usr/bin/env") (string-append "#!" (which "env"))))
>
> For cross-compilation, this should most likely be
> (string-append (assoc-ref inputs "coreutils") "/bin/env")
> or something like that instead. Likewise in other places.

Since the shebang line for node-gyp is specifically "#!/usr/bin/env
node", I wonder if it should use the node built by this package, rather
than a dynamic node.

More generally, I see that there are 355 directories installed under
"lib/node_modules/npm/node_modules" (which corresponds to the "deps"
path above). Most of them don't seem to be available as Guix packages
that could be depended upon by other Guix node packages. I'd guess
node-gyp may not be the only one with shebangs that ought to be patched.

On 8/8/21 6:29 PM, Pierre Langlois wrote:

> ... `node-gyp' needs

> node headers to compile against, packaged as a tarball, which it tries

> to download. Instead, we can run a `node-gyp --tarball <> configure'

> step to manually provide the tarball, which we can package separately

> for any given node version.

There is also a --nodedir option, which I found could work with
something like:

(string-append "--nodedir=" (assoc-ref inputs "node"))

That seems like it might be better, though I don't know all the
considerations for cross-compilation and such.

-Philip
P
P
Pierre Langlois wrote on 25 Sep 2021 12:24
(name . Philip McGrath)(address . philip@philipmcgrath.com)
87o88gq5p5.fsf@gmx.com
Hi Philip,

Philip McGrath <philip@philipmcgrath.com> writes:

Toggle quote (4 lines)
> I'm interested in the node-gyp part of this, which has come up in some other
> software I'm trying to package. These comments come with the caveat that my
> experience with node.js and npm is fairly shallow.

Thanks for your feedback!

Toggle quote (18 lines)
>
> On 8/10/21 2:28 PM, Maxime Devos wrote:
>> Pierre Langlois schreef op ma 09-08-2021 om 00:33 [+0100]:
>>> @@ -120,6 +120,10 @@
>>> (("'/usr/bin/env'")
>>> (string-append "'" (which "env") "'")))
>>>
>>> + ;; Fix /usr/bin/env shebang in node-gyp.
>>> + (substitute* "deps/npm/node_modules/node-gyp/bin/node-gyp.js"
>>> + (("#!/usr/bin/env") (string-append "#!" (which "env"))))
>> For cross-compilation, this should most likely be
>> (string-append (assoc-ref inputs "coreutils") "/bin/env")
>> or something like that instead. Likewise in other places.
>
> Since the shebang line for node-gyp is specifically "#!/usr/bin/env node", I
> wonder if it should use the node built by this package, rather than a dynamic
> node.

Yeah we could do that, although I generally prefer to follow whatever
the script already does, there could be a good reason for them to use
`env' no?

Toggle quote (5 lines)
> More generally, I see that there are 355 directories installed under
> "lib/node_modules/npm/node_modules" (which corresponds to the "deps"
> path above). Most of them don't seem to be available as Guix packages that could
> be depended upon by other Guix node packages.

Yeah that's tricky, ideally we should remove all the node_modules deps
and package them separately, I wonder if anybody tried to do that
already. I would suspect it to be quite a lot of work, sometimes
unbundling stops being worth and when it's hard to maintain dependencies
manually.

Hopefully we can get there one day though! I don't want to deter anybody
from trying :-), I might give it a go on a raindy day.

Toggle quote (3 lines)
> I'd guess node-gyp may not be the only one with shebangs that ought to
> be patched.

Yeah there could be others, although normally the patching phase from
the gnu build system should have taken care of most of them, hopefully
all, I'm not sure why it didn't work for /usr/bin/env though.

I would suggest we patch things as we encounter them, did you find
anymore issues when working on your package?

For instance, while working on a newer version of one of the packages in
this series, I saw we may need to patch GYP's python reference as well,
like so:

(substitute* "deps/npm/node_modules/node-gyp/gyp/gyp_main.py"
(("#!/usr/bin/env python")
(string-append "#!" (assoc-ref inputs "python") "/bin/python3")))

Only for node 14+. The reason seems to be that gyp still refers to
"python", but python2 is no longer a dependency for newer nodes. And it
seems GYP is perfectly happy with python3, and the shebang is fixed
upstream so a never node will be fine:

Maybe updating node would be better than this fix though.

Toggle quote (19 lines)
> On 8/8/21 6:29 PM, Pierre Langlois wrote:
>
>> ... `node-gyp' needs
>
>> node headers to compile against, packaged as a tarball, which it tries
>
>> to download. Instead, we can run a `node-gyp --tarball <> configure'
>
>> step to manually provide the tarball, which we can package separately
>
>> for any given node version.
>
> There is also a --nodedir option, which I found could work with something like:
>
> (string-append "--nodedir=" (assoc-ref inputs "node"))
>
> That seems like it might be better, though I don't know all the considerations
> for cross-compilation and such.

Oh that's a good idea, I didn't really like having to download the
headers separately from the main package, especially given we run
snippet on the source to remove bundled dependencies.

Trying this out this approach does work, but I needed to:

- Create a union directory with both node and libuv. The node package
only has headers for V8/node, but we also need libuv, so doing
something like this works:

(union-build node-sources
(list (assoc-ref inputs "node")
(assoc-ref inputs "libuv"))
#:create-all-directories? #t
#:log-port (%make-void-port "w"))

- For some reason, --nodedir didn't really "configure" gyp to use that
node directory, I think it's meant to be passed everytime you run
any gyp command. Instead I found that you can use and environment
variable:

(setenv "npm_config_nodedir" node-sources)

And that works for the packages in this series! That'll be much better
than before, I'll do it this way.

Thanks again for taking a look, I'll see if I can send updated patches
sometimes this weekend.

Pierre
-----BEGIN PGP SIGNATURE-----

iQFMBAEBCgA2FiEEctU9gYy29KFyWDdMqPyeRH9PfVQFAmFPAacYHHBpZXJyZS5s
YW5nbG9pc0BnbXguY29tAAoJEKj8nkR/T31UX4cH/iBUk2LanVkCltAcn4bCP3vo
+L+wCv7vDSWeU0D0H3VIVCutk2x+6nUcSLch8hansMGKIaDnfAzuf7o60cl7MLfh
v3JfY20Y8yFPFhRV4xPKTUmFt55cyLBkBfjDjP8hALTaON8wASSm802vMgELgMx/
TyaSWrcW8ObXr9M+721FyTKeG4qtHECunEuDJvJ1tPEo7b+MRN9I+PwkmRM1Uqh7
1DxOBg9UOL4Jt2VoDRbqO584j/+p3A+9uGzuLJ4R/Va+FPvSXuImQy//YwkHyW7r
eRd5NWFo9ch+OsfJUhORAVKGThwWWwh2bT36OgtfMXCd7S4ko/EP56eOaZ2taK8=
=rNU6
-----END PGP SIGNATURE-----

P
P
Philip McGrath wrote on 27 Sep 2021 00:02
(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
e2aec7f8-5930-367a-2cd2-b9dddc352fa3@philipmcgrath.com
Hi Pierre,

On 9/25/21 6:24 AM, Pierre Langlois wrote:
Toggle quote (8 lines)
> Philip McGrath <philip@philipmcgrath.com> writes:
>> Since the shebang line for node-gyp is specifically "#!/usr/bin/env node", I
>> wonder if it should use the node built by this package, rather than a dynamic
>> node.
>
> Yeah we could do that, although I generally prefer to follow whatever
> the script already does, there could be a good reason for them to use
> `env' no
I think it might be better to use `patch-shebang` from `(guix build
utils)` rather than `substitute*` these by hand, and it seems that
`patch-shebang` removed the indirection through `env`. My guess is most
of these cases are to accommodate the fact that `node` and `python` are
often installed to places other than `/usr/bin`.

>> I'd guess node-gyp may not be the only one with shebangs that ought to

>> be patched.

>

> Yeah there could be others, although normally the patching phase from

> the gnu build system should have taken care of most of them, hopefully

> all, I'm not sure why it didn't work for /usr/bin/env though.

>

> I would suggest we patch things as we encounter them, did you find

> anymore issues when working on your package?


Looking at `gnu-build-system`, it seems that the `'patch-shebangs` phase
only operates on files installed in the "/bin" and "/sbin"
subdirectories of the package's outputs. That restriction doesn't make
sense to me in general: for instance, what about "/libexec"? For Node
specifically, this misses a lot of stuff under "/lib/node_modules" and
"/lib/node_modules/npm/node_modules". I think I more general fix could
subsume the `'patch-npm-shebang` and `'patch-node-shebang` phases in
building Node, too.

> For instance, while working on a newer version of one of the packages in

> this series, I saw we may need to patch GYP's python reference as well,

> like so:

>

> (substitute* "deps/npm/node_modules/node-gyp/gyp/gyp_main.py"

> (("#!/usr/bin/env python")

> (string-append "#!" (assoc-ref inputs "python") "/bin/python3")))

>

> Only for node 14+. The reason seems to be that gyp still refers to

> "python", but python2 is no longer a dependency for newer nodes. And it

> seems GYP is perfectly happy with python3, and the shebang is fixed

> upstream so a never node will be fine:



I think in some places (but perhaps not enough places) Guix uses
`python-wrapper` to work around this ...

>

> Maybe updating node would be better than this fix though.

I'm not totally clear on whether the upstream fix is in 14.17.6 LTS,
but, if so, that seems great!

Toggle quote (15 lines)
>
>> More generally, I see that there are 355 directories installed under
>> "lib/node_modules/npm/node_modules" (which corresponds to the "deps"
>> path above). Most of them don't seem to be available as Guix packages that could
>> be depended upon by other Guix node packages.
>
> Yeah that's tricky, ideally we should remove all the node_modules deps
> and package them separately, I wonder if anybody tried to do that
> already. I would suspect it to be quite a lot of work, sometimes
> unbundling stops being worth and when it's hard to maintain dependencies
> manually.
>
> Hopefully we can get there one day though! I don't want to deter anybody
> from trying :-), I might give it a go on a raindy day.

Since these are developed and released with Node, and apparently we can
build them as part of the Node build process, I was thinking we could
just make packages that point to these versions we're already building.
It might be good to hear from someone who develops with node/npm, though
... I just use it to install software that I can't find packaged elsewhere.

Toggle quote (36 lines)
>
>> On 8/8/21 6:29 PM, Pierre Langlois wrote:
>>
>>> ... `node-gyp' needs
>>
>>> node headers to compile against, packaged as a tarball, which it tries
>>
>>> to download. Instead, we can run a `node-gyp --tarball <> configure'
>>
>>> step to manually provide the tarball, which we can package separately
>>
>>> for any given node version.
>>
>> There is also a --nodedir option, which I found could work with something like:
>>
>> (string-append "--nodedir=" (assoc-ref inputs "node"))
>>
>> That seems like it might be better, though I don't know all the considerations
>> for cross-compilation and such.
>
> Oh that's a good idea, I didn't really like having to download the
> headers separately from the main package, especially given we run
> snippet on the source to remove bundled dependencies.
>
> Trying this out this approach does work, but I needed to:
>
> - Create a union directory with both node and libuv. The node package
> only has headers for V8/node, but we also need libuv, so doing
> something like this works:
>
> (union-build node-sources
> (list (assoc-ref inputs "node")
> (assoc-ref inputs "libuv"))
> #:create-all-directories? #t
> #:log-port (%make-void-port "w"))

I found it worked to just add libuv as an input of packages built with
node-gyp. I hadn't tried to change `node-build-system`, but I think that
would be the place to do it.

Toggle quote (8 lines)
>
> - For some reason, --nodedir didn't really "configure" gyp to use that
> node directory, I think it's meant to be passed everytime you run
> any gyp command. Instead I found that you can use and environment
> variable:
>
> (setenv "npm_config_nodedir" node-sources)

That seems right. I believe there's a similar "npm_config_python" for
the Python executable to use.

Alternatively, I think it's possible to configure these in

Toggle quote (7 lines)
>
> And that works for the packages in this series! That'll be much better
> than before, I'll do it this way.
>
> Thanks again for taking a look, I'll see if I can send updated patches
> sometimes this weekend.

Glad it was useful!

For patching the shebangs, here's a variant of node-lts that worked for
me, though I think it would be even better to combine it with the
existing phases:

```
(define-public patched-node
(let ((node node-lts))
(package
(inherit node)
(arguments
(substitute-keyword-arguments (package-arguments node)
((#:phases standard-phases)
`(modify-phases ,standard-phases
(add-after 'patch-npm-shebang 'patch-more-shebangs
(lambda* (#:key inputs outputs #:allow-other-keys)
(define (append-map f lst)
(apply append (map f lst)))
;; from patch-shebangs
(define bin-directories
;;(match-lambda
;; ((_ . dir)
(lambda (pr)
(let ((dir (cdr pr)))
(list (string-append dir "/bin")
(string-append dir "/sbin")))))
(define output-bindirs
(append-map bin-directories outputs))
(define input-bindirs
;; Shebangs should refer to binaries of the target
system---i.e., from
;; "inputs", not from "native-inputs".
(append-map bin-directories inputs))
(define path
(append output-bindirs input-bindirs))
(with-directory-excursion
(string-append (assoc-ref outputs "out")
"/lib/node_modules/npm/node_modules")
(for-each
;;(cut patch-shebang <> path)
(lambda (file)
(patch-shebang file path))
;; from patch-generated-file-shebangs
(find-files "."
(lambda (file stat)
(and (eq? 'regular (stat:type stat))
(not (zero? (logand (stat:mode
stat) #o100)))))
#:stat lstat))))))))))))
```

-Philip
M
M
Maxime Devos wrote on 27 Sep 2021 12:11
(address . 49946@debbugs.gnu.org)
592a9a571437c4c053ceeb27ed0512deeb0667c6.camel@telenet.be
Philip McGrath schreef op zo 26-09-2021 om 18:02 [-0400]:
Toggle quote (5 lines)
> Looking at `gnu-build-system`, it seems that the `'patch-shebangs` phase
> only operates on files installed in the "/bin" and "/sbin"
> subdirectories of the package's outputs. That restriction doesn't make
> sense to me in general: for instance, what about "/libexec"?

'libexec' is included on core-updates{,-frozen}. I believe the idea of the
restriction is to avoid patching too much. E.g., "autoconf" has a file
share/autoconf/build-aux/config.guess with a #!/bin/sh shebang. It should
not be patched, otherwise tarballs created with "make dist" would include
a store path and hence be Guix-specific and architecture-specific.

Greetings,
Maxime.
-----BEGIN PGP SIGNATURE-----

iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYVGYyRccbWF4aW1lZGV2
b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7s12AQD9RgeUKLsp/NrdpynQ5KtLzf0d
a1QqfgnU6s2rGOREIAD/SHKNqg5x/YfmpJfybEGqoFkKoZQtpzZdNBi2pOOgOQ4=
=JNjt
-----END PGP SIGNATURE-----


P
P
Philip McGrath wrote on 29 Sep 2021 06:45
(address . 49946@debbugs.gnu.org)
ac0288a7-fdc2-e43f-40ce-c778b6a474d4@philipmcgrath.com
On 9/27/21 6:11 AM, Maxime Devos wrote:
Toggle quote (12 lines)
> Philip McGrath schreef op zo 26-09-2021 om 18:02 [-0400]:
>> Looking at `gnu-build-system`, it seems that the `'patch-shebangs` phase
>> only operates on files installed in the "/bin" and "/sbin"
>> subdirectories of the package's outputs. That restriction doesn't make
>> sense to me in general: for instance, what about "/libexec"?
>
> 'libexec' is included on core-updates{,-frozen}. I believe the idea of the
> restriction is to avoid patching too much. E.g., "autoconf" has a file
> share/autoconf/build-aux/config.guess with a #!/bin/sh shebang. It should
> not be patched, otherwise tarballs created with "make dist" would include
> a store path and hence be Guix-specific and architecture-specific.

That makes some sense. I would have thought checking that the file is
executable would catch most such cases, but, if this works for
`gnu-build-system`, great.

As I look at potentially making a patch, another thing that seems odd is
that `(gnu packages node)` exports node@10.24.0 as `node` (via
`define-public`), but node@14.16.0 as `node-lts`. Normally, if I saw
that there were packages `node` and `node-lts`, I'd assume that
`node-lts` was *older*. It's especially confusing because, at the
command line, `guix install node` refers to what in Scheme you have to
write as `node-lts`.

I wonder if it was a mistake, and should have used `define` rather than
`define-public`, since this code:

```
;; This should be the latest version of node that still builds without
;; depending on llhttp.
(define-public node-bootstrap
(hidden-package node))
```

seems to be trying to hide the older node.

It looks like `node` has only a few dependents, and it seems like at
least several of them only used it because it had the more obvious name.
The `node-build-system` uses `node-lts` as the `(default-node)`.

Would it make sense to change the names? Or just to remove the
`define-public` of `node`?

-Philip
P
P
Philip McGrath wrote on 29 Sep 2021 08:31
(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
682f2ed3-90bd-a9f8-1193-34bef399ca62@philipmcgrath.com
On 9/25/21 6:24 AM, Pierre Langlois wrote:
Toggle quote (14 lines)
> For instance, while working on a newer version of one of the packages in
> this series, I saw we may need to patch GYP's python reference as well,
> like so:
>
> (substitute* "deps/npm/node_modules/node-gyp/gyp/gyp_main.py"
> (("#!/usr/bin/env python")
> (string-append "#!" (assoc-ref inputs "python") "/bin/python3")))
>
> Only for node 14+. The reason seems to be that gyp still refers to
> "python", but python2 is no longer a dependency for newer nodes. And it
> seems GYP is perfectly happy with python3, and the shebang is fixed
> upstream so a never node will be fine:
> https://github.com/nodejs/node-gyp/pull/2355/files

I think this needs to be a `python` from `inputs` rather than
`native-inputs`, for cross-compilation, IIUC.

I tried building node 14.18.0, and it ran into other issues, but there
were still a number shebangs with `python` rather than `python3` in
various places, though I think they'd be fine with `python-wrapper`.

-Philip
P
P
Philip McGrath wrote on 1 Oct 2021 00:56
[PATCH 0/3] guix: node-build-system: Support compiling addons with node-gyp.
(address . 49946@debbugs.gnu.org)
20210930225611.2143194-1-philip@philipmcgrath.com
Hi,

I've reworked the part of the patch series dealing with node-gyp.

I'd like to find an NPM addon package to submit as part of this series, too,
basically as a test case. If I can find one that doesn't raise too many other
complications, I may send this in as a separate patch, but feel free to try it
with tree-sitter, too.

There are a few things I'm still not sure about. I haven't made
node-build-system add libuv as an implicit input, because I think some
node-gyp addons don't actually need libuv, but maybe it's common enough that
it should be done automatically.

Likewise, I haven't tried to change the issue of `node` referring to
`node-bootstrap`, but I still think it should be changed.

These patches are also on GitLab at

Improvements welcome!

-Philip


Philip McGrath (3):
gnu: node: Avoid duplicating build phases.
gnu: node: Update to 10.24.1 for bootstrapping.
guix: node-build-system: Support compiling addons with node-gyp.

gnu/packages/node.scm | 187 ++++++++++---------------------
guix/build-system/node.scm | 7 +-
guix/build/node-build-system.scm | 9 ++
3 files changed, 74 insertions(+), 129 deletions(-)

--
2.30.2
P
P
Philip McGrath wrote on 1 Oct 2021 00:56
[PATCH 1/3] gnu: node: Avoid duplicating build phases.
(address . 49946@debbugs.gnu.org)
20210930225611.2143194-2-philip@philipmcgrath.com
* gnu/packages/node.scm (node)[arguments]: Split 'patch-files phase
into 'patch-hardcoded-program-references and
'delete-problematic-tests. Adapt those phases and 'configure to work
unmodified on node-lts.
(node, node-lts)[inputs]: Use bash-minimal rather than bash.
(node-lts)[arguments]: Inherit 'patch-hardcoded-program-references,
'delete-problemating-tests, and 'configure phases from the bootstrap
node. Remove the 'patch-files phase, keeping its remaining
non-inherited work in a new 'replace-llhttp-sources phase.
---
gnu/packages/node.scm | 150 +++++++++++-------------------------------
1 file changed, 39 insertions(+), 111 deletions(-)

Toggle diff (216 lines)
diff --git a/gnu/packages/node.scm b/gnu/packages/node.scm
index f8ac95884c..34c2bfa9d4 100644
--- a/gnu/packages/node.scm
+++ b/gnu/packages/node.scm
@@ -9,6 +9,7 @@
;;; Copyright © 2020, 2021 Pierre Langlois <pierre.langlois@gmx.com>
;;; Copyright © 2020 Ricardo Wurmus <rekado@elephly.net>
;;; Copyright © 2021 Simon Tournier <zimon.toutoune@gmail.com>
+;;; Copyright © 2021 Philip McGrath <philip@philipmcgrath.com>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -105,14 +106,22 @@
#:test-target "test-ci-js"
#:phases
(modify-phases %standard-phases
- (add-before 'configure 'patch-files
+ (add-before 'configure 'patch-hardcoded-program-references
(lambda* (#:key inputs #:allow-other-keys)
+
;; Fix hardcoded /bin/sh references.
- (substitute* '("lib/child_process.js"
- "lib/internal/v8_prof_polyfill.js"
- "test/parallel/test-child-process-spawnsync-shell.js"
- "test/parallel/test-stdio-closed.js"
- "test/sequential/test-child-process-emfile.js")
+ (substitute*
+ (let ((common
+ '("lib/child_process.js"
+ "lib/internal/v8_prof_polyfill.js"
+ "test/parallel/test-child-process-spawnsync-shell.js"
+ "test/parallel/test-stdio-closed.js"
+ "test/sequential/test-child-process-emfile.js"))
+ ;; not in bootstap node:
+ (sigxfsz "test/parallel/test-fs-write-sigxfsz.js"))
+ (if (file-exists? sigxfsz)
+ (cons sigxfsz common)
+ common))
(("'/bin/sh'")
(string-append "'" (assoc-ref inputs "bash") "/bin/sh'")))
@@ -122,18 +131,28 @@
"test/parallel/test-child-process-exec-env.js")
(("'/usr/bin/env'")
(string-append "'" (assoc-ref inputs "coreutils")
- "/bin/env'")))
+ "/bin/env'")))))
+ (add-after 'patch-hardcoded-program-references 'delete-problematic-tests
+ (lambda* (#:key inputs #:allow-other-keys)
+ (define (delete-file-if-exists pth)
+ (when (file-exists? pth)
+ (delete-file pth)))
;; FIXME: These tests fail in the build container, but they don't
;; seem to be indicative of real problems in practice.
- (for-each delete-file
+ (for-each delete-file-if-exists
'("test/parallel/test-cluster-master-error.js"
"test/parallel/test-cluster-master-kill.js"
;; See also <https://github.com/nodejs/node/issues/25903>.
+ ;; (no longer exists in node-lts)
"test/sequential/test-performance.js"))
- ;; This requires a DNS resolver.
- (delete-file "test/parallel/test-dns.js")
+ ;; These require a DNS resolver.
+ (for-each delete-file-if-exists
+ '("test/parallel/test-dns.js"
+ ;; not in the bootstrap node:
+ "test/parallel/test-dns-lookupService-promises.js"))
+
;; FIXME: This test fails randomly:
;; https://github.com/nodejs/node/issues/31213
@@ -217,9 +236,13 @@
(setenv "CXX" ,(cxx-for-target))
(setenv "PKG_CONFIG" ,(pkg-config-for-target))
(apply invoke
- (string-append (assoc-ref (or native-inputs inputs)
- "python")
- "/bin/python")
+ (let ((python
+ (string-append (assoc-ref (or native-inputs inputs)
+ "python")
+ "/bin/python")))
+ (if (file-exists? python)
+ python
+ (string-append python "3")))
"configure" flags))))
(add-after 'patch-shebangs 'patch-npm-shebang
(lambda* (#:key outputs #:allow-other-keys)
@@ -256,7 +279,7 @@
(variable "NODE_PATH")
(files '("lib/node_modules")))))
(inputs
- `(("bash" ,bash)
+ `(("bash" ,bash-minimal)
("coreutils" ,coreutils)
("c-ares" ,c-ares)
("http-parser" ,http-parser)
@@ -711,103 +734,8 @@ source files.")
libuv "/lib:"
zlib "/lib"
"'],"))))))
- (replace 'configure
- ;; Node's configure script is actually a python script, so we can't
- ;; run it with bash.
- (lambda* (#:key outputs (configure-flags '()) native-inputs inputs
- #:allow-other-keys)
- (let* ((prefix (assoc-ref outputs "out"))
- (xflags ,(if (%current-target-system)
- `'("--cross-compiling"
- ,(string-append
- "--dest-cpu="
- (match (%current-target-system)
- ((? (cut string-prefix? "arm" <>))
- "arm")
- ((? (cut string-prefix? "aarch64" <>))
- "arm64")
- ((? (cut string-prefix? "i686" <>))
- "ia32")
- ((? (cut string-prefix? "x86_64" <>))
- "x64")
- ((? (cut string-prefix? "powerpc64" <>))
- "ppc64")
- (_ "unsupported"))))
- ''()))
- (flags (cons
- (string-append "--prefix=" prefix)
- (append xflags configure-flags))))
- (format #t "build directory: ~s~%" (getcwd))
- (format #t "configure flags: ~s~%" flags)
- ;; Node's configure script expects the CC environment variable to
- ;; be set.
- (setenv "CC_host" "gcc")
- (setenv "CXX_host" "g++")
- (setenv "CC" ,(cc-for-target))
- (setenv "CXX" ,(cxx-for-target))
- (setenv "PKG_CONFIG" ,(pkg-config-for-target))
- (apply invoke
- (string-append (assoc-ref (or native-inputs inputs)
- "python")
- "/bin/python3")
- "configure" flags))))
- (replace 'patch-files
+ (add-after 'delete-problematic-tests 'replace-llhttp-sources
(lambda* (#:key inputs #:allow-other-keys)
- ;; Fix hardcoded /bin/sh references.
- (substitute* '("lib/child_process.js"
- "lib/internal/v8_prof_polyfill.js"
- "test/parallel/test-child-process-spawnsync-shell.js"
- "test/parallel/test-fs-write-sigxfsz.js"
- "test/parallel/test-stdio-closed.js"
- "test/sequential/test-child-process-emfile.js")
- (("'/bin/sh'")
- (string-append "'" (assoc-ref inputs "bash") "/bin/sh'")))
-
- ;; Fix hardcoded /usr/bin/env references.
- (substitute* '("test/parallel/test-child-process-default-options.js"
- "test/parallel/test-child-process-env.js"
- "test/parallel/test-child-process-exec-env.js")
- (("'/usr/bin/env'")
- (string-append "'" (assoc-ref inputs "coreutils")
- "/bin/env'")))
-
- ;; FIXME: These tests fail in the build container, but they don't
- ;; seem to be indicative of real problems in practice.
- (for-each delete-file
- '("test/parallel/test-cluster-master-error.js"
- "test/parallel/test-cluster-master-kill.js"))
-
- ;; These require a DNS resolver.
- (for-each delete-file
- '("test/parallel/test-dns.js"
- "test/parallel/test-dns-lookupService-promises.js"))
-
- ;; FIXME: This test fails randomly:
- ;; https://github.com/nodejs/node/issues/31213
- (delete-file "test/parallel/test-net-listen-after-destroying-stdin.js")
-
- ;; FIXME: These tests fail on armhf-linux:
- ;; https://github.com/nodejs/node/issues/31970
- ,@(if (target-arm32?)
- '((for-each delete-file
- '("test/parallel/test-zlib.js"
- "test/parallel/test-zlib-brotli.js"
- "test/parallel/test-zlib-brotli-flush.js"
- "test/parallel/test-zlib-brotli-from-brotli.js"
- "test/parallel/test-zlib-brotli-from-string.js"
- "test/parallel/test-zlib-convenience-methods.js"
- "test/parallel/test-zlib-random-byte-pipes.js"
- "test/parallel/test-zlib-write-after-flush.js")))
- '())
-
- ;; These tests have an expiry date: they depend on the validity of
- ;; TLS certificates that are bundled with the source. We want this
- ;; package to be reproducible forever, so remove those.
- ;; TODO: Regenerate certs instead.
- (for-each delete-file
- '("test/parallel/test-tls-passphrase.js"
- "test/parallel/test-tls-server-verify.js"))
-
;; Replace pre-generated llhttp sources
(let ((llhttp (assoc-ref inputs "llhttp")))
(copy-file (string-append llhttp "/src/llhttp.c")
@@ -834,7 +762,7 @@ source files.")
("python" ,python)
("util-linux" ,util-linux)))
(inputs
- `(("bash" ,bash)
+ `(("bash" ,bash-minimal)
("coreutils" ,coreutils)
("c-ares" ,c-ares)
("icu4c" ,icu4c-67)
--
2.30.2
P
P
Philip McGrath wrote on 1 Oct 2021 00:56
[PATCH 2/3] gnu: node: Update to 10.24.1 for bootstrapping.
(address . 49946@debbugs.gnu.org)
20210930225611.2143194-3-philip@philipmcgrath.com
* gnu/packages/node.scm (node): Update to 10.24.1.
---
gnu/packages/node.scm | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)

Toggle diff (23 lines)
diff --git a/gnu/packages/node.scm b/gnu/packages/node.scm
index 34c2bfa9d4..6d9c3ccc71 100644
--- a/gnu/packages/node.scm
+++ b/gnu/packages/node.scm
@@ -57,14 +57,14 @@
(define-public node
(package
(name "node")
- (version "10.24.0")
+ (version "10.24.1")
(source (origin
(method url-fetch)
(uri (string-append "https://nodejs.org/dist/v" version
"/node-v" version ".tar.xz"))
(sha256
(base32
- "1k1srdis23782hnd1ymgczs78x9gqhv77v0am7yb54gqcspp70hm"))
+ "032801kg24j04xmf09m0vxzlcz86sv21s24lv9l4cfv08k1c4byp"))
(modules '((guix build utils)))
(snippet
`(begin
--
2.30.2
P
P
Philip McGrath wrote on 1 Oct 2021 00:56
[PATCH 3/3] guix: node-build-system: Support compiling addons with node-gyp.
(address . 49946@debbugs.gnu.org)
20210930225611.2143194-4-philip@philipmcgrath.com
* gnu/packages/node.scm (node)[arguments]: Replace 'patch-npm-shebang
and 'patch-node-shebang with a new 'patch-nested-shebangs that also
handles node-gyp and other shebangs under "/lib/node_modules".
[inputs]: Add Python for node-gyp as "python-for-target".
(node-lts)[inputs]: Likewise.
* guix/build-system/node.scm (lower): Add optional #:python argument.
* guix/build/node-build-system.scm (set-node-gyp-paths): New
function. Sets the "npm_config_nodedir" and "npm_config_python"
environment variables.
(%standard-phases): Add 'set-node-gyp-paths after 'set-paths.

Co-authored-by: Pierre Langlois <pierre.langlois@gmx.com>
---
gnu/packages/node.scm | 33 +++++++++++++++++---------------
guix/build-system/node.scm | 7 ++++++-
guix/build/node-build-system.scm | 9 +++++++++
3 files changed, 33 insertions(+), 16 deletions(-)

Toggle diff (137 lines)
diff --git a/gnu/packages/node.scm b/gnu/packages/node.scm
index 6d9c3ccc71..805a4f18fc 100644
--- a/gnu/packages/node.scm
+++ b/gnu/packages/node.scm
@@ -244,21 +244,22 @@
python
(string-append python "3")))
"configure" flags))))
- (add-after 'patch-shebangs 'patch-npm-shebang
- (lambda* (#:key outputs #:allow-other-keys)
- (let* ((bindir (string-append (assoc-ref outputs "out")
- "/bin"))
- (npm (string-append bindir "/npm"))
- (target (readlink npm)))
- (with-directory-excursion bindir
- (patch-shebang target (list bindir))))))
- (add-after 'install 'patch-node-shebang
- (lambda* (#:key outputs #:allow-other-keys)
- (let* ((bindir (string-append (assoc-ref outputs "out")
- "/bin"))
- (npx (readlink (string-append bindir "/npx"))))
- (with-directory-excursion bindir
- (patch-shebang npx (list bindir)))))))))
+ (add-after 'patch-shebangs 'patch-nested-shebangs
+ (lambda* (#:key inputs outputs #:allow-other-keys)
+ (let* ((prefix (assoc-ref outputs "out"))
+ (path (map (lambda (dir)
+ (string-append dir "/bin"))
+ (list prefix
+ (assoc-ref inputs "python-for-target")))))
+ (with-directory-excursion (string-append prefix "/lib/node_modules")
+ (for-each
+ (lambda (file)
+ (patch-shebang file path))
+ (find-files "."
+ (lambda (file stat)
+ (and (eq? 'regular (stat:type stat))
+ (not (zero? (logand (stat:mode stat) #o100)))))
+ #:stat lstat)))))))))
(native-inputs
`(;; Runtime dependencies for binaries used as a bootstrap.
("c-ares" ,c-ares)
@@ -281,6 +282,7 @@
(inputs
`(("bash" ,bash-minimal)
("coreutils" ,coreutils)
+ ("python-for-target" ,python-wrapper) ;; for node-gyp (supports python3)
("c-ares" ,c-ares)
("http-parser" ,http-parser)
("icu4c" ,icu4c)
@@ -764,6 +766,7 @@ source files.")
(inputs
`(("bash" ,bash-minimal)
("coreutils" ,coreutils)
+ ("python-for-target" ,python-wrapper) ;; for node-gyp (supports python3)
("c-ares" ,c-ares)
("icu4c" ,icu4c-67)
("libuv" ,libuv-for-node)
diff --git a/guix/build-system/node.scm b/guix/build-system/node.scm
index 98f63f87ef..3e49e67ff6 100644
--- a/guix/build-system/node.scm
+++ b/guix/build-system/node.scm
@@ -1,6 +1,8 @@
;;; GNU Guix --- Functional package management for GNU
;;; Copyright © 2016 Jelle Licht <jlicht@fsfe.org>
;;; Copyright © 2019 Timothy Sample <samplet@ngyro.com>
+;;; Copyright © 2021 Pierre Langlois <pierre.langlois@gmx.com>
+;;; Copyright © 2021 Philip McGrath <philip@philipmcgrath.com>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -24,6 +26,7 @@
#:use-module (guix search-paths)
#:use-module (guix build-system)
#:use-module (guix build-system gnu)
+ #:use-module (guix build-system python)
#:use-module (ice-9 match)
#:export (%node-build-system-modules
node-build
@@ -44,11 +47,12 @@
(define* (lower name
#:key source inputs native-inputs outputs system target
(node (default-node))
+ (python (default-python)) ;; for node-gyp
#:allow-other-keys
#:rest arguments)
"Return a bag for NAME."
(define private-keywords
- '(#:source #:target #:node #:inputs #:native-inputs))
+ '(#:source #:target #:node #:python #:inputs #:native-inputs))
(and (not target) ;XXX: no cross-compilation
(bag
@@ -62,6 +66,7 @@
;; Keep the standard inputs of 'gnu-build-system'.
,@(standard-packages)))
(build-inputs `(("node" ,node)
+ ("python" ,python)
,@native-inputs))
(outputs outputs)
(build node-build)
diff --git a/guix/build/node-build-system.scm b/guix/build/node-build-system.scm
index 70a367618e..5e62eb4784 100644
--- a/guix/build/node-build-system.scm
+++ b/guix/build/node-build-system.scm
@@ -2,6 +2,8 @@
;;; Copyright © 2015 David Thompson <davet@gnu.org>
;;; Copyright © 2016, 2020 Jelle Licht <jlicht@fsfe.org>
;;; Copyright © 2019, 2021 Timothy Sample <samplet@ngyro.com>
+;;; Copyright © 2021 Pierre Langlois <pierre.langlois@gmx.com>
+;;; Copyright © 2021 Philip McGrath <philip@philipmcgrath.com>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -46,6 +48,12 @@
(format #t "set HOME to ~s~%" (getenv "HOME")))))))
#t)
+(define* (set-node-gyp-paths #:key inputs #:allow-other-keys)
+ "Initialize environment variables needed for building native addons."
+ (setenv "npm_config_nodedir" (assoc-ref inputs "node"))
+ (setenv "npm_config_python" (assoc-ref inputs "python"))
+ #t)
+
(define (module-name module)
(let* ((package.json (string-append module "/package.json"))
(package-meta (call-with-input-file package.json read-json)))
@@ -144,6 +152,7 @@
(define %standard-phases
(modify-phases gnu:%standard-phases
+ (add-after 'set-paths 'set-node-gyp-paths set-node-gyp-paths)
(add-after 'unpack 'set-home set-home)
(add-before 'configure 'patch-dependencies patch-dependencies)
(replace 'configure configure)
--
2.30.2
P
P
Pierre Langlois wrote on 2 Oct 2021 13:30
Re: [PATCH 0/3] guix: node-build-system: Support compiling addons with node-gyp.
(name . Philip McGrath)(address . philip@philipmcgrath.com)
87r1d3venp.fsf@gmx.com
Hi Philip,

Philip McGrath <philip@philipmcgrath.com> writes:

Toggle quote (4 lines)
> Hi,
>
> I've reworked the part of the patch series dealing with node-gyp.

Nice, thanks for working on this! I'll take a look at the patches, sorry
I've not had too much time this week yet :-).

Toggle quote (5 lines)
> I'd like to find an NPM addon package to submit as part of this series, too,
> basically as a test case. If I can find one that doesn't raise too many other
> complications, I may send this in as a separate patch, but feel free to try it
> with tree-sitter, too.

Cool, I'll give it a test with the tree-sitter series.

Toggle quote (6 lines)
>
> There are a few things I'm still not sure about. I haven't made
> node-build-system add libuv as an implicit input, because I think some
> node-gyp addons don't actually need libuv, but maybe it's common enough that
> it should be done automatically.

I see, I assumed libuv was needed for all addons. I'm thinking given
node itself heavily depends on libuv anymays, it might be nice to add it
by default in the build system, but I don't really mind either way. I'll
check if I can just add libuv in the tree-sitter packages.

Toggle quote (4 lines)
>
> Likewise, I haven't tried to change the issue of `node` referring to
> `node-bootstrap`, but I still think it should be changed.

I agree it would be good to change this. I started working on updating
node last weekend but haven't had time to work on it after that. I think
we should probably name node always as "node-<version>":

node -> node-10
node-lts -> node-14

Then keep the node and node-lts names as aliases:

(define-public node node-14) ;; The latest, but move it to node-16 later
(define-public node-lts node-14)

However, moving the node variable to node-14 is likely to cause
rebuilds, so we'll have to check that.

Anyways, here's my WIP patch in case there's anything of interest
there. Note that it still has the bug you pointed out where I'm refering
to the host python instead of the target one (thanks for pointing it
out!).

Thanks,
Pierre
-----BEGIN PGP SIGNATURE-----

iQFMBAEBCgA2FiEEctU9gYy29KFyWDdMqPyeRH9PfVQFAmFYRQoYHHBpZXJyZS5s
YW5nbG9pc0BnbXguY29tAAoJEKj8nkR/T31U/pEIAJdXHFJ7Gdbyx+axxXdEanQN
Tls8O855DPywHIgUhgGC5j6Mz6gk4/mnnEXWhbduPn+ehqvUABql9k87wF2tb+YV
sbYsBX5Xqv8GVKHW1vl1WGnCcZCJFK+x3pwrzIvsaCkN+31E2HxkSZCVCIzIk+fb
zd6yoH+nq8ZUR6QqdhA9X13FZjgm4Fq8VeDzVYrMLrvjI5RvdM3e8kN9fAkGJJpw
1WaJYAW+f1BGgQFvwUkAgUk9kUe1rbVvnt5iwGAmqCnFKwcNIvD6wYyKieVvi7U8
sGnO5V/3voK0EbLUt86b9KgQMyeFUAr27eCXULJvp5fEm8+HdLYtt4fnDcncHTI=
=AGh/
-----END PGP SIGNATURE-----

P
P
Pierre Langlois wrote on 2 Oct 2021 13:03
Re: [PATCH 1/3] gnu: node: Avoid duplicating build phases.
(name . Philip McGrath)(address . philip@philipmcgrath.com)
87h7dzveiy.fsf@gmx.com
Hi Philip,

Philip McGrath <philip@philipmcgrath.com> writes:

Toggle quote (10 lines)
> * gnu/packages/node.scm (node)[arguments]: Split 'patch-files phase
> into 'patch-hardcoded-program-references and
> 'delete-problematic-tests. Adapt those phases and 'configure to work
> unmodified on node-lts.
> (node, node-lts)[inputs]: Use bash-minimal rather than bash.
> (node-lts)[arguments]: Inherit 'patch-hardcoded-program-references,
> 'delete-problemating-tests, and 'configure phases from the bootstrap
> node. Remove the 'patch-files phase, keeping its remaining
> non-inherited work in a new 'replace-llhttp-sources phase.

While I agree that most of the time, factoring out common code is a good
thing, I'm not sure it applies in the case of patching tests. The list
of tests is specific to a version and it's likely for each version to
need fixes. Having a common phase that describes the tests to patch for
2 versions (3 if we add node 16) is harder to maintain than three phases
IMO, even though they'll look similar indeed. Having to change commmon
code can also cause unnessecary rebuilds.

For example, I started working updating node last weekend and saw these
test changes:

- 14.16 -> 14.17: Delete test/parallel/test-https-agent-unref-socket.js,
requires networking
- 16: Extra test needs /bin/sh patched test/parallel/test-stdin-from-file-spawn.js"
A couple tests were renamed:
test/parallel/test-cluster-master-error.js -> test/parallel/test-cluster-primary-error.js
test/parallel/test-cluster-master-kill.js -> test/parallel/test-cluster-primary-kill.js

That being said, I definetely agree we should have a separate phase for
the replacement of the llhttp source, that's logically different from
patching tests, and is unlikely to change version to version.

Keeping the list of tests local to each packages allows to add node 16
while avoiding rebuilding the others, does this make sense? I could be
wrong here of course :-).

Thanks,
Pierre
-----BEGIN PGP SIGNATURE-----

iQFMBAEBCgA2FiEEctU9gYy29KFyWDdMqPyeRH9PfVQFAmFYRbUYHHBpZXJyZS5s
YW5nbG9pc0BnbXguY29tAAoJEKj8nkR/T31UzrcH/i7cT2CBqJuJMKem8JeCTV0g
Z1jWeCdY1zYvmulsczCn/35nIwFi+n+YBePIoxn9pfyl6FfrMXskMavafWUSk8K2
El5QB0ITYyWQ3w7R87GIonaMxc7Yqsy/SteTe3gqLPPQCO4N3kcx4viiigLK9wGF
Q8xSWrZZlsEQat7E/z3ID0uW1AQ3Usyk+GqjEYknJQOTyutN5T1LjwMHIrICftxr
aUZ6kKxRCBLy23kprbneZykV3DHsLwvl96gGBD2XK+rKuHjvhmHl41Sqh49xms92
dVtlA7AMR/ZPrWOvvjD8oPOi610sw8HhpuNjmrv8bE0ub7Q8dL9UEqEDQkzzz4I=
=S0Pr
-----END PGP SIGNATURE-----

P
P
Pierre Langlois wrote on 2 Oct 2021 13:49
Re: [PATCH 3/3] guix: node-build-system: Support compiling addons with node-gyp.
(name . Philip McGrath)(address . philip@philipmcgrath.com)
878rzbtyyn.fsf@gmx.com
Philip McGrath <philip@philipmcgrath.com> writes:

Toggle quote (11 lines)
> * gnu/packages/node.scm (node)[arguments]: Replace 'patch-npm-shebang
> and 'patch-node-shebang with a new 'patch-nested-shebangs that also
> handles node-gyp and other shebangs under "/lib/node_modules".
> [inputs]: Add Python for node-gyp as "python-for-target".
> (node-lts)[inputs]: Likewise.
> * guix/build-system/node.scm (lower): Add optional #:python argument.
> * guix/build/node-build-system.scm (set-node-gyp-paths): New
> function. Sets the "npm_config_nodedir" and "npm_config_python"
> environment variables.
> (%standard-phases): Add 'set-node-gyp-paths after 'set-paths.

Nice! I'll test this with the tree-sitter series. I just had one comment
inline, otherwise it looks good to me.

Do you want me to integrate it into the tree-sitter series or submit it
separately? It might make its way upstream quicker separately, in which
case I'd suggest to send it again in a new bug for more visibility.

Toggle quote (48 lines)
>
> Co-authored-by: Pierre Langlois <pierre.langlois@gmx.com>
> ---
> gnu/packages/node.scm | 33 +++++++++++++++++---------------
> guix/build-system/node.scm | 7 ++++++-
> guix/build/node-build-system.scm | 9 +++++++++
> 3 files changed, 33 insertions(+), 16 deletions(-)
>
> diff --git a/gnu/packages/node.scm b/gnu/packages/node.scm
> index 6d9c3ccc71..805a4f18fc 100644
> --- a/gnu/packages/node.scm
> +++ b/gnu/packages/node.scm
> @@ -244,21 +244,22 @@
> python
> (string-append python "3")))
> "configure" flags))))
> - (add-after 'patch-shebangs 'patch-npm-shebang
> - (lambda* (#:key outputs #:allow-other-keys)
> - (let* ((bindir (string-append (assoc-ref outputs "out")
> - "/bin"))
> - (npm (string-append bindir "/npm"))
> - (target (readlink npm)))
> - (with-directory-excursion bindir
> - (patch-shebang target (list bindir))))))
> - (add-after 'install 'patch-node-shebang
> - (lambda* (#:key outputs #:allow-other-keys)
> - (let* ((bindir (string-append (assoc-ref outputs "out")
> - "/bin"))
> - (npx (readlink (string-append bindir "/npx"))))
> - (with-directory-excursion bindir
> - (patch-shebang npx (list bindir)))))))))
> + (add-after 'patch-shebangs 'patch-nested-shebangs
> + (lambda* (#:key inputs outputs #:allow-other-keys)
> + (let* ((prefix (assoc-ref outputs "out"))
> + (path (map (lambda (dir)
> + (string-append dir "/bin"))
> + (list prefix
> + (assoc-ref inputs "python-for-target")))))
> + (with-directory-excursion (string-append prefix "/lib/node_modules")
> + (for-each
> + (lambda (file)
> + (patch-shebang file path))
> + (find-files "."
> + (lambda (file stat)
> + (and (eq? 'regular (stat:type stat))
> + (not (zero? (logand (stat:mode stat) #o100)))))
> + #:stat lstat)))))))))

Here you don't necessarily need with-directory-excursion. I see we also
have a executable-file? predicate function in (guix build utils), could
we use that? i.e:

(for-each
(lambda (file)
(patch-shebang file path))
(find-files (string-append prefix "/lib/node_modules") executable-file?))

Toggle quote (98 lines)
> (native-inputs
> `(;; Runtime dependencies for binaries used as a bootstrap.
> ("c-ares" ,c-ares)
> @@ -281,6 +282,7 @@
> (inputs
> `(("bash" ,bash-minimal)
> ("coreutils" ,coreutils)
> + ("python-for-target" ,python-wrapper) ;; for node-gyp (supports python3)
> ("c-ares" ,c-ares)
> ("http-parser" ,http-parser)
> ("icu4c" ,icu4c)
> @@ -764,6 +766,7 @@ source files.")
> (inputs
> `(("bash" ,bash-minimal)
> ("coreutils" ,coreutils)
> + ("python-for-target" ,python-wrapper) ;; for node-gyp (supports python3)
> ("c-ares" ,c-ares)
> ("icu4c" ,icu4c-67)
> ("libuv" ,libuv-for-node)
> diff --git a/guix/build-system/node.scm b/guix/build-system/node.scm
> index 98f63f87ef..3e49e67ff6 100644
> --- a/guix/build-system/node.scm
> +++ b/guix/build-system/node.scm
> @@ -1,6 +1,8 @@
> ;;; GNU Guix --- Functional package management for GNU
> ;;; Copyright © 2016 Jelle Licht <jlicht@fsfe.org>
> ;;; Copyright © 2019 Timothy Sample <samplet@ngyro.com>
> +;;; Copyright © 2021 Pierre Langlois <pierre.langlois@gmx.com>
> +;;; Copyright © 2021 Philip McGrath <philip@philipmcgrath.com>
> ;;;
> ;;; This file is part of GNU Guix.
> ;;;
> @@ -24,6 +26,7 @@
> #:use-module (guix search-paths)
> #:use-module (guix build-system)
> #:use-module (guix build-system gnu)
> + #:use-module (guix build-system python)
> #:use-module (ice-9 match)
> #:export (%node-build-system-modules
> node-build
> @@ -44,11 +47,12 @@
> (define* (lower name
> #:key source inputs native-inputs outputs system target
> (node (default-node))
> + (python (default-python)) ;; for node-gyp
> #:allow-other-keys
> #:rest arguments)
> "Return a bag for NAME."
> (define private-keywords
> - '(#:source #:target #:node #:inputs #:native-inputs))
> + '(#:source #:target #:node #:python #:inputs #:native-inputs))
>
> (and (not target) ;XXX: no cross-compilation
> (bag
> @@ -62,6 +66,7 @@
> ;; Keep the standard inputs of 'gnu-build-system'.
> ,@(standard-packages)))
> (build-inputs `(("node" ,node)
> + ("python" ,python)
> ,@native-inputs))
> (outputs outputs)
> (build node-build)
> diff --git a/guix/build/node-build-system.scm b/guix/build/node-build-system.scm
> index 70a367618e..5e62eb4784 100644
> --- a/guix/build/node-build-system.scm
> +++ b/guix/build/node-build-system.scm
> @@ -2,6 +2,8 @@
> ;;; Copyright © 2015 David Thompson <davet@gnu.org>
> ;;; Copyright © 2016, 2020 Jelle Licht <jlicht@fsfe.org>
> ;;; Copyright © 2019, 2021 Timothy Sample <samplet@ngyro.com>
> +;;; Copyright © 2021 Pierre Langlois <pierre.langlois@gmx.com>
> +;;; Copyright © 2021 Philip McGrath <philip@philipmcgrath.com>
> ;;;
> ;;; This file is part of GNU Guix.
> ;;;
> @@ -46,6 +48,12 @@
> (format #t "set HOME to ~s~%" (getenv "HOME")))))))
> #t)
>
> +(define* (set-node-gyp-paths #:key inputs #:allow-other-keys)
> + "Initialize environment variables needed for building native addons."
> + (setenv "npm_config_nodedir" (assoc-ref inputs "node"))
> + (setenv "npm_config_python" (assoc-ref inputs "python"))
> + #t)
> +
> (define (module-name module)
> (let* ((package.json (string-append module "/package.json"))
> (package-meta (call-with-input-file package.json read-json)))
> @@ -144,6 +152,7 @@
>
> (define %standard-phases
> (modify-phases gnu:%standard-phases
> + (add-after 'set-paths 'set-node-gyp-paths set-node-gyp-paths)
> (add-after 'unpack 'set-home set-home)
> (add-before 'configure 'patch-dependencies patch-dependencies)
> (replace 'configure configure)


The build system changes are now so nice and simple :-). I guess we
don't really need to set npm_config_python, it should be able to find
python in the PATH, but it doesn't hurt.

Thanks,
Pierre
-----BEGIN PGP SIGNATURE-----

iQFMBAEBCgA2FiEEctU9gYy29KFyWDdMqPyeRH9PfVQFAmFYSsAYHHBpZXJyZS5s
YW5nbG9pc0BnbXguY29tAAoJEKj8nkR/T31UbGwIAIkzKPG7t2Zzpwiq6OpznueP
jnWlHc9SSX470xck1bwv7Gf06J84rIh523jkmiMGCiPN0vz2/KXj1FZIB6ykcJfr
lXUfBu7gEWclAa8ilV2CwJHv1Xrn5oDXUPFdLxnmnUQtPaGKC7rwqWeNeHA6/oX3
rGLCN/5PiL3sM86Fym6sealM8t8wAIRU1SwyBr2JFthoAH5O5Dvol2Mtsv1Tckzh
LvQWIg0KR+l0/sogj2YHSnhqvXemnK1icBR4hxD/5quBqpiCvH1j5Qh+1ogcKq4A
WVvabJYzc3GQoLLuJTWtNEvIRDZMite9Og75DWaopRbWQBR0Pual/aY1x5YIdRk=
=R3ZV
-----END PGP SIGNATURE-----

P
P
Pierre Langlois wrote on 10 Feb 2022 23:39
[WIP PATCH v3 00/26] gnu: Add tree-sitter for emacs (and neovim as well?).
(address . 49946@debbugs.gnu.org)
87leyinxly.fsf@gmx.com
Hi Guix, hi Luis,

Here's a WIP V3 of my series for tree-sitter. I'm almost done with it, I
need to to another couple of passes to see if there's anything left to
cleanup.

I'm sharing it early as a WIP first, as I just noticed the other day
that Luis also contributed a patches for tree-sitter for neovim[0]. The
series I have here is quite a lot bigger though so I'm happy to rebase
it on top of yours, Luis :-). The package I had for tree-sitter looks
basically the same, I would then move it to a new file with all
tree-sitter related packages, if you're happy with that.

The other reason I'm posting this is this series, which while it only
supports emacs, actually builds grammars for some of the supported
languages, I picked a set that I think are popular, there are probably
more to add but I thought it was a good starting point. While the
grammars are originally in JSON, they are distributed as binaries it
seems so it'll be better if we can build them as part of Guix.

So I'm wondering, are these tree-sitter grammar packages also useful for
the neovim package for tree-sitter?

Anyways, see patches incoming in just a minute!

Thanks,
Pierre


Pierre Langlois (26):
gnu: Add rust-smallbitvec.
gnu: Add rust-html-escape.
gnu: Add rust-spin@0.7.
gnu: Add tree-sitter.
gnu: Add tree-sitter-cli.
gnu: Add tree-sitter-c.
gnu: Add tree-sitter-cpp.
gnu: Add tree-sitter-bash.
gnu: Add tree-sitter-css.
gnu: Add tree-sitter-go.
gnu: Add tree-sitter-html.
gnu: Add tree-sitter-java.
gnu: Add tree-sitter-javascript.
gnu: Add tree-sitter-json.
gnu: Add tree-sitter-julia.
gnu: Add tree-sitter-php.
gnu: Add tree-sitter-python.
gnu: Add tree-sitter-rust.
gnu: Add tree-sitter-typescript.
gnu: Add rust-tree-sitter-for-emacs.
gnu: rust-emacs-module: Update to 0.18.
gnu: rust-emacs-macros: Update to 0.17.
gnu: rust-emacs: Update to 0.18.
gnu: Add emacs-tree-sitter-core.
gnu: Add emacs-tree-sitter.
gnu: Add emacs-tree-sitter-langs.

gnu/local.mk | 2 +
gnu/packages/crates-io.scm | 154 +++-
.../rust-tree-sitter-text-provider-fix.patch | 98 +++
gnu/packages/tree-sitter.scm | 808 ++++++++++++++++++
4 files changed, 1044 insertions(+), 18 deletions(-)
create mode 100644 gnu/packages/patches/rust-tree-sitter-text-provider-fix.patch
create mode 100644 gnu/packages/tree-sitter.scm

--
2.34.0
-----BEGIN PGP SIGNATURE-----

iQFMBAEBCgA2FiEEctU9gYy29KFyWDdMqPyeRH9PfVQFAmIFlxkYHHBpZXJyZS5s
YW5nbG9pc0BnbXguY29tAAoJEKj8nkR/T31UMf8IAK06VuxmHQYuMO49zLApSwso
ONhTcSfLy8VNGjirKEjJHaErjMbJv4kN9PUyw9YvwzGVz1aCfIBQy5p6mItzNBE0
wxY2D5GT5YummMLzXPOMfrS+eA9Og7RSp0PKGC4pg6vI+z/9CdNACpwhWQIdMV/J
lI/f2nISCzVAKusx9PiCG1zPFrUuZtCu+wyOgK53z6Pah1/hYQvEUFMx+LoNHFri
eiZHIPswLMPAbzJAghTG7v5PQ8dluUQ1C6iD1uRT2xsbDuuc7TrmKwOCLYIIXcam
bu7TQmxSFsxibNObXKEilFX7Cmadp4pNEt4Z6Na8ekT6qZcWxUVPEcFtGfVxa2I=
=nW3K
-----END PGP SIGNATURE-----

P
P
Pierre Langlois wrote on 10 Feb 2022 23:55
[WIP PATCH v3 01/26] gnu: Add rust-smallbitvec.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220210225527.25189-1-pierre.langlois@gmx.com
* gnu/packages/crates-io.scm (rust-smallbitvec-2): New variable.
---
gnu/packages/crates-io.scm | 22 ++++++++++++++++++++++
1 file changed, 22 insertions(+)

Toggle diff (33 lines)
diff --git a/gnu/packages/crates-io.scm b/gnu/packages/crates-io.scm
index fa9a7d9ab5..d001e723ec 100644
--- a/gnu/packages/crates-io.scm
+++ b/gnu/packages/crates-io.scm
@@ -56099,6 +56099,28 @@ (define-public rust-sluice-0.4
("rust-futures-core-preview" ,rust-futures-core-preview-0.3)
("rust-futures-io-preview" ,rust-futures-io-preview-0.3))))))

+(define-public rust-smallbitvec-2
+ (package
+ (name "rust-smallbitvec")
+ (version "2.5.1")
+ (source (origin
+ (method url-fetch)
+ (uri (crate-uri "smallbitvec" version))
+ (file-name (string-append name "-" version ".tar.gz"))
+ (sha256
+ (base32
+ "0plrbldsjpwip3afbzd8fgrnvdhizcg5z4ncfqs4q6x4qjflzkkm"))))
+ (build-system cargo-build-system)
+ (arguments
+ `(#:cargo-development-inputs
+ (("rust-bit-vec" ,rust-bit-vec-0.4)
+ ("rust-rand" ,rust-rand-0.4))))
+ (home-page "https://github.com/servo/smallbitvec")
+ (synopsis "A bit vector optimized for size and inline storage")
+ (description "This package provides a bit vector optimized for size and
+inline storage")
+ (license (list license:expat license:asl2.0))))
+
(define-public rust-smallvec-1
(package
(name "rust-smallvec")
--
2.34.0
P
P
Pierre Langlois wrote on 10 Feb 2022 23:55
[WIP PATCH v3 02/26] gnu: Add rust-html-escape.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220210225527.25189-2-pierre.langlois@gmx.com
* gnu/packages/crates-io.scm (rust-html-escape-0.2): New variable.
---
gnu/packages/crates-io.scm | 24 ++++++++++++++++++++++++
1 file changed, 24 insertions(+)

Toggle diff (35 lines)
diff --git a/gnu/packages/crates-io.scm b/gnu/packages/crates-io.scm
index d001e723ec..cff703c427 100644
--- a/gnu/packages/crates-io.scm
+++ b/gnu/packages/crates-io.scm
@@ -26933,6 +26933,30 @@ (define-public rust-htmlescape-0.3
"This package provides a library for HTML entity encoding and decoding.")
(license (list license:asl2.0 license:expat license:mpl2.0))))

+(define-public rust-html-escape-0.2
+ (package
+ (name "rust-html-escape")
+ (version "0.2.9")
+ (source (origin
+ (method url-fetch)
+ (uri (crate-uri "html-escape" version))
+ (file-name (string-append name "-" version ".tar.gz"))
+ (sha256
+ (base32
+ "1dxw9lpckrqzzqgbkw64ckbajr4b7xxnjdn8adgzqf2mm40shvl1"))))
+ (build-system cargo-build-system)
+ (arguments
+ `(#:cargo-inputs
+ (("rust-utf8-width" ,rust-utf8-width-0.1))
+ #:cargo-development-inputs
+ (("rust-bencher" ,rust-bencher-0.1))))
+ (home-page "https://magiclen.org/html-escape")
+ (synopsis "Library for encoding and escaping special characters in HTML")
+ (description
+ "This package provides a library for encoding and escaping special
+characters in HTML, decoding and unescaping HTML entities as well.")
+ (license license:expat)))
+
(define-public rust-http-0.2
(package
(name "rust-http")
--
2.34.0
P
P
Pierre Langlois wrote on 10 Feb 2022 23:55
[WIP PATCH v3 03/26] gnu: Add rust-spin@0.7.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220210225527.25189-3-pierre.langlois@gmx.com
* gnu/packages/crates-io.scm (rust-spin-0.7): New variable.
---
gnu/packages/crates-io.scm | 13 +++++++++++++
1 file changed, 13 insertions(+)

Toggle diff (24 lines)
diff --git a/gnu/packages/crates-io.scm b/gnu/packages/crates-io.scm
index cff703c427..ad27298404 100644
--- a/gnu/packages/crates-io.scm
+++ b/gnu/packages/crates-io.scm
@@ -56716,6 +56716,19 @@ (define-public rust-spin-0.9
initializers are available.")
(license license:expat)))

+(define-public rust-spin-0.7
+ (package
+ (inherit rust-spin-0.9)
+ (name "rust-spin")
+ (version "0.7.1")
+ (source (origin
+ (method url-fetch)
+ (uri (crate-uri "spin" version))
+ (file-name (string-append name "-" version ".tar.gz"))
+ (sha256
+ (base32
+ "0qjips9f6fsvkyd7wj3a4gzaqknn2q4kkb19957pl86im56pna0k"))))))
+
(define-public rust-spin-0.5
(package
(inherit rust-spin-0.9)
--
2.34.0
P
P
Pierre Langlois wrote on 10 Feb 2022 23:55
[WIP PATCH v3 05/26] gnu: Add tree-sitter-cli.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220210225527.25189-5-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-cli): New variable.
---
gnu/packages/tree-sitter.scm | 68 +++++++++++++++++++++++++++++++++++-
1 file changed, 67 insertions(+), 1 deletion(-)

Toggle diff (86 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index e7d2ce113d..c2919e76b2 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -19,10 +19,13 @@
(define-module (gnu packages tree-sitter)
#:use-module (guix gexp)
#:use-module ((guix licenses) #:prefix license:)
+ #:use-module (guix build-system cargo)
#:use-module (guix build-system gnu)
#:use-module (guix git-download)
#:use-module (guix packages)
- #:use-module (guix utils))
+ #:use-module (guix utils)
+ #:use-module (gnu packages crates-graphics)
+ #:use-module (gnu packages crates-io))

(define-public tree-sitter
(package
@@ -74,3 +77,66 @@ (define-public tree-sitter

This package includes the @code{libtree-sitter} runtime library.")
(license license:expat)))
+
+(define-public tree-sitter-cli
+ (package (inherit tree-sitter)
+ (name "tree-sitter-cli")
+ (build-system cargo-build-system)
+ (arguments
+ `(;; Running test requires downloading fixtures, see the
+ ;; script/fetch-fixtures script.
+ #:tests? #f
+ ;; FIXME: Installing the sources for the tree-sitter Rust bindings
+ ;; doesn't work out of the box due to tree-sitter having multiple
+ ;; Rust packages in the same repository (bindings and CLI).
+ #:install-source? #f
+ #:cargo-inputs
+ (("rust-ansi-term" ,rust-ansi-term-0.12)
+ ("rust-anyhow" ,rust-anyhow-1)
+ ("rust-atty" ,rust-atty-0.2)
+ ("rust-clap" ,rust-clap-2)
+ ("rust-difference" ,rust-difference-2)
+ ("rust-dirs" ,rust-dirs-3)
+ ("rust-html-escape" ,rust-html-escape-0.2)
+ ("rust-libloading" ,rust-libloading-0.7)
+ ("rust-rustc-hash" ,rust-rustc-hash-1)
+ ("rust-serde" ,rust-serde-1)
+ ("rust-smallbitvec" ,rust-smallbitvec-2)
+ ("rust-spin" ,rust-spin-0.7)
+ ("rust-thiserror" ,rust-thiserror-1)
+ ("rust-tiny-http" ,rust-tiny-http-0.8)
+ ("rust-toml" ,rust-toml-0.5)
+ ("rust-walkdir" ,rust-walkdir-2)
+ ("rust-webbrowser" ,rust-webbrowser-0.5)
+ ("rust-which" ,rust-which-4))
+ #:cargo-development-inputs
+ (("rust-pretty-assertions" ,rust-pretty-assertions-0.7))
+ #:phases
+ (modify-phases %standard-phases
+ (add-after 'unpack 'delete-cargo.lock
+ (lambda _ (delete-file "Cargo.lock")))
+ (replace 'install
+ (lambda* (#:key outputs #:allow-other-keys)
+ (let* ((out (assoc-ref outputs "out"))
+ (bin (string-append out "/bin"))
+ (lib (string-append out "/lib")))
+ (mkdir-p bin)
+ (install-file "target/release/tree-sitter" bin)))))))
+ (synopsis "Incremental parsing system for programming tools")
+ (description "Tree-sitter is a parser generator tool and an incremental
+parsing library. It can build a concrete syntax tree for a source file and
+efficiently update the syntax tree as the source file is edited.
+
+Tree-sitter aims to be:
+
+@enumerate
+@item General enough to parse any programming language.
+@item Fast enough to parse on every keystroke in a text editor.
+@item Robust enough to provide useful results even in the presence of syntax
+errors.
+@item Dependency-free so that the runtime library (which is written in pure C)
+can be embedded in any application.
+@end enumerate
+
+This package includes the @command{tree-sitter} command-line tool.")
+ (license license:expat)))
--
2.34.0
P
P
Pierre Langlois wrote on 10 Feb 2022 23:55
[WIP PATCH v3 04/26] gnu: Add tree-sitter.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220210225527.25189-4-pierre.langlois@gmx.com
* gnu/package/tree-sitter.scm: New file.
* gnu/local.mk (GNU_SYSTEM_MODULES): Register it.
---
gnu/local.mk | 1 +
gnu/packages/tree-sitter.scm | 76 ++++++++++++++++++++++++++++++++++++
2 files changed, 77 insertions(+)
create mode 100644 gnu/packages/tree-sitter.scm

Toggle diff (94 lines)
diff --git a/gnu/local.mk b/gnu/local.mk
index 198c8f64a6..273d80a9c2 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -584,6 +584,7 @@ GNU_SYSTEM_MODULES = \
%D%/packages/tmux.scm \
%D%/packages/toolkits.scm \
%D%/packages/tor.scm \
+ %D%/packages/tree-sitter.scm \
%D%/packages/tv.scm \
%D%/packages/uglifyjs.scm \
%D%/packages/uml.scm \
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
new file mode 100644
index 0000000000..e7d2ce113d
--- /dev/null
+++ b/gnu/packages/tree-sitter.scm
@@ -0,0 +1,76 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2022 Pierre Langlois <pierre.langlois@gmx.com>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu packages tree-sitter)
+ #:use-module (guix gexp)
+ #:use-module ((guix licenses) #:prefix license:)
+ #:use-module (guix build-system gnu)
+ #:use-module (guix git-download)
+ #:use-module (guix packages)
+ #:use-module (guix utils))
+
+(define-public tree-sitter
+ (package
+ (name "tree-sitter")
+ (version "0.20.4")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "1yldgdbf3l5l4ki52abdf81nwkcbvg219gwr3ydcjwfsg7hf7zhz"))))
+ (build-system gnu-build-system)
+ (arguments
+ (list
+ #:tests? #f ;No check target.
+ #:phases
+ #~(modify-phases %standard-phases
+ (delete 'configure)
+ (add-before 'build 'set-cc
+ (lambda _
+ (setenv "CC"
+ #$(cc-for-target))))
+ (replace 'install
+ (lambda* (#:key outputs #:allow-other-keys)
+ (let* ((out (assoc-ref outputs "out")) (lib (string-append
+ out "/lib")))
+ (setenv "PREFIX" out)
+ (invoke "make" "install")))))))
+ (home-page "https://tree-sitter.github.io/tree-sitter/")
+ (synopsis "Incremental parsing system for programming tools")
+ (description
+ "Tree-sitter is a parser generator tool and an incremental
+parsing library. It can build a concrete syntax tree for a source file and
+efficiently update the syntax tree as the source file is edited.
+
+Tree-sitter aims to be:
+
+@enumerate
+@item General enough to parse any programming language.
+@item Fast enough to parse on every keystroke in a text editor.
+@item Robust enough to provide useful results even in the presence of syntax
+errors.
+@item Dependency-free so that the runtime library (which is written in pure C)
+can be embedded in any application.
+@end enumerate
+
+This package includes the @code{libtree-sitter} runtime library.")
+ (license license:expat)))
--
2.34.0
P
P
Pierre Langlois wrote on 10 Feb 2022 23:55
[WIP PATCH v3 07/26] gnu: Add tree-sitter-cpp.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220210225527.25189-7-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-cpp): New variable.
---
gnu/packages/tree-sitter.scm | 18 ++++++++++++++++++
1 file changed, 18 insertions(+)

Toggle diff (26 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index b62cb483c9..cc3bb2cec0 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -247,3 +247,21 @@ (define (source-file? basename)
(description
"This package provides a C grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-cpp
+ (package
+ (inherit tree-sitter-c)
+ (name "tree-sitter-cpp")
+ (version "0.20.0")
+ (source (tree-sitter-origin
+ name
+ version
+ "0hxcpdvyyig8njga1mxp4qcnbbnr1d0aiy27vahijwbh98b081nr"))
+ (inputs
+ (list node-nan
+ `(,tree-sitter-c "js")))
+ (home-page "https://github.com/tree-sitter/tree-sitter-cpp")
+ (synopsis "Tree-sitter C++ grammar")
+ (description
+ "This package provides a C++ grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.34.0
P
P
Pierre Langlois wrote on 10 Feb 2022 23:55
[WIP PATCH v3 08/26] gnu: Add tree-sitter-bash.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220210225527.25189-8-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-bash): New variable.
---
gnu/packages/tree-sitter.scm | 24 ++++++++++++++++++++++++
1 file changed, 24 insertions(+)

Toggle diff (32 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index cc3bb2cec0..827e1cdc4d 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -265,3 +265,27 @@ (define-public tree-sitter-cpp
(description
"This package provides a C++ grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-bash
+ (package (inherit tree-sitter-c)
+ (name "tree-sitter-bash")
+ (version "0.19.0")
+ (source (tree-sitter-origin
+ name
+ version
+ "18c030bb65r50i6z37iy7jb9z9i8i36y7b08dbc9bchdifqsijs5"))
+ (arguments
+ (substitute-keyword-arguments (package-arguments tree-sitter-c)
+ ((#:phases phases)
+ `(modify-phases ,phases
+ (replace 'delete-dependencies
+ (lambda _
+ (delete-dependencies '("tree-sitter-cli"
+ "node-gyp"
+ "prebuild"
+ "prebuild-install"))))))))
+ (home-page "https://github.com/tree-sitter/tree-sitter-bash")
+ (synopsis "Tree-sitter Bash grammar")
+ (description
+ "This package provides a Bash grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.34.0
P
P
Pierre Langlois wrote on 10 Feb 2022 23:55
[WIP PATCH v3 09/26] gnu: Add tree-sitter-css.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220210225527.25189-9-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-css): New variable.
---
gnu/packages/tree-sitter.scm | 14 ++++++++++++++
1 file changed, 14 insertions(+)

Toggle diff (22 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 827e1cdc4d..36b712c189 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -289,3 +289,17 @@ (define-public tree-sitter-bash
(description
"This package provides a Bash grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-css
+ (package (inherit tree-sitter-c)
+ (name "tree-sitter-css")
+ (version "0.19.0")
+ (source (tree-sitter-origin
+ name
+ version
+ "014jrlgi7zfza9g38hsr4vlbi8964i5p7iglaih6qmzaiml7bja2"))
+ (home-page "https://github.com/tree-sitter/tree-sitter-css")
+ (synopsis "Tree-sitter CSS grammar")
+ (description
+ "This package provides a CSS grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.34.0
P
P
Pierre Langlois wrote on 10 Feb 2022 23:55
[WIP PATCH v3 11/26] gnu: Add tree-sitter-html.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220210225527.25189-11-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-html): New variable.
---
gnu/packages/tree-sitter.scm | 14 ++++++++++++++
1 file changed, 14 insertions(+)

Toggle diff (22 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 47f102b0b3..e4679dc8ab 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -326,3 +326,17 @@ (define-public tree-sitter-go
(description
"This package provides a Golang grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-html
+ (package (inherit tree-sitter-c)
+ (name "tree-sitter-html")
+ (version "0.19.0")
+ (source (tree-sitter-origin
+ name
+ version
+ "1hg7vbcy7bir6b8x11v0a4x0glvqnsqc3i2ixiarbxmycbgl3axy"))
+ (home-page "https://github.com/tree-sitter/tree-sitter-html")
+ (synopsis "Tree-sitter HTML grammar")
+ (description
+ "This package provides a HTML grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.34.0
P
P
Pierre Langlois wrote on 10 Feb 2022 23:55
[WIP PATCH v3 12/26] gnu: Add tree-sitter-java.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220210225527.25189-12-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-java): New variable.
---
gnu/packages/tree-sitter.scm | 23 +++++++++++++++++++++++
1 file changed, 23 insertions(+)

Toggle diff (31 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index e4679dc8ab..760a7871df 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -340,3 +340,26 @@ (define-public tree-sitter-html
(description
"This package provides a HTML grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-java
+ (package (inherit tree-sitter-c)
+ (name "tree-sitter-java")
+ (version "0.19.1")
+ (source (tree-sitter-origin
+ name
+ version
+ "07zw9ygb45hnvlx9qlz7rlz8hc3byjy03d24v72i5iyhpiiwlhvl"))
+ (arguments
+ (substitute-keyword-arguments (package-arguments tree-sitter-c)
+ ((#:phases phases)
+ `(modify-phases ,phases
+ ;; The parse-example script tries to clone git repositories.
+ (add-after 'unpack 'remove-parse-example
+ (lambda _
+ (substitute* "package.json"
+ ((" && script\\/parse-examples") ""))))))))
+ (home-page "https://github.com/tree-sitter/tree-sitter-java")
+ (synopsis "Tree-sitter Java grammar")
+ (description
+ "This package provides a Java grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.34.0
P
P
Pierre Langlois wrote on 10 Feb 2022 23:55
[WIP PATCH v3 10/26] gnu: Add tree-sitter-go.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220210225527.25189-10-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-go): New variable.
---
gnu/packages/tree-sitter.scm | 23 +++++++++++++++++++++++
1 file changed, 23 insertions(+)

Toggle diff (31 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 36b712c189..47f102b0b3 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -303,3 +303,26 @@ (define-public tree-sitter-css
(description
"This package provides a CSS grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-go
+ (package (inherit tree-sitter-c)
+ (name "tree-sitter-go")
+ (version "0.19.1")
+ (source (tree-sitter-origin
+ name
+ version
+ "0nxs47vd2fc2fr0qlxq496y852rwg39flhg334s7dlyq7d3lcx4x"))
+ (arguments
+ (substitute-keyword-arguments (package-arguments tree-sitter-c)
+ ((#:phases phases)
+ `(modify-phases ,phases
+ ;; The parse-example script tries to clone git repositories.
+ (add-after 'unpack 'remove-parse-example
+ (lambda _
+ (substitute* "package.json"
+ ((" && script\\/parse-examples") ""))))))))
+ (home-page "https://github.com/tree-sitter/tree-sitter-go")
+ (synopsis "Tree-sitter Go grammar")
+ (description
+ "This package provides a Golang grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.34.0
P
P
Pierre Langlois wrote on 10 Feb 2022 23:55
[WIP PATCH v3 13/26] gnu: Add tree-sitter-javascript.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220210225527.25189-13-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-javascript): New variable.
---
gnu/packages/tree-sitter.scm | 24 ++++++++++++++++++++++++
1 file changed, 24 insertions(+)

Toggle diff (32 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 760a7871df..b728e6511b 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -363,3 +363,27 @@ (define-public tree-sitter-java
(description
"This package provides a Java grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-javascript
+ (package (inherit tree-sitter-c)
+ (name "tree-sitter-javascript")
+ (version "0.20.0")
+ (source (tree-sitter-origin
+ name
+ version
+ "175yrk382n2di0c2xn4gpv8y4n83x1lg4hqn04vabf0yqynlkq67"))
+ (arguments
+ (substitute-keyword-arguments (package-arguments tree-sitter-c)
+ ((#:phases phases)
+ `(modify-phases ,phases
+ ;; The parse-example script tries to clone git repositories.
+ (add-after 'unpack 'remove-parse-example
+ (lambda _
+ (substitute* "package.json"
+ ((" && script\\/parse-examples") ""))))))))
+ (home-page "https://github.com/tree-sitter/tree-sitter-javascript")
+ (synopsis "Tree-sitter Javascript grammar")
+ (description
+ "This package provides a Javascript grammar for the Tree-sitter
+library.")
+ (license license:expat)))
--
2.34.0
P
P
Pierre Langlois wrote on 10 Feb 2022 23:55
[WIP PATCH v3 14/26] gnu: Add tree-sitter-json.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220210225527.25189-14-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-json): New variable.
---
gnu/packages/tree-sitter.scm | 14 ++++++++++++++
1 file changed, 14 insertions(+)

Toggle diff (22 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index b728e6511b..d35edc533d 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -387,3 +387,17 @@ (define-public tree-sitter-javascript
"This package provides a Javascript grammar for the Tree-sitter
library.")
(license license:expat)))
+
+(define-public tree-sitter-json
+ (package (inherit tree-sitter-c)
+ (name "tree-sitter-json")
+ (version "0.19.0")
+ (source (tree-sitter-origin
+ name
+ version
+ "06pjh31bv9ja9hlnykk257a6zh8bsxg2fqa54al7qk1r4n9ksnff"))
+ (home-page "https://github.com/tree-sitter/tree-sitter-json")
+ (synopsis "Tree-sitter JSON grammar")
+ (description
+ "This package provides a JSON grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.34.0
P
P
Pierre Langlois wrote on 10 Feb 2022 23:55
[WIP PATCH v3 15/26] gnu: Add tree-sitter-julia.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220210225527.25189-15-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-julia): New variable.
---
gnu/packages/tree-sitter.scm | 23 +++++++++++++++++++++++
1 file changed, 23 insertions(+)

Toggle diff (31 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index d35edc533d..aeb541224a 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -401,3 +401,26 @@ (define-public tree-sitter-json
(description
"This package provides a JSON grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-julia
+ (package (inherit tree-sitter-c)
+ (name "tree-sitter-julia")
+ (version "0.19.0")
+ (source (tree-sitter-origin
+ name
+ version
+ "1pbnmvhy2gq4vg1b0sjzmjm4s2gsgdjh7h01yj8qrrqbcl29c463"))
+ (arguments
+ (substitute-keyword-arguments (package-arguments tree-sitter-c)
+ ((#:phases phases)
+ `(modify-phases ,phases
+ ;; The parse-example script tries to clone git repositories.
+ (add-after 'unpack 'remove-parse-example
+ (lambda _
+ (substitute* "package.json"
+ ((" && script\\/parse-examples") ""))))))))
+ (home-page "https://github.com/tree-sitter/tree-sitter-julia")
+ (synopsis "Tree-sitter Julia grammar")
+ (description
+ "This package provides a Julia grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.34.0
P
P
Pierre Langlois wrote on 10 Feb 2022 23:55
[WIP PATCH v3 16/26] gnu: Add tree-sitter-php.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220210225527.25189-16-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-php): New variable.
---
gnu/packages/tree-sitter.scm | 23 +++++++++++++++++++++++
1 file changed, 23 insertions(+)

Toggle diff (31 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index aeb541224a..072d2bf588 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -424,3 +424,26 @@ (define-public tree-sitter-julia
(description
"This package provides a Julia grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-php
+ (package (inherit tree-sitter-c)
+ (name "tree-sitter-php")
+ (version "0.19.0")
+ (source (tree-sitter-origin
+ name
+ version
+ "17cmybgpprw7w9d2v7lmc6zmr90d70g0jqq279gzg0mpwfzla53s"))
+ (arguments
+ (substitute-keyword-arguments (package-arguments tree-sitter-c)
+ ((#:phases phases)
+ `(modify-phases ,phases
+ ;; The parse-example script tries to clone git repositories.
+ (add-after 'unpack 'remove-parse-example
+ (lambda _
+ (substitute* "package.json"
+ ((" && script\\/parse-examples") ""))))))))
+ (home-page "https://github.com/tree-sitter/tree-sitter-php")
+ (synopsis "Tree-sitter PHP grammar")
+ (description
+ "This package provides a PHP grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.34.0
P
P
Pierre Langlois wrote on 10 Feb 2022 23:55
[WIP PATCH v3 17/26] gnu: Add tree-sitter-python.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220210225527.25189-17-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-python): New variable.
---
gnu/packages/tree-sitter.scm | 23 +++++++++++++++++++++++
1 file changed, 23 insertions(+)

Toggle diff (31 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 072d2bf588..05d6ade77d 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -447,3 +447,26 @@ (define-public tree-sitter-php
(description
"This package provides a PHP grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-python
+ (package (inherit tree-sitter-c)
+ (name "tree-sitter-python")
+ (version "0.19.1")
+ (source (tree-sitter-origin
+ name
+ version
+ "16i5mcpz4ld0rybffb0nxim4crb9fhnnq20b4bkjs7jjr3zcx7vq"))
+ (arguments
+ (substitute-keyword-arguments (package-arguments tree-sitter-c)
+ ((#:phases phases)
+ `(modify-phases ,phases
+ ;; The parse-example script tries to clone git repositories.
+ (add-after 'unpack 'remove-parse-example
+ (lambda _
+ (substitute* "package.json"
+ ((" && script\\/parse-examples") ""))))))))
+ (home-page "https://github.com/tree-sitter/tree-sitter-python")
+ (synopsis "Tree-sitter Python grammar")
+ (description
+ "This package provides a Python grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.34.0
P
P
Pierre Langlois wrote on 10 Feb 2022 23:55
[WIP PATCH v3 19/26] gnu: Add tree-sitter-typescript.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220210225527.25189-19-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-typescript): New variable.
---
gnu/packages/tree-sitter.scm | 87 ++++++++++++++++++++++++++++++++++++
1 file changed, 87 insertions(+)

Toggle diff (95 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 63e558f50f..2dcf4dae87 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -488,3 +488,90 @@ (define-public tree-sitter-rust
(description
"This package provides a Rust grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-typescript
+ (let ((commit "111b07762e86efab9a918b7c721f720c37e76b0a")
+ (revision "1"))
+ (package (inherit tree-sitter-c)
+ (name "tree-sitter-typescript")
+ (version (git-version "0.20.0" revision commit))
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-typescript")
+ (commit commit)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32 "1364bm3wbqqzvp03cvpx1w89bbqynb1v62i46gy7f6rhib9bf63a"))
+ (modules '((guix build utils)))
+ (snippet
+ '(begin
+ (for-each
+ (lambda (lang)
+ (with-directory-excursion lang
+ (delete-file "src/grammar.json")
+ (delete-file "src/node-types.json")
+ (delete-file "src/parser.c")
+ (delete-file-recursively "src/tree_sitter")))
+ '("typescript" "tsx"))
+ #t))))
+ (inputs
+ (list node-nan
+ `(,tree-sitter-javascript "js")))
+ (arguments
+ (substitute-keyword-arguments (package-arguments tree-sitter-c)
+ ((#:phases phases)
+ #~(modify-phases #$phases
+ ;; The parse-example script tries to clone git repositories.
+ (add-after 'unpack 'remove-parse-example
+ (lambda _
+ (substitute* "package.json"
+ ((" && script\\/parse-examples") ""))))
+ (add-after 'build 'build-gyp
+ (lambda* (#:key inputs #:allow-other-keys)
+ (invoke
+ (search-input-file
+ inputs
+ "/lib/node_modules/npm/bin/node-gyp-bin/node-gyp")
+ "build")))
+ (replace 'tree-sitter-generate
+ (lambda _
+ (for-each (lambda (dir)
+ (with-directory-excursion dir
+ (invoke "tree-sitter" "generate")))
+ '("typescript" "tsx"))))
+ (replace 'build-and-install-lib
+ (lambda* (#:key outputs #:allow-other-keys)
+ (use-modules (guix build json))
+ (let ((name (assoc-ref
+ (call-with-input-file "package.json" read-json)
+ "name"))
+ (lib (string-append #$output "/lib/tree-sitter/")))
+ (mkdir-p lib)
+ (define (source-file? basename)
+ "document me"
+ (let ((path (string-append src "/" basename)))
+ (if (file-exists? path)
+ path
+ #f)))
+ (for-each
+ (lambda (lang)
+ (let ((src (string-append #$output:js
+ "/lib/node_modules/"
+ name "/" lang "/src")))
+ (invoke #$(cc-for-target)
+ "-shared"
+ "-fPIC"
+ "-fno-exceptions"
+ "-O2"
+ "-g"
+ "-o" (string-append lib "/" lang ".so")
+ (string-append src "/scanner.c")
+ (string-append src "/parser.c"))))
+ '("typescript" "tsx")))))))))
+ (home-page "https://github.com/tree-sitter/tree-sitter-typescript")
+ (synopsis "Tree-sitter Typescript grammar")
+ (description
+ "This package provides Typescript and TSX grammars for the Tree-sitter
+library.")
+ (license license:expat))))
--
2.34.0
P
P
Pierre Langlois wrote on 10 Feb 2022 23:55
[WIP PATCH v3 18/26] gnu: Add tree-sitter-rust.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220210225527.25189-18-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-rust): New variable.
---
gnu/packages/tree-sitter.scm | 18 ++++++++++++++++++
1 file changed, 18 insertions(+)

Toggle diff (33 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 05d6ade77d..63e558f50f 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -26,6 +26,7 @@ (define-module (gnu packages tree-sitter)
#:use-module (guix git-download)
#:use-module (guix packages)
#:use-module (guix utils)
+ #:use-module (gnu packages algebra)
#:use-module (gnu packages crates-graphics)
#:use-module (gnu packages crates-io)
#:use-module (gnu packages node-xyz)
@@ -470,3 +471,20 @@ (define-public tree-sitter-python
(description
"This package provides a Python grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-rust
+ (package (inherit tree-sitter-c)
+ (name "tree-sitter-rust")
+ (version "0.20.1")
+ (source (tree-sitter-origin
+ name
+ version
+ "174j5pxwf80c4xniri39l3a6bb7nq96g2s8hh5sgv4i7xvbpfsmg"))
+ (native-inputs
+ (modify-inputs (package-native-inputs tree-sitter-c)
+ (prepend bc)))
+ (home-page "https://github.com/tree-sitter/tree-sitter-rust")
+ (synopsis "Tree-sitter Rust grammar")
+ (description
+ "This package provides a Rust grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.34.0
P
P
Pierre Langlois wrote on 10 Feb 2022 23:55
[WIP PATCH v3 21/26] gnu: rust-emacs-module: Update to 0.18.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220210225527.25189-21-pierre.langlois@gmx.com
* gnu/packages/crates-io.scm (rust-emacs-module-0.18): New variable.
(rust-emacs-module-0.10): Inherit from rust-emacs-module-0.18.
---
gnu/packages/crates-io.scm | 25 +++++++++++++++++++++----
1 file changed, 21 insertions(+), 4 deletions(-)

Toggle diff (55 lines)
diff --git a/gnu/packages/crates-io.scm b/gnu/packages/crates-io.scm
index ad27298404..c039591c56 100644
--- a/gnu/packages/crates-io.scm
+++ b/gnu/packages/crates-io.scm
@@ -19090,10 +19090,10 @@ (define-public rust-emacs-macros-0.11
(description "This package provides proc macros for Emacs modules.")
(license license:bsd-3)))

-(define-public rust-emacs-module-0.10
+(define-public rust-emacs-module-0.18
(package
(name "rust-emacs-module")
- (version "0.10.0")
+ (version "0.18.0")
(source
(origin
(method url-fetch)
@@ -19102,11 +19102,11 @@ (define-public rust-emacs-module-0.10
(string-append name "-" version ".tar.gz"))
(sha256
(base32
- "1gf9lz735xbkyir53dyv362drfx3nin5an5cx39kd8q8kjjwix5g"))))
+ "1ypjyyv2ca3vza4sia91ckxamgfk63yd8frkvg3d4ph4fk4pn1mk"))))
(build-system cargo-build-system)
(arguments
`(#:cargo-inputs
- (("rust-bindgen" ,rust-bindgen-0.48))))
+ (("rust-bindgen" ,rust-bindgen-0.59))))
(inputs
(list clang))
(home-page "https://github.com/ubolonton/emacs-module-rs")
@@ -19115,6 +19115,23 @@ (define-public rust-emacs-module-0.10
Emacs' support for dynamic modules.")
(license license:bsd-3)))

+(define-public rust-emacs-module-0.10
+ (package (inherit rust-emacs-module-0.18)
+ (name "rust-emacs-module")
+ (version "0.10.0")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (crate-uri "emacs_module" version))
+ (file-name
+ (string-append name "-" version ".tar.gz"))
+ (sha256
+ (base32
+ "1gf9lz735xbkyir53dyv362drfx3nin5an5cx39kd8q8kjjwix5g"))))
+ (arguments
+ `(#:cargo-inputs
+ (("rust-bindgen" ,rust-bindgen-0.48))))))
+
(define-public rust-email-0.0.20
(package
(name "rust-email")
--
2.34.0
P
P
Pierre Langlois wrote on 10 Feb 2022 23:55
[WIP PATCH v3 20/26] gnu: Add rust-tree-sitter-for-emacs.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220210225527.25189-20-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (rust-tree-sitter-for-emacs): New variable.
* gnu/packages/patches/rust-tree-sitter-text-provider-fix.patch: New file.
* gnu/local.mk (dist_patch_DATA): Register it.
---
gnu/local.mk | 1 +
.../rust-tree-sitter-text-provider-fix.patch | 98 +++++++++++++++++++
gnu/packages/tree-sitter.scm | 45 +++++++++
3 files changed, 144 insertions(+)
create mode 100644 gnu/packages/patches/rust-tree-sitter-text-provider-fix.patch

Toggle diff (182 lines)
diff --git a/gnu/local.mk b/gnu/local.mk
index 273d80a9c2..f2b6f0afdd 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -1811,6 +1811,7 @@ dist_patch_DATA = \
%D%/packages/patches/rust-nettle-sys-disable-vendor.patch \
%D%/packages/patches/rust-openssl-sys-no-vendor.patch \
%D%/packages/patches/rust-shell2batch-lint-fix.patch \
+ %D%/packages/patches/rust-tree-sitter-text-provider-fix.patch \
%D%/packages/patches/rust-wl-clipboard-rs-newer-wl.patch \
%D%/packages/patches/sbc-fix-build-non-x86.patch \
%D%/packages/patches/sbcl-aserve-add-HTML-5-elements.patch \
diff --git a/gnu/packages/patches/rust-tree-sitter-text-provider-fix.patch b/gnu/packages/patches/rust-tree-sitter-text-provider-fix.patch
new file mode 100644
index 0000000000..beda2d8391
--- /dev/null
+++ b/gnu/packages/patches/rust-tree-sitter-text-provider-fix.patch
@@ -0,0 +1,98 @@
+From 475b822f47bdc58d832533448b6f6d9818554f37 Mon Sep 17 00:00:00 2001
+From: =?UTF-8?q?Tu=E1=BA=A5n-Anh=20Nguy=E1=BB=85n?= <ubolonton@gmail.com>
+Date: Sun, 25 Jul 2021 13:11:52 +0700
+Subject: [PATCH] Allow TextProvider's iterators to generate owned text
+
+---
+ binding_rust/lib.rs | 33 ++++++++++++++++++---------------
+ 1 file changed, 18 insertions(+), 15 deletions(-)
+
+diff --git a/lib/binding_rust/lib.rs b/lib/binding_rust/lib.rs
+index 068a60285..79c3991ed 100644
+--- a/binding_rust/lib.rs
++++ b/binding_rust/lib.rs
+@@ -8,6 +8,7 @@ pub mod allocations;
+ use std::os::unix::io::AsRawFd;
+
+ use std::{
++ borrow::Cow,
+ char, error,
+ ffi::CStr,
+ fmt, hash, iter,
+@@ -162,7 +163,8 @@ pub struct QueryCaptures<'a, 'tree: 'a, T: TextProvider<'a>> {
+ }
+
+ pub trait TextProvider<'a> {
+- type I: Iterator<Item = &'a [u8]> + 'a;
++ type I: Iterator<Item = Cow<'a, [u8]>>;
++
+ fn text(&mut self, node: Node) -> Self::I;
+ }
+
+@@ -1799,19 +1801,19 @@ impl<'a, 'tree> QueryMatch<'a, 'tree> {
+ buffer2: &mut Vec<u8>,
+ text_provider: &mut impl TextProvider<'a>,
+ ) -> bool {
+- fn get_text<'a, 'b: 'a, I: Iterator<Item = &'b [u8]>>(
++ fn get_text<'a, 'b: 'a, I: Iterator<Item = Cow<'b, [u8]>>>(
+ buffer: &'a mut Vec<u8>,
+ mut chunks: I,
+- ) -> &'a [u8] {
+- let first_chunk = chunks.next().unwrap_or(&[]);
++ ) -> Cow<'a, [u8]> {
++ let first_chunk = chunks.next().unwrap_or(Cow::Owned(vec![0u8; 0]));
+ if let Some(next_chunk) = chunks.next() {
+ buffer.clear();
+- buffer.extend_from_slice(first_chunk);
+- buffer.extend_from_slice(next_chunk);
++ buffer.extend_from_slice(&first_chunk);
++ buffer.extend_from_slice(&next_chunk);
+ for chunk in chunks {
+- buffer.extend_from_slice(chunk);
++ buffer.extend_from_slice(&chunk);
+ }
+- buffer.as_slice()
++ Cow::Borrowed(buffer.as_slice())
+ } else {
+ first_chunk
+ }
+@@ -1835,7 +1837,7 @@ impl<'a, 'tree> QueryMatch<'a, 'tree> {
+ TextPredicate::CaptureMatchString(i, r, is_positive) => {
+ let node = self.nodes_for_capture_index(*i).next().unwrap();
+ let text = get_text(buffer1, text_provider.text(node));
+- r.is_match(text) == *is_positive
++ r.is_match(&text) == *is_positive
+ }
+ })
+ }
+@@ -1946,23 +1948,24 @@ impl<'cursor, 'tree> fmt::Debug for QueryMatch<'cursor, 'tree> {
+ }
+ }
+
+-impl<'a, F, I> TextProvider<'a> for F
++impl<'a, F, I, T> TextProvider<'a> for F
+ where
+ F: FnMut(Node) -> I,
+- I: Iterator<Item = &'a [u8]> + 'a,
++ T: Into<Cow<'a, [u8]>>,
++ I: Iterator<Item = T>,
+ {
+- type I = I;
++ type I = iter::Map<I, fn(T) -> Cow<'a, [u8]>>;
+
+ fn text(&mut self, node: Node) -> Self::I {
+- (self)(node)
++ (self)(node).map(T::into)
+ }
+ }
+
+ impl<'a> TextProvider<'a> for &'a [u8] {
+- type I = iter::Once<&'a [u8]>;
++ type I = iter::Once<Cow<'a, [u8]>>;
+
+ fn text(&mut self, node: Node) -> Self::I {
+- iter::once(&self[node.byte_range()])
++ iter::once(Cow::Borrowed(&self[node.byte_range()]))
+ }
+ }
+
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 2dcf4dae87..c86244a300 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -23,9 +23,11 @@ (define-module (gnu packages tree-sitter)
#:use-module (guix build-system gnu)
#:use-module (guix build-system node)
#:use-module (guix gexp)
+ #:use-module (guix download)
#:use-module (guix git-download)
#:use-module (guix packages)
#:use-module (guix utils)
+ #:use-module (gnu packages)
#:use-module (gnu packages algebra)
#:use-module (gnu packages crates-graphics)
#:use-module (gnu packages crates-io)
@@ -146,6 +148,49 @@ (define-public tree-sitter-cli
This package includes the @command{tree-sitter} command-line tool.")
(license license:expat)))

+;; We need to apply a patch in order to compile the rust bindings against the
+;; emacs tree-sitter module.
+;; See https://github.com/tree-sitter/tree-sitter/pull/1294
+(define-public rust-tree-sitter-for-emacs
+ (package
+ (name "rust-tree-sitter")
+ (version "0.20.0")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (crate-uri "tree-sitter" version))
+ (file-name (string-append name "-" version ".tar.gz"))
+ (sha256
+ (base32
+ "1yg4p54hsfsxxknjq492i8b4rvibzpl2zdvr2bwvwakqgah05v33"))
+ (patches (search-patches "rust-tree-sitter-text-provider-fix.patch"))
+ (modules '((guix build utils)))
+ (snippet
+ '(begin
+ ;; Force
+ (delete-file-recursively "src")
+ (delete-file "binding_rust/build.rs")
+ (with-output-to-file "binding_rust/build.rs"
+ (lambda _
+ (format #t "fn main() {~@
+ println!(\"cargo:rustc-link-lib=tree-sitter\");~@
+ }~%")))
+ #f))))
+ (build-system cargo-build-system)
+ (inputs (list tree-sitter))
+ (arguments
+ `(#:tests? #f ;; Running tests misinterprets comments as doc-tests.
+ #:cargo-inputs
+ (("rust-cc" ,rust-cc-1)
+ ("rust-lazy-static" ,rust-lazy-static-1)
+ ("rust-regex" ,rust-regex-1)
+ ("rust-spin" ,rust-spin-0.7))))
+ (home-page "https://tree-sitter.github.io/tree-sitter/")
+ (synopsis "Rust bindings to the Tree-sitter parsing library")
+ (description "This package provides Rust bindings to the Tree-sitter
+parsing library.")
+ (license license:expat)))
+
(define (tree-sitter-origin name version hash)
(origin
(method git-fetch)
--
2.34.0
P
P
Pierre Langlois wrote on 10 Feb 2022 23:55
[WIP PATCH v3 22/26] gnu: rust-emacs-macros: Update to 0.17.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220210225527.25189-22-pierre.langlois@gmx.com
* gnu/packages/crates-io.scm (rust-emacs-macros-0.17): New variable.
(rust-emacs-macros-0.11): Inherit from rust-emacs-macros-0.17.
---
gnu/packages/crates-io.scm | 32 +++++++++++++++++++++++++-------
1 file changed, 25 insertions(+), 7 deletions(-)

Toggle diff (55 lines)
diff --git a/gnu/packages/crates-io.scm b/gnu/packages/crates-io.scm
index c039591c56..ea2e9c9328 100644
--- a/gnu/packages/crates-io.scm
+++ b/gnu/packages/crates-io.scm
@@ -19066,8 +19066,31 @@ (define-public rust-emacs-0.11
Emacs' support for dynamic modules.")
(license license:bsd-3)))

-(define-public rust-emacs-macros-0.11
+(define-public rust-emacs-macros-0.17
(package
+ (name "rust-emacs-macros")
+ (version "0.17.0")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (crate-uri "emacs-macros" version))
+ (file-name (string-append name "-" version ".tar.gz"))
+ (sha256
+ (base32
+ "0qg1dcn5acbirq617qq2fgg9adswif2dnr292s3qnq62wzgnyrb9"))))
+ (build-system cargo-build-system)
+ (arguments
+ `(#:cargo-inputs
+ (("rust-darling" ,rust-darling-0.10)
+ ("rust-quote" ,rust-quote-1)
+ ("rust-syn" ,rust-syn-1))))
+ (home-page "https://github.com/ubolonton/emacs-module-rs")
+ (synopsis "Proc macros for Emacs modules")
+ (description "This package provides proc macros for Emacs modules.")
+ (license license:bsd-3)))
+
+(define-public rust-emacs-macros-0.11
+ (package (inherit rust-emacs-macros-0.17)
(name "rust-emacs-macros")
(version "0.11.0")
(source
@@ -19079,16 +19102,11 @@ (define-public rust-emacs-macros-0.11
(sha256
(base32
"0390y8vafxdi334hhgrzvcqjq3n5ckcmvilqcfp8vajjq8irrly6"))))
- (build-system cargo-build-system)
(arguments
`(#:cargo-inputs
(("rust-darling" ,rust-darling-0.9)
("rust-quote" ,rust-quote-0.6)
- ("rust-syn" ,rust-syn-0.15))))
- (home-page "https://github.com/ubolonton/emacs-module-rs")
- (synopsis "Proc macros for Emacs modules")
- (description "This package provides proc macros for Emacs modules.")
- (license license:bsd-3)))
+ ("rust-syn" ,rust-syn-0.15))))))

(define-public rust-emacs-module-0.18
(package
--
2.34.0
P
P
Pierre Langlois wrote on 10 Feb 2022 23:55
[WIP PATCH v3 24/26] gnu: Add emacs-tree-sitter-core.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220210225527.25189-24-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-emacs-module): New local variable.
(emacs-tree-sitter-core): New variable.
---
gnu/packages/tree-sitter.scm | 78 ++++++++++++++++++++++++++++++++++++
1 file changed, 78 insertions(+)

Toggle diff (93 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index c86244a300..893c02824d 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -20,6 +20,7 @@ (define-module (gnu packages tree-sitter)
#:use-module (guix gexp)
#:use-module ((guix licenses) #:prefix license:)
#:use-module (guix build-system cargo)
+ #:use-module (guix build-system emacs)
#:use-module (guix build-system gnu)
#:use-module (guix build-system node)
#:use-module (guix gexp)
@@ -620,3 +621,80 @@ (define (source-file? basename)
"This package provides Typescript and TSX grammars for the Tree-sitter
library.")
(license license:expat))))
+
+;; Local package definition solely for building the native emacs module
+;; written in Rust.
+(define tree-sitter-emacs-module
+ (package
+ (name "tree-sitter-emacs-module")
+ (version "0.17.0")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/emacs-tree-sitter/elisp-tree-sitter")
+ (commit version)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "0bl7709r4mhb5nmfcsiqj09ja8wn53x9nf5jrr5lq3n1lwz7qq66"))))
+ (build-system cargo-build-system)
+ (inputs
+ (list tree-sitter))
+ (arguments
+ `(#:cargo-inputs
+ (("rust-anyhow" ,rust-anyhow-1)
+ ("rust-emacs" ,rust-emacs-0.18)
+ ("rust-libloading" ,rust-libloading-0.7)
+ ("rust-once-cell" ,rust-once-cell-1)
+ ("rust-tree-sitter" ,rust-tree-sitter-for-emacs))
+ #:phases
+ (modify-phases %standard-phases
+ (add-after 'unpack 'chdir
+ (lambda _ (chdir "core")))
+ (add-after 'chdir 'delete-cargo.lock
+ (lambda _ (delete-file "Cargo.lock")))
+ (add-after 'delete-cargo.lock 'do-not-fetch-from-github
+ (lambda _
+ (substitute* "Cargo.toml"
+ (("\\[patch.*") "")
+ (("git = .*") ""))))
+ (replace 'install
+ (lambda* (#:key outputs #:allow-other-keys)
+ (let ((lib (string-append (assoc-ref outputs "out") "/lib")))
+ (install-file "target/release/libtsc_dyn.so" lib)))))))
+ (home-page #f)
+ (synopsis #f)
+ (description #f)
+ (license license:expat)))
+
+(define-public emacs-tree-sitter-core
+ (package
+ (name "emacs-tree-sitter-core")
+ (version (package-version tree-sitter-emacs-module))
+ (source (package-source tree-sitter-emacs-module))
+ (build-system emacs-build-system)
+ (native-inputs
+ (list tree-sitter-emacs-module))
+ (arguments
+ `(#:phases
+ (modify-phases %standard-phases
+ (add-after 'unpack 'chdir
+ (lambda _ (chdir "core")))
+ (add-after 'install 'install-module
+ (lambda* (#:key inputs outputs #:allow-other-keys)
+ (let ((elpa (elpa-directory (assoc-ref outputs "out")))
+ (module (string-append (assoc-ref inputs "tree-sitter-emacs-module")
+ "/lib/libtsc_dyn.so")))
+ ;; Writing "LOCAL" in this file prevents the package from
+ ;; trying to download the module from the internet.
+ (call-with-output-file (string-append elpa "/DYN-VERSION")
+ (lambda (port) (display "LOCAL" port)))
+ (substitute* "tsc-dyn-get.el"
+ (("defcustom tsc-dyn-dir tsc--dir")
+ (string-append "defcustom tsc-dyn-dir \"" elpa "\"")))
+ (copy-file module (string-append elpa "/tsc-dyn.so"))))))))
+ (home-page "https://github.com/emacs-tree-sitter/elisp-tree-sitter")
+ (synopsis "Tree-sitter bindings for Emacs Lisp, core library")
+ (description "This package provides core APIs of the Emacs binding for
+Tree-sitter, an incremental parsing system.")
+ (license license:expat)))
--
2.34.0
P
P
Pierre Langlois wrote on 10 Feb 2022 23:55
[WIP PATCH v3 23/26] gnu: rust-emacs: Update to 0.18.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220210225527.25189-23-pierre.langlois@gmx.com
* gnu/packages/crates-io.scm (rust-emacs-0.18): New variable.
(rust-emacs-0.11): Inherit from rust-emacs-0.18.
---
gnu/packages/crates-io.scm | 38 +++++++++++++++++++++++++++++++-------
1 file changed, 31 insertions(+), 7 deletions(-)

Toggle diff (70 lines)
diff --git a/gnu/packages/crates-io.scm b/gnu/packages/crates-io.scm
index ea2e9c9328..f9ea194438 100644
--- a/gnu/packages/crates-io.scm
+++ b/gnu/packages/crates-io.scm
@@ -19035,10 +19035,10 @@ (define-public rust-elf-0.0.10
"This package provides a pure-Rust library for parsing ELF files.")
(license (list license:expat license:asl2.0))))

-(define-public rust-emacs-0.11
+(define-public rust-emacs-0.18
(package
(name "rust-emacs")
- (version "0.11.0")
+ (version "0.18.0")
(source
(origin
(method url-fetch)
@@ -19047,16 +19047,18 @@ (define-public rust-emacs-0.11
(string-append name "-" version ".tar.gz"))
(sha256
(base32
- "1xn6nr8m3p24irlmv3i7c9n95yb10qc6aikwqv99qlmhbky5x0z7"))))
+ "0r860i73b2680i2fhdl2l1wwvvmf2zksncpckgkksdcx310ak5v7"))))
(build-system cargo-build-system)
(arguments
`(#:cargo-inputs
- (("rust-ctor" ,rust-ctor-0.1)
- ("rust-emacs-macros" ,rust-emacs-macros-0.11)
- ("rust-emacs-module" ,rust-emacs-module-0.10)
+ (("rust-anyhow" ,rust-anyhow-1)
+ ("rust-ctor" ,rust-ctor-0.1)
+ ("rust-emacs-macros" ,rust-emacs-macros-0.17)
+ ("rust-emacs-module" ,rust-emacs-module-0.18)
("rust-failure" ,rust-failure-0.1)
("rust-failure-derive" ,rust-failure-derive-0.1)
- ("rust-lazy-static" ,rust-lazy-static-1))))
+ ("rust-once-cell" ,rust-once-cell-1)
+ ("rust-thiserror" ,rust-thiserror-1))))
(inputs
(list clang))
(home-page "https://github.com/ubolonton/emacs-module-rs")
@@ -19066,6 +19068,28 @@ (define-public rust-emacs-0.11
Emacs' support for dynamic modules.")
(license license:bsd-3)))

+(define-public rust-emacs-0.11
+ (package (inherit rust-emacs-0.18)
+ (name "rust-emacs")
+ (version "0.11.0")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (crate-uri "emacs" version))
+ (file-name
+ (string-append name "-" version ".tar.gz"))
+ (sha256
+ (base32
+ "1xn6nr8m3p24irlmv3i7c9n95yb10qc6aikwqv99qlmhbky5x0z7"))))
+ (arguments
+ `(#:cargo-inputs
+ (("rust-ctor" ,rust-ctor-0.1)
+ ("rust-emacs-macros" ,rust-emacs-macros-0.11)
+ ("rust-emacs-module" ,rust-emacs-module-0.10)
+ ("rust-failure" ,rust-failure-0.1)
+ ("rust-failure-derive" ,rust-failure-derive-0.1)
+ ("rust-lazy-static" ,rust-lazy-static-1))))))
+
(define-public rust-emacs-macros-0.17
(package
(name "rust-emacs-macros")
--
2.34.0
P
P
Pierre Langlois wrote on 10 Feb 2022 23:55
[WIP PATCH v3 25/26] gnu: Add emacs-tree-sitter.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220210225527.25189-25-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (emacs-tree-sitter): New variable.
---
gnu/packages/tree-sitter.scm | 27 +++++++++++++++++++++++++++
1 file changed, 27 insertions(+)

Toggle diff (35 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 893c02824d..b412f3d031 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -698,3 +698,30 @@ (module (string-append (assoc-ref inputs "tree-sitter-emacs-module")
(description "This package provides core APIs of the Emacs binding for
Tree-sitter, an incremental parsing system.")
(license license:expat)))
+
+(define-public emacs-tree-sitter
+ (package
+ (name "emacs-tree-sitter")
+ (version (package-version emacs-tree-sitter-core))
+ (source (package-source emacs-tree-sitter-core))
+ (build-system emacs-build-system)
+ (propagated-inputs
+ (list emacs-tree-sitter-core))
+ (arguments
+ `(#:phases
+ (modify-phases %standard-phases
+ (add-after 'unpack 'chdir
+ (lambda _ (chdir "lisp"))))))
+ (home-page "https://github.com/emacs-tree-sitter/elisp-tree-sitter")
+ (synopsis "Tree-sitter bindings for Emacs Lisp")
+ (description "This package provides Emacs bindings for Tree-sitter, an
+incremental parsing library. It aims to be the foundation for a new breed of
+Emacs packages that understand code structurally. For example:
+
+@enumerate
+@item Faster, fine-grained code highlighting.
+@item More flexible code folding.
+@item Structural editing (like Paredit, or even better) for non-Lisp code.
+@item More informative indexing for imenu.
+@end enumerate")
+ (license license:expat)))
--
2.34.0
P
P
Pierre Langlois wrote on 10 Feb 2022 23:55
[WIP PATCH v3 26/26] gnu: Add emacs-tree-sitter-langs.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220210225527.25189-26-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (emacs-tree-sitter-langs): New variable.
---
gnu/packages/tree-sitter.scm | 81 ++++++++++++++++++++++++++++++++++++
1 file changed, 81 insertions(+)

Toggle diff (89 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index b412f3d031..00b91f5c91 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -725,3 +725,84 @@ (define-public emacs-tree-sitter
@item More informative indexing for imenu.
@end enumerate")
(license license:expat)))
+
+(define-public emacs-tree-sitter-langs
+ (package
+ (name "emacs-tree-sitter-langs")
+ (version "0.11.1")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/emacs-tree-sitter/tree-sitter-langs")
+ (commit version)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "0r3yw25czg1vcxk9015yd8p1n0z427z2l4hizfjy8w8jjjg7cxxf"))))
+ (build-system emacs-build-system)
+ ;; FIXME: Support for some languages, such as ruby, is still missing from
+ ;; the bundle.
+ (inputs
+ (list tree-sitter-bash
+ tree-sitter-c
+ tree-sitter-cpp
+ tree-sitter-css
+ tree-sitter-go
+ tree-sitter-html
+ tree-sitter-java
+ tree-sitter-javascript
+ tree-sitter-json
+ tree-sitter-julia
+ tree-sitter-php
+ tree-sitter-python
+ tree-sitter-rust
+ tree-sitter-typescript))
+ (propagated-inputs
+ (list emacs-tree-sitter))
+ (arguments
+ `(#:phases
+ (modify-phases %standard-phases
+ (add-before 'build 'fake-bundle
+ (lambda _
+ ;; The BUNDLE-VERSION file prevents the package from
+ ;; downloading libraries at load time.
+ (call-with-output-file "bin/BUNDLE-VERSION"
+ (lambda (port) (display ,version port)))))
+ (add-after 'install 'install-libs
+ (lambda* (#:key outputs inputs #:allow-other-keys)
+ (let ((elpa (elpa-directory (assoc-ref outputs "out"))))
+ (mkdir-p (string-append elpa "/bin"))
+ ;; The BUNDLE-VERSION file prevents the package from
+ ;; downloading libraries at load time.
+ (call-with-output-file (string-append elpa "/bin/BUNDLE-VERSION")
+ (lambda (port) (display ,version port)))
+
+ (define (install-lib lang)
+ (let ((input (assoc-ref inputs (string-append "tree-sitter-" lang))))
+ (symlink (string-append input "/lib/tree-sitter/" lang ".so")
+ (string-append elpa "/bin/" lang ".so"))))
+ (for-each (lambda (input)
+ (use-modules (ice-9 regex))
+ (cond
+ ((string-match "^tree-sitter-(.*)$" (car input))
+ => (lambda (m)
+ (install-lib (match:substring m 1))))))
+ inputs))))
+ (add-after 'install 'install-queries
+ (lambda* (#:key outputs #:allow-other-keys)
+ (let ((elpa (elpa-directory (assoc-ref outputs "out"))))
+ (copy-recursively "queries" (string-append elpa "/queries"))))))))
+ (home-page "https://ubolonton.github.io/emacs-tree-sitter/languages/")
+ (synopsis "Language support bundle for Tree-sitter")
+ (description "This package is a convenient language bundle for
+Tree-sitter. For each supported language, this package provides:
+
+@enumerate
+@item Pre-compiled grammar binaries.
+@item An optional highlights.scm file that provides highlighting patterns.
+This is mainly intended for major modes that are not aware of tree-sitter.
+@item Optional query patterns for other minor modes that provide high-level
+functionalities on top of tree-sitter, such as code folding, evil text
+objects, ...etc.
+@end enumerate")
+ (license license:expat)))
--
2.34.0
P
P
Pierre Langlois wrote on 18 Feb 2022 15:12
[PATCH v4 00/31] gnu: Add tree-sitter for emacs.
(address . 49946@debbugs.gnu.org)
877d9s2qa9.fsf@gmx.com
Hi Guix!

Here's a v4 that's in a pretty good shape IMO. Here's a summary of the
changes compared to previous versions, there were lots actually! :-)

- Removed bundled icu4c headers from `tree-sitter', as pointed out on

- Added a snippet that removes all files generated by the
`tree-sitter generate' command, to be used in all grammar packages.

- For each grammar package, provide two outputs:

* "out": With just the compiled grammar.

Toggle snippet (10 lines)
/gnu/store/mb2hcaanm6ld8rx19p1ihk2fs1m7lcj8-tree-sitter-c-0.20.1
??? lib
?   ??? tree-sitter
?   ??? c.so
??? share
??? doc
??? tree-sitter-c-0.20.1
??? LICENSE

* "js": With the JS package output, so that it may be used as a
dependency. For example C++ imports the C grammar, and Typescript
imports JavaScript.

- Removed the dependency on node-gyp!

By default, the `npm' build system builds JavaScript bindings for each
grammar, so that node applications may use them as a library to parse
a given language. We don't need this yet in Guix, we just want to
generate C code from the JS grammar description and compile it.

So when invoking `tree-sitter generate', we disable bindings
generation and then we ignore all custom scripts when installing the
JS package. This simplifies grammar packages a *lot*, given those
custom script often try to download binaries and would have had to be
patched.

- Grammars are compiled manually.

Previous versions relied on doing `tree-sitter test' so that a grammar
would be loaded and compiled by `tree-sitter' into its own cache. We
would then install the library from that cache location, which can
change across versions.

This felt like a hack, looking at the source code, we're just
compiling a couple of files at most, we can do that ourselve.

- I added support for a few extra languages, like ruby, C#, elixir and
ocaml.

- Last but very much not least, tests for emacs-tree-sitter-langs are
enabled!

This is important because this package doesn't just bundle all the
grammars, it also provides custom highlighting rules, which depend on
the grammar and can easily be out-of-sync and break. Thanks to
enabling these tests, I realized the php grammar need updating and
didn't work for example.

Let me know if you have any feedback on this series! It's taking me
quite some time to follow it through, it's not the easiest thing to
package :-/.

I do wonder if I'm going about it the right way for Guix though, for
instance I'm wondering if I should rewrite the grammar packages using a
new custom build system, do people think it would be worth it? That
could be done as a follow-up of course.

Thanks,
Pierre

Pierre Langlois (31):
gnu: Add rust-smallbitvec.
gnu: Add rust-html-escape.
gnu: Add rust-spin@0.7.
gnu: Add tree-sitter.
gnu: Add tree-sitter-cli.
gnu: Add tree-sitter-c.
gnu: Add tree-sitter-cpp.
gnu: Add tree-sitter-bash.
gnu: Add tree-sitter-css.
gnu: Add tree-sitter-c-sharp.
gnu: Add tree-sitter-elixir.
gnu: Add tree-sitter-elm.
gnu: Add tree-sitter-go.
gnu: Add tree-sitter-html.
gnu: Add tree-sitter-java.
gnu: Add tree-sitter-javascript.
gnu: Add tree-sitter-json.
gnu: Add tree-sitter-julia.
gnu: Add tree-sitter-ocaml.
gnu: Add tree-sitter-php.
gnu: Add tree-sitter-python.
gnu: Add tree-sitter-ruby.
gnu: Add tree-sitter-rust.
gnu: Add tree-sitter-typescript.
gnu: Add rust-tree-sitter-for-emacs.
gnu: rust-emacs-module: Update to 0.18.
gnu: rust-emacs-macros: Update to 0.17.
gnu: rust-emacs: Update to 0.18.
gnu: Add emacs-tree-sitter-core.
gnu: Add emacs-tree-sitter.
gnu: Add emacs-tree-sitter-langs.

gnu/local.mk | 2 +
gnu/packages/crates-io.scm | 154 ++-
.../rust-tree-sitter-text-provider-fix.patch | 98 ++
gnu/packages/tree-sitter.scm | 1088 +++++++++++++++++
4 files changed, 1324 insertions(+), 18 deletions(-)
create mode 100644 gnu/packages/patches/rust-tree-sitter-text-provider-fix.patch
create mode 100644 gnu/packages/tree-sitter.scm

--
2.34.0
-----BEGIN PGP SIGNATURE-----

iQFMBAEBCgA2FiEEctU9gYy29KFyWDdMqPyeRH9PfVQFAmIPr24YHHBpZXJyZS5s
YW5nbG9pc0BnbXguY29tAAoJEKj8nkR/T31U3M0IAKDg2dimfgTh9aElA7Vx2Rpd
JvCCl4RpfcvCSoxSTwwd1x8pzuCYlvjrTCSrT+byqxFVg0kjexTSPymtPKZRSyT5
VB2tzPGkaFaud2WwHptE2s7fRC45zUfXupyuc7cDA6hWtCTBCCvlcSVrvPCr6vXa
NGxC6gCuFb3samZPV/tImMBA5+h0R/HoZ9j3I8FFj9KwV3l7bbT0CG3Przdyfjpd
rvUO41AH4WChFT4p8haTG+Mlp/Pl+0LvaZewUBDNHJOrjeni9814Jqzx9nKEcyMr
/D1m3j7oWWAf1qDIXV9o8u1fqRZOPeBEdOieuJ4vtJWl3Va+1so5BKBxK89ySlg=
=8jYk
-----END PGP SIGNATURE-----

P
P
Pierre Langlois wrote on 18 Feb 2022 15:39
[PATCH v4 01/31] gnu: Add rust-smallbitvec.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220218143948.28989-1-pierre.langlois@gmx.com
* gnu/packages/crates-io.scm (rust-smallbitvec-2): New variable.
---
gnu/packages/crates-io.scm | 22 ++++++++++++++++++++++
1 file changed, 22 insertions(+)

Toggle diff (33 lines)
diff --git a/gnu/packages/crates-io.scm b/gnu/packages/crates-io.scm
index 6024c9c57a..291c289b5a 100644
--- a/gnu/packages/crates-io.scm
+++ b/gnu/packages/crates-io.scm
@@ -56109,6 +56109,28 @@ (define-public rust-sluice-0.4
("rust-futures-core-preview" ,rust-futures-core-preview-0.3)
("rust-futures-io-preview" ,rust-futures-io-preview-0.3))))))

+(define-public rust-smallbitvec-2
+ (package
+ (name "rust-smallbitvec")
+ (version "2.5.1")
+ (source (origin
+ (method url-fetch)
+ (uri (crate-uri "smallbitvec" version))
+ (file-name (string-append name "-" version ".tar.gz"))
+ (sha256
+ (base32
+ "0plrbldsjpwip3afbzd8fgrnvdhizcg5z4ncfqs4q6x4qjflzkkm"))))
+ (build-system cargo-build-system)
+ (arguments
+ `(#:cargo-development-inputs
+ (("rust-bit-vec" ,rust-bit-vec-0.4)
+ ("rust-rand" ,rust-rand-0.4))))
+ (home-page "https://github.com/servo/smallbitvec")
+ (synopsis "A bit vector optimized for size and inline storage")
+ (description "This package provides a bit vector optimized for size and
+inline storage")
+ (license (list license:expat license:asl2.0))))
+
(define-public rust-smallvec-1
(package
(name "rust-smallvec")
--
2.34.0
P
P
Pierre Langlois wrote on 18 Feb 2022 15:39
[PATCH v4 02/31] gnu: Add rust-html-escape.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220218143948.28989-2-pierre.langlois@gmx.com
* gnu/packages/crates-io.scm (rust-html-escape-0.2): New variable.
---
gnu/packages/crates-io.scm | 24 ++++++++++++++++++++++++
1 file changed, 24 insertions(+)

Toggle diff (35 lines)
diff --git a/gnu/packages/crates-io.scm b/gnu/packages/crates-io.scm
index 291c289b5a..986901740a 100644
--- a/gnu/packages/crates-io.scm
+++ b/gnu/packages/crates-io.scm
@@ -26943,6 +26943,30 @@ (define-public rust-htmlescape-0.3
"This package provides a library for HTML entity encoding and decoding.")
(license (list license:asl2.0 license:expat license:mpl2.0))))

+(define-public rust-html-escape-0.2
+ (package
+ (name "rust-html-escape")
+ (version "0.2.9")
+ (source (origin
+ (method url-fetch)
+ (uri (crate-uri "html-escape" version))
+ (file-name (string-append name "-" version ".tar.gz"))
+ (sha256
+ (base32
+ "1dxw9lpckrqzzqgbkw64ckbajr4b7xxnjdn8adgzqf2mm40shvl1"))))
+ (build-system cargo-build-system)
+ (arguments
+ `(#:cargo-inputs
+ (("rust-utf8-width" ,rust-utf8-width-0.1))
+ #:cargo-development-inputs
+ (("rust-bencher" ,rust-bencher-0.1))))
+ (home-page "https://magiclen.org/html-escape")
+ (synopsis "Library for encoding and escaping special characters in HTML")
+ (description
+ "This package provides a library for encoding and escaping special
+characters in HTML, decoding and unescaping HTML entities as well.")
+ (license license:expat)))
+
(define-public rust-http-0.2
(package
(name "rust-http")
--
2.34.0
P
P
Pierre Langlois wrote on 18 Feb 2022 15:39
[PATCH v4 03/31] gnu: Add rust-spin@0.7.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220218143948.28989-3-pierre.langlois@gmx.com
* gnu/packages/crates-io.scm (rust-spin-0.7): New variable.
---
gnu/packages/crates-io.scm | 13 +++++++++++++
1 file changed, 13 insertions(+)

Toggle diff (24 lines)
diff --git a/gnu/packages/crates-io.scm b/gnu/packages/crates-io.scm
index 986901740a..a4bd79607c 100644
--- a/gnu/packages/crates-io.scm
+++ b/gnu/packages/crates-io.scm
@@ -56726,6 +56726,19 @@ (define-public rust-spin-0.9
initializers are available.")
(license license:expat)))

+(define-public rust-spin-0.7
+ (package
+ (inherit rust-spin-0.9)
+ (name "rust-spin")
+ (version "0.7.1")
+ (source (origin
+ (method url-fetch)
+ (uri (crate-uri "spin" version))
+ (file-name (string-append name "-" version ".tar.gz"))
+ (sha256
+ (base32
+ "0qjips9f6fsvkyd7wj3a4gzaqknn2q4kkb19957pl86im56pna0k"))))))
+
(define-public rust-spin-0.5
(package
(inherit rust-spin-0.9)
--
2.34.0
P
P
Pierre Langlois wrote on 18 Feb 2022 15:39
[PATCH v4 04/31] gnu: Add tree-sitter.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220218143948.28989-4-pierre.langlois@gmx.com
* gnu/package/tree-sitter.scm: New file.
* gnu/local.mk (GNU_SYSTEM_MODULES): Register it.
---
gnu/local.mk | 1 +
gnu/packages/tree-sitter.scm | 83 ++++++++++++++++++++++++++++++++++++
2 files changed, 84 insertions(+)
create mode 100644 gnu/packages/tree-sitter.scm

Toggle diff (101 lines)
diff --git a/gnu/local.mk b/gnu/local.mk
index 433467ee39..97ab160c3c 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -584,6 +584,7 @@ GNU_SYSTEM_MODULES = \
%D%/packages/tmux.scm \
%D%/packages/toolkits.scm \
%D%/packages/tor.scm \
+ %D%/packages/tree-sitter.scm \
%D%/packages/tv.scm \
%D%/packages/uglifyjs.scm \
%D%/packages/uml.scm \
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
new file mode 100644
index 0000000000..10a3a6e57c
--- /dev/null
+++ b/gnu/packages/tree-sitter.scm
@@ -0,0 +1,83 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2022 Pierre Langlois <pierre.langlois@gmx.com>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu packages tree-sitter)
+ #:use-module (guix gexp)
+ #:use-module ((guix licenses) #:prefix license:)
+ #:use-module (guix build-system gnu)
+ #:use-module (guix git-download)
+ #:use-module (guix packages)
+ #:use-module (guix utils)
+ #:use-module (gnu packages icu4c))
+
+(define-public tree-sitter
+ (package
+ (name "tree-sitter")
+ (version "0.20.4")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "1yldgdbf3l5l4ki52abdf81nwkcbvg219gwr3ydcjwfsg7hf7zhz"))
+ (modules '((guix build utils)))
+ (snippet
+ '(begin
+ (delete-file-recursively "lib/src/unicode")))))
+ (inputs
+ (list icu4c))
+ (build-system gnu-build-system)
+ (arguments
+ (list
+ #:tests? #f ;No check target.
+ #:phases
+ #~(modify-phases %standard-phases
+ (delete 'configure)
+ (add-before 'build 'set-cc
+ (lambda _
+ (setenv "CC"
+ #$(cc-for-target))))
+ (replace 'install
+ (lambda* (#:key outputs #:allow-other-keys)
+ (let* ((out (assoc-ref outputs "out")) (lib (string-append
+ out "/lib")))
+ (setenv "PREFIX" out)
+ (invoke "make" "install")))))))
+ (home-page "https://tree-sitter.github.io/tree-sitter/")
+ (synopsis "Incremental parsing system for programming tools")
+ (description
+ "Tree-sitter is a parser generator tool and an incremental
+parsing library. It can build a concrete syntax tree for a source file and
+efficiently update the syntax tree as the source file is edited.
+
+Tree-sitter aims to be:
+
+@enumerate
+@item General enough to parse any programming language.
+@item Fast enough to parse on every keystroke in a text editor.
+@item Robust enough to provide useful results even in the presence of syntax
+errors.
+@item Dependency-free so that the runtime library (which is written in pure C)
+can be embedded in any application.
+@end enumerate
+
+This package includes the @code{libtree-sitter} runtime library.")
+ (license license:expat)))
--
2.34.0
P
P
Pierre Langlois wrote on 18 Feb 2022 15:39
[PATCH v4 05/31] gnu: Add tree-sitter-cli.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220218143948.28989-5-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-cli): New variable.
---
gnu/packages/tree-sitter.scm | 66 ++++++++++++++++++++++++++++++++++++
1 file changed, 66 insertions(+)

Toggle diff (85 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 10a3a6e57c..cbf8ecdc6c 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -19,10 +19,13 @@
(define-module (gnu packages tree-sitter)
#:use-module (guix gexp)
#:use-module ((guix licenses) #:prefix license:)
+ #:use-module (guix build-system cargo)
#:use-module (guix build-system gnu)
#:use-module (guix git-download)
#:use-module (guix packages)
#:use-module (guix utils)
+ #:use-module (gnu packages crates-graphics)
+ #:use-module (gnu packages crates-io)
#:use-module (gnu packages icu4c))

(define-public tree-sitter
@@ -81,3 +84,66 @@ (define-public tree-sitter

This package includes the @code{libtree-sitter} runtime library.")
(license license:expat)))
+
+(define-public tree-sitter-cli
+ (package (inherit tree-sitter)
+ (name "tree-sitter-cli")
+ (build-system cargo-build-system)
+ (arguments
+ `(;; Running test requires downloading fixtures, see the
+ ;; script/fetch-fixtures script.
+ #:tests? #f
+ ;; FIXME: Installing the sources for the tree-sitter Rust bindings
+ ;; doesn't work out of the box due to tree-sitter having multiple
+ ;; Rust packages in the same repository (bindings and CLI).
+ #:install-source? #f
+ #:cargo-inputs
+ (("rust-ansi-term" ,rust-ansi-term-0.12)
+ ("rust-anyhow" ,rust-anyhow-1)
+ ("rust-atty" ,rust-atty-0.2)
+ ("rust-clap" ,rust-clap-2)
+ ("rust-difference" ,rust-difference-2)
+ ("rust-dirs" ,rust-dirs-3)
+ ("rust-html-escape" ,rust-html-escape-0.2)
+ ("rust-libloading" ,rust-libloading-0.7)
+ ("rust-rustc-hash" ,rust-rustc-hash-1)
+ ("rust-serde" ,rust-serde-1)
+ ("rust-smallbitvec" ,rust-smallbitvec-2)
+ ("rust-spin" ,rust-spin-0.7)
+ ("rust-thiserror" ,rust-thiserror-1)
+ ("rust-tiny-http" ,rust-tiny-http-0.8)
+ ("rust-toml" ,rust-toml-0.5)
+ ("rust-walkdir" ,rust-walkdir-2)
+ ("rust-webbrowser" ,rust-webbrowser-0.5)
+ ("rust-which" ,rust-which-4))
+ #:cargo-development-inputs
+ (("rust-pretty-assertions" ,rust-pretty-assertions-0.7))
+ #:phases
+ (modify-phases %standard-phases
+ (add-after 'unpack 'delete-cargo.lock
+ (lambda _ (delete-file "Cargo.lock")))
+ (replace 'install
+ (lambda* (#:key outputs #:allow-other-keys)
+ (let* ((out (assoc-ref outputs "out"))
+ (bin (string-append out "/bin"))
+ (lib (string-append out "/lib")))
+ (mkdir-p bin)
+ (install-file "target/release/tree-sitter" bin)))))))
+ (synopsis "Incremental parsing system for programming tools")
+ (description "Tree-sitter is a parser generator tool and an incremental
+parsing library. It can build a concrete syntax tree for a source file and
+efficiently update the syntax tree as the source file is edited.
+
+Tree-sitter aims to be:
+
+@enumerate
+@item General enough to parse any programming language.
+@item Fast enough to parse on every keystroke in a text editor.
+@item Robust enough to provide useful results even in the presence of syntax
+errors.
+@item Dependency-free so that the runtime library (which is written in pure C)
+can be embedded in any application.
+@end enumerate
+
+This package includes the @command{tree-sitter} command-line tool.")
+ (license license:expat)))
--
2.34.0
P
P
Pierre Langlois wrote on 18 Feb 2022 15:39
[PATCH v4 10/31] gnu: Add tree-sitter-c-sharp.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220218143948.28989-10-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-c-sharp): New variable.
---
gnu/packages/tree-sitter.scm | 21 +++++++++++++++++++++
1 file changed, 21 insertions(+)

Toggle diff (32 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index f26732197e..5bb7e7549a 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -268,6 +268,27 @@ (define-public tree-sitter-cpp
"This package provides a C++ grammar for the Tree-sitter library.")
(license license:expat)))

+(define-public tree-sitter-c-sharp
+ (package (inherit tree-sitter-c)
+ (name "tree-sitter-c-sharp")
+ (version "0.19.1")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-c-sharp")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "054fmpf47cwh59gbg00sc0nl237ba4rnxi73miz39yqzcs87055r"))
+ (modules '((guix build utils)))
+ (snippet tree-sitter-delete-generated-files)))
+ (home-page "https://github.com/tree-sitter/tree-sitter-c-sharp")
+ (synopsis "Tree-sitter C# grammar")
+ (description
+ "This package provides a C# grammar for the Tree-sitter library.")
+ (license license:expat)))
+
(define-public tree-sitter-bash
(package (inherit tree-sitter-c)
(name "tree-sitter-bash")
--
2.34.0
P
P
Pierre Langlois wrote on 18 Feb 2022 15:39
[PATCH v4 07/31] gnu: Add tree-sitter-cpp.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220218143948.28989-7-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-cpp): New variable.
---
gnu/packages/tree-sitter.scm | 24 ++++++++++++++++++++++++
1 file changed, 24 insertions(+)

Toggle diff (32 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 920c386fdf..e82c6e60a9 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -243,3 +243,27 @@ (define (source-file path)
(description
"This package provides a C grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-cpp
+ (package
+ (inherit tree-sitter-c)
+ (name "tree-sitter-cpp")
+ (version "0.20.0")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-cpp")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "0hxcpdvyyig8njga1mxp4qcnbbnr1d0aiy27vahijwbh98b081nr"))
+ (modules '((guix build utils)))
+ (snippet tree-sitter-delete-generated-files)))
+ (inputs
+ (list `(,tree-sitter-c "js")))
+ (home-page "https://github.com/tree-sitter/tree-sitter-cpp")
+ (synopsis "Tree-sitter C++ grammar")
+ (description
+ "This package provides a C++ grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.34.0
P
P
Pierre Langlois wrote on 18 Feb 2022 15:39
[PATCH v4 08/31] gnu: Add tree-sitter-bash.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220218143948.28989-8-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-bash): New variable.
---
gnu/packages/tree-sitter.scm | 32 ++++++++++++++++++++++++++++++++
1 file changed, 32 insertions(+)

Toggle diff (40 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index e82c6e60a9..8f12559dde 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -267,3 +267,35 @@ (define-public tree-sitter-cpp
(description
"This package provides a C++ grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-bash
+ (package (inherit tree-sitter-c)
+ (name "tree-sitter-bash")
+ (version "0.19.0")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-bash")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "18c030bb65r50i6z37iy7jb9z9i8i36y7b08dbc9bchdifqsijs5"))
+ (modules '((guix build utils)))
+ (snippet tree-sitter-delete-generated-files)))
+ (arguments
+ (substitute-keyword-arguments (package-arguments tree-sitter-c)
+ ((#:phases phases)
+ `(modify-phases ,phases
+ (replace 'delete-dependencies
+ (lambda _
+ (delete-dependencies '("tree-sitter-cli"
+ "nan"
+ "node-gyp"
+ "prebuild"
+ "prebuild-install"))))))))
+ (home-page "https://github.com/tree-sitter/tree-sitter-bash")
+ (synopsis "Tree-sitter Bash grammar")
+ (description
+ "This package provides a Bash grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.34.0
P
P
Pierre Langlois wrote on 18 Feb 2022 15:39
[PATCH v4 09/31] gnu: Add tree-sitter-css.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220218143948.28989-9-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-css): New variable.
---
gnu/packages/tree-sitter.scm | 21 +++++++++++++++++++++
1 file changed, 21 insertions(+)

Toggle diff (29 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 8f12559dde..f26732197e 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -299,3 +299,24 @@ (define-public tree-sitter-bash
(description
"This package provides a Bash grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-css
+ (package (inherit tree-sitter-c)
+ (name "tree-sitter-css")
+ (version "0.19.0")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-css")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "014jrlgi7zfza9g38hsr4vlbi8964i5p7iglaih6qmzaiml7bja2"))
+ (modules '((guix build utils)))
+ (snippet tree-sitter-delete-generated-files)))
+ (home-page "https://github.com/tree-sitter/tree-sitter-css")
+ (synopsis "Tree-sitter CSS grammar")
+ (description
+ "This package provides a CSS grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.34.0
P
P
Pierre Langlois wrote on 18 Feb 2022 15:39
[PATCH v4 14/31] gnu: Add tree-sitter-html.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220218143948.28989-14-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-html): New variable.
---
gnu/packages/tree-sitter.scm | 21 +++++++++++++++++++++
1 file changed, 21 insertions(+)

Toggle diff (29 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index c1e58d9c8e..4bb9425eb1 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -430,3 +430,24 @@ (define-public tree-sitter-go
(description
"This package provides a Golang grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-html
+ (package (inherit tree-sitter-c)
+ (name "tree-sitter-html")
+ (version "0.19.0")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-html")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "1hg7vbcy7bir6b8x11v0a4x0glvqnsqc3i2ixiarbxmycbgl3axy"))
+ (modules '((guix build utils)))
+ (snippet tree-sitter-delete-generated-files)))
+ (home-page "https://github.com/tree-sitter/tree-sitter-html")
+ (synopsis "Tree-sitter HTML grammar")
+ (description
+ "This package provides a HTML grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.34.0
P
P
Pierre Langlois wrote on 18 Feb 2022 15:39
[PATCH v4 15/31] gnu: Add tree-sitter-java.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220218143948.28989-15-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-java): New variable.
---
gnu/packages/tree-sitter.scm | 21 +++++++++++++++++++++
1 file changed, 21 insertions(+)

Toggle diff (29 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 4bb9425eb1..d93f6a584a 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -451,3 +451,24 @@ (define-public tree-sitter-html
(description
"This package provides a HTML grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-java
+ (package (inherit tree-sitter-c)
+ (name "tree-sitter-java")
+ (version "0.19.1")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-java")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "07zw9ygb45hnvlx9qlz7rlz8hc3byjy03d24v72i5iyhpiiwlhvl"))
+ (modules '((guix build utils)))
+ (snippet tree-sitter-delete-generated-files)))
+ (home-page "https://github.com/tree-sitter/tree-sitter-java")
+ (synopsis "Tree-sitter Java grammar")
+ (description
+ "This package provides a Java grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.34.0
P
P
Pierre Langlois wrote on 18 Feb 2022 15:39
[PATCH v4 11/31] gnu: Add tree-sitter-elixir.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220218143948.28989-11-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-elixir): New variable.
---
gnu/packages/tree-sitter.scm | 35 +++++++++++++++++++++++++++++++++++
1 file changed, 35 insertions(+)

Toggle diff (43 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 5bb7e7549a..134d2768cf 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -341,3 +341,38 @@ (define-public tree-sitter-css
(description
"This package provides a CSS grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-elixir
+ (let ((commit "de20391afe5cb03ef1e8a8e43167e7b58cc52869")
+ (revision "1"))
+ (package (inherit tree-sitter-c)
+ (name "tree-sitter-elixir")
+ (version (git-version "0.19.0" revision commit))
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/elixir-lang/tree-sitter-elixir")
+ (commit commit)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "0zrkrwhw3g1vazsxcwrfd1fk4wvs9hdwmwp6073mfh370bz4140h"))
+ (modules '((guix build utils)))
+ (snippet tree-sitter-delete-generated-files)))
+ (arguments
+ (substitute-keyword-arguments (package-arguments tree-sitter-c)
+ ((#:phases phases)
+ `(modify-phases ,phases
+ (replace 'delete-dependencies
+ (lambda _
+ (delete-dependencies '("tree-sitter-cli"
+ "node-gyp"
+ "nan"
+ "prettier"))))))))
+ (home-page "https://elixir-lang.org/tree-sitter-elixir/")
+ (synopsis "Tree-sitter Elixir grammar")
+ (description
+ "This package provides a Elixir grammar for the Tree-sitter library.")
+ (license (list license:asl2.0
+ ;; Files in tests/corpus are under MIT license.
+ license:expat)))))
--
2.34.0
P
P
Pierre Langlois wrote on 18 Feb 2022 15:39
[PATCH v4 13/31] gnu: Add tree-sitter-go.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220218143948.28989-13-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-go): New variable.
---
gnu/packages/tree-sitter.scm | 21 +++++++++++++++++++++
1 file changed, 21 insertions(+)

Toggle diff (29 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 5ade030283..c1e58d9c8e 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -409,3 +409,24 @@ (define-public tree-sitter-elm
(description
"This package provides an Elm grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-go
+ (package (inherit tree-sitter-c)
+ (name "tree-sitter-go")
+ (version "0.19.1")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-go")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "0nxs47vd2fc2fr0qlxq496y852rwg39flhg334s7dlyq7d3lcx4x"))
+ (modules '((guix build utils)))
+ (snippet tree-sitter-delete-generated-files)))
+ (home-page "https://github.com/tree-sitter/tree-sitter-go")
+ (synopsis "Tree-sitter Go grammar")
+ (description
+ "This package provides a Golang grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.34.0
P
P
Pierre Langlois wrote on 18 Feb 2022 15:39
[PATCH v4 12/31] gnu: Add tree-sitter-elm.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220218143948.28989-12-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-elm): New variable.
---
gnu/packages/tree-sitter.scm | 33 +++++++++++++++++++++++++++++++++
1 file changed, 33 insertions(+)

Toggle diff (41 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 134d2768cf..5ade030283 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -376,3 +376,36 @@ (define-public tree-sitter-elixir
(license (list license:asl2.0
;; Files in tests/corpus are under MIT license.
license:expat)))))
+
+(define-public tree-sitter-elm
+ (package (inherit tree-sitter-c)
+ (name "tree-sitter-elm")
+ (version "5.5.1")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/elm-tooling/tree-sitter-elm")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "10hbi4vyj4hjixqswdcbvzl60prldczz29mlp02if61wvwiwvqrw"))
+ (modules '((guix build utils)))
+ (snippet tree-sitter-delete-generated-files)))
+ (arguments
+ (substitute-keyword-arguments (package-arguments tree-sitter-c)
+ ((#:phases phases)
+ `(modify-phases ,phases
+ (replace 'delete-dependencies
+ (lambda _
+ (delete-dependencies '("tree-sitter-cli"
+ "nan"
+ "node-gyp"
+ "@asgerf/dts-tree-sitter"
+ "prebuild"
+ "prebuild-install"))))))))
+ (home-page "https://elm-tooling.github.io/tree-sitter-elm/")
+ (synopsis "Tree-sitter Elm grammar")
+ (description
+ "This package provides an Elm grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.34.0
P
P
Pierre Langlois wrote on 18 Feb 2022 15:39
[PATCH v4 16/31] gnu: Add tree-sitter-javascript.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220218143948.28989-16-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-javascript): New variable.
---
gnu/packages/tree-sitter.scm | 22 ++++++++++++++++++++++
1 file changed, 22 insertions(+)

Toggle diff (30 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index d93f6a584a..976e4cf9ae 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -472,3 +472,25 @@ (define-public tree-sitter-java
(description
"This package provides a Java grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-javascript
+ (package (inherit tree-sitter-c)
+ (name "tree-sitter-javascript")
+ (version "0.20.0")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-javascript")
+ (commit (string-append "rust-" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "175yrk382n2di0c2xn4gpv8y4n83x1lg4hqn04vabf0yqynlkq67"))
+ (modules '((guix build utils)))
+ (snippet tree-sitter-delete-generated-files)))
+ (home-page "https://github.com/tree-sitter/tree-sitter-javascript")
+ (synopsis "Tree-sitter Javascript grammar")
+ (description
+ "This package provides Javascript and JSX grammars for the Tree-sitter
+library.")
+ (license license:expat)))
--
2.34.0
P
P
Pierre Langlois wrote on 18 Feb 2022 15:39
[PATCH v4 18/31] gnu: Add tree-sitter-julia.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220218143948.28989-18-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-julia): New variable.
---
gnu/packages/tree-sitter.scm | 21 +++++++++++++++++++++
1 file changed, 21 insertions(+)

Toggle diff (29 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index ea5a5a5156..081ec8caa0 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -515,3 +515,24 @@ (define-public tree-sitter-json
(description
"This package provides a JSON grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-julia
+ (package (inherit tree-sitter-c)
+ (name "tree-sitter-julia")
+ (version "0.19.0")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-julia")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "1pbnmvhy2gq4vg1b0sjzmjm4s2gsgdjh7h01yj8qrrqbcl29c463"))
+ (modules '((guix build utils)))
+ (snippet tree-sitter-delete-generated-files)))
+ (home-page "https://github.com/tree-sitter/tree-sitter-julia")
+ (synopsis "Tree-sitter Julia grammar")
+ (description
+ "This package provides a Julia grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.34.0
P
P
Pierre Langlois wrote on 18 Feb 2022 15:39
[PATCH v4 17/31] gnu: Add tree-sitter-json.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220218143948.28989-17-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-json): New variable.
---
gnu/packages/tree-sitter.scm | 21 +++++++++++++++++++++
1 file changed, 21 insertions(+)

Toggle diff (29 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 976e4cf9ae..ea5a5a5156 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -494,3 +494,24 @@ (define-public tree-sitter-javascript
"This package provides Javascript and JSX grammars for the Tree-sitter
library.")
(license license:expat)))
+
+(define-public tree-sitter-json
+ (package (inherit tree-sitter-c)
+ (name "tree-sitter-json")
+ (version "0.19.0")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-json")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "06pjh31bv9ja9hlnykk257a6zh8bsxg2fqa54al7qk1r4n9ksnff"))
+ (modules '((guix build utils)))
+ (snippet tree-sitter-delete-generated-files)))
+ (home-page "https://github.com/tree-sitter/tree-sitter-json")
+ (synopsis "Tree-sitter JSON grammar")
+ (description
+ "This package provides a JSON grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.34.0
P
P
Pierre Langlois wrote on 18 Feb 2022 15:39
[PATCH v4 20/31] gnu: Add tree-sitter-php.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220218143948.28989-20-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-php): New variable.
---
gnu/packages/tree-sitter.scm | 38 ++++++++++++++++++++++++++++++++++++
1 file changed, 38 insertions(+)

Toggle diff (46 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 904697154d..e64e248215 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -613,3 +613,41 @@ (define-public tree-sitter-ocaml
(description
"This package provides a OCaml grammar for the Tree-sitter library.")
(license license:expat))))
+
+(define-public tree-sitter-php
+ (let ((commit "435fa00006c0d1515c37fbb4dd6a9de284af75ab")
+ (revision "1"))
+ (package (inherit tree-sitter-c)
+ (name "tree-sitter-php")
+ (version (git-version "0.19.0" revision commit))
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-php")
+ (commit commit)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "05k4h58gi616gv41r0qqdb1x4rs8y94vghn2r10yczisgzq4vbad"))
+ (modules '((guix build utils)))
+ (snippet
+ '(begin
+ (delete-file "src/grammar.json")
+ (delete-file "src/node-types.json")
+ (delete-file "src/parser.c")
+ (delete-file-recursively "src/tree_sitter")
+ #t))))
+ (arguments
+ (substitute-keyword-arguments (package-arguments tree-sitter-c)
+ ((#:phases phases)
+ `(modify-phases ,phases
+ (replace 'delete-dependencies
+ (lambda _
+ (delete-dependencies '("tree-sitter-cli"
+ "nan"
+ "shelljs"))))))))
+ (home-page "https://github.com/tree-sitter/tree-sitter-php")
+ (synopsis "Tree-sitter PHP grammar")
+ (description
+ "This package provides a PHP grammar for the Tree-sitter library.")
+ (license license:expat))))
--
2.34.0
P
P
Pierre Langlois wrote on 18 Feb 2022 15:39
[PATCH v4 21/31] gnu: Add tree-sitter-python.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220218143948.28989-21-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-python): New variable.
---
gnu/packages/tree-sitter.scm | 23 +++++++++++++++++++++++
1 file changed, 23 insertions(+)

Toggle diff (31 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index e64e248215..cecce9af50 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -651,3 +651,26 @@ (define-public tree-sitter-php
(description
"This package provides a PHP grammar for the Tree-sitter library.")
(license license:expat))))
+
+(define-public tree-sitter-python
+ (let ((commit "ed0fe62e55dc617ed9dec8817ebf771aa7cf3c42")
+ (revision "1"))
+ (package (inherit tree-sitter-c)
+ (name "tree-sitter-python")
+ (version (git-version "0.19.1" revision commit))
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-python")
+ (commit commit)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "0wrfpg84mc3pzcrdi6n5fqwijkqr1nj5sqfnayb502krvqpjilal"))
+ (modules '((guix build utils)))
+ (snippet tree-sitter-delete-generated-files)))
+ (home-page "https://github.com/tree-sitter/tree-sitter-python")
+ (synopsis "Tree-sitter Python grammar")
+ (description
+ "This package provides a Python grammar for the Tree-sitter library.")
+ (license license:expat))))
--
2.34.0
P
P
Pierre Langlois wrote on 18 Feb 2022 15:39
[PATCH v4 22/31] gnu: Add tree-sitter-ruby.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220218143948.28989-22-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-ruby): New variable.
---
gnu/packages/tree-sitter.scm | 32 ++++++++++++++++++++++++++++++++
1 file changed, 32 insertions(+)

Toggle diff (40 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index cecce9af50..6fc9a0c91a 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -674,3 +674,35 @@ (define-public tree-sitter-python
(description
"This package provides a Python grammar for the Tree-sitter library.")
(license license:expat))))
+
+(define-public tree-sitter-ruby
+ (package (inherit tree-sitter-c)
+ (name "tree-sitter-ruby")
+ (version "0.19.0")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-ruby")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "0m3h4928rbs300wcb6776h9r88hi32rybbhcaf6rdympl5nzi83v"))
+ (modules '((guix build utils)))
+ (snippet tree-sitter-delete-generated-files)))
+ (arguments
+ (substitute-keyword-arguments (package-arguments tree-sitter-c)
+ ((#:phases phases)
+ `(modify-phases ,phases
+ (replace 'delete-dependencies
+ (lambda _
+ (delete-dependencies '("tree-sitter-cli"
+ "nan"
+ "node-gyp"
+ "prebuild"
+ "prebuild-install"))))))))
+ (home-page "https://github.com/tree-sitter/tree-sitter-ruby")
+ (synopsis "Tree-sitter Ruby grammar")
+ (description
+ "This package provides a Ruby grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.34.0
P
P
Pierre Langlois wrote on 18 Feb 2022 15:39
[PATCH v4 23/31] gnu: Add tree-sitter-rust.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220218143948.28989-23-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-rust): New variable.
---
gnu/packages/tree-sitter.scm | 25 +++++++++++++++++++++++++
1 file changed, 25 insertions(+)

Toggle diff (40 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 6fc9a0c91a..f4df5c5335 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -25,6 +25,7 @@ (define-module (gnu packages tree-sitter)
#:use-module (guix git-download)
#:use-module (guix packages)
#:use-module (guix utils)
+ #:use-module (gnu packages algebra)
#:use-module (gnu packages crates-graphics)
#:use-module (gnu packages crates-io)
#:use-module (gnu packages icu4c))
@@ -706,3 +707,27 @@ (define-public tree-sitter-ruby
(description
"This package provides a Ruby grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-rust
+ (package (inherit tree-sitter-c)
+ (name "tree-sitter-rust")
+ (version "0.20.1")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-rust")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "174j5pxwf80c4xniri39l3a6bb7nq96g2s8hh5sgv4i7xvbpfsmg"))
+ (modules '((guix build utils)))
+ (snippet tree-sitter-delete-generated-files)))
+ (native-inputs
+ (modify-inputs (package-native-inputs tree-sitter-c)
+ (prepend bc)))
+ (home-page "https://github.com/tree-sitter/tree-sitter-rust")
+ (synopsis "Tree-sitter Rust grammar")
+ (description
+ "This package provides a Rust grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.34.0
P
P
Pierre Langlois wrote on 18 Feb 2022 15:39
[PATCH v4 29/31] gnu: Add emacs-tree-sitter-core.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220218143948.28989-29-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-emacs-module): New local variable.
(emacs-tree-sitter-core): New variable.
---
gnu/packages/tree-sitter.scm | 78 ++++++++++++++++++++++++++++++++++++
1 file changed, 78 insertions(+)

Toggle diff (93 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 3c8b4588c4..e2af7cff5a 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -20,6 +20,7 @@ (define-module (gnu packages tree-sitter)
#:use-module (guix gexp)
#:use-module ((guix licenses) #:prefix license:)
#:use-module (guix build-system cargo)
+ #:use-module (guix build-system emacs)
#:use-module (guix build-system gnu)
#:use-module (guix build-system node)
#:use-module (guix download)
@@ -845,3 +846,80 @@ (define-public tree-sitter-typescript
"This package provides Typescript and TSX grammars for the Tree-sitter
library.")
(license license:expat))))
+
+;; Local package definition solely for building the native emacs module
+;; written in Rust.
+(define tree-sitter-emacs-module
+ (package
+ (name "tree-sitter-emacs-module")
+ (version "0.18.0")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/emacs-tree-sitter/elisp-tree-sitter")
+ (commit version)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "1sdvz827v436qijs6xafakkfw2d16bvp8frymd818rppjc7a9dif"))))
+ (build-system cargo-build-system)
+ (inputs
+ (list tree-sitter))
+ (arguments
+ `(#:cargo-inputs
+ (("rust-anyhow" ,rust-anyhow-1)
+ ("rust-emacs" ,rust-emacs-0.18)
+ ("rust-libloading" ,rust-libloading-0.7)
+ ("rust-once-cell" ,rust-once-cell-1)
+ ("rust-tree-sitter" ,rust-tree-sitter-for-emacs))
+ #:phases
+ (modify-phases %standard-phases
+ (add-after 'unpack 'chdir
+ (lambda _ (chdir "core")))
+ (add-after 'chdir 'delete-cargo.lock
+ (lambda _ (delete-file "Cargo.lock")))
+ (add-after 'delete-cargo.lock 'do-not-fetch-from-github
+ (lambda _
+ (substitute* "Cargo.toml"
+ (("\\[patch.*") "")
+ (("git = .*") ""))))
+ (replace 'install
+ (lambda* (#:key outputs #:allow-other-keys)
+ (let ((lib (string-append (assoc-ref outputs "out") "/lib")))
+ (install-file "target/release/libtsc_dyn.so" lib)))))))
+ (home-page #f)
+ (synopsis #f)
+ (description #f)
+ (license license:expat)))
+
+(define-public emacs-tree-sitter-core
+ (package
+ (name "emacs-tree-sitter-core")
+ (version (package-version tree-sitter-emacs-module))
+ (source (package-source tree-sitter-emacs-module))
+ (build-system emacs-build-system)
+ (native-inputs
+ (list tree-sitter-emacs-module))
+ (arguments
+ `(#:phases
+ (modify-phases %standard-phases
+ (add-after 'unpack 'chdir
+ (lambda _ (chdir "core")))
+ (add-after 'install 'install-module
+ (lambda* (#:key inputs outputs #:allow-other-keys)
+ (let ((elpa (elpa-directory (assoc-ref outputs "out")))
+ (module (string-append (assoc-ref inputs "tree-sitter-emacs-module")
+ "/lib/libtsc_dyn.so")))
+ ;; Writing the version number in this file prevents the package
+ ;; from trying to download the module from the internet.
+ (call-with-output-file (string-append elpa "/DYN-VERSION")
+ (lambda (port) (display ,version port)))
+ (substitute* "tsc-dyn-get.el"
+ (("defcustom tsc-dyn-dir tsc--dir")
+ (string-append "defcustom tsc-dyn-dir \"" elpa "\"")))
+ (copy-file module (string-append elpa "/tsc-dyn.so"))))))))
+ (home-page "https://github.com/emacs-tree-sitter/elisp-tree-sitter")
+ (synopsis "Tree-sitter bindings for Emacs Lisp, core library")
+ (description "This package provides core APIs of the Emacs binding for
+Tree-sitter, an incremental parsing system.")
+ (license license:expat)))
--
2.34.0
P
P
Pierre Langlois wrote on 18 Feb 2022 15:39
[PATCH v4 24/31] gnu: Add tree-sitter-typescript.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220218143948.28989-24-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-typescript): New variable.
---
gnu/packages/tree-sitter.scm | 69 ++++++++++++++++++++++++++++++++++++
1 file changed, 69 insertions(+)

Toggle diff (77 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index f4df5c5335..309b38386d 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -731,3 +731,72 @@ (define-public tree-sitter-rust
(description
"This package provides a Rust grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-typescript
+ (let ((commit "111b07762e86efab9a918b7c721f720c37e76b0a")
+ (revision "1"))
+ (package (inherit tree-sitter-c)
+ (name "tree-sitter-typescript")
+ (version (git-version "0.20.0" revision commit))
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-typescript")
+ (commit commit)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32 "1364bm3wbqqzvp03cvpx1w89bbqynb1v62i46gy7f6rhib9bf63a"))
+ (modules '((guix build utils)))
+ (snippet
+ '(begin
+ (delete-file "binding.gyp")
+ (delete-file-recursively "bindings")
+ (for-each
+ (lambda (lang)
+ (with-directory-excursion lang
+ (delete-file "src/grammar.json")
+ (delete-file "src/node-types.json")
+ (delete-file "src/parser.c")
+ (delete-file-recursively "src/tree_sitter")))
+ '("typescript" "tsx"))
+ #t))))
+ (inputs
+ (list `(,tree-sitter-javascript "js")))
+ (arguments
+ (substitute-keyword-arguments (package-arguments tree-sitter-c)
+ ((#:phases phases)
+ #~(modify-phases #$phases
+ (replace 'build
+ (lambda _
+ (for-each (lambda (dir)
+ (with-directory-excursion dir
+ (invoke "tree-sitter" "generate" "--no-bindings")))
+ '("typescript" "tsx"))))
+ (replace 'check
+ (lambda _
+ (for-each (lambda (dir)
+ (with-directory-excursion dir
+ (invoke "tree-sitter" "test")))
+ '("typescript" "tsx"))))
+ (replace 'install
+ (lambda _
+ (let ((lib (string-append #$output "/lib/tree-sitter/")))
+ (mkdir-p lib)
+ (for-each
+ (lambda (lang)
+ (invoke #$(cxx-for-target)
+ "-shared"
+ "-fPIC"
+ "-fno-exceptions"
+ "-O2"
+ "-g"
+ "-o" (string-append lib "/" lang ".so")
+ "-xc" (string-append lang "/src/scanner.c")
+ "-xc" (string-append lang "/src/parser.c")))
+ '("typescript" "tsx")))))))))
+ (home-page "https://github.com/tree-sitter/tree-sitter-typescript")
+ (synopsis "Tree-sitter Typescript grammar")
+ (description
+ "This package provides Typescript and TSX grammars for the Tree-sitter
+library.")
+ (license license:expat))))
--
2.34.0
P
P
Pierre Langlois wrote on 18 Feb 2022 15:39
[PATCH v4 26/31] gnu: rust-emacs-module: Update to 0.18.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220218143948.28989-26-pierre.langlois@gmx.com
* gnu/packages/crates-io.scm (rust-emacs-module-0.18): New variable.
(rust-emacs-module-0.10): Inherit from rust-emacs-module-0.18.
---
gnu/packages/crates-io.scm | 25 +++++++++++++++++++++----
1 file changed, 21 insertions(+), 4 deletions(-)

Toggle diff (55 lines)
diff --git a/gnu/packages/crates-io.scm b/gnu/packages/crates-io.scm
index a4bd79607c..23444f1e7b 100644
--- a/gnu/packages/crates-io.scm
+++ b/gnu/packages/crates-io.scm
@@ -19100,10 +19100,10 @@ (define-public rust-emacs-macros-0.11
(description "This package provides proc macros for Emacs modules.")
(license license:bsd-3)))

-(define-public rust-emacs-module-0.10
+(define-public rust-emacs-module-0.18
(package
(name "rust-emacs-module")
- (version "0.10.0")
+ (version "0.18.0")
(source
(origin
(method url-fetch)
@@ -19112,11 +19112,11 @@ (define-public rust-emacs-module-0.10
(string-append name "-" version ".tar.gz"))
(sha256
(base32
- "1gf9lz735xbkyir53dyv362drfx3nin5an5cx39kd8q8kjjwix5g"))))
+ "1ypjyyv2ca3vza4sia91ckxamgfk63yd8frkvg3d4ph4fk4pn1mk"))))
(build-system cargo-build-system)
(arguments
`(#:cargo-inputs
- (("rust-bindgen" ,rust-bindgen-0.48))))
+ (("rust-bindgen" ,rust-bindgen-0.59))))
(inputs
(list clang))
(home-page "https://github.com/ubolonton/emacs-module-rs")
@@ -19125,6 +19125,23 @@ (define-public rust-emacs-module-0.10
Emacs' support for dynamic modules.")
(license license:bsd-3)))

+(define-public rust-emacs-module-0.10
+ (package (inherit rust-emacs-module-0.18)
+ (name "rust-emacs-module")
+ (version "0.10.0")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (crate-uri "emacs_module" version))
+ (file-name
+ (string-append name "-" version ".tar.gz"))
+ (sha256
+ (base32
+ "1gf9lz735xbkyir53dyv362drfx3nin5an5cx39kd8q8kjjwix5g"))))
+ (arguments
+ `(#:cargo-inputs
+ (("rust-bindgen" ,rust-bindgen-0.48))))))
+
(define-public rust-email-0.0.20
(package
(name "rust-email")
--
2.34.0
P
P
Pierre Langlois wrote on 18 Feb 2022 15:39
[PATCH v4 25/31] gnu: Add rust-tree-sitter-for-emacs.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220218143948.28989-25-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (rust-tree-sitter-for-emacs): New variable.
* gnu/packages/patches/rust-tree-sitter-text-provider-fix.patch: New file.
* gnu/local.mk (dist_patch_DATA): Register it.
---
gnu/local.mk | 1 +
.../rust-tree-sitter-text-provider-fix.patch | 98 +++++++++++++++++++
gnu/packages/tree-sitter.scm | 45 +++++++++
3 files changed, 144 insertions(+)
create mode 100644 gnu/packages/patches/rust-tree-sitter-text-provider-fix.patch

Toggle diff (182 lines)
diff --git a/gnu/local.mk b/gnu/local.mk
index 97ab160c3c..0c113956d3 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -1775,6 +1775,7 @@ dist_patch_DATA = \
%D%/packages/patches/rust-nettle-sys-disable-vendor.patch \
%D%/packages/patches/rust-openssl-sys-no-vendor.patch \
%D%/packages/patches/rust-shell2batch-lint-fix.patch \
+ %D%/packages/patches/rust-tree-sitter-text-provider-fix.patch \
%D%/packages/patches/rust-wl-clipboard-rs-newer-wl.patch \
%D%/packages/patches/sbc-fix-build-non-x86.patch \
%D%/packages/patches/sbcl-aserve-add-HTML-5-elements.patch \
diff --git a/gnu/packages/patches/rust-tree-sitter-text-provider-fix.patch b/gnu/packages/patches/rust-tree-sitter-text-provider-fix.patch
new file mode 100644
index 0000000000..beda2d8391
--- /dev/null
+++ b/gnu/packages/patches/rust-tree-sitter-text-provider-fix.patch
@@ -0,0 +1,98 @@
+From 475b822f47bdc58d832533448b6f6d9818554f37 Mon Sep 17 00:00:00 2001
+From: =?UTF-8?q?Tu=E1=BA=A5n-Anh=20Nguy=E1=BB=85n?= <ubolonton@gmail.com>
+Date: Sun, 25 Jul 2021 13:11:52 +0700
+Subject: [PATCH] Allow TextProvider's iterators to generate owned text
+
+---
+ binding_rust/lib.rs | 33 ++++++++++++++++++---------------
+ 1 file changed, 18 insertions(+), 15 deletions(-)
+
+diff --git a/lib/binding_rust/lib.rs b/lib/binding_rust/lib.rs
+index 068a60285..79c3991ed 100644
+--- a/binding_rust/lib.rs
++++ b/binding_rust/lib.rs
+@@ -8,6 +8,7 @@ pub mod allocations;
+ use std::os::unix::io::AsRawFd;
+
+ use std::{
++ borrow::Cow,
+ char, error,
+ ffi::CStr,
+ fmt, hash, iter,
+@@ -162,7 +163,8 @@ pub struct QueryCaptures<'a, 'tree: 'a, T: TextProvider<'a>> {
+ }
+
+ pub trait TextProvider<'a> {
+- type I: Iterator<Item = &'a [u8]> + 'a;
++ type I: Iterator<Item = Cow<'a, [u8]>>;
++
+ fn text(&mut self, node: Node) -> Self::I;
+ }
+
+@@ -1799,19 +1801,19 @@ impl<'a, 'tree> QueryMatch<'a, 'tree> {
+ buffer2: &mut Vec<u8>,
+ text_provider: &mut impl TextProvider<'a>,
+ ) -> bool {
+- fn get_text<'a, 'b: 'a, I: Iterator<Item = &'b [u8]>>(
++ fn get_text<'a, 'b: 'a, I: Iterator<Item = Cow<'b, [u8]>>>(
+ buffer: &'a mut Vec<u8>,
+ mut chunks: I,
+- ) -> &'a [u8] {
+- let first_chunk = chunks.next().unwrap_or(&[]);
++ ) -> Cow<'a, [u8]> {
++ let first_chunk = chunks.next().unwrap_or(Cow::Owned(vec![0u8; 0]));
+ if let Some(next_chunk) = chunks.next() {
+ buffer.clear();
+- buffer.extend_from_slice(first_chunk);
+- buffer.extend_from_slice(next_chunk);
++ buffer.extend_from_slice(&first_chunk);
++ buffer.extend_from_slice(&next_chunk);
+ for chunk in chunks {
+- buffer.extend_from_slice(chunk);
++ buffer.extend_from_slice(&chunk);
+ }
+- buffer.as_slice()
++ Cow::Borrowed(buffer.as_slice())
+ } else {
+ first_chunk
+ }
+@@ -1835,7 +1837,7 @@ impl<'a, 'tree> QueryMatch<'a, 'tree> {
+ TextPredicate::CaptureMatchString(i, r, is_positive) => {
+ let node = self.nodes_for_capture_index(*i).next().unwrap();
+ let text = get_text(buffer1, text_provider.text(node));
+- r.is_match(text) == *is_positive
++ r.is_match(&text) == *is_positive
+ }
+ })
+ }
+@@ -1946,23 +1948,24 @@ impl<'cursor, 'tree> fmt::Debug for QueryMatch<'cursor, 'tree> {
+ }
+ }
+
+-impl<'a, F, I> TextProvider<'a> for F
++impl<'a, F, I, T> TextProvider<'a> for F
+ where
+ F: FnMut(Node) -> I,
+- I: Iterator<Item = &'a [u8]> + 'a,
++ T: Into<Cow<'a, [u8]>>,
++ I: Iterator<Item = T>,
+ {
+- type I = I;
++ type I = iter::Map<I, fn(T) -> Cow<'a, [u8]>>;
+
+ fn text(&mut self, node: Node) -> Self::I {
+- (self)(node)
++ (self)(node).map(T::into)
+ }
+ }
+
+ impl<'a> TextProvider<'a> for &'a [u8] {
+- type I = iter::Once<&'a [u8]>;
++ type I = iter::Once<Cow<'a, [u8]>>;
+
+ fn text(&mut self, node: Node) -> Self::I {
+- iter::once(&self[node.byte_range()])
++ iter::once(Cow::Borrowed(&self[node.byte_range()]))
+ }
+ }
+
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 309b38386d..3c8b4588c4 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -22,9 +22,11 @@ (define-module (gnu packages tree-sitter)
#:use-module (guix build-system cargo)
#:use-module (guix build-system gnu)
#:use-module (guix build-system node)
+ #:use-module (guix download)
#:use-module (guix git-download)
#:use-module (guix packages)
#:use-module (guix utils)
+ #:use-module (gnu packages)
#:use-module (gnu packages algebra)
#:use-module (gnu packages crates-graphics)
#:use-module (gnu packages crates-io)
@@ -150,6 +152,49 @@ (define-public tree-sitter-cli
This package includes the @command{tree-sitter} command-line tool.")
(license license:expat)))

+;; We need to apply a patch in order to compile the rust bindings against the
+;; emacs tree-sitter module.
+;; See https://github.com/tree-sitter/tree-sitter/pull/1294
+(define-public rust-tree-sitter-for-emacs
+ (package
+ (name "rust-tree-sitter")
+ (version "0.20.0")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (crate-uri "tree-sitter" version))
+ (file-name (string-append name "-" version ".tar.gz"))
+ (sha256
+ (base32
+ "1yg4p54hsfsxxknjq492i8b4rvibzpl2zdvr2bwvwakqgah05v33"))
+ (patches (search-patches "rust-tree-sitter-text-provider-fix.patch"))
+ (modules '((guix build utils)))
+ (snippet
+ '(begin
+ ;; Force
+ (delete-file-recursively "src")
+ (delete-file "binding_rust/build.rs")
+ (with-output-to-file "binding_rust/build.rs"
+ (lambda _
+ (format #t "fn main() {~@
+ println!(\"cargo:rustc-link-lib=tree-sitter\");~@
+ }~%")))
+ #f))))
+ (build-system cargo-build-system)
+ (inputs (list tree-sitter))
+ (arguments
+ `(#:tests? #f ;; Running tests misinterprets comments as doc-tests.
+ #:cargo-inputs
+ (("rust-cc" ,rust-cc-1)
+ ("rust-lazy-static" ,rust-lazy-static-1)
+ ("rust-regex" ,rust-regex-1)
+ ("rust-spin" ,rust-spin-0.7))))
+ (home-page "https://tree-sitter.github.io/tree-sitter/")
+ (synopsis "Rust bindings to the Tree-sitter parsing library")
+ (description "This package provides Rust bindings to the Tree-sitter
+parsing library.")
+ (license license:expat)))
+
(define tree-sitter-delete-generated-files
'(begin
(delete-file "binding.gyp")
--
2.34.0
P
P
Pierre Langlois wrote on 18 Feb 2022 15:39
[PATCH v4 27/31] gnu: rust-emacs-macros: Update to 0.17.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220218143948.28989-27-pierre.langlois@gmx.com
* gnu/packages/crates-io.scm (rust-emacs-macros-0.17): New variable.
(rust-emacs-macros-0.11): Inherit from rust-emacs-macros-0.17.
---
gnu/packages/crates-io.scm | 32 +++++++++++++++++++++++++-------
1 file changed, 25 insertions(+), 7 deletions(-)

Toggle diff (55 lines)
diff --git a/gnu/packages/crates-io.scm b/gnu/packages/crates-io.scm
index 23444f1e7b..154ad963a5 100644
--- a/gnu/packages/crates-io.scm
+++ b/gnu/packages/crates-io.scm
@@ -19076,8 +19076,31 @@ (define-public rust-emacs-0.11
Emacs' support for dynamic modules.")
(license license:bsd-3)))

-(define-public rust-emacs-macros-0.11
+(define-public rust-emacs-macros-0.17
(package
+ (name "rust-emacs-macros")
+ (version "0.17.0")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (crate-uri "emacs-macros" version))
+ (file-name (string-append name "-" version ".tar.gz"))
+ (sha256
+ (base32
+ "0qg1dcn5acbirq617qq2fgg9adswif2dnr292s3qnq62wzgnyrb9"))))
+ (build-system cargo-build-system)
+ (arguments
+ `(#:cargo-inputs
+ (("rust-darling" ,rust-darling-0.10)
+ ("rust-quote" ,rust-quote-1)
+ ("rust-syn" ,rust-syn-1))))
+ (home-page "https://github.com/ubolonton/emacs-module-rs")
+ (synopsis "Proc macros for Emacs modules")
+ (description "This package provides proc macros for Emacs modules.")
+ (license license:bsd-3)))
+
+(define-public rust-emacs-macros-0.11
+ (package (inherit rust-emacs-macros-0.17)
(name "rust-emacs-macros")
(version "0.11.0")
(source
@@ -19089,16 +19112,11 @@ (define-public rust-emacs-macros-0.11
(sha256
(base32
"0390y8vafxdi334hhgrzvcqjq3n5ckcmvilqcfp8vajjq8irrly6"))))
- (build-system cargo-build-system)
(arguments
`(#:cargo-inputs
(("rust-darling" ,rust-darling-0.9)
("rust-quote" ,rust-quote-0.6)
- ("rust-syn" ,rust-syn-0.15))))
- (home-page "https://github.com/ubolonton/emacs-module-rs")
- (synopsis "Proc macros for Emacs modules")
- (description "This package provides proc macros for Emacs modules.")
- (license license:bsd-3)))
+ ("rust-syn" ,rust-syn-0.15))))))

(define-public rust-emacs-module-0.18
(package
--
2.34.0
P
P
Pierre Langlois wrote on 18 Feb 2022 15:39
[PATCH v4 28/31] gnu: rust-emacs: Update to 0.18.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220218143948.28989-28-pierre.langlois@gmx.com
* gnu/packages/crates-io.scm (rust-emacs-0.18): New variable.
(rust-emacs-0.11): Inherit from rust-emacs-0.18.
---
gnu/packages/crates-io.scm | 38 +++++++++++++++++++++++++++++++-------
1 file changed, 31 insertions(+), 7 deletions(-)

Toggle diff (70 lines)
diff --git a/gnu/packages/crates-io.scm b/gnu/packages/crates-io.scm
index 154ad963a5..6c0376c01a 100644
--- a/gnu/packages/crates-io.scm
+++ b/gnu/packages/crates-io.scm
@@ -19045,10 +19045,10 @@ (define-public rust-elf-0.0.10
"This package provides a pure-Rust library for parsing ELF files.")
(license (list license:expat license:asl2.0))))

-(define-public rust-emacs-0.11
+(define-public rust-emacs-0.18
(package
(name "rust-emacs")
- (version "0.11.0")
+ (version "0.18.0")
(source
(origin
(method url-fetch)
@@ -19057,16 +19057,18 @@ (define-public rust-emacs-0.11
(string-append name "-" version ".tar.gz"))
(sha256
(base32
- "1xn6nr8m3p24irlmv3i7c9n95yb10qc6aikwqv99qlmhbky5x0z7"))))
+ "0r860i73b2680i2fhdl2l1wwvvmf2zksncpckgkksdcx310ak5v7"))))
(build-system cargo-build-system)
(arguments
`(#:cargo-inputs
- (("rust-ctor" ,rust-ctor-0.1)
- ("rust-emacs-macros" ,rust-emacs-macros-0.11)
- ("rust-emacs-module" ,rust-emacs-module-0.10)
+ (("rust-anyhow" ,rust-anyhow-1)
+ ("rust-ctor" ,rust-ctor-0.1)
+ ("rust-emacs-macros" ,rust-emacs-macros-0.17)
+ ("rust-emacs-module" ,rust-emacs-module-0.18)
("rust-failure" ,rust-failure-0.1)
("rust-failure-derive" ,rust-failure-derive-0.1)
- ("rust-lazy-static" ,rust-lazy-static-1))))
+ ("rust-once-cell" ,rust-once-cell-1)
+ ("rust-thiserror" ,rust-thiserror-1))))
(inputs
(list clang))
(home-page "https://github.com/ubolonton/emacs-module-rs")
@@ -19076,6 +19078,28 @@ (define-public rust-emacs-0.11
Emacs' support for dynamic modules.")
(license license:bsd-3)))

+(define-public rust-emacs-0.11
+ (package (inherit rust-emacs-0.18)
+ (name "rust-emacs")
+ (version "0.11.0")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (crate-uri "emacs" version))
+ (file-name
+ (string-append name "-" version ".tar.gz"))
+ (sha256
+ (base32
+ "1xn6nr8m3p24irlmv3i7c9n95yb10qc6aikwqv99qlmhbky5x0z7"))))
+ (arguments
+ `(#:cargo-inputs
+ (("rust-ctor" ,rust-ctor-0.1)
+ ("rust-emacs-macros" ,rust-emacs-macros-0.11)
+ ("rust-emacs-module" ,rust-emacs-module-0.10)
+ ("rust-failure" ,rust-failure-0.1)
+ ("rust-failure-derive" ,rust-failure-derive-0.1)
+ ("rust-lazy-static" ,rust-lazy-static-1))))))
+
(define-public rust-emacs-macros-0.17
(package
(name "rust-emacs-macros")
--
2.34.0
P
P
Pierre Langlois wrote on 18 Feb 2022 15:39
[PATCH v4 30/31] gnu: Add emacs-tree-sitter.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220218143948.28989-30-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (emacs-tree-sitter): New variable.
---
gnu/packages/tree-sitter.scm | 27 +++++++++++++++++++++++++++
1 file changed, 27 insertions(+)

Toggle diff (35 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index e2af7cff5a..9a427bed7f 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -923,3 +923,30 @@ (module (string-append (assoc-ref inputs "tree-sitter-emacs-module")
(description "This package provides core APIs of the Emacs binding for
Tree-sitter, an incremental parsing system.")
(license license:expat)))
+
+(define-public emacs-tree-sitter
+ (package
+ (name "emacs-tree-sitter")
+ (version (package-version emacs-tree-sitter-core))
+ (source (package-source emacs-tree-sitter-core))
+ (build-system emacs-build-system)
+ (propagated-inputs
+ (list emacs-tree-sitter-core))
+ (arguments
+ `(#:phases
+ (modify-phases %standard-phases
+ (add-after 'unpack 'chdir
+ (lambda _ (chdir "lisp"))))))
+ (home-page "https://github.com/emacs-tree-sitter/elisp-tree-sitter")
+ (synopsis "Tree-sitter bindings for Emacs Lisp")
+ (description "This package provides Emacs bindings for Tree-sitter, an
+incremental parsing library. It aims to be the foundation for a new breed of
+Emacs packages that understand code structurally. For example:
+
+@enumerate
+@item Faster, fine-grained code highlighting.
+@item More flexible code folding.
+@item Structural editing (like Paredit, or even better) for non-Lisp code.
+@item More informative indexing for imenu.
+@end enumerate")
+ (license license:expat)))
--
2.34.0
P
P
Pierre Langlois wrote on 18 Feb 2022 15:39
[PATCH v4 31/31] gnu: Add emacs-tree-sitter-langs.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220218143948.28989-31-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm
(tree-sitter-langs-grammar-bundle, emacs-tree-sitter-langs): New variables.
---
gnu/packages/tree-sitter.scm | 136 +++++++++++++++++++++++++++++++++++
1 file changed, 136 insertions(+)

Toggle diff (151 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 9a427bed7f..d1341a2ddd 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -23,6 +23,7 @@ (define-module (gnu packages tree-sitter)
#:use-module (guix build-system emacs)
#:use-module (guix build-system gnu)
#:use-module (guix build-system node)
+ #:use-module (guix build-system trivial)
#:use-module (guix download)
#:use-module (guix git-download)
#:use-module (guix packages)
@@ -950,3 +951,138 @@ (define-public emacs-tree-sitter
@item More informative indexing for imenu.
@end enumerate")
(license license:expat)))
+
+(define tree-sitter-langs-grammar-bundle
+ (package
+ (name "tree-sitter-langs-grammar-bundle")
+ (source #f)
+ (version "0.11.3")
+ (build-system trivial-build-system)
+ (inputs
+ ;; FIXME: Support for some languages is still left to package.
+ (list tree-sitter-bash
+ tree-sitter-c
+ tree-sitter-c-sharp
+ tree-sitter-cpp
+ tree-sitter-css
+ tree-sitter-elixir
+ tree-sitter-elm
+ tree-sitter-go
+ tree-sitter-html
+ tree-sitter-java
+ tree-sitter-javascript
+ tree-sitter-json
+ tree-sitter-julia
+ tree-sitter-ocaml
+ tree-sitter-php
+ tree-sitter-python
+ tree-sitter-rust
+ tree-sitter-ruby
+ tree-sitter-typescript))
+ (arguments
+ (list #:builder
+ (with-imported-modules '((guix build union)
+ (guix build utils))
+ #~(begin
+ (use-modules (ice-9 match)
+ (guix build union)
+ (guix build utils))
+ (union-build
+ #$output
+ (filter directory-exists?
+ (map (match-lambda
+ ((name directory)
+ (string-append directory "/lib/tree-sitter")))
+ '#$(package-inputs this-package))))
+ ;; The BUNDLE-VERSION file prevents emacs-tree-sitter-langs
+ ;; from downloading libraries at load time.
+ (call-with-output-file (string-append #$output "/BUNDLE-VERSION")
+ (lambda (port) (display #$version port)))))))
+ (synopsis #f)
+ (description #f)
+ (home-page #f)
+ (license #f)))
+
+(define-public emacs-tree-sitter-langs
+ (package
+ (name "emacs-tree-sitter-langs")
+ (version "0.11.3")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/emacs-tree-sitter/tree-sitter-langs")
+ (commit version)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "1sgpsjd3037pbvwk7lgw70gb4j0fyqikv4lwxwa6l5dwnqvqxgq6"))))
+ (build-system emacs-build-system)
+ (inputs
+ (list tree-sitter-langs-grammar-bundle))
+ (propagated-inputs
+ (list emacs-tree-sitter))
+ (arguments
+ (list
+ #:tests? #t
+ #:test-command ''("script/test")
+ #:phases
+ #~(modify-phases %standard-phases
+ (add-after 'unpack 'remove-cask
+ (lambda _
+ (substitute* "script/test"
+ (("cask") ""))))
+ (add-before 'check 'bundle-for-testing
+ (lambda* (#:key inputs #:allow-other-keys)
+ (delete-file-recursively "bin")
+ (symlink #$tree-sitter-langs-grammar-bundle "bin")))
+ (add-before 'check 'patch-tree-sitter-require-test
+ (lambda _
+ (use-modules (ice-9 regex))
+ ;; This test needs a git repositories with submodules for
+ ;; each languages in order to map all repositories. We patch
+ ;; the mapping function with one that invokes the tests for each
+ ;; packaged language.
+ (let ((supported-languages
+ (map (lambda (lib)
+ (match:substring
+ (string-match "(.*)\\.so$" (basename lib))
+ 1))
+ (find-files "bin" "\\.so$"))))
+ (substitute* "tree-sitter-langs-tests.el"
+ (("tree-sitter-langs--map-repos")
+ (call-with-output-string
+ (lambda (port)
+ (write `(lambda (fn)
+ (dolist (lang ',supported-languages)
+ (funcall fn lang)))
+ port))))))))
+ ;; Tests for queries will fail given those languages are not
+ ;; packages yet.
+ (add-before 'check 'remove-unused-highlight-queries
+ (lambda _
+ (delete-file-recursively "queries/hcl")
+ (delete-file-recursively "queries/pgn")))
+ (add-before 'install 'install-bundle
+ (lambda _
+ (let ((elpa (elpa-directory #$output)))
+ (mkdir-p elpa)
+ (symlink #$tree-sitter-langs-grammar-bundle
+ (string-append elpa "/bin")))))
+ (add-after 'install 'install-queries
+ (lambda* (#:key outputs #:allow-other-keys)
+ (let ((elpa (elpa-directory (assoc-ref outputs "out"))))
+ (copy-recursively "queries" (string-append elpa "/queries"))))))))
+ (home-page "https://ubolonton.github.io/emacs-tree-sitter/languages/")
+ (synopsis "Language support bundle for Tree-sitter")
+ (description "This package is a convenient language bundle for
+Tree-sitter. For each supported language, this package provides:
+
+@enumerate
+@item Pre-compiled grammar binaries.
+@item An optional highlights.scm file that provides highlighting patterns.
+This is mainly intended for major modes that are not aware of tree-sitter.
+@item Optional query patterns for other minor modes that provide high-level
+functionalities on top of tree-sitter, such as code folding, evil text
+objects, ...etc.
+@end enumerate")
+ (license license:expat)))
--
2.34.0
L
L
Luis Henrique Gomes Higino wrote on 7 Mar 2022 19:30
Re: [WIP PATCH v3 00/26] gnu: Add tree-sitter for emacs (and neovim as well?).
(name . Pierre Langlois)(address . pierre.langlois@gmx.com)(address . 49946@debbugs.gnu.org)
87lexl1goc.fsf@gmail.com
Hi Guix, Hi Pierre,

I apologize for my delay, the last few weeks were quite busy :-/.

Pierre Langlois <pierre.langlois@gmx.com> writes:

Toggle quote (11 lines)
> I'm sharing it early as a WIP first, as I just noticed the other
> day
> that Luis also contributed a patches for tree-sitter for
> neovim[0]. The
> series I have here is quite a lot bigger though so I'm happy to
> rebase
> it on top of yours, Luis :-). The package I had for tree-sitter
> looks
> basically the same, I would then move it to a new file with all
> tree-sitter related packages, if you're happy with that.

That's OK by me!

Toggle quote (4 lines)
> So I'm wondering, are these tree-sitter grammar packages also
> useful for
> the neovim package for tree-sitter?

I'm not exactly sure about this. Currently, most Neovim users
install their grammars through the installer included in
the user gets the version that is ABI compatible with
nvim-treesitter. That makes me uncertain if we could use the same
grammar packages for Neovim and Emacs, at least for now.

Toggle quote (8 lines)
> I do wonder if I'm going about it the right way for Guix though,
> for
> instance I'm wondering if I should rewrite the grammar packages
> using a
> new custom build system, do people think it would be worth it?
> That
> could be done as a follow-up of course.

I'm not a very experienced packager myself, but from what I saw in
your patch, it seems rather unnecessary, as the grammar packages
are very simple.

--
Luis H. Higino
P
P
Pierre Langlois wrote on 9 Mar 2022 14:46
(name . Luis Henrique Gomes Higino)(address . luishenriquegh2701@gmail.com)
87sfrrnpme.fsf@gmx.com
Hi Luis,

Luis Henrique Gomes Higino <luishenriquegh2701@gmail.com> writes:

Toggle quote (4 lines)
> Hi Guix, Hi Pierre,
>
> I apologize for my delay, the last few weeks were quite busy :-/.

No worries! :-).

Toggle quote (22 lines)
>
> Pierre Langlois <pierre.langlois@gmx.com> writes:
>
>> I'm sharing it early as a WIP first, as I just noticed the other day
>> that Luis also contributed a patches for tree-sitter for neovim[0]. The
>> series I have here is quite a lot bigger though so I'm happy to rebase
>> it on top of yours, Luis :-). The package I had for tree-sitter looks
>> basically the same, I would then move it to a new file with all
>> tree-sitter related packages, if you're happy with that.
>
> That's OK by me!
>
>> So I'm wondering, are these tree-sitter grammar packages also useful for
>> the neovim package for tree-sitter?
>
> I'm not exactly sure about this. Currently, most Neovim users install their
> grammars through the installer included in
> https://github.com/nvim-treesitter/nvim-treesitter. That ensures
> the user gets the version that is ABI compatible with nvim-treesitter. That
> makes me uncertain if we could use the same grammar packages for Neovim and
> Emacs, at least for now.

I see, I suppose for the moment we're not packaging nvim-treesitter so
we don't have to concern ourselves with how language grammars are
delivered to users.

In the future it might be cool to package it, out of curiosity I took a
look at it and it:

- [0]: Clones the repo with `git'.
- [1]: Run `npm install' and `tree-sitter generate'.
- [2]: Compiles the generated grammar with gcc.


The only concern I can think of is that when it runs `npm install', it's
likely that it downloads a `tree-sitter' binary, rather than use
Guix's tree-sitter-cli package. In my series I had to patch
package.json files for each grammar to remove `tree-sitter-cli' as a
dependency:


Otherwise it would pull in the official tree-sitter-cli node package,
which then downloads a `tree-sitter' binary :-/


Anyways, I don't think that's something we should concern ourselves with
for now. If we ever want to package nvim-treesitter, I think we might
have to patch it so it uses Guix-provided grammars instead of building
them locally. For emacs that wasn't so difficult to do, but the real
issue is making sure the grammar is compabible, and luckily the emacs
package had tests we could run for that. We would also need to run
tests if we're to package nvim-treesitter.

Toggle quote (9 lines)
>
>> I do wonder if I'm going about it the right way for Guix though, for
>> instance I'm wondering if I should rewrite the grammar packages using a
>> new custom build system, do people think it would be worth it? That
>> could be done as a follow-up of course.
>
> I'm not a very experienced packager myself, but from what I saw in your patch,
> it seems rather unnecessary, as the grammar packages are very simple.

That was also my impression :-). Since then I did play with building
one to see what it would look like, also as an escuse to go and learn
how they work. I'm still on the fence about it though, I've attached
what it could look like, if anybody else is also unsure. I think I'd
prefer to work on it as a potential follow-up after an initial set of
patches get merged.

Thanks,
Pierre
-----BEGIN PGP SIGNATURE-----

iQFMBAEBCgA2FiEEctU9gYy29KFyWDdMqPyeRH9PfVQFAmIov4kYHHBpZXJyZS5s
YW5nbG9pc0BnbXguY29tAAoJEKj8nkR/T31USi8H/0ImEqfzU20ae2q2Cb4BgTR3
v16AjRWRFrM9rU6Q2bwoj6tx3Coe/xpMmtVTHOwlwzunwKLttdmyzbFAVGIt7pOb
MLLuxBf8LsF42/BJdnFmwDS9sP+PAq2mVpSt0zG2VBs8QO/NhuuvTZwpsea8pBtL
kOF22UfKfrKwu7f/PyntMS5pubBWMxl4x0qvDhzeiMyVzvlJvoM/4EkG34fgt566
9eLsvjvjPpLVvNlvIy9sRNfEJKSUp8Ql0LADjnYYJa7JchVP74uBWUicIW0jRaWl
cnHnek6zjmtSKoxsP4L3W/+lY4/+UhIUg+fWAa0lps2ligbe20cqWGbq0c+XKvE=
=r8Qv
-----END PGP SIGNATURE-----

M
M
Maxime Devos wrote on 9 Mar 2022 22:04
Re: [bug#49946] [WIP PATCH v3 00/26] gnu: Add tree-sitter for emacs (and neovim as well?).
(address . 49946@debbugs.gnu.org)
24f64e912876f56fe40fbaeaec02ddd5e8ef2c4b.camel@telenet.be
Pierre Langlois schreef op wo 09-03-2022 om 13:46 [+0000]:
Toggle quote (2 lines)
> +           `("g++"

Wouldn't the cross-compiler be necessary here?
-----BEGIN PGP SIGNATURE-----

iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYikWRRccbWF4aW1lZGV2
b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7jLeAQD7x54jXP7DwNAFdiU0P2AfnzRg
1T+6QQiZew7hE/0JdgEAmnmx7kUo6O+Dk/RVqaW/e+JUEkZA2SX5RwMBa7eYAQ0=
=HGH4
-----END PGP SIGNATURE-----


P
P
Pierre Langlois wrote on 10 Mar 2022 12:17
(name . Maxime Devos)(address . maximedevos@telenet.be)
87zglykpmf.fsf@gmx.com
Hi Maxime,

Thanks for taking a look!

Maxime Devos <maximedevos@telenet.be> writes:

Toggle quote (6 lines)
> [[PGP Signed Part:Undecided]]
> Pierre Langlois schreef op wo 09-03-2022 om 13:46 [+0000]:
>> +           `("g++"
>
> Wouldn't the cross-compiler be necessary here?

Yeah it should, I've not yet implemented cross-compiling support in this
build-system yet, but I'm happy to do it!

Before continuing to work on this build-system though, do you have any
opinions about adding it to begin with? As opposed to the current
approach of defining a base grammar package (see tree-sitter-c) and have
the other grammar packages inherit from it. I'm wondering if it's worth
adding the build-system, when I don't think we should expect the number
of grammars we would package to grow too much. But then again, the
package definitions do look quite a bit nicer with the build-system, so
I'm a bit on the fence about this :-).

Thanks,
Pierre
-----BEGIN PGP SIGNATURE-----

iQFMBAEBCgA2FiEEctU9gYy29KFyWDdMqPyeRH9PfVQFAmIp4kgYHHBpZXJyZS5s
YW5nbG9pc0BnbXguY29tAAoJEKj8nkR/T31UHuUH/jZ3h6qi5xZVtDlUXssaPKdD
eU7Z3RR/sjxpmlIX12BR/OuFhzs9A/VNN0Cdstg/QSUi/HfdwAZKe8WziSP51wZF
I0ztihtC12N3HcADWG2PJBz0xpFUkSZqbcTBIlLc0M4hhkEMLqyQ/JrGH0O5En1e
u+RO2+LlaN9RS0g/+hTjPjE1HJpjNpFVX/anPX7Zoyn5kePaL16UJRomLD538X20
BpsLtvA4pu8OxZ/80zLqcGtxS/M4GPUoqTTuIdxPfiVSog6Rav3W8xdJqZ3L1VQX
2Hc6EEFQini2E7w976PD0WVtBBMLoaaCoRIMQC9gA0Jwa7oiVF8qh9BGXsZKAkk=
=BxcU
-----END PGP SIGNATURE-----

M
M
Maxime Devos wrote on 10 Mar 2022 13:36
(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
a1c721ae86cac9bd0e9a9f2ef02e75ac75253d17.camel@telenet.be
Pierre Langlois schreef op do 10-03-2022 om 11:17 [+0000]:
Toggle quote (14 lines)
> Before continuing to work on this build-system though, do you have
> any
> opinions about adding it to begin with?  As opposed to the current
> approach of defining a base grammar package (see tree-sitter-c) and
> have
> the other grammar packages inherit from it.  I'm wondering if it's
> worth
> adding the build-system, when I don't think we should expect the
> number
> of grammars we would package to grow too much.  But then again, the
> package definitions do look quite a bit nicer with the build-system,
> so
> I'm a bit on the fence about this :-).

Build systems only cost a module or two, there are quite a few tree
sitter packages (19 or so?) and the tree sitter packages have a lot
in common (custom test phases, install phases), so I wouldn't
hesitate to define a custom build system for tree-sitter stuff.

Even better would be to unify things a bit more, e.g. it looks like
tree-sitter-ocaml needs a custom 'install', 'build' and 'check' phase,
but they looks almost the same as the other phases for other treesitter
packages, so maybe 'tree-sitter-build-system' can be taught to mostly
automatically determine the exact invocation?

Also, 'use-modules' only really works on the top-level, I recommend
#:modules instead, there are some problems with using it in expression
context (I don't think they are documented anywhere though).

Greetings,
Maxime
-----BEGIN PGP SIGNATURE-----

iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYinw0xccbWF4aW1lZGV2
b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7p3ZAQDHSWusjpMwlDcrWyExpSyvUDj4
0uy5lRCgLpkHcC5KWgEAs/8hWfEvdUyqdi3XJFBEGxp523SHWM7oXAOjXbevNgU=
=hmgu
-----END PGP SIGNATURE-----


P
P
Pierre Langlois wrote on 10 Mar 2022 14:19
(name . Maxime Devos)(address . maximedevos@telenet.be)
87fsnpkjgg.fsf@gmx.com
Maxime Devos <maximedevos@telenet.be> writes:

Toggle quote (21 lines)
> [[PGP Signed Part:Undecided]]
> Pierre Langlois schreef op do 10-03-2022 om 11:17 [+0000]:
>> Before continuing to work on this build-system though, do you have
>> any
>> opinions about adding it to begin with?  As opposed to the current
>> approach of defining a base grammar package (see tree-sitter-c) and
>> have
>> the other grammar packages inherit from it.  I'm wondering if it's
>> worth
>> adding the build-system, when I don't think we should expect the
>> number
>> of grammars we would package to grow too much.  But then again, the
>> package definitions do look quite a bit nicer with the build-system,
>> so
>> I'm a bit on the fence about this :-).
>
> Build systems only cost a module or two, there are quite a few tree
> sitter packages (19 or so?) and the tree sitter packages have a lot
> in common (custom test phases, install phases), so I wouldn't
> hesitate to define a custom build system for tree-sitter stuff.

Sounds good, I'll continue in that direction for v5.

Toggle quote (7 lines)
>
> Even better would be to unify things a bit more, e.g. it looks like
> tree-sitter-ocaml needs a custom 'install', 'build' and 'check' phase,
> but they looks almost the same as the other phases for other treesitter
> packages, so maybe 'tree-sitter-build-system' can be taught to mostly
> automatically determine the exact invocation?

I'll see what can be done, sadly so far I couldn't find a way to do the
right thing automatically for ocaml and typescript. These are packages
that provide two grammars, and I didn't see a way to "discover" that
there are multiple languages supported. However maybe they could be
split into multiple package definitions, if I can find a way to do that,
maybe by passing new #:keyword parameters.

Another idea could be to call back into the build-system phase, for
example something like:

Toggle snippet (8 lines)
(replace 'build
(lambda _
(for-each (lambda (dir)
(with-directory-excursion dir
((assoc-ref tree-sitter:%standard-phases 'build))))
'("ocaml" "interface"))))

I don't know if that works yet, we could do that if other nicer
approaches don't work.

Toggle quote (5 lines)
>
> Also, 'use-modules' only really works on the top-level, I recommend
> #:modules instead, there are some problems with using it in expression
> context (I don't think they are documented anywhere though).

Ah, I see I had (use-modules (guix build json) (ice-9 regex)) in the
install phase for no reason indeed, those modules were already imported,
that's what you meant right?

Thanks for your input! I'll work on a v5.
Pierre
-----BEGIN PGP SIGNATURE-----

iQFMBAEBCgA2FiEEctU9gYy29KFyWDdMqPyeRH9PfVQFAmIqAX8YHHBpZXJyZS5s
YW5nbG9pc0BnbXguY29tAAoJEKj8nkR/T31URewH/iJ1rOj+497/oN1u3QAH0R+r
6scDBDANzi195NONgoOeI7POyRYh22LE1d5mFwgGqIWaNl6NYJhQD1i+aKYSk/8n
VhfvuaSM2KXNf8BX5BS0NXwJSAt/7qg5R9Q9ZqVNYQdx5pjRbwGg8KaQiAd5LYKE
40NdqyszEBIaPMyoaLVRFQfLk+C91b4aTspCWHldawHys6klG+deePUpzBb3fGVX
Auw5wPszK+1E4eXKp0ATA1x6qt2hO/9gAKj2DVWyqfuDfp4BFtcPFvmGZROCvtis
lpEhEmYQQJhVLmyZmzlhIn+3zYzAOuoGHOSwsJSusnVYXTd5AABd125jeG62ED4=
=sie6
-----END PGP SIGNATURE-----

M
M
Maxime Devos wrote on 10 Mar 2022 16:34
(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
4849eb7dd3faa87d81e4bfbdff88ab6496d9be9d.camel@telenet.be
Pierre Langlois schreef op do 10-03-2022 om 13:19 [+0000]:
Toggle quote (17 lines)
> I'll see what can be done, sadly so far I couldn't find a way to do
> the right thing automatically for ocaml and typescript.  These are
> packages that provide two grammars, and I didn't see a way to
> "discover" that there are multiple languages supported.  [...] maybe
> by passing new #:keyword parameters.
>
> Another idea could be to call back into the build-system phase, for
> example something like:
>
> --8<---------------cut here---------------start------------->8---
> (replace 'build
>   (lambda _
>     (for-each (lambda (dir)
>                 (with-directory-excursion dir
>                   ((assoc-ref tree-sitter:%standard-phases 'build))))
>               '("ocaml" "interface"))))

I would go for a keyword argument (#:languages 'auto by default,
or an explicit list like #:languages '("ocaml" "interface") for when
autodetection does not work), since it seems more declarative and
conciser than overriding the build phase.

Greetings,
Maxime.
-----BEGIN PGP SIGNATURE-----

iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYioadhccbWF4aW1lZGV2
b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7mDAAP9MWRpS6Ig0yLS+y0G4NqNnE0CK
s5x8jz0SOK96UVk5DAEAqWt00a3XDP7YFRaXiPeocZNjVsWMeMSqoJbeDMLlJw4=
=ywGA
-----END PGP SIGNATURE-----


P
P
Pierre Langlois wrote on 29 Mar 2022 21:42
[PATCH v5 00/27] gnu: Add tree-sitter for emacs.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220329194318.6106-1-pierre.langlois@gmx.com
Hi Guix!

Here's v5 of my tree-sitter series for emacs, highlights:

* New tree-sitter build-system for building grammars, based on the node
build-system.
* Rebased on top of master, moving the new tree-sitter package in a new
module.
* Dynamically link to libtree-sitter.so from rust packages instead of
bundling it.

I think it's in a pretty good shape now, WDYT?

Thanks,
Pierre

Pierre Langlois (27):
gnu: tree-sitter: Move to its own module.
gnu: Add tree-sitter-cli.
build-system: Add tree-sitter-build-system.
gnu: Add tree-sitter-c.
gnu: Add tree-sitter-cpp.
gnu: Add tree-sitter-bash.
gnu: Add tree-sitter-css.
gnu: Add tree-sitter-c-sharp.
gnu: Add tree-sitter-elixir.
gnu: Add tree-sitter-elm.
gnu: Add tree-sitter-go.
gnu: Add tree-sitter-html.
gnu: Add tree-sitter-java.
gnu: Add tree-sitter-javascript.
gnu: Add tree-sitter-json.
gnu: Add tree-sitter-julia.
gnu: Add tree-sitter-ocaml.
gnu: Add tree-sitter-php.
gnu: Add tree-sitter-python.
gnu: Add tree-sitter-ruby.
gnu: Add tree-sitter-rust.
gnu: Add tree-sitter-typescript.
gnu: Add rust-tree-sitter.
gnu: Add rust-tree-sitter-for-emacs.
gnu: Add emacs-tree-sitter-core.
gnu: Add emacs-tree-sitter.
gnu: Add emacs-tree-sitter-langs.

Makefile.am | 2 +
doc/guix.texi | 21 +-
gnu/local.mk | 2 +
.../rust-tree-sitter-text-provider-fix.patch | 98 ++
gnu/packages/text-editors.scm | 52 -
gnu/packages/tree-sitter.scm | 932 ++++++++++++++++++
gnu/packages/vim.scm | 1 +
guix/build-system/tree-sitter.scm | 190 ++++
guix/build/tree-sitter-build-system.scm | 153 +++
9 files changed, 1398 insertions(+), 53 deletions(-)
create mode 100644 gnu/packages/patches/rust-tree-sitter-text-provider-fix.patch
create mode 100644 gnu/packages/tree-sitter.scm
create mode 100644 guix/build-system/tree-sitter.scm
create mode 100644 guix/build/tree-sitter-build-system.scm

--
2.34.0
P
P
Pierre Langlois wrote on 29 Mar 2022 21:42
[PATCH v5 01/27] gnu: tree-sitter: Move to its own module.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220329194318.6106-2-pierre.langlois@gmx.com
* gnu/packages/text-editors.scm (tree-sitter): Move to ...
* gnu/packages/tree-sitter.scm: ... here, a new module.
* gnu/packages/vim.scm: Use (gnu packages tree-sitter).
* gnu/local.mk (GNU_SYSTEM_MODULES): Register tree-sitter.scm new module.
---
gnu/local.mk | 1 +
gnu/packages/text-editors.scm | 52 -----------------------
gnu/packages/tree-sitter.scm | 78 +++++++++++++++++++++++++++++++++++
gnu/packages/vim.scm | 1 +
4 files changed, 80 insertions(+), 52 deletions(-)
create mode 100644 gnu/packages/tree-sitter.scm

Toggle diff (175 lines)
diff --git a/gnu/local.mk b/gnu/local.mk
index a704161abc..03972441e6 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -587,6 +587,7 @@ GNU_SYSTEM_MODULES = \
%D%/packages/tmux.scm \
%D%/packages/toolkits.scm \
%D%/packages/tor.scm \
+ %D%/packages/tree-sitter.scm \
%D%/packages/tv.scm \
%D%/packages/uglifyjs.scm \
%D%/packages/uml.scm \
diff --git a/gnu/packages/text-editors.scm b/gnu/packages/text-editors.scm
index 7cbad042c9..ddbfef3e77 100644
--- a/gnu/packages/text-editors.scm
+++ b/gnu/packages/text-editors.scm
@@ -63,7 +63,6 @@ (define-module (gnu packages text-editors)
#:use-module (gnu packages gtk)
#:use-module (gnu packages guile)
#:use-module (gnu packages haskell-xyz)
- #:use-module (gnu packages icu4c)
#:use-module (gnu packages image)
#:use-module (gnu packages libbsd)
#:use-module (gnu packages libreoffice)
@@ -1167,54 +1166,3 @@ (define-public virtaal
provided by the Translate Toolkit, including XLIFF and PO.")
(home-page "https://virtaal.translatehouse.org/")
(license license:gpl2+)))
-
-(define-public tree-sitter
- (package
- (name "tree-sitter")
- (version "0.20.6")
- (source (origin
- (method git-fetch)
- (uri (git-reference
- (url "https://github.com/tree-sitter/tree-sitter")
- (commit (string-append "v" version))))
- (file-name (git-file-name name version))
- (sha256
- (base32
- "1z20518snyg0zp75qgs5bxmzjqws4dd19vnp6sya494za3qp5b6d"))
- (modules '((guix build utils)))
- (snippet '(begin
- ;; Remove bundled ICU parts
- (delete-file-recursively "lib/src/unicode")
- #t))))
- (build-system gnu-build-system)
- (inputs (list icu4c))
- (arguments
- (list #:phases
- '(modify-phases %standard-phases
- (delete 'configure))
- #:tests? #f ; there are no tests for the runtime library
- #:make-flags
- #~(list (string-append "PREFIX="
- #$output)
- (string-append "CC="
- #$(cc-for-target)))))
- (home-page "https://tree-sitter.github.io/tree-sitter/")
- (synopsis "Incremental parsing system for programming tools")
- (description
- "Tree-sitter is a parser generator tool and an incremental parsing
-library. It can build a concrete syntax tree for a source file and efficiently
-update the syntax tree as the source file is edited.
-
-Tree-sitter aims to be:
-
-@itemize
-@item General enough to parse any programming language
-@item Fast enough to parse on every keystroke in a text editor
-@item Robust enough to provide useful results even in the presence of syntax errors
-@item Dependency-free so that the runtime library (which is written in pure C)
-can be embedded in any application
-@end itemize
-
-This package includes the @code{libtree-sitter} runtime library.
-")
- (license license:expat)))
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
new file mode 100644
index 0000000000..7116a10ed7
--- /dev/null
+++ b/gnu/packages/tree-sitter.scm
@@ -0,0 +1,78 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2022 Luis Henrique Gomes Higino <luishenriquegh2701@gmail.com>
+;;; Copyright © 2022 Pierre Langlois <pierre.langlois@gmx.com>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu packages tree-sitter)
+ #:use-module (guix gexp)
+ #:use-module ((guix licenses) #:prefix license:)
+ #:use-module (guix build-system gnu)
+ #:use-module (guix git-download)
+ #:use-module (guix packages)
+ #:use-module (guix utils)
+ #:use-module (gnu packages icu4c))
+
+(define-public tree-sitter
+ (package
+ (name "tree-sitter")
+ (version "0.20.6")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "1z20518snyg0zp75qgs5bxmzjqws4dd19vnp6sya494za3qp5b6d"))
+ (modules '((guix build utils)))
+ (snippet '(begin
+ ;; Remove bundled ICU parts
+ (delete-file-recursively "lib/src/unicode")
+ #t))))
+ (build-system gnu-build-system)
+ (inputs (list icu4c))
+ (arguments
+ (list #:phases
+ '(modify-phases %standard-phases
+ (delete 'configure))
+ #:tests? #f ; there are no tests for the runtime library
+ #:make-flags
+ #~(list (string-append "PREFIX="
+ #$output)
+ (string-append "CC="
+ #$(cc-for-target)))))
+ (home-page "https://tree-sitter.github.io/tree-sitter/")
+ (synopsis "Incremental parsing system for programming tools")
+ (description
+ "Tree-sitter is a parser generator tool and an incremental parsing
+library. It can build a concrete syntax tree for a source file and efficiently
+update the syntax tree as the source file is edited.
+
+Tree-sitter aims to be:
+
+@itemize
+@item General enough to parse any programming language
+@item Fast enough to parse on every keystroke in a text editor
+@item Robust enough to provide useful results even in the presence of syntax errors
+@item Dependency-free so that the runtime library (which is written in pure C)
+can be embedded in any application
+@end itemize
+
+This package includes the @code{libtree-sitter} runtime library.
+")
+ (license license:expat)))
diff --git a/gnu/packages/vim.scm b/gnu/packages/vim.scm
index bc5e36c6f2..f2fc39586e 100644
--- a/gnu/packages/vim.scm
+++ b/gnu/packages/vim.scm
@@ -71,6 +71,7 @@ (define-module (gnu packages vim)
#:use-module (gnu packages tcl)
#:use-module (gnu packages text-editors)
#:use-module (gnu packages terminals)
+ #:use-module (gnu packages tree-sitter)
#:use-module (gnu packages xdisorg)
#:use-module (gnu packages xorg))

--
2.34.0
P
P
Pierre Langlois wrote on 29 Mar 2022 21:42
[PATCH v5 02/27] gnu: Add tree-sitter-cli.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220329194318.6106-3-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-cli): New variable.
---
gnu/packages/tree-sitter.scm | 79 ++++++++++++++++++++++++++++++++++++
1 file changed, 79 insertions(+)

Toggle diff (98 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 7116a10ed7..18c9fe0844 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -20,10 +20,13 @@
(define-module (gnu packages tree-sitter)
#:use-module (guix gexp)
#:use-module ((guix licenses) #:prefix license:)
+ #:use-module (guix build-system cargo)
#:use-module (guix build-system gnu)
#:use-module (guix git-download)
#:use-module (guix packages)
#:use-module (guix utils)
+ #:use-module (gnu packages crates-graphics)
+ #:use-module (gnu packages crates-io)
#:use-module (gnu packages icu4c))

(define-public tree-sitter
@@ -76,3 +79,79 @@ (define-public tree-sitter
This package includes the @code{libtree-sitter} runtime library.
")
(license license:expat)))
+
+(define-public tree-sitter-cli
+ (package (inherit tree-sitter)
+ (name "tree-sitter-cli")
+ (source (origin
+ (inherit (package-source tree-sitter))
+ (snippet
+ '(begin
+ ;; Remove the runtime library code and dynamically link to
+ ;; it instead.
+ (delete-file-recursively "lib/src")
+ (delete-file "lib/binding_rust/build.rs")
+ (with-output-to-file "lib/binding_rust/build.rs"
+ (lambda _
+ (format #t "fn main() {~@
+ println!(\"cargo:rustc-link-lib=tree-sitter\");~@
+ }~%")))
+ #t))))
+ (build-system cargo-build-system)
+ (inputs (list tree-sitter))
+ (arguments
+ `(;; Running test requires downloading fixtures, see the
+ ;; script/fetch-fixtures script.
+ #:tests? #f
+ ;; We're only packaging the CLI program so we do not need to install
+ ;; sources.
+ #:install-source? #f
+ #:cargo-inputs
+ (("rust-ansi-term" ,rust-ansi-term-0.12)
+ ("rust-anyhow" ,rust-anyhow-1)
+ ("rust-atty" ,rust-atty-0.2)
+ ("rust-clap" ,rust-clap-2)
+ ("rust-difference" ,rust-difference-2)
+ ("rust-html-escape" ,rust-html-escape-0.2)
+ ("rust-libloading" ,rust-libloading-0.7)
+ ("rust-rand" ,rust-rand-0.8)
+ ("rust-rustc-hash" ,rust-rustc-hash-1)
+ ("rust-semver" ,rust-semver-1)
+ ("rust-smallbitvec" ,rust-smallbitvec-2)
+ ("rust-thiserror" ,rust-thiserror-1)
+ ("rust-tiny-http" ,rust-tiny-http-0.8)
+ ("rust-toml" ,rust-toml-0.5)
+ ("rust-walkdir" ,rust-walkdir-2)
+ ("rust-webbrowser" ,rust-webbrowser-0.5)
+ ("rust-which" ,rust-which-4))
+ #:cargo-development-inputs
+ (("rust-pretty-assertions" ,rust-pretty-assertions-0.7))
+ #:phases
+ (modify-phases %standard-phases
+ (add-after 'unpack 'delete-cargo.lock
+ (lambda _ (delete-file "Cargo.lock")))
+ (replace 'install
+ (lambda* (#:key outputs #:allow-other-keys)
+ (let* ((out (assoc-ref outputs "out"))
+ (bin (string-append out "/bin"))
+ (lib (string-append out "/lib")))
+ (mkdir-p bin)
+ (install-file "target/release/tree-sitter" bin)))))))
+ (synopsis "Incremental parsing system for programming tools")
+ (description "Tree-sitter is a parser generator tool and an incremental
+parsing library. It can build a concrete syntax tree for a source file and
+efficiently update the syntax tree as the source file is edited.
+
+Tree-sitter aims to be:
+
+@enumerate
+@item General enough to parse any programming language.
+@item Fast enough to parse on every keystroke in a text editor.
+@item Robust enough to provide useful results even in the presence of syntax
+errors.
+@item Dependency-free so that the runtime library (which is written in pure C)
+can be embedded in any application.
+@end enumerate
+
+This package includes the @command{tree-sitter} command-line tool.")
+ (license license:expat)))
--
2.34.0
P
P
Pierre Langlois wrote on 29 Mar 2022 21:42
[PATCH v5 04/27] gnu: Add tree-sitter-c.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220329194318.6106-5-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-c): New variable.
---
gnu/packages/tree-sitter.scm | 33 +++++++++++++++++++++++++++++++++
1 file changed, 33 insertions(+)

Toggle diff (48 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 18c9fe0844..3f07b2d91c 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -22,6 +22,7 @@ (define-module (gnu packages tree-sitter)
#:use-module ((guix licenses) #:prefix license:)
#:use-module (guix build-system cargo)
#:use-module (guix build-system gnu)
+ #:use-module (guix build-system tree-sitter)
#:use-module (guix git-download)
#:use-module (guix packages)
#:use-module (guix utils)
@@ -155,3 +156,35 @@ (define-public tree-sitter-cli

This package includes the @command{tree-sitter} command-line tool.")
(license license:expat)))
+
+(define tree-sitter-delete-generated-files
+ '(begin
+ (delete-file "binding.gyp")
+ (delete-file-recursively "bindings")
+ (delete-file "src/grammar.json")
+ (delete-file "src/node-types.json")
+ (delete-file "src/parser.c")
+ (delete-file-recursively "src/tree_sitter")
+ #t))
+
+(define-public tree-sitter-c
+ (package
+ (name "tree-sitter-c")
+ (version "0.20.1")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-c")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "0454jziys33i4kbwnvi9xcck0fzya792ghy32ahgk1hhv96xga9w"))
+ (modules '((guix build utils)))
+ (snippet tree-sitter-delete-generated-files)))
+ (build-system tree-sitter-build-system)
+ (home-page "https://github.com/tree-sitter/tree-sitter-c")
+ (synopsis "Tree-sitter C grammar")
+ (description
+ "This package provides a C grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.34.0
P
P
Pierre Langlois wrote on 29 Mar 2022 21:42
[PATCH v5 05/27] gnu: Add tree-sitter-cpp.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220329194318.6106-6-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-cpp): New variable.
---
gnu/packages/tree-sitter.scm | 23 +++++++++++++++++++++++
1 file changed, 23 insertions(+)

Toggle diff (31 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 3f07b2d91c..5e25d48c05 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -188,3 +188,26 @@ (define-public tree-sitter-c
(description
"This package provides a C grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-cpp
+ (package
+ (name "tree-sitter-cpp")
+ (version "0.20.0")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-cpp")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "0hxcpdvyyig8njga1mxp4qcnbbnr1d0aiy27vahijwbh98b081nr"))
+ (modules '((guix build utils)))
+ (snippet tree-sitter-delete-generated-files)))
+ (build-system tree-sitter-build-system)
+ (inputs (list tree-sitter-c))
+ (home-page "https://github.com/tree-sitter/tree-sitter-cpp")
+ (synopsis "Tree-sitter C++ grammar")
+ (description
+ "This package provides a C++ grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.34.0
P
P
Pierre Langlois wrote on 29 Mar 2022 21:42
[PATCH v5 06/27] gnu: Add tree-sitter-bash.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220329194318.6106-7-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-bash): New variable.
---
gnu/packages/tree-sitter.scm | 22 ++++++++++++++++++++++
1 file changed, 22 insertions(+)

Toggle diff (30 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 5e25d48c05..084071fe3d 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -211,3 +211,25 @@ (define-public tree-sitter-cpp
(description
"This package provides a C++ grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-bash
+ (package
+ (name "tree-sitter-bash")
+ (version "0.19.0")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-bash")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "18c030bb65r50i6z37iy7jb9z9i8i36y7b08dbc9bchdifqsijs5"))
+ (modules '((guix build utils)))
+ (snippet tree-sitter-delete-generated-files)))
+ (build-system tree-sitter-build-system)
+ (home-page "https://github.com/tree-sitter/tree-sitter-bash")
+ (synopsis "Tree-sitter Bash grammar")
+ (description
+ "This package provides a Bash grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.34.0
P
P
Pierre Langlois wrote on 29 Mar 2022 21:42
[PATCH v5 07/27] gnu: Add tree-sitter-css.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220329194318.6106-8-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-css): New variable.
---
gnu/packages/tree-sitter.scm | 22 ++++++++++++++++++++++
1 file changed, 22 insertions(+)

Toggle diff (30 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 084071fe3d..d2f8326be0 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -233,3 +233,25 @@ (define-public tree-sitter-bash
(description
"This package provides a Bash grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-css
+ (package
+ (name "tree-sitter-css")
+ (version "0.19.0")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-css")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "014jrlgi7zfza9g38hsr4vlbi8964i5p7iglaih6qmzaiml7bja2"))
+ (modules '((guix build utils)))
+ (snippet tree-sitter-delete-generated-files)))
+ (build-system tree-sitter-build-system)
+ (home-page "https://github.com/tree-sitter/tree-sitter-css")
+ (synopsis "Tree-sitter CSS grammar")
+ (description
+ "This package provides a CSS grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.34.0
P
P
Pierre Langlois wrote on 29 Mar 2022 21:42
[PATCH v5 08/27] gnu: Add tree-sitter-c-sharp.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220329194318.6106-9-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-c-sharp): New variable.
---
gnu/packages/tree-sitter.scm | 22 ++++++++++++++++++++++
1 file changed, 22 insertions(+)

Toggle diff (33 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index d2f8326be0..de90ac2093 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -212,6 +212,28 @@ (define-public tree-sitter-cpp
"This package provides a C++ grammar for the Tree-sitter library.")
(license license:expat)))

+(define-public tree-sitter-c-sharp
+ (package
+ (name "tree-sitter-c-sharp")
+ (version "0.19.1")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-c-sharp")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "054fmpf47cwh59gbg00sc0nl237ba4rnxi73miz39yqzcs87055r"))
+ (modules '((guix build utils)))
+ (snippet tree-sitter-delete-generated-files)))
+ (build-system tree-sitter-build-system)
+ (home-page "https://github.com/tree-sitter/tree-sitter-c-sharp")
+ (synopsis "Tree-sitter C# grammar")
+ (description
+ "This package provides a C# grammar for the Tree-sitter library.")
+ (license license:expat)))
+
(define-public tree-sitter-bash
(package
(name "tree-sitter-bash")
--
2.34.0
P
P
Pierre Langlois wrote on 29 Mar 2022 21:43
[PATCH v5 10/27] gnu: Add tree-sitter-elm.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220329194318.6106-11-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-elm): New variable.
---
gnu/packages/tree-sitter.scm | 22 ++++++++++++++++++++++
1 file changed, 22 insertions(+)

Toggle diff (30 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 18ea00f7cf..0aafaf7c1b 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -303,3 +303,25 @@ (define-public tree-sitter-elixir
(license (list license:asl2.0
;; Files in tests/corpus are under MIT license.
license:expat)))))
+
+(define-public tree-sitter-elm
+ (package
+ (name "tree-sitter-elm")
+ (version "5.5.1")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/elm-tooling/tree-sitter-elm")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "10hbi4vyj4hjixqswdcbvzl60prldczz29mlp02if61wvwiwvqrw"))
+ (modules '((guix build utils)))
+ (snippet tree-sitter-delete-generated-files)))
+ (build-system tree-sitter-build-system)
+ (home-page "https://elm-tooling.github.io/tree-sitter-elm/")
+ (synopsis "Tree-sitter Elm grammar")
+ (description
+ "This package provides an Elm grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.34.0
P
P
Pierre Langlois wrote on 29 Mar 2022 21:43
[PATCH v5 11/27] gnu: Add tree-sitter-go.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220329194318.6106-12-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-go): New variable.
---
gnu/packages/tree-sitter.scm | 22 ++++++++++++++++++++++
1 file changed, 22 insertions(+)

Toggle diff (30 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 0aafaf7c1b..585b44b9f9 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -325,3 +325,25 @@ (define-public tree-sitter-elm
(description
"This package provides an Elm grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-go
+ (package
+ (name "tree-sitter-go")
+ (version "0.19.1")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-go")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "0nxs47vd2fc2fr0qlxq496y852rwg39flhg334s7dlyq7d3lcx4x"))
+ (modules '((guix build utils)))
+ (snippet tree-sitter-delete-generated-files)))
+ (build-system tree-sitter-build-system)
+ (home-page "https://github.com/tree-sitter/tree-sitter-go")
+ (synopsis "Tree-sitter Go grammar")
+ (description
+ "This package provides a Golang grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.34.0
P
P
Pierre Langlois wrote on 29 Mar 2022 21:43
[PATCH v5 09/27] gnu: Add tree-sitter-elixir.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220329194318.6106-10-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-elixir): New variable.
---
gnu/packages/tree-sitter.scm | 26 ++++++++++++++++++++++++++
1 file changed, 26 insertions(+)

Toggle diff (34 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index de90ac2093..18ea00f7cf 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -277,3 +277,29 @@ (define-public tree-sitter-css
(description
"This package provides a CSS grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-elixir
+ (let ((commit "de20391afe5cb03ef1e8a8e43167e7b58cc52869")
+ (revision "1"))
+ (package
+ (name "tree-sitter-elixir")
+ (version (git-version "0.19.0" revision commit))
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/elixir-lang/tree-sitter-elixir")
+ (commit commit)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "0zrkrwhw3g1vazsxcwrfd1fk4wvs9hdwmwp6073mfh370bz4140h"))
+ (modules '((guix build utils)))
+ (snippet tree-sitter-delete-generated-files)))
+ (build-system tree-sitter-build-system)
+ (home-page "https://elixir-lang.org/tree-sitter-elixir/")
+ (synopsis "Tree-sitter Elixir grammar")
+ (description
+ "This package provides a Elixir grammar for the Tree-sitter library.")
+ (license (list license:asl2.0
+ ;; Files in tests/corpus are under MIT license.
+ license:expat)))))
--
2.34.0
P
P
Pierre Langlois wrote on 29 Mar 2022 21:43
[PATCH v5 12/27] gnu: Add tree-sitter-html.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220329194318.6106-13-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-html): New variable.
---
gnu/packages/tree-sitter.scm | 22 ++++++++++++++++++++++
1 file changed, 22 insertions(+)

Toggle diff (30 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 585b44b9f9..15f832f8ca 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -347,3 +347,25 @@ (define-public tree-sitter-go
(description
"This package provides a Golang grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-html
+ (package
+ (name "tree-sitter-html")
+ (version "0.19.0")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-html")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "1hg7vbcy7bir6b8x11v0a4x0glvqnsqc3i2ixiarbxmycbgl3axy"))
+ (modules '((guix build utils)))
+ (snippet tree-sitter-delete-generated-files)))
+ (build-system tree-sitter-build-system)
+ (home-page "https://github.com/tree-sitter/tree-sitter-html")
+ (synopsis "Tree-sitter HTML grammar")
+ (description
+ "This package provides a HTML grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.34.0
P
P
Pierre Langlois wrote on 29 Mar 2022 21:43
[PATCH v5 13/27] gnu: Add tree-sitter-java.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220329194318.6106-14-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-java): New variable.
---
gnu/packages/tree-sitter.scm | 22 ++++++++++++++++++++++
1 file changed, 22 insertions(+)

Toggle diff (30 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 15f832f8ca..010a7c3751 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -369,3 +369,25 @@ (define-public tree-sitter-html
(description
"This package provides a HTML grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-java
+ (package
+ (name "tree-sitter-java")
+ (version "0.19.1")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-java")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "07zw9ygb45hnvlx9qlz7rlz8hc3byjy03d24v72i5iyhpiiwlhvl"))
+ (modules '((guix build utils)))
+ (snippet tree-sitter-delete-generated-files)))
+ (build-system tree-sitter-build-system)
+ (home-page "https://github.com/tree-sitter/tree-sitter-java")
+ (synopsis "Tree-sitter Java grammar")
+ (description
+ "This package provides a Java grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.34.0
P
P
Pierre Langlois wrote on 29 Mar 2022 21:43
[PATCH v5 14/27] gnu: Add tree-sitter-javascript.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220329194318.6106-15-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-javascript): New variable.
---
gnu/packages/tree-sitter.scm | 23 +++++++++++++++++++++++
1 file changed, 23 insertions(+)

Toggle diff (31 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 010a7c3751..30220a5a8c 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -391,3 +391,26 @@ (define-public tree-sitter-java
(description
"This package provides a Java grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-javascript
+ (package
+ (name "tree-sitter-javascript")
+ (version "0.20.0")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-javascript")
+ (commit (string-append "rust-" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "175yrk382n2di0c2xn4gpv8y4n83x1lg4hqn04vabf0yqynlkq67"))
+ (modules '((guix build utils)))
+ (snippet tree-sitter-delete-generated-files)))
+ (build-system tree-sitter-build-system)
+ (home-page "https://github.com/tree-sitter/tree-sitter-javascript")
+ (synopsis "Tree-sitter Javascript grammar")
+ (description
+ "This package provides Javascript and JSX grammars for the Tree-sitter
+library.")
+ (license license:expat)))
--
2.34.0
P
P
Pierre Langlois wrote on 29 Mar 2022 21:43
[PATCH v5 15/27] gnu: Add tree-sitter-json.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220329194318.6106-16-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-json): New variable.
---
gnu/packages/tree-sitter.scm | 22 ++++++++++++++++++++++
1 file changed, 22 insertions(+)

Toggle diff (30 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 30220a5a8c..d43e16af09 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -414,3 +414,25 @@ (define-public tree-sitter-javascript
"This package provides Javascript and JSX grammars for the Tree-sitter
library.")
(license license:expat)))
+
+(define-public tree-sitter-json
+ (package
+ (name "tree-sitter-json")
+ (version "0.19.0")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-json")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "06pjh31bv9ja9hlnykk257a6zh8bsxg2fqa54al7qk1r4n9ksnff"))
+ (modules '((guix build utils)))
+ (snippet tree-sitter-delete-generated-files)))
+ (build-system tree-sitter-build-system)
+ (home-page "https://github.com/tree-sitter/tree-sitter-json")
+ (synopsis "Tree-sitter JSON grammar")
+ (description
+ "This package provides a JSON grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.34.0
P
P
Pierre Langlois wrote on 29 Mar 2022 21:43
[PATCH v5 16/27] gnu: Add tree-sitter-julia.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220329194318.6106-17-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-julia): New variable.
---
gnu/packages/tree-sitter.scm | 22 ++++++++++++++++++++++
1 file changed, 22 insertions(+)

Toggle diff (30 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index d43e16af09..a535d8bc4a 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -436,3 +436,25 @@ (define-public tree-sitter-json
(description
"This package provides a JSON grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-julia
+ (package
+ (name "tree-sitter-julia")
+ (version "0.19.0")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-julia")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "1pbnmvhy2gq4vg1b0sjzmjm4s2gsgdjh7h01yj8qrrqbcl29c463"))
+ (modules '((guix build utils)))
+ (snippet tree-sitter-delete-generated-files)))
+ (build-system tree-sitter-build-system)
+ (home-page "https://github.com/tree-sitter/tree-sitter-julia")
+ (synopsis "Tree-sitter Julia grammar")
+ (description
+ "This package provides a Julia grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.34.0
P
P
Pierre Langlois wrote on 29 Mar 2022 21:43
[PATCH v5 17/27] gnu: Add tree-sitter-ocaml.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220329194318.6106-18-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-ocaml): New variable.
---
gnu/packages/tree-sitter.scm | 39 ++++++++++++++++++++++++++++++++++++
1 file changed, 39 insertions(+)

Toggle diff (47 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index a535d8bc4a..ff98044cc1 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -458,3 +458,42 @@ (define-public tree-sitter-julia
(description
"This package provides a Julia grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-ocaml
+ (let ((commit "0348562f385bc2bd67ecf181425e1afd6d454192")
+ (revision "1"))
+ (package
+ (name "tree-sitter-ocaml")
+ (version (git-version "0.19.0" revision commit))
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-ocaml")
+ (commit commit)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "0iqmwcz3c2ai4gyx4xli1rhn6hi6a0f60dn20f8jas9ham9dc2df"))
+ (modules '((guix build utils)))
+ (snippet
+ '(begin
+ (delete-file "binding.gyp")
+ (delete-file-recursively "bindings")
+ (for-each
+ (lambda (lang)
+ (with-directory-excursion lang
+ (delete-file "src/grammar.json")
+ (delete-file "src/node-types.json")
+ (delete-file "src/parser.c")
+ (delete-file-recursively "src/tree_sitter")))
+ '("ocaml" "interface"))
+ #t))))
+ (build-system tree-sitter-build-system)
+ (arguments
+ (list
+ #:grammar-directories '("ocaml" "interface")))
+ (home-page "https://github.com/tree-sitter/tree-sitter-ocaml")
+ (synopsis "Tree-sitter OCaml grammar")
+ (description
+ "This package provides a OCaml grammar for the Tree-sitter library.")
+ (license license:expat))))
--
2.34.0
P
P
Pierre Langlois wrote on 29 Mar 2022 21:43
[PATCH v5 18/27] gnu: Add tree-sitter-php.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220329194318.6106-19-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-php): New variable.
---
gnu/packages/tree-sitter.scm | 30 ++++++++++++++++++++++++++++++
1 file changed, 30 insertions(+)

Toggle diff (38 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index ff98044cc1..654445f71d 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -497,3 +497,33 @@ (define-public tree-sitter-ocaml
(description
"This package provides a OCaml grammar for the Tree-sitter library.")
(license license:expat))))
+
+(define-public tree-sitter-php
+ (let ((commit "435fa00006c0d1515c37fbb4dd6a9de284af75ab")
+ (revision "1"))
+ (package
+ (name "tree-sitter-php")
+ (version (git-version "0.19.0" revision commit))
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-php")
+ (commit commit)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "05k4h58gi616gv41r0qqdb1x4rs8y94vghn2r10yczisgzq4vbad"))
+ (modules '((guix build utils)))
+ (snippet
+ '(begin
+ (delete-file "src/grammar.json")
+ (delete-file "src/node-types.json")
+ (delete-file "src/parser.c")
+ (delete-file-recursively "src/tree_sitter")
+ #t))))
+ (build-system tree-sitter-build-system)
+ (home-page "https://github.com/tree-sitter/tree-sitter-php")
+ (synopsis "Tree-sitter PHP grammar")
+ (description
+ "This package provides a PHP grammar for the Tree-sitter library.")
+ (license license:expat))))
--
2.34.0
P
P
Pierre Langlois wrote on 29 Mar 2022 21:43
[PATCH v5 19/27] gnu: Add tree-sitter-python.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220329194318.6106-20-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-python): New variable.
---
gnu/packages/tree-sitter.scm | 24 ++++++++++++++++++++++++
1 file changed, 24 insertions(+)

Toggle diff (32 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 654445f71d..c43cafea2d 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -527,3 +527,27 @@ (define-public tree-sitter-php
(description
"This package provides a PHP grammar for the Tree-sitter library.")
(license license:expat))))
+
+(define-public tree-sitter-python
+ (let ((commit "ed0fe62e55dc617ed9dec8817ebf771aa7cf3c42")
+ (revision "1"))
+ (package
+ (name "tree-sitter-python")
+ (version (git-version "0.19.1" revision commit))
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-python")
+ (commit commit)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "0wrfpg84mc3pzcrdi6n5fqwijkqr1nj5sqfnayb502krvqpjilal"))
+ (modules '((guix build utils)))
+ (snippet tree-sitter-delete-generated-files)))
+ (build-system tree-sitter-build-system)
+ (home-page "https://github.com/tree-sitter/tree-sitter-python")
+ (synopsis "Tree-sitter Python grammar")
+ (description
+ "This package provides a Python grammar for the Tree-sitter library.")
+ (license license:expat))))
--
2.34.0
P
P
Pierre Langlois wrote on 29 Mar 2022 21:43
[PATCH v5 20/27] gnu: Add tree-sitter-ruby.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220329194318.6106-21-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-ruby): New variable.
---
gnu/packages/tree-sitter.scm | 22 ++++++++++++++++++++++
1 file changed, 22 insertions(+)

Toggle diff (30 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index c43cafea2d..c7b1998253 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -551,3 +551,25 @@ (define-public tree-sitter-python
(description
"This package provides a Python grammar for the Tree-sitter library.")
(license license:expat))))
+
+(define-public tree-sitter-ruby
+ (package
+ (name "tree-sitter-ruby")
+ (version "0.19.0")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-ruby")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "0m3h4928rbs300wcb6776h9r88hi32rybbhcaf6rdympl5nzi83v"))
+ (modules '((guix build utils)))
+ (snippet tree-sitter-delete-generated-files)))
+ (build-system tree-sitter-build-system)
+ (home-page "https://github.com/tree-sitter/tree-sitter-ruby")
+ (synopsis "Tree-sitter Ruby grammar")
+ (description
+ "This package provides a Ruby grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.34.0
P
P
Pierre Langlois wrote on 29 Mar 2022 21:43
[PATCH v5 21/27] gnu: Add tree-sitter-rust.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220329194318.6106-22-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-rust): New variable.
---
gnu/packages/tree-sitter.scm | 22 ++++++++++++++++++++++
1 file changed, 22 insertions(+)

Toggle diff (30 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index c7b1998253..c37183e131 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -573,3 +573,25 @@ (define-public tree-sitter-ruby
(description
"This package provides a Ruby grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-rust
+ (package
+ (name "tree-sitter-rust")
+ (version "0.20.1")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-rust")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "174j5pxwf80c4xniri39l3a6bb7nq96g2s8hh5sgv4i7xvbpfsmg"))
+ (modules '((guix build utils)))
+ (snippet tree-sitter-delete-generated-files)))
+ (build-system tree-sitter-build-system)
+ (home-page "https://github.com/tree-sitter/tree-sitter-rust")
+ (synopsis "Tree-sitter Rust grammar")
+ (description
+ "This package provides a Rust grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.34.0
P
P
Pierre Langlois wrote on 29 Mar 2022 21:43
[PATCH v5 23/27] gnu: Add rust-tree-sitter.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220329194318.6106-24-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (rust-tree-sitter): New variable.
---
gnu/packages/tree-sitter.scm | 39 ++++++++++++++++++++++++++++++++++++
1 file changed, 39 insertions(+)

Toggle diff (57 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 756dc0d540..389cb944b9 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -23,6 +23,7 @@ (define-module (gnu packages tree-sitter)
#:use-module (guix build-system cargo)
#:use-module (guix build-system gnu)
#:use-module (guix build-system tree-sitter)
+ #:use-module (guix download)
#:use-module (guix git-download)
#:use-module (guix packages)
#:use-module (guix utils)
@@ -157,6 +158,44 @@ (define-public tree-sitter-cli
This package includes the @command{tree-sitter} command-line tool.")
(license license:expat)))

+(define-public rust-tree-sitter
+ (package
+ (name "rust-tree-sitter")
+ (version "0.20.6")
+ (source (origin
+ (method url-fetch)
+ (uri (crate-uri "tree-sitter" version))
+ (file-name (string-append name "-" version ".tar.gz"))
+ (sha256
+ (base32
+ "0wcvxgnvj7ga1y7xa7wm0pmabkfj8936ifg8jacd4201cj0vgcq9"))
+ (modules '((guix build utils)))
+ (snippet
+ '(begin
+ ;; Remove the runtime library code and dynamically link to
+ ;; it instead.
+ (delete-file-recursively "src")
+ (delete-file "binding_rust/build.rs")
+ (with-output-to-file "binding_rust/build.rs"
+ (lambda _
+ (format #t "fn main() {~@
+ println!(\"cargo:rustc-link-lib=tree-sitter\");~@
+ }~%")))
+ #t))))
+ (build-system cargo-build-system)
+ (inputs (list tree-sitter))
+ (arguments
+ `(#:cargo-inputs
+ (("rust-cc" ,rust-cc-1)
+ ("rust-lazy-static" ,rust-lazy-static-1)
+ ("rust-regex" ,rust-regex-1)
+ ("rust-spin" ,rust-spin-0.7))))
+ (home-page "https://tree-sitter.github.io/tree-sitter/")
+ (synopsis "Rust bindings to the Tree-sitter parsing library")
+ (description "This package provides Rust bindings to the Tree-sitter
+parsing library.")
+ (license license:expat)))
+
(define tree-sitter-delete-generated-files
'(begin
(delete-file "binding.gyp")
--
2.34.0
P
P
Pierre Langlois wrote on 29 Mar 2022 21:43
[PATCH v5 24/27] gnu: Add rust-tree-sitter-for-emacs.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220329194318.6106-25-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (rust-tree-sitter-for-emacs): New variable.
* gnu/packages/patches/rust-tree-sitter-text-provider-fix.patch: New file.
* gnu/local.mk (dist_patch_DATA): Register it.
---
gnu/local.mk | 1 +
.../rust-tree-sitter-text-provider-fix.patch | 98 +++++++++++++++++++
gnu/packages/tree-sitter.scm | 14 +++
3 files changed, 113 insertions(+)
create mode 100644 gnu/packages/patches/rust-tree-sitter-text-provider-fix.patch

Toggle diff (148 lines)
diff --git a/gnu/local.mk b/gnu/local.mk
index 03972441e6..aa353a6f68 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -1780,6 +1780,7 @@ dist_patch_DATA = \
%D%/packages/patches/rust-nettle-sys-disable-vendor.patch \
%D%/packages/patches/rust-openssl-sys-no-vendor.patch \
%D%/packages/patches/rust-shell2batch-lint-fix.patch \
+ %D%/packages/patches/rust-tree-sitter-text-provider-fix.patch \
%D%/packages/patches/rust-wl-clipboard-rs-newer-wl.patch \
%D%/packages/patches/sbc-fix-build-non-x86.patch \
%D%/packages/patches/sbcl-aserve-add-HTML-5-elements.patch \
diff --git a/gnu/packages/patches/rust-tree-sitter-text-provider-fix.patch b/gnu/packages/patches/rust-tree-sitter-text-provider-fix.patch
new file mode 100644
index 0000000000..215e7fc18b
--- /dev/null
+++ b/gnu/packages/patches/rust-tree-sitter-text-provider-fix.patch
@@ -0,0 +1,98 @@
+From e3576b180488b8231e1fc0ca130748577579d129 Mon Sep 17 00:00:00 2001
+From: =?UTF-8?q?Tu=E1=BA=A5n-Anh=20Nguy=E1=BB=85n?= <ubolonton@gmail.com>
+Date: Sun, 25 Jul 2021 13:11:52 +0700
+Subject: [PATCH] Allow TextProvider's iterators to generate owned text
+
+---
+ binding_rust/lib.rs | 33 ++++++++++++++++++---------------
+ 1 file changed, 18 insertions(+), 15 deletions(-)
+
+diff --git a/lib/binding_rust/lib.rs b/lib/binding_rust/lib.rs
+index e88a411c..cf214d92 100644
+--- a/binding_rust/lib.rs
++++ b/binding_rust/lib.rs
+@@ -5,6 +5,7 @@ mod util;
+ use std::os::unix::io::AsRawFd;
+
+ use std::{
++ borrow::Cow,
+ char, error,
+ ffi::CStr,
+ fmt, hash, iter,
+@@ -183,7 +184,8 @@ pub struct QueryCaptures<'a, 'tree: 'a, T: TextProvider<'a>> {
+ }
+
+ pub trait TextProvider<'a> {
+- type I: Iterator<Item = &'a [u8]> + 'a;
++ type I: Iterator<Item = Cow<'a, [u8]>>;
++
+ fn text(&mut self, node: Node) -> Self::I;
+ }
+
+@@ -1840,19 +1842,19 @@ impl<'a, 'tree> QueryMatch<'a, 'tree> {
+ buffer2: &mut Vec<u8>,
+ text_provider: &mut impl TextProvider<'a>,
+ ) -> bool {
+- fn get_text<'a, 'b: 'a, I: Iterator<Item = &'b [u8]>>(
++ fn get_text<'a, 'b: 'a, I: Iterator<Item = Cow<'b, [u8]>>>(
+ buffer: &'a mut Vec<u8>,
+ mut chunks: I,
+- ) -> &'a [u8] {
+- let first_chunk = chunks.next().unwrap_or(&[]);
++ ) -> Cow<'a, [u8]> {
++ let first_chunk = chunks.next().unwrap_or(Cow::Owned(vec![0u8; 0]));
+ if let Some(next_chunk) = chunks.next() {
+ buffer.clear();
+- buffer.extend_from_slice(first_chunk);
+- buffer.extend_from_slice(next_chunk);
++ buffer.extend_from_slice(&first_chunk);
++ buffer.extend_from_slice(&next_chunk);
+ for chunk in chunks {
+- buffer.extend_from_slice(chunk);
++ buffer.extend_from_slice(&chunk);
+ }
+- buffer.as_slice()
++ Cow::Borrowed(buffer.as_slice())
+ } else {
+ first_chunk
+ }
+@@ -1888,7 +1890,7 @@ impl<'a, 'tree> QueryMatch<'a, 'tree> {
+ match node {
+ Some(node) => {
+ let text = get_text(buffer1, text_provider.text(node));
+- r.is_match(text) == *is_positive
++ r.is_match(&text) == *is_positive
+ }
+ None => true,
+ }
+@@ -2002,23 +2004,24 @@ impl<'cursor, 'tree> fmt::Debug for QueryMatch<'cursor, 'tree> {
+ }
+ }
+
+-impl<'a, F, I> TextProvider<'a> for F
++impl<'a, F, I, T> TextProvider<'a> for F
+ where
+ F: FnMut(Node) -> I,
+- I: Iterator<Item = &'a [u8]> + 'a,
++ T: Into<Cow<'a, [u8]>>,
++ I: Iterator<Item = T>,
+ {
+- type I = I;
++ type I = iter::Map<I, fn(T) -> Cow<'a, [u8]>>;
+
+ fn text(&mut self, node: Node) -> Self::I {
+- (self)(node)
++ (self)(node).map(T::into)
+ }
+ }
+
+ impl<'a> TextProvider<'a> for &'a [u8] {
+- type I = iter::Once<&'a [u8]>;
++ type I = iter::Once<Cow<'a, [u8]>>;
+
+ fn text(&mut self, node: Node) -> Self::I {
+- iter::once(&self[node.byte_range()])
++ iter::once(Cow::Borrowed(&self[node.byte_range()]))
+ }
+ }
+
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 389cb944b9..5209876431 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -27,6 +27,7 @@ (define-module (gnu packages tree-sitter)
#:use-module (guix git-download)
#:use-module (guix packages)
#:use-module (guix utils)
+ #:use-module (gnu packages)
#:use-module (gnu packages crates-graphics)
#:use-module (gnu packages crates-io)
#:use-module (gnu packages icu4c))
@@ -196,6 +197,19 @@ (define-public rust-tree-sitter
parsing library.")
(license license:expat)))

+;; We need to apply a patch in order to compile the rust bindings against the
+;; emacs tree-sitter module.
+;; See https://github.com/tree-sitter/tree-sitter/pull/1294
+(define-public rust-tree-sitter-for-emacs
+ (package (inherit rust-tree-sitter)
+ (source (origin
+ (inherit (package-source rust-tree-sitter))
+ (patches (search-patches
+ "rust-tree-sitter-text-provider-fix.patch"))))
+ ;; Do not show this package in the UI as it's only meant to be used for
+ ;; emacs's tree-sitter module.
+ (properties '((hidden? . #t)))))
+
(define tree-sitter-delete-generated-files
'(begin
(delete-file "binding.gyp")
--
2.34.0
P
P
Pierre Langlois wrote on 29 Mar 2022 21:43
[PATCH v5 26/27] gnu: Add emacs-tree-sitter.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220329194318.6106-27-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (emacs-tree-sitter): New variable.
---
gnu/packages/tree-sitter.scm | 27 +++++++++++++++++++++++++++
1 file changed, 27 insertions(+)

Toggle diff (35 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index fb3d13b3c7..97a20494ec 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -766,3 +766,30 @@ (module (string-append (assoc-ref inputs "tree-sitter-emacs-module")
(description "This package provides core APIs of the Emacs binding for
Tree-sitter, an incremental parsing system.")
(license license:expat)))
+
+(define-public emacs-tree-sitter
+ (package
+ (name "emacs-tree-sitter")
+ (version (package-version emacs-tree-sitter-core))
+ (source (package-source emacs-tree-sitter-core))
+ (build-system emacs-build-system)
+ (propagated-inputs
+ (list emacs-tree-sitter-core))
+ (arguments
+ `(#:phases
+ (modify-phases %standard-phases
+ (add-after 'unpack 'chdir
+ (lambda _ (chdir "lisp"))))))
+ (home-page "https://github.com/emacs-tree-sitter/elisp-tree-sitter")
+ (synopsis "Tree-sitter bindings for Emacs Lisp")
+ (description "This package provides Emacs bindings for Tree-sitter, an
+incremental parsing library. It aims to be the foundation for a new breed of
+Emacs packages that understand code structurally. For example:
+
+@enumerate
+@item Faster, fine-grained code highlighting.
+@item More flexible code folding.
+@item Structural editing (like Paredit, or even better) for non-Lisp code.
+@item More informative indexing for imenu.
+@end enumerate")
+ (license license:expat)))
--
2.34.0
P
P
Pierre Langlois wrote on 29 Mar 2022 21:43
[PATCH v5 22/27] gnu: Add tree-sitter-typescript.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220329194318.6106-23-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-typescript): New variable.
---
gnu/packages/tree-sitter.scm | 40 ++++++++++++++++++++++++++++++++++++
1 file changed, 40 insertions(+)

Toggle diff (48 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index c37183e131..756dc0d540 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -595,3 +595,43 @@ (define-public tree-sitter-rust
(description
"This package provides a Rust grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-typescript
+ (let ((commit "111b07762e86efab9a918b7c721f720c37e76b0a")
+ (revision "1"))
+ (package
+ (name "tree-sitter-typescript")
+ (version (git-version "0.20.0" revision commit))
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-typescript")
+ (commit commit)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32 "1364bm3wbqqzvp03cvpx1w89bbqynb1v62i46gy7f6rhib9bf63a"))
+ (modules '((guix build utils)))
+ (snippet
+ '(begin
+ (delete-file "binding.gyp")
+ (delete-file-recursively "bindings")
+ (for-each
+ (lambda (lang)
+ (with-directory-excursion lang
+ (delete-file "src/grammar.json")
+ (delete-file "src/node-types.json")
+ (delete-file "src/parser.c")
+ (delete-file-recursively "src/tree_sitter")))
+ '("typescript" "tsx"))
+ #t))))
+ (build-system tree-sitter-build-system)
+ (inputs (list tree-sitter-javascript))
+ (arguments
+ (list
+ #:grammar-directories '("typescript" "tsx")))
+ (home-page "https://github.com/tree-sitter/tree-sitter-typescript")
+ (synopsis "Tree-sitter Typescript grammar")
+ (description
+ "This package provides Typescript and TSX grammars for the Tree-sitter
+library.")
+ (license license:expat))))
--
2.34.0
P
P
Pierre Langlois wrote on 29 Mar 2022 21:43
[PATCH v5 25/27] gnu: Add emacs-tree-sitter-core.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220329194318.6106-26-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-emacs-module): New local variable.
(emacs-tree-sitter-core): New variable.
---
gnu/packages/tree-sitter.scm | 78 ++++++++++++++++++++++++++++++++++++
1 file changed, 78 insertions(+)

Toggle diff (93 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 5209876431..fb3d13b3c7 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -21,6 +21,7 @@ (define-module (gnu packages tree-sitter)
#:use-module (guix gexp)
#:use-module ((guix licenses) #:prefix license:)
#:use-module (guix build-system cargo)
+ #:use-module (guix build-system emacs)
#:use-module (guix build-system gnu)
#:use-module (guix build-system tree-sitter)
#:use-module (guix download)
@@ -688,3 +689,80 @@ (define-public tree-sitter-typescript
"This package provides Typescript and TSX grammars for the Tree-sitter
library.")
(license license:expat))))
+
+;; Local package definition solely for building the native emacs module
+;; written in Rust.
+(define tree-sitter-emacs-module
+ (package
+ (name "tree-sitter-emacs-module")
+ (version "0.18.0")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/emacs-tree-sitter/elisp-tree-sitter")
+ (commit version)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "1sdvz827v436qijs6xafakkfw2d16bvp8frymd818rppjc7a9dif"))))
+ (build-system cargo-build-system)
+ (inputs
+ (list tree-sitter))
+ (arguments
+ `(#:cargo-inputs
+ (("rust-anyhow" ,rust-anyhow-1)
+ ("rust-emacs" ,rust-emacs-0.18)
+ ("rust-libloading" ,rust-libloading-0.7)
+ ("rust-once-cell" ,rust-once-cell-1)
+ ("rust-tree-sitter" ,rust-tree-sitter-for-emacs))
+ #:phases
+ (modify-phases %standard-phases
+ (add-after 'unpack 'chdir
+ (lambda _ (chdir "core")))
+ (add-after 'chdir 'delete-cargo.lock
+ (lambda _ (delete-file "Cargo.lock")))
+ (add-after 'delete-cargo.lock 'do-not-fetch-from-github
+ (lambda _
+ (substitute* "Cargo.toml"
+ (("\\[patch.*") "")
+ (("git = .*") ""))))
+ (replace 'install
+ (lambda* (#:key outputs #:allow-other-keys)
+ (let ((lib (string-append (assoc-ref outputs "out") "/lib")))
+ (install-file "target/release/libtsc_dyn.so" lib)))))))
+ (home-page #f)
+ (synopsis #f)
+ (description #f)
+ (license license:expat)))
+
+(define-public emacs-tree-sitter-core
+ (package
+ (name "emacs-tree-sitter-core")
+ (version (package-version tree-sitter-emacs-module))
+ (source (package-source tree-sitter-emacs-module))
+ (build-system emacs-build-system)
+ (native-inputs
+ (list tree-sitter-emacs-module))
+ (arguments
+ `(#:phases
+ (modify-phases %standard-phases
+ (add-after 'unpack 'chdir
+ (lambda _ (chdir "core")))
+ (add-after 'install 'install-module
+ (lambda* (#:key inputs outputs #:allow-other-keys)
+ (let ((elpa (elpa-directory (assoc-ref outputs "out")))
+ (module (string-append (assoc-ref inputs "tree-sitter-emacs-module")
+ "/lib/libtsc_dyn.so")))
+ ;; Writing the version number in this file prevents the package
+ ;; from trying to download the module from the internet.
+ (call-with-output-file (string-append elpa "/DYN-VERSION")
+ (lambda (port) (display ,version port)))
+ (substitute* "tsc-dyn-get.el"
+ (("defcustom tsc-dyn-dir tsc--dir")
+ (string-append "defcustom tsc-dyn-dir \"" elpa "\"")))
+ (copy-file module (string-append elpa "/tsc-dyn.so"))))))))
+ (home-page "https://github.com/emacs-tree-sitter/elisp-tree-sitter")
+ (synopsis "Tree-sitter bindings for Emacs Lisp, core library")
+ (description "This package provides core APIs of the Emacs binding for
+Tree-sitter, an incremental parsing system.")
+ (license license:expat)))
--
2.34.0
P
P
Pierre Langlois wrote on 29 Mar 2022 21:43
[PATCH v5 27/27] gnu: Add emacs-tree-sitter-langs.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220329194318.6106-28-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (make-emacs-tree-sitter-langs-grammar-bundle):
New procedure.
(emacs-tree-sitter-langs): New variable.
---
gnu/packages/tree-sitter.scm | 137 +++++++++++++++++++++++++++++++++++
1 file changed, 137 insertions(+)

Toggle diff (152 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 97a20494ec..ae0478c3e4 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -24,6 +24,7 @@ (define-module (gnu packages tree-sitter)
#:use-module (guix build-system emacs)
#:use-module (guix build-system gnu)
#:use-module (guix build-system tree-sitter)
+ #:use-module (guix build-system trivial)
#:use-module (guix download)
#:use-module (guix git-download)
#:use-module (guix packages)
@@ -793,3 +794,139 @@ (define-public emacs-tree-sitter
@item More informative indexing for imenu.
@end enumerate")
(license license:expat)))
+
+(define (make-emacs-tree-sitter-langs-grammar-bundle version)
+ (package
+ (name "emacs-tree-sitter-langs-grammar-bundle")
+ (source #f)
+ (version version)
+ (build-system trivial-build-system)
+ (inputs
+ ;; FIXME: Support for some languages is still left to package.
+ (list tree-sitter-bash
+ tree-sitter-c
+ tree-sitter-c-sharp
+ tree-sitter-cpp
+ tree-sitter-css
+ tree-sitter-elixir
+ tree-sitter-elm
+ tree-sitter-go
+ tree-sitter-html
+ tree-sitter-java
+ tree-sitter-javascript
+ tree-sitter-json
+ tree-sitter-julia
+ tree-sitter-ocaml
+ tree-sitter-php
+ tree-sitter-python
+ tree-sitter-rust
+ tree-sitter-ruby
+ tree-sitter-typescript))
+ (arguments
+ (list #:builder
+ (with-imported-modules '((guix build union)
+ (guix build utils))
+ #~(begin
+ (use-modules (ice-9 match)
+ (guix build union)
+ (guix build utils))
+ (union-build
+ #$output
+ (filter directory-exists?
+ (map (match-lambda
+ ((name directory)
+ (string-append directory "/lib/tree-sitter")))
+ '#$(package-inputs this-package))))
+ ;; The BUNDLE-VERSION file prevents emacs-tree-sitter-langs
+ ;; from downloading libraries at load time.
+ (call-with-output-file (string-append #$output "/BUNDLE-VERSION")
+ (lambda (port) (display #$version port)))))))
+ (synopsis #f)
+ (description #f)
+ (home-page #f)
+ (license #f)))
+
+(define-public emacs-tree-sitter-langs
+ (package
+ (name "emacs-tree-sitter-langs")
+ (version "0.11.6")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/emacs-tree-sitter/tree-sitter-langs")
+ (commit version)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "0jyzpiyl3b88vwfzkda7ak77kl0b2nybvsz0j9cndl3pf95d62g0"))))
+ (build-system emacs-build-system)
+ (inputs
+ (list (make-emacs-tree-sitter-langs-grammar-bundle version)))
+ (propagated-inputs
+ (list emacs-tree-sitter))
+ (arguments
+ (list
+ #:tests? #t
+ #:test-command ''("script/test")
+ #:phases
+ #~(modify-phases %standard-phases
+ (add-after 'unpack 'remove-cask
+ (lambda _
+ (substitute* "script/test"
+ (("cask") ""))))
+ (add-before 'check 'bundle-for-testing
+ (lambda* (#:key inputs #:allow-other-keys)
+ (delete-file-recursively "bin")
+ (symlink #$(make-emacs-tree-sitter-langs-grammar-bundle version)
+ "bin")))
+ (add-before 'check 'patch-tree-sitter-require-test
+ (lambda _
+ (use-modules (ice-9 regex))
+ ;; This test needs a git repositories with submodules for
+ ;; each languages in order to map all repositories. We patch
+ ;; the mapping function with one that invokes the tests for each
+ ;; packaged language.
+ (let ((supported-languages
+ (map (lambda (lib)
+ (match:substring
+ (string-match "(.*)\\.so$" (basename lib))
+ 1))
+ (find-files "bin" "\\.so$"))))
+ (substitute* "tree-sitter-langs-tests.el"
+ (("tree-sitter-langs--map-repos")
+ (call-with-output-string
+ (lambda (port)
+ (write `(lambda (fn)
+ (dolist (lang ',supported-languages)
+ (funcall fn lang)))
+ port))))))))
+ ;; Tests for queries will fail given those languages are not
+ ;; packages yet.
+ (add-before 'check 'remove-unused-highlight-queries
+ (lambda _
+ (delete-file-recursively "queries/hcl")
+ (delete-file-recursively "queries/pgn")))
+ (add-before 'install 'install-bundle
+ (lambda _
+ (let ((elpa (elpa-directory #$output)))
+ (mkdir-p elpa)
+ (symlink #$(make-emacs-tree-sitter-langs-grammar-bundle version)
+ (string-append elpa "/bin")))))
+ (add-after 'install 'install-queries
+ (lambda* (#:key outputs #:allow-other-keys)
+ (let ((elpa (elpa-directory (assoc-ref outputs "out"))))
+ (copy-recursively "queries" (string-append elpa "/queries"))))))))
+ (home-page "https://ubolonton.github.io/emacs-tree-sitter/languages/")
+ (synopsis "Language support bundle for Tree-sitter")
+ (description "This package is a convenient language bundle for
+Tree-sitter. For each supported language, this package provides:
+
+@enumerate
+@item Pre-compiled grammar binaries.
+@item An optional highlights.scm file that provides highlighting patterns.
+This is mainly intended for major modes that are not aware of tree-sitter.
+@item Optional query patterns for other minor modes that provide high-level
+functionalities on top of tree-sitter, such as code folding, evil text
+objects, ...etc.
+@end enumerate")
+ (license license:expat)))
--
2.34.0
M
M
Maxime Devos wrote on 29 Mar 2022 21:55
0fa9d98cb9038a24e8596858a8d37d63071937a2.camel@telenet.be
Pierre Langlois schreef op di 29-03-2022 om 20:43 [+0100]:
Toggle quote (3 lines)
> +                  ;; The BUNDLE-VERSION file prevents emacs-tree-sitter-langs
> +                  ;; from downloading libraries at load time.

WDYT of patching emacs-tree-sitter-langs to not download, such that it
doesn't download and run non-Guix libraries behind the user's back?

Also, why do we need a bundle at all, would simply installing emacs-
tree-sitter, and, e.g., tree-sitter-java, just work?

Greetings,
Maxime.
-----BEGIN PGP SIGNATURE-----

iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYkNkLRccbWF4aW1lZGV2
b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7nVRAPwIdaQsOpgFBOK/ehutvI/QXruC
Hi1W1NqSM9IooZ3ofwEA4ZTo00vgUmC/5OF3ayew4/qEsg0xZ7mGzXItctyiXgE=
=3CK2
-----END PGP SIGNATURE-----


M
M
Maxime Devos wrote on 29 Mar 2022 21:57
Re: [bug#49946] [PATCH v5 01/27] gnu: tree-sitter: Move to its own module.
2c435f65e2b30348442ef5ebc043d8bcf4146ff2.camel@telenet.be
Pierre Langlois schreef op di 29-03-2022 om 20:42 [+0100]:
Toggle quote (3 lines)
> -(define-public tree-sitter
> -  (package

Is my local guix too old, or is this package not yet in guix?

Greetings,
Maxime.
-----BEGIN PGP SIGNATURE-----

iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYkNkuhccbWF4aW1lZGV2
b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7sGQAQCh04rh5ofRLSyiwBgbHXqyewab
KAtLDBqeT2YIykouAwEA09oeRT25sXP6GfB4dLTMQhIOnsC5J7CJISsivQNszws=
=2fNG
-----END PGP SIGNATURE-----


M
M
Maxime Devos wrote on 29 Mar 2022 21:58
7d5f974bd32fc3169cadf2ae0cd6194ddb9a080f.camel@telenet.be
Pierre Langlois schreef op di 29-03-2022 om 20:42 [+0100]:
Toggle quote (2 lines)
> -@item Dependency-free so that the runtime library (which is written in pure C)

It probably depends on glibc library, so strictly speaking this is
probably not technically correct ... but maybe good enough?

Greetings,
Maxime.
-----BEGIN PGP SIGNATURE-----

iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYkNk/RccbWF4aW1lZGV2
b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7lUTAQC6YU0eu0XBrYIH8elFv9cRUcFm
0A5dwyDbRp3wtcvRJwEA+wYzvqDZrmxreLxFjoGM0gkpxRWVcGHDSMx8NCeb6Qk=
=8t9g
-----END PGP SIGNATURE-----


M
M
Maxime Devos wrote on 29 Mar 2022 22:23
19886a6f03b8e8f16064c795a3c111abc7a4e5f6.camel@telenet.be
Maxime Devos schreef op di 29-03-2022 om 21:57 [+0200]:
Toggle quote (6 lines)
> Pierre Langlois schreef op di 29-03-2022 om 20:42 [+0100]:
> > -(define-public tree-sitter
> > -  (package
>
> Is my local guix too old, or is this package not yet in guix?

Nevermind, looks like my Guix was too old.
-----BEGIN PGP SIGNATURE-----

iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYkNq1RccbWF4aW1lZGV2
b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7vC4AP9dDcQZz7L2Tli/U75kSDdH6SNL
6epfBwwt1ypjUOj7ogEAvkf4jvnw6SVDbolaihkfDcEMrS5CiR0MKpB901Z+xws=
=F0JP
-----END PGP SIGNATURE-----


M
M
Mekeor Melire wrote on 15 May 2022 00:36
State of emacs-tree-sitter
(address . 49946@debbugs.gnu.org)(address . pierre.langlois@gmx.com)
87ee0vlofx.fsf@posteo.de
Hello,

what's the current state of this set of patches?

Pierre, do you happen to have a channel or so where I could try out the
patch-series?
P
P
Pierre Langlois wrote on 15 May 2022 14:14
(name . Mekeor Melire)(address . mekeor@posteo.de)
87tu9rasz6.fsf@gmx.com
Hi,

Mekeor Melire <mekeor@posteo.de> writes:

Toggle quote (4 lines)
> Hello,
>
> what's the current state of this set of patches?

I'm still working on it, sorry I've not had too much time for it these
days.

Toggle quote (4 lines)
>
> Pierre, do you happen to have a channel or so where I could try out the
> patch-series?

I don't have a channel, however I just pushed the series rebased on top

So if you'd like to test, you could use this instead of applying
patches:

guix pull --url=https://gitlab.com/planglois/guix--branch=emacs-tree-sitter

I suppose I could move to a channel, but that's a little bit more work,
and I'd delete it immediately after we have something upstream.

Let me know how testing goes!

Pierre
-----BEGIN PGP SIGNATURE-----

iQFMBAEBCgA2FiEEctU9gYy29KFyWDdMqPyeRH9PfVQFAmKA780YHHBpZXJyZS5s
YW5nbG9pc0BnbXguY29tAAoJEKj8nkR/T31UR0QH/3QLSqFH1KIz8i0XIALghG7T
jbOXjtvGgHNPaxFfddbkWmFtbQnkmiIFMD/jn3ISiG693XePXcIoGk7+EukoT4Ui
bro3gqqc9xaW9KHdVIap3sau3JP6tNm64194R4A+5J6SqIiewERxwslw5Qt4t2sd
pt065InGY4cLcoic9AxmYvYdncLTnWMkL2FGmTgRADWrcyT6vLIxIUzE+/ZuBnxa
2bAh701Kj2+VplpFN571v6pxR/2y2i2BBo+POARH2YXVukShmpO9UeR6/iHKcqhd
kQALwhCgXlScjyXOxABpnrffpGJNjSNPkkp0gGJ640YD7rsusRedBIiaJFOTpHQ=
=pDAj
-----END PGP SIGNATURE-----

P
P
Pierre Langlois wrote on 15 May 2022 14:20
Re: [bug#49946] [PATCH v5 27/27] gnu: Add emacs-tree-sitter-langs.
(name . Maxime Devos)(address . maximedevos@telenet.be)
87bkvz54cg.fsf@gmx.com
Hi Maxime, sorry for the late reply!

Maxime Devos <maximedevos@telenet.be> writes:

Toggle quote (8 lines)
> [[PGP Signed Part:Undecided]]
> Pierre Langlois schreef op di 29-03-2022 om 20:43 [+0100]:
>> +                  ;; The BUNDLE-VERSION file prevents emacs-tree-sitter-langs
>> +                  ;; from downloading libraries at load time.
>
> WDYT of patching emacs-tree-sitter-langs to not download, such that it
> doesn't download and run non-Guix libraries behind the user's back?

The way the current patchset works, by providing a compatible bundle, we
already prevent emacs-tree-sitter-langs from downloading binaries by
default. I agree we could go further though, and entirely remove the
code that downloads binaries.

However I'm not sure about it. Essentially, there is a
tree-sitter-langs-build.el file [0] that can either download binaries or
fetch sources and compile locally. So a user could decide to opt-out of
using Guix binaries and instead use an alternative method.

I 100% agree that by default we should make the package use Guix
libraries, this way it also /just works/. However, if somebody wants to
do things differently, I'm not sure we want to get in the way. In the
end, this isn't so different from letting people use an alternative
package manager if they like. We'd rather people used Guix of course :-).

I don't have a really strong opinion about this though, so if you still
prefer to delete the downloading code, I've attached an example patch
that entirely replace it with a bare-bones implementation as an example.

Let me know what you think!


Toggle quote (3 lines)
> Also, why do we need a bundle at all, would simply installing emacs-
> tree-sitter, and, e.g., tree-sitter-java, just work?

Yeah, having a bundle is fundamentally how this package works AFAICT.

I think the main reason is that in order to do highlighting effectively
in emacs, it's not enough to install the tree-sitter runtime and a
tree-sitter-<lang> grammar. You also want to "configure" how the
highlighting is done by providing custom "queries" expressions.

And this package provides queries for every language that it supports:

All that being said, I believe that long-term the idea is that upstream
language-specific packages would eventually gain support for tree-sitter
and then this bundle "glue" package will no longer be necessary.
Especially if one day emacs proper gains native support for tree-sitter
(I think I saw some discussions about that on emacs-devel last year).

But given this package is quite useful though, I'd be surprised if it
goes away soon.

Hope this makes sense!

Thanks,
Pierre

Toggle quote (5 lines)
>
> Greetings,
> Maxime.
>
> [[End of PGP Signed Part]]
-----BEGIN PGP SIGNATURE-----

iQFMBAEBCgA2FiEEctU9gYy29KFyWDdMqPyeRH9PfVQFAmKA+78YHHBpZXJyZS5s
YW5nbG9pc0BnbXguY29tAAoJEKj8nkR/T31UEhoH/RFOtPgATPZpLBDnq1Bh4Rft
WbZ+x+QktpWy/VZoAe5QFKIStEPQoQLQJK7NOhsHN844w/lSkVLf5DN7MSUQl6BK
6uEAKDp6gUgM8AvvMTd283ZJJwzHL1GE9y8/yX6gXuKxDHko041XhwJTT/xcDCry
X3xXTDft39FaC10KNUj+6UJRg1I+q0LF83Sk811L7IR4wVO6TeA/P5is7W51EUD+
M0T8F3VrMxgmnA4/KHBkvs4r/ZAnAPziG5mN69OsdZcWYW3+YhtAJsBaE89Mm8df
IygS4yupJsG/4PfurZhm0YXrpLi6PpURUAecmW6sr9O8c2t3mfIlAl8K+x83L9M=
=FtGC
-----END PGP SIGNATURE-----

From 7ad62ccef2446011dfbdfb2dbe8cc58f46fb05d8 Mon Sep 17 00:00:00 2001
From: Pierre Langlois <pierre.langlois@gmx.com>
Date: Sat, 2 Apr 2022 19:22:52 +0100
Subject: [PATCH] wip

---
gnu/packages/tree-sitter.scm | 49 ++++++++++++++++++++++--------------
1 file changed, 30 insertions(+), 19 deletions(-)

Toggle diff (95 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 7e14ebd1e3..d6892db6e3 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -795,11 +795,11 @@ (define-public emacs-tree-sitter
@end enumerate")
(license license:expat)))

-(define (make-emacs-tree-sitter-langs-grammar-bundle version)
+(define emacs-tree-sitter-langs-grammar-bundle
(package
(name "emacs-tree-sitter-langs-grammar-bundle")
(source #f)
- (version version)
+ (version (package-version tree-sitter))
(build-system trivial-build-system)
(inputs
;; FIXME: Support for some languages is still left to package.
@@ -836,11 +836,7 @@ (define (make-emacs-tree-sitter-langs-grammar-bundle version)
(map (match-lambda
((name directory)
(string-append directory "/lib/tree-sitter")))
- '#$(package-inputs this-package))))
- ;; The BUNDLE-VERSION file prevents emacs-tree-sitter-langs
- ;; from downloading libraries at load time.
- (call-with-output-file (string-append #$output "/BUNDLE-VERSION")
- (lambda (port) (display #$version port)))))))
+ '#$(package-inputs this-package))))))))
(synopsis #f)
(description #f)
(home-page #f)
@@ -861,7 +857,7 @@ (define-public emacs-tree-sitter-langs
"1p2zbb6ac7wi6x6zpbczcmpkb2p45md2csd2bj43d8s56ckzw5mp"))))
(build-system emacs-build-system)
(inputs
- (list (make-emacs-tree-sitter-langs-grammar-bundle version)))
+ (list emacs-tree-sitter-langs-grammar-bundle))
(propagated-inputs
(list emacs-tree-sitter))
(arguments
@@ -870,15 +866,36 @@ (define-public emacs-tree-sitter-langs
#:test-command ''("script/test")
#:phases
#~(modify-phases %standard-phases
+ (add-after 'unpack 'disable-downloader
+ (lambda _
+ (call-with-output-file "tree-sitter-langs-build.el"
+ (lambda (port)
+ (let ((on-load-message
+ (string-append
+ "tree-sitter-langs: Grammar bundle already installed "
+ "via Guix. Installing external grammars via this "
+ "function isn't supported, if a language you need is "
+ "missing please report a bug at bug-guix@gnu.org.")))
+ (format
+ port
+ ";;;###autoload
+ (defun tree-sitter-langs-install-grammars
+ (&optional skip-if-installed version os
+ keep-bundle)
+ (interactive)
+ (message \"~a\"))
+ (defconst tree-sitter-langs--queries-dir
+ (file-name-as-directory
+ (concat (file-name-directory (locate-library \"tree-sitter-langs.el\"))
+ \"queries\")))
+ (defun tree-sitter-langs--bin-dir () \"~a\")
+ (provide 'tree-sitter-langs-build)"
+ on-load-message
+ #$emacs-tree-sitter-langs-grammar-bundle))))))
(add-after 'unpack 'remove-cask
(lambda _
(substitute* "script/test"
(("cask") ""))))
- (add-before 'check 'bundle-for-testing
- (lambda* (#:key inputs #:allow-other-keys)
- (delete-file-recursively "bin")
- (symlink #$(make-emacs-tree-sitter-langs-grammar-bundle version)
- "bin")))
(add-before 'check 'patch-tree-sitter-require-test
(lambda _
(use-modules (ice-9 regex))
@@ -906,12 +923,6 @@ (define-public emacs-tree-sitter-langs
(lambda _
(delete-file-recursively "queries/hcl")
(delete-file-recursively "queries/pgn")))
- (add-before 'install 'install-bundle
- (lambda _
- (let ((elpa (elpa-directory #$output)))
- (mkdir-p elpa)
- (symlink #$(make-emacs-tree-sitter-langs-grammar-bundle version)
- (string-append elpa "/bin")))))
(add-after 'install 'install-queries
(lambda* (#:key outputs #:allow-other-keys)
(let ((elpa (elpa-directory (assoc-ref outputs "out"))))
--
2.36.0
P
P
Pierre Langlois wrote on 15 May 2022 15:16
Re: [bug#49946] [PATCH v5 01/27] gnu: tree-sitter: Move to its own module.
(name . Maxime Devos)(address . maximedevos@telenet.be)
87pmkf3p79.fsf@gmx.com
Maxime Devos <maximedevos@telenet.be> writes:

Toggle quote (7 lines)
> [[PGP Signed Part:Undecided]]
> Pierre Langlois schreef op di 29-03-2022 om 20:42 [+0100]:
>> -@item Dependency-free so that the runtime library (which is written in pure C)
>
> It probably depends on glibc library, so strictly speaking this is
> probably not technically correct ... but maybe good enough?

Oh actually, looking at a generated parser.c file, that statement appears to be
true, I was also surprised. The generated C code looks like a huge
table of rules, so it's more data than code really. For example:

I suppose it was easier to generate C code and compile it, rather than
define a custom binary format.

Thanks,
Pierre
-----BEGIN PGP SIGNATURE-----

iQFMBAEBCgA2FiEEctU9gYy29KFyWDdMqPyeRH9PfVQFAmKA/qoYHHBpZXJyZS5s
YW5nbG9pc0BnbXguY29tAAoJEKj8nkR/T31UJZIH/0kHz9qxm02VJ0fldDJHp0Gp
qxtEiFP8tSYDURGTyWj0LFWw0bozSeEm1R6IFfFRMly8bwF2RF6kGNbBq/wtS7t7
solVLlQnYF0pjgxf1UwXawN0MXlFrbRUGYXFBTmzNdpvPUN9bF1CwUgAci/5C8eC
yaPtgXrT/wdz7h+MhdkrzVWLXKRlCMI/FEYVvEYE/1ApaR2+QcyUNW36ASB+uevM
but+ECGWO+QIcRh2D6JI+ehym53ScE+LABhFpSSDFZpGMAf9hMTyTaQu6jzMxsXE
x9I0VzSDBPE91xFmr7lbYHQvMQVBGLw0DXLND5ACD1FhduzV3HtJpopan7qeT1o=
=d2qD
-----END PGP SIGNATURE-----

M
M
Maxime Devos wrote on 15 May 2022 16:33
Re: [bug#49946] [PATCH v5 27/27] gnu: Add emacs-tree-sitter-langs.
(name . Pierre Langlois)(address . pierre.langlois@gmx.com)(address . 49946@debbugs.gnu.org)
c99ef163889ca07f581961b445ce738f09db6c81.camel@telenet.be
Pierre Langlois schreef op zo 15-05-2022 om 13:20 [+0100]:
Toggle quote (9 lines)
> I 100% agree that by default we should make the package use Guix
> libraries, this way it also /just works/.  However, if somebody wants
> to
> do things differently, I'm not sure we want to get in the way.  In
> the
> end, this isn't so different from letting people use an alternative
> package manager if they like.  We'd rather people used Guix of course
> :-).

OK, my proposal to prevent any accidents (what if we misinterpreted
BUNDLE-VERSION / the semantics change in a future version, and now
binaries are downloaded by default), not to stop people from choosing
to do. Seems a bit risky though (see the bit about future changes in
semantics).

Greetings,
Maxime.
-----BEGIN PGP SIGNATURE-----

iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYoEPLxccbWF4aW1lZGV2
b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7iA1AP9B7dtADW2nf2ZKRAwVP+dfbJOX
U9zKtXhJfCX97YBOvgD/VpJvgGQvqEIB056DF2RuiZ14yTRBNMMnBcOiR0CG3AM=
=dLpL
-----END PGP SIGNATURE-----


P
P
Pierre Langlois wrote on 15 May 2022 16:55
(name . Maxime Devos)(address . maximedevos@telenet.be)
871qwu3k5x.fsf@gmx.com
Maxime Devos <maximedevos@telenet.be> writes:

Toggle quote (17 lines)
> [[PGP Signed Part:Undecided]]
> Pierre Langlois schreef op zo 15-05-2022 om 13:20 [+0100]:
>> I 100% agree that by default we should make the package use Guix
>> libraries, this way it also /just works/.  However, if somebody wants
>> to
>> do things differently, I'm not sure we want to get in the way.  In
>> the
>> end, this isn't so different from letting people use an alternative
>> package manager if they like.  We'd rather people used Guix of course
>> :-).
>
> OK, my proposal to prevent any accidents (what if we misinterpreted
> BUNDLE-VERSION / the semantics change in a future version, and now
> binaries are downloaded by default), not to stop people from choosing
> to do. Seems a bit risky though (see the bit about future changes in
> semantics).

Ah I see, that's a good point. I /think/ today we should be able to
catch such things thanks to the tests, if the package decides to
download binaries during testing it'll fail. But maybe it could, for
some reason, use Guix binaries during testing and then decide to
download its own when running outside of the build environment.

Now I'm conlicted :-), maybe rewriting the build.el file is the safest
option, and maybe if we get bug reports from people who would like to
use their own grammars we could revisit?
-----BEGIN PGP SIGNATURE-----

iQFMBAEBCgA2FiEEctU9gYy29KFyWDdMqPyeRH9PfVQFAmKBGCoYHHBpZXJyZS5s
YW5nbG9pc0BnbXguY29tAAoJEKj8nkR/T31U5mAH/AiJd6i7GkrdtZYUTYq28BR3
4AhEDc4+KkjCkXTGtxYBSf43ubF9gDFWl+QnNhbCvTS77UrGC1ya8et2oWCpv8HO
daSKKzRM20bT3A9eNAM0XuhzgKQSek/Cwbjb0k8rKg5yoJGc28ncR2EWX6xHKiRy
JtDoMjHGgIqObPoujNwZLs2f7DRPTXFkGv14JmEmImHfGLFHwVG7HVGGC9k77Rma
VbVqK1RWltrSXsv+PMJLfrSYR+G0lqIoeGkgYYsOUU3ZTkAirP3dKFRnATIN4Wce
CFk1L63jMslowHV5nAbQWSrQmtZ/YiK+VXW0mCzSkT7nDzaeRYfYh3fnriCxHmk=
=hsKp
-----END PGP SIGNATURE-----

P
P
Pierre Langlois wrote on 15 May 2022 17:22
Re: [PATCH v5 25/27] gnu: Add emacs-tree-sitter-core.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
87o7zy24lm.fsf@gmx.com
Pierre Langlois <pierre.langlois@gmx.com> writes:

Toggle quote (94 lines)
> * gnu/packages/tree-sitter.scm (tree-sitter-emacs-module): New local variable.
> (emacs-tree-sitter-core): New variable.
> ---
> gnu/packages/tree-sitter.scm | 78 ++++++++++++++++++++++++++++++++++++
> 1 file changed, 78 insertions(+)
>
> diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
> index 5209876431..fb3d13b3c7 100644
> --- a/gnu/packages/tree-sitter.scm
> +++ b/gnu/packages/tree-sitter.scm
> @@ -21,6 +21,7 @@ (define-module (gnu packages tree-sitter)
> #:use-module (guix gexp)
> #:use-module ((guix licenses) #:prefix license:)
> #:use-module (guix build-system cargo)
> + #:use-module (guix build-system emacs)
> #:use-module (guix build-system gnu)
> #:use-module (guix build-system tree-sitter)
> #:use-module (guix download)
> @@ -688,3 +689,80 @@ (define-public tree-sitter-typescript
> "This package provides Typescript and TSX grammars for the Tree-sitter
> library.")
> (license license:expat))))
> +
> +;; Local package definition solely for building the native emacs module
> +;; written in Rust.
> +(define tree-sitter-emacs-module
> + (package
> + (name "tree-sitter-emacs-module")
> + (version "0.18.0")
> + (source (origin
> + (method git-fetch)
> + (uri (git-reference
> + (url "https://github.com/emacs-tree-sitter/elisp-tree-sitter")
> + (commit version)))
> + (file-name (git-file-name name version))
> + (sha256
> + (base32
> + "1sdvz827v436qijs6xafakkfw2d16bvp8frymd818rppjc7a9dif"))))
> + (build-system cargo-build-system)
> + (inputs
> + (list tree-sitter))
> + (arguments
> + `(#:cargo-inputs
> + (("rust-anyhow" ,rust-anyhow-1)
> + ("rust-emacs" ,rust-emacs-0.18)
> + ("rust-libloading" ,rust-libloading-0.7)
> + ("rust-once-cell" ,rust-once-cell-1)
> + ("rust-tree-sitter" ,rust-tree-sitter-for-emacs))
> + #:phases
> + (modify-phases %standard-phases
> + (add-after 'unpack 'chdir
> + (lambda _ (chdir "core")))
> + (add-after 'chdir 'delete-cargo.lock
> + (lambda _ (delete-file "Cargo.lock")))
> + (add-after 'delete-cargo.lock 'do-not-fetch-from-github
> + (lambda _
> + (substitute* "Cargo.toml"
> + (("\\[patch.*") "")
> + (("git = .*") ""))))
> + (replace 'install
> + (lambda* (#:key outputs #:allow-other-keys)
> + (let ((lib (string-append (assoc-ref outputs "out") "/lib")))
> + (install-file "target/release/libtsc_dyn.so" lib)))))))
> + (home-page #f)
> + (synopsis #f)
> + (description #f)
> + (license license:expat)))
> +
> +(define-public emacs-tree-sitter-core
> + (package
> + (name "emacs-tree-sitter-core")
> + (version (package-version tree-sitter-emacs-module))
> + (source (package-source tree-sitter-emacs-module))
> + (build-system emacs-build-system)
> + (native-inputs
> + (list tree-sitter-emacs-module))
> + (arguments
> + `(#:phases
> + (modify-phases %standard-phases
> + (add-after 'unpack 'chdir
> + (lambda _ (chdir "core")))
> + (add-after 'install 'install-module
> + (lambda* (#:key inputs outputs #:allow-other-keys)
> + (let ((elpa (elpa-directory (assoc-ref outputs "out")))
> + (module (string-append (assoc-ref inputs "tree-sitter-emacs-module")
> + "/lib/libtsc_dyn.so")))
> + ;; Writing the version number in this file prevents the package
> + ;; from trying to download the module from the internet.
> + (call-with-output-file (string-append elpa "/DYN-VERSION")
> + (lambda (port) (display ,version port)))
> + (substitute* "tsc-dyn-get.el"
> + (("defcustom tsc-dyn-dir tsc--dir")
> + (string-append "defcustom tsc-dyn-dir \"" elpa "\"")))

note to self: We can use emacs-subsitute-variables here.
-----BEGIN PGP SIGNATURE-----

iQFMBAEBCgA2FiEEctU9gYy29KFyWDdMqPyeRH9PfVQFAmKBHTUYHHBpZXJyZS5s
YW5nbG9pc0BnbXguY29tAAoJEKj8nkR/T31UaMAH/Au2klug6iBs0luGllQavrBs
zIFwaDek1AvCzkm4RmH0TSxNf1ScrLIFuxjaaFo8BpSFbwwVScy7AZIUQC144Rc3
o0g4IaHVaiBjH6Q0gX6pBVWcNTeKkqjv5SuoFNjJnk12kePcPoslHraRXQ+I+rBL
EapYY/8jUnP+/p8qO97XOyZs33IFiH8HH6+wxY6Xll5xjfrLF6A/28x+clSGtq0Y
skMFqUmEank5128VDzzEjQFTyRIqPor5y2ICRkRhAilP9xvFn9K2e5mHiBAMNV4c
RmGuxjYudeph6V//NwGOJMh/vv9+Fcx2SMzNwetSA5b33Q6edI/K1qksfzB7jLQ=
=Nw7b
-----END PGP SIGNATURE-----

M
M
Maxime Devos wrote on 15 May 2022 18:05
Re: [bug#49946] [PATCH v5 27/27] gnu: Add emacs-tree-sitter-langs.
(name . Pierre Langlois)(address . pierre.langlois@gmx.com)(address . 49946@debbugs.gnu.org)
21d31e3130c8b0b6f105c3f35f31f56ecaa57f7d.camel@telenet.be
Pierre Langlois schreef op zo 15-05-2022 om 15:55 [+0100]:
Toggle quote (12 lines)
> [...]
>
> Ah I see, that's a good point. I /think/ today we should be able to
> catch such things thanks to the tests, if the package decides to
> download binaries during testing it'll fail. But maybe it could, for
> some reason, use Guix binaries during testing and then decide to
> download its own when running outside of the build environment.
>
> Now I'm conlicted :-), maybe rewriting the build.el file is the safest
> option, and maybe if we get bug reports from people who would like to
> use their own grammars we could revisit?

Seems like an option to me (I don't think we actually know yet if there
will be interest in downloading the grammars from upstream or not among
potential Guix users). Though FWIW, they can already use their own
grammars with "--with-source" transformations (to change an existing
grammar) or by packaging them (for new grammars) and the like (albeit
with Guix instead of Emacs).

Greetings,
Maxime.
-----BEGIN PGP SIGNATURE-----

iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYoEk2RccbWF4aW1lZGV2
b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7gokAP42AmdGn5rkuoD93rOr2B7H/S9+
kT9b76Bj+S6Kgl1mPQD9EDSASDY8EBVYMDvigsoCovpSn1MPbG5D8IEZp8aqtAk=
=Nl4U
-----END PGP SIGNATURE-----


P
P
Pierre Langlois wrote on 15 May 2022 20:28
[PATCH v6 00/27] gnu: Add tree-sitter for emacs.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
87mtfi63ut.fsf@gmx.com
Hi Guix!

Here's a v6 of the tree-sitter for emacs series.

Changes are:

- Rebased on master.

- Updated typescript grammar to 0.20.1.

- Remove code that downloads binaries from emacs-tree-sitter and
emacs-tree-sitter-langs. It's no longer possible to change the source
of grammars used by the package, and will now only use Guix-provided
ones.

Given this behaviour is different from upstream, we also emit a
message when grammars are loaded:

Toggle snippet (5 lines)
tree-sitter-langs: Grammar bundle already installed via Guix.
Installing external grammars via this function isn't supported, if a
language you need is missing please report a bug at bug-guix@gnu.org.

I also pushed this series to a branch on
that's useful for testing.

Thanks!
Pierre

Pierre Langlois (27):
gnu: tree-sitter: Move to its own module.
gnu: Add tree-sitter-cli.
build-system: Add tree-sitter-build-system.
gnu: Add tree-sitter-c.
gnu: Add tree-sitter-cpp.
gnu: Add tree-sitter-bash.
gnu: Add tree-sitter-css.
gnu: Add tree-sitter-c-sharp.
gnu: Add tree-sitter-elixir.
gnu: Add tree-sitter-elm.
gnu: Add tree-sitter-go.
gnu: Add tree-sitter-html.
gnu: Add tree-sitter-java.
gnu: Add tree-sitter-javascript.
gnu: Add tree-sitter-json.
gnu: Add tree-sitter-julia.
gnu: Add tree-sitter-ocaml.
gnu: Add tree-sitter-php.
gnu: Add tree-sitter-python.
gnu: Add tree-sitter-ruby.
gnu: Add tree-sitter-rust.
gnu: Add tree-sitter-typescript.
gnu: Add rust-tree-sitter.
gnu: Add rust-tree-sitter-for-emacs.
gnu: Add emacs-tree-sitter-core.
gnu: Add emacs-tree-sitter.
gnu: Add emacs-tree-sitter-langs.

Makefile.am | 2 +
doc/guix.texi | 21 +-
gnu/local.mk | 2 +
.../rust-tree-sitter-text-provider-fix.patch | 98 ++
gnu/packages/text-editors.scm | 52 -
gnu/packages/tree-sitter.scm | 947 ++++++++++++++++++
gnu/packages/vim.scm | 1 +
guix/build-system/tree-sitter.scm | 190 ++++
guix/build/tree-sitter-build-system.scm | 153 +++
9 files changed, 1413 insertions(+), 53 deletions(-)
create mode 100644 gnu/packages/patches/rust-tree-sitter-text-provider-fix.patch
create mode 100644 gnu/packages/tree-sitter.scm
create mode 100644 guix/build-system/tree-sitter.scm
create mode 100644 guix/build/tree-sitter-build-system.scm

--
2.36.0
-----BEGIN PGP SIGNATURE-----

iQFMBAEBCgA2FiEEctU9gYy29KFyWDdMqPyeRH9PfVQFAmKBR/oYHHBpZXJyZS5s
YW5nbG9pc0BnbXguY29tAAoJEKj8nkR/T31UCIwH+gKSBEStZ9vr7i3FJN+93nl2
mrqSpvXfFYpJM+VQpYUrCzk63x6drKwt72rB5AYFXEAKVC0uLGNwJII1jMpFqpQP
nM279qo4xGYsapRCqWvHnu61RaFIrgRkJANOOBCc14w+AXn5HsT8tz+75cVL7kwE
DtpVRBb/espvDyVH43mbiRpc91L7d6XzgAlgZqzKJlpnSX9VDTP6vooRvXeumQ4T
UqdCCQru8aIJIoQvqBcwmz3SCP0zS0g690a8YCHYK3gB7eUP8h+EU2r/0LSz5yXG
n/UzswwVn3x93CnoWkFB42eSrfQkI1xFusJZlBEBQkOv5XJD0DfzqO0O15zlz5U=
=+zvP
-----END PGP SIGNATURE-----

P
P
Pierre Langlois wrote on 15 May 2022 20:38
[PATCH v6 01/27] gnu: tree-sitter: Move to its own module.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220515183834.18771-1-pierre.langlois@gmx.com
* gnu/packages/text-editors.scm (tree-sitter): Move to ...
* gnu/packages/tree-sitter.scm: ... here, a new module.
* gnu/packages/vim.scm: Use (gnu packages tree-sitter).
* gnu/local.mk (GNU_SYSTEM_MODULES): Register tree-sitter.scm new module.
---
gnu/local.mk | 1 +
gnu/packages/text-editors.scm | 52 -----------------------
gnu/packages/tree-sitter.scm | 78 +++++++++++++++++++++++++++++++++++
gnu/packages/vim.scm | 1 +
4 files changed, 80 insertions(+), 52 deletions(-)
create mode 100644 gnu/packages/tree-sitter.scm

Toggle diff (175 lines)
diff --git a/gnu/local.mk b/gnu/local.mk
index cfb499915a..429a0b577b 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -589,6 +589,7 @@ GNU_SYSTEM_MODULES = \
%D%/packages/tmux.scm \
%D%/packages/toolkits.scm \
%D%/packages/tor.scm \
+ %D%/packages/tree-sitter.scm \
%D%/packages/tv.scm \
%D%/packages/uglifyjs.scm \
%D%/packages/uml.scm \
diff --git a/gnu/packages/text-editors.scm b/gnu/packages/text-editors.scm
index 7cbad042c9..ddbfef3e77 100644
--- a/gnu/packages/text-editors.scm
+++ b/gnu/packages/text-editors.scm
@@ -63,7 +63,6 @@ (define-module (gnu packages text-editors)
#:use-module (gnu packages gtk)
#:use-module (gnu packages guile)
#:use-module (gnu packages haskell-xyz)
- #:use-module (gnu packages icu4c)
#:use-module (gnu packages image)
#:use-module (gnu packages libbsd)
#:use-module (gnu packages libreoffice)
@@ -1167,54 +1166,3 @@ (define-public virtaal
provided by the Translate Toolkit, including XLIFF and PO.")
(home-page "https://virtaal.translatehouse.org/")
(license license:gpl2+)))
-
-(define-public tree-sitter
- (package
- (name "tree-sitter")
- (version "0.20.6")
- (source (origin
- (method git-fetch)
- (uri (git-reference
- (url "https://github.com/tree-sitter/tree-sitter")
- (commit (string-append "v" version))))
- (file-name (git-file-name name version))
- (sha256
- (base32
- "1z20518snyg0zp75qgs5bxmzjqws4dd19vnp6sya494za3qp5b6d"))
- (modules '((guix build utils)))
- (snippet '(begin
- ;; Remove bundled ICU parts
- (delete-file-recursively "lib/src/unicode")
- #t))))
- (build-system gnu-build-system)
- (inputs (list icu4c))
- (arguments
- (list #:phases
- '(modify-phases %standard-phases
- (delete 'configure))
- #:tests? #f ; there are no tests for the runtime library
- #:make-flags
- #~(list (string-append "PREFIX="
- #$output)
- (string-append "CC="
- #$(cc-for-target)))))
- (home-page "https://tree-sitter.github.io/tree-sitter/")
- (synopsis "Incremental parsing system for programming tools")
- (description
- "Tree-sitter is a parser generator tool and an incremental parsing
-library. It can build a concrete syntax tree for a source file and efficiently
-update the syntax tree as the source file is edited.
-
-Tree-sitter aims to be:
-
-@itemize
-@item General enough to parse any programming language
-@item Fast enough to parse on every keystroke in a text editor
-@item Robust enough to provide useful results even in the presence of syntax errors
-@item Dependency-free so that the runtime library (which is written in pure C)
-can be embedded in any application
-@end itemize
-
-This package includes the @code{libtree-sitter} runtime library.
-")
- (license license:expat)))
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
new file mode 100644
index 0000000000..7116a10ed7
--- /dev/null
+++ b/gnu/packages/tree-sitter.scm
@@ -0,0 +1,78 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2022 Luis Henrique Gomes Higino <luishenriquegh2701@gmail.com>
+;;; Copyright © 2022 Pierre Langlois <pierre.langlois@gmx.com>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu packages tree-sitter)
+ #:use-module (guix gexp)
+ #:use-module ((guix licenses) #:prefix license:)
+ #:use-module (guix build-system gnu)
+ #:use-module (guix git-download)
+ #:use-module (guix packages)
+ #:use-module (guix utils)
+ #:use-module (gnu packages icu4c))
+
+(define-public tree-sitter
+ (package
+ (name "tree-sitter")
+ (version "0.20.6")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "1z20518snyg0zp75qgs5bxmzjqws4dd19vnp6sya494za3qp5b6d"))
+ (modules '((guix build utils)))
+ (snippet '(begin
+ ;; Remove bundled ICU parts
+ (delete-file-recursively "lib/src/unicode")
+ #t))))
+ (build-system gnu-build-system)
+ (inputs (list icu4c))
+ (arguments
+ (list #:phases
+ '(modify-phases %standard-phases
+ (delete 'configure))
+ #:tests? #f ; there are no tests for the runtime library
+ #:make-flags
+ #~(list (string-append "PREFIX="
+ #$output)
+ (string-append "CC="
+ #$(cc-for-target)))))
+ (home-page "https://tree-sitter.github.io/tree-sitter/")
+ (synopsis "Incremental parsing system for programming tools")
+ (description
+ "Tree-sitter is a parser generator tool and an incremental parsing
+library. It can build a concrete syntax tree for a source file and efficiently
+update the syntax tree as the source file is edited.
+
+Tree-sitter aims to be:
+
+@itemize
+@item General enough to parse any programming language
+@item Fast enough to parse on every keystroke in a text editor
+@item Robust enough to provide useful results even in the presence of syntax errors
+@item Dependency-free so that the runtime library (which is written in pure C)
+can be embedded in any application
+@end itemize
+
+This package includes the @code{libtree-sitter} runtime library.
+")
+ (license license:expat)))
diff --git a/gnu/packages/vim.scm b/gnu/packages/vim.scm
index bb459933b0..c7f7697168 100644
--- a/gnu/packages/vim.scm
+++ b/gnu/packages/vim.scm
@@ -71,6 +71,7 @@ (define-module (gnu packages vim)
#:use-module (gnu packages tcl)
#:use-module (gnu packages text-editors)
#:use-module (gnu packages terminals)
+ #:use-module (gnu packages tree-sitter)
#:use-module (gnu packages xdisorg)
#:use-module (gnu packages xorg))

--
2.36.0
P
P
Pierre Langlois wrote on 15 May 2022 20:38
[PATCH v6 04/27] gnu: Add tree-sitter-c.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220515183834.18771-4-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-c): New variable.
---
gnu/packages/tree-sitter.scm | 33 +++++++++++++++++++++++++++++++++
1 file changed, 33 insertions(+)

Toggle diff (48 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 18c9fe0844..3f07b2d91c 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -22,6 +22,7 @@ (define-module (gnu packages tree-sitter)
#:use-module ((guix licenses) #:prefix license:)
#:use-module (guix build-system cargo)
#:use-module (guix build-system gnu)
+ #:use-module (guix build-system tree-sitter)
#:use-module (guix git-download)
#:use-module (guix packages)
#:use-module (guix utils)
@@ -155,3 +156,35 @@ (define-public tree-sitter-cli

This package includes the @command{tree-sitter} command-line tool.")
(license license:expat)))
+
+(define tree-sitter-delete-generated-files
+ '(begin
+ (delete-file "binding.gyp")
+ (delete-file-recursively "bindings")
+ (delete-file "src/grammar.json")
+ (delete-file "src/node-types.json")
+ (delete-file "src/parser.c")
+ (delete-file-recursively "src/tree_sitter")
+ #t))
+
+(define-public tree-sitter-c
+ (package
+ (name "tree-sitter-c")
+ (version "0.20.1")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-c")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "0454jziys33i4kbwnvi9xcck0fzya792ghy32ahgk1hhv96xga9w"))
+ (modules '((guix build utils)))
+ (snippet tree-sitter-delete-generated-files)))
+ (build-system tree-sitter-build-system)
+ (home-page "https://github.com/tree-sitter/tree-sitter-c")
+ (synopsis "Tree-sitter C grammar")
+ (description
+ "This package provides a C grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.36.0
P
P
Pierre Langlois wrote on 15 May 2022 20:38
[PATCH v6 05/27] gnu: Add tree-sitter-cpp.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220515183834.18771-5-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-cpp): New variable.
---
gnu/packages/tree-sitter.scm | 23 +++++++++++++++++++++++
1 file changed, 23 insertions(+)

Toggle diff (31 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 3f07b2d91c..5e25d48c05 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -188,3 +188,26 @@ (define-public tree-sitter-c
(description
"This package provides a C grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-cpp
+ (package
+ (name "tree-sitter-cpp")
+ (version "0.20.0")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-cpp")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "0hxcpdvyyig8njga1mxp4qcnbbnr1d0aiy27vahijwbh98b081nr"))
+ (modules '((guix build utils)))
+ (snippet tree-sitter-delete-generated-files)))
+ (build-system tree-sitter-build-system)
+ (inputs (list tree-sitter-c))
+ (home-page "https://github.com/tree-sitter/tree-sitter-cpp")
+ (synopsis "Tree-sitter C++ grammar")
+ (description
+ "This package provides a C++ grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.36.0
P
P
Pierre Langlois wrote on 15 May 2022 20:38
[PATCH v6 02/27] gnu: Add tree-sitter-cli.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220515183834.18771-2-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-cli): New variable.
---
gnu/packages/tree-sitter.scm | 79 ++++++++++++++++++++++++++++++++++++
1 file changed, 79 insertions(+)

Toggle diff (98 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 7116a10ed7..18c9fe0844 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -20,10 +20,13 @@
(define-module (gnu packages tree-sitter)
#:use-module (guix gexp)
#:use-module ((guix licenses) #:prefix license:)
+ #:use-module (guix build-system cargo)
#:use-module (guix build-system gnu)
#:use-module (guix git-download)
#:use-module (guix packages)
#:use-module (guix utils)
+ #:use-module (gnu packages crates-graphics)
+ #:use-module (gnu packages crates-io)
#:use-module (gnu packages icu4c))

(define-public tree-sitter
@@ -76,3 +79,79 @@ (define-public tree-sitter
This package includes the @code{libtree-sitter} runtime library.
")
(license license:expat)))
+
+(define-public tree-sitter-cli
+ (package (inherit tree-sitter)
+ (name "tree-sitter-cli")
+ (source (origin
+ (inherit (package-source tree-sitter))
+ (snippet
+ '(begin
+ ;; Remove the runtime library code and dynamically link to
+ ;; it instead.
+ (delete-file-recursively "lib/src")
+ (delete-file "lib/binding_rust/build.rs")
+ (with-output-to-file "lib/binding_rust/build.rs"
+ (lambda _
+ (format #t "fn main() {~@
+ println!(\"cargo:rustc-link-lib=tree-sitter\");~@
+ }~%")))
+ #t))))
+ (build-system cargo-build-system)
+ (inputs (list tree-sitter))
+ (arguments
+ `(;; Running test requires downloading fixtures, see the
+ ;; script/fetch-fixtures script.
+ #:tests? #f
+ ;; We're only packaging the CLI program so we do not need to install
+ ;; sources.
+ #:install-source? #f
+ #:cargo-inputs
+ (("rust-ansi-term" ,rust-ansi-term-0.12)
+ ("rust-anyhow" ,rust-anyhow-1)
+ ("rust-atty" ,rust-atty-0.2)
+ ("rust-clap" ,rust-clap-2)
+ ("rust-difference" ,rust-difference-2)
+ ("rust-html-escape" ,rust-html-escape-0.2)
+ ("rust-libloading" ,rust-libloading-0.7)
+ ("rust-rand" ,rust-rand-0.8)
+ ("rust-rustc-hash" ,rust-rustc-hash-1)
+ ("rust-semver" ,rust-semver-1)
+ ("rust-smallbitvec" ,rust-smallbitvec-2)
+ ("rust-thiserror" ,rust-thiserror-1)
+ ("rust-tiny-http" ,rust-tiny-http-0.8)
+ ("rust-toml" ,rust-toml-0.5)
+ ("rust-walkdir" ,rust-walkdir-2)
+ ("rust-webbrowser" ,rust-webbrowser-0.5)
+ ("rust-which" ,rust-which-4))
+ #:cargo-development-inputs
+ (("rust-pretty-assertions" ,rust-pretty-assertions-0.7))
+ #:phases
+ (modify-phases %standard-phases
+ (add-after 'unpack 'delete-cargo.lock
+ (lambda _ (delete-file "Cargo.lock")))
+ (replace 'install
+ (lambda* (#:key outputs #:allow-other-keys)
+ (let* ((out (assoc-ref outputs "out"))
+ (bin (string-append out "/bin"))
+ (lib (string-append out "/lib")))
+ (mkdir-p bin)
+ (install-file "target/release/tree-sitter" bin)))))))
+ (synopsis "Incremental parsing system for programming tools")
+ (description "Tree-sitter is a parser generator tool and an incremental
+parsing library. It can build a concrete syntax tree for a source file and
+efficiently update the syntax tree as the source file is edited.
+
+Tree-sitter aims to be:
+
+@enumerate
+@item General enough to parse any programming language.
+@item Fast enough to parse on every keystroke in a text editor.
+@item Robust enough to provide useful results even in the presence of syntax
+errors.
+@item Dependency-free so that the runtime library (which is written in pure C)
+can be embedded in any application.
+@end enumerate
+
+This package includes the @command{tree-sitter} command-line tool.")
+ (license license:expat)))
--
2.36.0
P
P
Pierre Langlois wrote on 15 May 2022 20:38
[PATCH v6 07/27] gnu: Add tree-sitter-css.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220515183834.18771-7-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-css): New variable.
---
gnu/packages/tree-sitter.scm | 22 ++++++++++++++++++++++
1 file changed, 22 insertions(+)

Toggle diff (30 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 084071fe3d..d2f8326be0 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -233,3 +233,25 @@ (define-public tree-sitter-bash
(description
"This package provides a Bash grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-css
+ (package
+ (name "tree-sitter-css")
+ (version "0.19.0")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-css")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "014jrlgi7zfza9g38hsr4vlbi8964i5p7iglaih6qmzaiml7bja2"))
+ (modules '((guix build utils)))
+ (snippet tree-sitter-delete-generated-files)))
+ (build-system tree-sitter-build-system)
+ (home-page "https://github.com/tree-sitter/tree-sitter-css")
+ (synopsis "Tree-sitter CSS grammar")
+ (description
+ "This package provides a CSS grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.36.0
P
P
Pierre Langlois wrote on 15 May 2022 20:38
[PATCH v6 06/27] gnu: Add tree-sitter-bash.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220515183834.18771-6-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-bash): New variable.
---
gnu/packages/tree-sitter.scm | 22 ++++++++++++++++++++++
1 file changed, 22 insertions(+)

Toggle diff (30 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 5e25d48c05..084071fe3d 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -211,3 +211,25 @@ (define-public tree-sitter-cpp
(description
"This package provides a C++ grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-bash
+ (package
+ (name "tree-sitter-bash")
+ (version "0.19.0")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-bash")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "18c030bb65r50i6z37iy7jb9z9i8i36y7b08dbc9bchdifqsijs5"))
+ (modules '((guix build utils)))
+ (snippet tree-sitter-delete-generated-files)))
+ (build-system tree-sitter-build-system)
+ (home-page "https://github.com/tree-sitter/tree-sitter-bash")
+ (synopsis "Tree-sitter Bash grammar")
+ (description
+ "This package provides a Bash grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.36.0
P
P
Pierre Langlois wrote on 15 May 2022 20:38
[PATCH v6 09/27] gnu: Add tree-sitter-elixir.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220515183834.18771-9-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-elixir): New variable.
---
gnu/packages/tree-sitter.scm | 26 ++++++++++++++++++++++++++
1 file changed, 26 insertions(+)

Toggle diff (34 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index de90ac2093..18ea00f7cf 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -277,3 +277,29 @@ (define-public tree-sitter-css
(description
"This package provides a CSS grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-elixir
+ (let ((commit "de20391afe5cb03ef1e8a8e43167e7b58cc52869")
+ (revision "1"))
+ (package
+ (name "tree-sitter-elixir")
+ (version (git-version "0.19.0" revision commit))
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/elixir-lang/tree-sitter-elixir")
+ (commit commit)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "0zrkrwhw3g1vazsxcwrfd1fk4wvs9hdwmwp6073mfh370bz4140h"))
+ (modules '((guix build utils)))
+ (snippet tree-sitter-delete-generated-files)))
+ (build-system tree-sitter-build-system)
+ (home-page "https://elixir-lang.org/tree-sitter-elixir/")
+ (synopsis "Tree-sitter Elixir grammar")
+ (description
+ "This package provides a Elixir grammar for the Tree-sitter library.")
+ (license (list license:asl2.0
+ ;; Files in tests/corpus are under MIT license.
+ license:expat)))))
--
2.36.0
P
P
Pierre Langlois wrote on 15 May 2022 20:38
[PATCH v6 08/27] gnu: Add tree-sitter-c-sharp.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220515183834.18771-8-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-c-sharp): New variable.
---
gnu/packages/tree-sitter.scm | 22 ++++++++++++++++++++++
1 file changed, 22 insertions(+)

Toggle diff (33 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index d2f8326be0..de90ac2093 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -212,6 +212,28 @@ (define-public tree-sitter-cpp
"This package provides a C++ grammar for the Tree-sitter library.")
(license license:expat)))

+(define-public tree-sitter-c-sharp
+ (package
+ (name "tree-sitter-c-sharp")
+ (version "0.19.1")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-c-sharp")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "054fmpf47cwh59gbg00sc0nl237ba4rnxi73miz39yqzcs87055r"))
+ (modules '((guix build utils)))
+ (snippet tree-sitter-delete-generated-files)))
+ (build-system tree-sitter-build-system)
+ (home-page "https://github.com/tree-sitter/tree-sitter-c-sharp")
+ (synopsis "Tree-sitter C# grammar")
+ (description
+ "This package provides a C# grammar for the Tree-sitter library.")
+ (license license:expat)))
+
(define-public tree-sitter-bash
(package
(name "tree-sitter-bash")
--
2.36.0
P
P
Pierre Langlois wrote on 15 May 2022 20:38
[PATCH v6 10/27] gnu: Add tree-sitter-elm.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220515183834.18771-10-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-elm): New variable.
---
gnu/packages/tree-sitter.scm | 22 ++++++++++++++++++++++
1 file changed, 22 insertions(+)

Toggle diff (30 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 18ea00f7cf..0aafaf7c1b 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -303,3 +303,25 @@ (define-public tree-sitter-elixir
(license (list license:asl2.0
;; Files in tests/corpus are under MIT license.
license:expat)))))
+
+(define-public tree-sitter-elm
+ (package
+ (name "tree-sitter-elm")
+ (version "5.5.1")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/elm-tooling/tree-sitter-elm")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "10hbi4vyj4hjixqswdcbvzl60prldczz29mlp02if61wvwiwvqrw"))
+ (modules '((guix build utils)))
+ (snippet tree-sitter-delete-generated-files)))
+ (build-system tree-sitter-build-system)
+ (home-page "https://elm-tooling.github.io/tree-sitter-elm/")
+ (synopsis "Tree-sitter Elm grammar")
+ (description
+ "This package provides an Elm grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.36.0
P
P
Pierre Langlois wrote on 15 May 2022 20:38
[PATCH v6 11/27] gnu: Add tree-sitter-go.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220515183834.18771-11-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-go): New variable.
---
gnu/packages/tree-sitter.scm | 22 ++++++++++++++++++++++
1 file changed, 22 insertions(+)

Toggle diff (30 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 0aafaf7c1b..585b44b9f9 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -325,3 +325,25 @@ (define-public tree-sitter-elm
(description
"This package provides an Elm grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-go
+ (package
+ (name "tree-sitter-go")
+ (version "0.19.1")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-go")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "0nxs47vd2fc2fr0qlxq496y852rwg39flhg334s7dlyq7d3lcx4x"))
+ (modules '((guix build utils)))
+ (snippet tree-sitter-delete-generated-files)))
+ (build-system tree-sitter-build-system)
+ (home-page "https://github.com/tree-sitter/tree-sitter-go")
+ (synopsis "Tree-sitter Go grammar")
+ (description
+ "This package provides a Golang grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.36.0
P
P
Pierre Langlois wrote on 15 May 2022 20:38
[PATCH v6 12/27] gnu: Add tree-sitter-html.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220515183834.18771-12-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-html): New variable.
---
gnu/packages/tree-sitter.scm | 22 ++++++++++++++++++++++
1 file changed, 22 insertions(+)

Toggle diff (30 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 585b44b9f9..15f832f8ca 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -347,3 +347,25 @@ (define-public tree-sitter-go
(description
"This package provides a Golang grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-html
+ (package
+ (name "tree-sitter-html")
+ (version "0.19.0")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-html")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "1hg7vbcy7bir6b8x11v0a4x0glvqnsqc3i2ixiarbxmycbgl3axy"))
+ (modules '((guix build utils)))
+ (snippet tree-sitter-delete-generated-files)))
+ (build-system tree-sitter-build-system)
+ (home-page "https://github.com/tree-sitter/tree-sitter-html")
+ (synopsis "Tree-sitter HTML grammar")
+ (description
+ "This package provides a HTML grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.36.0
P
P
Pierre Langlois wrote on 15 May 2022 20:38
[PATCH v6 13/27] gnu: Add tree-sitter-java.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220515183834.18771-13-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-java): New variable.
---
gnu/packages/tree-sitter.scm | 22 ++++++++++++++++++++++
1 file changed, 22 insertions(+)

Toggle diff (30 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 15f832f8ca..010a7c3751 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -369,3 +369,25 @@ (define-public tree-sitter-html
(description
"This package provides a HTML grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-java
+ (package
+ (name "tree-sitter-java")
+ (version "0.19.1")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-java")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "07zw9ygb45hnvlx9qlz7rlz8hc3byjy03d24v72i5iyhpiiwlhvl"))
+ (modules '((guix build utils)))
+ (snippet tree-sitter-delete-generated-files)))
+ (build-system tree-sitter-build-system)
+ (home-page "https://github.com/tree-sitter/tree-sitter-java")
+ (synopsis "Tree-sitter Java grammar")
+ (description
+ "This package provides a Java grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.36.0
P
P
Pierre Langlois wrote on 15 May 2022 20:38
[PATCH v6 14/27] gnu: Add tree-sitter-javascript.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220515183834.18771-14-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-javascript): New variable.
---
gnu/packages/tree-sitter.scm | 23 +++++++++++++++++++++++
1 file changed, 23 insertions(+)

Toggle diff (31 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 010a7c3751..30220a5a8c 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -391,3 +391,26 @@ (define-public tree-sitter-java
(description
"This package provides a Java grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-javascript
+ (package
+ (name "tree-sitter-javascript")
+ (version "0.20.0")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-javascript")
+ (commit (string-append "rust-" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "175yrk382n2di0c2xn4gpv8y4n83x1lg4hqn04vabf0yqynlkq67"))
+ (modules '((guix build utils)))
+ (snippet tree-sitter-delete-generated-files)))
+ (build-system tree-sitter-build-system)
+ (home-page "https://github.com/tree-sitter/tree-sitter-javascript")
+ (synopsis "Tree-sitter Javascript grammar")
+ (description
+ "This package provides Javascript and JSX grammars for the Tree-sitter
+library.")
+ (license license:expat)))
--
2.36.0
P
P
Pierre Langlois wrote on 15 May 2022 20:38
[PATCH v6 15/27] gnu: Add tree-sitter-json.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220515183834.18771-15-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-json): New variable.
---
gnu/packages/tree-sitter.scm | 22 ++++++++++++++++++++++
1 file changed, 22 insertions(+)

Toggle diff (30 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 30220a5a8c..d43e16af09 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -414,3 +414,25 @@ (define-public tree-sitter-javascript
"This package provides Javascript and JSX grammars for the Tree-sitter
library.")
(license license:expat)))
+
+(define-public tree-sitter-json
+ (package
+ (name "tree-sitter-json")
+ (version "0.19.0")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-json")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "06pjh31bv9ja9hlnykk257a6zh8bsxg2fqa54al7qk1r4n9ksnff"))
+ (modules '((guix build utils)))
+ (snippet tree-sitter-delete-generated-files)))
+ (build-system tree-sitter-build-system)
+ (home-page "https://github.com/tree-sitter/tree-sitter-json")
+ (synopsis "Tree-sitter JSON grammar")
+ (description
+ "This package provides a JSON grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.36.0
P
P
Pierre Langlois wrote on 15 May 2022 20:38
[PATCH v6 16/27] gnu: Add tree-sitter-julia.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220515183834.18771-16-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-julia): New variable.
---
gnu/packages/tree-sitter.scm | 22 ++++++++++++++++++++++
1 file changed, 22 insertions(+)

Toggle diff (30 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index d43e16af09..a535d8bc4a 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -436,3 +436,25 @@ (define-public tree-sitter-json
(description
"This package provides a JSON grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-julia
+ (package
+ (name "tree-sitter-julia")
+ (version "0.19.0")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-julia")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "1pbnmvhy2gq4vg1b0sjzmjm4s2gsgdjh7h01yj8qrrqbcl29c463"))
+ (modules '((guix build utils)))
+ (snippet tree-sitter-delete-generated-files)))
+ (build-system tree-sitter-build-system)
+ (home-page "https://github.com/tree-sitter/tree-sitter-julia")
+ (synopsis "Tree-sitter Julia grammar")
+ (description
+ "This package provides a Julia grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.36.0
P
P
Pierre Langlois wrote on 15 May 2022 20:38
[PATCH v6 17/27] gnu: Add tree-sitter-ocaml.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220515183834.18771-17-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-ocaml): New variable.
---
gnu/packages/tree-sitter.scm | 39 ++++++++++++++++++++++++++++++++++++
1 file changed, 39 insertions(+)

Toggle diff (47 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index a535d8bc4a..ff98044cc1 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -458,3 +458,42 @@ (define-public tree-sitter-julia
(description
"This package provides a Julia grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-ocaml
+ (let ((commit "0348562f385bc2bd67ecf181425e1afd6d454192")
+ (revision "1"))
+ (package
+ (name "tree-sitter-ocaml")
+ (version (git-version "0.19.0" revision commit))
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-ocaml")
+ (commit commit)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "0iqmwcz3c2ai4gyx4xli1rhn6hi6a0f60dn20f8jas9ham9dc2df"))
+ (modules '((guix build utils)))
+ (snippet
+ '(begin
+ (delete-file "binding.gyp")
+ (delete-file-recursively "bindings")
+ (for-each
+ (lambda (lang)
+ (with-directory-excursion lang
+ (delete-file "src/grammar.json")
+ (delete-file "src/node-types.json")
+ (delete-file "src/parser.c")
+ (delete-file-recursively "src/tree_sitter")))
+ '("ocaml" "interface"))
+ #t))))
+ (build-system tree-sitter-build-system)
+ (arguments
+ (list
+ #:grammar-directories '("ocaml" "interface")))
+ (home-page "https://github.com/tree-sitter/tree-sitter-ocaml")
+ (synopsis "Tree-sitter OCaml grammar")
+ (description
+ "This package provides a OCaml grammar for the Tree-sitter library.")
+ (license license:expat))))
--
2.36.0
P
P
Pierre Langlois wrote on 15 May 2022 20:38
[PATCH v6 18/27] gnu: Add tree-sitter-php.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220515183834.18771-18-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-php): New variable.
---
gnu/packages/tree-sitter.scm | 30 ++++++++++++++++++++++++++++++
1 file changed, 30 insertions(+)

Toggle diff (38 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index ff98044cc1..654445f71d 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -497,3 +497,33 @@ (define-public tree-sitter-ocaml
(description
"This package provides a OCaml grammar for the Tree-sitter library.")
(license license:expat))))
+
+(define-public tree-sitter-php
+ (let ((commit "435fa00006c0d1515c37fbb4dd6a9de284af75ab")
+ (revision "1"))
+ (package
+ (name "tree-sitter-php")
+ (version (git-version "0.19.0" revision commit))
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-php")
+ (commit commit)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "05k4h58gi616gv41r0qqdb1x4rs8y94vghn2r10yczisgzq4vbad"))
+ (modules '((guix build utils)))
+ (snippet
+ '(begin
+ (delete-file "src/grammar.json")
+ (delete-file "src/node-types.json")
+ (delete-file "src/parser.c")
+ (delete-file-recursively "src/tree_sitter")
+ #t))))
+ (build-system tree-sitter-build-system)
+ (home-page "https://github.com/tree-sitter/tree-sitter-php")
+ (synopsis "Tree-sitter PHP grammar")
+ (description
+ "This package provides a PHP grammar for the Tree-sitter library.")
+ (license license:expat))))
--
2.36.0
P
P
Pierre Langlois wrote on 15 May 2022 20:38
[PATCH v6 19/27] gnu: Add tree-sitter-python.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220515183834.18771-19-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-python): New variable.
---
gnu/packages/tree-sitter.scm | 24 ++++++++++++++++++++++++
1 file changed, 24 insertions(+)

Toggle diff (32 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 654445f71d..c43cafea2d 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -527,3 +527,27 @@ (define-public tree-sitter-php
(description
"This package provides a PHP grammar for the Tree-sitter library.")
(license license:expat))))
+
+(define-public tree-sitter-python
+ (let ((commit "ed0fe62e55dc617ed9dec8817ebf771aa7cf3c42")
+ (revision "1"))
+ (package
+ (name "tree-sitter-python")
+ (version (git-version "0.19.1" revision commit))
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-python")
+ (commit commit)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "0wrfpg84mc3pzcrdi6n5fqwijkqr1nj5sqfnayb502krvqpjilal"))
+ (modules '((guix build utils)))
+ (snippet tree-sitter-delete-generated-files)))
+ (build-system tree-sitter-build-system)
+ (home-page "https://github.com/tree-sitter/tree-sitter-python")
+ (synopsis "Tree-sitter Python grammar")
+ (description
+ "This package provides a Python grammar for the Tree-sitter library.")
+ (license license:expat))))
--
2.36.0
P
P
Pierre Langlois wrote on 15 May 2022 20:38
[PATCH v6 20/27] gnu: Add tree-sitter-ruby.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220515183834.18771-20-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-ruby): New variable.
---
gnu/packages/tree-sitter.scm | 22 ++++++++++++++++++++++
1 file changed, 22 insertions(+)

Toggle diff (30 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index c43cafea2d..c7b1998253 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -551,3 +551,25 @@ (define-public tree-sitter-python
(description
"This package provides a Python grammar for the Tree-sitter library.")
(license license:expat))))
+
+(define-public tree-sitter-ruby
+ (package
+ (name "tree-sitter-ruby")
+ (version "0.19.0")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-ruby")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "0m3h4928rbs300wcb6776h9r88hi32rybbhcaf6rdympl5nzi83v"))
+ (modules '((guix build utils)))
+ (snippet tree-sitter-delete-generated-files)))
+ (build-system tree-sitter-build-system)
+ (home-page "https://github.com/tree-sitter/tree-sitter-ruby")
+ (synopsis "Tree-sitter Ruby grammar")
+ (description
+ "This package provides a Ruby grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.36.0
P
P
Pierre Langlois wrote on 15 May 2022 20:38
[PATCH v6 22/27] gnu: Add tree-sitter-typescript.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220515183834.18771-22-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-typescript): New variable.
---
gnu/packages/tree-sitter.scm | 38 ++++++++++++++++++++++++++++++++++++
1 file changed, 38 insertions(+)

Toggle diff (46 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index c37183e131..04747f47e5 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -595,3 +595,41 @@ (define-public tree-sitter-rust
(description
"This package provides a Rust grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-typescript
+ (package
+ (name "tree-sitter-typescript")
+ (version "0.20.1")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-typescript")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32 "07fl9d968lal0aqj4f0n16p3n94cjkgfp54wynfr8gbdkjss5v5x"))
+ (modules '((guix build utils)))
+ (snippet
+ '(begin
+ (delete-file "binding.gyp")
+ (delete-file-recursively "bindings")
+ (for-each
+ (lambda (lang)
+ (with-directory-excursion lang
+ (delete-file "src/grammar.json")
+ (delete-file "src/node-types.json")
+ (delete-file "src/parser.c")
+ (delete-file-recursively "src/tree_sitter")))
+ '("typescript" "tsx"))
+ #t))))
+ (build-system tree-sitter-build-system)
+ (inputs (list tree-sitter-javascript))
+ (arguments
+ (list
+ #:grammar-directories '("typescript" "tsx")))
+ (home-page "https://github.com/tree-sitter/tree-sitter-typescript")
+ (synopsis "Tree-sitter Typescript grammar")
+ (description
+ "This package provides Typescript and TSX grammars for the Tree-sitter
+library.")
+ (license license:expat)))
--
2.36.0
P
P
Pierre Langlois wrote on 15 May 2022 20:38
[PATCH v6 25/27] gnu: Add emacs-tree-sitter-core.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220515183834.18771-25-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-emacs-module): New local variable.
(emacs-tree-sitter-core): New variable.
---
gnu/packages/tree-sitter.scm | 84 ++++++++++++++++++++++++++++++++++++
1 file changed, 84 insertions(+)

Toggle diff (99 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 4d1d3b2177..3e01474285 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -21,6 +21,7 @@ (define-module (gnu packages tree-sitter)
#:use-module (guix gexp)
#:use-module ((guix licenses) #:prefix license:)
#:use-module (guix build-system cargo)
+ #:use-module (guix build-system emacs)
#:use-module (guix build-system gnu)
#:use-module (guix build-system tree-sitter)
#:use-module (guix download)
@@ -686,3 +687,86 @@ (define-public tree-sitter-typescript
"This package provides Typescript and TSX grammars for the Tree-sitter
library.")
(license license:expat)))
+
+;; Local package definition solely for building the native emacs module
+;; written in Rust.
+(define tree-sitter-emacs-module
+ (package
+ (name "tree-sitter-emacs-module")
+ (version "0.18.0")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/emacs-tree-sitter/elisp-tree-sitter")
+ (commit version)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "1sdvz827v436qijs6xafakkfw2d16bvp8frymd818rppjc7a9dif"))))
+ (build-system cargo-build-system)
+ (inputs
+ (list tree-sitter))
+ (arguments
+ `(#:cargo-inputs
+ (("rust-anyhow" ,rust-anyhow-1)
+ ("rust-emacs" ,rust-emacs-0.18)
+ ("rust-libloading" ,rust-libloading-0.7)
+ ("rust-once-cell" ,rust-once-cell-1)
+ ("rust-tree-sitter" ,rust-tree-sitter-for-emacs))
+ #:phases
+ (modify-phases %standard-phases
+ (add-after 'unpack 'chdir
+ (lambda _ (chdir "core")))
+ (add-after 'chdir 'delete-cargo.lock
+ (lambda _ (delete-file "Cargo.lock")))
+ (add-after 'delete-cargo.lock 'do-not-fetch-from-github
+ (lambda _
+ (substitute* "Cargo.toml"
+ (("\\[patch.*") "")
+ (("git = .*") ""))))
+ (replace 'install
+ (lambda* (#:key outputs #:allow-other-keys)
+ (let ((lib (string-append (assoc-ref outputs "out") "/lib")))
+ (mkdir-p lib)
+ (copy-file "target/release/libtsc_dyn.so"
+ (string-append lib "/tsc-dyn.so"))))))))
+ (home-page #f)
+ (synopsis #f)
+ (description #f)
+ (license license:expat)))
+
+(define-public emacs-tree-sitter-core
+ (package
+ (name "emacs-tree-sitter-core")
+ (version (package-version tree-sitter-emacs-module))
+ (source (package-source tree-sitter-emacs-module))
+ (build-system emacs-build-system)
+ (native-inputs
+ (list tree-sitter-emacs-module))
+ (arguments
+ `(#:phases
+ (modify-phases %standard-phases
+ (add-after 'unpack 'chdir
+ (lambda _ (chdir "core")))
+ (add-before 'install 'find-tsc-dyn
+ (lambda* (#:key inputs outputs #:allow-other-keys)
+ (let ((elpa (elpa-directory (assoc-ref outputs "out")))
+ (module (search-input-file inputs "/lib/tsc-dyn.so")))
+ ;; Install the tsc-dyn module in site-lisp and the current
+ ;; directory for test to pass.
+ (install-file module elpa)
+ (copy-file module "tsc-dyn.so")
+ ;; We replace the tsc-dyn-get.el file with an empty stub to
+ ;; prevent the code from downloading the module.
+ (call-with-output-file "tsc-dyn-get.el"
+ (lambda (port)
+ (for-each
+ (lambda (sexp) (write sexp port))
+ '((defun tsc-dyn-get-ensure (requested)
+ nil)
+ (provide 'tsc-dyn-get)))))))))))
+ (home-page "https://github.com/emacs-tree-sitter/elisp-tree-sitter")
+ (synopsis "Tree-sitter bindings for Emacs Lisp, core library")
+ (description "This package provides core APIs of the Emacs binding for
+Tree-sitter, an incremental parsing system.")
+ (license license:expat)))
--
2.36.0
P
P
Pierre Langlois wrote on 15 May 2022 20:38
[PATCH v6 26/27] gnu: Add emacs-tree-sitter.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220515183834.18771-26-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (emacs-tree-sitter): New variable.
---
gnu/packages/tree-sitter.scm | 27 +++++++++++++++++++++++++++
1 file changed, 27 insertions(+)

Toggle diff (35 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 3e01474285..cb3f326367 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -770,3 +770,30 @@ (module (search-input-file inputs "/lib/tsc-dyn.so")))
(description "This package provides core APIs of the Emacs binding for
Tree-sitter, an incremental parsing system.")
(license license:expat)))
+
+(define-public emacs-tree-sitter
+ (package
+ (name "emacs-tree-sitter")
+ (version (package-version emacs-tree-sitter-core))
+ (source (package-source emacs-tree-sitter-core))
+ (build-system emacs-build-system)
+ (propagated-inputs
+ (list emacs-tree-sitter-core))
+ (arguments
+ `(#:phases
+ (modify-phases %standard-phases
+ (add-after 'unpack 'chdir
+ (lambda _ (chdir "lisp"))))))
+ (home-page "https://github.com/emacs-tree-sitter/elisp-tree-sitter")
+ (synopsis "Tree-sitter bindings for Emacs Lisp")
+ (description "This package provides Emacs bindings for Tree-sitter, an
+incremental parsing library. It aims to be the foundation for a new breed of
+Emacs packages that understand code structurally. For example:
+
+@enumerate
+@item Faster, fine-grained code highlighting.
+@item More flexible code folding.
+@item Structural editing (like Paredit, or even better) for non-Lisp code.
+@item More informative indexing for imenu.
+@end enumerate")
+ (license license:expat)))
--
2.36.0
P
P
Pierre Langlois wrote on 15 May 2022 20:38
[PATCH v6 27/27] gnu: Add emacs-tree-sitter-langs.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220515183834.18771-27-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (make-emacs-tree-sitter-langs-grammar-bundle):
New procedure.
(emacs-tree-sitter-langs): New variable.
---
gnu/packages/tree-sitter.scm | 148 +++++++++++++++++++++++++++++++++++
1 file changed, 148 insertions(+)

Toggle diff (163 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index cb3f326367..b80d912bfc 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -24,6 +24,7 @@ (define-module (gnu packages tree-sitter)
#:use-module (guix build-system emacs)
#:use-module (guix build-system gnu)
#:use-module (guix build-system tree-sitter)
+ #:use-module (guix build-system trivial)
#:use-module (guix download)
#:use-module (guix git-download)
#:use-module (guix packages)
@@ -797,3 +798,150 @@ (define-public emacs-tree-sitter
@item More informative indexing for imenu.
@end enumerate")
(license license:expat)))
+
+(define emacs-tree-sitter-langs-grammar-bundle
+ (package
+ (name "emacs-tree-sitter-langs-grammar-bundle")
+ (source #f)
+ (version (package-version tree-sitter))
+ (build-system trivial-build-system)
+ (inputs
+ ;; FIXME: Support for some languages is still left to package.
+ (list tree-sitter-bash
+ tree-sitter-c
+ tree-sitter-c-sharp
+ tree-sitter-cpp
+ tree-sitter-css
+ tree-sitter-elixir
+ tree-sitter-elm
+ tree-sitter-go
+ tree-sitter-html
+ tree-sitter-java
+ tree-sitter-javascript
+ tree-sitter-json
+ tree-sitter-julia
+ tree-sitter-ocaml
+ tree-sitter-php
+ tree-sitter-python
+ tree-sitter-rust
+ tree-sitter-ruby
+ tree-sitter-typescript))
+ (arguments
+ (list #:builder
+ (with-imported-modules '((guix build union)
+ (guix build utils))
+ #~(begin
+ (use-modules (ice-9 match)
+ (guix build union)
+ (guix build utils))
+ (union-build
+ #$output
+ (filter directory-exists?
+ (map (match-lambda
+ ((name directory)
+ (string-append directory "/lib/tree-sitter")))
+ '#$(package-inputs this-package))))))))
+ (synopsis #f)
+ (description #f)
+ (home-page #f)
+ (license #f)))
+
+(define-public emacs-tree-sitter-langs
+ (package
+ (name "emacs-tree-sitter-langs")
+ (version "0.12.0")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/emacs-tree-sitter/tree-sitter-langs")
+ (commit version)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "1p2zbb6ac7wi6x6zpbczcmpkb2p45md2csd2bj43d8s56ckzw5mp"))))
+ (build-system emacs-build-system)
+ (inputs
+ (list emacs-tree-sitter-langs-grammar-bundle))
+ (propagated-inputs
+ (list emacs-tree-sitter))
+ (arguments
+ (list
+ #:tests? #t
+ #:test-command ''("script/test")
+ #:phases
+ #~(modify-phases %standard-phases
+ (add-after 'unpack 'disable-downloader
+ (lambda _
+ (call-with-output-file "tree-sitter-langs-build.el"
+ (lambda (port)
+ (let ((on-load-message
+ (string-append
+ "tree-sitter-langs: Grammar bundle already installed "
+ "via Guix. Installing external grammars via this "
+ "function isn't supported, if a language you need is "
+ "missing please report a bug at bug-guix@gnu.org.")))
+ (format
+ port
+ ";;;###autoload
+ (defun tree-sitter-langs-install-grammars
+ (&optional skip-if-installed version os
+ keep-bundle)
+ (interactive)
+ (message \"~a\"))
+ (defconst tree-sitter-langs--queries-dir
+ (file-name-as-directory
+ (concat (file-name-directory (locate-library \"tree-sitter-langs.el\"))
+ \"queries\")))
+ (defun tree-sitter-langs--bin-dir () \"~a\")
+ (provide 'tree-sitter-langs-build)"
+ on-load-message
+ #$emacs-tree-sitter-langs-grammar-bundle))))))
+ (add-after 'unpack 'remove-cask
+ (lambda _
+ (substitute* "script/test"
+ (("cask") ""))))
+ (add-before 'check 'patch-tree-sitter-require-test
+ (lambda _
+ (use-modules (ice-9 regex))
+ ;; This test needs a git repositories with submodules for
+ ;; each languages in order to map all repositories. We patch
+ ;; the mapping function with one that invokes the tests for each
+ ;; packaged language.
+ (let ((supported-languages
+ (map (lambda (lib)
+ (match:substring
+ (string-match "(.*)\\.so$" (basename lib))
+ 1))
+ (find-files "bin" "\\.so$"))))
+ (substitute* "tree-sitter-langs-tests.el"
+ (("tree-sitter-langs--map-repos")
+ (call-with-output-string
+ (lambda (port)
+ (write `(lambda (fn)
+ (dolist (lang ',supported-languages)
+ (funcall fn lang)))
+ port))))))))
+ ;; Tests for queries will fail given those languages are not
+ ;; packages yet.
+ (add-before 'check 'remove-unused-highlight-queries
+ (lambda _
+ (delete-file-recursively "queries/hcl")
+ (delete-file-recursively "queries/pgn")))
+ (add-after 'install 'install-queries
+ (lambda* (#:key outputs #:allow-other-keys)
+ (let ((elpa (elpa-directory (assoc-ref outputs "out"))))
+ (copy-recursively "queries" (string-append elpa "/queries"))))))))
+ (home-page "https://ubolonton.github.io/emacs-tree-sitter/languages/")
+ (synopsis "Language support bundle for Tree-sitter")
+ (description "This package is a convenient language bundle for
+Tree-sitter. For each supported language, this package provides:
+
+@enumerate
+@item Pre-compiled grammar binaries.
+@item An optional highlights.scm file that provides highlighting patterns.
+This is mainly intended for major modes that are not aware of tree-sitter.
+@item Optional query patterns for other minor modes that provide high-level
+functionalities on top of tree-sitter, such as code folding, evil text
+objects, ...etc.
+@end enumerate")
+ (license license:expat)))
--
2.36.0
P
P
Pierre Langlois wrote on 15 May 2022 20:38
[PATCH v6 21/27] gnu: Add tree-sitter-rust.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220515183834.18771-21-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-rust): New variable.
---
gnu/packages/tree-sitter.scm | 22 ++++++++++++++++++++++
1 file changed, 22 insertions(+)

Toggle diff (30 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index c7b1998253..c37183e131 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -573,3 +573,25 @@ (define-public tree-sitter-ruby
(description
"This package provides a Ruby grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-rust
+ (package
+ (name "tree-sitter-rust")
+ (version "0.20.1")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-rust")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "174j5pxwf80c4xniri39l3a6bb7nq96g2s8hh5sgv4i7xvbpfsmg"))
+ (modules '((guix build utils)))
+ (snippet tree-sitter-delete-generated-files)))
+ (build-system tree-sitter-build-system)
+ (home-page "https://github.com/tree-sitter/tree-sitter-rust")
+ (synopsis "Tree-sitter Rust grammar")
+ (description
+ "This package provides a Rust grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.36.0
P
P
Pierre Langlois wrote on 15 May 2022 20:38
[PATCH v6 24/27] gnu: Add rust-tree-sitter-for-emacs.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220515183834.18771-24-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (rust-tree-sitter-for-emacs): New variable.
* gnu/packages/patches/rust-tree-sitter-text-provider-fix.patch: New file.
* gnu/local.mk (dist_patch_DATA): Register it.
---
gnu/local.mk | 1 +
.../rust-tree-sitter-text-provider-fix.patch | 98 +++++++++++++++++++
gnu/packages/tree-sitter.scm | 14 +++
3 files changed, 113 insertions(+)
create mode 100644 gnu/packages/patches/rust-tree-sitter-text-provider-fix.patch

Toggle diff (148 lines)
diff --git a/gnu/local.mk b/gnu/local.mk
index 429a0b577b..7681eda58d 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -1787,6 +1787,7 @@ dist_patch_DATA = \
%D%/packages/patches/rust-nettle-sys-disable-vendor.patch \
%D%/packages/patches/rust-openssl-sys-no-vendor.patch \
%D%/packages/patches/rust-shell2batch-lint-fix.patch \
+ %D%/packages/patches/rust-tree-sitter-text-provider-fix.patch \
%D%/packages/patches/rust-wl-clipboard-rs-newer-wl.patch \
%D%/packages/patches/sbc-fix-build-non-x86.patch \
%D%/packages/patches/sbcl-aserve-add-HTML-5-elements.patch \
diff --git a/gnu/packages/patches/rust-tree-sitter-text-provider-fix.patch b/gnu/packages/patches/rust-tree-sitter-text-provider-fix.patch
new file mode 100644
index 0000000000..215e7fc18b
--- /dev/null
+++ b/gnu/packages/patches/rust-tree-sitter-text-provider-fix.patch
@@ -0,0 +1,98 @@
+From e3576b180488b8231e1fc0ca130748577579d129 Mon Sep 17 00:00:00 2001
+From: =?UTF-8?q?Tu=E1=BA=A5n-Anh=20Nguy=E1=BB=85n?= <ubolonton@gmail.com>
+Date: Sun, 25 Jul 2021 13:11:52 +0700
+Subject: [PATCH] Allow TextProvider's iterators to generate owned text
+
+---
+ binding_rust/lib.rs | 33 ++++++++++++++++++---------------
+ 1 file changed, 18 insertions(+), 15 deletions(-)
+
+diff --git a/lib/binding_rust/lib.rs b/lib/binding_rust/lib.rs
+index e88a411c..cf214d92 100644
+--- a/binding_rust/lib.rs
++++ b/binding_rust/lib.rs
+@@ -5,6 +5,7 @@ mod util;
+ use std::os::unix::io::AsRawFd;
+
+ use std::{
++ borrow::Cow,
+ char, error,
+ ffi::CStr,
+ fmt, hash, iter,
+@@ -183,7 +184,8 @@ pub struct QueryCaptures<'a, 'tree: 'a, T: TextProvider<'a>> {
+ }
+
+ pub trait TextProvider<'a> {
+- type I: Iterator<Item = &'a [u8]> + 'a;
++ type I: Iterator<Item = Cow<'a, [u8]>>;
++
+ fn text(&mut self, node: Node) -> Self::I;
+ }
+
+@@ -1840,19 +1842,19 @@ impl<'a, 'tree> QueryMatch<'a, 'tree> {
+ buffer2: &mut Vec<u8>,
+ text_provider: &mut impl TextProvider<'a>,
+ ) -> bool {
+- fn get_text<'a, 'b: 'a, I: Iterator<Item = &'b [u8]>>(
++ fn get_text<'a, 'b: 'a, I: Iterator<Item = Cow<'b, [u8]>>>(
+ buffer: &'a mut Vec<u8>,
+ mut chunks: I,
+- ) -> &'a [u8] {
+- let first_chunk = chunks.next().unwrap_or(&[]);
++ ) -> Cow<'a, [u8]> {
++ let first_chunk = chunks.next().unwrap_or(Cow::Owned(vec![0u8; 0]));
+ if let Some(next_chunk) = chunks.next() {
+ buffer.clear();
+- buffer.extend_from_slice(first_chunk);
+- buffer.extend_from_slice(next_chunk);
++ buffer.extend_from_slice(&first_chunk);
++ buffer.extend_from_slice(&next_chunk);
+ for chunk in chunks {
+- buffer.extend_from_slice(chunk);
++ buffer.extend_from_slice(&chunk);
+ }
+- buffer.as_slice()
++ Cow::Borrowed(buffer.as_slice())
+ } else {
+ first_chunk
+ }
+@@ -1888,7 +1890,7 @@ impl<'a, 'tree> QueryMatch<'a, 'tree> {
+ match node {
+ Some(node) => {
+ let text = get_text(buffer1, text_provider.text(node));
+- r.is_match(text) == *is_positive
++ r.is_match(&text) == *is_positive
+ }
+ None => true,
+ }
+@@ -2002,23 +2004,24 @@ impl<'cursor, 'tree> fmt::Debug for QueryMatch<'cursor, 'tree> {
+ }
+ }
+
+-impl<'a, F, I> TextProvider<'a> for F
++impl<'a, F, I, T> TextProvider<'a> for F
+ where
+ F: FnMut(Node) -> I,
+- I: Iterator<Item = &'a [u8]> + 'a,
++ T: Into<Cow<'a, [u8]>>,
++ I: Iterator<Item = T>,
+ {
+- type I = I;
++ type I = iter::Map<I, fn(T) -> Cow<'a, [u8]>>;
+
+ fn text(&mut self, node: Node) -> Self::I {
+- (self)(node)
++ (self)(node).map(T::into)
+ }
+ }
+
+ impl<'a> TextProvider<'a> for &'a [u8] {
+- type I = iter::Once<&'a [u8]>;
++ type I = iter::Once<Cow<'a, [u8]>>;
+
+ fn text(&mut self, node: Node) -> Self::I {
+- iter::once(&self[node.byte_range()])
++ iter::once(Cow::Borrowed(&self[node.byte_range()]))
+ }
+ }
+
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 58ecc445a6..4d1d3b2177 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -27,6 +27,7 @@ (define-module (gnu packages tree-sitter)
#:use-module (guix git-download)
#:use-module (guix packages)
#:use-module (guix utils)
+ #:use-module (gnu packages)
#:use-module (gnu packages crates-graphics)
#:use-module (gnu packages crates-io)
#:use-module (gnu packages icu4c))
@@ -196,6 +197,19 @@ (define-public rust-tree-sitter
parsing library.")
(license license:expat)))

+;; We need to apply a patch in order to compile the rust bindings against the
+;; emacs tree-sitter module.
+;; See https://github.com/tree-sitter/tree-sitter/pull/1294
+(define-public rust-tree-sitter-for-emacs
+ (package (inherit rust-tree-sitter)
+ (source (origin
+ (inherit (package-source rust-tree-sitter))
+ (patches (search-patches
+ "rust-tree-sitter-text-provider-fix.patch"))))
+ ;; Do not show this package in the UI as it's only meant to be used for
+ ;; emacs's tree-sitter module.
+ (properties '((hidden? . #t)))))
+
(define tree-sitter-delete-generated-files
'(begin
(delete-file "binding.gyp")
--
2.36.0
P
P
Pierre Langlois wrote on 15 May 2022 20:38
[PATCH v6 23/27] gnu: Add rust-tree-sitter.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20220515183834.18771-23-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (rust-tree-sitter): New variable.
---
gnu/packages/tree-sitter.scm | 39 ++++++++++++++++++++++++++++++++++++
1 file changed, 39 insertions(+)

Toggle diff (57 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 04747f47e5..58ecc445a6 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -23,6 +23,7 @@ (define-module (gnu packages tree-sitter)
#:use-module (guix build-system cargo)
#:use-module (guix build-system gnu)
#:use-module (guix build-system tree-sitter)
+ #:use-module (guix download)
#:use-module (guix git-download)
#:use-module (guix packages)
#:use-module (guix utils)
@@ -157,6 +158,44 @@ (define-public tree-sitter-cli
This package includes the @command{tree-sitter} command-line tool.")
(license license:expat)))

+(define-public rust-tree-sitter
+ (package
+ (name "rust-tree-sitter")
+ (version "0.20.6")
+ (source (origin
+ (method url-fetch)
+ (uri (crate-uri "tree-sitter" version))
+ (file-name (string-append name "-" version ".tar.gz"))
+ (sha256
+ (base32
+ "0wcvxgnvj7ga1y7xa7wm0pmabkfj8936ifg8jacd4201cj0vgcq9"))
+ (modules '((guix build utils)))
+ (snippet
+ '(begin
+ ;; Remove the runtime library code and dynamically link to
+ ;; it instead.
+ (delete-file-recursively "src")
+ (delete-file "binding_rust/build.rs")
+ (with-output-to-file "binding_rust/build.rs"
+ (lambda _
+ (format #t "fn main() {~@
+ println!(\"cargo:rustc-link-lib=tree-sitter\");~@
+ }~%")))
+ #t))))
+ (build-system cargo-build-system)
+ (inputs (list tree-sitter))
+ (arguments
+ `(#:cargo-inputs
+ (("rust-cc" ,rust-cc-1)
+ ("rust-lazy-static" ,rust-lazy-static-1)
+ ("rust-regex" ,rust-regex-1)
+ ("rust-spin" ,rust-spin-0.7))))
+ (home-page "https://tree-sitter.github.io/tree-sitter/")
+ (synopsis "Rust bindings to the Tree-sitter parsing library")
+ (description "This package provides Rust bindings to the Tree-sitter
+parsing library.")
+ (license license:expat)))
+
(define tree-sitter-delete-generated-files
'(begin
(delete-file "binding.gyp")
--
2.36.0
P
P
Pierre Langlois wrote on 15 May 2022 23:19
Re: [PATCH v6 03/27] build-system: Add tree-sitter-build-system.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
8735hav6ch.fsf@gmx.com
I'm not sure why, but when I look at either
https://mail.gnu.org/archive/html/guix-patches/or https://issues.guix.gnu.org/49946
this patch always seems to be missing. I wonder why, I used `git
send-email' and didn't see any errors.

I'm sending it as an attachment in case people are not receiving it.

Thanks,
Pierre
-----BEGIN PGP SIGNATURE-----

iQFMBAEBCgA2FiEEctU9gYy29KFyWDdMqPyeRH9PfVQFAmKBbx4YHHBpZXJyZS5s
YW5nbG9pc0BnbXguY29tAAoJEKj8nkR/T31UpD0H/R2UmjNB728+xRjunXOXfXuj
8c5dnvbRYALM0crpZIlvn2I8P1E/gtdeTYnnJ8rLfC3cAqQDnNhSFM4YtjiepXz4
QOvwhcsM+wsZboR7v4/aYjPK59O2h65iJFtD0YnWu7NPGWRIuhXVRCcXsNfkNMJQ
9MQADyFGYqhP/z7xASDVdfKuDyTW+k5iqlhhVxn8s8NrwU9D87C4SEfgpNOjf3/E
rx6mW0W5texRqXGu9FPFNTII6AkEOFRSKaumgNS99HqHO4KhJEi0ahY3Po6R+3L+
LS3di6Cz7tImSkN4f3WpbbsoHtYIDDMnYz3q58lG17XaGEkipumWDrShWjNm8EE=
=nOaF
-----END PGP SIGNATURE-----

P
P
Pierre Langlois wrote on 25 Nov 2022 02:21
[PATCH v7 02/32] gnu: tree-sitter: Update to 0.20.7.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20221125012142.22579-3-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter): Update to 0.20.7.
---
gnu/packages/tree-sitter.scm | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)

Toggle diff (24 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 7116a10ed7..8ef4e87e41 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -29,7 +29,7 @@ (define-module (gnu packages tree-sitter)
(define-public tree-sitter
(package
(name "tree-sitter")
- (version "0.20.6")
+ (version "0.20.7")
(source (origin
(method git-fetch)
(uri (git-reference
@@ -38,7 +38,7 @@ (define-public tree-sitter
(file-name (git-file-name name version))
(sha256
(base32
- "1z20518snyg0zp75qgs5bxmzjqws4dd19vnp6sya494za3qp5b6d"))
+ "1nv2a2hr22w8ix71b6rkkxv9rfvhvwlmyql0g6lva9qzj4vy50p4"))
(modules '((guix build utils)))
(snippet '(begin
;; Remove bundled ICU parts
--
2.38.1
P
P
Pierre Langlois wrote on 25 Nov 2022 02:21
[PATCH v7 00/32] gnu: Add tree-sitter for emacs.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20221125012142.22579-1-pierre.langlois@gmx.com
Hi Guix!

With the recent news of tree-sitter support making it into emacs, I thought
it'd be a good idea to make sure this series is rebased and up-to-date,
hopefully nobody's working on this while not being aware of it.

This does not use the new native emacs support for tree-sitter, so the emacs
packages it introduces could eventually become obsolete. However the grammars
and the build system should be re-usable by emacs natively too.

On top of rebasing, I also:

- Updated to 0.20.7.
- Re-introduced a couple of rust crates needed by tree-sitter-cli.
- Added support for R and Haskell.
- Updated some of the grammars to make the emacs-tree-sitter-langs tests
happy.

Thanks,
Pierre

Pierre Langlois (32):
gnu: tree-sitter: Move to its own module.
gnu: tree-sitter: Update to 0.20.7.
gnu: Add rust-html-escape.
gnu: Add rust-smallbitvec-2.
gnu: Add tree-sitter-cli.
build-system: Add tree-sitter-build-system.
gnu: Add tree-sitter-c.
gnu: Add tree-sitter-cpp.
gnu: Add tree-sitter-bash.
gnu: Add tree-sitter-css.
gnu: Add tree-sitter-c-sharp.
gnu: Add tree-sitter-elixir.
gnu: Add tree-sitter-elm.
gnu: Add tree-sitter-go.
gnu: Add tree-sitter-haskell.
gnu: Add tree-sitter-html.
gnu: Add tree-sitter-java.
gnu: Add tree-sitter-javascript.
gnu: Add tree-sitter-json.
gnu: Add tree-sitter-julia.
gnu: Add tree-sitter-ocaml.
gnu: Add tree-sitter-php.
gnu: Add tree-sitter-python.
gnu: Add tree-sitter-r.
gnu: Add tree-sitter-ruby.
gnu: Add tree-sitter-rust.
gnu: Add tree-sitter-typescript.
gnu: Add rust-tree-sitter.
gnu: Add rust-tree-sitter-for-emacs.
gnu: Add emacs-tree-sitter-core.
gnu: Add emacs-tree-sitter.
gnu: Add emacs-tree-sitter-langs.

Makefile.am | 2 +
doc/guix.texi | 21 +-
gnu/local.mk | 2 +
gnu/packages/crates-io.scm | 46 +
.../rust-tree-sitter-text-provider-fix.patch | 98 ++
gnu/packages/text-editors.scm | 52 -
gnu/packages/tree-sitter.scm | 1001 +++++++++++++++++
gnu/packages/vim.scm | 1 +
guix/build-system/tree-sitter.scm | 190 ++++
guix/build/tree-sitter-build-system.scm | 153 +++
10 files changed, 1513 insertions(+), 53 deletions(-)
create mode 100644 gnu/packages/patches/rust-tree-sitter-text-provider-fix.patch
create mode 100644 gnu/packages/tree-sitter.scm
create mode 100644 guix/build-system/tree-sitter.scm
create mode 100644 guix/build/tree-sitter-build-system.scm

--
2.38.1
P
P
Pierre Langlois wrote on 25 Nov 2022 02:21
[PATCH v7 01/32] gnu: tree-sitter: Move to its own module.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20221125012142.22579-2-pierre.langlois@gmx.com
* gnu/packages/text-editors.scm (tree-sitter): Move to ...
* gnu/packages/tree-sitter.scm: ... here, a new module.
* gnu/packages/vim.scm: Use (gnu packages tree-sitter).
* gnu/local.mk (GNU_SYSTEM_MODULES): Register tree-sitter.scm new module.
---
gnu/local.mk | 1 +
gnu/packages/text-editors.scm | 52 -----------------------
gnu/packages/tree-sitter.scm | 78 +++++++++++++++++++++++++++++++++++
gnu/packages/vim.scm | 1 +
4 files changed, 80 insertions(+), 52 deletions(-)
create mode 100644 gnu/packages/tree-sitter.scm

Toggle diff (180 lines)
diff --git a/gnu/local.mk b/gnu/local.mk
index 7278c50e4f..7d11f4bb27 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -603,6 +603,7 @@ GNU_SYSTEM_MODULES = \
%D%/packages/tmux.scm \
%D%/packages/toolkits.scm \
%D%/packages/tor.scm \
+ %D%/packages/tree-sitter.scm \
%D%/packages/tv.scm \
%D%/packages/uglifyjs.scm \
%D%/packages/uml.scm \
diff --git a/gnu/packages/text-editors.scm b/gnu/packages/text-editors.scm
index ed77113726..57178e7b87 100644
--- a/gnu/packages/text-editors.scm
+++ b/gnu/packages/text-editors.scm
@@ -70,7 +70,6 @@ (define-module (gnu packages text-editors)
#:use-module (gnu packages gtk)
#:use-module (gnu packages guile)
#:use-module (gnu packages haskell-xyz)
- #:use-module (gnu packages icu4c)
#:use-module (gnu packages image)
#:use-module (gnu packages libbsd)
#:use-module (gnu packages libreoffice)
@@ -1219,57 +1218,6 @@ (define-public edlin
FreeDOS as a functional clone of the old MS-DOS program edlin.")
(license license:gpl2+)))
-(define-public tree-sitter
- (package
- (name "tree-sitter")
- (version "0.20.6")
- (source (origin
- (method git-fetch)
- (uri (git-reference
- (url "https://github.com/tree-sitter/tree-sitter")
- (commit (string-append "v" version))))
- (file-name (git-file-name name version))
- (sha256
- (base32
- "1z20518snyg0zp75qgs5bxmzjqws4dd19vnp6sya494za3qp5b6d"))
- (modules '((guix build utils)))
- (snippet '(begin
- ;; Remove bundled ICU parts
- (delete-file-recursively "lib/src/unicode")
- #t))))
- (build-system gnu-build-system)
- (inputs (list icu4c))
- (arguments
- (list #:phases
- '(modify-phases %standard-phases
- (delete 'configure))
- #:tests? #f ; there are no tests for the runtime library
- #:make-flags
- #~(list (string-append "PREFIX="
- #$output)
- (string-append "CC="
- #$(cc-for-target)))))
- (home-page "https://tree-sitter.github.io/tree-sitter/")
- (synopsis "Incremental parsing system for programming tools")
- (description
- "Tree-sitter is a parser generator tool and an incremental parsing
-library. It can build a concrete syntax tree for a source file and efficiently
-update the syntax tree as the source file is edited.
-
-Tree-sitter aims to be:
-
-@itemize
-@item General enough to parse any programming language
-@item Fast enough to parse on every keystroke in a text editor
-@item Robust enough to provide useful results even in the presence of syntax errors
-@item Dependency-free so that the runtime library (which is written in pure C)
-can be embedded in any application
-@end itemize
-
-This package includes the @code{libtree-sitter} runtime library.
-")
- (license license:expat)))
-
(define-public mle
(package
(name "mle")
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
new file mode 100644
index 0000000000..7116a10ed7
--- /dev/null
+++ b/gnu/packages/tree-sitter.scm
@@ -0,0 +1,78 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2022 Luis Henrique Gomes Higino <luishenriquegh2701@gmail.com>
+;;; Copyright © 2022 Pierre Langlois <pierre.langlois@gmx.com>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu packages tree-sitter)
+ #:use-module (guix gexp)
+ #:use-module ((guix licenses) #:prefix license:)
+ #:use-module (guix build-system gnu)
+ #:use-module (guix git-download)
+ #:use-module (guix packages)
+ #:use-module (guix utils)
+ #:use-module (gnu packages icu4c))
+
+(define-public tree-sitter
+ (package
+ (name "tree-sitter")
+ (version "0.20.6")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "1z20518snyg0zp75qgs5bxmzjqws4dd19vnp6sya494za3qp5b6d"))
+ (modules '((guix build utils)))
+ (snippet '(begin
+ ;; Remove bundled ICU parts
+ (delete-file-recursively "lib/src/unicode")
+ #t))))
+ (build-system gnu-build-system)
+ (inputs (list icu4c))
+ (arguments
+ (list #:phases
+ '(modify-phases %standard-phases
+ (delete 'configure))
+ #:tests? #f ; there are no tests for the runtime library
+ #:make-flags
+ #~(list (string-append "PREFIX="
+ #$output)
+ (string-append "CC="
+ #$(cc-for-target)))))
+ (home-page "https://tree-sitter.github.io/tree-sitter/")
+ (synopsis "Incremental parsing system for programming tools")
+ (description
+ "Tree-sitter is a parser generator tool and an incremental parsing
+library. It can build a concrete syntax tree for a source file and efficiently
+update the syntax tree as the source file is edited.
+
+Tree-sitter aims to be:
+
+@itemize
+@item General enough to parse any programming language
+@item Fast enough to parse on every keystroke in a text editor
+@item Robust enough to provide useful results even in the presence of syntax errors
+@item Dependency-free so that the runtime library (which is written in pure C)
+can be embedded in any application
+@end itemize
+
+This package includes the @code{libtree-sitter} runtime library.
+")
+ (license license:expat)))
diff --git a/gnu/packages/vim.scm b/gnu/packages/vim.scm
index 957f00a92e..0c44c3c114 100644
--- a/gnu/packages/vim.scm
+++ b/gnu/packages/vim.scm
@@ -72,6 +72,7 @@ (define-module (gnu packages vim)
#:use-module (gnu packages tcl)
#:use-module (gnu packages text-editors)
#:use-module (gnu packages terminals)
+ #:use-module (gnu packages tree-sitter)
#:use-module (gnu packages xdisorg)
#:use-module (gnu packages xorg))
--
2.38.1
P
P
Pierre Langlois wrote on 25 Nov 2022 02:21
[PATCH v7 04/32] gnu: Add rust-smallbitvec-2.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20221125012142.22579-5-pierre.langlois@gmx.com
* gnu/packages/crates-io.scm (rust-smallbitvec-2): New variable.
---
gnu/packages/crates-io.scm | 22 ++++++++++++++++++++++
1 file changed, 22 insertions(+)

Toggle diff (35 lines)
diff --git a/gnu/packages/crates-io.scm b/gnu/packages/crates-io.scm
index cb3d83de5d..9ca52587eb 100644
--- a/gnu/packages/crates-io.scm
+++ b/gnu/packages/crates-io.scm
@@ -53163,6 +53163,28 @@ (define-public rust-sluice-0.5
I/O programming.")
(license license:expat)))
+(define-public rust-smallbitvec-2
+ (package
+ (name "rust-smallbitvec")
+ (version "2.5.1")
+ (source (origin
+ (method url-fetch)
+ (uri (crate-uri "smallbitvec" version))
+ (file-name (string-append name "-" version ".tar.gz"))
+ (sha256
+ (base32
+ "0plrbldsjpwip3afbzd8fgrnvdhizcg5z4ncfqs4q6x4qjflzkkm"))))
+ (build-system cargo-build-system)
+ (arguments
+ `(#:cargo-development-inputs
+ (("rust-bit-vec" ,rust-bit-vec-0.4)
+ ("rust-rand" ,rust-rand-0.4))))
+ (home-page "https://github.com/servo/smallbitvec")
+ (synopsis "Bit vector optimized for size and inline storage")
+ (description "This package provides a bit vector optimized for size and
+inline storage.")
+ (license (list license:expat license:asl2.0))))
+
(define-public rust-smallvec-1
(package
(name "rust-smallvec")
--
2.38.1
P
P
Pierre Langlois wrote on 25 Nov 2022 02:21
[PATCH v7 03/32] gnu: Add rust-html-escape.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20221125012142.22579-4-pierre.langlois@gmx.com
* gnu/packages/crates-io.scm (rust-html-escape): New variable.
---
gnu/packages/crates-io.scm | 24 ++++++++++++++++++++++++
1 file changed, 24 insertions(+)

Toggle diff (37 lines)
diff --git a/gnu/packages/crates-io.scm b/gnu/packages/crates-io.scm
index 792779a28f..cb3d83de5d 100644
--- a/gnu/packages/crates-io.scm
+++ b/gnu/packages/crates-io.scm
@@ -25913,6 +25913,30 @@ (define-public rust-htmlescape-0.3
"This package provides a library for HTML entity encoding and decoding.")
(license (list license:asl2.0 license:expat license:mpl2.0))))
+(define-public rust-html-escape-0.2
+ (package
+ (name "rust-html-escape")
+ (version "0.2.12")
+ (source (origin
+ (method url-fetch)
+ (uri (crate-uri "html-escape" version))
+ (file-name (string-append name "-" version ".tar.gz"))
+ (sha256
+ (base32
+ "01f2v3c6j2rk5h2lhdbh62j07cm1fvzqw4vplj2sms83jpx5qc8m"))))
+ (build-system cargo-build-system)
+ (arguments
+ `(#:cargo-inputs
+ (("rust-utf8-width" ,rust-utf8-width-0.1))
+ #:cargo-development-inputs
+ (("rust-bencher" ,rust-bencher-0.1))))
+ (home-page "https://magiclen.org/html-escape")
+ (synopsis "Library for encoding and escaping special characters in HTML")
+ (description
+ "This package provides a library for encoding and escaping special
+characters in HTML, decoding and unescaping HTML entities as well.")
+ (license license:expat)))
+
(define-public rust-hts-sys-2
(package
(name "rust-hts-sys")
--
2.38.1
P
P
Pierre Langlois wrote on 25 Nov 2022 02:21
[PATCH v7 07/32] gnu: Add tree-sitter-c.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20221125012142.22579-8-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-c): New variable.
---
gnu/packages/tree-sitter.scm | 33 +++++++++++++++++++++++++++++++++
1 file changed, 33 insertions(+)

Toggle diff (50 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 30bf17b79b..4f75b0251a 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -22,6 +22,7 @@ (define-module (gnu packages tree-sitter)
#:use-module ((guix licenses) #:prefix license:)
#:use-module (guix build-system cargo)
#:use-module (guix build-system gnu)
+ #:use-module (guix build-system tree-sitter)
#:use-module (guix git-download)
#:use-module (guix packages)
#:use-module (guix utils)
@@ -156,3 +157,35 @@ (define-public tree-sitter-cli
This package includes the @command{tree-sitter} command-line tool.")
(license license:expat)))
+
+(define tree-sitter-delete-generated-files
+ '(begin
+ (delete-file "binding.gyp")
+ (delete-file-recursively "bindings")
+ (delete-file "src/grammar.json")
+ (delete-file "src/node-types.json")
+ (delete-file "src/parser.c")
+ (delete-file-recursively "src/tree_sitter")
+ #t))
+
+(define-public tree-sitter-c
+ (package
+ (name "tree-sitter-c")
+ (version "0.20.2")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-c")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "1w03r4l773ki4iq2xxsc2pqxf3pjsbybq3xq4glmnsihgylibn8v"))
+ (modules '((guix build utils)))
+ (snippet tree-sitter-delete-generated-files)))
+ (build-system tree-sitter-build-system)
+ (home-page "https://github.com/tree-sitter/tree-sitter-c")
+ (synopsis "Tree-sitter C grammar")
+ (description
+ "This package provides a C grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.38.1
P
P
Pierre Langlois wrote on 25 Nov 2022 02:21
[PATCH v7 08/32] gnu: Add tree-sitter-cpp.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20221125012142.22579-9-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-cpp): New variable.
---
gnu/packages/tree-sitter.scm | 25 +++++++++++++++++++++++++
1 file changed, 25 insertions(+)

Toggle diff (35 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 4f75b0251a..63dc990475 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -189,3 +189,28 @@ (define-public tree-sitter-c
(description
"This package provides a C grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-cpp
+ (let ((commit "5ead1e26c6ab71919db0f1880c46a278a93bc5ea")
+ (revision "1"))
+ (package
+ (name "tree-sitter-cpp")
+ (version (git-version "0.20.0" revision commit))
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-cpp")
+ (commit commit)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "1572qhfw1jjkm1q6c110lnnj2n384a97fgn645c5q9ikciv8kac7"))
+ (modules '((guix build utils)))
+ (snippet tree-sitter-delete-generated-files)))
+ (build-system tree-sitter-build-system)
+ (inputs (list tree-sitter-c))
+ (home-page "https://github.com/tree-sitter/tree-sitter-cpp")
+ (synopsis "Tree-sitter C++ grammar")
+ (description
+ "This package provides a C++ grammar for the Tree-sitter library.")
+ (license license:expat))))
--
2.38.1
P
P
Pierre Langlois wrote on 25 Nov 2022 02:21
[PATCH v7 05/32] gnu: Add tree-sitter-cli.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20221125012142.22579-6-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-cli): New variable.
---
gnu/packages/tree-sitter.scm | 80 ++++++++++++++++++++++++++++++++++++
1 file changed, 80 insertions(+)

Toggle diff (101 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 8ef4e87e41..30bf17b79b 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -20,10 +20,13 @@
(define-module (gnu packages tree-sitter)
#:use-module (guix gexp)
#:use-module ((guix licenses) #:prefix license:)
+ #:use-module (guix build-system cargo)
#:use-module (guix build-system gnu)
#:use-module (guix git-download)
#:use-module (guix packages)
#:use-module (guix utils)
+ #:use-module (gnu packages crates-graphics)
+ #:use-module (gnu packages crates-io)
#:use-module (gnu packages icu4c))
(define-public tree-sitter
@@ -76,3 +79,80 @@ (define-public tree-sitter
This package includes the @code{libtree-sitter} runtime library.
")
(license license:expat)))
+
+(define-public tree-sitter-cli
+ (package (inherit tree-sitter)
+ (name "tree-sitter-cli")
+ (source (origin
+ (inherit (package-source tree-sitter))
+ (snippet
+ '(begin
+ ;; Remove the runtime library code and dynamically link to
+ ;; it instead.
+ (delete-file-recursively "lib/src")
+ (delete-file "lib/binding_rust/build.rs")
+ (with-output-to-file "lib/binding_rust/build.rs"
+ (lambda _
+ (format #t "fn main() {~@
+ println!(\"cargo:rustc-link-lib=tree-sitter\");~@
+ }~%")))
+ #t))))
+ (build-system cargo-build-system)
+ (inputs (list tree-sitter))
+ (arguments
+ `(;; Running test requires downloading fixtures, see the
+ ;; script/fetch-fixtures script.
+ #:tests? #f
+ ;; We're only packaging the CLI program so we do not need to install
+ ;; sources.
+ #:install-source? #f
+ #:cargo-inputs
+ (("rust-ansi-term" ,rust-ansi-term-0.12)
+ ("rust-anyhow" ,rust-anyhow-1)
+ ("rust-atty" ,rust-atty-0.2)
+ ("rust-clap" ,rust-clap-2)
+ ("rust-difference" ,rust-difference-2)
+ ("rust-dirs" ,rust-dirs-3)
+ ("rust-html-escape" ,rust-html-escape-0.2)
+ ("rust-libloading" ,rust-libloading-0.7)
+ ("rust-rand" ,rust-rand-0.8)
+ ("rust-rustc-hash" ,rust-rustc-hash-1)
+ ("rust-semver" ,rust-semver-1)
+ ("rust-smallbitvec" ,rust-smallbitvec-2)
+ ("rust-thiserror" ,rust-thiserror-1)
+ ("rust-tiny-http" ,rust-tiny-http-0.8)
+ ("rust-toml" ,rust-toml-0.5)
+ ("rust-walkdir" ,rust-walkdir-2)
+ ("rust-webbrowser" ,rust-webbrowser-0.5)
+ ("rust-which" ,rust-which-4))
+ #:cargo-development-inputs
+ (("rust-pretty-assertions" ,rust-pretty-assertions-0.7))
+ #:phases
+ (modify-phases %standard-phases
+ (add-after 'unpack 'delete-cargo.lock
+ (lambda _ (delete-file "Cargo.lock")))
+ (replace 'install
+ (lambda* (#:key outputs #:allow-other-keys)
+ (let* ((out (assoc-ref outputs "out"))
+ (bin (string-append out "/bin"))
+ (lib (string-append out "/lib")))
+ (mkdir-p bin)
+ (install-file "target/release/tree-sitter" bin)))))))
+ (synopsis "Incremental parsing system for programming tools")
+ (description "Tree-sitter is a parser generator tool and an incremental
+parsing library. It can build a concrete syntax tree for a source file and
+efficiently update the syntax tree as the source file is edited.
+
+Tree-sitter aims to be:
+
+@enumerate
+@item General enough to parse any programming language.
+@item Fast enough to parse on every keystroke in a text editor.
+@item Robust enough to provide useful results even in the presence of syntax
+errors.
+@item Dependency-free so that the runtime library (which is written in pure C)
+can be embedded in any application.
+@end enumerate
+
+This package includes the @command{tree-sitter} command-line tool.")
+ (license license:expat)))
--
2.38.1
P
P
Pierre Langlois wrote on 25 Nov 2022 02:21
[PATCH v7 10/32] gnu: Add tree-sitter-css.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20221125012142.22579-11-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-css): New variable.
---
gnu/packages/tree-sitter.scm | 22 ++++++++++++++++++++++
1 file changed, 22 insertions(+)

Toggle diff (32 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index f583dba24d..e3ec418419 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -236,3 +236,25 @@ (define-public tree-sitter-bash
(description
"This package provides a Bash grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-css
+ (package
+ (name "tree-sitter-css")
+ (version "0.19.0")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-css")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "014jrlgi7zfza9g38hsr4vlbi8964i5p7iglaih6qmzaiml7bja2"))
+ (modules '((guix build utils)))
+ (snippet tree-sitter-delete-generated-files)))
+ (build-system tree-sitter-build-system)
+ (home-page "https://github.com/tree-sitter/tree-sitter-css")
+ (synopsis "Tree-sitter CSS grammar")
+ (description
+ "This package provides a CSS grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.38.1
P
P
Pierre Langlois wrote on 25 Nov 2022 02:21
[PATCH v7 11/32] gnu: Add tree-sitter-c-sharp.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20221125012142.22579-12-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-c-sharp): New variable.
---
gnu/packages/tree-sitter.scm | 22 ++++++++++++++++++++++
1 file changed, 22 insertions(+)

Toggle diff (35 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index e3ec418419..0f65e50440 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -215,6 +215,28 @@ (define-public tree-sitter-cpp
"This package provides a C++ grammar for the Tree-sitter library.")
(license license:expat))))
+(define-public tree-sitter-c-sharp
+ (package
+ (name "tree-sitter-c-sharp")
+ (version "0.19.1")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-c-sharp")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "054fmpf47cwh59gbg00sc0nl237ba4rnxi73miz39yqzcs87055r"))
+ (modules '((guix build utils)))
+ (snippet tree-sitter-delete-generated-files)))
+ (build-system tree-sitter-build-system)
+ (home-page "https://github.com/tree-sitter/tree-sitter-c-sharp")
+ (synopsis "Tree-sitter C# grammar")
+ (description
+ "This package provides a C# grammar for the Tree-sitter library.")
+ (license license:expat)))
+
(define-public tree-sitter-bash
(package
(name "tree-sitter-bash")
--
2.38.1
P
P
Pierre Langlois wrote on 25 Nov 2022 02:21
[PATCH v7 09/32] gnu: Add tree-sitter-bash.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20221125012142.22579-10-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-bash): New variable.
---
gnu/packages/tree-sitter.scm | 22 ++++++++++++++++++++++
1 file changed, 22 insertions(+)

Toggle diff (32 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 63dc990475..f583dba24d 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -214,3 +214,25 @@ (define-public tree-sitter-cpp
(description
"This package provides a C++ grammar for the Tree-sitter library.")
(license license:expat))))
+
+(define-public tree-sitter-bash
+ (package
+ (name "tree-sitter-bash")
+ (version "0.19.0")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-bash")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "18c030bb65r50i6z37iy7jb9z9i8i36y7b08dbc9bchdifqsijs5"))
+ (modules '((guix build utils)))
+ (snippet tree-sitter-delete-generated-files)))
+ (build-system tree-sitter-build-system)
+ (home-page "https://github.com/tree-sitter/tree-sitter-bash")
+ (synopsis "Tree-sitter Bash grammar")
+ (description
+ "This package provides a Bash grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.38.1
P
P
Pierre Langlois wrote on 25 Nov 2022 02:21
[PATCH v7 12/32] gnu: Add tree-sitter-elixir.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20221125012142.22579-13-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-elixir): New variable.
---
gnu/packages/tree-sitter.scm | 26 ++++++++++++++++++++++++++
1 file changed, 26 insertions(+)

Toggle diff (36 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 0f65e50440..3ffb91a1a7 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -280,3 +280,29 @@ (define-public tree-sitter-css
(description
"This package provides a CSS grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-elixir
+ (let ((commit "de20391afe5cb03ef1e8a8e43167e7b58cc52869")
+ (revision "1"))
+ (package
+ (name "tree-sitter-elixir")
+ (version (git-version "0.19.0" revision commit))
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/elixir-lang/tree-sitter-elixir")
+ (commit commit)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "0zrkrwhw3g1vazsxcwrfd1fk4wvs9hdwmwp6073mfh370bz4140h"))
+ (modules '((guix build utils)))
+ (snippet tree-sitter-delete-generated-files)))
+ (build-system tree-sitter-build-system)
+ (home-page "https://elixir-lang.org/tree-sitter-elixir/")
+ (synopsis "Tree-sitter Elixir grammar")
+ (description
+ "This package provides a Elixir grammar for the Tree-sitter library.")
+ (license (list license:asl2.0
+ ;; Files in tests/corpus are under MIT license.
+ license:expat)))))
--
2.38.1
P
P
Pierre Langlois wrote on 25 Nov 2022 02:21
[PATCH v7 13/32] gnu: Add tree-sitter-elm.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20221125012142.22579-14-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-elm): New variable.
---
gnu/packages/tree-sitter.scm | 22 ++++++++++++++++++++++
1 file changed, 22 insertions(+)

Toggle diff (32 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 3ffb91a1a7..11548f526b 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -306,3 +306,25 @@ (define-public tree-sitter-elixir
(license (list license:asl2.0
;; Files in tests/corpus are under MIT license.
license:expat)))))
+
+(define-public tree-sitter-elm
+ (package
+ (name "tree-sitter-elm")
+ (version "5.6.3")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/elm-tooling/tree-sitter-elm")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "0b5jpj8bnil1ylisyc4w48j8a30dyf3zylhidj73mlrb8rf7xm2s"))
+ (modules '((guix build utils)))
+ (snippet tree-sitter-delete-generated-files)))
+ (build-system tree-sitter-build-system)
+ (home-page "https://elm-tooling.github.io/tree-sitter-elm/")
+ (synopsis "Tree-sitter Elm grammar")
+ (description
+ "This package provides an Elm grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.38.1
P
P
Pierre Langlois wrote on 25 Nov 2022 02:21
[PATCH v7 14/32] gnu: Add tree-sitter-go.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20221125012142.22579-15-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-go): New variable.
---
gnu/packages/tree-sitter.scm | 24 ++++++++++++++++++++++++
1 file changed, 24 insertions(+)

Toggle diff (34 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 11548f526b..61d6eba777 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -328,3 +328,27 @@ (define-public tree-sitter-elm
(description
"This package provides an Elm grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-go
+ (let ((commit "05900faa3cdb5d2d8c8bd5e77ee698487e0a8611")
+ (revision "1"))
+ (package
+ (name "tree-sitter-go")
+ (version (git-version "0.19.1" revision commit))
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-go")
+ (commit commit)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "1qymkdi4qcnj8ywmsanb6pdl9zd71cbm6kzl87zk241h7dhkkkvz"))
+ (modules '((guix build utils)))
+ (snippet tree-sitter-delete-generated-files)))
+ (build-system tree-sitter-build-system)
+ (home-page "https://github.com/tree-sitter/tree-sitter-go")
+ (synopsis "Tree-sitter Go grammar")
+ (description
+ "This package provides a Golang grammar for the Tree-sitter library.")
+ (license license:expat))))
--
2.38.1
P
P
Pierre Langlois wrote on 25 Nov 2022 02:21
[PATCH v7 15/32] gnu: Add tree-sitter-haskell.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20221125012142.22579-16-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-haskell): New variable.
---
gnu/packages/tree-sitter.scm | 24 ++++++++++++++++++++++++
1 file changed, 24 insertions(+)

Toggle diff (34 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 61d6eba777..668788e582 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -352,3 +352,27 @@ (define-public tree-sitter-go
(description
"This package provides a Golang grammar for the Tree-sitter library.")
(license license:expat))))
+
+(define-public tree-sitter-haskell
+ (let ((commit "e30bdfd53eb28c73f26a68b77d436fd2140af167")
+ (revision "1"))
+ (package
+ (name "tree-sitter-haskell")
+ (version (git-version "0.14.0" revision commit))
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-haskell")
+ (commit commit)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "1ylgs6lv1dyh3wxf756fii070r32hzbgddrfjbi2v369vmvg337p"))
+ (modules '((guix build utils)))
+ (snippet tree-sitter-delete-generated-files)))
+ (build-system tree-sitter-build-system)
+ (home-page "https://github.com/tree-sitter/tree-sitter-haskell")
+ (synopsis "Tree-sitter Haskell grammar")
+ (description
+ "This package provides a Haskell grammar for the Tree-sitter library.")
+ (license license:expat))))
--
2.38.1
P
P
Pierre Langlois wrote on 25 Nov 2022 02:21
[PATCH v7 16/32] gnu: Add tree-sitter-html.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20221125012142.22579-17-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-html): New variable.
---
gnu/packages/tree-sitter.scm | 22 ++++++++++++++++++++++
1 file changed, 22 insertions(+)

Toggle diff (32 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 668788e582..f8edddccc5 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -376,3 +376,25 @@ (define-public tree-sitter-haskell
(description
"This package provides a Haskell grammar for the Tree-sitter library.")
(license license:expat))))
+
+(define-public tree-sitter-html
+ (package
+ (name "tree-sitter-html")
+ (version "0.19.0")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-html")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "1hg7vbcy7bir6b8x11v0a4x0glvqnsqc3i2ixiarbxmycbgl3axy"))
+ (modules '((guix build utils)))
+ (snippet tree-sitter-delete-generated-files)))
+ (build-system tree-sitter-build-system)
+ (home-page "https://github.com/tree-sitter/tree-sitter-html")
+ (synopsis "Tree-sitter HTML grammar")
+ (description
+ "This package provides a HTML grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.38.1
P
P
Pierre Langlois wrote on 25 Nov 2022 02:21
[PATCH v7 18/32] gnu: Add tree-sitter-javascript.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20221125012142.22579-19-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-javascript): New variable.
---
gnu/packages/tree-sitter.scm | 23 +++++++++++++++++++++++
1 file changed, 23 insertions(+)

Toggle diff (33 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 1782fa8e19..99fdd2de2c 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -420,3 +420,26 @@ (define-public tree-sitter-java
(description
"This package provides a Java grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-javascript
+ (package
+ (name "tree-sitter-javascript")
+ (version "0.20.0")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-javascript")
+ (commit (string-append "rust-" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "175yrk382n2di0c2xn4gpv8y4n83x1lg4hqn04vabf0yqynlkq67"))
+ (modules '((guix build utils)))
+ (snippet tree-sitter-delete-generated-files)))
+ (build-system tree-sitter-build-system)
+ (home-page "https://github.com/tree-sitter/tree-sitter-javascript")
+ (synopsis "Tree-sitter Javascript grammar")
+ (description
+ "This package provides Javascript and JSX grammars for the Tree-sitter
+library.")
+ (license license:expat)))
--
2.38.1
P
P
Pierre Langlois wrote on 25 Nov 2022 02:21
[PATCH v7 17/32] gnu: Add tree-sitter-java.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20221125012142.22579-18-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-java): New variable.
---
gnu/packages/tree-sitter.scm | 22 ++++++++++++++++++++++
1 file changed, 22 insertions(+)

Toggle diff (32 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index f8edddccc5..1782fa8e19 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -398,3 +398,25 @@ (define-public tree-sitter-html
(description
"This package provides a HTML grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-java
+ (package
+ (name "tree-sitter-java")
+ (version "0.20.0")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-java")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "1i9zfgqibinz3rkx6yws1wk49iys32x901dki65qihbxcmcfh341"))
+ (modules '((guix build utils)))
+ (snippet tree-sitter-delete-generated-files)))
+ (build-system tree-sitter-build-system)
+ (home-page "https://github.com/tree-sitter/tree-sitter-java")
+ (synopsis "Tree-sitter Java grammar")
+ (description
+ "This package provides a Java grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.38.1
P
P
Pierre Langlois wrote on 25 Nov 2022 02:21
[PATCH v7 19/32] gnu: Add tree-sitter-json.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20221125012142.22579-20-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-json): New variable.
---
gnu/packages/tree-sitter.scm | 22 ++++++++++++++++++++++
1 file changed, 22 insertions(+)

Toggle diff (32 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 99fdd2de2c..df9c8f7f4c 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -443,3 +443,25 @@ (define-public tree-sitter-javascript
"This package provides Javascript and JSX grammars for the Tree-sitter
library.")
(license license:expat)))
+
+(define-public tree-sitter-json
+ (package
+ (name "tree-sitter-json")
+ (version "0.19.0")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-json")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "06pjh31bv9ja9hlnykk257a6zh8bsxg2fqa54al7qk1r4n9ksnff"))
+ (modules '((guix build utils)))
+ (snippet tree-sitter-delete-generated-files)))
+ (build-system tree-sitter-build-system)
+ (home-page "https://github.com/tree-sitter/tree-sitter-json")
+ (synopsis "Tree-sitter JSON grammar")
+ (description
+ "This package provides a JSON grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.38.1
P
P
Pierre Langlois wrote on 25 Nov 2022 02:21
[PATCH v7 21/32] gnu: Add tree-sitter-ocaml.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20221125012142.22579-22-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-ocaml): New variable.
---
gnu/packages/tree-sitter.scm | 39 ++++++++++++++++++++++++++++++++++++
1 file changed, 39 insertions(+)

Toggle diff (49 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 0f9f9097a6..244eb2e8ec 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -487,3 +487,42 @@ (define-public tree-sitter-julia
(description
"This package provides a Julia grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-ocaml
+ (let ((commit "0348562f385bc2bd67ecf181425e1afd6d454192")
+ (revision "1"))
+ (package
+ (name "tree-sitter-ocaml")
+ (version (git-version "0.19.0" revision commit))
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-ocaml")
+ (commit commit)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "0iqmwcz3c2ai4gyx4xli1rhn6hi6a0f60dn20f8jas9ham9dc2df"))
+ (modules '((guix build utils)))
+ (snippet
+ '(begin
+ (delete-file "binding.gyp")
+ (delete-file-recursively "bindings")
+ (for-each
+ (lambda (lang)
+ (with-directory-excursion lang
+ (delete-file "src/grammar.json")
+ (delete-file "src/node-types.json")
+ (delete-file "src/parser.c")
+ (delete-file-recursively "src/tree_sitter")))
+ '("ocaml" "interface"))
+ #t))))
+ (build-system tree-sitter-build-system)
+ (arguments
+ (list
+ #:grammar-directories '("ocaml" "interface")))
+ (home-page "https://github.com/tree-sitter/tree-sitter-ocaml")
+ (synopsis "Tree-sitter OCaml grammar")
+ (description
+ "This package provides a OCaml grammar for the Tree-sitter library.")
+ (license license:expat))))
--
2.38.1
P
P
Pierre Langlois wrote on 25 Nov 2022 02:21
[PATCH v7 25/32] gnu: Add tree-sitter-ruby.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20221125012142.22579-26-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-ruby): New variable.
---
gnu/packages/tree-sitter.scm | 22 ++++++++++++++++++++++
1 file changed, 22 insertions(+)

Toggle diff (32 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 7d00ee209f..19e5308818 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -604,3 +604,25 @@ (define-public tree-sitter-r
(description
"This package provides a R grammar for the Tree-sitter library.")
(license license:expat))))
+
+(define-public tree-sitter-ruby
+ (package
+ (name "tree-sitter-ruby")
+ (version "0.19.0")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-ruby")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "0m3h4928rbs300wcb6776h9r88hi32rybbhcaf6rdympl5nzi83v"))
+ (modules '((guix build utils)))
+ (snippet tree-sitter-delete-generated-files)))
+ (build-system tree-sitter-build-system)
+ (home-page "https://github.com/tree-sitter/tree-sitter-ruby")
+ (synopsis "Tree-sitter Ruby grammar")
+ (description
+ "This package provides a Ruby grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.38.1
P
P
Pierre Langlois wrote on 25 Nov 2022 02:21
[PATCH v7 28/32] gnu: Add rust-tree-sitter.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20221125012142.22579-29-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (rust-tree-sitter): New variable.
---
gnu/packages/tree-sitter.scm | 38 ++++++++++++++++++++++++++++++++++++
1 file changed, 38 insertions(+)

Toggle diff (58 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 7cf9a41105..ca92fccfd6 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -23,6 +23,7 @@ (define-module (gnu packages tree-sitter)
#:use-module (guix build-system cargo)
#:use-module (guix build-system gnu)
#:use-module (guix build-system tree-sitter)
+ #:use-module (guix download)
#:use-module (guix git-download)
#:use-module (guix packages)
#:use-module (guix utils)
@@ -158,6 +159,43 @@ (define-public tree-sitter-cli
This package includes the @command{tree-sitter} command-line tool.")
(license license:expat)))
+(define-public rust-tree-sitter
+ (package
+ (name "rust-tree-sitter")
+ (version "0.20.7")
+ (source (origin
+ (method url-fetch)
+ (uri (crate-uri "tree-sitter" version))
+ (file-name (string-append name "-" version ".tar.gz"))
+ (sha256
+ (base32
+ "183niy9malzr4dm81swcgl05xkwqamim6ck0gw5xb6k78nprz6jl"))
+ (modules '((guix build utils)))
+ (snippet
+ '(begin
+ ;; Remove the runtime library code and dynamically link to
+ ;; it instead.
+ (delete-file-recursively "src")
+ (delete-file "binding_rust/build.rs")
+ (with-output-to-file "binding_rust/build.rs"
+ (lambda _
+ (format #t "fn main() {~@
+ println!(\"cargo:rustc-link-lib=tree-sitter\");~@
+ }~%")))
+ #t))))
+ (build-system cargo-build-system)
+ (inputs (list tree-sitter))
+ (arguments
+ `(#:cargo-inputs
+ (("rust-cc" ,rust-cc-1)
+ ("rust-lazy-static" ,rust-lazy-static-1)
+ ("rust-regex" ,rust-regex-1))))
+ (home-page "https://tree-sitter.github.io/tree-sitter/")
+ (synopsis "Rust bindings to the Tree-sitter parsing library")
+ (description "This package provides Rust bindings to the Tree-sitter
+parsing library.")
+ (license license:expat)))
+
(define tree-sitter-delete-generated-files
'(begin
(delete-file "binding.gyp")
--
2.38.1
P
P
Pierre Langlois wrote on 25 Nov 2022 02:21
[PATCH v7 30/32] gnu: Add emacs-tree-sitter-core.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20221125012142.22579-31-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-emacs-module): New local variable.
(emacs-tree-sitter-core): New variable.
---
gnu/packages/tree-sitter.scm | 84 ++++++++++++++++++++++++++++++++++++
1 file changed, 84 insertions(+)

Toggle diff (101 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index d0214f3a12..d23fa2fbd8 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -21,6 +21,7 @@ (define-module (gnu packages tree-sitter)
#:use-module (guix gexp)
#:use-module ((guix licenses) #:prefix license:)
#:use-module (guix build-system cargo)
+ #:use-module (guix build-system emacs)
#:use-module (guix build-system gnu)
#:use-module (guix build-system tree-sitter)
#:use-module (guix download)
@@ -738,3 +739,86 @@ (define-public tree-sitter-typescript
"This package provides Typescript and TSX grammars for the Tree-sitter
library.")
(license license:expat)))
+
+;; Local package definition solely for building the native emacs module
+;; written in Rust.
+(define tree-sitter-emacs-module
+ (package
+ (name "tree-sitter-emacs-module")
+ (version "0.18.0")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/emacs-tree-sitter/elisp-tree-sitter")
+ (commit version)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "1sdvz827v436qijs6xafakkfw2d16bvp8frymd818rppjc7a9dif"))))
+ (build-system cargo-build-system)
+ (inputs
+ (list tree-sitter))
+ (arguments
+ `(#:cargo-inputs
+ (("rust-anyhow" ,rust-anyhow-1)
+ ("rust-emacs" ,rust-emacs-0.18)
+ ("rust-libloading" ,rust-libloading-0.7)
+ ("rust-once-cell" ,rust-once-cell-1)
+ ("rust-tree-sitter" ,rust-tree-sitter-for-emacs))
+ #:phases
+ (modify-phases %standard-phases
+ (add-after 'unpack 'chdir
+ (lambda _ (chdir "core")))
+ (add-after 'chdir 'delete-cargo.lock
+ (lambda _ (delete-file "Cargo.lock")))
+ (add-after 'delete-cargo.lock 'do-not-fetch-from-github
+ (lambda _
+ (substitute* "Cargo.toml"
+ (("\\[patch.*") "")
+ (("git = .*") ""))))
+ (replace 'install
+ (lambda* (#:key outputs #:allow-other-keys)
+ (let ((lib (string-append (assoc-ref outputs "out") "/lib")))
+ (mkdir-p lib)
+ (copy-file "target/release/libtsc_dyn.so"
+ (string-append lib "/tsc-dyn.so"))))))))
+ (home-page #f)
+ (synopsis #f)
+ (description #f)
+ (license license:expat)))
+
+(define-public emacs-tree-sitter-core
+ (package
+ (name "emacs-tree-sitter-core")
+ (version (package-version tree-sitter-emacs-module))
+ (source (package-source tree-sitter-emacs-module))
+ (build-system emacs-build-system)
+ (native-inputs
+ (list tree-sitter-emacs-module))
+ (arguments
+ `(#:phases
+ (modify-phases %standard-phases
+ (add-after 'unpack 'chdir
+ (lambda _ (chdir "core")))
+ (add-before 'install 'find-tsc-dyn
+ (lambda* (#:key inputs outputs #:allow-other-keys)
+ (let ((elpa (elpa-directory (assoc-ref outputs "out")))
+ (module (search-input-file inputs "/lib/tsc-dyn.so")))
+ ;; Install the tsc-dyn module in site-lisp and the current
+ ;; directory for test to pass.
+ (install-file module elpa)
+ (copy-file module "tsc-dyn.so")
+ ;; We replace the tsc-dyn-get.el file with an empty stub to
+ ;; prevent the code from downloading the module.
+ (call-with-output-file "tsc-dyn-get.el"
+ (lambda (port)
+ (for-each
+ (lambda (sexp) (write sexp port))
+ '((defun tsc-dyn-get-ensure (requested)
+ nil)
+ (provide 'tsc-dyn-get)))))))))))
+ (home-page "https://github.com/emacs-tree-sitter/elisp-tree-sitter")
+ (synopsis "Tree-sitter bindings for Emacs Lisp, core library")
+ (description "This package provides core APIs of the Emacs binding for
+Tree-sitter, an incremental parsing system.")
+ (license license:expat)))
--
2.38.1
P
P
Pierre Langlois wrote on 25 Nov 2022 02:21
[PATCH v7 31/32] gnu: Add emacs-tree-sitter.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20221125012142.22579-32-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (emacs-tree-sitter): New variable.
---
gnu/packages/tree-sitter.scm | 27 +++++++++++++++++++++++++++
1 file changed, 27 insertions(+)

Toggle diff (37 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index d23fa2fbd8..26b0206575 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -822,3 +822,30 @@ (module (search-input-file inputs "/lib/tsc-dyn.so")))
(description "This package provides core APIs of the Emacs binding for
Tree-sitter, an incremental parsing system.")
(license license:expat)))
+
+(define-public emacs-tree-sitter
+ (package
+ (name "emacs-tree-sitter")
+ (version (package-version emacs-tree-sitter-core))
+ (source (package-source emacs-tree-sitter-core))
+ (build-system emacs-build-system)
+ (propagated-inputs
+ (list emacs-tree-sitter-core))
+ (arguments
+ `(#:phases
+ (modify-phases %standard-phases
+ (add-after 'unpack 'chdir
+ (lambda _ (chdir "lisp"))))))
+ (home-page "https://github.com/emacs-tree-sitter/elisp-tree-sitter")
+ (synopsis "Tree-sitter bindings for Emacs Lisp")
+ (description "This package provides Emacs bindings for Tree-sitter, an
+incremental parsing library. It aims to be the foundation for a new breed of
+Emacs packages that understand code structurally. For example:
+
+@enumerate
+@item Faster, fine-grained code highlighting.
+@item More flexible code folding.
+@item Structural editing (like Paredit, or even better) for non-Lisp code.
+@item More informative indexing for imenu.
+@end enumerate")
+ (license license:expat)))
--
2.38.1
P
P
Pierre Langlois wrote on 25 Nov 2022 02:21
[PATCH v7 32/32] gnu: Add emacs-tree-sitter-langs.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20221125012142.22579-33-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (make-emacs-tree-sitter-langs-grammar-bundle):
New procedure.
(emacs-tree-sitter-langs): New variable.
---
gnu/packages/tree-sitter.scm | 150 +++++++++++++++++++++++++++++++++++
1 file changed, 150 insertions(+)

Toggle diff (167 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 26b0206575..02d515bdc4 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -24,6 +24,7 @@ (define-module (gnu packages tree-sitter)
#:use-module (guix build-system emacs)
#:use-module (guix build-system gnu)
#:use-module (guix build-system tree-sitter)
+ #:use-module (guix build-system trivial)
#:use-module (guix download)
#:use-module (guix git-download)
#:use-module (guix packages)
@@ -849,3 +850,152 @@ (define-public emacs-tree-sitter
@item More informative indexing for imenu.
@end enumerate")
(license license:expat)))
+
+(define emacs-tree-sitter-langs-grammar-bundle
+ (package
+ (name "emacs-tree-sitter-langs-grammar-bundle")
+ (source #f)
+ (version (package-version tree-sitter))
+ (build-system trivial-build-system)
+ (inputs
+ ;; FIXME: Support for some languages is still left to package.
+ (list tree-sitter-bash
+ tree-sitter-c
+ tree-sitter-c-sharp
+ tree-sitter-cpp
+ tree-sitter-css
+ tree-sitter-elixir
+ tree-sitter-elm
+ tree-sitter-go
+ tree-sitter-haskell
+ tree-sitter-html
+ tree-sitter-java
+ tree-sitter-javascript
+ tree-sitter-json
+ tree-sitter-julia
+ tree-sitter-ocaml
+ tree-sitter-php
+ tree-sitter-python
+ tree-sitter-r
+ tree-sitter-rust
+ tree-sitter-ruby
+ tree-sitter-typescript))
+ (arguments
+ (list #:builder
+ (with-imported-modules '((guix build union)
+ (guix build utils))
+ #~(begin
+ (use-modules (ice-9 match)
+ (guix build union)
+ (guix build utils))
+ (union-build
+ #$output
+ (filter directory-exists?
+ (map (match-lambda
+ ((name directory)
+ (string-append directory "/lib/tree-sitter")))
+ '#$(package-inputs this-package))))))))
+ (synopsis #f)
+ (description #f)
+ (home-page #f)
+ (license #f)))
+
+(define-public emacs-tree-sitter-langs
+ (package
+ (name "emacs-tree-sitter-langs")
+ (version "0.12.6")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/emacs-tree-sitter/tree-sitter-langs")
+ (commit version)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "1qqf5naf4a9zm0am1rn9yn2b2s6q3h9fapsl5d546vbsyvak5q40"))))
+ (build-system emacs-build-system)
+ (inputs
+ (list emacs-tree-sitter-langs-grammar-bundle))
+ (propagated-inputs
+ (list emacs-tree-sitter))
+ (arguments
+ (list
+ #:tests? #t
+ #:test-command ''("script/test")
+ #:phases
+ #~(modify-phases %standard-phases
+ (add-after 'unpack 'disable-downloader
+ (lambda _
+ (call-with-output-file "tree-sitter-langs-build.el"
+ (lambda (port)
+ (let ((on-load-message
+ (string-append
+ "tree-sitter-langs: Grammar bundle already installed "
+ "via Guix. Installing external grammars via this "
+ "function isn't supported, if a language you need is "
+ "missing please report a bug at bug-guix@gnu.org.")))
+ (format
+ port
+ ";;;###autoload
+ (defun tree-sitter-langs-install-grammars
+ (&optional skip-if-installed version os
+ keep-bundle)
+ (interactive)
+ (message \"~a\"))
+ (defconst tree-sitter-langs--queries-dir
+ (file-name-as-directory
+ (concat (file-name-directory (locate-library \"tree-sitter-langs.el\"))
+ \"queries\")))
+ (defun tree-sitter-langs--bin-dir () \"~a\")
+ (provide 'tree-sitter-langs-build)"
+ on-load-message
+ #$emacs-tree-sitter-langs-grammar-bundle))))))
+ (add-after 'unpack 'remove-cask
+ (lambda _
+ (substitute* "script/test"
+ (("cask") ""))))
+ (add-before 'check 'patch-tree-sitter-require-test
+ (lambda _
+ (use-modules (ice-9 regex))
+ ;; This test needs a git repositories with submodules for
+ ;; each languages in order to map all repositories. We patch
+ ;; the mapping function with one that invokes the tests for each
+ ;; packaged language.
+ (let ((supported-languages
+ (map (lambda (lib)
+ (match:substring
+ (string-match "(.*)\\.so$" (basename lib))
+ 1))
+ (find-files "bin" "\\.so$"))))
+ (substitute* "tree-sitter-langs-tests.el"
+ (("tree-sitter-langs--map-repos")
+ (call-with-output-string
+ (lambda (port)
+ (write `(lambda (fn)
+ (dolist (lang ',supported-languages)
+ (funcall fn lang)))
+ port))))))))
+ ;; Tests for queries will fail given those languages are not
+ ;; packages yet.
+ (add-before 'check 'remove-unused-highlight-queries
+ (lambda _
+ (delete-file-recursively "queries/hcl")
+ (delete-file-recursively "queries/pgn")))
+ (add-after 'install 'install-queries
+ (lambda* (#:key outputs #:allow-other-keys)
+ (let ((elpa (elpa-directory (assoc-ref outputs "out"))))
+ (copy-recursively "queries" (string-append elpa "/queries"))))))))
+ (home-page "https://ubolonton.github.io/emacs-tree-sitter/languages/")
+ (synopsis "Language support bundle for Tree-sitter")
+ (description "This package is a convenient language bundle for
+Tree-sitter. For each supported language, this package provides:
+
+@enumerate
+@item Pre-compiled grammar binaries.
+@item An optional highlights.scm file that provides highlighting patterns.
+This is mainly intended for major modes that are not aware of tree-sitter.
+@item Optional query patterns for other minor modes that provide high-level
+functionalities on top of tree-sitter, such as code folding, evil text
+objects, ...etc.
+@end enumerate")
+ (license license:expat)))
--
2.38.1
P
P
Pierre Langlois wrote on 25 Nov 2022 02:21
[PATCH v7 20/32] gnu: Add tree-sitter-julia.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20221125012142.22579-21-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-julia): New variable.
---
gnu/packages/tree-sitter.scm | 22 ++++++++++++++++++++++
1 file changed, 22 insertions(+)

Toggle diff (32 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index df9c8f7f4c..0f9f9097a6 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -465,3 +465,25 @@ (define-public tree-sitter-json
(description
"This package provides a JSON grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-julia
+ (package
+ (name "tree-sitter-julia")
+ (version "0.19.0")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-julia")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "1pbnmvhy2gq4vg1b0sjzmjm4s2gsgdjh7h01yj8qrrqbcl29c463"))
+ (modules '((guix build utils)))
+ (snippet tree-sitter-delete-generated-files)))
+ (build-system tree-sitter-build-system)
+ (home-page "https://github.com/tree-sitter/tree-sitter-julia")
+ (synopsis "Tree-sitter Julia grammar")
+ (description
+ "This package provides a Julia grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.38.1
P
P
Pierre Langlois wrote on 25 Nov 2022 02:21
[PATCH v7 22/32] gnu: Add tree-sitter-php.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20221125012142.22579-23-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-php): New variable.
---
gnu/packages/tree-sitter.scm | 30 ++++++++++++++++++++++++++++++
1 file changed, 30 insertions(+)

Toggle diff (40 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 244eb2e8ec..0a1bd4739a 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -526,3 +526,33 @@ (define-public tree-sitter-ocaml
(description
"This package provides a OCaml grammar for the Tree-sitter library.")
(license license:expat))))
+
+(define-public tree-sitter-php
+ (let ((commit "435fa00006c0d1515c37fbb4dd6a9de284af75ab")
+ (revision "1"))
+ (package
+ (name "tree-sitter-php")
+ (version (git-version "0.19.0" revision commit))
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-php")
+ (commit commit)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "05k4h58gi616gv41r0qqdb1x4rs8y94vghn2r10yczisgzq4vbad"))
+ (modules '((guix build utils)))
+ (snippet
+ '(begin
+ (delete-file "src/grammar.json")
+ (delete-file "src/node-types.json")
+ (delete-file "src/parser.c")
+ (delete-file-recursively "src/tree_sitter")
+ #t))))
+ (build-system tree-sitter-build-system)
+ (home-page "https://github.com/tree-sitter/tree-sitter-php")
+ (synopsis "Tree-sitter PHP grammar")
+ (description
+ "This package provides a PHP grammar for the Tree-sitter library.")
+ (license license:expat))))
--
2.38.1
P
P
Pierre Langlois wrote on 25 Nov 2022 02:21
[PATCH v7 24/32] gnu: Add tree-sitter-r.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20221125012142.22579-25-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-r): New variable.
---
gnu/packages/tree-sitter.scm | 24 ++++++++++++++++++++++++
1 file changed, 24 insertions(+)

Toggle diff (34 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index c6256cd427..7d00ee209f 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -580,3 +580,27 @@ (define-public tree-sitter-python
(description
"This package provides a Python grammar for the Tree-sitter library.")
(license license:expat))))
+
+(define-public tree-sitter-r
+ (let ((commit "80efda55672d1293aa738f956c7ae384ecdc31b4")
+ (revision "1"))
+ (package
+ (name "tree-sitter-r")
+ (version (git-version "0.0.1" revision commit))
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/r-lib/tree-sitter-r")
+ (commit commit)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "1n7yxi2wf9xj8snw0b85a5w40vhf7x1pwirnwfk78ilr6hhz4ix9"))
+ (modules '((guix build utils)))
+ (snippet tree-sitter-delete-generated-files)))
+ (build-system tree-sitter-build-system)
+ (home-page "https://github.com/r-lib/tree-sitter-r")
+ (synopsis "Tree-sitter R grammar")
+ (description
+ "This package provides a R grammar for the Tree-sitter library.")
+ (license license:expat))))
--
2.38.1
P
P
Pierre Langlois wrote on 25 Nov 2022 02:21
[PATCH v7 23/32] gnu: Add tree-sitter-python.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20221125012142.22579-24-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-python): New variable.
---
gnu/packages/tree-sitter.scm | 24 ++++++++++++++++++++++++
1 file changed, 24 insertions(+)

Toggle diff (34 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 0a1bd4739a..c6256cd427 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -556,3 +556,27 @@ (define-public tree-sitter-php
(description
"This package provides a PHP grammar for the Tree-sitter library.")
(license license:expat))))
+
+(define-public tree-sitter-python
+ (let ((commit "ed0fe62e55dc617ed9dec8817ebf771aa7cf3c42")
+ (revision "1"))
+ (package
+ (name "tree-sitter-python")
+ (version (git-version "0.19.1" revision commit))
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-python")
+ (commit commit)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "0wrfpg84mc3pzcrdi6n5fqwijkqr1nj5sqfnayb502krvqpjilal"))
+ (modules '((guix build utils)))
+ (snippet tree-sitter-delete-generated-files)))
+ (build-system tree-sitter-build-system)
+ (home-page "https://github.com/tree-sitter/tree-sitter-python")
+ (synopsis "Tree-sitter Python grammar")
+ (description
+ "This package provides a Python grammar for the Tree-sitter library.")
+ (license license:expat))))
--
2.38.1
P
P
Pierre Langlois wrote on 25 Nov 2022 02:21
[PATCH v7 26/32] gnu: Add tree-sitter-rust.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20221125012142.22579-27-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-rust): New variable.
---
gnu/packages/tree-sitter.scm | 22 ++++++++++++++++++++++
1 file changed, 22 insertions(+)

Toggle diff (32 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 19e5308818..54f2affff2 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -626,3 +626,25 @@ (define-public tree-sitter-ruby
(description
"This package provides a Ruby grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-rust
+ (package
+ (name "tree-sitter-rust")
+ (version "0.20.1")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-rust")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "174j5pxwf80c4xniri39l3a6bb7nq96g2s8hh5sgv4i7xvbpfsmg"))
+ (modules '((guix build utils)))
+ (snippet tree-sitter-delete-generated-files)))
+ (build-system tree-sitter-build-system)
+ (home-page "https://github.com/tree-sitter/tree-sitter-rust")
+ (synopsis "Tree-sitter Rust grammar")
+ (description
+ "This package provides a Rust grammar for the Tree-sitter library.")
+ (license license:expat)))
--
2.38.1
P
P
Pierre Langlois wrote on 25 Nov 2022 02:21
[PATCH v7 27/32] gnu: Add tree-sitter-typescript.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20221125012142.22579-28-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-typescript): New variable.
---
gnu/packages/tree-sitter.scm | 38 ++++++++++++++++++++++++++++++++++++
1 file changed, 38 insertions(+)

Toggle diff (48 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 54f2affff2..7cf9a41105 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -648,3 +648,41 @@ (define-public tree-sitter-rust
(description
"This package provides a Rust grammar for the Tree-sitter library.")
(license license:expat)))
+
+(define-public tree-sitter-typescript
+ (package
+ (name "tree-sitter-typescript")
+ (version "0.20.1")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/tree-sitter/tree-sitter-typescript")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32 "07fl9d968lal0aqj4f0n16p3n94cjkgfp54wynfr8gbdkjss5v5x"))
+ (modules '((guix build utils)))
+ (snippet
+ '(begin
+ (delete-file "binding.gyp")
+ (delete-file-recursively "bindings")
+ (for-each
+ (lambda (lang)
+ (with-directory-excursion lang
+ (delete-file "src/grammar.json")
+ (delete-file "src/node-types.json")
+ (delete-file "src/parser.c")
+ (delete-file-recursively "src/tree_sitter")))
+ '("typescript" "tsx"))
+ #t))))
+ (build-system tree-sitter-build-system)
+ (inputs (list tree-sitter-javascript))
+ (arguments
+ (list
+ #:grammar-directories '("typescript" "tsx")))
+ (home-page "https://github.com/tree-sitter/tree-sitter-typescript")
+ (synopsis "Tree-sitter Typescript grammar")
+ (description
+ "This package provides Typescript and TSX grammars for the Tree-sitter
+library.")
+ (license license:expat)))
--
2.38.1
P
P
Pierre Langlois wrote on 25 Nov 2022 02:21
[PATCH v7 29/32] gnu: Add rust-tree-sitter-for-emacs.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20221125012142.22579-30-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (rust-tree-sitter-for-emacs): New variable.
* gnu/packages/patches/rust-tree-sitter-text-provider-fix.patch: New file.
* gnu/local.mk (dist_patch_DATA): Register it.
---
gnu/local.mk | 1 +
.../rust-tree-sitter-text-provider-fix.patch | 98 +++++++++++++++++++
gnu/packages/tree-sitter.scm | 14 +++
3 files changed, 113 insertions(+)
create mode 100644 gnu/packages/patches/rust-tree-sitter-text-provider-fix.patch

Toggle diff (150 lines)
diff --git a/gnu/local.mk b/gnu/local.mk
index 7d11f4bb27..961d90bee9 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -1832,6 +1832,7 @@ dist_patch_DATA = \
%D%/packages/patches/rust-nettle-disable-vendor.patch \
%D%/packages/patches/rust-nettle-sys-disable-vendor.patch \
%D%/packages/patches/rust-openssl-sys-no-vendor.patch \
+ %D%/packages/patches/rust-tree-sitter-text-provider-fix.patch \
%D%/packages/patches/rust-wl-clipboard-rs-newer-wl.patch \
%D%/packages/patches/sbc-fix-build-non-x86.patch \
%D%/packages/patches/sbcl-aserve-add-HTML-5-elements.patch \
diff --git a/gnu/packages/patches/rust-tree-sitter-text-provider-fix.patch b/gnu/packages/patches/rust-tree-sitter-text-provider-fix.patch
new file mode 100644
index 0000000000..215e7fc18b
--- /dev/null
+++ b/gnu/packages/patches/rust-tree-sitter-text-provider-fix.patch
@@ -0,0 +1,98 @@
+From e3576b180488b8231e1fc0ca130748577579d129 Mon Sep 17 00:00:00 2001
+From: =?UTF-8?q?Tu=E1=BA=A5n-Anh=20Nguy=E1=BB=85n?= <ubolonton@gmail.com>
+Date: Sun, 25 Jul 2021 13:11:52 +0700
+Subject: [PATCH] Allow TextProvider's iterators to generate owned text
+
+---
+ binding_rust/lib.rs | 33 ++++++++++++++++++---------------
+ 1 file changed, 18 insertions(+), 15 deletions(-)
+
+diff --git a/lib/binding_rust/lib.rs b/lib/binding_rust/lib.rs
+index e88a411c..cf214d92 100644
+--- a/binding_rust/lib.rs
++++ b/binding_rust/lib.rs
+@@ -5,6 +5,7 @@ mod util;
+ use std::os::unix::io::AsRawFd;
+
+ use std::{
++ borrow::Cow,
+ char, error,
+ ffi::CStr,
+ fmt, hash, iter,
+@@ -183,7 +184,8 @@ pub struct QueryCaptures<'a, 'tree: 'a, T: TextProvider<'a>> {
+ }
+
+ pub trait TextProvider<'a> {
+- type I: Iterator<Item = &'a [u8]> + 'a;
++ type I: Iterator<Item = Cow<'a, [u8]>>;
++
+ fn text(&mut self, node: Node) -> Self::I;
+ }
+
+@@ -1840,19 +1842,19 @@ impl<'a, 'tree> QueryMatch<'a, 'tree> {
+ buffer2: &mut Vec<u8>,
+ text_provider: &mut impl TextProvider<'a>,
+ ) -> bool {
+- fn get_text<'a, 'b: 'a, I: Iterator<Item = &'b [u8]>>(
++ fn get_text<'a, 'b: 'a, I: Iterator<Item = Cow<'b, [u8]>>>(
+ buffer: &'a mut Vec<u8>,
+ mut chunks: I,
+- ) -> &'a [u8] {
+- let first_chunk = chunks.next().unwrap_or(&[]);
++ ) -> Cow<'a, [u8]> {
++ let first_chunk = chunks.next().unwrap_or(Cow::Owned(vec![0u8; 0]));
+ if let Some(next_chunk) = chunks.next() {
+ buffer.clear();
+- buffer.extend_from_slice(first_chunk);
+- buffer.extend_from_slice(next_chunk);
++ buffer.extend_from_slice(&first_chunk);
++ buffer.extend_from_slice(&next_chunk);
+ for chunk in chunks {
+- buffer.extend_from_slice(chunk);
++ buffer.extend_from_slice(&chunk);
+ }
+- buffer.as_slice()
++ Cow::Borrowed(buffer.as_slice())
+ } else {
+ first_chunk
+ }
+@@ -1888,7 +1890,7 @@ impl<'a, 'tree> QueryMatch<'a, 'tree> {
+ match node {
+ Some(node) => {
+ let text = get_text(buffer1, text_provider.text(node));
+- r.is_match(text) == *is_positive
++ r.is_match(&text) == *is_positive
+ }
+ None => true,
+ }
+@@ -2002,23 +2004,24 @@ impl<'cursor, 'tree> fmt::Debug for QueryMatch<'cursor, 'tree> {
+ }
+ }
+
+-impl<'a, F, I> TextProvider<'a> for F
++impl<'a, F, I, T> TextProvider<'a> for F
+ where
+ F: FnMut(Node) -> I,
+- I: Iterator<Item = &'a [u8]> + 'a,
++ T: Into<Cow<'a, [u8]>>,
++ I: Iterator<Item = T>,
+ {
+- type I = I;
++ type I = iter::Map<I, fn(T) -> Cow<'a, [u8]>>;
+
+ fn text(&mut self, node: Node) -> Self::I {
+- (self)(node)
++ (self)(node).map(T::into)
+ }
+ }
+
+ impl<'a> TextProvider<'a> for &'a [u8] {
+- type I = iter::Once<&'a [u8]>;
++ type I = iter::Once<Cow<'a, [u8]>>;
+
+ fn text(&mut self, node: Node) -> Self::I {
+- iter::once(&self[node.byte_range()])
++ iter::once(Cow::Borrowed(&self[node.byte_range()]))
+ }
+ }
+
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index ca92fccfd6..d0214f3a12 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -27,6 +27,7 @@ (define-module (gnu packages tree-sitter)
#:use-module (guix git-download)
#:use-module (guix packages)
#:use-module (guix utils)
+ #:use-module (gnu packages)
#:use-module (gnu packages crates-graphics)
#:use-module (gnu packages crates-io)
#:use-module (gnu packages icu4c))
@@ -196,6 +197,19 @@ (define-public rust-tree-sitter
parsing library.")
(license license:expat)))
+;; We need to apply a patch in order to compile the rust bindings against the
+;; emacs tree-sitter module.
+;; See https://github.com/tree-sitter/tree-sitter/pull/1294
+(define-public rust-tree-sitter-for-emacs
+ (package (inherit rust-tree-sitter)
+ (source (origin
+ (inherit (package-source rust-tree-sitter))
+ (patches (search-patches
+ "rust-tree-sitter-text-provider-fix.patch"))))
+ ;; Do not show this package in the UI as it's only meant to be used for
+ ;; emacs's tree-sitter module.
+ (properties '((hidden? . #t)))))
+
(define tree-sitter-delete-generated-files
'(begin
(delete-file "binding.gyp")
--
2.38.1
(
Re: [bug#49946] [PATCH v7 01/32] gnu: tree-sitter: Move to its own module.
COL63T6V2GW7.38CF8YKKBV1FM@guix-framework
Heya,

On Fri Nov 25, 2022 at 1:21 AM GMT, Pierre Langlois wrote:
Toggle quote (19 lines)
> * gnu/packages/text-editors.scm (tree-sitter): Move to ...
> * gnu/packages/tree-sitter.scm: ... here, a new module.
> * gnu/packages/vim.scm: Use (gnu packages tree-sitter).
> * gnu/local.mk (GNU_SYSTEM_MODULES): Register tree-sitter.scm new module.

> --- /dev/null
> +++ b/gnu/packages/tree-sitter.scm

> @@ -0,0 +1,78 @@

> +(define-module (gnu packages tree-sitter)
> + #:use-module (guix gexp)
> + #:use-module ((guix licenses) #:prefix license:)
> + #:use-module (guix build-system gnu)
> + #:use-module (guix git-download)
> + #:use-module (guix packages)
> + #:use-module (guix utils)
> + #:use-module (gnu packages icu4c))

Please sort these alphabetically.

Toggle quote (16 lines)
> +(define-public tree-sitter
> + (package
> + (name "tree-sitter")
> + (version "0.20.6")
> + (source (origin
> + (method git-fetch)
> + (uri (git-reference
> + (url "https://github.com/tree-sitter/tree-sitter")
> + (commit (string-append "v" version))))
> + (file-name (git-file-name name version))
> + (sha256
> + (base32
> + "1z20518snyg0zp75qgs5bxmzjqws4dd19vnp6sya494za3qp5b6d"))
> + (modules '((guix build utils)))
> + (snippet '(begin

Please change it to use a gexp: ``(snippet #~(begin ...))''.

Toggle quote (4 lines)
> + ;; Remove bundled ICU parts
> + (delete-file-recursively "lib/src/unicode")
> + #t))))

This #T is now unnecessary.

Toggle quote (5 lines)
> + (arguments
> + (list #:phases
> + '(modify-phases %standard-phases
> + (delete 'configure))

Please use a gexp here, too.

Toggle quote (6 lines)
> + #:make-flags
> + #~(list (string-append "PREFIX="
> + #$output)
> + (string-append "CC="
> + #$(cc-for-target)))))

Maybe modify the indentation here a little.

(string-append "PREFIX=" #$output)
(string-append "CC=" #$(cc-for-target))

Toggle quote (4 lines)
> +
> +This package includes the @code{libtree-sitter} runtime library.
> +")

Please remove the newline at the end of the description.

-- (
(
Re: [bug#49946] [PATCH v7 02/32] gnu: tree-sitter: Update to 0.20.7.
COL65R4MPFQL.1GKVBNB2W5KWQ@guix-framework
On Fri Nov 25, 2022 at 1:21 AM GMT, Pierre Langlois wrote:
Toggle quote (2 lines)
> * gnu/packages/tree-sitter.scm (tree-sitter): Update to 0.20.7.

LGTM, but maybe you could use this commit to make the stylistic
changes I suggested in the last reply.

-- (
(
Re: [bug#49946] [PATCH v7 03/32] gnu: Add rust-html-escape.
COL6ASRYMCZ0.OXMK8SI1PLMN@guix-framework
On Fri Nov 25, 2022 at 1:21 AM GMT, Pierre Langlois wrote:
Toggle quote (13 lines)
> * gnu/packages/crates-io.scm (rust-html-escape): New variable.

> --- a/gnu/packages/crates-io.scm
> +++ b/gnu/packages/crates-io.scm

> @@ -25913,6 +25913,30 @@ (define-public rust-htmlescape-0.3

> + (arguments
> + `(#:cargo-inputs
> + (("rust-utf8-width" ,rust-utf8-width-0.1))
> + #:cargo-development-inputs
> + (("rust-bencher" ,rust-bencher-0.1))))

Please use LIST here:

(arguments
(list #:cargo-inputs
`(("rust-utf8-width" ,rust-utf8-width-0.1))
#:cargo-development-inputs
`(("rust-bencher" ,rust-bencher-0.1))))
Toggle quote (4 lines)
> + (description
> + "This package provides a library for encoding and escaping special
> +characters in HTML, decoding and unescaping HTML entities as well.")

This isn't quite a full, grammatical sentence (I guess it was lifted from
the crates.io page?) How about this?

(description
"This package provides a library for encoding special characters
as HTML entities and vice versa.")

-- (
(
Re: [bug#49946] [PATCH v7 04/32] gnu: Add rust-smallbitvec-2.
COL6B8A04EFW.I934W877NW7Q@guix-framework
On Fri Nov 25, 2022 at 1:21 AM GMT, Pierre Langlois wrote:
Toggle quote (11 lines)
> * gnu/packages/crates-io.scm (rust-smallbitvec-2): New variable.

> --- a/gnu/packages/crates-io.scm
> +++ b/gnu/packages/crates-io.scm

> @@ -53163,6 +53163,28 @@ (define-public rust-sluice-0.5

> + `(#:cargo-development-inputs
> + (("rust-bit-vec" ,rust-bit-vec-0.4)
> + ("rust-rand" ,rust-rand-0.4))))

LIST here please.

-- (
(
Re: [bug#49946] [PATCH v7 05/32] gnu: Add tree-sitter-cli.
COL6FVZHRLXS.3U96J8NTOCNUF@guix-framework
On Fri Nov 25, 2022 at 1:21 AM GMT, Pierre Langlois wrote:
Toggle quote (9 lines)
> * gnu/packages/tree-sitter.scm (tree-sitter-cli): New variable.

> --- a/gnu/packages/tree-sitter.scm
> +++ b/gnu/packages/tree-sitter.scm

> @@ -76,3 +79,80 @@ (define-public tree-sitter
> + (snippet
> + '(begin

Please use a gexp here.

Toggle quote (7 lines)
> + (with-output-to-file "lib/binding_rust/build.rs"
> + (lambda _
> + (format #t "fn main() {~@
> + println!(\"cargo:rustc-link-lib=tree-sitter\");~@
> + }~%")))
> + #t))))

The #T is redundant.

Toggle quote (3 lines)
> + (arguments
> + `(;; Running test requires downloading fixtures, see the

Please use LIST.

Toggle quote (3 lines)
> + ;; script/fetch-fixtures script.
> + #:tests? #f

Couldn't we fetch the fixtures ourselves with a secondary origin,

Toggle quote (12 lines)
> + #:phases
> + (modify-phases %standard-phases
> + (add-after 'unpack 'delete-cargo.lock
> + (lambda _ (delete-file "Cargo.lock")))
> + (replace 'install
> + (lambda* (#:key outputs #:allow-other-keys)
> + (let* ((out (assoc-ref outputs "out"))
> + (bin (string-append out "/bin"))
> + (lib (string-append out "/lib")))
> + (mkdir-p bin)
> + (install-file "target/release/tree-sitter" bin)))))))

Update this to modern Guix style, please:

#~(modify-phases %standard-phases
(add-after 'unpack 'delete-cargo-lock
(lambda _
(delete-file "Cargo.lock")))
(replace 'install
(lambda* (#:key outputs #:allow-other-keys)
(let ((bin (string-append #$output "/bin")))
(mkdir-p bin)
(install-file "target/release/tree-sitter" bin)))))

-- (
(
Re: [bug#49946] [PATCH v7 07/32] gnu: Add tree-sitter-c.
COL6ILV9JKI8.1W8VPFQS1VK68@guix-framework
On Fri Nov 25, 2022 at 1:21 AM GMT, Pierre Langlois wrote:
Toggle quote (20 lines)
> * gnu/packages/tree-sitter.scm (tree-sitter-c): New variable.

> --- a/gnu/packages/tree-sitter.scm
> +++ b/gnu/packages/tree-sitter.scm

> @@ -156,3 +157,35 @@ (define-public tree-sitter-cli

> This package includes the @command{tree-sitter} command-line tool.")
> (license license:expat)))
> +
> +(define tree-sitter-delete-generated-files
> + '(begin
> + (delete-file "binding.gyp")
> + (delete-file-recursively "bindings")
> + (delete-file "src/grammar.json")
> + (delete-file "src/node-types.json")
> + (delete-file "src/parser.c")
> + (delete-file-recursively "src/tree_sitter")
> + #t))

No need to outline this; just do

(snippet
#~(begin
(for-each delete-file
(list "binding.gyp"
"src/grammar.json"
"src/node-types.json"
"src/parser.c"))
(for-each delete-file-recursively
(list "bindings"
"src/tree_sitter"))))

Shouldn't some of this deletion (namely, that of pregenerated files)
be handled by tree-sitter, though?

Toggle quote (2 lines)
> + (build-system tree-sitter-build-system)

...I think you may have forgotten to send the patches for the
tree-sitter-build-system?

-- (
(
COL6JKW2WS5K.39NSGAIFEVXDQ@guix-framework
On Fri Nov 25, 2022 at 1:21 AM GMT, Pierre Langlois wrote:
Toggle quote (10 lines)
> +(define tree-sitter-delete-generated-files
> + '(begin
> + (delete-file "binding.gyp")
> + (delete-file-recursively "bindings")
> + (delete-file "src/grammar.json")
> + (delete-file "src/node-types.json")
> + (delete-file "src/parser.c")
> + (delete-file-recursively "src/tree_sitter")
> + #t))

Aah, I just realised that the other grammars use this... My point
about incorporating it into tree-sitter-build-system still stands,
though.

-- (
(
Re: [bug#49946] [PATCH v7 09/32] gnu: Add tree-sitter-bash.
COL6K5427TVS.1FBJASUHZ4H64@guix-framework
On Fri Nov 25, 2022 at 1:21 AM GMT, Pierre Langlois wrote:
Toggle quote (2 lines)
> * gnu/packages/tree-sitter.scm (tree-sitter-bash): New variable.

LGTM

-- (
(
Re: [bug#49946] [PATCH v7 10/32] gnu: Add tree-sitter-css.
COL6K8LDG2KW.VERV7ZKKQ9QD@guix-framework
On Fri Nov 25, 2022 at 1:21 AM GMT, Pierre Langlois wrote:
Toggle quote (2 lines)
> * gnu/packages/tree-sitter.scm (tree-sitter-css): New variable.

LGTM

-- (
(
Re: [bug#49946] [PATCH v7 11/32] gnu: Add tree-sitter-c-sharp.
COL6KDWOORD2.2T5IY9GY8JW2S@guix-framework
On Fri Nov 25, 2022 at 1:21 AM GMT, Pierre Langlois wrote:
Toggle quote (2 lines)
> * gnu/packages/tree-sitter.scm (tree-sitter-c-sharp): New variable.

LGTM

-- (
(
Re: [bug#49946] [PATCH v7 12/32] gnu: Add tree-sitter-elixir.
COL6KJCJPJ3Y.167VZEQ8HHEGH@guix-framework
On Fri Nov 25, 2022 at 1:21 AM GMT, Pierre Langlois wrote:
Toggle quote (2 lines)
> * gnu/packages/tree-sitter.scm (tree-sitter-elixir): New variable.

LGTM

-- (
(
Re: [bug#49946] [PATCH v7 13/32] gnu: Add tree-sitter-elm.
COL6KOI38RNL.39NAY37EOGJ7Y@guix-framework
On Fri Nov 25, 2022 at 1:21 AM GMT, Pierre Langlois wrote:
Toggle quote (2 lines)
> * gnu/packages/tree-sitter.scm (tree-sitter-elm): New variable.

LGTM

-- (
(
Re: [bug#49946] [PATCH v7 14/32] gnu: Add tree-sitter-go.
COL6LCX5PGKK.1CK3ULF8Z3V0B@guix-framework
On Fri Nov 25, 2022 at 1:21 AM GMT, Pierre Langlois wrote:
Toggle quote (11 lines)
> * gnu/packages/tree-sitter.scm (tree-sitter-go): New variable.

> --- a/gnu/packages/tree-sitter.scm
> +++ b/gnu/packages/tree-sitter.scm

> @@ -328,3 +328,27 @@ (define-public tree-sitter-elm

> +(define-public tree-sitter-go
> + (let ((commit "05900faa3cdb5d2d8c8bd5e77ee698487e0a8611")
> + (revision "1"))

Why are you using a commit? Please add an explanatory comment. And
start at (revision "0") :)

-- (
(
Re: [bug#49946] [PATCH v7 08/32] gnu: Add tree-sitter-cpp.
COL6LL2R2LQ8.8RBHP6L94GEX@guix-framework
On Fri Nov 25, 2022 at 1:21 AM GMT, Pierre Langlois wrote:
Toggle quote (2 lines)
> * gnu/packages/tree-sitter.scm (tree-sitter-cpp): New variable.

LGTM (apart from nonexistant tree-sitter-build-system :))

-- (
(
COL6M6ZM2W4G.1W3E85IW5NLX4@guix-framework
On Fri Nov 25, 2022 at 1:21 AM GMT, Pierre Langlois wrote:
Toggle quote (11 lines)
> * gnu/packages/tree-sitter.scm (tree-sitter-cpp): New variable.

> --- a/gnu/packages/tree-sitter.scm
> +++ b/gnu/packages/tree-sitter.scm

> @@ -189,3 +189,28 @@ (define-public tree-sitter-c

> +(define-public tree-sitter-cpp
> + (let ((commit "5ead1e26c6ab71919db0f1880c46a278a93bc5ea")
> + (revision "1"))

Please an an explanation of why you're using a commit, and start
at revision "0".

-- (
(
Re: [bug#49946] [PATCH v7 16/32] gnu: Add tree-sitter-html.
COL6N1MTB927.2SKCFOCU086DY@guix-framework
On Fri Nov 25, 2022 at 1:21 AM GMT, Pierre Langlois wrote:
Toggle quote (2 lines)
> * gnu/packages/tree-sitter.scm (tree-sitter-html): New variable.

LGTM

-- (
(
Re: [bug#49946] [PATCH v7 17/32] gnu: Add tree-sitter-java.
COL6N7Y16N9G.18T87BEDTCKBG@guix-framework
On Fri Nov 25, 2022 at 1:21 AM GMT, Pierre Langlois wrote:
Toggle quote (2 lines)
> * gnu/packages/tree-sitter.scm (tree-sitter-java): New variable.

LGTM

-- (
(
Re: [bug#49946] [PATCH v7 18/32] gnu: Add tree-sitter-javascript.
COL6NBGJQZZK.IX1F03X1K6GK@guix-framework
On Fri Nov 25, 2022 at 1:21 AM GMT, Pierre Langlois wrote:
Toggle quote (2 lines)
> * gnu/packages/tree-sitter.scm (tree-sitter-javascript): New variable.

LGTM

-- (
(
Re: [bug#49946] [PATCH v7 19/32] gnu: Add tree-sitter-json.
COL6NEI5O9HZ.33P8V2BH3TAC0@guix-framework
On Fri Nov 25, 2022 at 1:21 AM GMT, Pierre Langlois wrote:
Toggle quote (2 lines)
> * gnu/packages/tree-sitter.scm (tree-sitter-json): New variable.

LGTM

-- (
(
Re: [bug#49946] [PATCH v7 20/32] gnu: Add tree-sitter-julia.
COL6OWILLA7L.20EWC3SVZ3SFT@guix-framework
On Fri Nov 25, 2022 at 1:21 AM GMT, Pierre Langlois wrote:
Toggle quote (2 lines)
> * gnu/packages/tree-sitter.scm (tree-sitter-julia): New variable.

LGTM

-- (
(
Re: [bug#49946] [PATCH v7 15/32] gnu: Add tree-sitter-haskell.
COL6P0M0U6IZ.1TKXVR8D9Y4RF@guix-framework
On Fri Nov 25, 2022 at 1:21 AM GMT, Pierre Langlois wrote:
Toggle quote (10 lines)
> * gnu/packages/tree-sitter.scm (tree-sitter-haskell): New variable.

> --- a/gnu/packages/tree-sitter.scm
> +++ b/gnu/packages/tree-sitter.scm

> @@ -352,3 +352,27 @@ (define-public tree-sitter-go
> +(define-public tree-sitter-haskell
> + (let ((commit "e30bdfd53eb28c73f26a68b77d436fd2140af167")
> + (revision "1"))

Explanatory comment and start at revision 0.
Toggle quote (2 lines)
> + (package
> + (name "tree-sitter-haskell")
^
Indentation is off; two spaces needed here...

Toggle quote (2 lines)
> + (source (origin
> + (method git-fetch)
^
...and here.

-- (
(
Re: [bug#49946] [PATCH v7 21/32] gnu: Add tree-sitter-ocaml.
COL6QXZI3TJJ.19HUXU5I3IQ2O@guix-framework
On Fri Nov 25, 2022 at 1:21 AM GMT, Pierre Langlois wrote:
Toggle quote (10 lines)
> * gnu/packages/tree-sitter.scm (tree-sitter-ocaml): New variable.

> --- a/gnu/packages/tree-sitter.scm
> +++ b/gnu/packages/tree-sitter.scm

> @@ -487,3 +487,42 @@ (define-public tree-sitter-julia

> + (let ((commit "0348562f385bc2bd67ecf181425e1afd6d454192")
> + (revision "1"))

Explain and start at 0.

Toggle quote (11 lines)
> + (delete-file "binding.gyp")
> + (delete-file-recursively "bindings")
> + (for-each
> + (lambda (lang)
> + (with-directory-excursion lang
> + (delete-file "src/grammar.json")
> + (delete-file "src/node-types.json")
> + (delete-file "src/parser.c")
> + (delete-file-recursively "src/tree_sitter")))
> + '("ocaml" "interface"))

This stuff should definitely be handled by the
tree-sitter-build-system...

Toggle quote (4 lines)
> + (arguments
> + (list
> + #:grammar-directories '("ocaml" "interface")))

Unnecessary newline and use of a quoted list; please use

(list #:grammar-directories (list "ocaml" "interface"))

-- (
(
Re: [bug#49946] [PATCH v7 22/32] gnu: Add tree-sitter-php.
COL6RAJJ9QZT.1TTCX9K3FMFH5@guix-framework
On Fri Nov 25, 2022 at 1:21 AM GMT, Pierre Langlois wrote:
Toggle quote (10 lines)
> * gnu/packages/tree-sitter.scm (tree-sitter-php): New variable.

> --- a/gnu/packages/tree-sitter.scm
> +++ b/gnu/packages/tree-sitter.scm

> @@ -526,3 +526,33 @@ (define-public tree-sitter-ocaml

> + (let ((commit "435fa00006c0d1515c37fbb4dd6a9de284af75ab")
> + (revision "1"))

Explain and start at 0.

-- (
(
Re: [bug#49946] [PATCH v7 23/32] gnu: Add tree-sitter-python.
COL6ROOMWQ35.22SZGJDYHW47Y@guix-framework
On Fri Nov 25, 2022 at 1:21 AM GMT, Pierre Langlois wrote:
Toggle quote (10 lines)
> * gnu/packages/tree-sitter.scm (tree-sitter-python): New variable.

> --- a/gnu/packages/tree-sitter.scm
> +++ b/gnu/packages/tree-sitter.scm

> @@ -556,3 +556,27 @@ (define-public tree-sitter-php

> + (let ((commit "ed0fe62e55dc617ed9dec8817ebf771aa7cf3c42")
> + (revision "1"))

Explain and start at 0.

-- (
(
Re: [bug#49946] [PATCH v7 24/32] gnu: Add tree-sitter-r.
COL6RWZ609J5.122QNFSCWHFGD@guix-framework
On Fri Nov 25, 2022 at 1:21 AM GMT, Pierre Langlois wrote:
Toggle quote (11 lines)
> * gnu/packages/tree-sitter.scm (tree-sitter-r): New variable.

> --- a/gnu/packages/tree-sitter.scm
> +++ b/gnu/packages/tree-sitter.scm

> @@ -580,3 +580,27 @@ (define-public tree-sitter-python

> +(define-public tree-sitter-r
> + (let ((commit "80efda55672d1293aa738f956c7ae384ecdc31b4")
> + (revision "1"))

Explain and start at 0.

-- (
(
Re: [bug#49946] [PATCH v7 25/32] gnu: Add tree-sitter-ruby.
COL6S5TGCMKE.QS0IEBWIQVKX@guix-framework
On Fri Nov 25, 2022 at 1:21 AM GMT, Pierre Langlois wrote:
Toggle quote (2 lines)
> * gnu/packages/tree-sitter.scm (tree-sitter-ruby): New variable.

LGTM

-- (
(
Re: [bug#49946] [PATCH v7 26/32] gnu: Add tree-sitter-rust.
COL6S9S91M82.1N4LWJOM9LOJL@guix-framework
On Fri Nov 25, 2022 at 1:21 AM GMT, Pierre Langlois wrote:
Toggle quote (2 lines)
> * gnu/packages/tree-sitter.scm (tree-sitter-rust): New variable.

LGTM

-- (
(
Re: [bug#49946] [PATCH v7 27/32] gnu: Add tree-sitter-typescript.
COL6SNQUAS2E.18M018RM5VUA8@guix-framework
On Fri Nov 25, 2022 at 1:21 AM GMT, Pierre Langlois wrote:
Toggle quote (11 lines)
> * gnu/packages/tree-sitter.scm (tree-sitter-typescript): New variable.

> --- a/gnu/packages/tree-sitter.scm
> +++ b/gnu/packages/tree-sitter.scm

> @@ -648,3 +648,41 @@ (define-public tree-sitter-rust

> + (arguments
> + (list
> + #:grammar-directories '("typescript" "tsx")))

Superfluous newline.

-- (
(
Re: [bug#49946] [PATCH v7 28/32] gnu: Add rust-tree-sitter.
COL6TPPIQOLK.3CUNK5CQND5K4@guix-framework
On Fri Nov 25, 2022 at 1:21 AM GMT, Pierre Langlois wrote:
Toggle quote (19 lines)
> * gnu/packages/tree-sitter.scm (rust-tree-sitter): New variable.

> --- a/gnu/packages/tree-sitter.scm
> +++ b/gnu/packages/tree-sitter.scm

> @@ -158,6 +159,43 @@ (define-public tree-sitter-cli

> + '(begin
> + ;; Remove the runtime library code and dynamically link to
> + ;; it instead.
> + (delete-file-recursively "src")
> + (delete-file "binding_rust/build.rs")
> + (with-output-to-file "binding_rust/build.rs"
> + (lambda _
> + (format #t "fn main() {~@
> + println!(\"cargo:rustc-link-lib=tree-sitter\");~@
> + }~%")))
> + #t))))

Maybe you should actually extract *this* snippet, as you also use
it with tree-sitter-cli?

Toggle quote (6 lines)
> + (arguments
> + `(#:cargo-inputs
> + (("rust-cc" ,rust-cc-1)
> + ("rust-lazy-static" ,rust-lazy-static-1)
> + ("rust-regex" ,rust-regex-1))))

(list ...) please.

-- (
(
Re: [bug#49946] [PATCH v7 29/32] gnu: Add rust-tree-sitter-for-emacs.
COL6YZ9URVWA.388EGOJKP3TEZ@guix-framework
On Fri Nov 25, 2022 at 1:21 AM GMT, Pierre Langlois wrote:
Toggle quote (21 lines)
> * gnu/packages/tree-sitter.scm (rust-tree-sitter-for-emacs): New variable.
> * gnu/packages/patches/rust-tree-sitter-text-provider-fix.patch: New file.
> * gnu/local.mk (dist_patch_DATA): Register it.

> --- a/gnu/packages/tree-sitter.scm
> +++ b/gnu/packages/tree-sitter.scm

> @@ -196,6 +197,19 @@ (define-public rust-tree-sitter
> +;; We need to apply a patch in order to compile the rust bindings against the
> +;; emacs tree-sitter module.
> +;; See https://github.com/tree-sitter/tree-sitter/pull/1294
> +(define-public rust-tree-sitter-for-emacs
> + (package (inherit rust-tree-sitter)
> + (source (origin
> + (inherit (package-source rust-tree-sitter))
> + (patches (search-patches
> + "rust-tree-sitter-text-provider-fix.patch"))))
> + ;; Do not show this package in the UI as it's only meant to be used for
> + ;; emacs's tree-sitter module.
> + (properties '((hidden? . #t)))))

You can use ``hidden-package'', which is a bit nicer than ``properties''
imo:

(hidden-package
(package (inherit rust-tree-sitter)
...))

-- (
(
Re: [bug#49946] [PATCH v7 30/32] gnu: Add emacs-tree-sitter-core.
COL72SLW5HTE.1K3Y67DCEMPPJ@guix-framework
On Fri Nov 25, 2022 at 1:21 AM GMT, Pierre Langlois wrote:
Toggle quote (3 lines)
> * gnu/packages/tree-sitter.scm (tree-sitter-emacs-module): New local variable.
> (emacs-tree-sitter-core): New variable.

Please split these two into separate patches.

Toggle quote (8 lines)
> --- a/gnu/packages/tree-sitter.scm
> +++ b/gnu/packages/tree-sitter.scm

> @@ -738,3 +739,86 @@ (define-public tree-sitter-typescript

> + (arguments
> + `(#:cargo-inputs

(list #:cargo-inputs
...)

Toggle quote (18 lines)
> + #:phases
> + (modify-phases %standard-phases
> + (add-after 'unpack 'chdir
> + (lambda _ (chdir "core")))
> + (add-after 'chdir 'delete-cargo.lock
> + (lambda _ (delete-file "Cargo.lock")))
> + (add-after 'delete-cargo.lock 'do-not-fetch-from-github
> + (lambda _
> + (substitute* "Cargo.toml"
> + (("\\[patch.*") "")
> + (("git = .*") ""))))
> + (replace 'install
> + (lambda* (#:key outputs #:allow-other-keys)
> + (let ((lib (string-append (assoc-ref outputs "out") "/lib")))
> + (mkdir-p lib)
> + (copy-file "target/release/libtsc_dyn.so"
> + (string-append lib "/tsc-dyn.so"))))))))

Please use gexps and #$output here:

(replace 'install
(lambda _
(let ((lib (string-append #$output "/lib")))
(mkdir-p lib)
(copy-file "target/release/libtsc_dyn.so"
(string-append lib "/tsc-dyn.so")))))

Toggle quote (4 lines)
> + (name "emacs-tree-sitter-core")
> + (version (package-version tree-sitter-emacs-module))
> + (source (package-source tree-sitter-emacs-module))

This ``version'' and ``source'' are unnecessary.

Toggle quote (3 lines)
> + (arguments
> + `(#:phases

LIST, please.

Toggle quote (2 lines)
> + (modify-phases %standard-phases

Gexp this, please.

Toggle quote (2 lines)
> + (let ((elpa (elpa-directory (assoc-ref outputs "out")))

Please use #$output.

Toggle quote (2 lines)
> + (module (search-input-file inputs "/lib/tsc-dyn.so")))

The leading / is redundant; this will do:

(search-input-file inputs "lib/tsc-dyn.so")

-- (
(
COL73W51TW1Q.1V6UY8SHDKAVE@guix-framework
On Fri Nov 25, 2022 at 1:21 AM GMT, Pierre Langlois wrote:
Toggle quote (6 lines)
> +(define-public emacs-tree-sitter-core
> + (package
> + (name "emacs-tree-sitter-core")
> + (version (package-version tree-sitter-emacs-module))
> + (source (package-source tree-sitter-emacs-module))

Never mind re ``source'' and ``version''; I just realised you didn't inherit from the
tree-sitter-emacs-module package :)

-- (
(
Re: [bug#49946] [PATCH v7 31/32] gnu: Add emacs-tree-sitter.
COL748WAHH3I.1TJK4IY2EYIIE@guix-framework
On Fri Nov 25, 2022 at 1:21 AM GMT, Pierre Langlois wrote:
Toggle quote (10 lines)
> * gnu/packages/tree-sitter.scm (emacs-tree-sitter): New variable.

> --- a/gnu/packages/tree-sitter.scm
> +++ b/gnu/packages/tree-sitter.scm

> @@ -822,3 +822,30 @@ (module (search-input-file inputs "/lib/tsc-dyn.so")))

> + (arguments
> + `(#:phases

LIST please.

Toggle quote (2 lines)
> + (modify-phases %standard-phases

Aaaand gexp.

-- (
(
Re: [bug#49946] [PATCH v7 32/32] gnu: Add emacs-tree-sitter-langs.
COL77TIV1EVV.Q7OVWZ55S2RC@guix-framework
On Fri Nov 25, 2022 at 1:21 AM GMT, Pierre Langlois wrote:
Toggle quote (3 lines)
> * gnu/packages/tree-sitter.scm (make-emacs-tree-sitter-langs-grammar-bundle):
> New procedure.

Doesn't look like a procedure to me :) It probably should also be
a hidden-package, no?

Toggle quote (10 lines)
> (emacs-tree-sitter-langs): New variable.

> --- a/gnu/packages/tree-sitter.scm
> +++ b/gnu/packages/tree-sitter.scm

> @@ -849,3 +850,152 @@ (define-public emacs-tree-sitter

> + (synopsis #f)
> + (description #f)

Please add a description and synopsis.

Toggle quote (3 lines)
> + (list
> + #:tests? #t

Superfluous newline; please use:

(list #:tests? #t ...)

Toggle quote (5 lines)
> + (add-after 'install 'install-queries
> + (lambda* (#:key outputs #:allow-other-keys)
> + (let ((elpa (elpa-directory (assoc-ref outputs "out"))))
> + (copy-recursively "queries" (string-append elpa "/queries"))))))))

Please use #$output here.

-- (
P
P
Pierre Langlois wrote on 25 Nov 2022 02:57
[PATCH v7 06/32] build-system: Add tree-sitter-build-system.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
87r0xrcpei.fsf@gmx.com
It appears this email is making it through, so here it is as an
attachment:
-----BEGIN PGP SIGNATURE-----

iQFMBAEBCgA2FiEEctU9gYy29KFyWDdMqPyeRH9PfVQFAmOAlWYYHHBpZXJyZS5s
YW5nbG9pc0BnbXguY29tAAoJEKj8nkR/T31UIo0H/iRioXzQ/9ulOwlNlgn2+FEi
fgAUL+jF3uYEX96rWrBLIY2EDdGU1O+e/ymxxkonDkg5VV3Hp+qYNm+IfSPoxtdg
frfk+60D0BIS82gTve6FWnjBCtlzVFLEHb6XuRk9+pgKOh2s/JsqLp+DFAYCWoFN
onncjb7bBlNIsfH2cYfx5Bmj3GLXbNP1InCChE0WMSNCY7ZdBX1ph6Spsf6s0syL
FjKiy8nLz852YsReR1RMxGr2fdxNQCbWMFr3w+3NoyfE5RXhsTfD4b+TlZv0Gjp8
5ToWkOV7Eys2YmjahRbCXx9H9IuCFi+qWMsBO+CdTNagyAlBv7I9VCwRSBRlfZg=
=lxXa
-----END PGP SIGNATURE-----

P
P
Pierre Langlois wrote on 25 Nov 2022 11:18
Re: [bug#49946] [PATCH v7 01/32] gnu: tree-sitter: Move to its own module.
(name . ()(address . paren@disroot.org)
87bkovcp1d.fsf@gmx.com
Hi!

"(" <paren@disroot.org> writes:

Toggle quote (2 lines)
> Heya,

Thanks for taking a look and spending time reviewing this, this is
great. I'll work on your suggestions over the weekend.

Thanks,
Pierre
-----BEGIN PGP SIGNATURE-----

iQFMBAEBCgA2FiEEctU9gYy29KFyWDdMqPyeRH9PfVQFAmOAlz4YHHBpZXJyZS5s
YW5nbG9pc0BnbXguY29tAAoJEKj8nkR/T31U4EMH/0g5S5KVVbeY8cjzZVqpsx6+
Tl4OOJsyxDLMKFJDDRRSU/bvGvySUZr9oK378V6/bDKzIiXH4xAZXYyCaztqvT7c
LesrgtOi5OiV/S2GBJxY693a0J4OCHd6Hbq96SPnWecowt+0l+rDiCplc7E3g1xP
eKgDHlV1qmPFKOTLkSpgisT0jN0hlfKxrT3JN0kI8P+VjfYTT7xCE46+sU+Hz8As
d1qr1F1GTIPKGRFmpugbAbnqTp/dyAtcTcsGQtgKpg+QqqQjhbtbWuD8Mr5UscWm
Igyr2sScK+PA07mlBBYCWYCAOWxyq9mY0OIyrOr0OOWyMU6+e/2VDdvJ1eA5vW0=
=uKDU
-----END PGP SIGNATURE-----

L
L
Luis Henrique Gomes Higino wrote on 25 Jan 2023 23:04
About migrating support from emacs-tree-sitter to treesit
874jsemfcy.fsf@gmail.com
Hi guix,

I was taking a look at this patch series again and wondering if it
could
be modified to support the new built in treesit module coming in
Emacs 29 instead of emacs-tree-sitter, as the latter recommends
using
the native integration going forward [1]. Maybe we should leave
this to when
Emacs 29 is released, but I think working on this in advance could
result on it being delivered earlier, which would be great. What
do you think Pierre?

[1]:

PS: I'm willing to help

Regards,
--
Luis H. Higino
L
L
Luis Henrique Gomes Higino wrote on 30 Jan 2023 00:10
874js8ewsw.fsf@gmail.com
Hi guix,

Luis Henrique Gomes Higino <luishenriquegh2701@gmail.com> writes:

Toggle quote (24 lines)
> Hi guix,
>
> I was taking a look at this patch series again and wondering if
> it
> could
> be modified to support the new built in treesit module coming in
> Emacs 29 instead of emacs-tree-sitter, as the latter recommends
> using
> the native integration going forward [1]. Maybe we should leave
> this
> to when
> Emacs 29 is released, but I think working on this in advance
> could
> result on it being delivered earlier, which would be great. What
> do
> you think Pierre?
>
> [1]:
> https://github.com/emacs-tree-sitter/elisp-tree-sitter/commit/d3eab879e9b0ccc9582113d81a93ad06f3c6bfb1
>
> PS: I'm willing to help
>
> Regards,

I played around this idea this weekend and I got `treesit' to use
the Guix installed grammars :) !

First, I applied a patch that added tree-sitter as an input for
emacs-next [1].


Then, I did the edits below on top of Pierre's latest patch
series:
From fb96ca9c6039d07856c8f3525919ed9036b39470 Mon Sep 17 00:00:00 2001
From: Luis Henrique Gomes Higino <luishenriquegh2701@gmail.com>
Date: Sun, 29 Jan 2023 20:24:04 -0300
Subject: [PATCH] Support treesit module from emacs

---
gnu/packages/emacs.scm | 8 +++++++-
guix/build/tree-sitter-build-system.scm | 2 +-
2 files changed, 8 insertions(+), 2 deletions(-)

Toggle diff (34 lines)
diff --git a/gnu/packages/emacs.scm b/gnu/packages/emacs.scm
index 951c1ad6df..193547ff42 100644
--- a/gnu/packages/emacs.scm
+++ b/gnu/packages/emacs.scm
@@ -409,7 +409,13 @@ (define-public emacs-next
(prepend sqlite tree-sitter)))
(native-inputs
(modify-inputs (package-native-inputs emacs)
- (prepend autoconf))))))
+ (prepend autoconf)))
+ (native-search-paths
+ (cons
+ (search-path-specification
+ (variable "TREE_SITTER_GRAMMARS")
+ (files '("lib/tree-sitter")))
+ (package-native-search-paths emacs))))))
(define-public emacs-next-pgtk
(package
diff --git a/guix/build/tree-sitter-build-system.scm b/guix/build/tree-sitter-build-system.scm
index 574b0f2a1c..ee1aa9976c 100644
--- a/guix/build/tree-sitter-build-system.scm
+++ b/guix/build/tree-sitter-build-system.scm
@@ -114,7 +114,7 @@ (define (compile-language dir)
"-fno-exceptions"
"-O2"
"-g"
- "-o" ,(string-append lib "/" lang ".so")
+ "-o" ,(string-append lib "/libtree-sitter-" lang ".so")
;; An additional `scanner.{c,cc}' file is sometimes
;; provided.
,@(cond
--
2.39.1
Then, I ran `./pre-inst-env guix shell emacs-next tree-sitter-c
tree-sitter-cpp --container --preserve="^TERM$" -- emacs -Q -nw`
and executed the following code inside that Emacs session:

Toggle snippet (10 lines)
(defvar guix-treesit-grammar-paths
(let ((grammars-path (if (getenv "TREE_SITTER_GRAMMARS")
(getenv "TREE_SITTER_GRAMMARS")
"")))
(split-string grammars-path ":")))

(if (treesit-available-p)
(setq treesit-extra-load-path guix-treesit-grammar-paths))

Doing these steps, Emacs' treesit was able to use the grammars
installed by Guix in c-ts-mode. I don't know if a native
search-path is the appropriate approach or how to include the
above elisp (or something similar) in the emacs-next package, but
I hope this helps Pierre or anyone else with more understanding to
do the v8 of this patch series :).

Regards,
--
Luis H. Higino
P
P
Pierre Langlois wrote on 6 Feb 2023 01:00
(name . Luis Henrique Gomes Higino)(address . luishenriquegh2701@gmail.com)
87wn4vzmes.fsf@gmx.com
Hi Luis,

Luis Henrique Gomes Higino <luishenriquegh2701@gmail.com> writes:

Toggle quote (31 lines)
> Hi guix,
>
> Luis Henrique Gomes Higino <luishenriquegh2701@gmail.com> writes:
>
>> Hi guix,
>>
>> I was taking a look at this patch series again and wondering if it
>> could
>> be modified to support the new built in treesit module coming in
>> Emacs 29 instead of emacs-tree-sitter, as the latter recommends using
>> the native integration going forward [1]. Maybe we should leave this
>> to when
>> Emacs 29 is released, but I think working on this in advance could
>> result on it being delivered earlier, which would be great. What do
>> you think Pierre?
>>
>> [1]:
>> https://github.com/emacs-tree-sitter/elisp-tree-sitter/commit/d3eab879e9b0ccc9582113d81a93ad06f3c6bfb1
>>
>> PS: I'm willing to help
>>
>> Regards,
>
> I played around this idea this weekend and I got `treesit' to use the Guix
> installed grammars :) !
>
> First, I applied a patch that added tree-sitter as an input for emacs-next
> [1]. [1]: https://issues.guix.gnu.org/61139
>
> Then, I did the edits below on top of Pierre's latest patch series:

This is super cool, thanks for looking into this! and sorry for replying
so late. I agree that we should try and get these grammars integrated
into emacs as soon as we can, it's the end goal after all (by that I
mean that I'm not sure the Rust-based emacs module will live on after
emacs 29 is released).

But anyways, it's great that you got it working with minimal changes,
looks like there won't be any blockers :-).

Regarding the patch series, I do have a new version in the pipeline
(fixing style and improving testing) but I'm afraid I've had no time to
work on it in the past month and a half. I should have more time
starting from next weekend.

Thanks!
Pierre
Z
Z
zimoun wrote on 7 Feb 2023 12:25
Re: bug#49946: [PATCH 00/31] Tree-sitter, node-gyp addon support and emacs-tree-sitter
(name . Pierre Langlois)(address . pierre.langlois@gmx.com)(address . 49946@debbugs.gnu.org)
864jrxogxt.fsf_-_@gmail.com
Hi,

Sorry, I have totally overlooked this tree-sitter story. :-) And I have
not read all the patch versions and thread yet.


On Fri, 25 Nov 2022 at 01:57, Pierre Langlois <pierre.langlois@gmx.com> wrote:

Toggle quote (18 lines)
> +(define* (install #:key target grammar-directories outputs #:allow-other-keys)
> + (let ((lib (string-append (assoc-ref outputs "out")
> + "/lib/tree-sitter")))
> + (mkdir-p lib)
> + (define (compile-language dir)
> + (with-directory-excursion dir
> + (let ((lang (assoc-ref (call-with-input-file "src/grammar.json"
> + read-json)
> + "name"))
> + (source-file (lambda (path)
> + (if (file-exists? path)
> + path
> + #f))))
> + (apply invoke
> + `(,(if target
> + (string-append target "-g++")
> + "g++")

Why is g++ required? Is gcc not enough? I thought from Tree-sitter
documentation that it was only using C files and not C++; but…

Toggle quote (13 lines)
> + "-shared"
> + "-fPIC"
> + "-fno-exceptions"
> + "-O2"
> + "-g"
> + "-o" ,(string-append lib "/" lang ".so")
> + ;; An additional `scanner.{c,cc}' file is sometimes
> + ;; provided.
> + ,@(cond
> + ((source-file "src/scanner.c")
> + => (lambda (file) (list "-xc" "-std=c99" file)))
> + ((source-file "src/scanner.cc")

…apparently not.


Cheers,
simon
A
A
Andrew Tropin wrote on 9 Feb 2023 11:11
Re: [bug#49946] [PATCH v7 01/32] gnu: tree-sitter: Move to its own module.
87h6vvgnd6.fsf@trop.in
On 2022-11-25 10:18, Pierre Langlois wrote:

Toggle quote (12 lines)
> Hi!
>
> "(" <paren@disroot.org> writes:
>
>> Heya,
>
> Thanks for taking a look and spending time reviewing this, this is
> great. I'll work on your suggestions over the weekend.
>
> Thanks,
> Pierre

Hi Pierre!

Thank you very much for all the work.

I applied tree-sitter and tree-sitter-cli patches, adjusted them,
addressed comments of unmatching-paren and succesfully built respective
packages. I'll push them later today or tomorrow. Just letting you
know to prevent duplication of effort.

Also, later I'll make a separate message on tree-sitter grammars and the
way I see they can be added to Guix and Emacs.

--
Best regards,
Andrew Tropin
-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEEKEGaxlA4dEDH6S/6IgjSCVjB3rAFAmPkxrUACgkQIgjSCVjB
3rBRPBAAh1rKPtGDdm5rBCSYsq1ZFQU/Oll61v+jU/KvEe3AtjlKrSmY8YHk9UGN
QxfKSxyKvN/G7UwRw/bvA23dBASbqGtGu8hGahgnqrpgGA4Cb8PA6Es7PwHF+1PK
/2Itaw8MXBtkzB0IgGbCD9sg/boCWz6dJrgSt64XgwGTYcB1raQ21otS3xJXEOgS
hg3zibLKaWr7IvRN5IJoBTZtdBCvBOdJuoaEal/GiFy1/L4e0M6HF6Anjt6nQMaw
JqoULivxsc3+wuClhRVU0Q73Zyd7hrVxqyzcKr6u/te9ISbaclDBCJMnb5CDtZbE
byfJM+FdfBmcyr86xmP95qU/O8L1kqX+1sjWX8w7WtDwojI9H0syr+4FZVumW28g
C1zid3H5Q+7MiZuLkBiIbe/rRjAPuqmET4uZrY0MUrGtWKsPxevHGxje9Jsp9wrT
sLYPRENOSh4odT+GwQ2pNJ1s9l0r4kuhmb9EU0FrPQvPqZeJKVitoSJOHFgG4kcc
m9YS7OwVJWZNbywQL8BGsuAo18IGKoSYi0E4QgfDo3SA4WbdQoafwB9yxkqn6Pyv
joqCcRPfvMiAoEBj92lzTsXc3NZ7RXbu8jDoxPeXZQTkhV7M5rzj+VBkoCxMN8ic
nszqceL/pTBOdGeZjJnZ5fms9jbejJkXuCmNhSeBjsLQ4QvE6Tg=
=xAGo
-----END PGP SIGNATURE-----

Z
Z
zimoun wrote on 9 Feb 2023 13:39
86pmaj3td2.fsf@gmail.com
Hi,

On Thu, 09 Feb 2023 at 14:11, Andrew Tropin <andrew@trop.in> wrote:

Toggle quote (2 lines)
> I applied tree-sitter and tree-sitter-cli patches,

Just to be sure to understand, you have only applied 02/32 and 05/32,
right?


[bug#49946] [PATCH v7 02/32] gnu: tree-sitter: Update to 0.20.7.
id:20221125012142.22579-3-pierre.langlois@gmx.com

[bug#49946] [PATCH v7 05/32] gnu: Add tree-sitter-cli.
id:20221125012142.22579-6-pierre.langlois@gmx.com

Leaving out all the others, right?

Cheers,
simon
A
A
Andrew Tropin wrote on 9 Feb 2023 15:04
87cz6jgcku.fsf@trop.in
On 2023-02-09 13:39, zimoun wrote:

Toggle quote (20 lines)
> Hi,
>
> On Thu, 09 Feb 2023 at 14:11, Andrew Tropin <andrew@trop.in> wrote:
>
>> I applied tree-sitter and tree-sitter-cli patches,
>
> Just to be sure to understand, you have only applied 02/32 and 05/32,
> right?
>
>
> [bug#49946] [PATCH v7 02/32] gnu: tree-sitter: Update to 0.20.7.
> id:20221125012142.22579-3-pierre.langlois@gmx.com
> http://issues.guix.gnu.org/msgid/20221125012142.22579-3-pierre.langlois@gmx.com
>
> [bug#49946] [PATCH v7 05/32] gnu: Add tree-sitter-cli.
> id:20221125012142.22579-6-pierre.langlois@gmx.com
> http://issues.guix.gnu.org/msgid/20221125012142.22579-6-pierre.langlois@gmx.com
>
> Leaving out all the others, right?

Merged first 5 patches from 01 to 05, also added one more commit, which
addresses some things from reviews and one commit, which adds html
grammar.

The html grammar is added for the testing purposes. It relies on
generated parser.c and scanner.c and we will need to repackage it using
grammar.js instead. I'm not sure if a separate build system is needed
for this, I guess we can just rewrite tree-sitter-grammar function,
which generates packages as in example with tree-sitter-grammar-html:

Anyway, I only skimmed tree-sitter-build-system source code, and plan to
read it carefully, evaluate and either introduce new build system or
just move all needed parts to tree-sitter-grammar function. WDYT?
After we done with it we can package all other grammars.

According to emacs integration: I already have a working prototype on my
local setup, which uses builtin treesit package, it's similiar to what
Luis mentioned with search-paths. We just need to figure out a few
minor details and I'll add it.

--
Best regards,
Andrew Tropin
-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEEKEGaxlA4dEDH6S/6IgjSCVjB3rAFAmPk/VEACgkQIgjSCVjB
3rAxMA/+JdVS1H4fFoGJGI2LltWc6BsUUlPHQN5ELd1HcGNnzXQtX9jWNJwhkoK8
QHRYE/cWlaZmtdLCuSN9paEqbStVJ+SKYdGcOdTw7C4Riw0o8ceE5I9PNShro5Wh
XfvB7hDiG3rpMnlvlwnhHmXsCk+SPntMNO9B5P+0f3O92YsQ0j2k8iPLeALSueDt
/cwNfc83SSHJ5aQEvuIcKsmjMXe+/52CMiOfieVUIMHY8LrLZH0c47wwbYqDE5K4
5Vl5spxSgwrPoIX4v79x+EPdfN16ui/kkAf96ihW03oo6SAMJHeZIB0fDYJQgEB5
Uc9tqv3uovrOjIDyNsGGHWh6SSU2pFslpay/KsK6NHlV0g7hVS5DseKmNCPXHtTh
Bo0tJQwSA5hMO4yl0XWGIWLzs0VpiA0aEWkoOumR9KATD0pLXv8bBO1aUNhv/SVP
HJeUqTPpo536OfayFb7Kp0+QQDZS3RgoLP2sFp+ddTBWd3vwAXv0OCqjujKOjpqF
YqUXzAwanE7qiNWy1rtC8ASAmFQe0M9GEGypbMgwVrb052aiYQ5O8oyf+MkSjT34
q1ZEVJybV+xp/X7iALow0wZDAbBAuCYmBW4qIOEEslJdTGBNtseLdiSU2+1UjVMF
2Jf27fAKxwCEkTbzxHiN2xJG3lRld/3FMSwA3bZXWhcMj+2nor0=
=AfIp
-----END PGP SIGNATURE-----

A
A
Andrew Tropin wrote on 10 Feb 2023 10:21
Re: [bug#49946] [PATCH 00/31] Tree-sitter, node-gyp addon support and emacs-tree-sitter
87edqxri41.fsf@trop.in
On 2021-08-08 23:29, Pierre Langlois wrote:

Toggle quote (26 lines)
> 4. Finally, emacs support!
>
> emacs-tree-sitter can be used for syntax highlighting, replacing
> font-lock. The tree-sitter runtime library is built as a Rust crate
> and exposed to emacs using a module thanks to the rust-emacs crate.
>
> Then emacs-tree-sitter-core provides just the core APIs as exposed by
> the rust module, then emacs-tree-sitter integrates it with emacs'
> syntax highlighting (and probably other things).
>
> Finally, we bundle together the various language grammars into a
> single emacs-tree-sitter-langs package so that they can get loaded
> along with their respective major modes. Note that there are grammars
> left to support which I've not yet managed to package, but what we
> have already is a good start I think :-). So far I've tested C++,
> Rust, Python and JavaScript and JSON.
>
> * gnu: Add rust-bindgen@0.56.
> * gnu: Add rust-tree-sitter.
> * gnu: rust-emacs-module: Update to 0.16.
> * gnu: rust-emacs-macros: Update to 0.17.
> * gnu: rust-emacs: Update to 0.17.
> * gnu: Add emacs-tree-sitter-core.
> * gnu: Add emacs-tree-sitter.
> * gnu: Add emacs-tree-sitter-langs.

Hi Pierre!

Implemented a loading of tree-sitter grammars in Emacs using guix
native-search-paths and built-in treesit package. I did it for emacs
package, but it will work only with Emacs 29 and greater (right now it's
emacs-next and similiar packages), until emacs package updated to
version 29 it won't take any effect on it except providing one more
environment variable in guix profile.

I tested it with emacs-next-pgtk, the patch is attached below, please
let me know what do you think!
From b5ecd4e3734e9dd0bc76ebe95cab9c43aa85a3fe Mon Sep 17 00:00:00 2001
From: Andrew Tropin <andrew@trop.in>
Date: Fri, 10 Feb 2023 12:32:12 +0400
Subject: [PATCH] gnu: emacs: Add TREE_SITTER_GRAMMAR_PATH support.

gnu/packages/emacs.scm (emacs)[native-search-paths]: Add a search-path for
tree-sitter grammars.
gnu/packages/aux-files/emacs/guix-emacs.el: Add directories from
TREE_SITTER_GRAMMAR_PATH to treesit-extra-load-path.
---
gnu/packages/aux-files/emacs/guix-emacs.el | 7 +++++++
gnu/packages/emacs.scm | 10 +++++++++-
2 files changed, 16 insertions(+), 1 deletion(-)

Toggle diff (41 lines)
diff --git a/gnu/packages/aux-files/emacs/guix-emacs.el b/gnu/packages/aux-files/emacs/guix-emacs.el
index 56dbcb8d67..708093267d 100644
--- a/gnu/packages/aux-files/emacs/guix-emacs.el
+++ b/gnu/packages/aux-files/emacs/guix-emacs.el
@@ -76,6 +76,13 @@ The files in the list do not have extensions (.el, .elc)."
(when (file-directory-p pkg-dir)
(package-load-descriptor pkg-dir)))))))))))
+;; If emacs built with tree-sitter, read the value of the environment variable
+;; to make tree-sitter grammars available in emacs out-of-the-box.
+(with-eval-after-load 'treesit
+ (when-let ((grammar-path (getenv "TREE_SITTER_GRAMMAR_PATH")))
+ (mapcar (lambda (x) (add-to-list 'treesit-extra-load-path x))
+ (split-string grammar-path ":"))))
+
(provide 'guix-emacs)
;;; guix-emacs.el ends here
diff --git a/gnu/packages/emacs.scm b/gnu/packages/emacs.scm
index 4ce41deb88..a3cc9d2e22 100644
--- a/gnu/packages/emacs.scm
+++ b/gnu/packages/emacs.scm
@@ -367,7 +367,15 @@ (define* (emacs-byte-compile-directory dir)
(files '("lib/emacs/native-site-lisp")))
(search-path-specification
(variable "INFOPATH")
- (files '("share/info")))))
+ (files '("share/info")))
+ ;; tree-sitter support is not yet available in emacs 28, but this
+ ;; search path won't harm and also will be benifitial for
+ ;; emacs-next and other emacs-* packages, which have tree-sitter
+ ;; support enabled. Please, remove this comment, when emacs
+ ;; package is updated to 29.
+ (search-path-specification
+ (variable "TREE_SITTER_GRAMMAR_PATH")
+ (files '("lib/tree-sitter")))))
(home-page "https://www.gnu.org/software/emacs/")
(synopsis "The extensible, customizable, self-documenting text editor")
--
2.39.1
--
Best regards,
Andrew Tropin
-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEEKEGaxlA4dEDH6S/6IgjSCVjB3rAFAmPmDI4ACgkQIgjSCVjB
3rAqMw/9EEOnkMMcWqA2434QvKazGX6eiN+bTLNcPz+5IZuVuKqbbU2k2F0tATEC
Vy05t+XPVvhELzizGJRk/gxYwsuMpULNqu6YIuO2+asP45iw4tnH5Vdvf6RLOHKW
WH+Fr+Fo6wDaEGztDCEY4ud/j1bsdtgrxxWgUmE6VgzBt7UJl1Y6LVJh0tSn/gNl
tseYkXcYHn1bQUCHBIA03w45WNZ8bQMxRXGzvmiNUEPWJhGbkluk3TQuGXXezKKn
O3tLuKwERxG4gRTC9Oj3uPWcv4RCxbQ2binNamT1KGFVpq1VdwA8WbDu41gp4Nk+
TIEw+G2JcbaxC3Om/p8oJsVsbYUaHtZlzysLhSjeg1mSQdqh6WanwuYif2cVFWQo
uUEt7+rcfZyiLeDqlgiUZO5gv86k+LyB3q4/wf4W0B0Ij8AblLMMNgAeUukamq5p
KfhuTRdAawL7ZijPCQ3QuDRJJ/x9OsK9s+WTNV2tITxfWwm6uQgtizwrpGwnUaPW
3gb/hwdhgdNjlsrICod9+8X++akY2bXcCJTfWKebeKsgzTj4fR5MIxIjeYDInd2E
asuVGsep6y5lxCsz4gN4bMEQcDjpj5B1Y6zsi6FiiMylkuqWeHvWZZet3/LAQJXD
ZxZRufLGz/zSpsW4W4nXKnr9LCgiY1HxthOYJHduveVpWIGFOz8=
=ohCg
-----END PGP SIGNATURE-----

A
A
Andrew Tropin wrote on 10 Feb 2023 13:52
Re: [bug#49946] [PATCH v7 01/32] gnu: tree-sitter: Move to its own module.
87a61lr8cp.fsf@trop.in
On 2023-02-09 18:04, Andrew Tropin wrote:

Toggle quote (38 lines)
> On 2023-02-09 13:39, zimoun wrote:
>
>> Hi,
>>
>> On Thu, 09 Feb 2023 at 14:11, Andrew Tropin <andrew@trop.in> wrote:
>>
>>> I applied tree-sitter and tree-sitter-cli patches,
>>
>> Just to be sure to understand, you have only applied 02/32 and 05/32,
>> right?
>>
>>
>> [bug#49946] [PATCH v7 02/32] gnu: tree-sitter: Update to 0.20.7.
>> id:20221125012142.22579-3-pierre.langlois@gmx.com
>> http://issues.guix.gnu.org/msgid/20221125012142.22579-3-pierre.langlois@gmx.com
>>
>> [bug#49946] [PATCH v7 05/32] gnu: Add tree-sitter-cli.
>> id:20221125012142.22579-6-pierre.langlois@gmx.com
>> http://issues.guix.gnu.org/msgid/20221125012142.22579-6-pierre.langlois@gmx.com
>>
>> Leaving out all the others, right?
>
> Merged first 5 patches from 01 to 05, also added one more commit, which
> addresses some things from reviews and one commit, which adds html
> grammar.
>
> The html grammar is added for the testing purposes. It relies on
> generated parser.c and scanner.c and we will need to repackage it using
> grammar.js instead. I'm not sure if a separate build system is needed
> for this, I guess we can just rewrite tree-sitter-grammar function,
> which generates packages as in example with tree-sitter-grammar-html:
> https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/tree-sitter.scm?h=53b00b91b73bd60412d5bd057e22e6d63194a7f7#n158
>
> Anyway, I only skimmed tree-sitter-build-system source code, and plan to
> read it carefully, evaluate and either introduce new build system or
> just move all needed parts to tree-sitter-grammar function. WDYT?
> After we done with it we can package all other grammars.

Ok, I realized that the proper build process for tree-sitter grammars is
a little harder than I expected, tree-sitter-build system make sense. I
reviewed it, made a small change:

Toggle snippet (20 lines)
@@ -29,7 +29,7 @@ (define-module (guix build tree-sitter-build-system)
;; Commentary:
;;
;; Build procedures for tree-sitter grammar packages. This is the
-;; builder-side code, which builds on top fo the node build-system.
+;; builder-side code, which builds on top of the node build-system.
;;
;; Tree-sitter grammars are written in JavaScript and compiled to a native
;; shared object. The `tree-sitter generate' command invokes `node' in order
@@ -114,7 +114,7 @@ (define (compile-language dir)
"-fno-exceptions"
"-O2"
"-g"
- "-o" ,(string-append lib "/" lang ".so")
+ "-o" ,(string-append lib "/libtree-sitter-" lang ".so")
;; An additional `scanner.{c,cc}' file is sometimes
;; provided.
,@(cond

rewrote html grammar to use this build system and made it work with
built-in treesit package. Also, tried examples of c and cpp grammars
from patches in this thread.

If you ok with it, I'll push the build system to master and update the
html grammar accordingly.

The final result will look like this:

Toggle snippet (44 lines)
(define tree-sitter-delete-generated-files
#~(begin
(delete-file "binding.gyp")
(delete-file-recursively "bindings")
(delete-file "src/grammar.json")
(delete-file "src/node-types.json")
(delete-file "src/parser.c")
(delete-file-recursively "src/tree_sitter")))

(define* (tree-sitter-grammar
language language-for-synopsis version commit hash
#:key
(repository-url
(format #f "https://github.com/tree-sitter/tree-sitter-~a" language))
(inputs '()))
(let ((synopsis (string-append language-for-synopsis
" grammar for tree-sitter"))
(name (string-append "tree-sitter-grammar-" language)))
(package
(name name)
(version version)
(home-page repository-url)
(source (origin
(method git-fetch)
(uri (git-reference
(url repository-url)
(commit commit)))
(file-name (git-file-name name version))
(sha256 (base32 hash))
(modules '((guix build utils)))
(snippet tree-sitter-delete-generated-files)))
(build-system tree-sitter-build-system)
(inputs inputs)
(synopsis synopsis)
(description (string-append synopsis "."))
(license license:expat))))

(define-public tree-sitter-grammar-html
(tree-sitter-grammar
"html" "HTML"
"0.19.0" "v0.19.0"
"1hg7vbcy7bir6b8x11v0a4x0glvqnsqc3i2ixiarbxmycbgl3axy"))

After that we can bring the rest of the grammars.

Toggle quote (6 lines)
>
> According to emacs integration: I already have a working prototype on
> my local setup, which uses builtin treesit package, it's similiar to
> what Luis mentioned with search-paths. We just need to figure out a
> few minor details and I'll add it.

I've sent a patch for emacs+tree-sitter integration in a different reply
in this thread.

Thank you very much for all the work, it's really valuable!

--
Best regards,
Andrew Tropin
-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEEKEGaxlA4dEDH6S/6IgjSCVjB3rAFAmPmPfYACgkQIgjSCVjB
3rBBlw/+JVj9+tV2fnaR8Sg7YKRiNtEEkun7gKRnvxm5JTewugn0334ZGowHdy0v
uUIoW2yZDibsXALRgJIb6LBFJ+oZ3S7i9xjOWaNK4u2GXQWjw7h6zjXBShLJeonU
CkhkcmpYrETAqUiYYkNCmD7GCYWarCuW6M59gVlfSkMHaWYw1Qze631kCOxDMXi6
ffmXBPyFqc7mk/tSTmOhyBQFsghnOi8o/64/bZUAviPvzfUnn/N27T4NoZXLEhjG
xlspWlkYCC090zfhc59APZv9f39hXgS2SmE7gS16nwdVQ7lMMTpryiCTczzbfAhr
OsjYzWhGqWrcile0m9+f/aY5bGUhwS2CgeYvao1N3CPYQrjqIKfjL4ZchWwNjGQd
DtGioV+ejMtF8A28b3Yqs50LFTntpbUyyNHF3lFoAmPMRXD1CQFk8uUfYn/ev+1f
qVlcI/rNZaFyxBUJfMPjah/s4Ea4IL/HkUzagdXHYpoYMVOWDXMncBv67PTFOMfT
cG3lvdsvNofFy9MD4XLtOxtgvJVTtA59XZxHaIG76qxN22fYS8Xw4qHJKbdb3PjP
wWWTyD1FoXXkU7aoRmTv1FeuPhTo3GlzkTCuJGO4UNE718raP/bA9CfhdDaAdJBu
BdMfT1zfGauSM4DfyxpJjrdr+DuPUNxPFXYTw1yPtRme1aCIy4s=
=WXQy
-----END PGP SIGNATURE-----

P
P
Pierre Langlois wrote on 10 Feb 2023 16:48
(name . Andrew Tropin)(address . andrew@trop.in)
87o7q1qzol.fsf@gmx.com
Hi Andrew, thanks for pushing this along! It's great to see things
getting merged.

Andrew Tropin <andrew@trop.in> writes:

Toggle quote (45 lines)
> [[PGP Signed Part:Undecided]]
> On 2023-02-09 18:04, Andrew Tropin wrote:
>
>> On 2023-02-09 13:39, zimoun wrote:
>>
>>> Hi,
>>>
>>> On Thu, 09 Feb 2023 at 14:11, Andrew Tropin <andrew@trop.in> wrote:
>>>
>>>> I applied tree-sitter and tree-sitter-cli patches,
>>>
>>> Just to be sure to understand, you have only applied 02/32 and 05/32,
>>> right?
>>>
>>>
>>> [bug#49946] [PATCH v7 02/32] gnu: tree-sitter: Update to 0.20.7.
>>> id:20221125012142.22579-3-pierre.langlois@gmx.com
>>> http://issues.guix.gnu.org/msgid/20221125012142.22579-3-pierre.langlois@gmx.com
>>>
>>> [bug#49946] [PATCH v7 05/32] gnu: Add tree-sitter-cli.
>>> id:20221125012142.22579-6-pierre.langlois@gmx.com
>>> http://issues.guix.gnu.org/msgid/20221125012142.22579-6-pierre.langlois@gmx.com
>>>
>>> Leaving out all the others, right?
>>
>> Merged first 5 patches from 01 to 05, also added one more commit, which
>> addresses some things from reviews and one commit, which adds html
>> grammar.
>>
>> The html grammar is added for the testing purposes. It relies on
>> generated parser.c and scanner.c and we will need to repackage it using
>> grammar.js instead. I'm not sure if a separate build system is needed
>> for this, I guess we can just rewrite tree-sitter-grammar function,
>> which generates packages as in example with tree-sitter-grammar-html:
>> https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/tree-sitter.scm?h=53b00b91b73bd60412d5bd057e22e6d63194a7f7#n158
>>
>> Anyway, I only skimmed tree-sitter-build-system source code, and plan to
>> read it carefully, evaluate and either introduce new build system or
>> just move all needed parts to tree-sitter-grammar function. WDYT?
>> After we done with it we can package all other grammars.
>
> Ok, I realized that the proper build process for tree-sitter grammars is
> a little harder than I expected, tree-sitter-build system make sense. I
> reviewed it, made a small change:

Ah great, I was going to comment to try and push for us to keep the
build system. I originally went with a template package and inheritance,
but Maxime suggested moving to a build-system which ended up making the
package definitions a *lot* nicer IMO (see previous discussion here
https://issues.guix.gnu.org/49946#144).It also allows us to deal with
grammars that depend on each other more nicely I think.

Toggle quote (75 lines)
>
> @@ -29,7 +29,7 @@ (define-module (guix build tree-sitter-build-system)
> ;; Commentary:
> ;;
> ;; Build procedures for tree-sitter grammar packages. This is the
> -;; builder-side code, which builds on top fo the node build-system.
> +;; builder-side code, which builds on top of the node build-system.
> ;;
> ;; Tree-sitter grammars are written in JavaScript and compiled to a native
> ;; shared object. The `tree-sitter generate' command invokes `node' in order
> @@ -114,7 +114,7 @@ (define (compile-language dir)
> "-fno-exceptions"
> "-O2"
> "-g"
> - "-o" ,(string-append lib "/" lang ".so")
> + "-o" ,(string-append lib "/libtree-sitter-" lang ".so")
> ;; An additional `scanner.{c,cc}' file is sometimes
> ;; provided.
> ,@(cond
>
>
> rewrote html grammar to use this build system and made it work with
> built-in treesit package. Also, tried examples of c and cpp grammars
> from patches in this thread.
>
> If you ok with it, I'll push the build system to master and update the
> html grammar accordingly.
>
> The final result will look like this:
>
> (define tree-sitter-delete-generated-files
> #~(begin
> (delete-file "binding.gyp")
> (delete-file-recursively "bindings")
> (delete-file "src/grammar.json")
> (delete-file "src/node-types.json")
> (delete-file "src/parser.c")
> (delete-file-recursively "src/tree_sitter")))
>
> (define* (tree-sitter-grammar
> language language-for-synopsis version commit hash
> #:key
> (repository-url
> (format #f "https://github.com/tree-sitter/tree-sitter-~a" language))
> (inputs '()))
> (let ((synopsis (string-append language-for-synopsis
> " grammar for tree-sitter"))
> (name (string-append "tree-sitter-grammar-" language)))
> (package
> (name name)
> (version version)
> (home-page repository-url)
> (source (origin
> (method git-fetch)
> (uri (git-reference
> (url repository-url)
> (commit commit)))
> (file-name (git-file-name name version))
> (sha256 (base32 hash))
> (modules '((guix build utils)))
> (snippet tree-sitter-delete-generated-files)))
> (build-system tree-sitter-build-system)
> (inputs inputs)
> (synopsis synopsis)
> (description (string-append synopsis "."))
> (license license:expat))))
>
> (define-public tree-sitter-grammar-html
> (tree-sitter-grammar
> "html" "HTML"
> "0.19.0" "v0.19.0"
> "1hg7vbcy7bir6b8x11v0a4x0glvqnsqc3i2ixiarbxmycbgl3axy"))
>
> After that we can bring the rest of the grammars.

I would suggest to rmeove the `tree-sitter-grammar' function, and keep
grammars as "regular" package records, even though it's a little bit
more verbose:

Toggle snippet (23 lines)
(define-public tree-sitter-html
(package
(name "tree-sitter-html")
(version "0.19.0")
(source (origin
(method git-fetch)
(uri (git-reference
(url "https://github.com/tree-sitter/tree-sitter-html")
(commit (string-append "v" version))))
(file-name (git-file-name name version))
(sha256
(base32
"1hg7vbcy7bir6b8x11v0a4x0glvqnsqc3i2ixiarbxmycbgl3axy"))
(modules '((guix build utils)))
(snippet tree-sitter-delete-generated-files)))
(build-system tree-sitter-build-system)
(home-page "https://github.com/tree-sitter/tree-sitter-html")
(synopsis "Tree-sitter HTML grammar")
(description
"This package provides a HTML grammar for the Tree-sitter library.")
(license license:expat)))

This way, they look like any other package in Guix, which makes it
easier for us to apply automatic changes in the future if needed (for
example like how the input format could be automically updated for all
"simple" package definitions, but had to be manual whenever custom code
refactoring was done). Does this make sense?

Toggle quote (12 lines)
>
>>
>> According to emacs integration: I already have a working prototype on
>> my local setup, which uses builtin treesit package, it's similiar to
>> what Luis mentioned with search-paths. We just need to figure out a
>> few minor details and I'll add it.
>
> I've sent a patch for emacs+tree-sitter integration in a different reply
> in this thread.
>
> Thank you very much for all the work, it's really valuable!

:-)

Thanks,
Pierre
-----BEGIN PGP SIGNATURE-----

iQFMBAEBCgA2FiEEctU9gYy29KFyWDdMqPyeRH9PfVQFAmPmadoYHHBpZXJyZS5s
YW5nbG9pc0BnbXguY29tAAoJEKj8nkR/T31UcrMIAIoyUPydzsRC71FQXoR04D20
mLjtBKgsGhP+GNkZZFIyJgr8GmxdUE2fAW/CCegUgigVpfOLCdBkMphp8v1W9ij7
l8PQktmn+vPuRnu6gbitrvfUlz0Mfa2E3t0fO3xdEV8TUpHbJLomShL7cRjU1IIy
H1gaUrNDv8F5zEY1D5lEmMWzek+za8BewO9u7EPUR8AMv7i1fnyvWFIsqNJt3Mwl
Scm4drDAA2cEbergnh8tBrLO3k87W6DoHV2ullhzCKCWWoHNEgff+sAoRmGu59B7
P942Jg2kekzKy+Hc8rl+c/2yuv/lEo6VpF6AKXWJBNqlLFkM+TKrys8jvnR0YMc=
=5EOX
-----END PGP SIGNATURE-----

P
P
Pierre Langlois wrote on 10 Feb 2023 17:14
Re: bug#49946: [PATCH 00/31] Tree-sitter, node-gyp addon support and emacs-tree-sitter
(name . zimoun)(address . zimon.toutoune@gmail.com)
87k00pqyu4.fsf@gmx.com
Hi!

zimoun <zimon.toutoune@gmail.com> writes:

Toggle quote (44 lines)
> Hi,
>
> Sorry, I have totally overlooked this tree-sitter story. :-) And I have
> not read all the patch versions and thread yet.
>
>
> On Fri, 25 Nov 2022 at 01:57, Pierre Langlois <pierre.langlois@gmx.com> wrote:
>
>> +(define* (install #:key target grammar-directories outputs #:allow-other-keys)
>> + (let ((lib (string-append (assoc-ref outputs "out")
>> + "/lib/tree-sitter")))
>> + (mkdir-p lib)
>> + (define (compile-language dir)
>> + (with-directory-excursion dir
>> + (let ((lang (assoc-ref (call-with-input-file "src/grammar.json"
>> + read-json)
>> + "name"))
>> + (source-file (lambda (path)
>> + (if (file-exists? path)
>> + path
>> + #f))))
>> + (apply invoke
>> + `(,(if target
>> + (string-append target "-g++")
>> + "g++")
>
> Why is g++ required? Is gcc not enough? I thought from Tree-sitter
> documentation that it was only using C files and not C++; but…
>
>> + "-shared"
>> + "-fPIC"
>> + "-fno-exceptions"
>> + "-O2"
>> + "-g"
>> + "-o" ,(string-append lib "/" lang ".so")
>> + ;; An additional `scanner.{c,cc}' file is sometimes
>> + ;; provided.
>> + ,@(cond
>> + ((source-file "src/scanner.c")
>> + => (lambda (file) (list "-xc" "-std=c99" file)))
>> + ((source-file "src/scanner.cc")
>
> …apparently not.

Yeah this is a little strange. IIUC, the final code generated from the
grammar.js->grammar.json->grammar.c process is plain C, but grammar
packages may also provide some custom hand-written code in terms of a
scanner.{c,cc} file.

Thanks,
Pierre
-----BEGIN PGP SIGNATURE-----

iQFMBAEBCgA2FiEEctU9gYy29KFyWDdMqPyeRH9PfVQFAmPmbiMYHHBpZXJyZS5s
YW5nbG9pc0BnbXguY29tAAoJEKj8nkR/T31Ufh8H/2wXw3F0BZ1URGv+OtovTkzA
lD0M1W0RWrY/Qr0mkq0kRjsnpGVj0gFxAAJ3trN6Cq9lhS1A9eEqjiJdfbBf5RUi
eO0p7PhtQb8yxjyV08NRIGcvYw0Hv0d96z1B0SICIqluTHHiyvJUP0qTmehzLEsy
DBade3OhY30ApjxIITQdCxjBpIqUf5ZXC5XIWzIfUAncymAMUm9/hIDSgemGU5zL
mdTd3Z+XZL7bJ5e/eyQ0m2cWWwDcknZEPdDSac4qCEBqWdTUAJj+hjVKRA93CcZL
j0N0cMrLo+3b/CHqMjjMcmUQbS4KDGhwAxsApeQj/52sB9lBEki8GJNfCaDwCbc=
=FBLh
-----END PGP SIGNATURE-----

P
P
Pierre Langlois wrote on 10 Feb 2023 17:22
Re: [bug#49946] [PATCH 00/31] Tree-sitter, node-gyp addon support and emacs-tree-sitter
(name . Andrew Tropin)(address . andrew@trop.in)
87zg9lo4qq.fsf@gmx.com
Hi,

Andrew Tropin <andrew@trop.in> writes:

Toggle quote (41 lines)
> [[PGP Signed Part:Undecided]]
> On 2021-08-08 23:29, Pierre Langlois wrote:
>
>> 4. Finally, emacs support!
>>
>> emacs-tree-sitter can be used for syntax highlighting, replacing
>> font-lock. The tree-sitter runtime library is built as a Rust crate
>> and exposed to emacs using a module thanks to the rust-emacs crate.
>>
>> Then emacs-tree-sitter-core provides just the core APIs as exposed by
>> the rust module, then emacs-tree-sitter integrates it with emacs'
>> syntax highlighting (and probably other things).
>>
>> Finally, we bundle together the various language grammars into a
>> single emacs-tree-sitter-langs package so that they can get loaded
>> along with their respective major modes. Note that there are grammars
>> left to support which I've not yet managed to package, but what we
>> have already is a good start I think :-). So far I've tested C++,
>> Rust, Python and JavaScript and JSON.
>>
>> * gnu: Add rust-bindgen@0.56.
>> * gnu: Add rust-tree-sitter.
>> * gnu: rust-emacs-module: Update to 0.16.
>> * gnu: rust-emacs-macros: Update to 0.17.
>> * gnu: rust-emacs: Update to 0.17.
>> * gnu: Add emacs-tree-sitter-core.
>> * gnu: Add emacs-tree-sitter.
>> * gnu: Add emacs-tree-sitter-langs.
>
> Hi Pierre!
>
> Implemented a loading of tree-sitter grammars in Emacs using guix
> native-search-paths and built-in treesit package. I did it for emacs
> package, but it will work only with Emacs 29 and greater (right now it's
> emacs-next and similiar packages), until emacs package updated to
> version 29 it won't take any effect on it except providing one more
> environment variable in guix profile.
>
> I tested it with emacs-next-pgtk, the patch is attached below, please
> let me know what do you think!

This is cool, thanks! I'm not familiar with how upstream emacs loads
grammars so I can't comment too much on the details. That being said, I
think it'd be good to get it to work with emacs 28 as well, I've been
using it for the past 6 months and it's working nicely. I'm happy to
take care of that part of the work though and let others focus on
emacs-next.

We have the following packages for emacs 28 support:

Toggle quote (9 lines)
>> * gnu: Add rust-bindgen@0.56.
>> * gnu: Add rust-tree-sitter.
>> * gnu: rust-emacs-module: Update to 0.16.
>> * gnu: rust-emacs-macros: Update to 0.17.
>> * gnu: rust-emacs: Update to 0.17.
>> * gnu: Add emacs-tree-sitter-core.
>> * gnu: Add emacs-tree-sitter.
>> * gnu: Add emacs-tree-sitter-langs.

Most of those would eventually go away I think, except probably
emacs-tree-sitter-langs, which is really useful to provide basic
tree-sitter syntax highlighint support to many language modes that may
not need to be changed themselves.

For instance, there is a PR to eventually move over to using the core

Toggle quote (57 lines)
>
> From b5ecd4e3734e9dd0bc76ebe95cab9c43aa85a3fe Mon Sep 17 00:00:00 2001
> From: Andrew Tropin <andrew@trop.in>
> Date: Fri, 10 Feb 2023 12:32:12 +0400
> Subject: [PATCH] gnu: emacs: Add TREE_SITTER_GRAMMAR_PATH support.
>
> gnu/packages/emacs.scm (emacs)[native-search-paths]: Add a search-path for
> tree-sitter grammars.
> gnu/packages/aux-files/emacs/guix-emacs.el: Add directories from
> TREE_SITTER_GRAMMAR_PATH to treesit-extra-load-path.
> ---
> gnu/packages/aux-files/emacs/guix-emacs.el | 7 +++++++
> gnu/packages/emacs.scm | 10 +++++++++-
> 2 files changed, 16 insertions(+), 1 deletion(-)
>
> diff --git a/gnu/packages/aux-files/emacs/guix-emacs.el b/gnu/packages/aux-files/emacs/guix-emacs.el
> index 56dbcb8d67..708093267d 100644
> --- a/gnu/packages/aux-files/emacs/guix-emacs.el
> +++ b/gnu/packages/aux-files/emacs/guix-emacs.el
> @@ -76,6 +76,13 @@ The files in the list do not have extensions (.el, .elc)."
> (when (file-directory-p pkg-dir)
> (package-load-descriptor pkg-dir)))))))))))
>
> +;; If emacs built with tree-sitter, read the value of the environment variable
> +;; to make tree-sitter grammars available in emacs out-of-the-box.
> +(with-eval-after-load 'treesit
> + (when-let ((grammar-path (getenv "TREE_SITTER_GRAMMAR_PATH")))
> + (mapcar (lambda (x) (add-to-list 'treesit-extra-load-path x))
> + (split-string grammar-path ":"))))
> +
> (provide 'guix-emacs)
>
> ;;; guix-emacs.el ends here
> diff --git a/gnu/packages/emacs.scm b/gnu/packages/emacs.scm
> index 4ce41deb88..a3cc9d2e22 100644
> --- a/gnu/packages/emacs.scm
> +++ b/gnu/packages/emacs.scm
> @@ -367,7 +367,15 @@ (define* (emacs-byte-compile-directory dir)
> (files '("lib/emacs/native-site-lisp")))
> (search-path-specification
> (variable "INFOPATH")
> - (files '("share/info")))))
> + (files '("share/info")))
> + ;; tree-sitter support is not yet available in emacs 28, but this
> + ;; search path won't harm and also will be benifitial for
> + ;; emacs-next and other emacs-* packages, which have tree-sitter
> + ;; support enabled. Please, remove this comment, when emacs
> + ;; package is updated to 29.
> + (search-path-specification
> + (variable "TREE_SITTER_GRAMMAR_PATH")
> + (files '("lib/tree-sitter")))))
>
> (home-page "https://www.gnu.org/software/emacs/")
> (synopsis "The extensible, customizable, self-documenting text editor")
> --
> 2.39.1

LGTM!


Thanks,
Pierre
-----BEGIN PGP SIGNATURE-----

iQFMBAEBCgA2FiEEctU9gYy29KFyWDdMqPyeRH9PfVQFAmPmcv4YHHBpZXJyZS5s
YW5nbG9pc0BnbXguY29tAAoJEKj8nkR/T31UMH4H/0Ax9b3V09Wzp5u2bNaEMwNf
LIRZO79oReySOCO8+TRQdtpIZrPXIbK+XbyTsgw48luezLOOEf7sE32tap05SiFj
ZyfpPuEbxDglw8vHNx/vo72DpklF0RdCNEWRSk4MbswALWXb301XroeOy6/GUMJq
170hVM9zjkrtt1cNbiO7+cFUQK+eK2Z3z646Wv/dOYVMlOW1LBLDqcj3rEA1vn/m
koLZthidyeAIT4Inm0ze7rSyqyGLjjocpkT5VzL81El5B/Riqqrh3vuD6tCFB5uQ
/58Zu7PjhnVCcUskr/gwUNYFuIPggHWSIn4yOmI5wCuVB+sU9+6IDYyHjGvfkyI=
=nLvU
-----END PGP SIGNATURE-----

P
P
Pierre Langlois wrote on 10 Feb 2023 18:02
Re: [bug#49946] [PATCH v7 01/32] gnu: tree-sitter: Move to its own module.
(name . Andrew Tropin)(address . andrew@trop.in)
87v8k9o3iz.fsf@gmx.com
Pierre Langlois <pierre.langlois@gmx.com> writes:

Toggle quote (86 lines)
> [[PGP Signed Part:Undecided]]
> Hi Andrew, thanks for pushing this along! It's great to see things
> getting merged.
>
> Andrew Tropin <andrew@trop.in> writes:
>
>> [[PGP Signed Part:Undecided]]
>> On 2023-02-09 18:04, Andrew Tropin wrote:
>>
>>> On 2023-02-09 13:39, zimoun wrote:
>>>
>>>> Hi,
>>>>
>>>> On Thu, 09 Feb 2023 at 14:11, Andrew Tropin <andrew@trop.in> wrote:
>>>>
>>>>> I applied tree-sitter and tree-sitter-cli patches,
>>>>
>>>> Just to be sure to understand, you have only applied 02/32 and 05/32,
>>>> right?
>>>>
>>>>
>>>> [bug#49946] [PATCH v7 02/32] gnu: tree-sitter: Update to 0.20.7.
>>>> id:20221125012142.22579-3-pierre.langlois@gmx.com
>>>> http://issues.guix.gnu.org/msgid/20221125012142.22579-3-pierre.langlois@gmx.com
>>>>
>>>> [bug#49946] [PATCH v7 05/32] gnu: Add tree-sitter-cli.
>>>> id:20221125012142.22579-6-pierre.langlois@gmx.com
>>>> http://issues.guix.gnu.org/msgid/20221125012142.22579-6-pierre.langlois@gmx.com
>>>>
>>>> Leaving out all the others, right?
>>>
>>> Merged first 5 patches from 01 to 05, also added one more commit, which
>>> addresses some things from reviews and one commit, which adds html
>>> grammar.
>>>
>>> The html grammar is added for the testing purposes. It relies on
>>> generated parser.c and scanner.c and we will need to repackage it using
>>> grammar.js instead. I'm not sure if a separate build system is needed
>>> for this, I guess we can just rewrite tree-sitter-grammar function,
>>> which generates packages as in example with tree-sitter-grammar-html:
>>> https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/tree-sitter.scm?h=53b00b91b73bd60412d5bd057e22e6d63194a7f7#n158
>>>
>>> Anyway, I only skimmed tree-sitter-build-system source code, and plan to
>>> read it carefully, evaluate and either introduce new build system or
>>> just move all needed parts to tree-sitter-grammar function. WDYT?
>>> After we done with it we can package all other grammars.
>>
>> Ok, I realized that the proper build process for tree-sitter grammars is
>> a little harder than I expected, tree-sitter-build system make sense. I
>> reviewed it, made a small change:
>
> Ah great, I was going to comment to try and push for us to keep the
> build system. I originally went with a template package and inheritance,
> but Maxime suggested moving to a build-system which ended up making the
> package definitions a *lot* nicer IMO (see previous discussion here
> https://issues.guix.gnu.org/49946#144). It also allows us to deal with
> grammars that depend on each other more nicely I think.
>
>>
>> @@ -29,7 +29,7 @@ (define-module (guix build tree-sitter-build-system)
>> ;; Commentary:
>> ;;
>> ;; Build procedures for tree-sitter grammar packages. This is the
>> -;; builder-side code, which builds on top fo the node build-system.
>> +;; builder-side code, which builds on top of the node build-system.
>> ;;
>> ;; Tree-sitter grammars are written in JavaScript and compiled to a native
>> ;; shared object. The `tree-sitter generate' command invokes `node' in order
>> @@ -114,7 +114,7 @@ (define (compile-language dir)
>> "-fno-exceptions"
>> "-O2"
>> "-g"
>> - "-o" ,(string-append lib "/" lang ".so")
>> + "-o" ,(string-append lib "/libtree-sitter-" lang ".so")
>> ;; An additional `scanner.{c,cc}' file is sometimes
>> ;; provided.
>> ,@(cond
>>
>>
>> rewrote html grammar to use this build system and made it work with
>> built-in treesit package. Also, tried examples of c and cpp grammars
>> from patches in this thread.
>>
>> If you ok with it, I'll push the build system to master and update the
>> html grammar accordingly.

Oh, I forgot to say, this change to the build system LGTM! I'm really
happy to see it merged soon :-). The path change will probably break the
emacs-28-based tree-sitter support, but that's OK, it's better for the
build-system to be made to target emacs 29's builtin support. I'm sure I
can work around for emacs 28.

Thanks,
Pierre
-----BEGIN PGP SIGNATURE-----

iQFMBAEBCgA2FiEEctU9gYy29KFyWDdMqPyeRH9PfVQFAmPmeSQYHHBpZXJyZS5s
YW5nbG9pc0BnbXguY29tAAoJEKj8nkR/T31U2NMIAKtMcusAtlfJdTuxpVzNPJCK
SMiWLULheY6nGPuNqTrsLA8ZmKsD572JJwLVuT4DMB8Vsf4a2FaL9NPAblTiOgsj
Ho9WXtdnQ966/sTz6TQLX2YV/7EkfCCAavDTMYrYtpeMnph8XC50/6dRPf65XwsU
35iifeIO6hIb8anTv1SWVxUtBFlMb8K0AILBfiCCWfhHP4gl6CHEYtCzzXIZs3iP
uDq9bt0l34YhfXLeO4hbIv+bzF37JauhnlW0lxDY4sRcH83C66sVLVr7eUUNiuBu
fanpQKDZDFwWANN+lhbUWOxw95RmMGu4/Cr3CW77r7ip+QA7zrw9z+VXUqI3URo=
=beyk
-----END PGP SIGNATURE-----

P
P
Pierre Langlois wrote on 10 Feb 2023 18:04
(name . Andrew Tropin)(address . andrew@trop.in)
87fsbdo34v.fsf@gmx.com
Hi,

Andrew Tropin <andrew@trop.in> writes:

Toggle quote (27 lines)
> [[PGP Signed Part:Undecided]]
> On 2023-02-09 13:39, zimoun wrote:
>
>> Hi,
>>
>> On Thu, 09 Feb 2023 at 14:11, Andrew Tropin <andrew@trop.in> wrote:
>>
>>> I applied tree-sitter and tree-sitter-cli patches,
>>
>> Just to be sure to understand, you have only applied 02/32 and 05/32,
>> right?
>>
>>
>> [bug#49946] [PATCH v7 02/32] gnu: tree-sitter: Update to 0.20.7.
>> id:20221125012142.22579-3-pierre.langlois@gmx.com
>> http://issues.guix.gnu.org/msgid/20221125012142.22579-3-pierre.langlois@gmx.com
>>
>> [bug#49946] [PATCH v7 05/32] gnu: Add tree-sitter-cli.
>> id:20221125012142.22579-6-pierre.langlois@gmx.com
>> http://issues.guix.gnu.org/msgid/20221125012142.22579-6-pierre.langlois@gmx.com
>>
>> Leaving out all the others, right?
>
> Merged first 5 patches from 01 to 05, also added one more commit, which
> addresses some things from reviews and one commit, which adds html
> grammar.

I had also started rebasing addressing comments on the tree-sitter-cli
package, and had a few extra changes left that haven't been pushed.

I realised we could remove a now redundant comment, and while enabling
*some* tests, I realized we should directly depend on node and dot, so
that people may use the `tree-sitter' command, without needing them in
the same profile.

What do you think of the following two patches?
-----BEGIN PGP SIGNATURE-----

iQFMBAEBCgA2FiEEctU9gYy29KFyWDdMqPyeRH9PfVQFAmPmeyAYHHBpZXJyZS5s
YW5nbG9pc0BnbXguY29tAAoJEKj8nkR/T31UX5kIAIrZysgv8hWcXNW4Jsu5EcWn
wok8kyxdTQWA+WbISavEq142olVZSGNmrJhBLv/EvciUCyVnxBFH+U8ru50d00al
2kvqE1agfVRD7UwrZy19liPfH8rWGP1r8INRwuUR3SHiyIXv7ANhef1iFtTKMWNx
4mB8aVyPkV5jz1WQom6PIhpU/yx0xijSq6dL83qQTccPPdjcBARrjasv29ctNbOe
ilnUJdroSd+72lJApV6mieLbYTLSFMsiJIPctX/ZnSorLPcLdEjZXSBUD6RC0Qbw
2LnavQAHyOnQLNskpBg1w/9Qf2H6ia/wTkL0gWkNQYAcKDVKo7Rxa0LFZtsuLAA=
=753I
-----END PGP SIGNATURE-----

From b24b04015261cfea2143be63671877c9c2c8d458 Mon Sep 17 00:00:00 2001
From: Pierre Langlois <pierre.langlois@gmx.com>
Date: Fri, 10 Feb 2023 16:10:25 +0000
Subject: [PATCH 1/2] gnu: Remove tree-sitter comment from emacs.scm imports.

* gnu/packages/emacs.scm: Remove tree-sitter comment.
---
gnu/packages/emacs.scm | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

Toggle diff (15 lines)
diff --git a/gnu/packages/emacs.scm b/gnu/packages/emacs.scm
index 4ce41deb88..4791570d12 100644
--- a/gnu/packages/emacs.scm
+++ b/gnu/packages/emacs.scm
@@ -76,7 +76,7 @@ (define-module (gnu packages emacs)
#:use-module (gnu packages sqlite)
#:use-module (gnu packages texinfo)
#:use-module (gnu packages tls)
- #:use-module (gnu packages tree-sitter) ; for tree-sitter
+ #:use-module (gnu packages tree-sitter)
#:use-module (gnu packages web) ; for jansson
#:use-module (gnu packages webkit)
#:use-module (gnu packages xml)
--
2.39.1
From 8225d51edeb528c51d30ad0f225eb96be5586a37 Mon Sep 17 00:00:00 2001
From: Pierre Langlois <pierre.langlois@gmx.com>
Date: Tue, 29 Mar 2022 20:13:11 +0100
Subject: [PATCH 2/2] gnu: tree-sitter-cli: Enable some tests and add node and
dot.

* gnu/packages/tree-sitter.scm (tree-sitter-cli)[inputs]: Add graphviz and
node-lts.
[arguments]<#:cargo-test-flags>: Skip tests that require downloading grammars.
<#:phases>: Add 'patch-node and 'patch-dot phases. Tweak install phase.
---
gnu/packages/tree-sitter.scm | 49 ++++++++++++++++++++++++++++--------
1 file changed, 39 insertions(+), 10 deletions(-)

Toggle diff (79 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 2e44339ca2..4331ffdd97 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -24,7 +24,9 @@ (define-module (gnu packages tree-sitter)
#:use-module ((guix licenses) #:prefix license:)
#:use-module (gnu packages crates-graphics)
#:use-module (gnu packages crates-io)
+ #:use-module (gnu packages graphviz)
#:use-module (gnu packages icu4c)
+ #:use-module (gnu packages node)
#:use-module (guix build-system cargo)
#:use-module (guix build-system gnu)
#:use-module (guix gexp)
@@ -96,13 +98,30 @@ (define-public tree-sitter-cli
println!(\"cargo:rustc-link-lib=tree-sitter\");~@
}~%")))))))
(build-system cargo-build-system)
- (inputs (list tree-sitter))
+ (inputs
+ (list tree-sitter graphviz node-lts))
(arguments
(list
- ;; Running test requires downloading fixtures, see the
- ;; script/fetch-fixtures script, which fetches grammars. Maybe it make
- ;; sence to run tests in the grammar's packages?
- #:tests? #f
+ #:cargo-test-flags
+ ''("--release" "--"
+ ;; Skip tests which rely on downloading grammar fixtures. It is
+ ;; difficult to support such tests given upstream does not encode
+ ;; which version of the grammars are expected.
+ ;; Instead, we do run some tests for each grammar in the tree-sitter
+ ;; build-system, by running `tree-sitter test'. This isn't as
+ ;; complete as running all tests from tree-sitter-cli, but it's a
+ ;; good compromise compared to maintaining two different sets of
+ ;; grammars (Guix packages vs test fixtures).
+ "--skip=tests::corpus_test"
+ "--skip=tests::highlight_test"
+ "--skip=tests::node_test"
+ "--skip=tests::parser_test"
+ "--skip=tests::pathological_test"
+ "--skip=tests::query_test"
+ "--skip=tests::tags_test"
+ "--skip=tests::test_highlight_test"
+ "--skip=tests::test_tags_test"
+ "--skip=tests::tree_test")
;; We're only packaging the CLI program so we do not need to install
;; sources.
#:install-source? #f
@@ -132,11 +151,21 @@ (define-public tree-sitter-cli
(add-after 'unpack 'delete-cargo-lock
(lambda _
(delete-file "Cargo.lock")))
- (replace 'install
- (lambda* (#:key outputs #:allow-other-keys)
- (let ((bin (string-append #$output "/bin")))
- (mkdir-p bin)
- (install-file "target/release/tree-sitter" bin)))))))
+ (add-after 'unpack 'patch-node
+ (lambda _
+ (substitute* "cli/src/generate/mod.rs"
+ (("Command::new\\(\"node\"\\)")
+ (string-append "Command::new(\"" #$node-lts "/bin/node\")")))))
+ (add-after 'unpack 'patch-dot
+ (lambda _
+ (substitute* "cli/src/util.rs"
+ (("Command::new\\(\"dot\"\\)")
+ (string-append "Command::new(\"" #$graphviz "/bin/dot\")")))))
+ (replace 'install
+ (lambda _
+ (let ((bin (string-append #$output "/bin")))
+ (mkdir-p bin)
+ (install-file "target/release/tree-sitter" bin)))))))
(description "Tree-sitter is a parser generator tool and an incremental
parsing library. It can build a concrete syntax tree for a source file and
efficiently update the syntax tree as the source file is edited.
--
2.39.1
Thanks,
Pierre
L
L
Liliana Marie Prikler wrote on 10 Feb 2023 19:02
Re: [bug#49946] [PATCH 00/31] Tree-sitter, node-gyp addon support and emacs-tree-sitter
5759483f7ac184a50a316401a9cc2f67d1719506.camel@gmail.com
Am Freitag, dem 10.02.2023 um 13:21 +0400 schrieb Andrew Tropin:

Toggle quote (8 lines)
> + ;; tree-sitter support is not yet available in emacs 28,
> but this
> + ;; search path won't harm and also will be benifitial for
> + ;; emacs-next and other emacs-* packages, which have
> tree-sitter
> + ;; support enabled. Please, remove this comment, when
> emacs
> + ;; package is updated to 29.
IIRC it's "beneficial". Otherwise LGTM
A
A
Andrew Tropin wrote on 11 Feb 2023 09:12
87357cipsr.fsf@trop.in
On 2023-02-10 19:02, Liliana Marie Prikler wrote:

Toggle quote (12 lines)
> Am Freitag, dem 10.02.2023 um 13:21 +0400 schrieb Andrew Tropin:
>
>> + ;; tree-sitter support is not yet available in emacs 28,
>> but this
>> + ;; search path won't harm and also will be benifitial for
>> + ;; emacs-next and other emacs-* packages, which have
>> tree-sitter
>> + ;; support enabled. Please, remove this comment, when
>> emacs
>> + ;; package is updated to 29.
> IIRC it's "beneficial". Otherwise LGTM

Nice catch, thank you very much, pushed as
c77a3d4554175ee5320ccc713aa21b1ba3e6b8a3

--
Best regards,
Andrew Tropin
-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEEKEGaxlA4dEDH6S/6IgjSCVjB3rAFAmPnTeQACgkQIgjSCVjB
3rCZJQ//RH8pLMpK4PzGyyOrxNotwGD8OZUAI5GMbOFa30W7e+8B+HdPPsLqZFXK
iLDo9v1FH5llI6Mw35xT9OPchw+yHtrv6+a3sn9zavNP+tW5P/oqrI3djjiNHHET
Og8hYKC+ZjJ9jJFocW8hIyMFhcsc+YmNgTrSIhoZe2rzGCIJUPOik7oSHFg3nnji
Pp2ZjUMiocisCOZix0i3AhAwm+lx9xmlrObNyUFGyPs+Sv3LWkKBGsX28FzfoHaX
VBhVdwldUBGFHyS8htlDHDCScjz8ZbwJ9bBjX8C2yLkbajzqrHhvwIfwVM0mg6Bc
g4JH/J7XSI2LONvJRRmD3lOC00NiF9ko6Q0cPpj8Tg87Dsvh9bBrnEm56g6AjOYA
qzJ634kH2Ik7zKoFY4xiKzER/FCJR5Ci0zfLrX3udCM53UP7KkGolPB6RnIVBQiv
KyeEs3ZhF/Hkh9KwuuC+xIVRKO1tC/+cgwv5ldzLA2u4IFOmsbUfDYwMM1Qjztgm
FPl7p/drMH3qREOmrT5HUCplxs0wqLUKX3PlNBZy/uErJ5SKkiiU68xddu9DdtA9
2tG0g4SrVBGRjBis5DvGg9RtNee6apuJFjug+mOXZMNkO8O2WgtDE4OBVGXMoPlX
vPAdMdURrc2iBbrxIoQVQZ/4zMoyAxs2rS6OplCezewFA5M6yiw=
=EYYK
-----END PGP SIGNATURE-----

A
A
Andrew Tropin wrote on 12 Feb 2023 06:55
Re: [bug#49946] [PATCH v7 01/32] gnu: tree-sitter: Move to its own module.
(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
87mt5jfmvl.fsf@trop.in
On 2023-02-10 15:48, Pierre Langlois wrote:

Toggle quote (141 lines)
> Hi Andrew, thanks for pushing this along! It's great to see things
> getting merged.
>
> Andrew Tropin <andrew@trop.in> writes:
>
>> [[PGP Signed Part:Undecided]]
>> On 2023-02-09 18:04, Andrew Tropin wrote:
>>
>>> On 2023-02-09 13:39, zimoun wrote:
>>>
>>>> Hi,
>>>>
>>>> On Thu, 09 Feb 2023 at 14:11, Andrew Tropin <andrew@trop.in> wrote:
>>>>
>>>>> I applied tree-sitter and tree-sitter-cli patches,
>>>>
>>>> Just to be sure to understand, you have only applied 02/32 and 05/32,
>>>> right?
>>>>
>>>>
>>>> [bug#49946] [PATCH v7 02/32] gnu: tree-sitter: Update to 0.20.7.
>>>> id:20221125012142.22579-3-pierre.langlois@gmx.com
>>>> http://issues.guix.gnu.org/msgid/20221125012142.22579-3-pierre.langlois@gmx.com
>>>>
>>>> [bug#49946] [PATCH v7 05/32] gnu: Add tree-sitter-cli.
>>>> id:20221125012142.22579-6-pierre.langlois@gmx.com
>>>> http://issues.guix.gnu.org/msgid/20221125012142.22579-6-pierre.langlois@gmx.com
>>>>
>>>> Leaving out all the others, right?
>>>
>>> Merged first 5 patches from 01 to 05, also added one more commit, which
>>> addresses some things from reviews and one commit, which adds html
>>> grammar.
>>>
>>> The html grammar is added for the testing purposes. It relies on
>>> generated parser.c and scanner.c and we will need to repackage it using
>>> grammar.js instead. I'm not sure if a separate build system is needed
>>> for this, I guess we can just rewrite tree-sitter-grammar function,
>>> which generates packages as in example with tree-sitter-grammar-html:
>>> https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/tree-sitter.scm?h=53b00b91b73bd60412d5bd057e22e6d63194a7f7#n158
>>>
>>> Anyway, I only skimmed tree-sitter-build-system source code, and plan to
>>> read it carefully, evaluate and either introduce new build system or
>>> just move all needed parts to tree-sitter-grammar function. WDYT?
>>> After we done with it we can package all other grammars.
>>
>> Ok, I realized that the proper build process for tree-sitter grammars is
>> a little harder than I expected, tree-sitter-build system make sense. I
>> reviewed it, made a small change:
>
> Ah great, I was going to comment to try and push for us to keep the
> build system. I originally went with a template package and inheritance,
> but Maxime suggested moving to a build-system which ended up making the
> package definitions a *lot* nicer IMO (see previous discussion here
> https://issues.guix.gnu.org/49946#144). It also allows us to deal with
> grammars that depend on each other more nicely I think.
>
>>
>> @@ -29,7 +29,7 @@ (define-module (guix build tree-sitter-build-system)
>> ;; Commentary:
>> ;;
>> ;; Build procedures for tree-sitter grammar packages. This is the
>> -;; builder-side code, which builds on top fo the node build-system.
>> +;; builder-side code, which builds on top of the node build-system.
>> ;;
>> ;; Tree-sitter grammars are written in JavaScript and compiled to a native
>> ;; shared object. The `tree-sitter generate' command invokes `node' in order
>> @@ -114,7 +114,7 @@ (define (compile-language dir)
>> "-fno-exceptions"
>> "-O2"
>> "-g"
>> - "-o" ,(string-append lib "/" lang ".so")
>> + "-o" ,(string-append lib "/libtree-sitter-" lang ".so")
>> ;; An additional `scanner.{c,cc}' file is sometimes
>> ;; provided.
>> ,@(cond
>>
>>
>> rewrote html grammar to use this build system and made it work with
>> built-in treesit package. Also, tried examples of c and cpp grammars
>> from patches in this thread.
>>
>> If you ok with it, I'll push the build system to master and update the
>> html grammar accordingly.
>>
>> The final result will look like this:
>>
>> (define tree-sitter-delete-generated-files
>> #~(begin
>> (delete-file "binding.gyp")
>> (delete-file-recursively "bindings")
>> (delete-file "src/grammar.json")
>> (delete-file "src/node-types.json")
>> (delete-file "src/parser.c")
>> (delete-file-recursively "src/tree_sitter")))
>>
>> (define* (tree-sitter-grammar
>> language language-for-synopsis version commit hash
>> #:key
>> (repository-url
>> (format #f "https://github.com/tree-sitter/tree-sitter-~a" language))
>> (inputs '()))
>> (let ((synopsis (string-append language-for-synopsis
>> " grammar for tree-sitter"))
>> (name (string-append "tree-sitter-grammar-" language)))
>> (package
>> (name name)
>> (version version)
>> (home-page repository-url)
>> (source (origin
>> (method git-fetch)
>> (uri (git-reference
>> (url repository-url)
>> (commit commit)))
>> (file-name (git-file-name name version))
>> (sha256 (base32 hash))
>> (modules '((guix build utils)))
>> (snippet tree-sitter-delete-generated-files)))
>> (build-system tree-sitter-build-system)
>> (inputs inputs)
>> (synopsis synopsis)
>> (description (string-append synopsis "."))
>> (license license:expat))))
>>
>> (define-public tree-sitter-grammar-html
>> (tree-sitter-grammar
>> "html" "HTML"
>> "0.19.0" "v0.19.0"
>> "1hg7vbcy7bir6b8x11v0a4x0glvqnsqc3i2ixiarbxmycbgl3axy"))
>>
>> After that we can bring the rest of the grammars.
>
> I would suggest to rmeove the `tree-sitter-grammar' function, and keep
> grammars as "regular" package records, even though it's a little bit
> more verbose:
>
> --8<---------------cut here---------------start------------->8---
> (define-public tree-sitter-html
> (package
> (name "tree-sitter-html")

It seems tree-sitter-html mimics upstream package name and probably make
more sense than tree-sitter-grammar-html used by me.

Toggle quote (26 lines)
> (version "0.19.0")
> (source (origin
> (method git-fetch)
> (uri (git-reference
> (url "https://github.com/tree-sitter/tree-sitter-html")
> (commit (string-append "v" version))))
> (file-name (git-file-name name version))
> (sha256
> (base32
> "1hg7vbcy7bir6b8x11v0a4x0glvqnsqc3i2ixiarbxmycbgl3axy"))
> (modules '((guix build utils)))
> (snippet tree-sitter-delete-generated-files)))
> (build-system tree-sitter-build-system)
> (home-page "https://github.com/tree-sitter/tree-sitter-html")
> (synopsis "Tree-sitter HTML grammar")
> (description
> "This package provides a HTML grammar for the Tree-sitter library.")
> (license license:expat)))
> --8<---------------cut here---------------end--------------->8---
>
> This way, they look like any other package in Guix, which makes it
> easier for us to apply automatic changes in the future if needed (for
> example like how the input format could be automically updated for all
> "simple" package definitions, but had to be manual whenever custom code
> refactoring was done). Does this make sense?

Make sense, but on the other hand we already have hunspell, aspell
dictionaries and probably a few more others, which are very similiar in
spirit and we already have to keep in mind their existence on such
automatic code updates.

It looks that the packages differ only in url for the source code, lang
name and sometimes in inputs. Having template package function can make
management of shared parts more centralized, reduce possibility of
copy-paste mistakes, when the description wasn't updated and so on and
can reduce the amount of the code overall (which also reduces the change
of introducing an error).

I don't have a strong opinion on this topic, but leaning towards the
template function slightly more, however I'm completely ok with the
standalone package definitions as well. WDYT?

Toggle quote (19 lines)
>
>>
>>>
>>> According to emacs integration: I already have a working prototype on
>>> my local setup, which uses builtin treesit package, it's similiar to
>>> what Luis mentioned with search-paths. We just need to figure out a
>>> few minor details and I'll add it.
>>
>> I've sent a patch for emacs+tree-sitter integration in a different reply
>> in this thread.
>>
>> Thank you very much for all the work, it's really valuable!
>
> :-)
>
> Thanks,
> Pierre
>

--
Best regards,
Andrew Tropin
-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEEKEGaxlA4dEDH6S/6IgjSCVjB3rAFAmPof24ACgkQIgjSCVjB
3rDSmw/+Need86jXW8YJuPd3UOS/Hb6S7YGCi9Dibxf8EOGfSxd5+YRo5JYNKGlh
+xCPSfU6wEEbxH1zVY0AejsJkJxhizK0jDkTn4ACQImi/FpmWKrpf/UpiMIsMdhw
o7OUY20sZNs0Xb3WSLLb5azjCfj0teEzMEBT/vmL0pGnzFdEwDQb24aE4euQB3LJ
0Kyyh+V6RbV6AtQHbeAYVMdDlPSkzgBw7uAhvafmgLD+tObP+1G4oP/QTt+VaJCN
Sqtmwz8nzTzMkn4KaWux1w0/kxefM8VbIWe16XPZiGG2A2Tjtyovl+F8Y1NmfdBQ
6+Ljxr0OgBBMOh62UHR5CAMc0celSQEl2gGbon9Sk08YDi1xq4j1v3a3h/f8ZIR5
Pou0VI8wHJaU/2UFo8yDQkCotnHFQT0UcY8hdy1gPDYVf9U0hOwoKonjDBmPCmCK
srwwlEI6RyAfza14lCHd/i/HumD8BxmBZ6ZTRqkeqXFMFKl2XllyQfsqpoO1sfn5
0cDxnjKgfaJ9dVcXAKRnDbH0oTWmxIfbyF96MqRT+t/cZNWr1tQ5Tp1+ByHCY5S8
5ETp91rnSt9xR+IvKvZbbP33PwRTpTMay1QsJSYK1YJCjDMEZMguEAlO8hMNJrB/
ygQUnYs87q4R1GfyJF0a0eHb2HQyTI5acpSRx4b+Jg4+TlOc34A=
=WmHB
-----END PGP SIGNATURE-----

A
A
Andrew Tropin wrote on 12 Feb 2023 07:05
(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
87k00nfmfp.fsf@trop.in
On 2023-02-10 17:02, Pierre Langlois wrote:

Toggle quote (94 lines)
> Pierre Langlois <pierre.langlois@gmx.com> writes:
>
>> [[PGP Signed Part:Undecided]]
>> Hi Andrew, thanks for pushing this along! It's great to see things
>> getting merged.
>>
>> Andrew Tropin <andrew@trop.in> writes:
>>
>>> [[PGP Signed Part:Undecided]]
>>> On 2023-02-09 18:04, Andrew Tropin wrote:
>>>
>>>> On 2023-02-09 13:39, zimoun wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> On Thu, 09 Feb 2023 at 14:11, Andrew Tropin <andrew@trop.in> wrote:
>>>>>
>>>>>> I applied tree-sitter and tree-sitter-cli patches,
>>>>>
>>>>> Just to be sure to understand, you have only applied 02/32 and 05/32,
>>>>> right?
>>>>>
>>>>>
>>>>> [bug#49946] [PATCH v7 02/32] gnu: tree-sitter: Update to 0.20.7.
>>>>> id:20221125012142.22579-3-pierre.langlois@gmx.com
>>>>> http://issues.guix.gnu.org/msgid/20221125012142.22579-3-pierre.langlois@gmx.com
>>>>>
>>>>> [bug#49946] [PATCH v7 05/32] gnu: Add tree-sitter-cli.
>>>>> id:20221125012142.22579-6-pierre.langlois@gmx.com
>>>>> http://issues.guix.gnu.org/msgid/20221125012142.22579-6-pierre.langlois@gmx.com
>>>>>
>>>>> Leaving out all the others, right?
>>>>
>>>> Merged first 5 patches from 01 to 05, also added one more commit, which
>>>> addresses some things from reviews and one commit, which adds html
>>>> grammar.
>>>>
>>>> The html grammar is added for the testing purposes. It relies on
>>>> generated parser.c and scanner.c and we will need to repackage it using
>>>> grammar.js instead. I'm not sure if a separate build system is needed
>>>> for this, I guess we can just rewrite tree-sitter-grammar function,
>>>> which generates packages as in example with tree-sitter-grammar-html:
>>>> https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/tree-sitter.scm?h=53b00b91b73bd60412d5bd057e22e6d63194a7f7#n158
>>>>
>>>> Anyway, I only skimmed tree-sitter-build-system source code, and plan to
>>>> read it carefully, evaluate and either introduce new build system or
>>>> just move all needed parts to tree-sitter-grammar function. WDYT?
>>>> After we done with it we can package all other grammars.
>>>
>>> Ok, I realized that the proper build process for tree-sitter grammars is
>>> a little harder than I expected, tree-sitter-build system make sense. I
>>> reviewed it, made a small change:
>>
>> Ah great, I was going to comment to try and push for us to keep the
>> build system. I originally went with a template package and inheritance,
>> but Maxime suggested moving to a build-system which ended up making the
>> package definitions a *lot* nicer IMO (see previous discussion here
>> https://issues.guix.gnu.org/49946#144). It also allows us to deal with
>> grammars that depend on each other more nicely I think.
>>
>>>
>>> @@ -29,7 +29,7 @@ (define-module (guix build tree-sitter-build-system)
>>> ;; Commentary:
>>> ;;
>>> ;; Build procedures for tree-sitter grammar packages. This is the
>>> -;; builder-side code, which builds on top fo the node build-system.
>>> +;; builder-side code, which builds on top of the node build-system.
>>> ;;
>>> ;; Tree-sitter grammars are written in JavaScript and compiled to a native
>>> ;; shared object. The `tree-sitter generate' command invokes `node' in order
>>> @@ -114,7 +114,7 @@ (define (compile-language dir)
>>> "-fno-exceptions"
>>> "-O2"
>>> "-g"
>>> - "-o" ,(string-append lib "/" lang ".so")
>>> + "-o" ,(string-append lib "/libtree-sitter-" lang ".so")
>>> ;; An additional `scanner.{c,cc}' file is sometimes
>>> ;; provided.
>>> ,@(cond
>>>
>>>
>>> rewrote html grammar to use this build system and made it work with
>>> built-in treesit package. Also, tried examples of c and cpp grammars
>>> from patches in this thread.
>>>
>>> If you ok with it, I'll push the build system to master and update the
>>> html grammar accordingly.
>
> Oh, I forgot to say, this change to the build system LGTM! I'm really
> happy to see it merged soon :-). The path change will probably break the
> emacs-28-based tree-sitter support, but that's OK, it's better for the
> build-system to be made to target emacs 29's builtin support. I'm sure I
> can work around for emacs 28.

Actually, I think we can build grammars with both names, just providing
two .so files instead of one. If you won't find a better workaround we
can go this way.

--
Best regards,
Andrew Tropin
-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEEKEGaxlA4dEDH6S/6IgjSCVjB3rAFAmPogaoACgkQIgjSCVjB
3rCzYhAAhapiaBqqMI9uquCvZGkU4uQMxNu5pOQwndhO6o7tqdta1BI2N0new05f
G5STYsZWqap5hT/lFaUf6zvJhQx8ypAxnWGYRVUOuwqWbx3ryu+2uIGcueO0Dp57
sh8lnB3SrmrJKrtm3iDZBi+14kpPC+HG4rHDyva87MyPUpuc3O6pwKrTT+mbxbam
Ym3dj1M9+HJIU+/1/yj6ch5JxhgVKCiDdFZwRQCtdmXCEW3QnCdxjAzYfso6dxl8
fOsCWsCl5JdssfHwA+k4dlyyK5ljNDEWJpqc9uW7SvYGukQsi6UuXchUFcU9JFRj
9bC0txaMNh/B8c/Of6cQ+gub87htVLW7A7ylZcOJobkPqPwZutnOOvNfVt6yzIwg
vaEDrBPfg8rXSpiP1sr3/2OI5KQp6mEGXSEYMRlYrhD3Pe8KMYJgo4Q/yK7cSvVX
7OciKztHrnH79Uvx72ZOUagIkANM3eaeXYrYd5cnAIBtE/gfbE6iTxr995oFZc+x
iMTdbqgu2ha1KcAERsSDtRKvye+s3MTNay9ldqt6oIZhZ1QeCP2GRVt69FU6Jfd/
s7FYB2+FE/GD+aHvJaneZ+vTzSI00PFavOjPhmscNLAV3zENqpo6sbTzHcC2opE3
SLA4V2TIQRS2IoGLiPUKXpmeg9FmTmmXih/kk0NrkRpJEbgdTDg=
=vT2W
-----END PGP SIGNATURE-----

A
A
Andrew Tropin wrote on 12 Feb 2023 07:28
(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
87h6vrfldq.fsf@trop.in
On 2023-02-10 17:04, Pierre Langlois wrote:

Toggle quote (39 lines)
> Hi,
>
> Andrew Tropin <andrew@trop.in> writes:
>
>> [[PGP Signed Part:Undecided]]
>> On 2023-02-09 13:39, zimoun wrote:
>>
>>> Hi,
>>>
>>> On Thu, 09 Feb 2023 at 14:11, Andrew Tropin <andrew@trop.in> wrote:
>>>
>>>> I applied tree-sitter and tree-sitter-cli patches,
>>>
>>> Just to be sure to understand, you have only applied 02/32 and 05/32,
>>> right?
>>>
>>>
>>> [bug#49946] [PATCH v7 02/32] gnu: tree-sitter: Update to 0.20.7.
>>> id:20221125012142.22579-3-pierre.langlois@gmx.com
>>> http://issues.guix.gnu.org/msgid/20221125012142.22579-3-pierre.langlois@gmx.com
>>>
>>> [bug#49946] [PATCH v7 05/32] gnu: Add tree-sitter-cli.
>>> id:20221125012142.22579-6-pierre.langlois@gmx.com
>>> http://issues.guix.gnu.org/msgid/20221125012142.22579-6-pierre.langlois@gmx.com
>>>
>>> Leaving out all the others, right?
>>
>> Merged first 5 patches from 01 to 05, also added one more commit, which
>> addresses some things from reviews and one commit, which adds html
>> grammar.
>
> I had also started rebasing addressing comments on the tree-sitter-cli
> package, and had a few extra changes left that haven't been pushed.
>
> I realised we could remove a now redundant comment, and while enabling
> *some* tests, I realized we should directly depend on node and dot, so
> that people may use the `tree-sitter' command, without needing them in
> the same profile.

Sounds very valid! :)

Toggle quote (4 lines)
>
> What do you think of the following two patches?
>

I applied them locally and tested, LGTM. Will push them a little later
with the rest of tree-sitter related changes.

Toggle quote (118 lines)
> From b24b04015261cfea2143be63671877c9c2c8d458 Mon Sep 17 00:00:00 2001
> From: Pierre Langlois <pierre.langlois@gmx.com> Date: Fri, 10 Feb 2023
> 16:10:25 +0000 Subject: [PATCH 1/2] gnu: Remove tree-sitter comment
> from emacs.scm imports.
>
> * gnu/packages/emacs.scm: Remove tree-sitter comment.
> ---
> gnu/packages/emacs.scm | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/gnu/packages/emacs.scm b/gnu/packages/emacs.scm
> index 4ce41deb88..4791570d12 100644
> --- a/gnu/packages/emacs.scm
> +++ b/gnu/packages/emacs.scm
> @@ -76,7 +76,7 @@ (define-module (gnu packages emacs)
> #:use-module (gnu packages sqlite)
> #:use-module (gnu packages texinfo)
> #:use-module (gnu packages tls)
> - #:use-module (gnu packages tree-sitter) ; for tree-sitter
> + #:use-module (gnu packages tree-sitter)
> #:use-module (gnu packages web) ; for jansson
> #:use-module (gnu packages webkit)
> #:use-module (gnu packages xml)
> --
> 2.39.1
>
> From 8225d51edeb528c51d30ad0f225eb96be5586a37 Mon Sep 17 00:00:00 2001
> From: Pierre Langlois <pierre.langlois@gmx.com>
> Date: Tue, 29 Mar 2022 20:13:11 +0100
> Subject: [PATCH 2/2] gnu: tree-sitter-cli: Enable some tests and add node and
> dot.
>
> * gnu/packages/tree-sitter.scm (tree-sitter-cli)[inputs]: Add graphviz and
> node-lts.
> [arguments]<#:cargo-test-flags>: Skip tests that require downloading grammars.
> <#:phases>: Add 'patch-node and 'patch-dot phases. Tweak install phase.
> ---
> gnu/packages/tree-sitter.scm | 49 ++++++++++++++++++++++++++++--------
> 1 file changed, 39 insertions(+), 10 deletions(-)
>
> diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
> index 2e44339ca2..4331ffdd97 100644
> --- a/gnu/packages/tree-sitter.scm
> +++ b/gnu/packages/tree-sitter.scm
> @@ -24,7 +24,9 @@ (define-module (gnu packages tree-sitter)
> #:use-module ((guix licenses) #:prefix license:)
> #:use-module (gnu packages crates-graphics)
> #:use-module (gnu packages crates-io)
> + #:use-module (gnu packages graphviz)
> #:use-module (gnu packages icu4c)
> + #:use-module (gnu packages node)
> #:use-module (guix build-system cargo)
> #:use-module (guix build-system gnu)
> #:use-module (guix gexp)
> @@ -96,13 +98,30 @@ (define-public tree-sitter-cli
> println!(\"cargo:rustc-link-lib=tree-sitter\");~@
> }~%")))))))
> (build-system cargo-build-system)
> - (inputs (list tree-sitter))
> + (inputs
> + (list tree-sitter graphviz node-lts))
> (arguments
> (list
> - ;; Running test requires downloading fixtures, see the
> - ;; script/fetch-fixtures script, which fetches grammars. Maybe it make
> - ;; sence to run tests in the grammar's packages?
> - #:tests? #f
> + #:cargo-test-flags
> + ''("--release" "--"
> + ;; Skip tests which rely on downloading grammar fixtures. It is
> + ;; difficult to support such tests given upstream does not encode
> + ;; which version of the grammars are expected.
> + ;; Instead, we do run some tests for each grammar in the tree-sitter
> + ;; build-system, by running `tree-sitter test'. This isn't as
> + ;; complete as running all tests from tree-sitter-cli, but it's a
> + ;; good compromise compared to maintaining two different sets of
> + ;; grammars (Guix packages vs test fixtures).
> + "--skip=tests::corpus_test"
> + "--skip=tests::highlight_test"
> + "--skip=tests::node_test"
> + "--skip=tests::parser_test"
> + "--skip=tests::pathological_test"
> + "--skip=tests::query_test"
> + "--skip=tests::tags_test"
> + "--skip=tests::test_highlight_test"
> + "--skip=tests::test_tags_test"
> + "--skip=tests::tree_test")
> ;; We're only packaging the CLI program so we do not need to install
> ;; sources.
> #:install-source? #f
> @@ -132,11 +151,21 @@ (define-public tree-sitter-cli
> (add-after 'unpack 'delete-cargo-lock
> (lambda _
> (delete-file "Cargo.lock")))
> - (replace 'install
> - (lambda* (#:key outputs #:allow-other-keys)
> - (let ((bin (string-append #$output "/bin")))
> - (mkdir-p bin)
> - (install-file "target/release/tree-sitter" bin)))))))
> + (add-after 'unpack 'patch-node
> + (lambda _
> + (substitute* "cli/src/generate/mod.rs"
> + (("Command::new\\(\"node\"\\)")
> + (string-append "Command::new(\"" #$node-lts "/bin/node\")")))))
> + (add-after 'unpack 'patch-dot
> + (lambda _
> + (substitute* "cli/src/util.rs"
> + (("Command::new\\(\"dot\"\\)")
> + (string-append "Command::new(\"" #$graphviz "/bin/dot\")")))))
> + (replace 'install
> + (lambda _
> + (let ((bin (string-append #$output "/bin")))
> + (mkdir-p bin)
> + (install-file "target/release/tree-sitter" bin)))))))
> (description "Tree-sitter is a parser generator tool and an incremental
> parsing library. It can build a concrete syntax tree for a source file and
> efficiently update the syntax tree as the source file is edited.

--
Best regards,
Andrew Tropin
-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEEKEGaxlA4dEDH6S/6IgjSCVjB3rAFAmPohwEACgkQIgjSCVjB
3rDX8Q//fq1bUPBoBeQsdHaLPP9tiNPlptv9SBf3kR8etAwXfbM4p3+AZxmnUbwV
/JyHMqoNegDfpEya+7lGXkQ4vVGGgXx7HEK5DJ24wbXOWzUAOWAmAcXia5ssbKHx
bsbxbFiq4lBrtx7rj6Zbd4cXkgNXso/aMqH5EKZ+f0aXiLE7EkkB6K+gq8Hh2J0k
HrZg2+OrMxVP1xIub8MwqVbLpJLL5GSBlvh+DJ7HYW8ZTM6CI5tTkW7ai4fWXAJA
/2iXpm/kLBMeTTApcCzdWt7/jpDXNtNq7SXLPWHdAe5K0yHD30zk2SpaMyaJzW/5
LjUQrtuW8oLG6+FhzQCV7o37lIQesewHYdCExHayEjjpv56wlLUogkl8YJGDjFmr
se12rvlSKMEj07pdy3Z3aM6Wm9ci40ptBarK61SdNgWJI9vZExWnNwwVm7lviNm4
l3GdU2UuaWPiF/Ckp9JdODjmdKhA9q6qBkvUckoqxGuRmtszq89saI2UuzEuIclT
bTKxuhAaCPxbKK6ybYSJQj3RdtDaoqP9OPffVN95XUeXKoBvQKyPQZNEIeFbz7iv
/JmrJfCutSJfuDYsC3tmn/i6IWCghiB0SZAiQCxv5CdqNsS6dDNikFtksD0NtJvY
zfT9K42b77KseaN/YH7rZ5eXwutL8GVQ1KNd4hEyfUlGUuZdpWM=
=3LwD
-----END PGP SIGNATURE-----

A
A
Andrew Tropin wrote on 12 Feb 2023 08:42
Re: [bug#49946] [PATCH v7 06/32] build-system: Add tree-sitter-build-system.
(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
87edqvfhy7.fsf@trop.in
On 2022-11-25 01:57, Pierre Langlois wrote:

Toggle quote (436 lines)
> It appears this email is making it through, so here it is as an
> attachment:
>
> From 4a3c1fff8460a03bfb7c1aada9863205cd6f22fd Mon Sep 17 00:00:00 2001
> From: Pierre Langlois <pierre.langlois@gmx.com>
> Date: Tue, 29 Mar 2022 20:13:34 +0100
> Subject: [PATCH v7 06/32] build-system: Add tree-sitter-build-system.
>
> * guix/build-system/tree-sitter.scm: New module.
> * guix/build/tree-sitter-build-system.scm: Likewise.
> * Makefile.am (MODULES): Add them.
> * doc/guix.texi: Document it.
> ---
> Makefile.am | 2 +
> doc/guix.texi | 21 ++-
> guix/build-system/tree-sitter.scm | 190 ++++++++++++++++++++++++
> guix/build/tree-sitter-build-system.scm | 153 +++++++++++++++++++
> 4 files changed, 365 insertions(+), 1 deletion(-)
> create mode 100644 guix/build-system/tree-sitter.scm
> create mode 100644 guix/build/tree-sitter-build-system.scm
>
> diff --git a/Makefile.am b/Makefile.am
> index c3af23b68e..a16c4fcd7e 100644
> --- a/Makefile.am
> +++ b/Makefile.am
> @@ -178,6 +178,7 @@ MODULES = \
> guix/build-system/ruby.scm \
> guix/build-system/scons.scm \
> guix/build-system/texlive.scm \
> + guix/build-system/tree-sitter.scm \
> guix/build-system/trivial.scm \
> guix/ftp-client.scm \
> guix/http-client.scm \
> @@ -234,6 +235,7 @@ MODULES = \
> guix/build/ruby-build-system.scm \
> guix/build/scons-build-system.scm \
> guix/build/texlive-build-system.scm \
> + guix/build/tree-sitter-build-system.scm \
> guix/build/waf-build-system.scm \
> guix/build/haskell-build-system.scm \
> guix/build/julia-build-system.scm \
> diff --git a/doc/guix.texi b/doc/guix.texi
> index e547d469f4..4e997f7176 100644
> --- a/doc/guix.texi
> +++ b/doc/guix.texi
> @@ -79,7 +79,7 @@ Copyright @copyright{} 2020 Jack Hill@*
> Copyright @copyright{} 2020 Naga Malleswari@*
> Copyright @copyright{} 2020, 2021 Brice Waegeneire@*
> Copyright @copyright{} 2020 R Veera Kumar@*
> -Copyright @copyright{} 2020, 2021 Pierre Langlois@*
> +Copyright @copyright{} 2020, 2021, 2022 Pierre Langlois@*
> Copyright @copyright{} 2020 pinoaffe@*
> Copyright @copyright{} 2020 André Batista@*
> Copyright @copyright{} 2020, 2021 Alexandru-Sergiu Marton@*
> @@ -9732,6 +9732,25 @@ be specified with the @code{#:node} parameter which defaults to
> @code{node}.
> @end defvr
>
> +@defvr {Scheme Variable} tree-sitter-build-system
> +
> +This variable is exported by @code{(guix build-system tree-sitter)}. It
> +implements procedures to compile grammars for the
> +@url{https://tree-sitter.github.io/tree-sitter/, Tree-sitter} parsing
> +library. It essentially runs @code{tree-sitter generate} to translate
> +@code{grammar.js} grammars to JSON and then to C. Which it then
> +compiles to native code.
> +
> +Tree-sitter packages may support multiple grammars, so this build system
> +supports a @code{#:grammar-directories} keyword to specify a list of
> +locations where a @code{grammar.js} file may be found.
> +
> +Grammars sometimes depend on each other, such as C++ depending on C and
> +TypeScript depending on JavaScript. You may use inputs to declare such
> +dependencies.
> +
> +@end defvr
> +
> Lastly, for packages that do not need anything as sophisticated, a
> ``trivial'' build system is provided. It is trivial in the sense that
> it provides basically no support: it does not pull any implicit inputs,
> diff --git a/guix/build-system/tree-sitter.scm b/guix/build-system/tree-sitter.scm
> new file mode 100644
> index 0000000000..aeb96e3ef5
> --- /dev/null
> +++ b/guix/build-system/tree-sitter.scm
> @@ -0,0 +1,190 @@
> +;;; GNU Guix --- Functional package management for GNU
> +;;; Copyright © 2022 Pierre Langlois <pierre.langlois@gmx.com>
> +;;;
> +;;; This file is part of GNU Guix.
> +;;;
> +;;; GNU Guix is free software; you can redistribute it and/or modify it
> +;;; under the terms of the GNU General Public License as published by
> +;;; the Free Software Foundation; either version 3 of the License, or (at
> +;;; your option) any later version.
> +;;;
> +;;; GNU Guix is distributed in the hope that it will be useful, but
> +;;; WITHOUT ANY WARRANTY; without even the implied warranty of
> +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> +;;; GNU General Public License for more details.
> +;;;
> +;;; You should have received a copy of the GNU General Public License
> +;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
> +
> +(define-module (guix build-system tree-sitter)
> + #:use-module (guix store)
> + #:use-module (guix utils)
> + #:use-module (guix packages)
> + #:use-module (guix gexp)
> + #:use-module (guix monads)
> + #:use-module (guix search-paths)
> + #:use-module (guix build-system)
> + #:use-module (guix build-system gnu)
> + #:use-module (guix build-system node)
> + #:use-module (ice-9 match)
> + #:export (%tree-sitter-build-system-modules
> + tree-sitter-build
> + tree-sitter-build-system))
> +
> +(define %tree-sitter-build-system-modules
> + ;; Build-side modules imported by default.
> + `((guix build tree-sitter-build-system)
> + ,@%node-build-system-modules))
> +
> +(define* (lower name
> + #:key source inputs native-inputs outputs system target
> + #:allow-other-keys
> + #:rest arguments)
> + "Return a bag for NAME from the given arguments."
> + (define private-keywords
> + `(#:inputs #:native-inputs #:outputs ,@(if target
> + '()
> + '(#:target))))
> + (define node
> + (module-ref (resolve-interface '(gnu packages node))
> + 'node-lts))
> + (define tree-sitter
> + (module-ref (resolve-interface '(gnu packages tree-sitter))
> + 'tree-sitter))
> + (define tree-sitter-cli
> + (module-ref (resolve-interface '(gnu packages tree-sitter))
> + 'tree-sitter-cli))
> + ;; Grammars depend on each other via JS modules, which we package into a
> + ;; dedicated js output.
> + (define grammar-inputs
> + (map (match-lambda
> + ((name package)
> + `(,name ,package "js")))
> + inputs))
> + (bag
> + (name name)
> + (system system) (target target)
> + (build-inputs `(,@(if source
> + `(("source" ,source))
> + '())
> + ("node" ,node)
> + ("tree-sitter-cli" ,tree-sitter-cli)
> + ,@native-inputs
> + ,@(if target '() grammar-inputs)
> + ;; Keep the standard inputs of 'gnu-build-system'.
> + ,@(if target
> + (standard-cross-packages target 'host)
> + '())
> + ,@(standard-packages)))
> + (host-inputs `(("tree-sitter" ,tree-sitter)
> + ,@(if target grammar-inputs '())))
> + ;; Keep the standard inputs of 'gnu-buid-system'.
> + (target-inputs (if target
> + (standard-cross-packages target 'target)
> + '()))
> + ;; XXX: this is a hack to get around issue #41569.
> + (outputs (match outputs
> + (("out") (cons "js" outputs))
> + (_ outputs)))
> + (build (if target tree-sitter-cross-build tree-sitter-build))
> + (arguments (strip-keyword-arguments private-keywords arguments))))
> +
> +(define* (tree-sitter-build name inputs
> + #:key
> + source
> + (phases '%standard-phases)
> + (grammar-directories '("."))
> + (tests? #t)
> + (outputs '("out" "js"))
> + (search-paths '())
> + (system (%current-system))
> + (guile #f)
> + (imported-modules %tree-sitter-build-system-modules)
> + (modules '((guix build utils)
> + (guix build tree-sitter-build-system))))
> + (define builder
> + (with-imported-modules imported-modules
> + #~(begin
> + (use-modules #$@(sexp->gexp modules))
> + (tree-sitter-build #:name #$name
> + #:source #+source
> + #:system #$system
> + #:phases #$phases
> + #:tests? #$tests?
> + #:grammar-directories '#$grammar-directories
> + #:outputs #$(outputs->gexp outputs)
> + #:search-paths '#$(sexp->gexp
> + (map search-path-specification->sexp
> + search-paths))
> + #:inputs #$(input-tuples->gexp inputs)))))
> +
> + (mlet %store-monad ((guile (package->derivation (or guile (default-guile))
> + system #:graft? #f)))
> + (gexp->derivation name builder
> + #:system system
> + #:guile-for-build guile)))
> +
> +(define* (tree-sitter-cross-build name
> + #:key
> + target
> + build-inputs target-inputs host-inputs
> + guile source
> + (phases '%standard-phases)
> + (grammar-directories '("."))
> + (tests? #t)
> + (outputs '("out" "js"))
> + (search-paths '())
> + (native-search-paths '())
> + (system (%current-system))
> + (build (nix-system->gnu-triplet system))
> + (imported-modules %tree-sitter-build-system-modules)
> + (modules '((guix build utils)
> + (guix build tree-sitter-build-system))))
> + (define builder
> + (with-imported-modules imported-modules
> + #~(begin
> + (use-modules #$@(sexp->gexp modules))
> +
> + (define %build-host-inputs
> + #+(input-tuples->gexp build-inputs))
> +
> + (define %build-target-inputs
> + (append #$(input-tuples->gexp host-inputs)
> + #+(input-tuples->gexp target-inputs)))
> +
> + (define %build-inputs
> + (append %build-host-inputs %build-target-inputs))
> +
> + (tree-sitter-build #:name #$name
> + #:source #+source
> + #:system #$system
> + #:build #$build
> + #:target #$target
> + #:phases #$phases
> + #:tests? #$tests?
> + #:grammar-directories '#$grammar-directories
> + #:outputs #$(outputs->gexp outputs)
> + #:inputs %build-target-inputs
> + #:native-inputs %build-host-inputs
> + #:search-paths '#$(sexp->gexp
> + (map search-path-specification->sexp
> + search-paths))
> + #:native-search-paths '#$(sexp->gexp
> + (map
> + search-path-specification->sexp
> + native-search-paths))))))
> +
> + (mlet %store-monad ((guile (package->derivation (or guile (default-guile))
> + system #:graft? #f)))
> + (gexp->derivation name builder
> + #:system system
> + #:target target
> + #:guile-for-build guile)))
> +
> +(define tree-sitter-build-system
> + (build-system
> + (name 'tree-sitter)
> + (description "The Tree-sitter grammar build system")
> + (lower lower)))
> +
> +;;; tree-sitter.scm ends here
> diff --git a/guix/build/tree-sitter-build-system.scm b/guix/build/tree-sitter-build-system.scm
> new file mode 100644
> index 0000000000..574b0f2a1c
> --- /dev/null
> +++ b/guix/build/tree-sitter-build-system.scm
> @@ -0,0 +1,153 @@
> +;;; GNU Guix --- Functional package management for GNU
> +;;; Copyright © 2022 Pierre Langlois <pierre.langlois@gmx.com>
> +;;;
> +;;; This file is part of GNU Guix.
> +;;;
> +;;; GNU Guix is free software; you can redistribute it and/or modify it
> +;;; under the terms of the GNU General Public License as published by
> +;;; the Free Software Foundation; either version 3 of the License, or (at
> +;;; your option) any later version.
> +;;;
> +;;; GNU Guix is distributed in the hope that it will be useful, but
> +;;; WITHOUT ANY WARRANTY; without even the implied warranty of
> +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> +;;; GNU General Public License for more details.
> +;;;
> +;;; You should have received a copy of the GNU General Public License
> +;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
> +
> +(define-module (guix build tree-sitter-build-system)
> + #:use-module ((guix build node-build-system) #:prefix node:)
> + #:use-module (guix build json)
> + #:use-module (guix build utils)
> + #:use-module (ice-9 match)
> + #:use-module (ice-9 regex)
> + #:use-module (srfi srfi-1)
> + #:export (%standard-phases
> + tree-sitter-build))
> +
> +;; Commentary:
> +;;
> +;; Build procedures for tree-sitter grammar packages. This is the
> +;; builder-side code, which builds on top fo the node build-system.
> +;;
> +;; Tree-sitter grammars are written in JavaScript and compiled to a native
> +;; shared object. The `tree-sitter generate' command invokes `node' in order
> +;; to evaluate the grammar.js into a grammar.json file, which is then
> +;; translated into C code. We then compile the C code ourselves. Packages
> +;; also sometimes add extra manually written C/C++ code.
> +;;
> +;; In order to support grammars depending on each other, such as C and C++,
> +;; JavaScript and TypeScript, this build-system installs the source of the
> +;; node module in a dedicated "js" output.
> +;;
> +;; Code:
> +
> +(define* (patch-dependencies #:key inputs #:allow-other-keys)
> + "Rewrite dependencies in 'package.json'. We remove all runtime dependencies
> +and replace development dependencies with tree-sitter grammar node modules."
> +
> + (define (rewrite package.json)
> + (map (match-lambda
> + (("dependencies" @ . _)
> + '("dependencies" @))
> + (("devDependencies" @ . _)
> + `("devDependencies" @
> + ,@(filter-map (match-lambda
> + ((key . directory)
> + (let ((node-module
> + (string-append directory
> + "/lib/node_modules/"
> + key)))
> + (and (directory-exists? node-module)
> + `(,key . ,node-module)))))
> + (alist-delete "node" inputs))))
> + (other other))
> + package.json))
> +
> + (node:with-atomic-json-file-replacement "package.json"
> + (match-lambda
> + (('@ . package.json)
> + (cons '@ (rewrite package.json))))))
> +
> +;; FIXME: The node build-system's configure phase does not support
> +;; cross-compiling so we re-define it.
> +(define* (configure #:key native-inputs inputs #:allow-other-keys)
> + (invoke (search-input-file (or native-inputs inputs) "/bin/npm")
> + "--offline" "--ignore-scripts" "install"))
> +
> +(define* (build #:key grammar-directories #:allow-other-keys)
> + (for-each (lambda (dir)
> + (with-directory-excursion dir
> + ;; Avoid generating binding code for other languages, we do
> + ;; not support this use-case yet and it relies on running
> + ;; `node-gyp' to build native addons.
> + (invoke "tree-sitter" "generate" "--no-bindings")))
> + grammar-directories))
> +
> +(define* (check #:key grammar-directories tests? #:allow-other-keys)
> + (when tests?
> + (for-each (lambda (dir)
> + (with-directory-excursion dir
> + (invoke "tree-sitter" "test")))
> + grammar-directories)))
> +
> +(define* (install #:key target grammar-directories outputs #:allow-other-keys)
> + (let ((lib (string-append (assoc-ref outputs "out")
> + "/lib/tree-sitter")))
> + (mkdir-p lib)
> + (define (compile-language dir)
> + (with-directory-excursion dir
> + (let ((lang (assoc-ref (call-with-input-file "src/grammar.json"
> + read-json)
> + "name"))
> + (source-file (lambda (path)
> + (if (file-exists? path)
> + path
> + #f))))
> + (apply invoke
> + `(,(if target
> + (string-append target "-g++")
> + "g++")
> + "-shared"
> + "-fPIC"
> + "-fno-exceptions"
> + "-O2"
> + "-g"
> + "-o" ,(string-append lib "/" lang ".so")
> + ;; An additional `scanner.{c,cc}' file is sometimes
> + ;; provided.
> + ,@(cond
> + ((source-file "src/scanner.c")
> + => (lambda (file) (list "-xc" "-std=c99" file)))
> + ((source-file "src/scanner.cc")
> + => (lambda (file) (list file)))
> + (else '()))
> + "-xc" "src/parser.c")))))
> + (for-each compile-language grammar-directories)))
> +
> +(define* (install-js #:key native-inputs inputs outputs #:allow-other-keys)
> + (invoke (search-input-file (or native-inputs inputs) "/bin/npm")
> + "--prefix" (assoc-ref outputs "js")
> + "--global"
> + "--offline"
> + "--loglevel" "info"
> + "--production"
> + ;; Skip scripts to prevent building bindings via GYP.
> + "--ignore-scripts"
> + "install" "../package.tgz"))
> +
> +(define %standard-phases
> + (modify-phases node:%standard-phases
> + (replace 'patch-dependencies patch-dependencies)
> + (replace 'configure configure)
> + (replace 'build build)
> + (replace 'check check)
> + (replace 'install install)
> + (add-after 'install 'install-js install-js)))
> +
> +(define* (tree-sitter-build #:key inputs (phases %standard-phases)
> + #:allow-other-keys #:rest args)
> + (apply node:node-build #:inputs inputs #:phases phases args))
> +
> +;;; tree-sitter-build-system.scm ends here

Applied, slightly reformatted, pushed as
dbd4d2d0707b486f1e2c8659e94e1d3b15e4351e

Also, migrated tree-sitter-grammar fn to this build system.

I still try to get the taste of packaging grammars with
tree-sitter-grammar helper and just as plain packages and try to
understand how good/bad template function idea is in
This message was truncated. Download the full message here.
-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEEKEGaxlA4dEDH6S/6IgjSCVjB3rAFAmPomGAACgkQIgjSCVjB
3rBNCg/+MLQ2JZC58PmhThF3IiJ8y2w/elM/me0Nlr1MT47sR7+y3lYAvSDSy8fN
Sl0KEgobI3VhXieJysH0WQAlh7+f2JnVrr4kFOG6Gw2VSKIW2/EjaBdIKgYbf4Tc
VkTd+vi6m1WuJj+rYG+CwdmnQpnXTMYa9tMhV2aNA7H+v/RyXuxbFWzz+6XUuIi0
BIXNt/3X+TqtdaH4c/WkZFOSFrWsJh2k9bj6w/rKDumKD7EdyVDlI6nR0COkD1Ua
Datz/POOcruXrj/EYqBas4phSOHWmP9uxkIv6mA/6+7yxS6jKEagTBJyUtRSQXPJ
NQombe3oEZQWBhxSyQTSjbUL0FmQYKBjF5SBCV6xWK9Hbis1HiaAhUNsQaauazzl
n6GN4pH8kkAAAmdTn97Q0gJKZTr0dUCVkJfIxCJq2FLK9qqn1F7zAVDuNcA2Z3ga
ylu38/ue92e5MlFO3rgUpP8TcwvXs2uahWdMkqjglCXPcXq3YytMb9KN/ihFN/VV
OQW4ul4a/6P0sHGYfCUwipGEEr+OD9p20aK0eH/fRNy/YbZ9rms15FBgZ+fecknr
qrmJnQgcA9jC2rA0ieNB6UcDDxdtDxU8ieeePZbnYWoK9TaMQtAm8+xwQjFVG4wl
u56wQZN9vEphzPoqAPk5HsTL766UAGetQeQBy1x7sRmn3MAfjMo=
=uq1j
-----END PGP SIGNATURE-----

P
P
Pierre Langlois wrote on 12 Feb 2023 13:07
Re: [bug#49946] [PATCH v7 01/32] gnu: tree-sitter: Move to its own module.
(name . Andrew Tropin)(address . andrew@trop.in)
871qmvm5qh.fsf@gmx.com
Hi,

Andrew Tropin <andrew@trop.in> writes:

Toggle quote (147 lines)
> [[PGP Signed Part:Undecided]]
> On 2023-02-10 15:48, Pierre Langlois wrote:
>
>> Hi Andrew, thanks for pushing this along! It's great to see things
>> getting merged.
>>
>> Andrew Tropin <andrew@trop.in> writes:
>>
>>> [[PGP Signed Part:Undecided]]
>>> On 2023-02-09 18:04, Andrew Tropin wrote:
>>>
>>>> On 2023-02-09 13:39, zimoun wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> On Thu, 09 Feb 2023 at 14:11, Andrew Tropin <andrew@trop.in> wrote:
>>>>>
>>>>>> I applied tree-sitter and tree-sitter-cli patches,
>>>>>
>>>>> Just to be sure to understand, you have only applied 02/32 and 05/32,
>>>>> right?
>>>>>
>>>>>
>>>>> [bug#49946] [PATCH v7 02/32] gnu: tree-sitter: Update to 0.20.7.
>>>>> id:20221125012142.22579-3-pierre.langlois@gmx.com
>>>>> http://issues.guix.gnu.org/msgid/20221125012142.22579-3-pierre.langlois@gmx.com
>>>>>
>>>>> [bug#49946] [PATCH v7 05/32] gnu: Add tree-sitter-cli.
>>>>> id:20221125012142.22579-6-pierre.langlois@gmx.com
>>>>> http://issues.guix.gnu.org/msgid/20221125012142.22579-6-pierre.langlois@gmx.com
>>>>>
>>>>> Leaving out all the others, right?
>>>>
>>>> Merged first 5 patches from 01 to 05, also added one more commit, which
>>>> addresses some things from reviews and one commit, which adds html
>>>> grammar.
>>>>
>>>> The html grammar is added for the testing purposes. It relies on
>>>> generated parser.c and scanner.c and we will need to repackage it using
>>>> grammar.js instead. I'm not sure if a separate build system is needed
>>>> for this, I guess we can just rewrite tree-sitter-grammar function,
>>>> which generates packages as in example with tree-sitter-grammar-html:
>>>> https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/tree-sitter.scm?h=53b00b91b73bd60412d5bd057e22e6d63194a7f7#n158
>>>>
>>>> Anyway, I only skimmed tree-sitter-build-system source code, and plan to
>>>> read it carefully, evaluate and either introduce new build system or
>>>> just move all needed parts to tree-sitter-grammar function. WDYT?
>>>> After we done with it we can package all other grammars.
>>>
>>> Ok, I realized that the proper build process for tree-sitter grammars is
>>> a little harder than I expected, tree-sitter-build system make sense. I
>>> reviewed it, made a small change:
>>
>> Ah great, I was going to comment to try and push for us to keep the
>> build system. I originally went with a template package and inheritance,
>> but Maxime suggested moving to a build-system which ended up making the
>> package definitions a *lot* nicer IMO (see previous discussion here
>> https://issues.guix.gnu.org/49946#144). It also allows us to deal with
>> grammars that depend on each other more nicely I think.
>>
>>>
>>> @@ -29,7 +29,7 @@ (define-module (guix build tree-sitter-build-system)
>>> ;; Commentary:
>>> ;;
>>> ;; Build procedures for tree-sitter grammar packages. This is the
>>> -;; builder-side code, which builds on top fo the node build-system.
>>> +;; builder-side code, which builds on top of the node build-system.
>>> ;;
>>> ;; Tree-sitter grammars are written in JavaScript and compiled to a native
>>> ;; shared object. The `tree-sitter generate' command invokes `node' in order
>>> @@ -114,7 +114,7 @@ (define (compile-language dir)
>>> "-fno-exceptions"
>>> "-O2"
>>> "-g"
>>> - "-o" ,(string-append lib "/" lang ".so")
>>> + "-o" ,(string-append lib "/libtree-sitter-" lang ".so")
>>> ;; An additional `scanner.{c,cc}' file is sometimes
>>> ;; provided.
>>> ,@(cond
>>>
>>>
>>> rewrote html grammar to use this build system and made it work with
>>> built-in treesit package. Also, tried examples of c and cpp grammars
>>> from patches in this thread.
>>>
>>> If you ok with it, I'll push the build system to master and update the
>>> html grammar accordingly.
>>>
>>> The final result will look like this:
>>>
>>> (define tree-sitter-delete-generated-files
>>> #~(begin
>>> (delete-file "binding.gyp")
>>> (delete-file-recursively "bindings")
>>> (delete-file "src/grammar.json")
>>> (delete-file "src/node-types.json")
>>> (delete-file "src/parser.c")
>>> (delete-file-recursively "src/tree_sitter")))
>>>
>>> (define* (tree-sitter-grammar
>>> language language-for-synopsis version commit hash
>>> #:key
>>> (repository-url
>>> (format #f "https://github.com/tree-sitter/tree-sitter-~a" language))
>>> (inputs '()))
>>> (let ((synopsis (string-append language-for-synopsis
>>> " grammar for tree-sitter"))
>>> (name (string-append "tree-sitter-grammar-" language)))
>>> (package
>>> (name name)
>>> (version version)
>>> (home-page repository-url)
>>> (source (origin
>>> (method git-fetch)
>>> (uri (git-reference
>>> (url repository-url)
>>> (commit commit)))
>>> (file-name (git-file-name name version))
>>> (sha256 (base32 hash))
>>> (modules '((guix build utils)))
>>> (snippet tree-sitter-delete-generated-files)))
>>> (build-system tree-sitter-build-system)
>>> (inputs inputs)
>>> (synopsis synopsis)
>>> (description (string-append synopsis "."))
>>> (license license:expat))))
>>>
>>> (define-public tree-sitter-grammar-html
>>> (tree-sitter-grammar
>>> "html" "HTML"
>>> "0.19.0" "v0.19.0"
>>> "1hg7vbcy7bir6b8x11v0a4x0glvqnsqc3i2ixiarbxmycbgl3axy"))
>>>
>>> After that we can bring the rest of the grammars.
>>
>> I would suggest to rmeove the `tree-sitter-grammar' function, and keep
>> grammars as "regular" package records, even though it's a little bit
>> more verbose:
>>
>> --8<---------------cut here---------------start------------->8---
>> (define-public tree-sitter-html
>> (package
>> (name "tree-sitter-html")
>
> It seems tree-sitter-html mimics upstream package name and probably make
> more sense than tree-sitter-grammar-html used by me.

Yeah, at some point I think I had named the packages with "grammar" as
well, but thought it was a bit of a mouthful. I'm also thinking one day
we may build language bindings as part of the build system (Rust and
NodeJS I think ATM), so those packages could do more than ship the
grammar in the future (although we don't know if we'll ever really need
that).

Toggle quote (43 lines)
>
>> (version "0.19.0")
>> (source (origin
>> (method git-fetch)
>> (uri (git-reference
>> (url "https://github.com/tree-sitter/tree-sitter-html")
>> (commit (string-append "v" version))))
>> (file-name (git-file-name name version))
>> (sha256
>> (base32
>> "1hg7vbcy7bir6b8x11v0a4x0glvqnsqc3i2ixiarbxmycbgl3axy"))
>> (modules '((guix build utils)))
>> (snippet tree-sitter-delete-generated-files)))
>> (build-system tree-sitter-build-system)
>> (home-page "https://github.com/tree-sitter/tree-sitter-html")
>> (synopsis "Tree-sitter HTML grammar")
>> (description
>> "This package provides a HTML grammar for the Tree-sitter library.")
>> (license license:expat)))
>> --8<---------------cut here---------------end--------------->8---
>>
>> This way, they look like any other package in Guix, which makes it
>> easier for us to apply automatic changes in the future if needed (for
>> example like how the input format could be automically updated for all
>> "simple" package definitions, but had to be manual whenever custom code
>> refactoring was done). Does this make sense?
>
> Make sense, but on the other hand we already have hunspell, aspell
> dictionaries and probably a few more others, which are very similiar in
> spirit and we already have to keep in mind their existence on such
> automatic code updates.
>
> It looks that the packages differ only in url for the source code, lang
> name and sometimes in inputs. Having template package function can make
> management of shared parts more centralized, reduce possibility of
> copy-paste mistakes, when the description wasn't updated and so on and
> can reduce the amount of the code overall (which also reduces the change
> of introducing an error).
>
> I don't have a strong opinion on this topic, but leaning towards the
> template function slightly more, however I'm completely ok with the
> standalone package definitions as well. WDYT?

I can think of both cost/benefits to the template so I don't have a
strong opinion either :-).

I do like the template to make sure people don't forget to delete
generated files, that's quite important as it seems upstream packages
often check-in the generated C code. Although, we could probably assert
that with in the build-system phase? I'll think about that.

On the other hand, I wonder how the template works for packages that
provide multiple grammars (see ocaml and typescript for example). I
guess we could use the template for trivial packages, and standalone
definitions for more complex ones? In general, if we keep the template
interface really simple, then I'm happy with it.

Thanks,
Pierre
-----BEGIN PGP SIGNATURE-----

iQFMBAEBCgA2FiEEctU9gYy29KFyWDdMqPyeRH9PfVQFAmPo2nYYHHBpZXJyZS5s
YW5nbG9pc0BnbXguY29tAAoJEKj8nkR/T31UEu4H/28hYTTd7UXk3XP09t/L9LAK
J6ZU8tN0dghON14h5vy/KhMGXhbBDpqHLTPmMPcyjRiQbkcGhh7bPaUDtoQ1P916
Ytmfgc9vpqjznFL2YmqcUzAW6pklnhvTJt8wz3RVEwg/7DO7piMSkW9zW7eM7S43
+EhhIHfC+My3+hQKaKjTxIzCo+J51cFFYALnrY+eaoxOOb3/zg6RFdEVjj/fxOYc
tX3T1MTeAOtgQIShKjQj6TF9qUGLvT5B2t6WNDH9OE+zsXCLtsMbXnfCsWOjvwUl
ARug7+XwQKVVMjlCPbqVkGRQoUwRqwIZBljWCeDyJzrF17Vp6st2UHRpA2kLN38=
=4Eku
-----END PGP SIGNATURE-----

P
P
Pierre Langlois wrote on 12 Feb 2023 13:24
(name . Andrew Tropin)(address . andrew@trop.in)
87r0uvkr06.fsf@gmx.com
Andrew Tropin <andrew@trop.in> writes:

Toggle quote (101 lines)
> [[PGP Signed Part:Undecided]]
> On 2023-02-10 17:02, Pierre Langlois wrote:
>
>> Pierre Langlois <pierre.langlois@gmx.com> writes:
>>
>>> [[PGP Signed Part:Undecided]]
>>> Hi Andrew, thanks for pushing this along! It's great to see things
>>> getting merged.
>>>
>>> Andrew Tropin <andrew@trop.in> writes:
>>>
>>>> [[PGP Signed Part:Undecided]]
>>>> On 2023-02-09 18:04, Andrew Tropin wrote:
>>>>
>>>>> On 2023-02-09 13:39, zimoun wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> On Thu, 09 Feb 2023 at 14:11, Andrew Tropin <andrew@trop.in> wrote:
>>>>>>
>>>>>>> I applied tree-sitter and tree-sitter-cli patches,
>>>>>>
>>>>>> Just to be sure to understand, you have only applied 02/32 and 05/32,
>>>>>> right?
>>>>>>
>>>>>>
>>>>>> [bug#49946] [PATCH v7 02/32] gnu: tree-sitter: Update to 0.20.7.
>>>>>> id:20221125012142.22579-3-pierre.langlois@gmx.com
>>>>>> http://issues.guix.gnu.org/msgid/20221125012142.22579-3-pierre.langlois@gmx.com
>>>>>>
>>>>>> [bug#49946] [PATCH v7 05/32] gnu: Add tree-sitter-cli.
>>>>>> id:20221125012142.22579-6-pierre.langlois@gmx.com
>>>>>> http://issues.guix.gnu.org/msgid/20221125012142.22579-6-pierre.langlois@gmx.com
>>>>>>
>>>>>> Leaving out all the others, right?
>>>>>
>>>>> Merged first 5 patches from 01 to 05, also added one more commit, which
>>>>> addresses some things from reviews and one commit, which adds html
>>>>> grammar.
>>>>>
>>>>> The html grammar is added for the testing purposes. It relies on
>>>>> generated parser.c and scanner.c and we will need to repackage it using
>>>>> grammar.js instead. I'm not sure if a separate build system is needed
>>>>> for this, I guess we can just rewrite tree-sitter-grammar function,
>>>>> which generates packages as in example with tree-sitter-grammar-html:
>>>>> https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/tree-sitter.scm?h=53b00b91b73bd60412d5bd057e22e6d63194a7f7#n158
>>>>>
>>>>> Anyway, I only skimmed tree-sitter-build-system source code, and plan to
>>>>> read it carefully, evaluate and either introduce new build system or
>>>>> just move all needed parts to tree-sitter-grammar function. WDYT?
>>>>> After we done with it we can package all other grammars.
>>>>
>>>> Ok, I realized that the proper build process for tree-sitter grammars is
>>>> a little harder than I expected, tree-sitter-build system make sense. I
>>>> reviewed it, made a small change:
>>>
>>> Ah great, I was going to comment to try and push for us to keep the
>>> build system. I originally went with a template package and inheritance,
>>> but Maxime suggested moving to a build-system which ended up making the
>>> package definitions a *lot* nicer IMO (see previous discussion here
>>> https://issues.guix.gnu.org/49946#144). It also allows us to deal with
>>> grammars that depend on each other more nicely I think.
>>>
>>>>
>>>> @@ -29,7 +29,7 @@ (define-module (guix build tree-sitter-build-system)
>>>> ;; Commentary:
>>>> ;;
>>>> ;; Build procedures for tree-sitter grammar packages. This is the
>>>> -;; builder-side code, which builds on top fo the node build-system.
>>>> +;; builder-side code, which builds on top of the node build-system.
>>>> ;;
>>>> ;; Tree-sitter grammars are written in JavaScript and compiled to a native
>>>> ;; shared object. The `tree-sitter generate' command invokes `node' in order
>>>> @@ -114,7 +114,7 @@ (define (compile-language dir)
>>>> "-fno-exceptions"
>>>> "-O2"
>>>> "-g"
>>>> - "-o" ,(string-append lib "/" lang ".so")
>>>> + "-o" ,(string-append lib "/libtree-sitter-" lang ".so")
>>>> ;; An additional `scanner.{c,cc}' file is sometimes
>>>> ;; provided.
>>>> ,@(cond
>>>>
>>>>
>>>> rewrote html grammar to use this build system and made it work with
>>>> built-in treesit package. Also, tried examples of c and cpp grammars
>>>> from patches in this thread.
>>>>
>>>> If you ok with it, I'll push the build system to master and update the
>>>> html grammar accordingly.
>>
>> Oh, I forgot to say, this change to the build system LGTM! I'm really
>> happy to see it merged soon :-). The path change will probably break the
>> emacs-28-based tree-sitter support, but that's OK, it's better for the
>> build-system to be made to target emacs 29's builtin support. I'm sure I
>> can work around for emacs 28.
>
> Actually, I think we can build grammars with both names, just providing
> two .so files instead of one. If you won't find a better workaround we
> can go this way.

That's no problem, I was able to work-around it quite easily. The
emacs-tree-sitter-langs package expects all the grammars supported to be
in a single directory, so we have to bundle them. So we can quite
easily create symlinks from the bundle with the expected names,
something like:

Toggle snippet (56 lines)
(define-public emacs-tree-sitter-langs-grammar-bundle
(package
(name "emacs-tree-sitter-langs-grammar-bundle")
(source #f)
(version (package-version tree-sitter))
(build-system trivial-build-system)
(inputs
;; FIXME: Support for some languages is still left to package.
(list tree-sitter-bash
tree-sitter-c
tree-sitter-c-sharp
tree-sitter-cpp
tree-sitter-css
tree-sitter-elixir
tree-sitter-elm
tree-sitter-go
tree-sitter-haskell
tree-sitter-html
tree-sitter-java
tree-sitter-javascript
tree-sitter-json
tree-sitter-julia
tree-sitter-lua
tree-sitter-ocaml
tree-sitter-php
tree-sitter-python
tree-sitter-r
tree-sitter-rust
tree-sitter-ruby
tree-sitter-typescript))
(arguments
(list #:builder
(with-imported-modules '((guix build union)
(guix build utils))
#~(begin
(use-modules (ice-9 match)
(srfi srfi-1)
(guix build union)
(guix build utils))
(mkdir-p #$output)
(for-each
(lambda (lib)
(symlink lib
(string-append #$output "/"
(substring (basename lib)
(string-length "libtree-sitter-")))))
(append-map (match-lambda
((name directory)
(find-files directory "libtree-sitter-.*\\.so$")))
'#$(package-inputs this-package)))))))
(synopsis #f)
(description #f)
(home-page #f)
(license #f)))

Giving us:

Toggle snippet (26 lines)
/gnu/store/fr9h59wgn55ilfifvm5df7xzxfwh38pc-emacs-tree-sitter-langs-grammar-bundle-0.20.7
??? bash.so -> /gnu/store/9qzvcdlpryjl44klfq0i2liqf6lsp5jq-tree-sitter-bash-0.19.0/lib/tree-sitter/libtree-sitter-bash.so
??? cpp.so -> /gnu/store/my02kq2dr6h48nmrl2dnfnm5rggx837w-tree-sitter-cpp-0.20.0-1.5ead1e2/lib/tree-sitter/libtree-sitter-cpp.so
??? c_sharp.so -> /gnu/store/mp8jvbhx5xlgj5bxa52zhmk5x8i7md5h-tree-sitter-c-sharp-0.19.1/lib/tree-sitter/libtree-sitter-c_sharp.so
??? c.so -> /gnu/store/y9ln76yx68q495vx8bnqdy87x7k8ihy5-tree-sitter-c-0.20.2/lib/tree-sitter/libtree-sitter-c.so
??? css.so -> /gnu/store/fwji59sdwvqpqyb94m55qw1ak92cmys9-tree-sitter-css-0.19.0/lib/tree-sitter/libtree-sitter-css.so
??? elixir.so -> /gnu/store/f6ismnxf7hlh1sq87zzarw56avvrzsl9-tree-sitter-elixir-0.19.0-1.de20391/lib/tree-sitter/libtree-sitter-elixir.so
??? elm.so -> /gnu/store/56zjl2ljvba3z1j6zfmpfa8mq19a3g3a-tree-sitter-elm-5.6.3/lib/tree-sitter/libtree-sitter-elm.so
??? go.so -> /gnu/store/qfy5sw6za96wkszbi21adxbxzggvjz3g-tree-sitter-go-0.19.1-1.05900fa/lib/tree-sitter/libtree-sitter-go.so
??? haskell.so -> /gnu/store/cq3chmqcb8g1nf5mzb6yhzdd6x9gvg1v-tree-sitter-haskell-0.14.0-1.e30bdfd/lib/tree-sitter/libtree-sitter-haskell.so
??? html.so -> /gnu/store/5pqfyjrg3yyvxaxidk690ffls3yb7wbi-tree-sitter-html-0.19.0/lib/tree-sitter/libtree-sitter-html.so
??? javascript.so -> /gnu/store/28s09v3dfb1c9bdkq2791z9abxnih66p-tree-sitter-javascript-0.20.0/lib/tree-sitter/libtree-sitter-javascript.so
??? java.so -> /gnu/store/i7jlqf5hbv0rhlkp4qlyc2d5ndw69dx4-tree-sitter-java-0.20.1/lib/tree-sitter/libtree-sitter-java.so
??? json.so -> /gnu/store/5dv4r74p6gd9w2ncs6pjrhz5hbw47262-tree-sitter-json-0.19.0/lib/tree-sitter/libtree-sitter-json.so
??? julia.so -> /gnu/store/5k37g1sdsllgh64p0w6ggabsni6jqlkr-tree-sitter-julia-0.19.0/lib/tree-sitter/libtree-sitter-julia.so
??? lua.so -> /gnu/store/nqzn4a6kgb2rx4y44pxdm4sqf3pzcpz1-tree-sitter-lua-0.0.14/lib/tree-sitter/libtree-sitter-lua.so
??? ocaml_interface.so -> /gnu/store/3h7krcj3xxclirb8afxh65ipabw1821l-tree-sitter-ocaml-0.19.0-1.0348562/lib/tree-sitter/libtree-sitter-ocaml_interface.so
??? ocaml.so -> /gnu/store/3h7krcj3xxclirb8afxh65ipabw1821l-tree-sitter-ocaml-0.19.0-1.0348562/lib/tree-sitter/libtree-sitter-ocaml.so
??? php.so -> /gnu/store/ymxf5m8jhihbrag7v2pghgydj3byp7wh-tree-sitter-php-0.19.0-1.435fa00/lib/tree-sitter/libtree-sitter-php.so
??? python.so -> /gnu/store/ggmzicwfxb7gz1rr9lfkx8cak62bfw7v-tree-sitter-python-0.19.1-1.ed0fe62/lib/tree-sitter/libtree-sitter-python.so
??? r.so -> /gnu/store/y9dxbnbb5dyf0rq3kpar7ip4w3lq6sb9-tree-sitter-r-0.0.1-1.80efda5/lib/tree-sitter/libtree-sitter-r.so
??? ruby.so -> /gnu/store/ky8n30dw16ck6byaqnhbf9ib7xp7j0yw-tree-sitter-ruby-0.20.0/lib/tree-sitter/libtree-sitter-ruby.so
??? rust.so -> /gnu/store/25zdpwrgq1xibhv7xpg64i4g71xah6g1-tree-sitter-rust-0.20.1/lib/tree-sitter/libtree-sitter-rust.so
??? tsx.so -> /gnu/store/wqpcphz855yjrginwqrymd3xzzxb8k8l-tree-sitter-typescript-0.20.1/lib/tree-sitter/libtree-sitter-tsx.so
??? typescript.so -> /gnu/store/wqpcphz855yjrginwqrymd3xzzxb8k8l-tree-sitter-typescript-0.20.1/lib/tree-sitter/libtree-sitter-typescript.so
-----BEGIN PGP SIGNATURE-----

iQFMBAEBCgA2FiEEctU9gYy29KFyWDdMqPyeRH9PfVQFAmPo20kYHHBpZXJyZS5s
YW5nbG9pc0BnbXguY29tAAoJEKj8nkR/T31UcoMIAI+yOPj9qsXR35dX5JAx+Ny1
I/xsUxUzjvFTuF5FDfMfmW8vhLxBIYjU+YS0SZ1lsXSevWkcX740Zlc/WkwsYBFF
/6wvqyFLQexh6ZKTCHcwHaYiuJjJCvdUf28/erdXMdsTb/PKwC2/yuRC/6ZLNwba
Qy7rhEZ0uAxUzst5RPR9mOQO64leNZx7Oftsc6fyjxkQ3LoHyn97yVvDS8MOSTux
aoKaVvUZ6ka31BkHhzRzr47FpBxJSc3jBWkVepbhTVO4Noa07GnrWaMhdHTLa4nC
vOD2EyGsicDVt/U4MGtAl+d6m5hgRWrj/RUDAADE9Kc4dfuNhfM05NNQxCPOWJQ=
=aR2f
-----END PGP SIGNATURE-----

P
P
Pierre Langlois wrote on 12 Feb 2023 13:29
(name . Andrew Tropin)(address . andrew@trop.in)
87mt5jkqxq.fsf@gmx.com
Andrew Tropin <andrew@trop.in> writes:

Toggle quote (51 lines)
> [[PGP Signed Part:Undecided]]
> On 2023-02-10 17:04, Pierre Langlois wrote:
>
>> Hi,
>>
>> Andrew Tropin <andrew@trop.in> writes:
>>
>>> [[PGP Signed Part:Undecided]]
>>> On 2023-02-09 13:39, zimoun wrote:
>>>
>>>> Hi,
>>>>
>>>> On Thu, 09 Feb 2023 at 14:11, Andrew Tropin <andrew@trop.in> wrote:
>>>>
>>>>> I applied tree-sitter and tree-sitter-cli patches,
>>>>
>>>> Just to be sure to understand, you have only applied 02/32 and 05/32,
>>>> right?
>>>>
>>>>
>>>> [bug#49946] [PATCH v7 02/32] gnu: tree-sitter: Update to 0.20.7.
>>>> id:20221125012142.22579-3-pierre.langlois@gmx.com
>>>> http://issues.guix.gnu.org/msgid/20221125012142.22579-3-pierre.langlois@gmx.com
>>>>
>>>> [bug#49946] [PATCH v7 05/32] gnu: Add tree-sitter-cli.
>>>> id:20221125012142.22579-6-pierre.langlois@gmx.com
>>>> http://issues.guix.gnu.org/msgid/20221125012142.22579-6-pierre.langlois@gmx.com
>>>>
>>>> Leaving out all the others, right?
>>>
>>> Merged first 5 patches from 01 to 05, also added one more commit, which
>>> addresses some things from reviews and one commit, which adds html
>>> grammar.
>>
>> I had also started rebasing addressing comments on the tree-sitter-cli
>> package, and had a few extra changes left that haven't been pushed.
>>
>> I realised we could remove a now redundant comment, and while enabling
>> *some* tests, I realized we should directly depend on node and dot, so
>> that people may use the `tree-sitter' command, without needing them in
>> the same profile.
>
> Sounds very valid! :)
>
>>
>> What do you think of the following two patches?
>>
>
> I applied them locally and tested, LGTM. Will push them a little later
> with the rest of tree-sitter related changes.

Cool, thanks!
-----BEGIN PGP SIGNATURE-----

iQFMBAEBCgA2FiEEctU9gYy29KFyWDdMqPyeRH9PfVQFAmPo26EYHHBpZXJyZS5s
YW5nbG9pc0BnbXguY29tAAoJEKj8nkR/T31Ud0oH/Avlqm9XpD6494RX98u1sdtJ
AAvvQlySa6IDncVeI6PpeFsMDs3F9IcskzmPLcd5N/pTLSMTIjIguImVwWC7wfiB
Vz/KKWwTq9gMx4lBRPQweULp+M7ATP+Y7fHj8u8/KZCrAVr49T6LrfP2ZVc+VktP
sM2nJkzVozVxQdoJt4B5lXHvrSHnSNptrdpbwzC+ZjI3jNLRN2ewAMr9CrurJWS6
wcQojyw8E+84/DD7bfajAwMzjOFH4BN4tsL86Z0eIRcsvQRPldWJiyGag4AWhRS3
G2G6XwVKRvYgB7N6PqDHUwyflBYPcvcJ8Cm/T2fYbkdXR1tmFOv172+zvqxIQsY=
=557e
-----END PGP SIGNATURE-----

A
A
Andrew Tropin wrote on 14 Feb 2023 14:24
(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
87zg9g753h.fsf@trop.in
On 2023-02-12 12:07, Pierre Langlois wrote:

Toggle quote (215 lines)
> Hi,
>
> Andrew Tropin <andrew@trop.in> writes:
>
>> [[PGP Signed Part:Undecided]]
>> On 2023-02-10 15:48, Pierre Langlois wrote:
>>
>>> Hi Andrew, thanks for pushing this along! It's great to see things
>>> getting merged.
>>>
>>> Andrew Tropin <andrew@trop.in> writes:
>>>
>>>> [[PGP Signed Part:Undecided]]
>>>> On 2023-02-09 18:04, Andrew Tropin wrote:
>>>>
>>>>> On 2023-02-09 13:39, zimoun wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> On Thu, 09 Feb 2023 at 14:11, Andrew Tropin <andrew@trop.in> wrote:
>>>>>>
>>>>>>> I applied tree-sitter and tree-sitter-cli patches,
>>>>>>
>>>>>> Just to be sure to understand, you have only applied 02/32 and 05/32,
>>>>>> right?
>>>>>>
>>>>>>
>>>>>> [bug#49946] [PATCH v7 02/32] gnu: tree-sitter: Update to 0.20.7.
>>>>>> id:20221125012142.22579-3-pierre.langlois@gmx.com
>>>>>> http://issues.guix.gnu.org/msgid/20221125012142.22579-3-pierre.langlois@gmx.com
>>>>>>
>>>>>> [bug#49946] [PATCH v7 05/32] gnu: Add tree-sitter-cli.
>>>>>> id:20221125012142.22579-6-pierre.langlois@gmx.com
>>>>>> http://issues.guix.gnu.org/msgid/20221125012142.22579-6-pierre.langlois@gmx.com
>>>>>>
>>>>>> Leaving out all the others, right?
>>>>>
>>>>> Merged first 5 patches from 01 to 05, also added one more commit, which
>>>>> addresses some things from reviews and one commit, which adds html
>>>>> grammar.
>>>>>
>>>>> The html grammar is added for the testing purposes. It relies on
>>>>> generated parser.c and scanner.c and we will need to repackage it using
>>>>> grammar.js instead. I'm not sure if a separate build system is needed
>>>>> for this, I guess we can just rewrite tree-sitter-grammar function,
>>>>> which generates packages as in example with tree-sitter-grammar-html:
>>>>> https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/tree-sitter.scm?h=53b00b91b73bd60412d5bd057e22e6d63194a7f7#n158
>>>>>
>>>>> Anyway, I only skimmed tree-sitter-build-system source code, and plan to
>>>>> read it carefully, evaluate and either introduce new build system or
>>>>> just move all needed parts to tree-sitter-grammar function. WDYT?
>>>>> After we done with it we can package all other grammars.
>>>>
>>>> Ok, I realized that the proper build process for tree-sitter grammars is
>>>> a little harder than I expected, tree-sitter-build system make sense. I
>>>> reviewed it, made a small change:
>>>
>>> Ah great, I was going to comment to try and push for us to keep the
>>> build system. I originally went with a template package and inheritance,
>>> but Maxime suggested moving to a build-system which ended up making the
>>> package definitions a *lot* nicer IMO (see previous discussion here
>>> https://issues.guix.gnu.org/49946#144). It also allows us to deal with
>>> grammars that depend on each other more nicely I think.
>>>
>>>>
>>>> @@ -29,7 +29,7 @@ (define-module (guix build tree-sitter-build-system)
>>>> ;; Commentary:
>>>> ;;
>>>> ;; Build procedures for tree-sitter grammar packages. This is the
>>>> -;; builder-side code, which builds on top fo the node build-system.
>>>> +;; builder-side code, which builds on top of the node build-system.
>>>> ;;
>>>> ;; Tree-sitter grammars are written in JavaScript and compiled to a native
>>>> ;; shared object. The `tree-sitter generate' command invokes `node' in order
>>>> @@ -114,7 +114,7 @@ (define (compile-language dir)
>>>> "-fno-exceptions"
>>>> "-O2"
>>>> "-g"
>>>> - "-o" ,(string-append lib "/" lang ".so")
>>>> + "-o" ,(string-append lib "/libtree-sitter-" lang ".so")
>>>> ;; An additional `scanner.{c,cc}' file is sometimes
>>>> ;; provided.
>>>> ,@(cond
>>>>
>>>>
>>>> rewrote html grammar to use this build system and made it work with
>>>> built-in treesit package. Also, tried examples of c and cpp grammars
>>>> from patches in this thread.
>>>>
>>>> If you ok with it, I'll push the build system to master and update the
>>>> html grammar accordingly.
>>>>
>>>> The final result will look like this:
>>>>
>>>> (define tree-sitter-delete-generated-files
>>>> #~(begin
>>>> (delete-file "binding.gyp")
>>>> (delete-file-recursively "bindings")
>>>> (delete-file "src/grammar.json")
>>>> (delete-file "src/node-types.json")
>>>> (delete-file "src/parser.c")
>>>> (delete-file-recursively "src/tree_sitter")))
>>>>
>>>> (define* (tree-sitter-grammar
>>>> language language-for-synopsis version commit hash
>>>> #:key
>>>> (repository-url
>>>> (format #f "https://github.com/tree-sitter/tree-sitter-~a" language))
>>>> (inputs '()))
>>>> (let ((synopsis (string-append language-for-synopsis
>>>> " grammar for tree-sitter"))
>>>> (name (string-append "tree-sitter-grammar-" language)))
>>>> (package
>>>> (name name)
>>>> (version version)
>>>> (home-page repository-url)
>>>> (source (origin
>>>> (method git-fetch)
>>>> (uri (git-reference
>>>> (url repository-url)
>>>> (commit commit)))
>>>> (file-name (git-file-name name version))
>>>> (sha256 (base32 hash))
>>>> (modules '((guix build utils)))
>>>> (snippet tree-sitter-delete-generated-files)))
>>>> (build-system tree-sitter-build-system)
>>>> (inputs inputs)
>>>> (synopsis synopsis)
>>>> (description (string-append synopsis "."))
>>>> (license license:expat))))
>>>>
>>>> (define-public tree-sitter-grammar-html
>>>> (tree-sitter-grammar
>>>> "html" "HTML"
>>>> "0.19.0" "v0.19.0"
>>>> "1hg7vbcy7bir6b8x11v0a4x0glvqnsqc3i2ixiarbxmycbgl3axy"))
>>>>
>>>> After that we can bring the rest of the grammars.
>>>
>>> I would suggest to rmeove the `tree-sitter-grammar' function, and keep
>>> grammars as "regular" package records, even though it's a little bit
>>> more verbose:
>>>
>>> --8<---------------cut here---------------start------------->8---
>>> (define-public tree-sitter-html
>>> (package
>>> (name "tree-sitter-html")
>>
>> It seems tree-sitter-html mimics upstream package name and probably make
>> more sense than tree-sitter-grammar-html used by me.
>
> Yeah, at some point I think I had named the packages with "grammar" as
> well, but thought it was a bit of a mouthful. I'm also thinking one day
> we may build language bindings as part of the build system (Rust and
> NodeJS I think ATM), so those packages could do more than ship the
> grammar in the future (although we don't know if we'll ever really need
> that).
>
>>
>>> (version "0.19.0")
>>> (source (origin
>>> (method git-fetch)
>>> (uri (git-reference
>>> (url "https://github.com/tree-sitter/tree-sitter-html")
>>> (commit (string-append "v" version))))
>>> (file-name (git-file-name name version))
>>> (sha256
>>> (base32
>>> "1hg7vbcy7bir6b8x11v0a4x0glvqnsqc3i2ixiarbxmycbgl3axy"))
>>> (modules '((guix build utils)))
>>> (snippet tree-sitter-delete-generated-files)))
>>> (build-system tree-sitter-build-system)
>>> (home-page "https://github.com/tree-sitter/tree-sitter-html")
>>> (synopsis "Tree-sitter HTML grammar")
>>> (description
>>> "This package provides a HTML grammar for the Tree-sitter library.")
>>> (license license:expat)))
>>> --8<---------------cut here---------------end--------------->8---
>>>
>>> This way, they look like any other package in Guix, which makes it
>>> easier for us to apply automatic changes in the future if needed (for
>>> example like how the input format could be automically updated for all
>>> "simple" package definitions, but had to be manual whenever custom code
>>> refactoring was done). Does this make sense?
>>
>> Make sense, but on the other hand we already have hunspell, aspell
>> dictionaries and probably a few more others, which are very similiar in
>> spirit and we already have to keep in mind their existence on such
>> automatic code updates.
>>
>> It looks that the packages differ only in url for the source code, lang
>> name and sometimes in inputs. Having template package function can make
>> management of shared parts more centralized, reduce possibility of
>> copy-paste mistakes, when the description wasn't updated and so on and
>> can reduce the amount of the code overall (which also reduces the change
>> of introducing an error).
>>
>> I don't have a strong opinion on this topic, but leaning towards the
>> template function slightly more, however I'm completely ok with the
>> standalone package definitions as well. WDYT?
>
> I can think of both cost/benefits to the template so I don't have a
> strong opinion either :-).
>
> I do like the template to make sure people don't forget to delete
> generated files, that's quite important as it seems upstream packages
> often check-in the generated C code. Although, we could probably assert
> that with in the build-system phase? I'll think about that.
>
> On the other hand, I wonder how the template works for packages that
> provide multiple grammars (see ocaml and typescript for example). I
> guess we could use the template for trivial packages, and standalone
> definitions for more complex ones? In general, if we keep the template
> interface really simple, then I'm happy with it.

Hi Pierre!

I spend two days trying grammars with and without helper function and
found hepler quite helpful to reduce boilerplate and errors from
copypaste, so I went the way with helper. The logic inside is quite
trivial, the only downside I found so far is that in cases when
repository url constructed automatically I can't easily open the repo
url in the browser.

I packaged all the grammars from this thread and a few more on top of
it. Updated them to usually latest versions, added some comments, when
needed.

If I forgot to reply on something or you have any comments/ideas, let me
know! :)

Kudos to Pierre and everyone, who helped with all the tree-sitter stuff.

--
Best regards,
Andrew Tropin
-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEEKEGaxlA4dEDH6S/6IgjSCVjB3rAFAmPri3IACgkQIgjSCVjB
3rC06g//XvLbq36ylpj8Um6MOevrYtlA45HNQGZwSQx1SYQmEcl5x+toCRlobD5Q
KcQWhFHXr7Fz29fdhC4lFbI3mJNe0+RIwkRiG0rpA0X7Z90rbSfvzziSygg8voGu
4/k+vShxTJbbDcdA6WuscaidGW3ZLi2AOM/zD1EwPhgf4T6x0Heh4C5cVxsaunLU
ydH36yz9/RXQHQR/yHfN74lMt43/+RRtHFgoz74ULreCOgYRIdbPF6Yr5UCbeYeN
qbhsRoFZLm/RsTxkWisiw7+pjm1oLnfJc33gOd+F/n/LmBW4Lnup9VXJo/jiCgZR
+50ZvMsDgJh+aYpng5kLtJHYJLtHTWNyukS0Xdx6gcZ6UP3GNJI2JN96dXRfjPQe
DX0rgWRfWIJDcBtpsUq1oIVZDzjaNXo4TpMed2XVLNE+H4deF2UgzR9IwGnaWPbc
Sw7cdcugMy9CxBYanz79EUtrzt5J20f4xE6uGYkXrSCwkdpgMscx3wO1i03QOCcA
AxvsALkayQrRSAR7qTuZBTPF57pjQedZgP1aKft6BkHN8PPcz7DKAerzcfDdxdTt
6iV932tmR4r2S7JeU9fF8q/7R7vtPLES4lxZivPnmrzN1d6tAvmTDhFQVEqXGTbS
xsjeSAxuzQvcOWioRDVxlmcZtfZBr2kZDBv+wOL5+yVWUZ13SQY=
=LF4M
-----END PGP SIGNATURE-----

P
P
Pierre Langlois wrote on 17 Feb 2023 13:37
[PATCH v8 0/7] Add tree-sitter support for Emacs 28
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20230217123722.15143-1-pierre.langlois@gmx.com
Hi Guix!

This is a rebase of the emacs-tree-sitter and emacs-tree-sitter-langs packages
on top of tree-sitter grammars that were merged. This brings tree-sitter to
Emacs 28, via a module.

When Emacs 29 is released with native support, some of this work may no longer
be required. However, the final emacs-tree-sitter-langs package is likely to
live on as a convenient way to add syntax highlighting support to major modes
that do not yet have native support. As it naturally will take time for
packages to transition over.

See this issue to track making emacs-tree-sitter-langs compatible with Emacs

Anymore feedback on these last few patches? I'm happy to finally be close to
finishing this work!

Thanks,
Pierre

Pierre Langlois (7):
gnu: Add tree-sitter-lua.
gnu: Add tree-sitter-rust@0.20.2.
gnu: Add rust-tree-sitter.
gnu: Add rust-tree-sitter-for-emacs.
gnu: Add emacs-tree-sitter-core.
gnu: Add emacs-tree-sitter.
gnu: Add emacs-tree-sitter-langs.

gnu/local.mk | 1 +
.../rust-tree-sitter-text-provider-fix.patch | 98 +++++
gnu/packages/tree-sitter.scm | 378 ++++++++++++++++++
3 files changed, 477 insertions(+)
create mode 100644 gnu/packages/patches/rust-tree-sitter-text-provider-fix.patch

--
2.39.1
P
P
Pierre Langlois wrote on 17 Feb 2023 13:37
[PATCH v8 1/7] gnu: Add tree-sitter-lua.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20230217123722.15143-2-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-lua): New variable.
---
gnu/packages/tree-sitter.scm | 7 +++++++
1 file changed, 7 insertions(+)

Toggle diff (20 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 8dda7737ca..395ae10d88 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -359,6 +359,13 @@ (define-public tree-sitter-julia
"1pbnmvhy2gq4vg1b0sjzmjm4s2gsgdjh7h01yj8qrrqbcl29c463"
"0.19.0"))
+(define-public tree-sitter-lua
+ (tree-sitter-grammar
+ "lua" "Lua"
+ "1hhffz45ilni4g9idsbpp1aymdjkjnyd4491xbdmr47bglf9cgc1"
+ "0.0.14"
+ #:repository-url "https://github.com/MunifTanjim/tree-sitter-lua"))
+
(define-public tree-sitter-ocaml
(tree-sitter-grammar
"ocaml" "OCaml (.ml and .mli)"
--
2.39.1
P
P
Pierre Langlois wrote on 17 Feb 2023 13:37
[PATCH v8 3/7] gnu: Add rust-tree-sitter.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20230217123722.15143-4-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (rust-tree-sitter): New variable.
---
gnu/packages/tree-sitter.scm | 37 ++++++++++++++++++++++++++++++++++++
1 file changed, 37 insertions(+)

Toggle diff (57 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index b202ebf4d3..7256ad02e4 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -30,6 +30,7 @@ (define-module (gnu packages tree-sitter)
#:use-module (guix build-system cargo)
#:use-module (guix build-system gnu)
#:use-module (guix build-system tree-sitter)
+ #:use-module (guix download)
#:use-module (guix gexp)
#:use-module (guix git-download)
#:use-module (guix packages)
@@ -188,6 +189,42 @@ (define-public tree-sitter-cli
This package includes the @command{tree-sitter} command-line tool.")
(license license:expat)))
+(define-public rust-tree-sitter
+ (package
+ (name "rust-tree-sitter")
+ (version "0.20.7")
+ (source (origin
+ (method url-fetch)
+ (uri (crate-uri "tree-sitter" version))
+ (file-name (string-append name "-" version ".tar.gz"))
+ (sha256
+ (base32
+ "183niy9malzr4dm81swcgl05xkwqamim6ck0gw5xb6k78nprz6jl"))
+ (modules '((guix build utils)))
+ (snippet
+ #~(begin
+ ;; Remove the runtime library code and dynamically link to
+ ;; it instead.
+ (delete-file-recursively "src")
+ (delete-file "binding_rust/build.rs")
+ (with-output-to-file "binding_rust/build.rs"
+ (lambda _
+ (format #t "fn main() {~@
+ println!(\"cargo:rustc-link-lib=tree-sitter\");~@
+ }~%")))))))
+ (build-system cargo-build-system)
+ (inputs (list tree-sitter))
+ (arguments
+ `(#:cargo-inputs
+ (("rust-cc" ,rust-cc-1)
+ ("rust-lazy-static" ,rust-lazy-static-1)
+ ("rust-regex" ,rust-regex-1))))
+ (home-page "https://tree-sitter.github.io/tree-sitter/")
+ (synopsis "Rust bindings to the Tree-sitter parsing library")
+ (description "This package provides Rust bindings to the Tree-sitter
+parsing library.")
+ (license license:expat)))
+
(define (tree-sitter-delete-generated-files grammar-directories)
#~(begin
(use-modules (guix build utils))
--
2.39.1
P
P
Pierre Langlois wrote on 17 Feb 2023 13:37
[PATCH v8 2/7] gnu: Add tree-sitter-rust@0.20.2.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20230217123722.15143-3-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-rust-0.20.2): New variable.
---
gnu/packages/tree-sitter.scm | 7 +++++++
1 file changed, 7 insertions(+)

Toggle diff (20 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 395ae10d88..b202ebf4d3 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -419,6 +419,13 @@ (define-public tree-sitter-rust
"149jhy01mqvavwa8jlxb8bnn7sxpfq2x1w35si6zn60b7kqjlx8f"
"0.20.3"))
+;; This version is needed by emacs-tree-sitter-langs.
+(define-public tree-sitter-rust-0.20.2
+ (tree-sitter-grammar
+ "rust" "Rust"
+ "03sd5hv9hc6ww1d53f4w5lbx3bbnkbm4hd7k38n2dmn9ldnmdf39"
+ "0.20.2"))
+
(define-public tree-sitter-clojure
(tree-sitter-grammar
"clojure" "Clojure"
--
2.39.1
P
P
Pierre Langlois wrote on 17 Feb 2023 13:37
[PATCH v8 6/7] gnu: Add emacs-tree-sitter.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20230217123722.15143-7-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (emacs-tree-sitter): New variable.
---
gnu/packages/tree-sitter.scm | 28 ++++++++++++++++++++++++++++
1 file changed, 28 insertions(+)

Toggle diff (38 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 12f2d108d9..f8be967e21 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -623,3 +623,31 @@ (define-public emacs-tree-sitter-core
(description "This package provides core APIs of the Emacs binding for
Tree-sitter, an incremental parsing system.")
(license license:expat)))
+
+(define-public emacs-tree-sitter
+ (package
+ (name "emacs-tree-sitter")
+ (version (package-version emacs-tree-sitter-core))
+ (source (package-source emacs-tree-sitter-core))
+ (build-system emacs-build-system)
+ (propagated-inputs
+ (list emacs-tree-sitter-core))
+ (arguments
+ (list
+ #:phases
+ #~(modify-phases %standard-phases
+ (add-after 'unpack 'chdir
+ (lambda _ (chdir "lisp"))))))
+ (home-page "https://emacs-tree-sitter.github.io")
+ (synopsis "Tree-sitter bindings for Emacs Lisp")
+ (description "This package provides Emacs bindings for Tree-sitter, an
+incremental parsing library. It aims to be the foundation for a new breed of
+Emacs packages that understand code structurally. For example:
+
+@enumerate
+@item Faster, fine-grained code highlighting.
+@item More flexible code folding.
+@item Structural editing (like Paredit, or even better) for non-Lisp code.
+@item More informative indexing for imenu.
+@end enumerate")
+ (license license:expat)))
--
2.39.1
P
P
Pierre Langlois wrote on 17 Feb 2023 13:37
[PATCH v8 5/7] gnu: Add emacs-tree-sitter-core.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20230217123722.15143-6-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (tree-sitter-emacs-module): New local variable.
(emacs-tree-sitter-core): New variable.
---
gnu/packages/tree-sitter.scm | 85 ++++++++++++++++++++++++++++++++++++
1 file changed, 85 insertions(+)

Toggle diff (102 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index bb9972cb00..12f2d108d9 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -29,6 +29,7 @@ (define-module (gnu packages tree-sitter)
#:use-module (gnu packages icu4c)
#:use-module (gnu packages node)
#:use-module (guix build-system cargo)
+ #:use-module (guix build-system emacs)
#:use-module (guix build-system gnu)
#:use-module (guix build-system tree-sitter)
#:use-module (guix download)
@@ -538,3 +539,87 @@ (define-public tree-sitter-racket
(git-version "0.1.0" revision commit)
#:repository-url "https://github.com/6cdh/tree-sitter-racket"
#:commit commit)))
+
+;; Local package definition solely for building the native emacs module
+;; written in Rust.
+(define tree-sitter-emacs-module
+ (package
+ (name "tree-sitter-emacs-module")
+ (version "0.18.0")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/emacs-tree-sitter/elisp-tree-sitter")
+ (commit version)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "1sdvz827v436qijs6xafakkfw2d16bvp8frymd818rppjc7a9dif"))))
+ (build-system cargo-build-system)
+ (inputs
+ (list tree-sitter))
+ (arguments
+ (list
+ #:cargo-inputs
+ `(("rust-anyhow" ,rust-anyhow-1)
+ ("rust-emacs" ,rust-emacs-0.18)
+ ("rust-libloading" ,rust-libloading-0.7)
+ ("rust-once-cell" ,rust-once-cell-1)
+ ("rust-tree-sitter" ,rust-tree-sitter-for-emacs))
+ #:phases
+ #~(modify-phases %standard-phases
+ (add-after 'unpack 'chdir
+ (lambda _ (chdir "core")))
+ (add-after 'chdir 'delete-cargo.lock
+ (lambda _ (delete-file "Cargo.lock")))
+ (add-after 'delete-cargo.lock 'do-not-fetch-from-github
+ (lambda _
+ (substitute* "Cargo.toml"
+ (("\\[patch.*") "")
+ (("git = .*") ""))))
+ (replace 'install
+ (lambda _
+ (let ((lib (string-append #$output "/lib")))
+ (mkdir-p lib)
+ (copy-file "target/release/libtsc_dyn.so"
+ (string-append lib "/tsc-dyn.so"))))))))
+ (home-page #f)
+ (synopsis #f)
+ (description #f)
+ (license license:expat)))
+
+(define-public emacs-tree-sitter-core
+ (package
+ (name "emacs-tree-sitter-core")
+ (version (package-version tree-sitter-emacs-module))
+ (source (package-source tree-sitter-emacs-module))
+ (build-system emacs-build-system)
+ (native-inputs
+ (list tree-sitter-emacs-module))
+ (arguments
+ (list
+ #:phases
+ #~(modify-phases %standard-phases
+ (add-after 'unpack 'chdir
+ (lambda _ (chdir "core")))
+ (add-before 'install 'find-tsc-dyn
+ (lambda* (#:key inputs #:allow-other-keys)
+ (let ((module (search-input-file inputs "/lib/tsc-dyn.so")))
+ ;; Install the tsc-dyn module in site-lisp and the current
+ ;; directory for test to pass.
+ (install-file module (elpa-directory #$output))
+ (copy-file module "tsc-dyn.so")
+ ;; We replace the tsc-dyn-get.el file with an empty stub to
+ ;; prevent the code from downloading the module.
+ (call-with-output-file "tsc-dyn-get.el"
+ (lambda (port)
+ (for-each
+ (lambda (sexp) (write sexp port))
+ '((defun tsc-dyn-get-ensure (requested)
+ nil)
+ (provide 'tsc-dyn-get)))))))))))
+ (home-page "https://emacs-tree-sitter.github.io")
+ (synopsis "Tree-sitter bindings for Emacs Lisp, core library")
+ (description "This package provides core APIs of the Emacs binding for
+Tree-sitter, an incremental parsing system.")
+ (license license:expat)))
--
2.39.1
P
P
Pierre Langlois wrote on 17 Feb 2023 13:37
[PATCH v8 4/7] gnu: Add rust-tree-sitter-for-emacs.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20230217123722.15143-5-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (rust-tree-sitter-for-emacs): New variable.
* gnu/packages/patches/rust-tree-sitter-text-provider-fix.patch: New file.
* gnu/local.mk (dist_patch_DATA): Register it.
---
gnu/local.mk | 1 +
.../rust-tree-sitter-text-provider-fix.patch | 98 +++++++++++++++++++
gnu/packages/tree-sitter.scm | 14 +++
3 files changed, 113 insertions(+)
create mode 100644 gnu/packages/patches/rust-tree-sitter-text-provider-fix.patch

Toggle diff (150 lines)
diff --git a/gnu/local.mk b/gnu/local.mk
index 5fcbdd4586..1dd98592b5 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -1831,6 +1831,7 @@ dist_patch_DATA = \
%D%/packages/patches/rust-nettle-disable-vendor.patch \
%D%/packages/patches/rust-nettle-sys-disable-vendor.patch \
%D%/packages/patches/rust-openssl-sys-no-vendor.patch \
+ %D%/packages/patches/rust-tree-sitter-text-provider-fix.patch \
%D%/packages/patches/rust-wl-clipboard-rs-newer-wl.patch \
%D%/packages/patches/rw-igraph-0.10.patch \
%D%/packages/patches/sbc-fix-build-non-x86.patch \
diff --git a/gnu/packages/patches/rust-tree-sitter-text-provider-fix.patch b/gnu/packages/patches/rust-tree-sitter-text-provider-fix.patch
new file mode 100644
index 0000000000..215e7fc18b
--- /dev/null
+++ b/gnu/packages/patches/rust-tree-sitter-text-provider-fix.patch
@@ -0,0 +1,98 @@
+From e3576b180488b8231e1fc0ca130748577579d129 Mon Sep 17 00:00:00 2001
+From: =?UTF-8?q?Tu=E1=BA=A5n-Anh=20Nguy=E1=BB=85n?= <ubolonton@gmail.com>
+Date: Sun, 25 Jul 2021 13:11:52 +0700
+Subject: [PATCH] Allow TextProvider's iterators to generate owned text
+
+---
+ binding_rust/lib.rs | 33 ++++++++++++++++++---------------
+ 1 file changed, 18 insertions(+), 15 deletions(-)
+
+diff --git a/lib/binding_rust/lib.rs b/lib/binding_rust/lib.rs
+index e88a411c..cf214d92 100644
+--- a/binding_rust/lib.rs
++++ b/binding_rust/lib.rs
+@@ -5,6 +5,7 @@ mod util;
+ use std::os::unix::io::AsRawFd;
+
+ use std::{
++ borrow::Cow,
+ char, error,
+ ffi::CStr,
+ fmt, hash, iter,
+@@ -183,7 +184,8 @@ pub struct QueryCaptures<'a, 'tree: 'a, T: TextProvider<'a>> {
+ }
+
+ pub trait TextProvider<'a> {
+- type I: Iterator<Item = &'a [u8]> + 'a;
++ type I: Iterator<Item = Cow<'a, [u8]>>;
++
+ fn text(&mut self, node: Node) -> Self::I;
+ }
+
+@@ -1840,19 +1842,19 @@ impl<'a, 'tree> QueryMatch<'a, 'tree> {
+ buffer2: &mut Vec<u8>,
+ text_provider: &mut impl TextProvider<'a>,
+ ) -> bool {
+- fn get_text<'a, 'b: 'a, I: Iterator<Item = &'b [u8]>>(
++ fn get_text<'a, 'b: 'a, I: Iterator<Item = Cow<'b, [u8]>>>(
+ buffer: &'a mut Vec<u8>,
+ mut chunks: I,
+- ) -> &'a [u8] {
+- let first_chunk = chunks.next().unwrap_or(&[]);
++ ) -> Cow<'a, [u8]> {
++ let first_chunk = chunks.next().unwrap_or(Cow::Owned(vec![0u8; 0]));
+ if let Some(next_chunk) = chunks.next() {
+ buffer.clear();
+- buffer.extend_from_slice(first_chunk);
+- buffer.extend_from_slice(next_chunk);
++ buffer.extend_from_slice(&first_chunk);
++ buffer.extend_from_slice(&next_chunk);
+ for chunk in chunks {
+- buffer.extend_from_slice(chunk);
++ buffer.extend_from_slice(&chunk);
+ }
+- buffer.as_slice()
++ Cow::Borrowed(buffer.as_slice())
+ } else {
+ first_chunk
+ }
+@@ -1888,7 +1890,7 @@ impl<'a, 'tree> QueryMatch<'a, 'tree> {
+ match node {
+ Some(node) => {
+ let text = get_text(buffer1, text_provider.text(node));
+- r.is_match(text) == *is_positive
++ r.is_match(&text) == *is_positive
+ }
+ None => true,
+ }
+@@ -2002,23 +2004,24 @@ impl<'cursor, 'tree> fmt::Debug for QueryMatch<'cursor, 'tree> {
+ }
+ }
+
+-impl<'a, F, I> TextProvider<'a> for F
++impl<'a, F, I, T> TextProvider<'a> for F
+ where
+ F: FnMut(Node) -> I,
+- I: Iterator<Item = &'a [u8]> + 'a,
++ T: Into<Cow<'a, [u8]>>,
++ I: Iterator<Item = T>,
+ {
+- type I = I;
++ type I = iter::Map<I, fn(T) -> Cow<'a, [u8]>>;
+
+ fn text(&mut self, node: Node) -> Self::I {
+- (self)(node)
++ (self)(node).map(T::into)
+ }
+ }
+
+ impl<'a> TextProvider<'a> for &'a [u8] {
+- type I = iter::Once<&'a [u8]>;
++ type I = iter::Once<Cow<'a, [u8]>>;
+
+ fn text(&mut self, node: Node) -> Self::I {
+- iter::once(&self[node.byte_range()])
++ iter::once(Cow::Borrowed(&self[node.byte_range()]))
+ }
+ }
+
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index 7256ad02e4..bb9972cb00 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -22,6 +22,7 @@
(define-module (gnu packages tree-sitter)
#:use-module ((guix licenses) #:prefix license:)
+ #:use-module (gnu packages)
#:use-module (gnu packages crates-graphics)
#:use-module (gnu packages crates-io)
#:use-module (gnu packages graphviz)
@@ -225,6 +226,19 @@ (define-public rust-tree-sitter
parsing library.")
(license license:expat)))
+;; We need to apply a patch in order to compile the rust bindings against the
+;; emacs tree-sitter module.
+;; See https://github.com/tree-sitter/tree-sitter/pull/1294
+(define-public rust-tree-sitter-for-emacs
+ (package (inherit rust-tree-sitter)
+ (source (origin
+ (inherit (package-source rust-tree-sitter))
+ (patches (search-patches
+ "rust-tree-sitter-text-provider-fix.patch"))))
+ ;; Do not show this package in the UI as it's only meant to be used for
+ ;; emacs's tree-sitter module.
+ (properties '((hidden? . #t)))))
+
(define (tree-sitter-delete-generated-files grammar-directories)
#~(begin
(use-modules (guix build utils))
--
2.39.1
P
P
Pierre Langlois wrote on 17 Feb 2023 13:37
[PATCH v8 7/7] gnu: Add emacs-tree-sitter-langs.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
20230217123722.15143-8-pierre.langlois@gmx.com
* gnu/packages/tree-sitter.scm (emacs-tree-sitter-langs-grammar-bundle): New
variable.
(emacs-tree-sitter-langs): New variable.
---
gnu/packages/tree-sitter.scm | 200 +++++++++++++++++++++++++++++++++++
1 file changed, 200 insertions(+)

Toggle diff (217 lines)
diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
index f8be967e21..2a05179bb4 100644
--- a/gnu/packages/tree-sitter.scm
+++ b/gnu/packages/tree-sitter.scm
@@ -32,6 +32,7 @@ (define-module (gnu packages tree-sitter)
#:use-module (guix build-system emacs)
#:use-module (guix build-system gnu)
#:use-module (guix build-system tree-sitter)
+ #:use-module (guix build-system trivial)
#:use-module (guix download)
#:use-module (guix gexp)
#:use-module (guix git-download)
@@ -651,3 +652,202 @@ (define-public emacs-tree-sitter
@item More informative indexing for imenu.
@end enumerate")
(license license:expat)))
+
+;; tree-sitter-langs expects all supported grammars to be bundled in a single
+;; directory. Doing this statically also allows us to check that grammars are
+;; compatible.
+(define emacs-tree-sitter-langs-grammar-bundle
+ (package
+ (name "emacs-tree-sitter-langs-grammar-bundle")
+ (source #f)
+ (version (package-version tree-sitter))
+ (build-system trivial-build-system)
+ (inputs
+ ;; When adding support for new languages, also add their names into the
+ ;; list in the package emacs-tree-sitter-langs package description.
+ ;; FIXME: Support for some languages is still left to package.
+ (list tree-sitter-bash
+ tree-sitter-c
+ tree-sitter-c-sharp
+ tree-sitter-cpp
+ tree-sitter-clojure
+ tree-sitter-css
+ tree-sitter-elixir
+ tree-sitter-elm
+ tree-sitter-go
+ tree-sitter-haskell
+ tree-sitter-html
+ tree-sitter-java
+ tree-sitter-javascript
+ tree-sitter-json
+ tree-sitter-julia
+ tree-sitter-lua
+ tree-sitter-markdown-gfm
+ tree-sitter-ocaml
+ tree-sitter-php
+ tree-sitter-python
+ tree-sitter-r
+ tree-sitter-ruby
+ tree-sitter-rust-0.20.2
+ tree-sitter-typescript))
+ (arguments
+ (list
+ #:builder
+ (with-imported-modules '((guix build union)
+ (guix build utils))
+ #~(begin
+ (use-modules (ice-9 match)
+ (srfi srfi-1)
+ (guix build union)
+ (guix build utils))
+ (mkdir-p #$output)
+ (for-each
+ (lambda (lib)
+ ;; tree-sitter-langs expects grammars to be as "<lang>.so"
+ ;; rather than "libtree-sitter-<lang>.so".
+ (symlink lib
+ (string-append
+ #$output
+ "/"
+ (substring (basename lib)
+ (string-length "libtree-sitter-")))))
+ (append-map (match-lambda
+ ((name directory)
+ (find-files directory "libtree-sitter-.*\\.so$")))
+ '#$(package-inputs this-package)))))))
+ (synopsis #f)
+ (description #f)
+ (home-page #f)
+ (license #f)))
+
+(define-public emacs-tree-sitter-langs
+ (package
+ (name "emacs-tree-sitter-langs")
+ (version "0.12.16")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/emacs-tree-sitter/tree-sitter-langs")
+ (commit version)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "1i8lhv404ijj52jjxrmljnrhby9jlrdgpymn2jfs4kml0x7qhsna"))))
+ (build-system emacs-build-system)
+ (inputs
+ (list emacs-tree-sitter-langs-grammar-bundle))
+ (propagated-inputs
+ (list emacs-tree-sitter))
+ (arguments
+ (list
+ #:tests? #t
+ #:test-command ''("script/test")
+ #:phases
+ #~(modify-phases %standard-phases
+ (add-after 'unpack 'disable-downloader
+ (lambda _
+ (call-with-output-file "tree-sitter-langs-build.el"
+ (lambda (port)
+ (let ((on-load-message
+ (string-append
+ "tree-sitter-langs: Grammar bundle already installed "
+ "via Guix. Installing external grammars via this "
+ "function isn't supported, if a language you need is "
+ "missing please report a bug at bug-guix@gnu.org.")))
+ (format
+ port
+ ";;;###autoload
+ (defun tree-sitter-langs-install-grammars
+ (&optional skip-if-installed version os
+ keep-bundle)
+ (interactive)
+ (message \"~a\"))
+ (defconst tree-sitter-langs--queries-dir
+ (file-name-as-directory
+ (concat (file-name-directory
+ (locate-library \"tree-sitter-langs.el\"))
+ \"queries\")))
+ (defun tree-sitter-langs--bin-dir () \"~a\")
+ (provide 'tree-sitter-langs-build)"
+ on-load-message
+ #$emacs-tree-sitter-langs-grammar-bundle))))))
+ (add-after 'unpack 'remove-cask
+ (lambda _
+ (substitute* "script/test"
+ (("cask") ""))))
+ (add-before 'check 'patch-tree-sitter-require-test
+ (lambda _
+ (use-modules (ice-9 regex))
+ ;; This test needs a git repositories with submodules for
+ ;; each languages in order to map all repositories. We patch
+ ;; the mapping function with one that invokes the tests for
+ ;; each packaged language.
+ (let ((supported-languages
+ (map (lambda (lib)
+ (match:substring
+ (string-match "(.*)\\.so$" (basename lib))
+ 1))
+ (find-files "bin" "\\.so$"))))
+ (substitute* "tree-sitter-langs-tests.el"
+ (("tree-sitter-langs--map-repos")
+ (call-with-output-string
+ (lambda (port)
+ (write `(lambda (fn)
+ (dolist (lang ',supported-languages)
+ (funcall fn lang)))
+ port))))))))
+ ;; Tests for queries will fail given those languages are not
+ ;; packaged yet.
+ (add-before 'check 'remove-unused-highlight-queries
+ (lambda _
+ (delete-file-recursively "queries/hcl")
+ (delete-file-recursively "queries/pgn")
+ (delete-file-recursively "queries/yaml")))
+ (add-after 'install 'install-queries
+ (lambda _
+ (copy-recursively
+ "queries"
+ (string-append (elpa-directory #$output) "/queries")))))))
+ (home-page "https://emacs-tree-sitter.github.io/languages/")
+ (synopsis "Language support bundle for Tree-sitter")
+ (description "This package is a convenient language bundle for
+Tree-sitter. For each supported language, this package provides:
+
+@enumerate
+@item Pre-compiled grammar binaries.
+@item An optional highlights.scm file that provides highlighting patterns.
+This is mainly intended for major modes that are not aware of tree-sitter.
+@item Optional query patterns for other minor modes that provide high-level
+functionalities on top of tree-sitter, such as code folding, evil text
+objects, ...etc.
+@end enumerate
+
+The following languages are currently supported in Guix:
+
+@itemize
+@item Bash
+@item C
+@item C#
+@item C++
+@item Clojure
+@item CSS
+@item Elixir
+@item Elm
+@item Go
+@item Haskell
+@item HTML
+@item Java
+@item JavaScript
+@item JSON
+@item Julia
+@item Lua
+@item Markdown
+@item Ocaml
+@item PHP
+@item Python
+@item R
+@item Ruby
+@item Rust
+@item Typescript
+@end itemize")
+ (license license:expat)))
--
2.39.1
P
P
Pierre Langlois wrote on 17 Feb 2023 13:38
Re: [bug#49946] [PATCH v7 01/32] gnu: tree-sitter: Move to its own module.
(name . Andrew Tropin)(address . andrew@trop.in)
873574v53t.fsf@gmx.com
Hi,

Andrew Tropin <andrew@trop.in> writes:

Toggle quote (236 lines)
> [[PGP Signed Part:Undecided]]
> On 2023-02-12 12:07, Pierre Langlois wrote:
>
>> Hi,
>>
>> Andrew Tropin <andrew@trop.in> writes:
>>
>>> [[PGP Signed Part:Undecided]]
>>> On 2023-02-10 15:48, Pierre Langlois wrote:
>>>
>>>> Hi Andrew, thanks for pushing this along! It's great to see things
>>>> getting merged.
>>>>
>>>> Andrew Tropin <andrew@trop.in> writes:
>>>>
>>>>> [[PGP Signed Part:Undecided]]
>>>>> On 2023-02-09 18:04, Andrew Tropin wrote:
>>>>>
>>>>>> On 2023-02-09 13:39, zimoun wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> On Thu, 09 Feb 2023 at 14:11, Andrew Tropin <andrew@trop.in> wrote:
>>>>>>>
>>>>>>>> I applied tree-sitter and tree-sitter-cli patches,
>>>>>>>
>>>>>>> Just to be sure to understand, you have only applied 02/32 and 05/32,
>>>>>>> right?
>>>>>>>
>>>>>>>
>>>>>>> [bug#49946] [PATCH v7 02/32] gnu: tree-sitter: Update to 0.20.7.
>>>>>>> id:20221125012142.22579-3-pierre.langlois@gmx.com
>>>>>>> http://issues.guix.gnu.org/msgid/20221125012142.22579-3-pierre.langlois@gmx.com
>>>>>>>
>>>>>>> [bug#49946] [PATCH v7 05/32] gnu: Add tree-sitter-cli.
>>>>>>> id:20221125012142.22579-6-pierre.langlois@gmx.com
>>>>>>> http://issues.guix.gnu.org/msgid/20221125012142.22579-6-pierre.langlois@gmx.com
>>>>>>>
>>>>>>> Leaving out all the others, right?
>>>>>>
>>>>>> Merged first 5 patches from 01 to 05, also added one more commit, which
>>>>>> addresses some things from reviews and one commit, which adds html
>>>>>> grammar.
>>>>>>
>>>>>> The html grammar is added for the testing purposes. It relies on
>>>>>> generated parser.c and scanner.c and we will need to repackage it using
>>>>>> grammar.js instead. I'm not sure if a separate build system is needed
>>>>>> for this, I guess we can just rewrite tree-sitter-grammar function,
>>>>>> which generates packages as in example with tree-sitter-grammar-html:
>>>>>> https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/tree-sitter.scm?h=53b00b91b73bd60412d5bd057e22e6d63194a7f7#n158
>>>>>>
>>>>>> Anyway, I only skimmed tree-sitter-build-system source code, and plan to
>>>>>> read it carefully, evaluate and either introduce new build system or
>>>>>> just move all needed parts to tree-sitter-grammar function. WDYT?
>>>>>> After we done with it we can package all other grammars.
>>>>>
>>>>> Ok, I realized that the proper build process for tree-sitter grammars is
>>>>> a little harder than I expected, tree-sitter-build system make sense. I
>>>>> reviewed it, made a small change:
>>>>
>>>> Ah great, I was going to comment to try and push for us to keep the
>>>> build system. I originally went with a template package and inheritance,
>>>> but Maxime suggested moving to a build-system which ended up making the
>>>> package definitions a *lot* nicer IMO (see previous discussion here
>>>> https://issues.guix.gnu.org/49946#144). It also allows us to deal with
>>>> grammars that depend on each other more nicely I think.
>>>>
>>>>>
>>>>> @@ -29,7 +29,7 @@ (define-module (guix build tree-sitter-build-system)
>>>>> ;; Commentary:
>>>>> ;;
>>>>> ;; Build procedures for tree-sitter grammar packages. This is the
>>>>> -;; builder-side code, which builds on top fo the node build-system.
>>>>> +;; builder-side code, which builds on top of the node build-system.
>>>>> ;;
>>>>> ;; Tree-sitter grammars are written in JavaScript and compiled to a native
>>>>> ;; shared object. The `tree-sitter generate' command invokes `node' in order
>>>>> @@ -114,7 +114,7 @@ (define (compile-language dir)
>>>>> "-fno-exceptions"
>>>>> "-O2"
>>>>> "-g"
>>>>> - "-o" ,(string-append lib "/" lang ".so")
>>>>> + "-o" ,(string-append lib "/libtree-sitter-" lang ".so")
>>>>> ;; An additional `scanner.{c,cc}' file is sometimes
>>>>> ;; provided.
>>>>> ,@(cond
>>>>>
>>>>>
>>>>> rewrote html grammar to use this build system and made it work with
>>>>> built-in treesit package. Also, tried examples of c and cpp grammars
>>>>> from patches in this thread.
>>>>>
>>>>> If you ok with it, I'll push the build system to master and update the
>>>>> html grammar accordingly.
>>>>>
>>>>> The final result will look like this:
>>>>>
>>>>> (define tree-sitter-delete-generated-files
>>>>> #~(begin
>>>>> (delete-file "binding.gyp")
>>>>> (delete-file-recursively "bindings")
>>>>> (delete-file "src/grammar.json")
>>>>> (delete-file "src/node-types.json")
>>>>> (delete-file "src/parser.c")
>>>>> (delete-file-recursively "src/tree_sitter")))
>>>>>
>>>>> (define* (tree-sitter-grammar
>>>>> language language-for-synopsis version commit hash
>>>>> #:key
>>>>> (repository-url
>>>>> (format #f "https://github.com/tree-sitter/tree-sitter-~a" language))
>>>>> (inputs '()))
>>>>> (let ((synopsis (string-append language-for-synopsis
>>>>> " grammar for tree-sitter"))
>>>>> (name (string-append "tree-sitter-grammar-" language)))
>>>>> (package
>>>>> (name name)
>>>>> (version version)
>>>>> (home-page repository-url)
>>>>> (source (origin
>>>>> (method git-fetch)
>>>>> (uri (git-reference
>>>>> (url repository-url)
>>>>> (commit commit)))
>>>>> (file-name (git-file-name name version))
>>>>> (sha256 (base32 hash))
>>>>> (modules '((guix build utils)))
>>>>> (snippet tree-sitter-delete-generated-files)))
>>>>> (build-system tree-sitter-build-system)
>>>>> (inputs inputs)
>>>>> (synopsis synopsis)
>>>>> (description (string-append synopsis "."))
>>>>> (license license:expat))))
>>>>>
>>>>> (define-public tree-sitter-grammar-html
>>>>> (tree-sitter-grammar
>>>>> "html" "HTML"
>>>>> "0.19.0" "v0.19.0"
>>>>> "1hg7vbcy7bir6b8x11v0a4x0glvqnsqc3i2ixiarbxmycbgl3axy"))
>>>>>
>>>>> After that we can bring the rest of the grammars.
>>>>
>>>> I would suggest to rmeove the `tree-sitter-grammar' function, and keep
>>>> grammars as "regular" package records, even though it's a little bit
>>>> more verbose:
>>>>
>>>> --8<---------------cut here---------------start------------->8---
>>>> (define-public tree-sitter-html
>>>> (package
>>>> (name "tree-sitter-html")
>>>
>>> It seems tree-sitter-html mimics upstream package name and probably make
>>> more sense than tree-sitter-grammar-html used by me.
>>
>> Yeah, at some point I think I had named the packages with "grammar" as
>> well, but thought it was a bit of a mouthful. I'm also thinking one day
>> we may build language bindings as part of the build system (Rust and
>> NodeJS I think ATM), so those packages could do more than ship the
>> grammar in the future (although we don't know if we'll ever really need
>> that).
>>
>>>
>>>> (version "0.19.0")
>>>> (source (origin
>>>> (method git-fetch)
>>>> (uri (git-reference
>>>> (url "https://github.com/tree-sitter/tree-sitter-html")
>>>> (commit (string-append "v" version))))
>>>> (file-name (git-file-name name version))
>>>> (sha256
>>>> (base32
>>>> "1hg7vbcy7bir6b8x11v0a4x0glvqnsqc3i2ixiarbxmycbgl3axy"))
>>>> (modules '((guix build utils)))
>>>> (snippet tree-sitter-delete-generated-files)))
>>>> (build-system tree-sitter-build-system)
>>>> (home-page "https://github.com/tree-sitter/tree-sitter-html")
>>>> (synopsis "Tree-sitter HTML grammar")
>>>> (description
>>>> "This package provides a HTML grammar for the Tree-sitter library.")
>>>> (license license:expat)))
>>>> --8<---------------cut here---------------end--------------->8---
>>>>
>>>> This way, they look like any other package in Guix, which makes it
>>>> easier for us to apply automatic changes in the future if needed (for
>>>> example like how the input format could be automically updated for all
>>>> "simple" package definitions, but had to be manual whenever custom code
>>>> refactoring was done). Does this make sense?
>>>
>>> Make sense, but on the other hand we already have hunspell, aspell
>>> dictionaries and probably a few more others, which are very similiar in
>>> spirit and we already have to keep in mind their existence on such
>>> automatic code updates.
>>>
>>> It looks that the packages differ only in url for the source code, lang
>>> name and sometimes in inputs. Having template package function can make
>>> management of shared parts more centralized, reduce possibility of
>>> copy-paste mistakes, when the description wasn't updated and so on and
>>> can reduce the amount of the code overall (which also reduces the change
>>> of introducing an error).
>>>
>>> I don't have a strong opinion on this topic, but leaning towards the
>>> template function slightly more, however I'm completely ok with the
>>> standalone package definitions as well. WDYT?
>>
>> I can think of both cost/benefits to the template so I don't have a
>> strong opinion either :-).
>>
>> I do like the template to make sure people don't forget to delete
>> generated files, that's quite important as it seems upstream packages
>> often check-in the generated C code. Although, we could probably assert
>> that with in the build-system phase? I'll think about that.
>>
>> On the other hand, I wonder how the template works for packages that
>> provide multiple grammars (see ocaml and typescript for example). I
>> guess we could use the template for trivial packages, and standalone
>> definitions for more complex ones? In general, if we keep the template
>> interface really simple, then I'm happy with it.
>
> Hi Pierre!
>
> I spend two days trying grammars with and without helper function and
> found hepler quite helpful to reduce boilerplate and errors from
> copypaste, so I went the way with helper. The logic inside is quite
> trivial, the only downside I found so far is that in cases when
> repository url constructed automatically I can't easily open the repo
> url in the browser.
>
> I packaged all the grammars from this thread and a few more on top of
> it. Updated them to usually latest versions, added some comments, when
> needed.
>
> If I forgot to reply on something or you have any comments/ideas, let me
> know! :)
>
> Kudos to Pierre and everyone, who helped with all the tree-sitter stuff.

Thank you for landing all this work!
-----BEGIN PGP SIGNATURE-----

iQFMBAEBCgA2FiEEctU9gYy29KFyWDdMqPyeRH9PfVQFAmPvdWYYHHBpZXJyZS5s
YW5nbG9pc0BnbXguY29tAAoJEKj8nkR/T31Uyb0IALOneatOvzeS+nrtS0ryGEga
hwPLzlJH4ZJfPd9zMHz0p1z0H191C155dP1ejGYnTK0QJzepqjVypzvbUl8MKVkt
p4aCm3Aid9yYW2QJ9pfmE0mc4PNH0je82Fps9w8QwYZwT2lc7rRDer1UAzM22LBD
GLTiLeTLWbAlA8/klpaOURkPs7E57BrHLbO554B0Vf5OHC+zp9SD7WdohTfgUC3O
gjXGCDgTrF+n4S+YR40fZRX+AEClfAYL6GGXgn6OelmX6Z1F2qOWTTcmCbgG6em3
ge/EbPSqJfvjQySLUXYUx8wEwAdvs/n8XI6znnuSH14IEVM+H6ce3gXPv3PcZNs=
=2XjL
-----END PGP SIGNATURE-----

P
P
Pierre Langlois wrote on 17 Feb 2023 13:48
Re: [PATCH v8 5/7] gnu: Add emacs-tree-sitter-core.
(address . 49946@debbugs.gnu.org)(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
87sff4tpyz.fsf@gmx.com
Pierre Langlois <pierre.langlois@gmx.com> writes:

Toggle quote (80 lines)
> * gnu/packages/tree-sitter.scm (tree-sitter-emacs-module): New local variable.
> (emacs-tree-sitter-core): New variable.
> ---
> gnu/packages/tree-sitter.scm | 85 ++++++++++++++++++++++++++++++++++++
> 1 file changed, 85 insertions(+)
>
> diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm
> index bb9972cb00..12f2d108d9 100644
> --- a/gnu/packages/tree-sitter.scm
> +++ b/gnu/packages/tree-sitter.scm
> @@ -29,6 +29,7 @@ (define-module (gnu packages tree-sitter)
> #:use-module (gnu packages icu4c)
> #:use-module (gnu packages node)
> #:use-module (guix build-system cargo)
> + #:use-module (guix build-system emacs)
> #:use-module (guix build-system gnu)
> #:use-module (guix build-system tree-sitter)
> #:use-module (guix download)
> @@ -538,3 +539,87 @@ (define-public tree-sitter-racket
> (git-version "0.1.0" revision commit)
> #:repository-url "https://github.com/6cdh/tree-sitter-racket"
> #:commit commit)))
> +
> +;; Local package definition solely for building the native emacs module
> +;; written in Rust.
> +(define tree-sitter-emacs-module
> + (package
> + (name "tree-sitter-emacs-module")
> + (version "0.18.0")
> + (source (origin
> + (method git-fetch)
> + (uri (git-reference
> + (url "https://github.com/emacs-tree-sitter/elisp-tree-sitter")
> + (commit version)))
> + (file-name (git-file-name name version))
> + (sha256
> + (base32
> + "1sdvz827v436qijs6xafakkfw2d16bvp8frymd818rppjc7a9dif"))))
> + (build-system cargo-build-system)
> + (inputs
> + (list tree-sitter))
> + (arguments
> + (list
> + #:cargo-inputs
> + `(("rust-anyhow" ,rust-anyhow-1)
> + ("rust-emacs" ,rust-emacs-0.18)
> + ("rust-libloading" ,rust-libloading-0.7)
> + ("rust-once-cell" ,rust-once-cell-1)
> + ("rust-tree-sitter" ,rust-tree-sitter-for-emacs))
> + #:phases
> + #~(modify-phases %standard-phases
> + (add-after 'unpack 'chdir
> + (lambda _ (chdir "core")))
> + (add-after 'chdir 'delete-cargo.lock
> + (lambda _ (delete-file "Cargo.lock")))
> + (add-after 'delete-cargo.lock 'do-not-fetch-from-github
> + (lambda _
> + (substitute* "Cargo.toml"
> + (("\\[patch.*") "")
> + (("git = .*") ""))))
> + (replace 'install
> + (lambda _
> + (let ((lib (string-append #$output "/lib")))
> + (mkdir-p lib)
> + (copy-file "target/release/libtsc_dyn.so"
> + (string-append lib "/tsc-dyn.so"))))))))
> + (home-page #f)
> + (synopsis #f)
> + (description #f)
> + (license license:expat)))
> +
> +(define-public emacs-tree-sitter-core
> + (package
> + (name "emacs-tree-sitter-core")
> + (version (package-version tree-sitter-emacs-module))
> + (source (package-source tree-sitter-emacs-module))
> + (build-system emacs-build-system)
> + (native-inputs
> + (list tree-sitter-emacs-module))

Wait, I think this should have been a regular input, not a native-input,
otherwise it'll break if we ever add cross-compilation support.

Toggle quote (27 lines)
> + (arguments
> + (list
> + #:phases
> + #~(modify-phases %standard-phases
> + (add-after 'unpack 'chdir
> + (lambda _ (chdir "core")))
> + (add-before 'install 'find-tsc-dyn
> + (lambda* (#:key inputs #:allow-other-keys)
> + (let ((module (search-input-file inputs "/lib/tsc-dyn.so")))
> + ;; Install the tsc-dyn module in site-lisp and the current
> + ;; directory for test to pass.
> + (install-file module (elpa-directory #$output))
> + (copy-file module "tsc-dyn.so")
> + ;; We replace the tsc-dyn-get.el file with an empty stub to
> + ;; prevent the code from downloading the module.
> + (call-with-output-file "tsc-dyn-get.el"
> + (lambda (port)
> + (for-each
> + (lambda (sexp) (write sexp port))
> + '((defun tsc-dyn-get-ensure (requested)
> + nil)
> + (provide 'tsc-dyn-get)))))))))))
> + (home-page "https://emacs-tree-sitter.github.io")
> + (synopsis "Tree-sitter bindings for Emacs Lisp, core library")
> + (description "This package provides core APIs of the Emacs binding for
> +Tree-sitter, an incremental parsing system.")
> + (license license:expat)))
-----BEGIN PGP SIGNATURE-----

iQFMBAEBCgA2FiEEctU9gYy29KFyWDdMqPyeRH9PfVQFAmPveEQYHHBpZXJyZS5s
YW5nbG9pc0BnbXguY29tAAoJEKj8nkR/T31UvgAIAIGI9qpyfnkSFMMdo9vduDxa
X+e+0p8MxMjx/vBdHY322TYbKE0nFObTTt45clczxVrhDv/zq97VqSHyF0l1O5nr
0w7IztFyhml01PLMmfxBOuMnZO5cREglvC9Ev9CIDLNfPEzFRlPliG2VbMK3L6RX
lx+Yf3a7FGxhxSvmy4bAPzEPofayoui4DyD/4Tb+ShvLict5i8Xq1YwuTYZ/y0oS
Gp1Gkv6vERZn4l9BFBLqGRR1OSqdC+SmqQ0Cv9vMj0GXLns8cFGfspYrA87KmwiT
8Q/pCPCqQDBdLcdxX1yckC7JYRLOJEtBZKkJmUZtlpw5sfYvRALzwugtRKJRgOw=
=q6wG
-----END PGP SIGNATURE-----

?