stavola.xyz / mlf
A human-friendly DSL for ATProto Lexicons
27
fork

Configure Feed

Select the types of activity you want to include in your feed.

dev instructions for tree-sitter-mlf fail #5

open opened by
vmx.cx

I've tried to look into tree-sitter-mlf. It didn't build for me with the dev instructions of the README. This can well be a local env issue, but I thought I report it though.

> git clone https://tangled.org/stavola.xyz/mlf
Cloning into 'mlf'...
remote: Enumerating objects: 1410, done.
remote: Counting objects: 100% (1410/1410), done.
remote: Compressing objects: 100% (1079/1079), done.
remote: Total 1410 (delta 636), reused 572 (delta 202), pack-reused 0 (from 0)
Receiving objects: 100% (1410/1410), 518.79 KiB | 2.98 MiB/s, done.
Resolving deltas: 100% (636/636), done.
> cd mlf/tree-sitter-mlf
> npm install

> tree-sitter-mlf@0.1.0 install
> node-gyp rebuild

gyp info it worked if it ends with ok
gyp info using node-gyp@12.2.0
gyp info using node@24.15.0 | linux | x64
gyp info find Python using Python version 3.13.2 found at "/usr/bin/python3"

gyp info spawn /usr/bin/python3
gyp info spawn args [
gyp info spawn args '/home/vmx/.local/lib/node_modules/npm/node_modules/node-gyp/gyp/gyp_main.py',
gyp info spawn args 'binding.gyp',
gyp info spawn args '-f',
gyp info spawn args 'make',
gyp info spawn args '-I',
gyp info spawn args '/tmp/mlf/tree-sitter-mlf/build/config.gypi',
gyp info spawn args '-I',
gyp info spawn args '/home/vmx/.local/lib/node_modules/npm/node_modules/node-gyp/addon.gypi',
gyp info spawn args '-I',
gyp info spawn args '/home/vmx/.cache/node-gyp/24.15.0/include/node/common.gypi',
gyp info spawn args '-Dlibrary=shared_library',
gyp info spawn args '-Dvisibility=default',
gyp info spawn args '-Dnode_root_dir=/home/vmx/.cache/node-gyp/24.15.0',
gyp info spawn args '-Dnode_gyp_dir=/home/vmx/.local/lib/node_modules/npm/node_modules/node-gyp',
gyp info spawn args '-Dnode_lib_file=/home/vmx/.cache/node-gyp/24.15.0/<(target_arch)/node.lib',
gyp info spawn args '-Dmodule_root_dir=/tmp/mlf/tree-sitter-mlf',
gyp info spawn args '-Dnode_engine=v8',
gyp info spawn args '--depth=.',
gyp info spawn args '--no-parallel',
gyp info spawn args '--generator-output',
gyp info spawn args 'build',
gyp info spawn args '-Goutput_dir=.'
gyp info spawn args ]
gyp info spawn make
gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ]
make: Entering directory '/tmp/mlf/tree-sitter-mlf/build'
  TOUCH Release/obj.target/node_modules/node-addon-api/node_addon_api_except.stamp
  CXX(target) Release/obj.target/tree_sitter_mlf_binding/bindings/node/binding.o
../bindings/node/binding.cc:1:10: fatal error: tree_sitter/parser.h: No such file or directory
    1 | #include "tree_sitter/parser.h"
      |          ^~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
make: *** [tree_sitter_mlf_binding.target.mk:118: Release/obj.target/tree_sitter_mlf_binding/bindings/node/binding.o] Error 1
make: Leaving directory '/tmp/mlf/tree-sitter-mlf/build'
gyp ERR! build error 
gyp ERR! stack Error: `make` failed with exit code: 2
gyp ERR! stack at ChildProcess.<anonymous> (/home/vmx/.local/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:219:23)
gyp ERR! System Linux 6.12.19-amd64
gyp ERR! command "/home/vmx/.local/bin/node" "/home/vmx/.local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
gyp ERR! cwd /tmp/mlf/tree-sitter-mlf
gyp ERR! node -v v24.15.0
gyp ERR! node-gyp -v v12.2.0
gyp ERR! $npm_package_name tree-sitter-mlf
gyp ERR! $npm_package_version 0.1.0
gyp ERR! not ok 
npm error code 1
npm error path /tmp/mlf/tree-sitter-mlf
npm error command failed
npm error command sh -c node-gyp rebuild
npm error A complete log of this run can be found in: /home/vmx/.npm/_logs/2026-04-16T20_07_29_061Z-debug-0.log

But when I then run a npx tree-sitter generate it works:

> npx tree-sitter generate
Adding an install script to package.json
Adding a prebuildify script to package.json
Adding files to package.json
Updated build.rs with the /utf-8 flag for Windows compilation
Replacing index.js with new binding API
Replacing binding.cc with new binding API
> $ npm install

> tree-sitter-mlf@0.1.0 install
> node-gyp-build

gyp info it worked if it ends with ok
gyp info using node-gyp@12.2.0
gyp info using node@24.15.0 | linux | x64
gyp info find Python using Python version 3.13.2 found at "/usr/bin/python3"

gyp info spawn /usr/bin/python3
gyp info spawn args [
gyp info spawn args '/home/vmx/.local/lib/node_modules/npm/node_modules/node-gyp/gyp/gyp_main.py',
gyp info spawn args 'binding.gyp',
gyp info spawn args '-f',
gyp info spawn args 'make',
gyp info spawn args '-I',
gyp info spawn args '/tmp/mlf/tree-sitter-mlf/build/config.gypi',
gyp info spawn args '-I',
gyp info spawn args '/home/vmx/.local/lib/node_modules/npm/node_modules/node-gyp/addon.gypi',
gyp info spawn args '-I',
gyp info spawn args '/home/vmx/.cache/node-gyp/24.15.0/include/node/common.gypi',
gyp info spawn args '-Dlibrary=shared_library',
gyp info spawn args '-Dvisibility=default',
gyp info spawn args '-Dnode_root_dir=/home/vmx/.cache/node-gyp/24.15.0',
gyp info spawn args '-Dnode_gyp_dir=/home/vmx/.local/lib/node_modules/npm/node_modules/node-gyp',
gyp info spawn args '-Dnode_lib_file=/home/vmx/.cache/node-gyp/24.15.0/<(target_arch)/node.lib',
gyp info spawn args '-Dmodule_root_dir=/tmp/mlf/tree-sitter-mlf',
gyp info spawn args '-Dnode_engine=v8',
gyp info spawn args '--depth=.',
gyp info spawn args '--no-parallel',
gyp info spawn args '--generator-output',
gyp info spawn args 'build',
gyp info spawn args '-Goutput_dir=.'
gyp info spawn args ]
gyp info spawn make
gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ]
make: Entering directory '/tmp/mlf/tree-sitter-mlf/build'
  TOUCH Release/obj.target/node_modules/node-addon-api/node_addon_api_except.stamp
  CXX(target) Release/obj.target/tree_sitter_mlf_binding/bindings/node/binding.o
  CC(target) Release/obj.target/tree_sitter_mlf_binding/src/parser.o
  SOLINK_MODULE(target) Release/obj.target/tree_sitter_mlf_binding.node
  COPY Release/tree_sitter_mlf_binding.node
make: Leaving directory '/tmp/mlf/tree-sitter-mlf/build'
gyp info ok 

up to date, audited 28 packages in 4s

5 packages are looking for funding
  run `npm fund` for details

found 0 vulnerabilities

It's the same with Node.js v25.9.0.

The diff after the generate is:

diff --git a/tree-sitter-mlf/bindings/node/binding.cc b/tree-sitter-mlf/bindings/node/binding.cc
index 6812e05..29fd8e6 100644
--- a/tree-sitter-mlf/bindings/node/binding.cc
+++ b/tree-sitter-mlf/bindings/node/binding.cc
@@ -1,28 +1,20 @@
-#include "tree_sitter/parser.h"
-#include <node.h>
-#include "nan.h"
+#include <napi.h>
 
-using namespace v8;
+typedef struct TSLanguage TSLanguage;
 
-extern "C" TSLanguage * tree_sitter_mlf();
+extern "C" TSLanguage *tree_sitter_mlf();
 
-namespace {
+// "tree-sitter", "language" hashed with BLAKE2
+const napi_type_tag LANGUAGE_TYPE_TAG = {
+  0x8AF2E5212AD58ABF, 0xD5006CAD83ABBA16
+};
 
-NAN_METHOD(New) {}
-
-void Init(Local<Object> exports, Local<Object> module) {
-  Local<FunctionTemplate> tpl = Nan::New<FunctionTemplate>(New);
-  tpl->SetClassName(Nan::New("Language").ToLocalChecked());
-  tpl->InstanceTemplate()->SetInternalFieldCount(1);
-
-  Local<Function> constructor = Nan::GetFunction(tpl).ToLocalChecked();
-  Local<Object> instance = constructor->NewInstance(Nan::GetCurrentContext()).ToLocalChecked();
-  Nan::SetInternalFieldPointer(instance, 0, tree_sitter_mlf());
-
-  Nan::Set(instance, Nan::New("name").ToLocalChecked(), Nan::New("mlf").ToLocalChecked());
-  Nan::Set(module, Nan::New("exports").ToLocalChecked(), instance);
+Napi::Object Init(Napi::Env env, Napi::Object exports) {
+    exports["name"] = Napi::String::New(env, "mlf");
+    auto language = Napi::External<TSLanguage>::New(env, tree_sitter_mlf());
+    language.TypeTag(&LANGUAGE_TYPE_TAG);
+    exports["language"] = language;
+    return exports;
 }
 
-NODE_MODULE(tree_sitter_mlf_binding, Init)
-
-}  // namespace
+NODE_API_MODULE(tree_sitter_mlf_binding, Init)
diff --git a/tree-sitter-mlf/bindings/node/index.js b/tree-sitter-mlf/bindings/node/index.js
index 0761eab..6657bcf 100644
--- a/tree-sitter-mlf/bindings/node/index.js
+++ b/tree-sitter-mlf/bindings/node/index.js
@@ -1,18 +1,6 @@
-try {
-  module.exports = require("../../build/Release/tree_sitter_mlf_binding");
-} catch (error1) {
-  if (error1.code !== 'MODULE_NOT_FOUND') {
-    throw error1;
-  }
-  try {
-    module.exports = require("../../build/Debug/tree_sitter_mlf_binding");
-  } catch (error2) {
-    if (error2.code !== 'MODULE_NOT_FOUND') {
-      throw error2;
-    }
-    throw error1
-  }
-}
+const root = require("path").join(__dirname, "..", "..");
+
+module.exports = require("node-gyp-build")(root);
 
 try {
   module.exports.nodeTypeInfo = require("../../src/node-types.json");
diff --git a/tree-sitter-mlf/bindings/rust/build.rs b/tree-sitter-mlf/bindings/rust/build.rs
index fe8ba98..68f2b22 100644
--- a/tree-sitter-mlf/bindings/rust/build.rs
+++ b/tree-sitter-mlf/bindings/rust/build.rs
@@ -7,6 +7,9 @@ fn main() {
         .flag_if_supported("-Wno-unused-parameter")
         .flag_if_supported("-Wno-unused-but-set-variable")
         .flag_if_supported("-Wno-trigraphs");
+    #[cfg(target_env = "msvc")]
+    c_config.flag("-utf-8");
+
     let parser_path = src_dir.join("parser.c");
     c_config.file(&parser_path);
 
diff --git a/tree-sitter-mlf/package.json b/tree-sitter-mlf/package.json
index d99e1cc..b1f32e9 100644
--- a/tree-sitter-mlf/package.json
+++ b/tree-sitter-mlf/package.json
@@ -11,6 +11,14 @@
     "lexicon",
     "atproto"
   ],
+  "files": [
+    "grammar.js",
+    "binding.gyp",
+    "prebuilds/**",
+    "bindings/node/*",
+    "queries/*",
+    "src/**"
+  ],
   "author": "Matt",
   "license": "MIT",
   "dependencies": {
@@ -36,5 +44,9 @@
         "mlf"
       ]
     }
-  ]
+  ],
+  "scripts": {
+    "install": "node-gyp-build",
+    "prebuildify": "prebuildify --napi --strip"
+  }
 }
diff --git a/tree-sitter-mlf/src/grammar.json b/tree-sitter-mlf/src/grammar.json
index 44a8ffd..eaf54d5 100644
--- a/tree-sitter-mlf/src/grammar.json
+++ b/tree-sitter-mlf/src/grammar.json
@@ -1,5 +1,4 @@
 {
-  "$schema": "https://tree-sitter.github.io/tree-sitter/assets/schemas/grammar.schema.json",
   "name": "mlf",
   "rules": {
     "source_file": {
@@ -1626,6 +1625,5 @@
   "precedences": [],
   "externals": [],
   "inline": [],
-  "supertypes": [],
-  "reserved": {}
-}
\ No newline at end of file
+  "supertypes": []
+}
diff --git a/tree-sitter-mlf/src/node-types.json b/tree-sitter-mlf/src/node-types.json
index fbb8301..f8be5c0 100644
--- a/tree-sitter-mlf/src/node-types.json
+++ b/tree-sitter-mlf/src/node-types.json
@@ -853,7 +853,6 @@
   {
     "type": "source_file",
     "named": true,
-    "root": true,
     "fields": {},
     "children": {
       "multiple": true,
@@ -1093,8 +1092,7 @@
   },
   {
     "type": "comment",
-    "named": true,
-    "extra": true
+    "named": true
   },
   {
     "type": "constrained",
@@ -1106,8 +1104,7 @@
   },
   {
     "type": "doc_comment",
-    "named": true,
-    "extra": true
+    "named": true
   },
   {
     "type": "error",
@@ -1155,11 +1152,11 @@
   },
   {
     "type": "string",
-    "named": false
+    "named": true
   },
   {
     "type": "string",
-    "named": true
+    "named": false
   },
   {
     "type": "subscription",
stavola.xyz

Yeah this is probably my fault for forgetting to commit the output of a generate command during one of the previous updates. If we had CI/CD set up here I would probably have a test for this before allowing a merge, but I guess I can also look to see if jj supports pre-commit hooks at the very least.

Anyway, I am away from my computer but I can fix it a bit later. If you happen to submit an PR before then I'll merge it for you.

vmx.cx (author)

The .gitignore file might contain things that should be committed. I let you fix it properly. I got myself unblocked locally.

May I also request to upgrade to tree-sitter 0.26? The reason I was looking into it was, that I wanted some static syntax-highlighted HTML. tree-sitter v0.22 doesn't support trees-itter highlight --css-classes which is what I wanted to use.

stavola.xyz

Sure, I'll look into this. Good suggestion!

stavola.xyz

Fixed in #25.

sign up or login to add to the discussion
Labels

None yet.

assignee

None yet.

Participants 2
AT URI
at://did:plc:3mdq56yhyqq5k6d4guztheaf/sh.tangled.repo.issue/3mjna5wp5vf22