forked from
anil.recoil.org/monopam-myspace
My aggregated monorepo of OCaml code, automaintained
Add design: per-cell filename attribute and message protocol migration
Port jtw_client from old RPC/IDL layer to the JSON message protocol,
then add optional filename field to Merlin-facing messages so cells
can specify .mli for correct interface syntax handling.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
+128
+128
docs/plans/2026-02-20-filename-and-message-protocol-design.md
+128
docs/plans/2026-02-20-filename-and-message-protocol-design.md
···
1
+
# Per-cell filename attribute and message protocol migration
2
+
3
+
## Problem
4
+
5
+
Two related issues:
6
+
7
+
1. **No filename support for Merlin.** All code cells are treated as `.ml`
8
+
(implementation) by Merlin. Tutorials that show `.mli` interface syntax
9
+
get incorrect error diagnostics because Merlin parses them as
10
+
implementation code. Merlin determines `.ml` vs `.mli` from
11
+
`Mconfig.query.filename` — currently unset, defaulting to `Impl`.
12
+
13
+
2. **x-ocaml's jtw_client still uses the old RPC layer.** The js_top_worker
14
+
`worker.ml` already speaks the new JSON message protocol (`message.ml`),
15
+
and a matching client library (`js_top_worker_client_msg.ml`) exists.
16
+
But x-ocaml's `jtw_client.ml` still uses `Js_top_worker_client_fut`
17
+
(the old `deriving rpcty` / IDL-based client), creating an unnecessary
18
+
dependency on `js_top_worker-rpc`, `rpclib`, and `ppx_deriving_rpc`.
19
+
20
+
## Design
21
+
22
+
### Part 1: Port jtw_client.ml to the message protocol
23
+
24
+
**x-ocaml/src/dune** — Replace `js_top_worker-client_fut` with
25
+
`js_top_worker-client.msg`.
26
+
27
+
**x-ocaml/src/jtw_client.ml** — Rewrite to use
28
+
`Js_top_worker_client_msg` instead of `Js_top_worker_client_fut`. The
29
+
new client speaks JSON messages directly — no type conversions from
30
+
`Toplevel_api_gen` needed.
31
+
32
+
The current `jtw_client.ml` converts between three type systems:
33
+
- `X_protocol` (x-ocaml's internal protocol)
34
+
- `Toplevel_api_gen` (the old RPC types)
35
+
- `Protocol` (merlin-js protocol types for completions, errors, etc.)
36
+
37
+
After the port, `jtw_client.ml` converts only between:
38
+
- `X_protocol` (x-ocaml's internal protocol)
39
+
- `Js_top_worker_message.Message` (simple record types with string fields)
40
+
41
+
The `Js_top_worker_client_msg` API:
42
+
- `create url` — create worker, returns `t`
43
+
- `init t config` — initialise, returns `unit Lwt.t`
44
+
- `eval t ?env_id code` — evaluate code, returns `output Lwt.t`
45
+
- `eval_stream t ?env_id code` — streaming eval, returns `eval_event Lwt_stream.t`
46
+
- `complete t ?env_id source position` — completions, returns `Msg.completions Lwt.t`
47
+
- `type_at t ?env_id source position` — types, returns `Msg.type_info list Lwt.t`
48
+
- `errors t ?env_id source` — errors, returns `Msg.error list Lwt.t`
49
+
- `create_env t env_id` / `destroy_env t env_id` — environment management
50
+
- `terminate t` — shut down worker
51
+
52
+
### Part 2: Add filename to the message protocol
53
+
54
+
**js_top_worker/idl/message.ml** — Add `filename : string option` to
55
+
the three Merlin-facing client messages:
56
+
57
+
```ocaml
58
+
| Complete of { cell_id : int; env_id : string; source : string;
59
+
position : int; filename : string option }
60
+
| TypeAt of { cell_id : int; env_id : string; source : string;
61
+
position : int; filename : string option }
62
+
| Errors of { cell_id : int; env_id : string; source : string;
63
+
filename : string option }
64
+
```
65
+
66
+
Add JSON serialization (`"filename"` field, omitted when None) and
67
+
parsing (with `get_string_opt` for backwards compatibility).
68
+
69
+
**js_top_worker/idl/js_top_worker_client_msg.ml** — Add optional
70
+
`?filename` parameter to `complete`, `type_at`, `errors`.
71
+
72
+
**js_top_worker/client/ocaml-worker.js** — Add optional `filename`
73
+
parameter to the JS client methods.
74
+
75
+
**js_top_worker/lib/impl.ml** — Accept optional filename in `config`:
76
+
77
+
```ocaml
78
+
let config ?filename () =
79
+
let initial = Merlin_kernel.Mconfig.initial in
80
+
let query = match filename with
81
+
| Some f -> { initial.query with filename = f }
82
+
| None -> initial.query
83
+
in
84
+
{ initial with
85
+
merlin = { initial.merlin with stdlib = Some path };
86
+
query }
87
+
```
88
+
89
+
Thread through `wdispatch`, `query_errors`, `complete_prefix`, and
90
+
`type_enclosing`.
91
+
92
+
**js_top_worker/lib/worker.ml** — Extract filename from messages and
93
+
pass to `M.query_errors`, `M.complete_prefix`, `M.type_enclosing`.
94
+
This requires the `Impl.Make(S)` functions to accept an optional
95
+
`?filename` parameter.
96
+
97
+
### Part 3: x-ocaml integration
98
+
99
+
**x-ocaml WebComponent** — Read `data-filename` from the element, pass
100
+
it through when sending Merlin requests via the backend.
101
+
102
+
**x-ocaml/src/jtw_client.ml** — Thread filename from x-ocaml to the
103
+
`Js_top_worker_client_msg` API.
104
+
105
+
**Authoring format** — A new per-cell attribute:
106
+
107
+
```
108
+
{@ocaml interactive filename=map.mli [
109
+
val map : ('a -> 'b) -> 'a list -> 'b list
110
+
]}
111
+
```
112
+
113
+
The odoc plugin emits `<x-ocaml mode="interactive" data-filename="map.mli">`.
114
+
115
+
## What doesn't change
116
+
117
+
- `X_protocol` — the builtin merlin-js backend still uses this.
118
+
- `Protocol` (merlin-js.protocol) — still used by the builtin backend
119
+
and merlin-codemirror.
120
+
- `add_cmi` in impl.ml — cell dependency CMI generation is always `.ml`.
121
+
- `exec` / `Eval` messages — execution always goes through `Toploop`
122
+
which is `.ml` toplevel syntax.
123
+
124
+
## Backwards compatibility
125
+
126
+
The `filename` field is optional in the JSON protocol. Workers that don't
127
+
understand it will ignore it. Clients that don't send it get the existing
128
+
behaviour (treated as `.ml`).