forked from
anil.recoil.org/monopam-myspace
My aggregated monorepo of OCaml code, automaintained
Add week 12 weeknotes with Zarr reprojection test notebook
Weeknotes 2026-12: standalone repro for the TESSERA PCA overlay
misalignment issue. Fetches embeddings from the Zarr store at
specific coordinates, runs PCA via TensorFlow.js, and overlays
on a Leaflet map. Shows input bbox and overlay bounds for
debugging the reprojection.
interactive_map_zarr: add debug widgets showing map coordinates
and drawn bbox.
gen_blog_index: fix @children_order to come before {0 Blog}.
Regenerated blog indexes.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
+274
-13
+1
-1
scripts/gen_blog_index.ml
+1
-1
scripts/gen_blog_index.ml
···
135
135
Printf.sprintf "{- %s %s}" (post_link p) p.published)
136
136
|> String.concat "\n"
137
137
in
138
-
Printf.sprintf "{0 Blog}\n\n@children_order %s\n\n@recent-posts\n{ul\n%s\n}\n"
138
+
Printf.sprintf "@children_order %s\n\n{0 Blog}\n\n@recent-posts\n{ul\n%s\n}\n"
139
139
children_order post_items
140
140
141
141
let ensure_dir path =
+1
-1
site/blog/2026/03/index.mld
+1
-1
site/blog/2026/03/index.mld
+153
-11
site/blog/2026/03/review.mld
+153
-11
site/blog/2026/03/review.mld
···
1
-
{0 Review of the last few weeks}
1
+
{0 Review of the last few months}
2
2
3
3
It's time to take a step back and write a retrospective on the last few months of
4
4
vibecoding with Claude.
5
+
6
+
{image!./vibecoding.png}
5
7
6
8
{1 What's been done}
7
9
···
15
17
in that it can completely replace what's in dune now, but doesn't extend the rules for the new features of
16
18
odoc. Since then we've merged the first part of it -- though that's the bit that
17
19
wasn't written by Claude but by my colleague {{:https://choum.net/panglesd/}Paul-Elliot}. Fortunately after a brief
18
-
sabbatical working on {{:https://docs.slipshow.org/en/stable/}slipshow}, he's now back working with us and we'll be meeting soon to discuss the
19
-
next steps to get the rest of it merged.
20
+
sabbatical working on {{:https://docs.slipshow.org/en/stable/}slipshow}, he's now back working with us and we'll be meeting soon to discuss the next steps to get the rest of it merged.
20
21
21
-
In the mean time though, I've extended the rules by quite a lot. We've now got smarter rules that don't
22
-
pull in as many dependencies as the current PR, we've got support for assets, we've got support for source
23
-
rendering, there's markdown output and sherlodoc native support so you can run sherlodoc queries on the
24
-
command line. You can configure the prefix for your doc output, and pass arbitrary options to the
22
+
In the mean time though, I've extended the rules by quite a lot. We've now got {{:https://github.com/ocaml/dune/commit/a49e98a88d7358074afffedfb0f6ee922208cdb7}smarter rules} that don't
23
+
pull in as many dependencies as the current PR, we've got {{:https://github.com/ocaml/dune/commit/0aa0170938b92342a609476abab67b960c356bd5}support for assets}, we've got support for {{:https://github.com/ocaml/dune/commit/91b08307c118193fc46a707891c28721c1778916}source
24
+
rendering}, there's {{:https://github.com/ocaml/dune/commit/4cb2b33e98634eba8b9a267a6dd9bcfc959575a4}markdown output} and {{:https://github.com/ocaml/dune/commit/76f61319a21ba8b03feeb0c15e2a567ca2114489}sherlodoc native support} so you can run sherlodoc queries on the
25
+
command line. You can configure the {{:https://github.com/ocaml/dune/commit/91b08307c118193fc46a707891c28721c1778916}prefix} for your doc output, and pass {{:https://github.com/ocaml/dune/commit/48a5cbb79b24606304dc25f965c22aa5e4cb898e}arbitrary options} to the
25
26
various invocations of odoc. All of this is being used as part of the process of generating this site,
26
27
but that's about all the testing its had. This will all have to be carefully reviewed then either
27
28
tacked onto the end of the current PR or we'll make new PRs for these. Very likely the latter, as the
···
29
30
30
31
{2 Odoc}
31
32
32
-
Odoc itself hasn't had {i much} work done to it. What {i has} been done though is a new plugin system that
33
+
OxCaml support for odoc was contributed by Luke Maurer early on after OxCaml was released.
34
+
However, this only fixed the build of odoc, it didn't give it any mode or layouts, nor
35
+
any of the other new features of OxCaml. I asked Claude to look through the way the
36
+
toplevel prints these annotations and port them to odoc, and that's been implemented on
37
+
this site. For example, see {!Base.Uniform_array.val-length}.
38
+
39
+
One of the earlier things I did was to give Odoc a nice new plugin system that
33
40
has been hugely enabling for building the new features below. I'm using dune's {{:https://dune.readthedocs.io/en/stable/sites.html#sites}site} feature for
34
41
the plugins, which really "just worked". It was very easy to add the feature and creating the plugins
35
42
has been equally easy. Building a whole variety of plugins has also been very useful in testing the
36
-
shape of the plugin API - I've made numerous changes to it as I've built various plugins and they expose
37
-
problems. I've also done a couple of more minor changes - one to allow 'custom tags', which the plugin
38
-
system uses heavily, and some improvements to the source rendering - there are many more links now
43
+
shape of the plugin API, and I've made numerous changes to it as I've built various plugins and they expose
44
+
problems. To use the plugins, you need a way to write text that the plugins will operate on, and
45
+
there are a couple of obvious ways to do this. The first is to allow {{:https://ocaml.org/manual/5.4/ocamldoc.html#sss:ocamldoc-custom-tags}custom tags}, a feature of ocamldoc
46
+
that odoc didn't support, and the second is to annotate source-code blocks with metadata that the
47
+
plugins can recognise. I also added some improvements to the source rendering - there are many more links now
39
48
and we've got the ability to link to source from doc comments.
40
49
41
50
Let's take a brief look through the plugins I've made.
···
76
85
77
86
- {{!/odoc-docsite/page-index}odoc-docsite} which produces a more modern SPA-style site
78
87
- {{!/odoc-jons-plugins/page-index}jons-shell} which produces this website
88
+
89
+
{2 Js_top_worker}
90
+
91
+
I fixed various things in js_top_worker. We've got [#require] working so that you can
92
+
load libraries from the code blocks, it's better at figuring out which libraries to
93
+
load and which are already present in the worker.
94
+
95
+
One important change to js_top_worker is the ability to use interactive widgets. This
96
+
requires coordination between the main javascript thread, or frontend, and the web-worker
97
+
backend which is actually running the OCaml code. I've been using {{!/note/page-index}Note}
98
+
as the FRP library to make this nice to work with. A nice demo of this is the
99
+
{{!/js_top_worker-widget-leaflet/page-index}Leaflet widget}.
100
+
101
+
{2 day10}
102
+
103
+
Day10 is Mark's {{:https://github.com/mtelvers/day10}build tool} that allows fast building
104
+
of opam packages in a similar way to
105
+
{{:https://github.com/ocurrent/ocaml-docs-ci}ocaml-docs-ci}. I've had dune insert the rest
106
+
of the docs-CI logic into it so that I can now use day10 to build docs for OCaml and for
107
+
OxCaml.
108
+
109
+
{2 TESSERA notebooks}
110
+
One of the most exciting things to come out of our group recently has been {{:https://geotessera.org/}TESSERA},
111
+
a pixel-wise Earth observation foundation model. The code and demos for this are mostly
112
+
in Python, particularly using Jupyter notebooks. OCaml notebooks have obviously long been
113
+
an interest of mine, so I ported the {{:https://github.com/ucam-eo/tessera-interactive-map}simple notebook} to OCaml, and more specifically
114
+
to {{!/site/notebooks/page-"interactive_map"}run in the browser}. This was pretty straightforward (aside from needing a coordinate
115
+
transform), but very slow to download the hundreds of megs of tiles required. So I then
116
+
switched to using the zarr format to stream just the areas of interest, and this
117
+
was much faster.
118
+
119
+
{1 Retrospective}
120
+
121
+
There's quite a variety of different projects that I've made progress on, and its given
122
+
me a lot of experience working with LLMs for code generation. It's been an eye-opening
123
+
experience and it's clear that the way we code is fundamentally changed. With a good
124
+
few projects under my belt now it's time to take a deep breath and assess exactly how
125
+
the landscape has changed.
126
+
127
+
{2 Attribution}
128
+
129
+
The first thing is the commits. My early purely-agentic commits were all authored by me
130
+
and co-authored by Claude. This is a lie. I couldn't carry on like this so I've switched
131
+
now to having the commits authored by "Jon's Agent". My plan is to rewrite the author
132
+
when I've gone through it line by line, and even then, I feel that Claude ought to be marked
133
+
as Author and I should just be adding my "Reviewed-by" line onto it. In either case, for
134
+
a pull request to be made the human in the loop has the responsibility to justify the
135
+
changes, and therefore has to be totally familiar with both the changes and the code
136
+
being changed. I'm not at all fundamentally opposed to having LLMs involved in the
137
+
process of making changes to open source code, but in my experience so far, there's a
138
+
huge amount of effort that needs to go into it even after you've got working code and
139
+
tests. I've tested the waters with a {{:https://github.com/ocaml/odoc/pull/1402}simple
140
+
bugfix or two}, and even these one-lines needed careful thought and attention before
141
+
I felt I could make a PR.
142
+
143
+
{2 Bug-discovery}
144
+
145
+
One thing that I've found tremendously useful is narrowing down bugs. Armed with a repro,
146
+
setting Claude off to track down issues has been a wonderful time saver. Additionally,
147
+
asking it to explain the issue in detail with links to the source is very handy indeed.
148
+
It's not, however, able to discover {i all} bugs, even with a lot of time. When working
149
+
on the fix for a {{:https://github.com/ocaml/odoc/pull/1400}particularly nasty bug}, I
150
+
found that with the patch applied we'd get a different error somewhere deep in some of Jane Street's async
151
+
ecosystem. I had a good suspicion of what the problem was give the changes that had been
152
+
made already, as the code I had altered had an analogue elsewhere in the codebase that
153
+
hadn't been fixed, so I thought this would be quite a good test for Claude. I gave it
154
+
lots of hints, but it flailed at the problem for several hours, often giving up,
155
+
sometimes blaming the OxCaml compiler, and sometimes upstream OCaml. In the end I gave
156
+
up, implemented what I thought would be the fix, and indeed the problem went away.
157
+
158
+
{2 API boundaries}
159
+
160
+
I've found it very helpful to have API boundaries to help structure the code that
161
+
Claude has been producing. Anil has long been enthusiastically pushing the idea that
162
+
we should write the [mli] files first, which constrain what types and values are
163
+
available between the modules. We can then write tests that target these interfaces,
164
+
and then adjust them where the tests have shown them to be inelegant or downright
165
+
unusable. We can then write the implementations and watch the tests start to pass.
166
+
A particularly interesting example of this is the odoc plugin interface. The
167
+
experience of writing several very different plugins that all extended odoc in
168
+
different directions
169
+
was very helpful, and I adjusted the interface quite a few times. I also
170
+
adjusted the {i documentation}, where qualities about how the interface {i behaved}
171
+
that weren't obvious from the types could be carefully noted, for example how
172
+
scripts might be made to behave correctly when the odoc pages were in an SPA shell.
173
+
174
+
{2 Failure modes}
175
+
176
+
When I was working on the {{!//blog/2025/12/page-"claude-and-dune"}dune rules}, I made
177
+
the mistake of going too long without giving Claude some architectural constraints, and
178
+
I ended up with a Big Ball of Code that I then had to spend time unpicking and teasing
179
+
apart into sensible looking modules. I had rather hopefully believed this to be an
180
+
Opus < 4.5 problem, but when adding the new features to day10 I mentioned above I hit
181
+
a very similar situation, when it just added vast amounts of code to the CLI to
182
+
implement the new features, and it was all very unstructured and unsatisfactory. This
183
+
was despite going through a design process where we went through the goals, the use
184
+
cases and desired features, but crucially, not at the level of the code.
185
+
186
+
Another failure mode I observed was {i my} failure. It's very easy, and very tempting,
187
+
to get your agent to do the next neat thing on the roadmap. Especially when you've
188
+
just spent a while going through the design and planning for the previous feature and
189
+
Claude has got started on it. The problem is that this can generate a large amount of
190
+
code that kind-of-works but has a bunch of bugs, which can end up costing a lot more
191
+
time and effort. The cost of starting the agent going
192
+
is much smaller than the cost of wading through the results, and it's quite easy to
193
+
end up drowning under a load of very interesting and partly cool half results. I'm very
194
+
much reminded of Dr Ian Malcolm's words from Jurrasic Park: "your scientists were so
195
+
preoccupied with whether or not they could, they didn't stop to think if they should."
196
+
197
+
{1 What's next}
198
+
199
+
The problem with everything I've done is that, as of right now, it's not usable,
200
+
at least by anyone but me.
201
+
While it's technically possible to add my opam-repo to your switch, and install my
202
+
versions of odoc, dune, and my various plugins, nobody is actually going to do that.
203
+
Worse than that, people might get their agents to just grab the source and mutate it
204
+
further, just diluting the efforts going into it.
205
+
206
+
Fortunately I've been talking with {{:https://choum.net/panglesd/}Paul Elliot}, who
207
+
has volunteered to shepherd the dune PR through to completion. I'll be working with
208
+
him on this of course, but I'm hoping he'll be doing the lion's share of the work.
209
+
210
+
The OxCaml work will be taken on by {{:https://github.com/art-w}art-w} who's already
211
+
done an excellent job getting Luke Maurer's patches into shape and PR'd to ocaml/odoc.
212
+
213
+
I think the odoc plugin experience was very educational, and I think the next step there
214
+
is to carefully consider how this ought to be used, how it will interact with the
215
+
dune rules, how it would affect documentation on ocaml.org. The experience of the
216
+
last few weeks will be really important in framing the discussion to be had there.
217
+
218
+
219
+
220
+
79
221
80
222
81
223
site/blog/2026/03/vibecoding.png
site/blog/2026/03/vibecoding.png
This is a binary file and will not be displayed.
+106
site/blog/2026/03/weeknotes-2026-12.mld
+106
site/blog/2026/03/weeknotes-2026-12.mld
···
1
+
{0 Weeknotes 2026 week 12}
2
+
3
+
@published 2026-03-23
4
+
5
+
{1 What did I do?}
6
+
7
+
End of term this week, so my tutorial interviews kept me quite busy for some of the week.
8
+
Then Paul-Elliot has returned from his sojourn working on Slipshow, so I had a useful
9
+
few hours working with him to try and begin the process of handing over the dune odoc
10
+
PR.
11
+
12
+
I mentioned {{!//blog/2026/03/page-"weeknotes-2026-11"}last week} that the
13
+
TESSERA reprojection was causing issues with the overlay alignment. Here's
14
+
a standalone test showing the Zarr-based pipeline — fetching embeddings,
15
+
running PCA, and overlaying the result on a Leaflet map.
16
+
17
+
{@ocaml kind=setup[
18
+
#require "tessera-zarr-jsoo";;
19
+
#require "tessera-viz-jsoo";;
20
+
#require "tessera-tfjs";;
21
+
#require "js_top_worker-widget-leaflet";;
22
+
open Widget_leaflet;;
23
+
register ();;
24
+
(* Load fzstd (Zstd decompressor) and TensorFlow.js *)
25
+
let () =
26
+
let open Js_of_ocaml in
27
+
let import url = Js.Unsafe.fun_call
28
+
(Js.Unsafe.get Js.Unsafe.global (Js.string "importScripts"))
29
+
[| Js.Unsafe.inject (Js.string url) |] in
30
+
import "https://cdn.jsdelivr.net/npm/fzstd@0.1.1/umd/index.js";
31
+
import "https://cdn.jsdelivr.net/npm/@tensorflow/tfjs@4/dist/tf.min.js"
32
+
]}
33
+
34
+
{2 Reprojection test}
35
+
36
+
Create the map, fetch embeddings from the Zarr store, run PCA via
37
+
TensorFlow.js, and overlay — all in one cell so the async pipeline
38
+
completes before the overlay is drawn:
39
+
40
+
{@ocaml x autorun[
41
+
let status_view text =
42
+
let open Widget.View in
43
+
Element { tag = "div"; attrs = [Style ("padding", "8px"); Style ("font-family", "monospace")];
44
+
children = [Text text] }
45
+
46
+
let () = Widget.display ~id:"status" ~handlers:[] (status_view "Initialising...")
47
+
48
+
let map = Leaflet_map.create
49
+
~center:(52.30690, -0.03296) ~zoom:14 ~height:"500px"
50
+
()
51
+
52
+
let bbox = Geotessera.{
53
+
min_lat = 52.29924; min_lon = -0.05845;
54
+
max_lat = 52.31745; max_lon = -0.00755;
55
+
}
56
+
57
+
let downsample mat ~h ~w ~max_pixels =
58
+
let n = h * w in
59
+
if n <= max_pixels then (mat, h, w)
60
+
else
61
+
let stride = int_of_float (ceil (sqrt (float_of_int n /. float_of_int max_pixels))) in
62
+
let h' = (h + stride - 1) / stride in
63
+
let w' = (w + stride - 1) / stride in
64
+
let out = Linalg.create_mat ~rows:(h' * w') ~cols:mat.Linalg.cols in
65
+
for i = 0 to h' - 1 do
66
+
for j = 0 to w' - 1 do
67
+
let si = min (i * stride) (h - 1) in
68
+
let sj = min (j * stride) (w - 1) in
69
+
for f = 0 to mat.Linalg.cols - 1 do
70
+
Linalg.mat_set out (i * w' + j) f
71
+
(Linalg.mat_get mat (si * w + sj) f)
72
+
done
73
+
done
74
+
done;
75
+
(out, h', w')
76
+
77
+
let () =
78
+
Widget.update ~id:"status" (status_view "Opening Zarr store...");
79
+
Lwt.async (fun () ->
80
+
let open Lwt.Syntax in
81
+
let* store = Tessera_zarr_jsoo.open_store () in
82
+
let progress msg = Widget.update ~id:"status" (status_view msg) in
83
+
let* (mat_full, h_full, w_full, geo_bounds) =
84
+
Tessera_zarr.fetch_region ~progress ~store bbox in
85
+
Widget.update ~id:"status"
86
+
(status_view (Printf.sprintf "Fetched %d×%d. Downsampling..." h_full w_full));
87
+
let (mat, h, w) = downsample mat_full ~h:h_full ~w:w_full ~max_pixels:200_000 in
88
+
let bounds = Leaflet_map.{
89
+
south = geo_bounds.Geotessera.min_lat;
90
+
north = geo_bounds.Geotessera.max_lat;
91
+
west = geo_bounds.Geotessera.min_lon;
92
+
east = geo_bounds.Geotessera.max_lon;
93
+
} in
94
+
Widget.update ~id:"status"
95
+
(status_view (Printf.sprintf "Computing PCA on %d×%d mosaic..." h w));
96
+
let proj = Tfjs.pca mat ~n_components:3 in
97
+
let img = Viz.pca_to_rgba ~width:w ~height:h proj in
98
+
let url = Viz_jsoo.to_data_url img in
99
+
Leaflet_map.add_image_overlay map ~url ~bounds ~opacity:0.7 ();
100
+
Widget.update ~id:"status"
101
+
(status_view (Printf.sprintf
102
+
"Done. Input bbox: S%.5f W%.5f N%.5f E%.5f | Overlay bounds: S%.5f W%.5f N%.5f E%.5f"
103
+
bbox.min_lat bbox.min_lon bbox.max_lat bbox.max_lon
104
+
bounds.south bounds.west bounds.north bounds.east));
105
+
Lwt.return_unit)
106
+
]}
+1
site/blog/2026/index.mld
+1
site/blog/2026/index.mld
···
2
2
3
3
{0 2026}
4
4
5
+
- {{!//blog/2026/03/page-"weeknotes-2026-12"}Weeknotes 2026 week 12}
5
6
- {{!//blog/2026/03/page-"weeknotes-2026-11"}Weeknotes 2026 week 11}
6
7
- {{!//blog/2026/03/page-"weeknotes-2026-10"}Weeknotes 2026 week 10}
7
8
- {{!//blog/2026/03/page-"weeknotes-2026-09"}Weeknotes 2026 week 9}
+2
site/blog/index.mld
+2
site/blog/index.mld
···
1
1
@children_order 2026/ 2025/
2
+
2
3
{0 Blog}
3
4
4
5
@recent-posts
5
6
{ul
7
+
{- {{!//blog/2026/03/page-"weeknotes-2026-12"}Weeknotes 2026 week 12} 2026-03-23}
6
8
{- {{!//blog/2026/03/page-"weeknotes-2026-11"}Weeknotes 2026 week 11} 2026-03-18}
7
9
{- {{!//blog/2026/03/page-"weeknotes-2026-10"}Weeknotes 2026 week 10} 2026-03-09}
8
10
{- {{!//blog/2026/03/page-"weeknotes-2026-09"}Weeknotes 2026 week 9} 2026-03-02}
+10
site/notebooks/interactive_map_zarr.mld
+10
site/notebooks/interactive_map_zarr.mld
···
58
58
59
59
(* --- Map widget (typed interface) --- *)
60
60
61
+
let () = Widget.display ~id:"debug" ~handlers:[] (status_view "No bbox drawn yet.")
62
+
61
63
let map_ref : Leaflet_map.t option ref = ref None
62
64
let map_get () = match !map_ref with Some m -> m | None -> failwith "map not ready"
63
65
64
66
let map = Leaflet_map.create
65
67
~center:(52.2, 0.12) ~zoom:13 ~height:"500px"
68
+
~on_move:(fun info ->
69
+
Widget.update ~id:"status"
70
+
(status_view (Printf.sprintf "Center: %.5f, %.5f Zoom: %d Bounds: S%.5f W%.5f N%.5f E%.5f"
71
+
info.center.lat info.center.lng info.zoom
72
+
info.bounds.south info.bounds.west info.bounds.north info.bounds.east)))
66
73
~on_bbox_drawn:(fun b ->
74
+
Widget.update ~id:"debug"
75
+
(status_view (Printf.sprintf "Drawn bbox: S%.5f W%.5f N%.5f E%.5f"
76
+
b.south b.west b.north b.east));
67
77
set_bbox (Some Geotessera.{
68
78
min_lat = b.south; min_lon = b.west;
69
79
max_lat = b.north; max_lon = b.east }))