commits
git-subtree-dir: zarr-v3-unix
git-subtree-mainline: 6ffc4c48fb24b9fcbcfa3c5c8de5d671dd2a0958
git-subtree-dir: zarr-v3
git-subtree-mainline: d0a6b6e4ea4e0ee781bf5b362f98f87e78a74a8c
git-subtree-dir: tessera-zarr-jsoo
git-subtree-mainline: 2e92b020ac834f83b5cf688a3527134853ce2c7f
git-subtree-dir: tessera-zarr
git-subtree-mainline: c2a183f36f2ae88071e26ee0c928e7f68af99b29
git-subtree-dir: tessera-viz-jsoo
git-subtree-mainline: f9ade889fb7b9cd362344c4ba009ea66a88c02a9
git-subtree-dir: tessera-viz
git-subtree-mainline: e1d983c816a8aabc9d3d32334320729ab9c36dfa
git-subtree-dir: tessera-tfjs
git-subtree-mainline: 004a57f9d8c4e336e275779513b942050e27d9c5
git-subtree-dir: tessera-npy
git-subtree-mainline: f29ae0701c40dac876bb49000a8810ce5384380f
git-subtree-dir: tessera-linalg
git-subtree-mainline: 723589fad4ec9abd6d3c617676aa4e04aeaec4c6
git-subtree-dir: tessera-geotessera-jsoo
git-subtree-mainline: 302737a1763b3ae1d044a5798f22eecd10b5365b
git-subtree-dir: tessera-geotessera
git-subtree-mainline: 9e3f4147448aec3cf35196707b4431dd3e157eb1
git-subtree-dir: x-ocaml
git-subtree-mainline: 35dd20e7ad43f7955764e4d8875e7cbe96e14b1a
git-subtree-dir: onnxrt
git-subtree-mainline: 54c812b7e1a2c46c839039ccf440ba12702878a7
git-subtree-dir: odoc-standalone
git-subtree-mainline: dd129f1e9abf3399a50c89b013e513d52c00c09a
git-subtree-dir: odoc-scrollycode-extension
git-subtree-mainline: f496642810a99759b8d890b5eb1c681df7988b46
git-subtree-dir: odoc-rfc-extension
git-subtree-mainline: 592def120c1b5337987d26a4b9290e793edfc032
git-subtree-dir: odoc-msc-extension
git-subtree-mainline: edfa701a110c683896ecbafc5b1cb808961b2b26
git-subtree-dir: odoc-mermaid-extension
git-subtree-mainline: 96e281e3afa3437bb9060bd79644678933b04079
git-subtree-dir: odoc-jons-plugins
git-subtree-mainline: daa259098e19230abd9f4638fe689e6a790b6fb1
git-subtree-dir: odoc-interactive-extension
git-subtree-mainline: 926fe64be5e89fc9180336c056dbc62b6dba3563
git-subtree-dir: odoc-dot-extension
git-subtree-mainline: caf130d3f251a05d336dbb24ab6d2657846f1911
git-subtree-dir: odoc-docsite
git-subtree-mainline: 2c5a5e8a49443e3e6b54da6914c38671fbe2b252
git-subtree-dir: odoc-admonition-extension
git-subtree-mainline: ad8aff724ee0e7ef0ebaee87a50ac59fa27d6f55
git-subtree-dir: mime_printer
git-subtree-mainline: c765607d45a3ab83cb493ae0bfb4e89538264565
git-subtree-dir: merlin-js
git-subtree-mainline: f7288408dbe3f820a89e380f0bd5342f87b60b31
git-subtree-dir: js_top_worker
git-subtree-mainline: a1f7d54a8e2f9544cc792949ab0e2cd148e380d3
git-subtree-dir: day11
git-subtree-mainline: b6573b3047013233d7e5a1ced9d5b9268238a53b
git-subtree-dir: day10
git-subtree-mainline: 70b7b73a70cf3ce8fe4f8f550d21cf0669525115
git-subtree-dir: jsoo-code-mirror
git-subtree-mainline: d250a25f04b2072adaed840497e8dccea368a0ff
Copies build.yml from origin's tangled mirror into the odoc subtree
so it survives monopam push. Runs dune runtest for odoc and
odoc-parser via nixery.
Combines the effect of origin commits 9ec78243f, b6f4cb732, c4eb108bb
(initial workflow + missing tool deps).
Odoc performance investigation: 14 commits reducing total allocation
by 40% and wall time by 14% on odoc_driver core (OxCaml switch).
Key optimisations:
- Flatten hidden PPX-monomorphization includes at load time
- Memoize doc comment parsing + semantic analysis
- O(n) shadow detection (was O(n^2))
- Hash-first identifier compare (eliminates most Map lookup compare_val)
- Cache mode printing (avoid Format.asprintf per arg)
- segment_to_string: direct string concat instead of Format.asprintf
- Buffered HTML output (avoid channel mutex per chunk)
- Stream Renderer.page children via Seq.t (bounded peak memory)
Details in commits a1802364..a92f00fb.
Generated output from gen_blog_index.exe — should not be checked in.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add @page-tags metadata to 39 blog posts across 10 tags (ai, docs-ci,
meta, notebooks, ocaml, odoc, plugins, teaching, tessera, weeknotes)
- Generate tag pages with post lists via gen_blog_index.exe
- @page-tags plugin now uses resolved odoc references for chip links
instead of hardcoded URLs
- gen_atom.ml: deterministic timestamps, CLI args, UTF-8-safe truncation
- gen_rules.ml: add markdown generation phase (@markdown alias), atom.xml
and tag page diff rules under @runtest
- Include weeknotes-2026-15 in blog indexes
- Skip dotfiles in blog scanner (fixes Emacs lockfile crash)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add @page-tags metadata to 39 blog posts across 10 tags (ai, docs-ci,
meta, notebooks, ocaml, odoc, plugins, teaching, tessera, weeknotes)
- Generate tag pages with post lists via gen_blog_index.exe
- @page-tags plugin now uses resolved odoc references for chip links
instead of hardcoded URLs
- gen_atom.ml: deterministic timestamps, CLI args, UTF-8-safe truncation
- gen_rules.ml: add markdown generation phase (@markdown alias), atom.xml
and tag page diff rules under @runtest
- Include weeknotes-2026-15 in blog indexes
- Skip dotfiles in blog scanner (fixes Emacs lockfile crash)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Previously html-generate built the entire tree of Html.elt values for
all pages in a library before writing anything. For core.odocl (11k
pages) peak RSS reached ~2 GB, problematic for CI environments with
limited memory (e.g., GitHub Actions).
Change Renderer.page.children from `page list Lazy.t` to `page Seq.t`
and switch subpage generators to Seq.map. Each child's Html.elt tree
is now built only when traverse pulls it, and the previous sibling's
tree becomes unreachable as traverse moves on. Peak memory during
rendering is now bounded by the ancestor chain (O(depth)) rather than
the whole library (O(pages)).
Impact on core.odocl html-generate:
- Peak RSS: 2.00 GB -> 1.81 GB (-190 MB, -10%)
- Incremental rendering cost (above unmarshal baseline):
730 MB -> 250 MB (-66%)
The remaining ~1.27 GB floor comes from unmarshaling the full
Lang.Compilation_unit (148 MB on disk -> ~1.3 GB in memory, typical
for Marshal-deserialised OCaml records).
HTML output bit-for-bit identical (verified).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
render previously built the entire tree of Html.elt values for all
pages before any output was written. For large libraries like core
(11,000+ pages), peak RSS reached ~2 GB during html-generate.
Make the `children` field of Renderer.page a lazy thunk, so each
level's subpages are only constructed when traverse reaches them.
In combination with traverse's updated iteration pattern (which
destructures and discards the parent page record before descending),
this reduces peak memory and allows GC to collect processed pages.
Effect is modest on its own (~5% peak RSS) because siblings at each
level are still built eagerly when the lazy is forced. Fuller
streaming would require refactoring subpage generation to produce
children one at a time. This commit establishes the API shape for
that future work.
HTML output is bit-for-bit identical — verified deterministic
between two runs of the same input.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Two small fixes to avoid generic compare_val in html-generate hot paths:
1. Url.is_prefix: try physical equality (==) before structural (=).
URLs for the same path are shared within a page, so == catches
the common case without walking the parent chain.
2. Link.drop_shared_prefix: l1 and l2 are string lists; use
String.equal instead of (=) to avoid generic comparison dispatch.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Format.formatter_of_out_channel writes to the channel on every buffer
flush. Each write takes the OCaml multicore channel mutex (trylock +
unlock), which callgrind showed at ~3% self-time on html-generate.
Accumulate output in a Buffer and write the whole file in one call
at the end. The mutex is taken once per page instead of many times.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
segment_to_string was called millions of times during HTML generation
(one of the top allocation sites: ~60% of minor heap allocation
according to statmemprof). It used Format.asprintf "%a%s" with a
pretty-printer that either does nothing (for Module/Page/LeafPage/
File/SourcePage kinds — the common case) or emits "<kind>-".
Replace with a direct match: return name unchanged for passthrough
kinds, otherwise concatenate prefix directly. Eliminates Format
buffer allocation per call.
Results:
- core.odocl html-generate: 45.7 GB -> 23.4 GB (-49% alloc)
- core.odocl html-generate: 9.53s -> 7.33s (-23%)
- stdlib.odocl html-generate: 1.74s -> 0.52s (-70%)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Identifier.compare was using Stdlib.compare on ikey strings, which
dispatches through the generic compare_val C function. Replace with
hash comparison first (Int.compare on precomputed ihash), falling
back to String.compare only when hashes collide.
This eliminates most string comparisons in Map lookups since
identifiers with different hashes are immediately distinguished.
Callgrind showed compare_val at 12% self-time on indexed_container_intf.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
is_shadowed used List.mem which triggers compare_val (generic
structural comparison) on Ident.t values. Replace with List.exists
using Ident.same (stamp-based integer comparison), reducing per-
element comparison cost from O(tree_size) to O(1).
Indexed_container_intf read_impl: 2.48s -> 2.16s (-13%).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Emit ODOC_PHASE_TIME lines on stderr when ODOC_GC_STATS=1 is set,
breaking down the compile command into unmarshal (Cmt_format.read_cmt),
read_impl (Cmt.read_implementation), and compile (Xref2.Compile)
phases. Useful for identifying which phase dominates on large files.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
value_name_exists, type_name_exists, etc. did a linear List.exists
scan of remaining items for each item processed — O(n^2) total.
Callgrind showed this as 16% of CPU on Container_intf (150K items).
Replace with a single pre-pass that counts names per kind in a
Hashtbl, then O(1) check-and-decrement during processing.
Result: read_impl drops from 3.77s to 2.87s (-24%) on Container_intf.
The function disappears entirely from the callgrind profile.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Two optimizations to Doc_attr.attached:
1. Skip entirely when skip_doc_parsing is true (inside a stop block).
Returns empty docs immediately, avoiding both parsing and semantic
analysis for items that will never render.
2. Cache the full ast_to_comment result by raw doc text. The semantic
analysis (code block trimming, tag processing) was running for all
150K doc comment instances even though the parse AST was cached.
With 33 unique texts, this eliminates 99.98% of semantic processing.
Together: read_impl drops from 3.77s to 3.70s on Container_intf.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
OxCaml mode annotations (locality, uniqueness, linearity, etc.) were
printed via Format.asprintf per argument type, allocating a fresh
Format buffer each time. With 150K items × multiple args, this was
24% of allocation in statmemprof.
Cache the result per mode constant value (small finite set — ~16
entries). Reduces minor heap allocation by 20%.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When flattening hidden includes, wrap the spliced items in
Comment `Stop markers so the document generator's skip-until-Stop
logic skips them. Without this, dunder-named items that were
previously hidden inside Include wrappers would be fully rendered
as top-level items, causing a 3x HTML size regression on base.
Also propagates skip_doc_parsing through nested read_signature calls
so that items inside stop blocks at any nesting depth have their doc
parsing skipped.
Result: HTML output for base drops from 188 MB to 19 MB (vs 145 MB
baseline). 869 empty dunder-named pages eliminated.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Three layered optimizations for odoc compile/link performance on
ppx_template-heavy packages (base, core):
1. Loader: flatten includes whose expansion items all have __ names
(ppx_template monomorphization duplicates) into the enclosing
signature, eliminating nested Include nodes that caused 10K+
redundant traversals during compile/link.
2. Loader: memoize Odoc_parser.parse_comment by raw text string.
Container_intf has 155K doc comments but only 33 unique texts
(99.98% cache hit rate), saving ~3.4s of parser time per compile.
3. Link: short-circuit comment_docs when the doc AST contains no
references, headings, or modules to resolve — avoids rebuilding
155K doc ASTs word-by-word via List.map.
Also adds instrumentation (gated by ODOC_GC_STATS=1):
- Per-subprocess Gc.quick_stat reporting via stderr
- Per-phase include_ call counting with per-location breakdown
- Doc parse timing and cache hit stats
- Per-item timing in the cmt loader
- Driver: top-10-by-allocation report per phase with include counts
- Driver: track all subprocesses (including silent/dependency ones)
Results on odoc_driver core (vs better-website baseline):
- Compile: 128 GB → 94 GB (-27%)
- Link: 73 GB → 56 GB (-23%)
- Wall time: 549s → 499s (-9%)
- HTML-gen: +28% allocation (known; items bypass internal_value
fast-skip due to ValueName.Std tag — deferred to follow-up)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Extend from_page in skeleton.ml to emit one child entry per heading
in addition to the whole-page entry. Each heading entry uses the
heading's Label.t identifier (which carries the fragment anchor) and
stores the heading text as its doc content for tokenisation.
Before: searching for a word returns only the page.
After: searching returns the page AND the specific heading, with a
qualified name that includes the anchor (e.g.
ht.heading-test.section-about-parseff).
Verified via sherlodoc index + search on a test fixture with two
headings: "section about parseff" and "section about tessera". Each
heading appears as a separate search result, and searching for
"section" returns both headings.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Page_tags now uses register_with_link so it gets access to the
cross-reference environment at link time. For each tag, it calls
Env.lookup_page_by_path with (TCurrentPackage, ["tags"; tag]) to
verify that a corresponding tag page exists under site/tags/.
If the lookup fails, the build crashes with a clear message:
@page-tags: no page found for tag 'foo'. Create
site/tags/foo.mld before using this tag.
This enforces the invariant that tag chip links always resolve —
you can't use a tag until you've created its index page.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Page_tags now uses register_with_link so it gets access to the
cross-reference environment at link time. For each tag, it calls
Env.lookup_page_by_path with (TCurrentPackage, ["tags"; tag]) to
verify that a corresponding tag page exists under site/tags/.
If the lookup fails, the build crashes with a clear message:
@page-tags: no page found for tag 'foo'. Create
site/tags/foo.mld before using this tag.
This enforces the invariant that tag chip links always resolve —
you can't use a tag until you've created its index page.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The inline-extensions CSS (margin-note, kbd) was registered as a
separate support file (extensions/inline-extensions.css) but the
shell's page <link> only referenced extensions/jon-shell.css — so
margin notes rendered with no styling.
Fix: concatenate inline_extensions_css into the jon-shell.css support
file at registration time. The shell's existing <link> now picks up
both the shell styles and the inline-extension styles in one request.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The inline-extensions CSS (margin-note, kbd) was registered as a
separate support file (extensions/inline-extensions.css) but the
shell's page <link> only referenced extensions/jon-shell.css — so
margin notes rendered with no styling.
Fix: concatenate inline_extensions_css into the jon-shell.css support
file at registration time. The shell's existing <link> now picks up
both the shell styles and the inline-extension styles in one request.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Three things bundled:
1. Margin note (`{&margin ...}`) now floats to the right of the
paragraph as a proper sidenote, with a responsive fallback that
collapses to a block on narrow screens. The inline-chip look is
gone.
2. Two new inline extensions:
- `{&image SRC "alt"}` → `<img src="SRC" alt="alt">`
- `{&linked-image URL SRC "alt"}` → `<a href="URL"><img ...></a>`
Both parse a whitespace-separated payload with quoted values.
3. `@figure` V3 form: if the body paragraph starts with a
`{&linked-image ...}` or `{&image ...}` inline, that inline
supplies the image and the remaining paragraph inlines become the
`<figcaption>` content — so bold, italic, links and references in
captions now render. V1 (attribute-only, plain-text caption) still
works as a fallback when no image inline is present.
Plus five placeholder tag pages under site/tags/ so the `@page-tags`
chips in the demo page actually link somewhere that exists, and an
updated demo page showing all three features side-by-side.
The link-phase validation for @page-tags (fail-build-if-tag-page-
missing) isn't in this commit — Env.lookup_page_by_path / by_name did
not reliably find pages under the site package. Needs more research
into how the monorepo's site package exposes pages to the Env at
link phase.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Three things bundled:
1. Margin note (`{&margin ...}`) now floats to the right of the
paragraph as a proper sidenote, with a responsive fallback that
collapses to a block on narrow screens. The inline-chip look is
gone.
2. Two new inline extensions:
- `{&image SRC "alt"}` → `<img src="SRC" alt="alt">`
- `{&linked-image URL SRC "alt"}` → `<a href="URL"><img ...></a>`
Both parse a whitespace-separated payload with quoted values.
3. `@figure` V3 form: if the body paragraph starts with a
`{&linked-image ...}` or `{&image ...}` inline, that inline
supplies the image and the remaining paragraph inlines become the
`<figcaption>` content — so bold, italic, links and references in
captions now render. V1 (attribute-only, plain-text caption) still
works as a fallback when no image inline is present.
Plus five placeholder tag pages under site/tags/ so the `@page-tags`
chips in the demo page actually link somewhere that exists, and an
updated demo page showing all three features side-by-side.
The link-phase validation for @page-tags (fail-build-if-tag-page-
missing) isn't in this commit — Env.lookup_page_by_path / by_name did
not reliably find pages under the site package. Needs more research
into how the monorepo's site package exposes pages to the Env at
link phase.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Single page exercising all four of the recent extensions: @page-tags,
@figure with link, {&kbd} and {&margin} custom inlines, and
sherlodoc-indexable page prose. Useful for eyeballing how the new
features render end to end.
Also escapes a literal @davesnx at-mention in weeknotes-2026-15.mld
that was parsing as a block tag.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Odoc performance investigation: 14 commits reducing total allocation
by 40% and wall time by 14% on odoc_driver core (OxCaml switch).
Key optimisations:
- Flatten hidden PPX-monomorphization includes at load time
- Memoize doc comment parsing + semantic analysis
- O(n) shadow detection (was O(n^2))
- Hash-first identifier compare (eliminates most Map lookup compare_val)
- Cache mode printing (avoid Format.asprintf per arg)
- segment_to_string: direct string concat instead of Format.asprintf
- Buffered HTML output (avoid channel mutex per chunk)
- Stream Renderer.page children via Seq.t (bounded peak memory)
Details in commits a1802364..a92f00fb.
- Add @page-tags metadata to 39 blog posts across 10 tags (ai, docs-ci,
meta, notebooks, ocaml, odoc, plugins, teaching, tessera, weeknotes)
- Generate tag pages with post lists via gen_blog_index.exe
- @page-tags plugin now uses resolved odoc references for chip links
instead of hardcoded URLs
- gen_atom.ml: deterministic timestamps, CLI args, UTF-8-safe truncation
- gen_rules.ml: add markdown generation phase (@markdown alias), atom.xml
and tag page diff rules under @runtest
- Include weeknotes-2026-15 in blog indexes
- Skip dotfiles in blog scanner (fixes Emacs lockfile crash)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add @page-tags metadata to 39 blog posts across 10 tags (ai, docs-ci,
meta, notebooks, ocaml, odoc, plugins, teaching, tessera, weeknotes)
- Generate tag pages with post lists via gen_blog_index.exe
- @page-tags plugin now uses resolved odoc references for chip links
instead of hardcoded URLs
- gen_atom.ml: deterministic timestamps, CLI args, UTF-8-safe truncation
- gen_rules.ml: add markdown generation phase (@markdown alias), atom.xml
and tag page diff rules under @runtest
- Include weeknotes-2026-15 in blog indexes
- Skip dotfiles in blog scanner (fixes Emacs lockfile crash)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Previously html-generate built the entire tree of Html.elt values for
all pages in a library before writing anything. For core.odocl (11k
pages) peak RSS reached ~2 GB, problematic for CI environments with
limited memory (e.g., GitHub Actions).
Change Renderer.page.children from `page list Lazy.t` to `page Seq.t`
and switch subpage generators to Seq.map. Each child's Html.elt tree
is now built only when traverse pulls it, and the previous sibling's
tree becomes unreachable as traverse moves on. Peak memory during
rendering is now bounded by the ancestor chain (O(depth)) rather than
the whole library (O(pages)).
Impact on core.odocl html-generate:
- Peak RSS: 2.00 GB -> 1.81 GB (-190 MB, -10%)
- Incremental rendering cost (above unmarshal baseline):
730 MB -> 250 MB (-66%)
The remaining ~1.27 GB floor comes from unmarshaling the full
Lang.Compilation_unit (148 MB on disk -> ~1.3 GB in memory, typical
for Marshal-deserialised OCaml records).
HTML output bit-for-bit identical (verified).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
render previously built the entire tree of Html.elt values for all
pages before any output was written. For large libraries like core
(11,000+ pages), peak RSS reached ~2 GB during html-generate.
Make the `children` field of Renderer.page a lazy thunk, so each
level's subpages are only constructed when traverse reaches them.
In combination with traverse's updated iteration pattern (which
destructures and discards the parent page record before descending),
this reduces peak memory and allows GC to collect processed pages.
Effect is modest on its own (~5% peak RSS) because siblings at each
level are still built eagerly when the lazy is forced. Fuller
streaming would require refactoring subpage generation to produce
children one at a time. This commit establishes the API shape for
that future work.
HTML output is bit-for-bit identical — verified deterministic
between two runs of the same input.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Two small fixes to avoid generic compare_val in html-generate hot paths:
1. Url.is_prefix: try physical equality (==) before structural (=).
URLs for the same path are shared within a page, so == catches
the common case without walking the parent chain.
2. Link.drop_shared_prefix: l1 and l2 are string lists; use
String.equal instead of (=) to avoid generic comparison dispatch.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Format.formatter_of_out_channel writes to the channel on every buffer
flush. Each write takes the OCaml multicore channel mutex (trylock +
unlock), which callgrind showed at ~3% self-time on html-generate.
Accumulate output in a Buffer and write the whole file in one call
at the end. The mutex is taken once per page instead of many times.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
segment_to_string was called millions of times during HTML generation
(one of the top allocation sites: ~60% of minor heap allocation
according to statmemprof). It used Format.asprintf "%a%s" with a
pretty-printer that either does nothing (for Module/Page/LeafPage/
File/SourcePage kinds — the common case) or emits "<kind>-".
Replace with a direct match: return name unchanged for passthrough
kinds, otherwise concatenate prefix directly. Eliminates Format
buffer allocation per call.
Results:
- core.odocl html-generate: 45.7 GB -> 23.4 GB (-49% alloc)
- core.odocl html-generate: 9.53s -> 7.33s (-23%)
- stdlib.odocl html-generate: 1.74s -> 0.52s (-70%)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Identifier.compare was using Stdlib.compare on ikey strings, which
dispatches through the generic compare_val C function. Replace with
hash comparison first (Int.compare on precomputed ihash), falling
back to String.compare only when hashes collide.
This eliminates most string comparisons in Map lookups since
identifiers with different hashes are immediately distinguished.
Callgrind showed compare_val at 12% self-time on indexed_container_intf.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
is_shadowed used List.mem which triggers compare_val (generic
structural comparison) on Ident.t values. Replace with List.exists
using Ident.same (stamp-based integer comparison), reducing per-
element comparison cost from O(tree_size) to O(1).
Indexed_container_intf read_impl: 2.48s -> 2.16s (-13%).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Emit ODOC_PHASE_TIME lines on stderr when ODOC_GC_STATS=1 is set,
breaking down the compile command into unmarshal (Cmt_format.read_cmt),
read_impl (Cmt.read_implementation), and compile (Xref2.Compile)
phases. Useful for identifying which phase dominates on large files.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
value_name_exists, type_name_exists, etc. did a linear List.exists
scan of remaining items for each item processed — O(n^2) total.
Callgrind showed this as 16% of CPU on Container_intf (150K items).
Replace with a single pre-pass that counts names per kind in a
Hashtbl, then O(1) check-and-decrement during processing.
Result: read_impl drops from 3.77s to 2.87s (-24%) on Container_intf.
The function disappears entirely from the callgrind profile.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Two optimizations to Doc_attr.attached:
1. Skip entirely when skip_doc_parsing is true (inside a stop block).
Returns empty docs immediately, avoiding both parsing and semantic
analysis for items that will never render.
2. Cache the full ast_to_comment result by raw doc text. The semantic
analysis (code block trimming, tag processing) was running for all
150K doc comment instances even though the parse AST was cached.
With 33 unique texts, this eliminates 99.98% of semantic processing.
Together: read_impl drops from 3.77s to 3.70s on Container_intf.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
OxCaml mode annotations (locality, uniqueness, linearity, etc.) were
printed via Format.asprintf per argument type, allocating a fresh
Format buffer each time. With 150K items × multiple args, this was
24% of allocation in statmemprof.
Cache the result per mode constant value (small finite set — ~16
entries). Reduces minor heap allocation by 20%.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When flattening hidden includes, wrap the spliced items in
Comment `Stop markers so the document generator's skip-until-Stop
logic skips them. Without this, dunder-named items that were
previously hidden inside Include wrappers would be fully rendered
as top-level items, causing a 3x HTML size regression on base.
Also propagates skip_doc_parsing through nested read_signature calls
so that items inside stop blocks at any nesting depth have their doc
parsing skipped.
Result: HTML output for base drops from 188 MB to 19 MB (vs 145 MB
baseline). 869 empty dunder-named pages eliminated.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Three layered optimizations for odoc compile/link performance on
ppx_template-heavy packages (base, core):
1. Loader: flatten includes whose expansion items all have __ names
(ppx_template monomorphization duplicates) into the enclosing
signature, eliminating nested Include nodes that caused 10K+
redundant traversals during compile/link.
2. Loader: memoize Odoc_parser.parse_comment by raw text string.
Container_intf has 155K doc comments but only 33 unique texts
(99.98% cache hit rate), saving ~3.4s of parser time per compile.
3. Link: short-circuit comment_docs when the doc AST contains no
references, headings, or modules to resolve — avoids rebuilding
155K doc ASTs word-by-word via List.map.
Also adds instrumentation (gated by ODOC_GC_STATS=1):
- Per-subprocess Gc.quick_stat reporting via stderr
- Per-phase include_ call counting with per-location breakdown
- Doc parse timing and cache hit stats
- Per-item timing in the cmt loader
- Driver: top-10-by-allocation report per phase with include counts
- Driver: track all subprocesses (including silent/dependency ones)
Results on odoc_driver core (vs better-website baseline):
- Compile: 128 GB → 94 GB (-27%)
- Link: 73 GB → 56 GB (-23%)
- Wall time: 549s → 499s (-9%)
- HTML-gen: +28% allocation (known; items bypass internal_value
fast-skip due to ValueName.Std tag — deferred to follow-up)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Extend from_page in skeleton.ml to emit one child entry per heading
in addition to the whole-page entry. Each heading entry uses the
heading's Label.t identifier (which carries the fragment anchor) and
stores the heading text as its doc content for tokenisation.
Before: searching for a word returns only the page.
After: searching returns the page AND the specific heading, with a
qualified name that includes the anchor (e.g.
ht.heading-test.section-about-parseff).
Verified via sherlodoc index + search on a test fixture with two
headings: "section about parseff" and "section about tessera". Each
heading appears as a separate search result, and searching for
"section" returns both headings.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Page_tags now uses register_with_link so it gets access to the
cross-reference environment at link time. For each tag, it calls
Env.lookup_page_by_path with (TCurrentPackage, ["tags"; tag]) to
verify that a corresponding tag page exists under site/tags/.
If the lookup fails, the build crashes with a clear message:
@page-tags: no page found for tag 'foo'. Create
site/tags/foo.mld before using this tag.
This enforces the invariant that tag chip links always resolve —
you can't use a tag until you've created its index page.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Page_tags now uses register_with_link so it gets access to the
cross-reference environment at link time. For each tag, it calls
Env.lookup_page_by_path with (TCurrentPackage, ["tags"; tag]) to
verify that a corresponding tag page exists under site/tags/.
If the lookup fails, the build crashes with a clear message:
@page-tags: no page found for tag 'foo'. Create
site/tags/foo.mld before using this tag.
This enforces the invariant that tag chip links always resolve —
you can't use a tag until you've created its index page.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The inline-extensions CSS (margin-note, kbd) was registered as a
separate support file (extensions/inline-extensions.css) but the
shell's page <link> only referenced extensions/jon-shell.css — so
margin notes rendered with no styling.
Fix: concatenate inline_extensions_css into the jon-shell.css support
file at registration time. The shell's existing <link> now picks up
both the shell styles and the inline-extension styles in one request.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The inline-extensions CSS (margin-note, kbd) was registered as a
separate support file (extensions/inline-extensions.css) but the
shell's page <link> only referenced extensions/jon-shell.css — so
margin notes rendered with no styling.
Fix: concatenate inline_extensions_css into the jon-shell.css support
file at registration time. The shell's existing <link> now picks up
both the shell styles and the inline-extension styles in one request.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Three things bundled:
1. Margin note (`{&margin ...}`) now floats to the right of the
paragraph as a proper sidenote, with a responsive fallback that
collapses to a block on narrow screens. The inline-chip look is
gone.
2. Two new inline extensions:
- `{&image SRC "alt"}` → `<img src="SRC" alt="alt">`
- `{&linked-image URL SRC "alt"}` → `<a href="URL"><img ...></a>`
Both parse a whitespace-separated payload with quoted values.
3. `@figure` V3 form: if the body paragraph starts with a
`{&linked-image ...}` or `{&image ...}` inline, that inline
supplies the image and the remaining paragraph inlines become the
`<figcaption>` content — so bold, italic, links and references in
captions now render. V1 (attribute-only, plain-text caption) still
works as a fallback when no image inline is present.
Plus five placeholder tag pages under site/tags/ so the `@page-tags`
chips in the demo page actually link somewhere that exists, and an
updated demo page showing all three features side-by-side.
The link-phase validation for @page-tags (fail-build-if-tag-page-
missing) isn't in this commit — Env.lookup_page_by_path / by_name did
not reliably find pages under the site package. Needs more research
into how the monorepo's site package exposes pages to the Env at
link phase.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Three things bundled:
1. Margin note (`{&margin ...}`) now floats to the right of the
paragraph as a proper sidenote, with a responsive fallback that
collapses to a block on narrow screens. The inline-chip look is
gone.
2. Two new inline extensions:
- `{&image SRC "alt"}` → `<img src="SRC" alt="alt">`
- `{&linked-image URL SRC "alt"}` → `<a href="URL"><img ...></a>`
Both parse a whitespace-separated payload with quoted values.
3. `@figure` V3 form: if the body paragraph starts with a
`{&linked-image ...}` or `{&image ...}` inline, that inline
supplies the image and the remaining paragraph inlines become the
`<figcaption>` content — so bold, italic, links and references in
captions now render. V1 (attribute-only, plain-text caption) still
works as a fallback when no image inline is present.
Plus five placeholder tag pages under site/tags/ so the `@page-tags`
chips in the demo page actually link somewhere that exists, and an
updated demo page showing all three features side-by-side.
The link-phase validation for @page-tags (fail-build-if-tag-page-
missing) isn't in this commit — Env.lookup_page_by_path / by_name did
not reliably find pages under the site package. Needs more research
into how the monorepo's site package exposes pages to the Env at
link phase.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Single page exercising all four of the recent extensions: @page-tags,
@figure with link, {&kbd} and {&margin} custom inlines, and
sherlodoc-indexable page prose. Useful for eyeballing how the new
features render end to end.
Also escapes a literal @davesnx at-mention in weeknotes-2026-15.mld
that was parsing as a block tag.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>