commits
Sweep crdt, delegation, dsp, ocamlfind, pid1, qemu, requests, scc,
sdls, sigstore, sle, spacedata, tls, vz, and prune: remove opam
dependencies that no dune stanza or source file references. Verified
each removal with a clean build and test run; the lint sweep is now
empty.
Sweep ocaml-scc, ocaml-scitt, ocaml-sdls, ocaml-sdnv, ocaml-sgp4,
ocaml-sle, ocaml-spake2, ocaml-sqlite, ocaml-srp to declare every
test-scope dependency that is actually used at test time.
The Usage block had unbound [tc_frame] / [frame] / [raw_bytes] free
variables. Wrap each flow in a function ([uplink], [downlink],
[feed_cltus]). The Eio transport block referenced a single [flow]
used as both source and sink, mismatched [(_, [`Transport e])] /
[(_, [`Error of string])] error variants, and missed the [`Error _]
case. Take [~source] / [~sink] separately, handle every error
variant via [Fmt.epr], and add [eio] / [eio.core] / [fmt] mdx
libraries.
Remove libraries declared in '(libraries ...)' clauses but unreferenced
by any module in the same source tree, as flagged by 'monopam lint'
after the new Dead_lib detection landed. Touches 131 dune files across
~80 packages.
A few stanzas needed a positive correction instead of a pure removal:
- ocaml-git/bin/diag: depended on eio_main + bytesrw-eio for an
Eio_posix.run call site; the umbrella was overkill, switch to the
precise eio_posix package.
- ocaml-scaleway/lib, ocaml-s3/lib: scaleway.mli / s3.mli reference
Eio_unix.Stdenv.base; eio.unix is required and was missing.
- merlint/lib: pulled bytesrw + nox-opam.bytesrw to surface
Opam_bytesrw, used by rule e915 and lint helpers.
Stanzas where Dead_lib was a false positive (transitive dep needed
for module visibility, virtual-library impls) are left untouched —
e.g. helix.jx.jsoo for ocaml-globe/demo retains its (libraries ...)
entry because it provides the impl of the helix.jx virtual lib.
Run mdx on lib/scc.mli so the {[ ... ]} odoc block now type-checks
the synchronisation + channel-coding flow.
The example referenced free `tc_frame`, `frame`, `raw_bytes`, used a
bare unqualified `I5` constructor (the variant lives in
Scc.Coding.Reed_solomon), and `let ... in` chains that do not parse
at the top level. Restructured as toplevel bindings: a real
`tc_frame`, a `223 * 5` byte interleaved-RS frame, the qualified
`Scc.Coding.Reed_solomon.I5` constructor, and `Scc.Sync.Cltu_sync`
fed the encoded CLTU back through the parser with an
`assert (Bytes.equal decoded tc_frame)` documenting the round-trip.
Asserted `Bytes.length codeword = 255 * 5` so the documented
RS(255,223) shape at interleave 5 is verified at mdx test time, not
buried in prose.
The READMEs all share the standard install/overlay snippet, but the
sh blocks lacked the "<!-- $MDX skip -->" directive. `dune test`
would shell out to `opam install` against the live switch, which
either prompts interactively or fails with a package conflict —
either way diffing as a test failure.
Bulk-add skip directives in front of every install/overlay block.
Also collapse the doubled "non-deterministic + skip" stack on three
READMEs (memtrace, ocaml-dpop, ocaml-pid1, ocaml-yaml, merlint) where
`skip` already implies the runtime is bypassed.
Renames 35 packages to make blacksun forks distinguishable from their
opam-repository upstreams. Module names (Git.x, Tls.x, ...) stay bare;
opam package names and dune (public_name) findlib references move to
nox-X. After this commit, zero local package names overlap with
opam-repository.
Renamed:
- nox-git, nox-irmin
- nox-crypto, nox-crypto-pk, nox-crypto-rng, nox-crypto-ec
- nox-tls, nox-tls-eio, nox-tar, nox-tar-eio, nox-tty, nox-tty-eio
- nox-arp, nox-ca-certs, nox-cbor, nox-cookie, nox-crc, nox-csv
- nox-gpt, nox-hkdf, nox-http, nox-jwt, nox-kdf, nox-loc
- nox-memtrace, nox-pds, nox-sexp, nox-slack, nox-toml
- nox-websocket, nox-x509, nox-xdge, nox-yaml
Also drops orphan tar-mirage and tar-unix opam templates that had no
matching package stanza.
Pure formatting changes from `dune fmt`: doc comment placement moves
from above the binding to below it for `type`s, multi-line `match`
expressions collapse onto one line where they fit, and infix operator
applications pick up spaces (`Soup.($?)` -> `Soup.( $? )`). No
semantic changes.
Follow up to the module rename: update the remaining callers that
still referenced [Err] (library [claude.ml{,i}], [client.ml], the test
driver [test.ml]), and fix one stray [^ e] string concatenation in
hermest's CLI that needed [Json.Error.to_string e] now that
[Json.of_string] yields a structured error.
Warning 69 (unused-field, mutable-never-assigned). Four independent
record fields were flagged as mutable but the code only mutates their
referents in place, never rebinds the record slot itself:
- ocaml-wal/lib/wal.ml: [t.file] (the Eio file resource; methods call
Eio.File.pwrite_all etc., the slot is set once at open time).
- ocaml-block/lib/block.ml: [Memory.state.data] (the backing bytes,
written via Bytes.blit_string; [Bytes.t] is already mutable).
- ocaml-sse/lib/sse.ml: [Parser.t.data_buf] (a Buffer.t, written via
Buffer.add_*; the slot never changes).
- ocaml-zephyr/lib/zephyr.ml: drop [mode : Read | Write] entirely —
set at open-time, read nowhere. The open_read / open_write
constructors already distinguish the two call shapes, so mode
tracking was redundant.
Previously the eight git-x subcommands sat flat at top level (split,
check, fix, verify, filter-paths, split-commit, drop-commit, reword),
with 'split' ambiguous between the subtree split and the per-directory
commit split.
Regrouped into two namespaces that mirror the object they act on:
git-x tree
split (was: git-x split)
add (new — inject a standalone history under a prefix)
drop (was: git-x filter-paths)
check (was: git-x check)
fix (was: git-x fix)
verify (was: git-x verify)
git-x commit
split (was: git-x split-commit)
drop (was: git-x drop-commit)
reword (was: git-x reword)
Each subcommand lives in cmd_<group>_<verb>.{ml,mli}; cmd_tree.ml and
cmd_commit.ml are the Cmd.group wrappers. git_x.ml registers just the
two groups.
'tree add' is a thin wrapper over Git.Subtree.add, which already
existed in the library but had no CLI exposure. It accepts a ref (e.g.
FETCH_HEAD after 'git fetch URL REF') and a --prefix, then builds a
subtree-merge commit with the current user's git config identity.
Log source names are updated to match (git-x.tree.split,
git-x.tree.fix). The cram test under test/cram/tree_split.t is
updated to use the new 'git-x tree split' invocation throughout.
Covers every package whose .opam was missing tags, misspelled a
singleton tag, or only carried a foreign org marker. The canonical
vocabulary now lives in .merlint as E915's topics: list; this pass
makes the monorepo pass E915 on the packages whose dune-project is
the source of truth.
Cleanups: drop singletons cop1/sle/sdls/uslp/s3/sdk; rename log to
logging (json-logs, vlog); add org:blacksun to formerly-Mirage-only
packages (ca-certs, tar, tls, x509, crypto-ec).
Additions to packages missing tags: cfdp-eio, cop1-eio, ltp-eio,
scc-eio, sdls-eio, sle-eio, yamlrw-eio, yamlrw-unix, ocaml-merlin,
scitt-atp, atp-lexicon-scitt, and 10 ocaml-atp subpackages.
Drops the redundant "t" suffix. Library name, opam package, module
(Csvt -> Csv), and directory all rename in lockstep. Every downstream
reference in interop tests, libraries, and docs updated.
Test_helpers wasn't a real test module so it's renamed to helpers.ml/.mli
to satisfy the test-file naming convention. Replace
Alcotest.fail (Fmt.str ...) with Alcotest.failf in test_index and
test_manifest.
Same refactor pattern as xmlt (d786b041, 067b745c):
- Dropped the custom error ADT (Missing_header, Missing_column,
Bad_value, Truncated_row, Encode_error, Invalid_utf8), error_to_string,
pp_error.
- module Error = Textloc.Error (re-exports shared infrastructure).
- module Sort = Row | Field | Header with to_string, pp, kinded,
or_kind.
- Internal helpers raise Textloc.Error.msgf/msg; try/with at boundaries.
- decode/decode_string/decode_channel/decode_file/fold_channel/fold_file
now return (_, string) result.
- Added primed variants returning (_, Error.t) result.
- Path threading: push_array for row index, push_object for column
name -- errors carry structural context.
- Added ?max_rows / ?max_cols limits on decode.
- Kept Invalid_utf8_encode of int exception.
Tests tightened to STRUCTURAL assertions (exact frames list, exact
message) instead of substring checks.
Same tightening retrofitted to xmlt's context tests -- 12 tests
now assert exact Textloc.Error.t shape (full frame list including
outer-first ordering, exact message) rather than 'contains'.
Downstream fixes: ocaml-cdm collapses its typed error ADT to
[Parse_error of string]; 38 interop test files updated from
[Alcotest.failf "%a" Csvt.pp_error e] to ["%s" e].
csvt: 60 tests pass. xmlt: 177 tests pass.
Generate .opam.template files with x-quality-* fields based on
detected package features:
- x-quality-build: has lib/ with .ml files
- x-quality-test: has test/ with .ml files
- x-quality-fuzz: has fuzz/ with .ml files
- x-quality-interop: has test/interop/ directory
- x-quality-cram: has test/*.t/ directories
These fields are picked up by dune's opam generation and will be
checked by merlint E910 for consistency.
Also: add fmt dep to ocaml-sse/lib/dune (Fmt.pf used without dep).
monopam quality — scans packages for quality features, caches by
git commit hash. 166 packages: build=163, test=162, fuzz=94,
interop=39, doc=42.
Standard vocabulary based on crates.io categories, erratique/opam
conventions, and monorepo domain coverage:
Org: org:blacksun
Domain: aerospace, codec, crypto, network, storage, git, merkle
Purpose: cli, test, bench, format, log, system
Protocol: ccsds, uslp, cop1, sdls, sle, atproto, tls, http, json, binary
Cross-cutting: eio, simulation, math, compression
Tags placed in dune-project (package ...) stanzas via (tags ...).
Propagated to .opam files by dune's opam generation.
Git.Repository.open_repo acquires file descriptors. When 168 packages
share one Eio switch, all fds stay open until the switch exits.
Fix: wrap each per-package push in its own Eio.Switch.run.
Also adds push-stress.t: push 50 packages to catch fd leaks.
112 files across the monorepo. Printf.sprintf → Fmt.str,
Printf.printf → Fmt.pr for consistent formatting library usage.
LDPC and Turbo interop tests reimplemented the CCSDS encoder in
Python ("we implement the full CCSDS H-matrix construction",
"we implement the full CCSDS turbo encoder here"). Not interop.
SCC dariol83 interop extended with Reed-Solomon traces using
dariol83's RS codec.
These used pure Python generators (struct.pack, manual bitfield
encoding) that reimplemented the spec — not independent oracles.
Per interop-testing skill: "The generator MUST call the upstream
tool's public API. Never reimplement, transcribe, or inline."
A Python script that packs the same bits our OCaml code does
validates nothing about cross-implementation compatibility.
Proper replacements:
- COP-1: dariol83/ccsds has COP-1 support (Java)
- SCC: dariol83/ccsds has CLTU/randomizer (Java)
- TCF: spacepackets has CDS/CUC time code support (Python)
Pass Eio switch from top-level Eio_main.run down to all
Git.Repository.open_repo calls instead of creating tiny
per-call Switch.run scopes that close prematurely.
- Update .ocamlformat to 0.29.0 across all 591 files
- csvt: reuse single Buffer.t for field reads (no alloc per field)
- sexpt: Obj members decoded from stream into Dict, typed Variant GADT
- Reformat all source files for 0.29.0
- ocaml-rice: CCSDS 121.0-B lossless compression (Rice/Golomb)
- ocaml-udpcl: RFC 7122 UDP convergence layer for Bundle Protocol
- ocaml-erasure: CCSDS 131.5-B erasure correcting codes (GF(2^8))
- ocaml-short-ldpc: CCSDS 131.4-B short block-length LDPC
- ocaml-opm: CCSDS 502.0-B Orbit Parameter Message (KVN)
- ocaml-aem: CCSDS 504.0-B Attitude Ephemeris Message (KVN)
- ocaml-tdm: CCSDS 503.0-B Tracking Data Message (KVN)
- ocaml-rdm: CCSDS 508.1-B Re-entry Data Message (KVN)
Create a single package matching the CCSDS standard name
"Synchronization and Channel Coding" (131.0-B, 231.0-B):
Scc.Sync — CLTU/BCH, ASM markers, stream sync parsers
Scc.Coding — randomizer, RS interleaving, FEC presets
API: Scc.Sync.Cltu.encode, Scc.Coding.Randomizer.create, etc.
Remove ocaml-cltu, ocaml-ccsds-coding, ocaml-transport (superseded).
The Usage block had unbound [tc_frame] / [frame] / [raw_bytes] free
variables. Wrap each flow in a function ([uplink], [downlink],
[feed_cltus]). The Eio transport block referenced a single [flow]
used as both source and sink, mismatched [(_, [`Transport e])] /
[(_, [`Error of string])] error variants, and missed the [`Error _]
case. Take [~source] / [~sink] separately, handle every error
variant via [Fmt.epr], and add [eio] / [eio.core] / [fmt] mdx
libraries.
Remove libraries declared in '(libraries ...)' clauses but unreferenced
by any module in the same source tree, as flagged by 'monopam lint'
after the new Dead_lib detection landed. Touches 131 dune files across
~80 packages.
A few stanzas needed a positive correction instead of a pure removal:
- ocaml-git/bin/diag: depended on eio_main + bytesrw-eio for an
Eio_posix.run call site; the umbrella was overkill, switch to the
precise eio_posix package.
- ocaml-scaleway/lib, ocaml-s3/lib: scaleway.mli / s3.mli reference
Eio_unix.Stdenv.base; eio.unix is required and was missing.
- merlint/lib: pulled bytesrw + nox-opam.bytesrw to surface
Opam_bytesrw, used by rule e915 and lint helpers.
Stanzas where Dead_lib was a false positive (transitive dep needed
for module visibility, virtual-library impls) are left untouched —
e.g. helix.jx.jsoo for ocaml-globe/demo retains its (libraries ...)
entry because it provides the impl of the helix.jx virtual lib.
Run mdx on lib/scc.mli so the {[ ... ]} odoc block now type-checks
the synchronisation + channel-coding flow.
The example referenced free `tc_frame`, `frame`, `raw_bytes`, used a
bare unqualified `I5` constructor (the variant lives in
Scc.Coding.Reed_solomon), and `let ... in` chains that do not parse
at the top level. Restructured as toplevel bindings: a real
`tc_frame`, a `223 * 5` byte interleaved-RS frame, the qualified
`Scc.Coding.Reed_solomon.I5` constructor, and `Scc.Sync.Cltu_sync`
fed the encoded CLTU back through the parser with an
`assert (Bytes.equal decoded tc_frame)` documenting the round-trip.
Asserted `Bytes.length codeword = 255 * 5` so the documented
RS(255,223) shape at interleave 5 is verified at mdx test time, not
buried in prose.
The READMEs all share the standard install/overlay snippet, but the
sh blocks lacked the "<!-- $MDX skip -->" directive. `dune test`
would shell out to `opam install` against the live switch, which
either prompts interactively or fails with a package conflict —
either way diffing as a test failure.
Bulk-add skip directives in front of every install/overlay block.
Also collapse the doubled "non-deterministic + skip" stack on three
READMEs (memtrace, ocaml-dpop, ocaml-pid1, ocaml-yaml, merlint) where
`skip` already implies the runtime is bypassed.
Renames 35 packages to make blacksun forks distinguishable from their
opam-repository upstreams. Module names (Git.x, Tls.x, ...) stay bare;
opam package names and dune (public_name) findlib references move to
nox-X. After this commit, zero local package names overlap with
opam-repository.
Renamed:
- nox-git, nox-irmin
- nox-crypto, nox-crypto-pk, nox-crypto-rng, nox-crypto-ec
- nox-tls, nox-tls-eio, nox-tar, nox-tar-eio, nox-tty, nox-tty-eio
- nox-arp, nox-ca-certs, nox-cbor, nox-cookie, nox-crc, nox-csv
- nox-gpt, nox-hkdf, nox-http, nox-jwt, nox-kdf, nox-loc
- nox-memtrace, nox-pds, nox-sexp, nox-slack, nox-toml
- nox-websocket, nox-x509, nox-xdge, nox-yaml
Also drops orphan tar-mirage and tar-unix opam templates that had no
matching package stanza.
Follow up to the module rename: update the remaining callers that
still referenced [Err] (library [claude.ml{,i}], [client.ml], the test
driver [test.ml]), and fix one stray [^ e] string concatenation in
hermest's CLI that needed [Json.Error.to_string e] now that
[Json.of_string] yields a structured error.
Warning 69 (unused-field, mutable-never-assigned). Four independent
record fields were flagged as mutable but the code only mutates their
referents in place, never rebinds the record slot itself:
- ocaml-wal/lib/wal.ml: [t.file] (the Eio file resource; methods call
Eio.File.pwrite_all etc., the slot is set once at open time).
- ocaml-block/lib/block.ml: [Memory.state.data] (the backing bytes,
written via Bytes.blit_string; [Bytes.t] is already mutable).
- ocaml-sse/lib/sse.ml: [Parser.t.data_buf] (a Buffer.t, written via
Buffer.add_*; the slot never changes).
- ocaml-zephyr/lib/zephyr.ml: drop [mode : Read | Write] entirely —
set at open-time, read nowhere. The open_read / open_write
constructors already distinguish the two call shapes, so mode
tracking was redundant.
Previously the eight git-x subcommands sat flat at top level (split,
check, fix, verify, filter-paths, split-commit, drop-commit, reword),
with 'split' ambiguous between the subtree split and the per-directory
commit split.
Regrouped into two namespaces that mirror the object they act on:
git-x tree
split (was: git-x split)
add (new — inject a standalone history under a prefix)
drop (was: git-x filter-paths)
check (was: git-x check)
fix (was: git-x fix)
verify (was: git-x verify)
git-x commit
split (was: git-x split-commit)
drop (was: git-x drop-commit)
reword (was: git-x reword)
Each subcommand lives in cmd_<group>_<verb>.{ml,mli}; cmd_tree.ml and
cmd_commit.ml are the Cmd.group wrappers. git_x.ml registers just the
two groups.
'tree add' is a thin wrapper over Git.Subtree.add, which already
existed in the library but had no CLI exposure. It accepts a ref (e.g.
FETCH_HEAD after 'git fetch URL REF') and a --prefix, then builds a
subtree-merge commit with the current user's git config identity.
Log source names are updated to match (git-x.tree.split,
git-x.tree.fix). The cram test under test/cram/tree_split.t is
updated to use the new 'git-x tree split' invocation throughout.
Covers every package whose .opam was missing tags, misspelled a
singleton tag, or only carried a foreign org marker. The canonical
vocabulary now lives in .merlint as E915's topics: list; this pass
makes the monorepo pass E915 on the packages whose dune-project is
the source of truth.
Cleanups: drop singletons cop1/sle/sdls/uslp/s3/sdk; rename log to
logging (json-logs, vlog); add org:blacksun to formerly-Mirage-only
packages (ca-certs, tar, tls, x509, crypto-ec).
Additions to packages missing tags: cfdp-eio, cop1-eio, ltp-eio,
scc-eio, sdls-eio, sle-eio, yamlrw-eio, yamlrw-unix, ocaml-merlin,
scitt-atp, atp-lexicon-scitt, and 10 ocaml-atp subpackages.
Same refactor pattern as xmlt (d786b041, 067b745c):
- Dropped the custom error ADT (Missing_header, Missing_column,
Bad_value, Truncated_row, Encode_error, Invalid_utf8), error_to_string,
pp_error.
- module Error = Textloc.Error (re-exports shared infrastructure).
- module Sort = Row | Field | Header with to_string, pp, kinded,
or_kind.
- Internal helpers raise Textloc.Error.msgf/msg; try/with at boundaries.
- decode/decode_string/decode_channel/decode_file/fold_channel/fold_file
now return (_, string) result.
- Added primed variants returning (_, Error.t) result.
- Path threading: push_array for row index, push_object for column
name -- errors carry structural context.
- Added ?max_rows / ?max_cols limits on decode.
- Kept Invalid_utf8_encode of int exception.
Tests tightened to STRUCTURAL assertions (exact frames list, exact
message) instead of substring checks.
Same tightening retrofitted to xmlt's context tests -- 12 tests
now assert exact Textloc.Error.t shape (full frame list including
outer-first ordering, exact message) rather than 'contains'.
Downstream fixes: ocaml-cdm collapses its typed error ADT to
[Parse_error of string]; 38 interop test files updated from
[Alcotest.failf "%a" Csvt.pp_error e] to ["%s" e].
csvt: 60 tests pass. xmlt: 177 tests pass.
Generate .opam.template files with x-quality-* fields based on
detected package features:
- x-quality-build: has lib/ with .ml files
- x-quality-test: has test/ with .ml files
- x-quality-fuzz: has fuzz/ with .ml files
- x-quality-interop: has test/interop/ directory
- x-quality-cram: has test/*.t/ directories
These fields are picked up by dune's opam generation and will be
checked by merlint E910 for consistency.
Also: add fmt dep to ocaml-sse/lib/dune (Fmt.pf used without dep).
Standard vocabulary based on crates.io categories, erratique/opam
conventions, and monorepo domain coverage:
Org: org:blacksun
Domain: aerospace, codec, crypto, network, storage, git, merkle
Purpose: cli, test, bench, format, log, system
Protocol: ccsds, uslp, cop1, sdls, sle, atproto, tls, http, json, binary
Cross-cutting: eio, simulation, math, compression
Tags placed in dune-project (package ...) stanzas via (tags ...).
Propagated to .opam files by dune's opam generation.
These used pure Python generators (struct.pack, manual bitfield
encoding) that reimplemented the spec — not independent oracles.
Per interop-testing skill: "The generator MUST call the upstream
tool's public API. Never reimplement, transcribe, or inline."
A Python script that packs the same bits our OCaml code does
validates nothing about cross-implementation compatibility.
Proper replacements:
- COP-1: dariol83/ccsds has COP-1 support (Java)
- SCC: dariol83/ccsds has CLTU/randomizer (Java)
- TCF: spacepackets has CDS/CUC time code support (Python)
- ocaml-rice: CCSDS 121.0-B lossless compression (Rice/Golomb)
- ocaml-udpcl: RFC 7122 UDP convergence layer for Bundle Protocol
- ocaml-erasure: CCSDS 131.5-B erasure correcting codes (GF(2^8))
- ocaml-short-ldpc: CCSDS 131.4-B short block-length LDPC
- ocaml-opm: CCSDS 502.0-B Orbit Parameter Message (KVN)
- ocaml-aem: CCSDS 504.0-B Attitude Ephemeris Message (KVN)
- ocaml-tdm: CCSDS 503.0-B Tracking Data Message (KVN)
- ocaml-rdm: CCSDS 508.1-B Re-entry Data Message (KVN)
Create a single package matching the CCSDS standard name
"Synchronization and Channel Coding" (131.0-B, 231.0-B):
Scc.Sync — CLTU/BCH, ASM markers, stream sync parsers
Scc.Coding — randomizer, RS interleaving, FEC presets
API: Scc.Sync.Cltu.encode, Scc.Coding.Randomizer.create, etc.
Remove ocaml-cltu, ocaml-ccsds-coding, ocaml-transport (superseded).