commits
Pure formatting changes from `dune fmt`: doc comment placement moves
from above the binding to below it for `type`s, multi-line `match`
expressions collapse onto one line where they fit, and infix operator
applications pick up spaces (`Soup.($?)` -> `Soup.( $? )`). No
semantic changes.
Follow up to the module rename: update the remaining callers that
still referenced [Err] (library [claude.ml{,i}], [client.ml], the test
driver [test.ml]), and fix one stray [^ e] string concatenation in
hermest's CLI that needed [Json.Error.to_string e] now that
[Json.of_string] yields a structured error.
Warning 69 (unused-field, mutable-never-assigned). Four independent
record fields were flagged as mutable but the code only mutates their
referents in place, never rebinds the record slot itself:
- ocaml-wal/lib/wal.ml: [t.file] (the Eio file resource; methods call
Eio.File.pwrite_all etc., the slot is set once at open time).
- ocaml-block/lib/block.ml: [Memory.state.data] (the backing bytes,
written via Bytes.blit_string; [Bytes.t] is already mutable).
- ocaml-sse/lib/sse.ml: [Parser.t.data_buf] (a Buffer.t, written via
Buffer.add_*; the slot never changes).
- ocaml-zephyr/lib/zephyr.ml: drop [mode : Read | Write] entirely —
set at open-time, read nowhere. The open_read / open_write
constructors already distinguish the two call shapes, so mode
tracking was redundant.
Previously the eight git-x subcommands sat flat at top level (split,
check, fix, verify, filter-paths, split-commit, drop-commit, reword),
with 'split' ambiguous between the subtree split and the per-directory
commit split.
Regrouped into two namespaces that mirror the object they act on:
git-x tree
split (was: git-x split)
add (new — inject a standalone history under a prefix)
drop (was: git-x filter-paths)
check (was: git-x check)
fix (was: git-x fix)
verify (was: git-x verify)
git-x commit
split (was: git-x split-commit)
drop (was: git-x drop-commit)
reword (was: git-x reword)
Each subcommand lives in cmd_<group>_<verb>.{ml,mli}; cmd_tree.ml and
cmd_commit.ml are the Cmd.group wrappers. git_x.ml registers just the
two groups.
'tree add' is a thin wrapper over Git.Subtree.add, which already
existed in the library but had no CLI exposure. It accepts a ref (e.g.
FETCH_HEAD after 'git fetch URL REF') and a --prefix, then builds a
subtree-merge commit with the current user's git config identity.
Log source names are updated to match (git-x.tree.split,
git-x.tree.fix). The cram test under test/cram/tree_split.t is
updated to use the new 'git-x tree split' invocation throughout.
The EverParse .3d subtraction (FrameLength - 7) requires an SMT proof
that the value won't underflow. Add ~self_constraint on FrameLength
so the .3d emits { (FrameLength >= 7) } and F* verification passes.
Both packages use Wire.Codec but had no c/ subdirectory, so merlint's
global check flagged them as missing the EverParse 3D entry point.
Add a minimal c/ for each: gen.ml that calls Wire_3d.main on the
codec(s), a dune file with the standard (alias gen) layout, and an
empty dune.inc placeholder. Running `dune build @<pkg>/c/gen` regenerates
dune.inc + the .3d, .h, .c, etc. when the EverParse 3D toolchain is on
PATH.
For proximity1 the codec is the single Proximity1.codec; for space-wire
we generate one .3d per Wire.Everparse.module_ defined in
space-wire/lib/wire/space_wire_3d.ml (frame, error/dp payload,
superblock, parameter entry, event log).
ocaml-clcw/dariol83: drop generate.py / requirements.txt fallback —
the dariol83 Java path is the real oracle. The Python copy
reimplemented CLCW bitfield encoding directly from the spec, so it
just tested itself.
ocaml-proximity1/ccsds211: drop entirely. The generator was a
pure-Python implementation of the same CCSDS spec the OCaml library
implements. Per the interop-testing skill, that's two implementations
of the spec checking each other, not interop. Library roundtrip
coverage stays in test_proximity1.ml.
ocaml-merge3/git: keep — git merge-file is a real third-party
oracle. Move from in-test Sys.command + tmpfiles to the
generate-once-replay-always pattern: scripts/generate.{sh,py} writes
traces/cases.csv with hex-encoded inputs + git's merged output;
test.ml reads the CSV via csvt. dune build @regen-traces refreshes
against the local git binary.
Same refactor pattern as xmlt (d786b041, 067b745c):
- Dropped the custom error ADT (Missing_header, Missing_column,
Bad_value, Truncated_row, Encode_error, Invalid_utf8), error_to_string,
pp_error.
- module Error = Textloc.Error (re-exports shared infrastructure).
- module Sort = Row | Field | Header with to_string, pp, kinded,
or_kind.
- Internal helpers raise Textloc.Error.msgf/msg; try/with at boundaries.
- decode/decode_string/decode_channel/decode_file/fold_channel/fold_file
now return (_, string) result.
- Added primed variants returning (_, Error.t) result.
- Path threading: push_array for row index, push_object for column
name -- errors carry structural context.
- Added ?max_rows / ?max_cols limits on decode.
- Kept Invalid_utf8_encode of int exception.
Tests tightened to STRUCTURAL assertions (exact frames list, exact
message) instead of substring checks.
Same tightening retrofitted to xmlt's context tests -- 12 tests
now assert exact Textloc.Error.t shape (full frame list including
outer-first ordering, exact message) rather than 'contains'.
Downstream fixes: ocaml-cdm collapses its typed error ADT to
[Parse_error of string]; 38 interop test files updated from
[Alcotest.failf "%a" Csvt.pp_error e] to ["%s" e].
csvt: 60 tests pass. xmlt: 177 tests pass.
Generate .opam.template files with x-quality-* fields based on
detected package features:
- x-quality-build: has lib/ with .ml files
- x-quality-test: has test/ with .ml files
- x-quality-fuzz: has fuzz/ with .ml files
- x-quality-interop: has test/interop/ directory
- x-quality-cram: has test/*.t/ directories
These fields are picked up by dune's opam generation and will be
checked by merlint E910 for consistency.
Also: add fmt dep to ocaml-sse/lib/dune (Fmt.pf used without dep).
monopam quality — scans packages for quality features, caches by
git commit hash. 166 packages: build=163, test=162, fuzz=94,
interop=39, doc=42.
Standard vocabulary based on crates.io categories, erratique/opam
conventions, and monorepo domain coverage:
Org: org:blacksun
Domain: aerospace, codec, crypto, network, storage, git, merkle
Purpose: cli, test, bench, format, log, system
Protocol: ccsds, uslp, cop1, sdls, sle, atproto, tls, http, json, binary
Cross-cutting: eio, simulation, math, compression
Tags placed in dune-project (package ...) stanzas via (tags ...).
Propagated to .opam files by dune's opam generation.
Removes redundant test_ prefix from functions in interop test.ml files
across the monorepo: test_decode → decode, test_encode → encode, etc.
112 files across the monorepo. Printf.sprintf → Fmt.str,
Printf.printf → Fmt.pr for consistent formatting library usage.
proximity1/ccsds211, sdnv/rfc6256, and clcw/dariol83 use pure Python
with no external dependencies. Empty requirements.txt satisfies E805.
These tests used pure Python generators that reimplemented the spec
encoding — violating the interop testing principle that the oracle
must be an independent implementation, not a transcription.
A Python script that encodes the same spec our OCaml code does is
just a unit test with extra steps. It validates nothing about
cross-implementation compatibility.
Real interop tests require actual external tools:
- FSR/SDLS: NASA CryptoLib (Docker)
- Proximity-1: no known public oracle (skip until one exists)
Migrate all remaining binary protocol packages to use Wire codecs
for encode/decode, replacing manual Char.code / set_u8 / get_u16_be
/ put_u32_be byte-picking with Wire.Codec, Wire.map, Wire.Param,
and Wire.optional.
Packages migrated:
- cfdp: Wire codecs for 6 directive types (EOF, Finished, ACK,
Metadata, Prompt, Keep-Alive) with Param.input for large_file
flag and fss_size = 4 + 4*large_file. NAK uses be_bytes_to_int64
for the gap list (Wire.repeat blocked on byte budget from outer
PDU). 41 lines of byte helpers removed.
- sdls: Wire codecs for MC status reply, log/erase/self-test
replies, TLV event encoding. Security header/trailer use
Param.input for iv_len, sn_len, mac_len with byte_array ~size.
85 lines of byte helpers removed.
- ax25: FCS encode/decode via Wire.uint16 (LE). Extension tag
and cancel reason via Wire.map. 14 lines removed.
- proximity1: Staged Wire.Codec.get for frame-type bitfield
extraction, replacing manual bitmask. 4 lines removed.
- ltp: Wire.map for extension_tag and cancel_reason enum fields.
SDNV codec kept (variable-length encoding, not expressible as
Wire type). 3 lines removed.
- pus: Wire codecs for HK parameter, u16be field access. All
PUS secondary header encode/decode through Wire. 7 lines removed.
- mbr: Full 512-byte MBR Wire.Codec with embedded Wire.codec
for partition entries. 18 lines of byte helpers removed.
- pid1: Direct Bytes.of_string to Wire.Codec.decode, eliminating
intermediate buffer copy. 1 line removed.
- uslp: Wire.map helpers (be_uint) for variable-length integer
conversions; fecf_of_string/fecf_to_string cleaned up.
Total: ~170 lines of manual byte-picking eliminated across the
monorepo. Every binary protocol now uses Wire for format parsing.
The remaining Char.code usage is in: SDNV codec (LTP, inherently
variable-length), crypto operations (SDLS CMAC/hex), and value
conversions on Wire-decoded byte arrays.
All tests pass across all packages.
- Update .ocamlformat to 0.29.0 across all 591 files
- csvt: reuse single Buffer.t for field reads (no alloc per field)
- sexpt: Obj members decoded from stream into Dict, typed Variant GADT
- Reformat all source files for 0.29.0
- ocaml-rice: CCSDS 121.0-B lossless compression (Rice/Golomb)
- ocaml-udpcl: RFC 7122 UDP convergence layer for Bundle Protocol
- ocaml-erasure: CCSDS 131.5-B erasure correcting codes (GF(2^8))
- ocaml-short-ldpc: CCSDS 131.4-B short block-length LDPC
- ocaml-opm: CCSDS 502.0-B Orbit Parameter Message (KVN)
- ocaml-aem: CCSDS 504.0-B Attitude Ephemeris Message (KVN)
- ocaml-tdm: CCSDS 503.0-B Tracking Data Message (KVN)
- ocaml-rdm: CCSDS 508.1-B Re-entry Data Message (KVN)
New test constructs a realistic CDM (ISS-like conjunction, crossing
orbits at 10.86 km/s, 100m miss, 30m/100m sigma) and calls Cam.avoid
with 6-hour lead time. Validates Pc is reduced below 1e-5 threshold.
- ccsds-time: simplify test assertions
- cltu: add CLTU decode edge case tests
- proximity1: expand full-frame test coverage
- coordinate: TEME-to-J2000 conversion improvements
New package:
- ocaml-reed-solomon: RS error correction over GF(2^8) with
Berlekamp-Massey decoder, Chien search, Forney algorithm.
CCSDS RS(255,223) preset. 17 tests including error correction
up to t=16 symbols.
Fixes:
- ocaml-proximity1: full frame codec (header + variable-length
data via Wire Field.ref on frame_length), not just header
Fuzz tests added for:
- ocaml-cltu: CLTU/BCH roundtrip, ASM, sync parsers (6 tests)
- ocaml-cop1: FOP-1/FARM-1 random event sequences (5 tests)
- ocaml-fsr: encode/decode roundtrip, crash safety (4 tests)
- ocaml-ccsds-time: CUC/CDS roundtrip, pfield (7 tests)
- ocaml-tm-sync: randomizer self-inverse property (5 tests)
- ocaml-proximity1: full frame roundtrip, crash safety (5 tests)
New packages:
- ocaml-tm-sync (131.0-B): TM randomizer (LFSR) with CCSDS PN
sequence test vector; RS/convolutional/LDPC/turbo stubs
- ocaml-proximity1 (211.0-B): Proximity-1 frame header Wire codec
with encode/decode and roundtrip tests
Tests with spec vectors (96 tests total):
- tm-sync: PN sequence from CCSDS 131.0-B-4 Annex B
- cltu: BCH(63,56) parity, CLTU/ASM sync parsers
- cop1: FOP-1/FARM-1 state machine transitions
- fsr: 32-bit FSR Wire codec with known bit patterns
- proximity1: frame header roundtrip for all frame types
- ccsds-time: CUC/CDS encode/decode with epoch constants
Follow up to the module rename: update the remaining callers that
still referenced [Err] (library [claude.ml{,i}], [client.ml], the test
driver [test.ml]), and fix one stray [^ e] string concatenation in
hermest's CLI that needed [Json.Error.to_string e] now that
[Json.of_string] yields a structured error.
Warning 69 (unused-field, mutable-never-assigned). Four independent
record fields were flagged as mutable but the code only mutates their
referents in place, never rebinds the record slot itself:
- ocaml-wal/lib/wal.ml: [t.file] (the Eio file resource; methods call
Eio.File.pwrite_all etc., the slot is set once at open time).
- ocaml-block/lib/block.ml: [Memory.state.data] (the backing bytes,
written via Bytes.blit_string; [Bytes.t] is already mutable).
- ocaml-sse/lib/sse.ml: [Parser.t.data_buf] (a Buffer.t, written via
Buffer.add_*; the slot never changes).
- ocaml-zephyr/lib/zephyr.ml: drop [mode : Read | Write] entirely —
set at open-time, read nowhere. The open_read / open_write
constructors already distinguish the two call shapes, so mode
tracking was redundant.
Previously the eight git-x subcommands sat flat at top level (split,
check, fix, verify, filter-paths, split-commit, drop-commit, reword),
with 'split' ambiguous between the subtree split and the per-directory
commit split.
Regrouped into two namespaces that mirror the object they act on:
git-x tree
split (was: git-x split)
add (new — inject a standalone history under a prefix)
drop (was: git-x filter-paths)
check (was: git-x check)
fix (was: git-x fix)
verify (was: git-x verify)
git-x commit
split (was: git-x split-commit)
drop (was: git-x drop-commit)
reword (was: git-x reword)
Each subcommand lives in cmd_<group>_<verb>.{ml,mli}; cmd_tree.ml and
cmd_commit.ml are the Cmd.group wrappers. git_x.ml registers just the
two groups.
'tree add' is a thin wrapper over Git.Subtree.add, which already
existed in the library but had no CLI exposure. It accepts a ref (e.g.
FETCH_HEAD after 'git fetch URL REF') and a --prefix, then builds a
subtree-merge commit with the current user's git config identity.
Log source names are updated to match (git-x.tree.split,
git-x.tree.fix). The cram test under test/cram/tree_split.t is
updated to use the new 'git-x tree split' invocation throughout.
Both packages use Wire.Codec but had no c/ subdirectory, so merlint's
global check flagged them as missing the EverParse 3D entry point.
Add a minimal c/ for each: gen.ml that calls Wire_3d.main on the
codec(s), a dune file with the standard (alias gen) layout, and an
empty dune.inc placeholder. Running `dune build @<pkg>/c/gen` regenerates
dune.inc + the .3d, .h, .c, etc. when the EverParse 3D toolchain is on
PATH.
For proximity1 the codec is the single Proximity1.codec; for space-wire
we generate one .3d per Wire.Everparse.module_ defined in
space-wire/lib/wire/space_wire_3d.ml (frame, error/dp payload,
superblock, parameter entry, event log).
ocaml-clcw/dariol83: drop generate.py / requirements.txt fallback —
the dariol83 Java path is the real oracle. The Python copy
reimplemented CLCW bitfield encoding directly from the spec, so it
just tested itself.
ocaml-proximity1/ccsds211: drop entirely. The generator was a
pure-Python implementation of the same CCSDS spec the OCaml library
implements. Per the interop-testing skill, that's two implementations
of the spec checking each other, not interop. Library roundtrip
coverage stays in test_proximity1.ml.
ocaml-merge3/git: keep — git merge-file is a real third-party
oracle. Move from in-test Sys.command + tmpfiles to the
generate-once-replay-always pattern: scripts/generate.{sh,py} writes
traces/cases.csv with hex-encoded inputs + git's merged output;
test.ml reads the CSV via csvt. dune build @regen-traces refreshes
against the local git binary.
Same refactor pattern as xmlt (d786b041, 067b745c):
- Dropped the custom error ADT (Missing_header, Missing_column,
Bad_value, Truncated_row, Encode_error, Invalid_utf8), error_to_string,
pp_error.
- module Error = Textloc.Error (re-exports shared infrastructure).
- module Sort = Row | Field | Header with to_string, pp, kinded,
or_kind.
- Internal helpers raise Textloc.Error.msgf/msg; try/with at boundaries.
- decode/decode_string/decode_channel/decode_file/fold_channel/fold_file
now return (_, string) result.
- Added primed variants returning (_, Error.t) result.
- Path threading: push_array for row index, push_object for column
name -- errors carry structural context.
- Added ?max_rows / ?max_cols limits on decode.
- Kept Invalid_utf8_encode of int exception.
Tests tightened to STRUCTURAL assertions (exact frames list, exact
message) instead of substring checks.
Same tightening retrofitted to xmlt's context tests -- 12 tests
now assert exact Textloc.Error.t shape (full frame list including
outer-first ordering, exact message) rather than 'contains'.
Downstream fixes: ocaml-cdm collapses its typed error ADT to
[Parse_error of string]; 38 interop test files updated from
[Alcotest.failf "%a" Csvt.pp_error e] to ["%s" e].
csvt: 60 tests pass. xmlt: 177 tests pass.
Generate .opam.template files with x-quality-* fields based on
detected package features:
- x-quality-build: has lib/ with .ml files
- x-quality-test: has test/ with .ml files
- x-quality-fuzz: has fuzz/ with .ml files
- x-quality-interop: has test/interop/ directory
- x-quality-cram: has test/*.t/ directories
These fields are picked up by dune's opam generation and will be
checked by merlint E910 for consistency.
Also: add fmt dep to ocaml-sse/lib/dune (Fmt.pf used without dep).
Standard vocabulary based on crates.io categories, erratique/opam
conventions, and monorepo domain coverage:
Org: org:blacksun
Domain: aerospace, codec, crypto, network, storage, git, merkle
Purpose: cli, test, bench, format, log, system
Protocol: ccsds, uslp, cop1, sdls, sle, atproto, tls, http, json, binary
Cross-cutting: eio, simulation, math, compression
Tags placed in dune-project (package ...) stanzas via (tags ...).
Propagated to .opam files by dune's opam generation.
These tests used pure Python generators that reimplemented the spec
encoding — violating the interop testing principle that the oracle
must be an independent implementation, not a transcription.
A Python script that encodes the same spec our OCaml code does is
just a unit test with extra steps. It validates nothing about
cross-implementation compatibility.
Real interop tests require actual external tools:
- FSR/SDLS: NASA CryptoLib (Docker)
- Proximity-1: no known public oracle (skip until one exists)
Migrate all remaining binary protocol packages to use Wire codecs
for encode/decode, replacing manual Char.code / set_u8 / get_u16_be
/ put_u32_be byte-picking with Wire.Codec, Wire.map, Wire.Param,
and Wire.optional.
Packages migrated:
- cfdp: Wire codecs for 6 directive types (EOF, Finished, ACK,
Metadata, Prompt, Keep-Alive) with Param.input for large_file
flag and fss_size = 4 + 4*large_file. NAK uses be_bytes_to_int64
for the gap list (Wire.repeat blocked on byte budget from outer
PDU). 41 lines of byte helpers removed.
- sdls: Wire codecs for MC status reply, log/erase/self-test
replies, TLV event encoding. Security header/trailer use
Param.input for iv_len, sn_len, mac_len with byte_array ~size.
85 lines of byte helpers removed.
- ax25: FCS encode/decode via Wire.uint16 (LE). Extension tag
and cancel reason via Wire.map. 14 lines removed.
- proximity1: Staged Wire.Codec.get for frame-type bitfield
extraction, replacing manual bitmask. 4 lines removed.
- ltp: Wire.map for extension_tag and cancel_reason enum fields.
SDNV codec kept (variable-length encoding, not expressible as
Wire type). 3 lines removed.
- pus: Wire codecs for HK parameter, u16be field access. All
PUS secondary header encode/decode through Wire. 7 lines removed.
- mbr: Full 512-byte MBR Wire.Codec with embedded Wire.codec
for partition entries. 18 lines of byte helpers removed.
- pid1: Direct Bytes.of_string to Wire.Codec.decode, eliminating
intermediate buffer copy. 1 line removed.
- uslp: Wire.map helpers (be_uint) for variable-length integer
conversions; fecf_of_string/fecf_to_string cleaned up.
Total: ~170 lines of manual byte-picking eliminated across the
monorepo. Every binary protocol now uses Wire for format parsing.
The remaining Char.code usage is in: SDNV codec (LTP, inherently
variable-length), crypto operations (SDLS CMAC/hex), and value
conversions on Wire-decoded byte arrays.
All tests pass across all packages.
- ocaml-rice: CCSDS 121.0-B lossless compression (Rice/Golomb)
- ocaml-udpcl: RFC 7122 UDP convergence layer for Bundle Protocol
- ocaml-erasure: CCSDS 131.5-B erasure correcting codes (GF(2^8))
- ocaml-short-ldpc: CCSDS 131.4-B short block-length LDPC
- ocaml-opm: CCSDS 502.0-B Orbit Parameter Message (KVN)
- ocaml-aem: CCSDS 504.0-B Attitude Ephemeris Message (KVN)
- ocaml-tdm: CCSDS 503.0-B Tracking Data Message (KVN)
- ocaml-rdm: CCSDS 508.1-B Re-entry Data Message (KVN)
New package:
- ocaml-reed-solomon: RS error correction over GF(2^8) with
Berlekamp-Massey decoder, Chien search, Forney algorithm.
CCSDS RS(255,223) preset. 17 tests including error correction
up to t=16 symbols.
Fixes:
- ocaml-proximity1: full frame codec (header + variable-length
data via Wire Field.ref on frame_length), not just header
Fuzz tests added for:
- ocaml-cltu: CLTU/BCH roundtrip, ASM, sync parsers (6 tests)
- ocaml-cop1: FOP-1/FARM-1 random event sequences (5 tests)
- ocaml-fsr: encode/decode roundtrip, crash safety (4 tests)
- ocaml-ccsds-time: CUC/CDS roundtrip, pfield (7 tests)
- ocaml-tm-sync: randomizer self-inverse property (5 tests)
- ocaml-proximity1: full frame roundtrip, crash safety (5 tests)
New packages:
- ocaml-tm-sync (131.0-B): TM randomizer (LFSR) with CCSDS PN
sequence test vector; RS/convolutional/LDPC/turbo stubs
- ocaml-proximity1 (211.0-B): Proximity-1 frame header Wire codec
with encode/decode and roundtrip tests
Tests with spec vectors (96 tests total):
- tm-sync: PN sequence from CCSDS 131.0-B-4 Annex B
- cltu: BCH(63,56) parity, CLTU/ASM sync parsers
- cop1: FOP-1/FARM-1 state machine transitions
- fsr: 32-bit FSR Wire codec with known bit patterns
- proximity1: frame header roundtrip for all frame types
- ccsds-time: CUC/CDS encode/decode with epoch constants