commits
Sweep [(nox-csv :with-test)] into 21 packages that load CSV-format
interop test vectors via [Csv.decode_file] but didn't declare
[nox-csv] as a test-scope opam dep. Most are CCSDS protocol
implementations whose [test/interop/.../test.ml] decodes vendor-
supplied vectors stored as CSV alongside the test fixtures.
ocaml-aos, ocaml-ax25, ocaml-bundle, ocaml-clcw, ocaml-cose,
ocaml-fsr, ocaml-hkdf, ocaml-merge3, ocaml-protobuf, ocaml-pus,
ocaml-reed-solomon, ocaml-rice, ocaml-s3, ocaml-space-packet,
ocaml-tc, ocaml-tcf, ocaml-tcpcl, ocaml-tm, ocaml-turbo, ocaml-uslp,
ocaml-viterbi.
Each verified by [dune build] + [dune runtest].
The block had a stray [in] after [Bytes.make 1024 '\x00'] and used a
top-level [match] expression. Promote to top-level [let], handle
[decompress] via [let () = match ... with | Ok ...] + assert, switch
[Printf.printf]/[Printf.eprintf] to [assert]/[Fmt.failwith], and add
[fmt] to the mdx libraries.
The READMEs all share the standard install/overlay snippet, but the
sh blocks lacked the "<!-- $MDX skip -->" directive. `dune test`
would shell out to `opam install` against the live switch, which
either prompts interactively or fails with a package conflict —
either way diffing as a test failure.
Bulk-add skip directives in front of every install/overlay block.
Also collapse the doubled "non-deterministic + skip" stack on three
READMEs (memtrace, ocaml-dpop, ocaml-pid1, ocaml-yaml, merlint) where
`skip` already implies the runtime is bypassed.
Renames 35 packages to make blacksun forks distinguishable from their
opam-repository upstreams. Module names (Git.x, Tls.x, ...) stay bare;
opam package names and dune (public_name) findlib references move to
nox-X. After this commit, zero local package names overlap with
opam-repository.
Renamed:
- nox-git, nox-irmin
- nox-crypto, nox-crypto-pk, nox-crypto-rng, nox-crypto-ec
- nox-tls, nox-tls-eio, nox-tar, nox-tar-eio, nox-tty, nox-tty-eio
- nox-arp, nox-ca-certs, nox-cbor, nox-cookie, nox-crc, nox-csv
- nox-gpt, nox-hkdf, nox-http, nox-jwt, nox-kdf, nox-loc
- nox-memtrace, nox-pds, nox-sexp, nox-slack, nox-toml
- nox-websocket, nox-x509, nox-xdge, nox-yaml
Also drops orphan tar-mirage and tar-unix opam templates that had no
matching package stanza.
Pure formatting changes from `dune fmt`: doc comment placement moves
from above the binding to below it for `type`s, multi-line `match`
expressions collapse onto one line where they fit, and infix operator
applications pick up spaces (`Soup.($?)` -> `Soup.( $? )`). No
semantic changes.
Object combinators: [Object.mem] -> [Object.member], [Object.opt_mem]
-> [Object.opt_member], [Object.case_mem] -> [Object.case_member]. The
sibling submodules [Object.Mem] / [Object.Mems] become
[Object.Member] / [Object.Members]. RFC 8259 §4 calls these
"name/value pairs, referred to as the members", so mirror the spec
name rather than the shortened [mem].
[Object.finish] -> [Object.seal]. "Seal" reads as "close the map, no
more members added", which is what the operation does.
Value constructors/queries: [Value.mem] (function) -> [Value.member];
[Value.mem_find] -> [Value.member_key]; [Value.mem_names] ->
[Value.member_names]; [Value.mem_keys] -> [Value.member_keys].
[type mem = ...] -> [type member = ...]; [type object'] still points
at [member list].
Downstream (~80 files across slack, sbom, stripe, sigstore, requests,
claude, irmin, freebox) updated via perl-pie. dune build clean,
dune test ocaml-json clean.
Follow up to the module rename: update the remaining callers that
still referenced [Err] (library [claude.ml{,i}], [client.ml], the test
driver [test.ml]), and fix one stray [^ e] string concatenation in
hermest's CLI that needed [Json.Error.to_string e] now that
[Json.of_string] yields a structured error.
Commit c680a7407 switched [best_split_k] to use the [select_k]
heuristic (floor(log2(mean))), reasoning that this is what Rice
coding normally does per CCSDS 121.0-B-3. That's mathematically
true but practically wrong for this library:
libaec (our CCSDS 121 reference oracle in test/interop/libaec/)
implements its own [best_split_k] via an exhaustive scan with a
first-found-wins tiebreak. On tie blocks — inputs where multiple k
values produce the same total bit length, e.g. an 8-bit uniform
ramp of 16 samples where k=2 and k=3 both yield 72 bits — libaec
picks whichever k comes first in [0..kmax]. The heuristic picks
floor(log2(mean)), which disagrees on those ties.
The interop test [compress] case [ramp_bs16_bps8] fails byte-for-
byte after the heuristic switch, emitting a 1-bit-shifted output
that phase-propagates through the entire stream. Interop-exact
match against libaec is non-negotiable for a reference-aligned
library.
Restore the exhaustive scan and keep [select_k]'s reasoning as a
comment so the next maintainer knows why we search instead of
computing. [select_k] itself is deleted (no callers).
Test result:
interop/libaec: 8/8 pass (compress + decompress × 4 vectors)
fuzz: 6/6 pass
[best_split_k] was doing an O(kmax) exhaustive search for the split
parameter that minimises the encoded length. That's not what Rice
coding normally does.
CCSDS 121.0-B-3 §5.1.3 specifies the split parameter selection
directly: k = floor(log2(block_sum / J)) clamped to [0, kmax]. This
is the Rice/Golomb optimum for the geometric distribution the
residuals follow, so there's no reason to search.
The file already had [select_k] implementing exactly this heuristic,
but it was unreachable — a warning-32 from the build sweep surfaced
it. Wiring it into [best_split_k]:
let best_split_k residuals res_ofs count id_len bps is_ref kmax =
let k = select_k residuals res_ofs count kmax in
let len = id_len + (if is_ref then bps else 0)
+ split_encoded_len residuals res_ofs count k in
(k, len)
Also delete Bitwriter.bit_length — a module-signature entry that was
never called from outside Bitwriter.
All 6 fuzz tests still pass (correlated compresses well, identical
data compresses, random data roundtrips, etc.), confirming the
heuristic matches what the exhaustive scan picked on tested inputs.
Warning 69 (unused-field, mutable-never-assigned). Four independent
record fields were flagged as mutable but the code only mutates their
referents in place, never rebinds the record slot itself:
- ocaml-wal/lib/wal.ml: [t.file] (the Eio file resource; methods call
Eio.File.pwrite_all etc., the slot is set once at open time).
- ocaml-block/lib/block.ml: [Memory.state.data] (the backing bytes,
written via Bytes.blit_string; [Bytes.t] is already mutable).
- ocaml-sse/lib/sse.ml: [Parser.t.data_buf] (a Buffer.t, written via
Buffer.add_*; the slot never changes).
- ocaml-zephyr/lib/zephyr.ml: drop [mode : Read | Write] entirely —
set at open-time, read nowhere. The open_read / open_write
constructors already distinguish the two call shapes, so mode
tracking was redundant.
Previously the eight git-x subcommands sat flat at top level (split,
check, fix, verify, filter-paths, split-commit, drop-commit, reword),
with 'split' ambiguous between the subtree split and the per-directory
commit split.
Regrouped into two namespaces that mirror the object they act on:
git-x tree
split (was: git-x split)
add (new — inject a standalone history under a prefix)
drop (was: git-x filter-paths)
check (was: git-x check)
fix (was: git-x fix)
verify (was: git-x verify)
git-x commit
split (was: git-x split-commit)
drop (was: git-x drop-commit)
reword (was: git-x reword)
Each subcommand lives in cmd_<group>_<verb>.{ml,mli}; cmd_tree.ml and
cmd_commit.ml are the Cmd.group wrappers. git_x.ml registers just the
two groups.
'tree add' is a thin wrapper over Git.Subtree.add, which already
existed in the library but had no CLI exposure. It accepts a ref (e.g.
FETCH_HEAD after 'git fetch URL REF') and a --prefix, then builds a
subtree-merge commit with the current user's git config identity.
Log source names are updated to match (git-x.tree.split,
git-x.tree.fix). The cram test under test/cram/tree_split.t is
updated to use the new 'git-x tree split' invocation throughout.
Drops the redundant "t" suffix. Library name, opam package, module
(Csvt -> Csv), and directory all rename in lockstep. Every downstream
reference in interop tests, libraries, and docs updated.
Split encode_ccsds_block into best_split_k/encode_second_extension/
encode_uncompressed/encode_split. Pull decode_zero_blocks/decode_second
_extension/decode_uncompressed/decode_split out of decode_ccsds_blocks.
Compress now delegates each block to encode_one_block and a
count_zero_run helper; decompress factors out decode_rsi /
decompress_blocks. Move Alcotest.run from test_rice.ml into a new
test.ml exporting the rice suite. Add fuzz_rice.mli interface.
Same refactor pattern as xmlt (d786b041, 067b745c):
- Dropped the custom error ADT (Missing_header, Missing_column,
Bad_value, Truncated_row, Encode_error, Invalid_utf8), error_to_string,
pp_error.
- module Error = Textloc.Error (re-exports shared infrastructure).
- module Sort = Row | Field | Header with to_string, pp, kinded,
or_kind.
- Internal helpers raise Textloc.Error.msgf/msg; try/with at boundaries.
- decode/decode_string/decode_channel/decode_file/fold_channel/fold_file
now return (_, string) result.
- Added primed variants returning (_, Error.t) result.
- Path threading: push_array for row index, push_object for column
name -- errors carry structural context.
- Added ?max_rows / ?max_cols limits on decode.
- Kept Invalid_utf8_encode of int exception.
Tests tightened to STRUCTURAL assertions (exact frames list, exact
message) instead of substring checks.
Same tightening retrofitted to xmlt's context tests -- 12 tests
now assert exact Textloc.Error.t shape (full frame list including
outer-first ordering, exact message) rather than 'contains'.
Downstream fixes: ocaml-cdm collapses its typed error ADT to
[Parse_error of string]; 38 interop test files updated from
[Alcotest.failf "%a" Csvt.pp_error e] to ["%s" e].
csvt: 60 tests pass. xmlt: 177 tests pass.
Generate .opam.template files with x-quality-* fields based on
detected package features:
- x-quality-build: has lib/ with .ml files
- x-quality-test: has test/ with .ml files
- x-quality-fuzz: has fuzz/ with .ml files
- x-quality-interop: has test/interop/ directory
- x-quality-cram: has test/*.t/ directories
These fields are picked up by dune's opam generation and will be
checked by merlint E910 for consistency.
Also: add fmt dep to ocaml-sse/lib/dune (Fmt.pf used without dep).
monopam quality — scans packages for quality features, caches by
git commit hash. 166 packages: build=163, test=162, fuzz=94,
interop=39, doc=42.
Standard vocabulary based on crates.io categories, erratique/opam
conventions, and monorepo domain coverage:
Org: org:blacksun
Domain: aerospace, codec, crypto, network, storage, git, merkle
Purpose: cli, test, bench, format, log, system
Protocol: ccsds, uslp, cop1, sdls, sle, atproto, tls, http, json, binary
Cross-cutting: eio, simulation, math, compression
Tags placed in dune-project (package ...) stanzas via (tags ...).
Propagated to .opam files by dune's opam generation.
- ocaml-cose/lib/cose.mli: add blank lines between val groups (warning 50)
- ocaml-mal/lib/mal.mli: same fix
- ocaml-rice/lib/rice.ml: remove stray ) after Fmt.invalid_arg
- ocaml-lfsr: remove stray ) in lib and test
- ocaml-idc: remove stray ) after Fmt.invalid_arg
- ocaml-aem, ocaml-adm, ocaml-opm, ocaml-tdm, ocaml-odm: fix Fmt.str
paren placement (args outside parens)
- ocaml-xmlt: fix bench/fuzz Fmt.str parens
- ocaml-oci/test/interop/registry: split into test_index.ml,
test_manifest.ml, test_config.ml with test.ml runner
- ocaml-gpt: exclude c/ dir (EverParse UINT64 pending)
- fmt dependency added to 11 coding packages
ocaml-{erasure,viterbi,turbo,short-ldpc,rice,lfsr,ldpc,idc,hcomp}/lib/dune
and ocaml-crc/bench/dune, ocaml-xmlt/fuzz/dune all use Fmt.str without
depending on fmt.
Removes redundant test_ prefix from functions in interop test.ml files
across the monorepo: test_decode → decode, test_encode → encode, etc.
112 files across the monorepo. Printf.sprintf → Fmt.str,
Printf.printf → Fmt.pr for consistent formatting library usage.
The RICE encoder now aggregates consecutive zero blocks into a single
header with a block count, matching libaec's encoding. This produces
significantly more compact output for constant or near-constant data
(e.g. 2 bytes vs 4 bytes for 64 constant 8-bit samples).
Implements the CCSDS 121.0-B-3 §5.1 ROS (rest of segment) encoding:
FS(0..3) → 1..4 blocks
FS(4) → rest of segment
FS(n≥5) → n-1 blocks (shifted to avoid collision with ROS)
The interop test now verifies byte-exact compression match against
libaec for all vectors including constant data.
Replace live C stubs (requiring libaec at test time) with the standard
generate-once-replay-always pattern. The generator compresses test
vectors with libaec and commits the traces; the test decompresses with
ocaml-rice and verifies byte-exact match.
Constant-data vectors are excluded from compress comparison — CCSDS
121.0-B-3 allows implementation-dependent zero-block encodings. Both
produce valid bitstreams; decompress interop is verified for all vectors.
Created 7 new READMEs: xmlt, dsp, demod, rtlsdr, erasure,
short-ldpc, ccsds (meta-package with full protocol suite table).
Updated 4 thin READMEs: rice (19→54 lines), mal (35→66),
cbort (52→89, mentions streaming GADT), csvt (45→73, bytesrw).
Each has: title, spec reference, quick start example, API overview.
- Update .ocamlformat to 0.29.0 across all 591 files
- csvt: reuse single Buffer.t for field reads (no alloc per field)
- sexpt: Obj members decoded from stream into Dict, typed Variant GADT
- Reformat all source files for 0.29.0
Remove ocaml-rice/test/debug/ (temporary libaec debug stubs).
Update ocaml-crypto, ocaml-csvt, ocaml-sexpt, ocaml-tomlt, ocaml-xmlt.
Replace closure-based codec with GADT: Text | Text_map | Element |
El | Raw | Map | Const | Option | Rec | List. decode/encode are
generic interpreters. El builder uses Dict + Type.Id.
150 tests + 10 XTCE pass. Public API unchanged.
Replace opaque closure record with a GADT that preserves codec
structure, following the approach from Buenzli's "An Alphabet for
Your Data Soups" paper:
- GADT constructors for each S-expression sort (Atom, List, Obj,
Any, Map, Rec, Variant, Pair, Triple, etc.)
- dec_fun GADT with Type.Id for unordered record member decoding
- Heterogeneous Dict for buffering typed member values
- Structural encode/decode by pattern matching on GADT
- New query/update API: get_mem, get_nth, update_mem, delete_mem
Internal redesign only — 'a t stays abstract, all existing tests
pass unchanged.
ocaml-idc (CCSDS 122.0-B):
- Implement CDF 9/7 lifting wavelet for lossy compression (was stubbed)
- 4-step lifting scheme with standard coefficients (α,β,γ,δ,K)
- Wavelet ID in header for format-aware decompression
- Tests: PSNR >40dB roundtrip, constant exact, 9/7 beats 5/3 on smooth
ocaml-hcomp (CCSDS 123.0-B):
- Fix 16-bit entropy coder truncation bug: escape values need bps+1 bits
(zigzag-mapped residuals can reach 2*(2^bps-1), caught by new fuzz test)
- 8 new tests: wire format (4 configs), spectral decorrelation, bps 1-16
- 2 new fuzz tests: roundtrip across all bps values, output length invariant
ocaml-rice (CCSDS 121.0-B):
- 9 new tests: wire format (5 exact byte checks), edge cases (4)
- 2 new fuzz tests: correlated data compresses well, determinism
- libaec interop test skeleton (test/interop/libaec/) — cross-validates
ocaml-rice against libaec via aec CLI, skips gracefully if not installed
Rice (CCSDS 121.0-B):
- Fix prediction error mapper per spec (modular arithmetic with
theta-based branching, matching libaec reference implementation)
- Fix select_k to use floor instead of round
- Add bounds check on decompress sample count
- All 11 tests + 4 fuzz tests pass
ODM consolidation (CCSDS 502.0-B):
- Merge OPM into ocaml-odm alongside OEM
- Access via Odm.Oem and Odm.Opm submodules
- Backward-compatible: Odm.of_kvn_string still works for OEM
ADM (CCSDS 504.0-B):
- Create ocaml-adm from ocaml-aem
- Access via Adm.Aem submodule (APM to be added)
ocaml-ccsds index:
- Meta-package with index.mld documenting the full protocol suite
- Organized by Blue Book / Green Book / Related Standards
- Links to CCSDS PDF specs for each standard
- Lists all implemented + not-yet-implemented specs
Replace manual Bytes.get/set in tml.ml with Wire.Codec for
TML PDU header (8 bytes) and context message body (12 bytes).
All 85 SLE tests pass.
- ocaml-rice: CCSDS 121.0-B lossless compression (Rice/Golomb)
- ocaml-udpcl: RFC 7122 UDP convergence layer for Bundle Protocol
- ocaml-erasure: CCSDS 131.5-B erasure correcting codes (GF(2^8))
- ocaml-short-ldpc: CCSDS 131.4-B short block-length LDPC
- ocaml-opm: CCSDS 502.0-B Orbit Parameter Message (KVN)
- ocaml-aem: CCSDS 504.0-B Attitude Ephemeris Message (KVN)
- ocaml-tdm: CCSDS 503.0-B Tracking Data Message (KVN)
- ocaml-rdm: CCSDS 508.1-B Re-entry Data Message (KVN)
Sweep [(nox-csv :with-test)] into 21 packages that load CSV-format
interop test vectors via [Csv.decode_file] but didn't declare
[nox-csv] as a test-scope opam dep. Most are CCSDS protocol
implementations whose [test/interop/.../test.ml] decodes vendor-
supplied vectors stored as CSV alongside the test fixtures.
ocaml-aos, ocaml-ax25, ocaml-bundle, ocaml-clcw, ocaml-cose,
ocaml-fsr, ocaml-hkdf, ocaml-merge3, ocaml-protobuf, ocaml-pus,
ocaml-reed-solomon, ocaml-rice, ocaml-s3, ocaml-space-packet,
ocaml-tc, ocaml-tcf, ocaml-tcpcl, ocaml-tm, ocaml-turbo, ocaml-uslp,
ocaml-viterbi.
Each verified by [dune build] + [dune runtest].
The READMEs all share the standard install/overlay snippet, but the
sh blocks lacked the "<!-- $MDX skip -->" directive. `dune test`
would shell out to `opam install` against the live switch, which
either prompts interactively or fails with a package conflict —
either way diffing as a test failure.
Bulk-add skip directives in front of every install/overlay block.
Also collapse the doubled "non-deterministic + skip" stack on three
READMEs (memtrace, ocaml-dpop, ocaml-pid1, ocaml-yaml, merlint) where
`skip` already implies the runtime is bypassed.
Renames 35 packages to make blacksun forks distinguishable from their
opam-repository upstreams. Module names (Git.x, Tls.x, ...) stay bare;
opam package names and dune (public_name) findlib references move to
nox-X. After this commit, zero local package names overlap with
opam-repository.
Renamed:
- nox-git, nox-irmin
- nox-crypto, nox-crypto-pk, nox-crypto-rng, nox-crypto-ec
- nox-tls, nox-tls-eio, nox-tar, nox-tar-eio, nox-tty, nox-tty-eio
- nox-arp, nox-ca-certs, nox-cbor, nox-cookie, nox-crc, nox-csv
- nox-gpt, nox-hkdf, nox-http, nox-jwt, nox-kdf, nox-loc
- nox-memtrace, nox-pds, nox-sexp, nox-slack, nox-toml
- nox-websocket, nox-x509, nox-xdge, nox-yaml
Also drops orphan tar-mirage and tar-unix opam templates that had no
matching package stanza.
Object combinators: [Object.mem] -> [Object.member], [Object.opt_mem]
-> [Object.opt_member], [Object.case_mem] -> [Object.case_member]. The
sibling submodules [Object.Mem] / [Object.Mems] become
[Object.Member] / [Object.Members]. RFC 8259 §4 calls these
"name/value pairs, referred to as the members", so mirror the spec
name rather than the shortened [mem].
[Object.finish] -> [Object.seal]. "Seal" reads as "close the map, no
more members added", which is what the operation does.
Value constructors/queries: [Value.mem] (function) -> [Value.member];
[Value.mem_find] -> [Value.member_key]; [Value.mem_names] ->
[Value.member_names]; [Value.mem_keys] -> [Value.member_keys].
[type mem = ...] -> [type member = ...]; [type object'] still points
at [member list].
Downstream (~80 files across slack, sbom, stripe, sigstore, requests,
claude, irmin, freebox) updated via perl-pie. dune build clean,
dune test ocaml-json clean.
Follow up to the module rename: update the remaining callers that
still referenced [Err] (library [claude.ml{,i}], [client.ml], the test
driver [test.ml]), and fix one stray [^ e] string concatenation in
hermest's CLI that needed [Json.Error.to_string e] now that
[Json.of_string] yields a structured error.
Commit c680a7407 switched [best_split_k] to use the [select_k]
heuristic (floor(log2(mean))), reasoning that this is what Rice
coding normally does per CCSDS 121.0-B-3. That's mathematically
true but practically wrong for this library:
libaec (our CCSDS 121 reference oracle in test/interop/libaec/)
implements its own [best_split_k] via an exhaustive scan with a
first-found-wins tiebreak. On tie blocks — inputs where multiple k
values produce the same total bit length, e.g. an 8-bit uniform
ramp of 16 samples where k=2 and k=3 both yield 72 bits — libaec
picks whichever k comes first in [0..kmax]. The heuristic picks
floor(log2(mean)), which disagrees on those ties.
The interop test [compress] case [ramp_bs16_bps8] fails byte-for-
byte after the heuristic switch, emitting a 1-bit-shifted output
that phase-propagates through the entire stream. Interop-exact
match against libaec is non-negotiable for a reference-aligned
library.
Restore the exhaustive scan and keep [select_k]'s reasoning as a
comment so the next maintainer knows why we search instead of
computing. [select_k] itself is deleted (no callers).
Test result:
interop/libaec: 8/8 pass (compress + decompress × 4 vectors)
fuzz: 6/6 pass
[best_split_k] was doing an O(kmax) exhaustive search for the split
parameter that minimises the encoded length. That's not what Rice
coding normally does.
CCSDS 121.0-B-3 §5.1.3 specifies the split parameter selection
directly: k = floor(log2(block_sum / J)) clamped to [0, kmax]. This
is the Rice/Golomb optimum for the geometric distribution the
residuals follow, so there's no reason to search.
The file already had [select_k] implementing exactly this heuristic,
but it was unreachable — a warning-32 from the build sweep surfaced
it. Wiring it into [best_split_k]:
let best_split_k residuals res_ofs count id_len bps is_ref kmax =
let k = select_k residuals res_ofs count kmax in
let len = id_len + (if is_ref then bps else 0)
+ split_encoded_len residuals res_ofs count k in
(k, len)
Also delete Bitwriter.bit_length — a module-signature entry that was
never called from outside Bitwriter.
All 6 fuzz tests still pass (correlated compresses well, identical
data compresses, random data roundtrips, etc.), confirming the
heuristic matches what the exhaustive scan picked on tested inputs.
Warning 69 (unused-field, mutable-never-assigned). Four independent
record fields were flagged as mutable but the code only mutates their
referents in place, never rebinds the record slot itself:
- ocaml-wal/lib/wal.ml: [t.file] (the Eio file resource; methods call
Eio.File.pwrite_all etc., the slot is set once at open time).
- ocaml-block/lib/block.ml: [Memory.state.data] (the backing bytes,
written via Bytes.blit_string; [Bytes.t] is already mutable).
- ocaml-sse/lib/sse.ml: [Parser.t.data_buf] (a Buffer.t, written via
Buffer.add_*; the slot never changes).
- ocaml-zephyr/lib/zephyr.ml: drop [mode : Read | Write] entirely —
set at open-time, read nowhere. The open_read / open_write
constructors already distinguish the two call shapes, so mode
tracking was redundant.
Previously the eight git-x subcommands sat flat at top level (split,
check, fix, verify, filter-paths, split-commit, drop-commit, reword),
with 'split' ambiguous between the subtree split and the per-directory
commit split.
Regrouped into two namespaces that mirror the object they act on:
git-x tree
split (was: git-x split)
add (new — inject a standalone history under a prefix)
drop (was: git-x filter-paths)
check (was: git-x check)
fix (was: git-x fix)
verify (was: git-x verify)
git-x commit
split (was: git-x split-commit)
drop (was: git-x drop-commit)
reword (was: git-x reword)
Each subcommand lives in cmd_<group>_<verb>.{ml,mli}; cmd_tree.ml and
cmd_commit.ml are the Cmd.group wrappers. git_x.ml registers just the
two groups.
'tree add' is a thin wrapper over Git.Subtree.add, which already
existed in the library but had no CLI exposure. It accepts a ref (e.g.
FETCH_HEAD after 'git fetch URL REF') and a --prefix, then builds a
subtree-merge commit with the current user's git config identity.
Log source names are updated to match (git-x.tree.split,
git-x.tree.fix). The cram test under test/cram/tree_split.t is
updated to use the new 'git-x tree split' invocation throughout.
Split encode_ccsds_block into best_split_k/encode_second_extension/
encode_uncompressed/encode_split. Pull decode_zero_blocks/decode_second
_extension/decode_uncompressed/decode_split out of decode_ccsds_blocks.
Compress now delegates each block to encode_one_block and a
count_zero_run helper; decompress factors out decode_rsi /
decompress_blocks. Move Alcotest.run from test_rice.ml into a new
test.ml exporting the rice suite. Add fuzz_rice.mli interface.
Same refactor pattern as xmlt (d786b041, 067b745c):
- Dropped the custom error ADT (Missing_header, Missing_column,
Bad_value, Truncated_row, Encode_error, Invalid_utf8), error_to_string,
pp_error.
- module Error = Textloc.Error (re-exports shared infrastructure).
- module Sort = Row | Field | Header with to_string, pp, kinded,
or_kind.
- Internal helpers raise Textloc.Error.msgf/msg; try/with at boundaries.
- decode/decode_string/decode_channel/decode_file/fold_channel/fold_file
now return (_, string) result.
- Added primed variants returning (_, Error.t) result.
- Path threading: push_array for row index, push_object for column
name -- errors carry structural context.
- Added ?max_rows / ?max_cols limits on decode.
- Kept Invalid_utf8_encode of int exception.
Tests tightened to STRUCTURAL assertions (exact frames list, exact
message) instead of substring checks.
Same tightening retrofitted to xmlt's context tests -- 12 tests
now assert exact Textloc.Error.t shape (full frame list including
outer-first ordering, exact message) rather than 'contains'.
Downstream fixes: ocaml-cdm collapses its typed error ADT to
[Parse_error of string]; 38 interop test files updated from
[Alcotest.failf "%a" Csvt.pp_error e] to ["%s" e].
csvt: 60 tests pass. xmlt: 177 tests pass.
Generate .opam.template files with x-quality-* fields based on
detected package features:
- x-quality-build: has lib/ with .ml files
- x-quality-test: has test/ with .ml files
- x-quality-fuzz: has fuzz/ with .ml files
- x-quality-interop: has test/interop/ directory
- x-quality-cram: has test/*.t/ directories
These fields are picked up by dune's opam generation and will be
checked by merlint E910 for consistency.
Also: add fmt dep to ocaml-sse/lib/dune (Fmt.pf used without dep).
Standard vocabulary based on crates.io categories, erratique/opam
conventions, and monorepo domain coverage:
Org: org:blacksun
Domain: aerospace, codec, crypto, network, storage, git, merkle
Purpose: cli, test, bench, format, log, system
Protocol: ccsds, uslp, cop1, sdls, sle, atproto, tls, http, json, binary
Cross-cutting: eio, simulation, math, compression
Tags placed in dune-project (package ...) stanzas via (tags ...).
Propagated to .opam files by dune's opam generation.
- ocaml-cose/lib/cose.mli: add blank lines between val groups (warning 50)
- ocaml-mal/lib/mal.mli: same fix
- ocaml-rice/lib/rice.ml: remove stray ) after Fmt.invalid_arg
- ocaml-lfsr: remove stray ) in lib and test
- ocaml-idc: remove stray ) after Fmt.invalid_arg
- ocaml-aem, ocaml-adm, ocaml-opm, ocaml-tdm, ocaml-odm: fix Fmt.str
paren placement (args outside parens)
- ocaml-xmlt: fix bench/fuzz Fmt.str parens
- ocaml-oci/test/interop/registry: split into test_index.ml,
test_manifest.ml, test_config.ml with test.ml runner
- ocaml-gpt: exclude c/ dir (EverParse UINT64 pending)
- fmt dependency added to 11 coding packages
The RICE encoder now aggregates consecutive zero blocks into a single
header with a block count, matching libaec's encoding. This produces
significantly more compact output for constant or near-constant data
(e.g. 2 bytes vs 4 bytes for 64 constant 8-bit samples).
Implements the CCSDS 121.0-B-3 §5.1 ROS (rest of segment) encoding:
FS(0..3) → 1..4 blocks
FS(4) → rest of segment
FS(n≥5) → n-1 blocks (shifted to avoid collision with ROS)
The interop test now verifies byte-exact compression match against
libaec for all vectors including constant data.
Replace live C stubs (requiring libaec at test time) with the standard
generate-once-replay-always pattern. The generator compresses test
vectors with libaec and commits the traces; the test decompresses with
ocaml-rice and verifies byte-exact match.
Constant-data vectors are excluded from compress comparison — CCSDS
121.0-B-3 allows implementation-dependent zero-block encodings. Both
produce valid bitstreams; decompress interop is verified for all vectors.
Created 7 new READMEs: xmlt, dsp, demod, rtlsdr, erasure,
short-ldpc, ccsds (meta-package with full protocol suite table).
Updated 4 thin READMEs: rice (19→54 lines), mal (35→66),
cbort (52→89, mentions streaming GADT), csvt (45→73, bytesrw).
Each has: title, spec reference, quick start example, API overview.
Replace opaque closure record with a GADT that preserves codec
structure, following the approach from Buenzli's "An Alphabet for
Your Data Soups" paper:
- GADT constructors for each S-expression sort (Atom, List, Obj,
Any, Map, Rec, Variant, Pair, Triple, etc.)
- dec_fun GADT with Type.Id for unordered record member decoding
- Heterogeneous Dict for buffering typed member values
- Structural encode/decode by pattern matching on GADT
- New query/update API: get_mem, get_nth, update_mem, delete_mem
Internal redesign only — 'a t stays abstract, all existing tests
pass unchanged.
ocaml-idc (CCSDS 122.0-B):
- Implement CDF 9/7 lifting wavelet for lossy compression (was stubbed)
- 4-step lifting scheme with standard coefficients (α,β,γ,δ,K)
- Wavelet ID in header for format-aware decompression
- Tests: PSNR >40dB roundtrip, constant exact, 9/7 beats 5/3 on smooth
ocaml-hcomp (CCSDS 123.0-B):
- Fix 16-bit entropy coder truncation bug: escape values need bps+1 bits
(zigzag-mapped residuals can reach 2*(2^bps-1), caught by new fuzz test)
- 8 new tests: wire format (4 configs), spectral decorrelation, bps 1-16
- 2 new fuzz tests: roundtrip across all bps values, output length invariant
ocaml-rice (CCSDS 121.0-B):
- 9 new tests: wire format (5 exact byte checks), edge cases (4)
- 2 new fuzz tests: correlated data compresses well, determinism
- libaec interop test skeleton (test/interop/libaec/) — cross-validates
ocaml-rice against libaec via aec CLI, skips gracefully if not installed
Rice (CCSDS 121.0-B):
- Fix prediction error mapper per spec (modular arithmetic with
theta-based branching, matching libaec reference implementation)
- Fix select_k to use floor instead of round
- Add bounds check on decompress sample count
- All 11 tests + 4 fuzz tests pass
ODM consolidation (CCSDS 502.0-B):
- Merge OPM into ocaml-odm alongside OEM
- Access via Odm.Oem and Odm.Opm submodules
- Backward-compatible: Odm.of_kvn_string still works for OEM
ADM (CCSDS 504.0-B):
- Create ocaml-adm from ocaml-aem
- Access via Adm.Aem submodule (APM to be added)
ocaml-ccsds index:
- Meta-package with index.mld documenting the full protocol suite
- Organized by Blue Book / Green Book / Related Standards
- Links to CCSDS PDF specs for each standard
- Lists all implemented + not-yet-implemented specs
- ocaml-rice: CCSDS 121.0-B lossless compression (Rice/Golomb)
- ocaml-udpcl: RFC 7122 UDP convergence layer for Bundle Protocol
- ocaml-erasure: CCSDS 131.5-B erasure correcting codes (GF(2^8))
- ocaml-short-ldpc: CCSDS 131.4-B short block-length LDPC
- ocaml-opm: CCSDS 502.0-B Orbit Parameter Message (KVN)
- ocaml-aem: CCSDS 504.0-B Attitude Ephemeris Message (KVN)
- ocaml-tdm: CCSDS 503.0-B Tracking Data Message (KVN)
- ocaml-rdm: CCSDS 508.1-B Re-entry Data Message (KVN)