star-lite/docs/size-bucketed.png

reviewed

This is a binary file and will not be displayed.

star-lite/docs/vs-raw.png

reviewed

This is a binary file and will not be displayed.

+13 -2

star-lite/readme.md

reviewed

··· 4 4 5 5 STAR-lite describes both a binary encoding, and an efficient algorithm to verify or transform sorted key-record pairs into stream-ordered CAR files. Together, they make STAR-lite suitable as both a network transport, and for long-term repo archiving, without sacrificing interoperability. 6 6 7 7 - STAR-lite files shine when zstd-compressed. 8 8 - 9 7 10 8 ### Compared to [CARs][car]: 11 9 ··· 20 18 - Content verification requires a complete scan of all content 21 19 - No support for sparse archives or "CAR slice"-like proofs (yet) 22 20 - Disk spilling required for memory-bounded streaming of large archives 21 21 + 22 22 + 23 23 + ### Compression-oriented 24 24 + 25 25 + STAR-lite files shine when compressed: almost 2x smaller than CARs for the same compression settings; almost 6x smaller than raw CARs with zstd. 26 26 + 27 27 + CAR vs STAR-lite for various compression configs: 28 28 + 29 29 +  30 30 + 31 31 + CAR vs STAR-lite for identical compression settings, grouped by CAR size: 32 32 + 33 33 +  23 34 24 35 25 36 ## Format