microcosm.blue / star
STreaming ARchives: stricter, verifiable, deterministic, highly compressible alternatives to CAR files for atproto repositories.
atproto car
9
fork

Configure Feed

Select the types of activity you want to include in your feed.

read-out optimization note

phil 2953ecf2 b8868ad4

+3
+3
star-lite/readme.md
··· 187 187 188 188 Since our depth-first walk finalizes children before parents, and the final parent finalizes last, we must unfortunately buffer all serialized CAR frames while the tree is walked. The good news is that a disk-spill-friendly byte log works well for this buffering. 189 189 190 + There is a nice optimization available if read-out performance suffers from the random access patterns (eg., on HDDs or networked block storage): keep a separate byte log for records and MST nodes. Most of the data is in the records, and they are inserted in slow-storage-friendly read-out order. 191 + 192 + 190 193 #### Pseudo-code 191 194 192 195 ```python