Problem
Hosting holds memory nothing needs, and none of it is sized for the 1 GiB edges (sjo1 runs hosting at 61 MiB + 48 MiB swap under memory pressure, baseline-memory.md). Read at 041de81 (v1.3.3, which includes #29 streamed promotion):
packages/@wispplace/observability/src/core.ts:85-95,146-151,318-326: every process keeps a 5000-entry log ring and a 10 000-entry request-metric ring, filled withunshift(O(n) per entry). Hosting never reads them:getLogs/getMetricsare called only from tests (grep), and VictoriaLogs/probe.py read console output, metrics go through OTLP.apps/hosting-service/src/lib/cache-manager.ts:372:siteCacheholds up to 5000 site manifests (file_cids+file_objects, every path, CID and CAS key of a site) with no byte cap. A 1000-file manifest is ~230 KiB of heap, so the count cap alone allows over 1 GiB.apps/hosting-service/src/index.ts:42+cache-manager.ts:317: the expiry sweep runs every 30 min, but the manifest TTL is 10 s (cache-manager.ts:347). Expired manifests stay resident until the same key is read again or the sweep runs.apps/hosting-service/src/lib/storage.ts:29: hot tier defaults to 100 MiB on every box. No deploy config setsHOT_CACHE_SIZE(grep ofdeploy/), so sjo1 inherits it.storage.ts:676-698: entries past the 60 s TTL are refused on read but their bytes stay until LRU or count eviction pushes them out.packages/@wispplace/tiered-storage/src/TieredStorage.ts:67-72,318,350,393: every buffered read returnsnew Uint8Array(result.data), a full copy of an object up to 200 MiB, on top of the tier's own buffer. Hosting's buffered readers (file-serving.ts:260,275,304,redirects.ts:35,html-prewarm.ts:64) only hash, decode or serve those bytes.
Solution
- observability:
setInMemoryRetention({ logs, metrics })resizes the rings, and 0 keeps nothing at all. Defaults stay 5000/10 000 for the other services. Hosting'sindex.tssets both to 0. Console lines, Loki pushes, OTLP metrics and the deduping error tracker (500 entries, needed for Loki's aggregated errors) are unchanged. - manifests:
siteCachegetsmaxSize=MANIFEST_CACHE_SIZE(default 16 MiB) with a JSON-shape byte estimator (estimateJsonBytes). The count cap stays asMANIFEST_CACHE_COUNT(default 5000). The estimator errs high against measured Bun 1.4.2 heap (33.8 MiB estimated vs 21.9 MiB measured for the bench's 2000 manifests). Hosting's sweep interval goes from 30 min to 60 s, so lapsed entries in every namespace are freed within about a minute. The sweep is a single pass over at most ~42k entries. - hot tier: the default is 32 MiB, sized for a 1 GiB edge, and
HOT_CACHE_SIZEstill overrides it.TTLMemoryTieralready walked its timestamps on every write. That same pass now also deletes expired entries instead of only dropping timestamps of LRU-evicted ones. - buffered reads:
TieredStorage.get/getWithMetadataaccept{ borrowData: true }, which returns the tier's bytes without the copy, the same idea as the existingborrowChunksfor streams. Metadata is still cloned. Hosting's five buffered call sites borrow. I checked that Bun neither detaches nor mutates aUint8Arrayserved throughbufferResponseBody/Response, and the integration suite runs unchanged. - rejected: deriving defaults from
os.totalmem(). In containers it reports host RAM, and a fixed default is predictable and easy to raise per edge. Also rejected: dropping the rings from the package entirely, since main-app/firehose may still want them and that would be a wider change. A per-object hot cap and fixing theupdateAccessStatsbacklog (below) are left for follow-ups.
Response bytes and headers do not change. The bench hashes status + every header except date, x-cache-tier and tier-dependent framing (content-length/transfer-encoding) + body for all 6040 distinct requests, and before and after give the same digest 8c31749101555b17, with 0 responses changing between passes.
Before
Method, identical for both columns: apps/hosting-service/scripts/memory-bench.ts (committed). It seeds a disk-source store (2000 per-site index.html, pooled HTML/CSS/JS/WebP, two 2–4 MiB HTML pages, four 3–16 MiB MP4s), then starts src/index.ts with scripts/memory-bench-preload.ts. The preload fakes postgres (fresh jsonb-parsed rows per query, nothing cached on the fake side) and runs no Redis or S3. Sites carry 25 files (90%), 200 (9%) or 1000 (1%), so 2000 manifests hold 101k paths. The client in the same container walks all sites once (warmup), then 4 more passes in fixed permutations: /, /assets/style.css, /assets/app.js, plus a video on 2% of sites. It uses Accept-Encoding: gzip and concurrency 16, 30 200 requests in total, then idles 75 s. Each sample is taken after Bun.gc(true) and gives process.memoryUsage().rss and the bun:jsc heap size.
Environment: docker run --rm --memory 1g --cpus 2 -v <worktree>:/w -w /w/apps/hosting-service oven/bun:1.4.2-alpine bun scripts/memory-bench.ts (Linux VM on an M-series Mac, Bun 1.4.2 as in production). For "before" the three scripts/memory-bench*.ts files are copied unchanged into a detached 041de81 worktree. Three runs each, alternating before/after.
| run | phase | RSS MiB | JS heap MiB | manifests resident (est. MiB) | hot items (MiB) | log/metric ring |
|---|---|---|---|---|---|---|
| before#1 | after warmup | 192.3 | 42.3 | 2000 (n/a) | 500 (3.0) | 4/6041 |
| before#1 | steady state | 242.7 | 42.9 | 2000 (n/a) | 500 (3.0) | 4/10000 |
| before#1 | idle 75s | 177.1 | 36.1 | 2000 (n/a) | 500 (3.0) | 4/10000 |
| before#2 | after warmup | 193.1 | 42.2 | 2000 (n/a) | 500 (3.0) | 4/6041 |
| before#2 | steady state | 233.7 | 42.9 | 2000 (n/a) | 500 (3.0) | 4/10000 |
| before#2 | idle 75s | 173.0 | 36.0 | 2000 (n/a) | 500 (3.0) | 4/10000 |
| before#3 | after warmup | 192.5 | 42.3 | 2000 (n/a) | 500 (3.0) | 4/6041 |
| before#3 | steady state | 244.9 | 42.9 | 2000 (n/a) | 500 (3.0) | 4/10000 |
| before#3 | idle 75s | 172.1 | 36.0 | 2000 (n/a) | 500 (3.0) | 4/10000 |
After
| run | phase | RSS MiB | JS heap MiB | manifests resident (est. MiB) | hot items (MiB) | log/metric ring |
|---|---|---|---|---|---|---|
| after#1 | after warmup | 184.0 | 30.7 | 957 (16.0) | 500 (3.0) | 0/0 |
| after#1 | steady state | 203.2 | 30.8 | 953 (16.0) | 500 (3.0) | 0/0 |
| after#1 | idle 75s | 152.4 | 13.5 | 0 (0.0) | 500 (3.0) | 0/0 |
| after#2 | after warmup | 199.5 | 30.6 | 956 (16.0) | 500 (3.0) | 0/0 |
| after#2 | steady state | 213.4 | 30.8 | 951 (15.9) | 500 (3.0) | 0/0 |
| after#2 | idle 75s | 153.7 | 13.5 | 0 (0.0) | 500 (3.0) | 0/0 |
| after#3 | after warmup | 192.0 | 30.7 | 957 (16.0) | 500 (3.0) | 0/0 |
| after#3 | steady state | 210.3 | 30.8 | 953 (16.0) | 500 (3.0) | 0/0 |
| after#3 | idle 75s | 132.0 | 13.6 | 0 (0.0) | 500 (3.0) | 0/0 |
Summary (median of 3, MiB):
| phase | metric | before | after | Δ |
|---|---|---|---|---|
| after warmup | JS heap | 42.3 (42.2–42.3) | 30.7 (30.6–30.7) | -11.6 |
| after warmup | RSS | 192.5 (192.3–193.1) | 192.0 (184.0–199.5) | -0.5 |
| steady state | JS heap | 42.9 (42.9–42.9) | 30.8 (30.8–30.8) | -12.1 |
| steady state | RSS | 242.7 (233.7–244.9) | 210.3 (203.2–213.4) | -32.4 |
| idle 75s | JS heap | 36.0 (36.0–36.1) | 13.5 (13.5–13.6) | -22.5 |
| idle 75s | RSS | 173.0 (172.1–177.1) | 152.4 (132.0–153.7) | -20.6 |
The idle drop of about 21 MiB of JS heap matches what the bench's 2000 manifests measure on their own (21.9 MiB), plus the two rings. RSS is far noisier than heap: about 100 MiB of it is allocator retention from streaming the large files, which this PR does not touch.
Tests after rebase (bun test --isolate, REDIS_URL= S3_BUCKET= PRIVATE_S3_BUCKET=): hosting-service 349 pass / 3 skip / 0 fail; tiered-storage 192 pass; observability 14 pass / 2 skip. Root bun test --isolate: 1813 pass / 89 skip / 0 fail. New tests that fail on 041de81: retention.test.ts (2), TieredStorage.borrow.test.ts (2 of 3; the copy-by-default test passes on both), storage.test.ts "releases expired entries on the next write…" and the 32MB default, and cache-manager.test.ts estimator + "evicts the oldest manifests once their estimated bytes exceed MANIFEST_CACHE_SIZE". bun run check reports only the existing blog/src/pages/rss.xml.ts astro error, which 041de81 has too. Biome is clean on the changed files.
Not measured / caveats
- Production effect is unmeasured: staging is down. Edge cgroup memory, Caddy latency and hit rates after a deploy are still to check.
- The bench uses disk-source mode. In production, cold is S3 behind a warm disk tier, so absolute RSS differs. On the previous base, steady-state heap was inflated by queued
.metaaccess-stat rewrites. #29 stopped updating access stats on cold reads, and steady-state heap is now flat in both columns, so that backlog no longer shows here. Warm-tier hits in production still rewrite.metaon every read. - At 16 MiB the bench keeps ~955 of 2000 manifests, so the rest are re-queried. Production's manifest working set per 10 s window is unknown. Edges with more RAM should raise
MANIFEST_CACHE_SIZE(andHOT_CACHE_SIZE), and nothing indeploy/sets either yet. - Hot tier vs streamed promotion (#29). Streamed warm/cold hits now promote into hot when placement allows hot (HTML, CSS, JS) and the object is at most
streamHotPromotionMaxBytes(256 KiB, hosting's default). With a 32 MiB / 500-item hot tier, 256 KiB is 1/128 of the byte cap. The count cap binds first while the average promoted object is ≤ 64 KiB, and the byte cap binds above that. No single streamed promotion can evict more than 256 KiB, and both promotion paths write throughTTLMemoryTier.set, so expired entries are freed on the same pass. Measured with the same bench and digest (single runs,--idle 0):- default config: hot settles at 500 items / 3.0 MiB, up from 1.5 MiB before #29, so it is count-bound.
HOT_CACHE_COUNT=5000: the bench's whole promotable set is 2061 items / 6.4 MiB on both041de81and this branch, so 32 MiB is far from binding here.HOT_CACHE_COUNT=5000 HOT_CACHE_SIZE=4194304: hot holds 975 items / 4.0 MiB, so the byte cap is enforced on the streamed-promotion path, and the response digest is unchanged (8c31749101555b17).- Buffered reads (prewarm, on-demand rewrite,
_redirects) still promote objects of any size up to the hot cap, e.g. the bench's 2–4 MiB HTML pages. 32 MiB keeps those to a handful of entries. A per-object cap for buffered promotion would be a separate change. - The hit-rate effect of 100→32 MiB in production is unmeasured. Expired hot entries are released on the next write, not on a timer.
- The
borrowDatawin is transient garbage (one object-sized copy per buffered read). GC'd snapshots cannot show it, and it is covered by tests rather than by the bench.