hey! ran into a cluster of quota/deletion edges on atcr — we resolved ourselves, but three things seem actionable:
-
UI manifest delete returns "Server error" — trash icon on artifacts under
zat.dev/stream/cacheconsistently 500'd with an opaque "Server Error". briefly looking atDeleteManifestHandler, suspects felt likepdsClient.DeleteRecordcall or the cache-db delete; whatever it was, the toast swallows the real error text. -
over-quota accounts can't self-prune via the registry API. token endpoint denies all write scopes while over quota — including
delete:scope=repository:zat.dev/stream:pull,push,delete → 403 "quota exceeded: 5.02 GB / 5.00 GB … Delete images to free space"
so afaict the error's own instruction isn't followable through the API (and manifest DELETE is bearer-only, so no basic-auth path either)
- usage figure lags deletion we ended up self-serving the cleanup by deleting our
io.atcr.manifest/io.atcr.tagrecords directly on our PDS. but the quota number appeared to stay at 5.02/5.00 (maybe until the hold's nightly GC frees the layers), so the API stays write-locked for up to ~24h after deleting everything, unless im missing something there
how we got in this situation, in case its a useful datapoint: kaniko --cache-repo=atcr.io/zat.dev/stream/cache filled 5 GB fast (6 cache manifests + ~50 CI tags)
Thanks for the report. Based on the logs, your OAuth session with your ATProto account had expired even though the UI still showed you as logged in. That's what caused the 500 errors on delete, it couldn't reach your PDS.
That said, even if the delete had gone through, there's a bug affecting images with a slash in the name (like
stream/cache) where the delete silently succeeds without actually removing the record. You would have hit that too, just less visibly.Deleting the records directly from your PDS is generally my go-to when the UI isn't cooperating.
As for deleting from the client side: yes, those are blocked while you're over quota. It was never tested, and the assumption was that clients would send a
deletescope on its own, notpull,push,delete.Finally, quotas clear on a garbage collection run every 24 hours. That's by design for now, but open to improvement later. GC ran a couple of hours after your deletes, so you should now be under the limit, I currently see 4.7GB on my end.