There are a few guiding principles behind this implementation:
- Processes that cannot be separated onto different hosts or containers should be combined into one coherent process.
- Ability to easily and continuously back up data is paramount.
- Models and processes that exist for historical reasons rather than essential reasons should be dropped.
- Full compatibility with Tangled's appview must be retained.
As such, I've made a few decisions about how Mnemosyne will be built:
- All knot state along with Git data will be stored in Postgres. We'll have to implement traits for Gitoxide to allow for this, but ideally this isn't a super hard blocker. Delta compression may end up being more difficult, but I think that's largely an implementation detail that we can address after we get end to end working.
- The result will be a single shipped binary. We'll have multiple tasks sharing the same sqlx pool for whatever things need to happen in parallel.
- Because everything is in Postgres, we'll ideally have native capability for WAL-G. This means we can have continuous journaled backups to e.g. B2 (this should probably be configurable per knot instance) and I'd like the knot to provide backup health monitoring at a simple check level. We'll probably also ship a standalone cronjob backup drill kinda thing you can implement to verify your production backups actually work.
- We'll need to look into what work has already been fleshed out re: repo migrations. Ideally this is something we can accommodate in the implementation from the start.
- The ability to load balance between knot replicas becomes available with the move to host all data in Postgres; we should take advantage of that.