commits
Signed-off-by: Seongmin Lee <git@boltless.me>
Signed-off-by: Seongmin Lee <git@boltless.me>
Signed-off-by: Seongmin Lee <git@boltless.me>
Signed-off-by: Seongmin Lee <git@boltless.me>
Signed-off-by: Seongmin Lee <git@boltless.me>
monospace font for textarea in dev app is so common that we can just
apply it as an opt-out style for all textareas
Signed-off-by: Seongmin Lee <git@boltless.me>
Add sandboxed atmosphere environment for local testing. This new vm
contains everything required to run local test appview including PLC,
PDS, Jetstream (listening to single PDS), knot and spindle.
I'm using my custom `tngl.boltless.dev` domain which resolves to
`127.0.0.1` without any proxy.
PLC: plc.tngl.boltless.dev
PDS: pds.tngl.boltless.dev
Relay: relay.tngl.boltless.dev
Jetstream: jetstream.tngl.boltless.dev
Knot: knot.tngl.boltless.dev
Spindle: spindle.tngl.boltless.dev
TLS is supported with caddy service running inside the vm.
note: `pds.env` file here is hard copy to be used for contrib/scripts.
note: upgraded pds package in order to set email settings
Signed-off-by: Seongmin Lee <git@boltless.me>
Signed-off-by: Seongmin Lee <git@boltless.me>
Signed-off-by: Seongmin Lee <git@boltless.me>
Signed-off-by: Seongmin Lee <git@boltless.me>
Signed-off-by: Seongmin Lee <git@boltless.me>
`sh.tangled.pipeline` events are now completely generated & streamed
from spindle
Signed-off-by: Seongmin Lee <git@boltless.me>
spindle will emit `sh.tangled.pipeline` event on:
- `sh.tangled.git.refUpdate` events from knot stream
- live create/update events of `sh.tangled.repo.pull` records
Signed-off-by: Seongmin Lee <git@boltless.me>
Spindle will sync git repo when new repo is registered
Spindle will listen to `sh.tangled.git.refUpdate` event from knot
stream and sync its local git repo instead. Spindle's git repo will
sparse-checkout only `/.tangled/workflows` directory.
Spindle now requires git version >=2.49 for `--revision` flag in `git
clone` command.
References:
- <https://stackoverflow.com/q/47541033/13150270>
- <https://stackoverflow.com/q/600079/13150270>
Signed-off-by: Seongmin Lee <git@boltless.me>
This single persistent directory can be used for storing general spindle
data like db, motd file and upcoming sparse-clone git repos.
db path will be `${DATA_DIR}/spindle.db`
Signed-off-by: Seongmin Lee <git@boltless.me>
spindle-tap will collect/stream record events from:
- users dynamically added by spindle (spindle members | collaborators of
repos using spindle)
- any users with `sh.tangled.repo.pull` collection
It might be bit inefficient considering it will also stream repo
creation events from PR authors due to second rule, but at least we now
have backfill logic and Sync 1.1 based syncing.
This inefficiency can be fixed later by modifying upstream tap cli or
embedding tap into spindle.
```
+--------- all tangled users --------+
| |
| +-- users known to spindle-tap --+ |
| | (PR author / manually added) | |
| | | |
| | +----------------------------+ | |
| | | users known to spindle | | |
| | | (members / collaborators) | | |
| | +----------------------------+ | |
| +--------------------------------+ |
+------------------------------------+
```
Close: <https://tangled.org/tangled.org/core/issues/341>
Signed-off-by: Seongmin Lee <git@boltless.me>
Signed-off-by: Seongmin Lee <git@boltless.me>
- did-method-plc
- bluesky-jetstream
- bluesky-relay
- tap
Signed-off-by: Seongmin Lee <git@boltless.me>
This new db migration won't migrate existing records in repos table.
Instead, it will simply rename the legacy table to `repos_old` and
create a new one with same name.
repo backfill will be done with tap
Signed-off-by: Seongmin Lee <git@boltless.me>
We will start using our own forked version of indigo package.
Signed-off-by: Seongmin Lee <git@boltless.me>
create new one if it's missing
Signed-off-by: Seongmin Lee <git@boltless.me>
This commit won't work without following spindle rewrite to use tap and
introduce backfill because repos table is empty yet.
Signed-off-by: Seongmin Lee <git@boltless.me>
1. Use repo AT-URI as identifier.
2. Use `dom` field rather than `obj` to filter by repository. So now
it's "user with role A in repo B can do action D to field C" where
`A,B,C,D` are `sub,dom,obj,act`.
3. Manage app-logic rules in embedded csv file which won't be saved in
db and load to memory on start. This makes app's global rbac rule
change easier as we just need to edit the csv file.
Many permission check methods are missing, but should be enough to test
this new RBAC enforcer package in spindle.
Related issue: <https://tangled.org/tangled.org/core/issues/282>
Signed-off-by: Seongmin Lee <git@boltless.me>
After refactoring record deletion logic, we only need
`db.GetReactionStatus`
Signed-off-by: Seongmin Lee <git@boltless.me>
- upsert public key to handle record update event
- don't delete by pair of name and key. delete by name or rkey instead.
Signed-off-by: Seongmin Lee <git@boltless.me>
Most service flow will be:
1. start db transaction
2. run db operation
3. run PDS operation
4. rollback db if anything above failed
5. commit transaction
If PDS operation succeed, don't try rollback anymore. The ingester will
backfill the missed db operations.
Signed-off-by: Seongmin Lee <git@boltless.me>
Signed-off-by: Seongmin Lee <git@boltless.me>
migrate tables: `stars`, `reactions`, `follows`, `public_keys`
Two major changes:
1. Remove autoincrement id for these tables.
AUTOINCREMENT primary key does not help much for these tables and only
introduces slice performance overhead. Use default `rowid` with
non-autoincrement integer instead.
2. Remove unique constraints other than `(did, rkey)`
We cannot block users creating non-unique atproto records. Appview needs
to handle those properly. For example, if user unstar a repo, appview
should delete all existing star records pointing to that repo.
To allow this, remove all constraints other than `(did, rkey)`.
Minor changes done while migrating tables:
- rename `thread_at` in `reactions` to `subject_at` to match with other
tables
- follow common column names like `did` and `created`
- allow self-follow (similar reason to 2nd major change. we should block
it from service layer instead)
Signed-off-by: Seongmin Lee <git@boltless.me>
Appview cannot modify the user-owned record on repository deletion
Signed-off-by: Seongmin Lee <git@boltless.me>
Signed-off-by: Seongmin Lee <git@boltless.me>
- RBAC should be enforced on service logic.
- We should not check for referenced records existence from db due to
the nature of atproto.
- Comment depth validation is not necessary. We can accept them and just
don't render replies with deeper depth.
Move markdown sanitizer to dedicated package to avoid import cycle
Signed-off-by: Seongmin Lee <git@boltless.me>
Move blobPattern, treePattern, and pathAfterRefRE to package-level
vars so they are compiled once and reused across GetRepoInfo and
path resolution calls instead of recompiling on every request.
Signed-off-by: Matías Insaurralde <matias@insaurral.de>
the learn-more button in the features set caused an overflow on home
page.
Signed-off-by: oppiliappan <me@oppi.li>
Signed-off-by: marco.tngl.sh <dev@marco.social>
Signed-off-by: marco.tngl.sh <dev@marco.social>
Signed-off-by: marco.tngl.sh <dev@marco.social>
Signed-off-by: marco.tngl.sh <dev@marco.social>
Add {ctrl,cmd}+Enter shortcut to submit issues.
See https://tangled.org/tangled.org/core/issues/412
Signed-off-by: marco.tngl.sh <dev@marco.social>
When reviewing a pull request with multiple files, the currently visible
file is now highlighted (bolded) in the filetree sidebar. This makes it
easier to track your position when scrolling through large diffs and
lose track of the header of the file you're looking at.
The highlight updates on scroll and also responds to clicks on filetree
links.
Signed-off-by: Jes Olson <j3s@c3f.net>
Add MermaidJS diagram rendering for markdown content (READMEs, issues,
PRs) using goldmark-mermaid with client-side rendering. Mermaid fenced
code blocks are transformed to <pre class="mermaid"> elements, which
MermaidJS picks up and renders in the browser.
Changes:
- Add go.abhg.dev/goldmark/mermaid dependency with client-side render mode
- Add mermaid extender before highlighting in goldmark pipeline
- Allow "mermaid" class on <pre> elements through bluemonday sanitizer
- Conditionally load MermaidJS from CDN only on pages with diagrams
- Add basic mermaid container styling with dark mode support
- Add tests verifying mermaid blocks produce correct HTML output
Closes https://tangled.org/tangled.org/core/issues/424
Signed-off-by: Nate Spilman <nate.spilman@gmail.com>
Signed-off-by: Seongmin Lee <git@boltless.me>
Signed-off-by: Jes Olson <j3s@c3f.net>
We seem to be hammering bsky's PDS, which explains the 429s. Perhaps
re-using the session alleviates the pressure. It's hard to test this
locally since their rate limits vanish (different host/IP).
Signed-off-by: Anirudh Oppiliappan <anirudh@tangled.org>
We're hitting 429s due to high load. This allows the appview to bypass
otherwise normal rate limits on the PDS.
Signed-off-by: Anirudh Oppiliappan <anirudh@tangled.org>
Signed-off-by: oppiliappan <me@oppi.li>
We have a wildcard record in place now. Should've started with this...
Signed-off-by: Anirudh Oppiliappan <anirudh@tangled.org>
Signed-off-by: Anirudh Oppiliappan <anirudh@tangled.org>
Signed-off-by: oppiliappan <me@oppi.li>
Signed-off-by: oppiliappan <me@oppi.li>
Signed-off-by: oppiliappan <me@oppi.li>
Signed-off-by: oppiliappan <me@oppi.li>
fetches recent tangled posts for use on landing page
Signed-off-by: oppiliappan <me@oppi.li>
Signed-off-by: Seongmin Lee <git@boltless.me>
Add sandboxed atmosphere environment for local testing. This new vm
contains everything required to run local test appview including PLC,
PDS, Jetstream (listening to single PDS), knot and spindle.
I'm using my custom `tngl.boltless.dev` domain which resolves to
`127.0.0.1` without any proxy.
PLC: plc.tngl.boltless.dev
PDS: pds.tngl.boltless.dev
Relay: relay.tngl.boltless.dev
Jetstream: jetstream.tngl.boltless.dev
Knot: knot.tngl.boltless.dev
Spindle: spindle.tngl.boltless.dev
TLS is supported with caddy service running inside the vm.
note: `pds.env` file here is hard copy to be used for contrib/scripts.
note: upgraded pds package in order to set email settings
Signed-off-by: Seongmin Lee <git@boltless.me>
Spindle will sync git repo when new repo is registered
Spindle will listen to `sh.tangled.git.refUpdate` event from knot
stream and sync its local git repo instead. Spindle's git repo will
sparse-checkout only `/.tangled/workflows` directory.
Spindle now requires git version >=2.49 for `--revision` flag in `git
clone` command.
References:
- <https://stackoverflow.com/q/47541033/13150270>
- <https://stackoverflow.com/q/600079/13150270>
Signed-off-by: Seongmin Lee <git@boltless.me>
spindle-tap will collect/stream record events from:
- users dynamically added by spindle (spindle members | collaborators of
repos using spindle)
- any users with `sh.tangled.repo.pull` collection
It might be bit inefficient considering it will also stream repo
creation events from PR authors due to second rule, but at least we now
have backfill logic and Sync 1.1 based syncing.
This inefficiency can be fixed later by modifying upstream tap cli or
embedding tap into spindle.
```
+--------- all tangled users --------+
| |
| +-- users known to spindle-tap --+ |
| | (PR author / manually added) | |
| | | |
| | +----------------------------+ | |
| | | users known to spindle | | |
| | | (members / collaborators) | | |
| | +----------------------------+ | |
| +--------------------------------+ |
+------------------------------------+
```
Close: <https://tangled.org/tangled.org/core/issues/341>
Signed-off-by: Seongmin Lee <git@boltless.me>
1. Use repo AT-URI as identifier.
2. Use `dom` field rather than `obj` to filter by repository. So now
it's "user with role A in repo B can do action D to field C" where
`A,B,C,D` are `sub,dom,obj,act`.
3. Manage app-logic rules in embedded csv file which won't be saved in
db and load to memory on start. This makes app's global rbac rule
change easier as we just need to edit the csv file.
Many permission check methods are missing, but should be enough to test
this new RBAC enforcer package in spindle.
Related issue: <https://tangled.org/tangled.org/core/issues/282>
Signed-off-by: Seongmin Lee <git@boltless.me>
migrate tables: `stars`, `reactions`, `follows`, `public_keys`
Two major changes:
1. Remove autoincrement id for these tables.
AUTOINCREMENT primary key does not help much for these tables and only
introduces slice performance overhead. Use default `rowid` with
non-autoincrement integer instead.
2. Remove unique constraints other than `(did, rkey)`
We cannot block users creating non-unique atproto records. Appview needs
to handle those properly. For example, if user unstar a repo, appview
should delete all existing star records pointing to that repo.
To allow this, remove all constraints other than `(did, rkey)`.
Minor changes done while migrating tables:
- rename `thread_at` in `reactions` to `subject_at` to match with other
tables
- follow common column names like `did` and `created`
- allow self-follow (similar reason to 2nd major change. we should block
it from service layer instead)
Signed-off-by: Seongmin Lee <git@boltless.me>
- RBAC should be enforced on service logic.
- We should not check for referenced records existence from db due to
the nature of atproto.
- Comment depth validation is not necessary. We can accept them and just
don't render replies with deeper depth.
Move markdown sanitizer to dedicated package to avoid import cycle
Signed-off-by: Seongmin Lee <git@boltless.me>
When reviewing a pull request with multiple files, the currently visible
file is now highlighted (bolded) in the filetree sidebar. This makes it
easier to track your position when scrolling through large diffs and
lose track of the header of the file you're looking at.
The highlight updates on scroll and also responds to clicks on filetree
links.
Signed-off-by: Jes Olson <j3s@c3f.net>
Add MermaidJS diagram rendering for markdown content (READMEs, issues,
PRs) using goldmark-mermaid with client-side rendering. Mermaid fenced
code blocks are transformed to <pre class="mermaid"> elements, which
MermaidJS picks up and renders in the browser.
Changes:
- Add go.abhg.dev/goldmark/mermaid dependency with client-side render mode
- Add mermaid extender before highlighting in goldmark pipeline
- Allow "mermaid" class on <pre> elements through bluemonday sanitizer
- Conditionally load MermaidJS from CDN only on pages with diagrams
- Add basic mermaid container styling with dark mode support
- Add tests verifying mermaid blocks produce correct HTML output
Closes https://tangled.org/tangled.org/core/issues/424
Signed-off-by: Nate Spilman <nate.spilman@gmail.com>
Signed-off-by: Seongmin Lee <git@boltless.me>