this repo has no description
docs: add multi-source support design
Design for adding source metadata (type, network, channel, user ID,
display name) to links, quotes, and images. Enables per-source
duplicate detection and API filtering for multi-community support.
+208
+208
docs/plans/2026-02-13-multi-source-support-design.md
+208
docs/plans/2026-02-13-multi-source-support-design.md
···
1
+
# Multi-Source Support
2
+
3
+
**Date:** 2026-02-13
4
+
**Status:** Proposed
5
+
6
+
## Problem
7
+
8
+
Tumble currently serves a single community with no tracking of where
9
+
posts originate. As usage expands to multiple IRC channels, Slack
10
+
teams, and potentially Discord servers, posts need metadata about
11
+
their source. This enables per-source duplicate detection, per-source
12
+
filtering, and eventually source-scoped UI views.
13
+
14
+
## Solution
15
+
16
+
Add source metadata columns to all three content tables (links,
17
+
quotes, images). Duplicate detection becomes scoped per source.
18
+
API endpoints accept optional source fields on creation and support
19
+
filtering by source on reads. No UI changes in this phase.
20
+
21
+
## Data Model
22
+
23
+
Five nullable columns added to `ircLink`, `quote`, and `image`:
24
+
25
+
| Column | Type | Purpose |
26
+
|--------|------|---------|
27
+
| `source_type` | VARCHAR(50) | Platform: "irc", "slack", "discord", "api", "web" |
28
+
| `source_network` | VARCHAR(255) | IRC server, Slack team ID, Discord guild ID |
29
+
| `source_channel` | VARCHAR(255) | IRC channel, Slack/Discord channel ID |
30
+
| `source_user_id` | VARCHAR(255) | Platform-specific user ID (null for IRC) |
31
+
| `source_user_name` | VARCHAR(255) | Display/mention name at time of post (null for IRC) |
32
+
33
+
Composite index on `(source_type, source_network, source_channel)`
34
+
on each table.
35
+
36
+
The existing `user` column is unchanged. It continues to store the
37
+
poster's name. The new `source_user_id` and `source_user_name` fields
38
+
provide platform-specific identity alongside it.
39
+
40
+
Platform mapping:
41
+
42
+
| Field | IRC | Slack | Discord |
43
+
|-------|-----|-------|---------|
44
+
| `source_type` | "irc" | "slack" | "discord" |
45
+
| `source_network` | server hostname | team ID | guild ID |
46
+
| `source_channel` | channel name | channel ID | channel ID |
47
+
| `source_user_id` | null | Slack user ID | Discord user ID |
48
+
| `source_user_name` | null | mention name | display name |
49
+
50
+
GORM model additions (same for all three structs):
51
+
52
+
```go
53
+
SourceType *string `json:"source_type,omitempty" gorm:"column:source_type"`
54
+
SourceNetwork *string `json:"source_network,omitempty" gorm:"column:source_network"`
55
+
SourceChannel *string `json:"source_channel,omitempty" gorm:"column:source_channel"`
56
+
SourceUserID *string `json:"source_user_id,omitempty" gorm:"column:source_user_id"`
57
+
SourceUserName *string `json:"source_user_name,omitempty" gorm:"column:source_user_name"`
58
+
```
59
+
60
+
Pointer types for correct null handling and `omitempty` serialization.
61
+
62
+
## DDL
63
+
64
+
GORM AutoMigrate handles schema changes from the struct definitions.
65
+
No manual DDL required. Columns are added automatically on next
66
+
application startup.
67
+
68
+
## Data Backfill
69
+
70
+
All existing rows predate multi-source support and originate from IRC.
71
+
A standalone SQL script (run once, manually) backfills them:
72
+
73
+
```sql
74
+
UPDATE ircLink SET source_type = 'irc', source_network = 'jameswhite.org', source_channel = '#soggies' WHERE source_type IS NULL;
75
+
UPDATE quote SET source_type = 'irc', source_network = 'jameswhite.org', source_channel = '#soggies' WHERE source_type IS NULL;
76
+
UPDATE image SET source_type = 'irc', source_network = 'jameswhite.org', source_channel = '#soggies' WHERE source_type IS NULL;
77
+
```
78
+
79
+
No backfill code in the application. No dead code paths.
80
+
81
+
## Duplicate Detection
82
+
83
+
Currently global: any POST of an existing URL returns 208. Changes to
84
+
be scoped by `source_type + source_network + source_channel`.
85
+
86
+
**Rules:**
87
+
- POST with source fields: check duplicates within that source tuple
88
+
only. Same URL from a different source is a new link (201).
89
+
- POST with null/omitted source fields: fall back to global duplicate
90
+
check (backward compatibility).
91
+
- Same URL within same source: 208 Already Reported with previous
92
+
submission info, as before.
93
+
94
+
**Example:**
95
+
1. `https://example.com` from `slack / T12345 / C67890` -> 201
96
+
2. Same URL from `irc / jameswhite.org / #soggies` -> 201
97
+
3. Same URL from `slack / T12345 / C67890` again -> 208
98
+
99
+
Images: same scoping applied to MD5 deduplication.
100
+
101
+
Quotes: no duplicate detection currently, no change.
102
+
103
+
## API Changes
104
+
105
+
### POST Endpoints
106
+
107
+
`POST /api/v1/links`, `POST /api/v1/quotes` accept new optional
108
+
fields:
109
+
110
+
```json
111
+
{
112
+
"url": "https://example.com",
113
+
"user": "stahnma",
114
+
"source_type": "slack",
115
+
"source_network": "T12345",
116
+
"source_channel": "C67890",
117
+
"source_user_id": "U99999",
118
+
"source_user_name": "stahnma"
119
+
}
120
+
```
121
+
122
+
All source fields are optional. Omitting them stores nulls.
123
+
124
+
### GET Endpoints
125
+
126
+
`GET /api/v1/links`, `GET /api/v1/quotes`, `GET /api/v1/search` gain
127
+
three optional query parameters:
128
+
129
+
- `source_type` - filter by platform
130
+
- `source_network` - filter by team/server (requires source_type)
131
+
- `source_channel` - filter by channel (requires source_type +
132
+
source_network)
133
+
134
+
Cumulative filtering. Omitting all returns everything (current
135
+
behavior).
136
+
137
+
### Response Payloads
138
+
139
+
Source fields included with `omitempty`. Null source fields are
140
+
omitted from responses:
141
+
142
+
```json
143
+
{
144
+
"id": 1234,
145
+
"url": "https://example.com",
146
+
"user": "stahnma",
147
+
"title": "Example",
148
+
"source_type": "slack",
149
+
"source_network": "T12345",
150
+
"source_channel": "C67890",
151
+
"source_user_id": "U99999",
152
+
"source_user_name": "stahnma"
153
+
}
154
+
```
155
+
156
+
## Store Interface Changes
157
+
158
+
`GetLinkByURL` becomes source-scoped. New filter struct for query
159
+
methods:
160
+
161
+
```go
162
+
type SourceFilter struct {
163
+
SourceType *string
164
+
SourceNetwork *string
165
+
SourceChannel *string
166
+
}
167
+
```
168
+
169
+
Used by `GetLinks`, `GetQuotes`, `GetImages`, and the duplicate
170
+
detection lookup. When all fields are nil, behaves identically to
171
+
current implementation.
172
+
173
+
`TimelineItem` gains source fields for future frontend use.
174
+
175
+
## Testing
176
+
177
+
- Duplicate detection: same URL + same source = 208; same URL +
178
+
different source = 201; null source = global fallback
179
+
- Filtering: source params return correct subset; params combine to
180
+
narrow results; no params returns everything
181
+
- Backfill: existing rows have correct values after running script
182
+
- Serialization: source fields omitted from JSON when null; present
183
+
when populated
184
+
- Round-trip: POST with source fields, GET returns them; POST
185
+
without, GET omits them
186
+
187
+
## API Documentation
188
+
189
+
Update `internal/assets/openapi.json` to reflect all API changes:
190
+
191
+
- Add `source_type`, `source_network`, `source_channel`,
192
+
`source_user_id`, and `source_user_name` to request schemas for
193
+
`POST /api/v1/links` and `POST /api/v1/quotes`
194
+
- Add `source_type`, `source_network`, `source_channel` as optional
195
+
query parameters on `GET /api/v1/links`, `GET /api/v1/quotes`, and
196
+
`GET /api/v1/search`
197
+
- Add source fields to all response schemas (links, quotes, images)
198
+
- Document the scoped duplicate detection behavior (208 is now
199
+
per-source when source fields are provided)
200
+
201
+
## Out of Scope
202
+
203
+
- UI changes
204
+
- Authentication/authorization per source
205
+
- User identity lookup table for name history tracking
206
+
- Slack, Discord, or other client/bot implementations
207
+
- Changes to the existing `source` request parameter (controls
208
+
response format for irc/api/html — separate concept)