···11+MIT License
22+33+Copyright (c) 2026 Michael Stahnke, et al
44+55+Permission is hereby granted, free of charge, to any person obtaining a copy
66+of this software and associated documentation files (the "Software"), to deal
77+in the Software without restriction, including without limitation the rights
88+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
99+copies of the Software, and to permit persons to whom the Software is
1010+furnished to do so, subject to the following conditions:
1111+1212+The above copyright notice and this permission notice shall be included in all
1313+copies or substantial portions of the Software.
1414+1515+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
1616+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
1717+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
1818+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
1919+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
2020+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
2121+SOFTWARE.
+119-316
README.md
···4455## History
6677-Tumble was a "Wouldn't it be cool?" project handed to [Scott Schnedier](https://github.com/sschneid) back in 2004. The idea was to create a website similar to a tumbleblog. Obviously, eventually tumblr ccame along and the rest was history.
77+Tumble was a "Wouldn't it be cool?" project handed to [Scott Schnedier](https://github.com/sschneid) back in 2004. The idea was to create a website similar to a tumbleblog. Eventually tumblr came along and the rest was history.
8899-## Deployment
99+The project was originally written in Perl and has since been rewritten in Go.
10101111-Run
1111+# Developer Guide
12121313-`make build`
1313+## Quick Start
14141515-Have a configuration file.
1515+This project uses [Flox](https://flox.dev) to manage its development environment. The Flox environment provides Go, make, jq, SQLite, ImageMagick, and libxml2.
16161717-See the systemd unit files in the `contrib` directory for more information.
1717+```bash
1818+# Activate the development environment
1919+flox activate
18202121+# Build
2222+make build
19232020-## Database Support
2424+# Configure (edit to match your environment)
2525+cp conf/config.yaml.sqlite conf/config.yaml
2626+vi conf/config.yaml
21272222-Tumble now supports both **MySQL** and **SQLite** databases. Choose the one that fits your needs:
2828+# Run
2929+make restart
3030+```
23312424-- **SQLite**: Recommended for development, testing, and small deployments. No separate database server required.
2525-- **MySQL**: Recommended for production deployments with higher traffic.
3232+Tumble will automatically create database tables on first startup.
26332727-## Development Workflows
3434+## Configuration
28352929-### Database Migrations
3030-3131-Tumble uses **GORM AutoMigrate** to manage database schemas. Migrations are automatically applied on application startup.
3232-3333-- **Mechanism**: The application checks the database schema on startup and creates/updates tables to match the Go structs in `internal/data/store.go`.
3636+Tumble uses [Viper](https://github.com/spf13/viper) for configuration via YAML files, environment variables, or defaults.
34373535-### Testing Infrastructure
3838+**Configuration search paths:** `conf/`, current directory (`.`)
36393737-A robust test infrastructure is available for rapid development:
3838-3939-- **Run Tests**: `make test-api` runs the integration tests.
4040-- **Test Database**: `make test-db` creates a fresh, disposable SQLite database (`tumble-test.sqlite`), runs migrations, and loads sample fixtures.
4141-- **Run with Test DB**: `make run-test` starts the application using the test database.
4242-- **Backup/Restore**: `make backup` and `make restore` allow you to snapshot your current development database.
4343-4444-To customize the test environment, you can edit `conf/config-test.yaml`.
4545-4646-### Manual MySQL Verification
4747-4848-To validate MySQL migration support without Docker, you can run against a local MySQL server:
4949-5050-1. **Create Local Database/User**:
5151-5252- ```bash
5353- mysql -u root -p -e "CREATE DATABASE tumble_test;"
5454- mysql -u root -p -e "CREATE USER 'tumble'@'localhost' IDENTIFIED BY 'password';"
5555- mysql -u root -p -e "GRANT ALL PRIVILEGES ON tumble_test.* TO 'tumble'@'localhost';"
5656- ```
5757-5858-2. **Configure**: Check `conf/config-test-mysql.yaml` matches your local credentials.
5959-6060-3. **Run Application**:
6161-6262- ```bash
6363- bin/tumble conf/config-test-mysql.yaml
6464- ```
6565-6666-4. **Load Fixtures**:
6767- ```bash
6868- export DRIVER=mysql
6969- export MYSQL_PASSWORD=password
7070- ./tests/load_fixtures.sh
7171- ```
7272-7373-## Quick Setup
7474-7575-### 1. Configure Database and Application
7676-7777-Tumble uses **Viper** for configuration, allowing you to configure the application using a file (`config.yaml`), environment variables, or default values.
7878-7979-**Configuration Search Paths:**
8080-8181-- `conf/`
8282-- `htdocs/`
8383-- Current directory (`.`)
8484-8585-**Configuration File (config.yaml):**
8686-8787-**For SQLite (default):**
4040+### SQLite (recommended for development)
88418942```yaml
9043driver: sqlite
···9447request_timeout: 2s
9548```
96499797-**For MySQL:**
5050+### MySQL
98519952```yaml
10053driver: mysql
···10659baseurl: your.domain.com
10760```
10861109109-**Environment Variables:**
6262+### Environment Variables
11063111111-You can override any configuration value using environment variables prefixed with `TUMBLE_`. Use underscores (`_`) to access nested keys.
6464+Override any config value with the `TUMBLE_` prefix. Use underscores for nested keys.
11265113113-- `TUMBLE_PORT=9090`
114114-- `TUMBLE_DRIVER=mysql`
115115-- `TUMBLE_DATABASE=production_db`
116116-- `TUMBLE_MODE=development` (Options: `development`, `production`. Default: `production`)
117117-- `TUMBLE_EMBED_ASSETS=true` (Options: `true`, `false`. Default: `true`)
118118-- `TUMBLE_LOGGING_LEVEL=debug`
119119-- `TUMBLE_REQUEST_TIMEOUT=2s` (Default: `2s`)
120120-- `TUMBLE_CLICK_SIGNING_KEY=your-secret` (Optional, enables signed click tracking)
6666+| Variable | Description | Default |
6767+|----------|-------------|---------|
6868+| `TUMBLE_PORT` | Listen port | `8080` |
6969+| `TUMBLE_DRIVER` | Database driver (`sqlite`, `mysql`) | `mysql` |
7070+| `TUMBLE_DATABASE` | Database name or file path | |
7171+| `TUMBLE_MODE` | `development` or `production` | `production` |
7272+| `TUMBLE_EMBED_ASSETS` | Load assets from binary or filesystem | `true` |
7373+| `TUMBLE_LOGGING_LEVEL` | Log level (`debug`, `info`, `warn`, `error`) | `info` |
7474+| `TUMBLE_REQUEST_TIMEOUT` | HTTP request timeout | `2s` |
7575+| `TUMBLE_ADMIN_SECRET` | Secret for admin API operations | |
7676+| `TUMBLE_CLICK_SIGNING_KEY` | Secret for signed click tracking | |
1217712278### Environment Modes (`TUMBLE_MODE`)
12379124124-- **development**:
125125- - **Logging**: Text format, Debug level, Full SQL query logging.
126126- - **Errors**: Displays detailed error messages in the browser.
127127-- **production** (default):
128128- - **Logging**: JSON format, Info level, Error-only SQL logging.
129129- - **Errors**: Displays generic "Internal Server Error" message to users.
8080+- **development**: Text-format logging, debug level, full SQL query logging, detailed error messages in the browser.
8181+- **production** (default): JSON-format logging, info level, error-only SQL logging, generic error messages to users.
1308213183### Asset Embedding (`TUMBLE_EMBED_ASSETS`)
13284133133-Controls whether templates and static assets are loaded from the embedded binary or from the filesystem.
8585+Controls whether templates and static assets are loaded from the compiled binary or the filesystem.
13486135135-- **true** (default): Assets are loaded from the compiled binary. This is the recommended setting for deployment - you only need the binary and a config file.
136136-- **false**: Assets are loaded from the filesystem (`internal/templates/views/` and `internal/assets/`). Templates are re-parsed on every request, enabling hot-reload during development.
8787+- **true** (default): Assets are loaded from the binary. Only the binary and a config file are needed for deployment.
8888+- **false**: Assets are loaded from `internal/templates/views/` and `internal/assets/`. Templates are re-parsed on every request, enabling hot-reload during development.
13789138138-This setting is independent of `TUMBLE_MODE`, allowing you to run in development mode (for verbose logging and detailed errors) while still using embedded assets for deployment.
9090+This setting is independent of `TUMBLE_MODE`.
13991140140-**Deployment Example:**
9292+## Database Support
14193142142-```yaml
143143-mode: development # Get detailed error messages and text logging
144144-embed_assets: true # But still use embedded assets (no source checkout needed)
145145-```
9494+Tumble supports both **MySQL** and **SQLite** databases.
14695147147-### 2. Initialize Database
9696+- **SQLite**: Recommended for development, testing, and small deployments. No separate database server required.
9797+- **MySQL**: Recommended for production deployments with higher traffic.
14898149149-Simply start the application! Tumble will automatically detect your database type (configured in `config.yaml` or via env vars) and create the necessary tables if they don't exist.
9999+### Migrations
150100151151-### 3. Configure Web Server
101101+Tumble uses **GORM AutoMigrate** to manage database schemas. Migrations are automatically applied on application startup based on the Go structs in `internal/data/store.go`.
152102153153-1. Update `/etc/httpd/conf.d/tumble.conf` with your server configuration
154154-2. Disable SELinux or set proper context
155155-3. Start httpd:
103103+### MySQL Setup
156104157157-```bash
158158-chkconfig httpd on
159159-service httpd start
160160-```
161161-162162-## Detailed Setup (MySQL)
163163-164164-If you prefer manual MySQL setup:
105105+To set up a MySQL database manually:
165106166107```bash
167167-# Install MySQL
168168-yum install mysql-server
169169-service mysqld start
170170-chkconfig mysqld on
171171-172172-# Create database and user
173173-mysql -u root -p < sql/sql_setup
174174-175175-# Run schema
176176-mysql -u tumble -p tumble < sql/schema.mysql
108108+mysql -u root -p < sql/user_and_database_setup.mysql
177109```
178110179179-## Migration from MySQL to SQLite
111111+Then configure `conf/config.yaml` with your MySQL credentials.
180112181181-1. Export your MySQL data:
113113+## Development
182114183183- ```bash
184184- mysqldump -u tumble -p tumble > tumble_backup.sql
185185- ```
115115+### Makefile Targets
186116187187-2. Update `config.yaml` to use SQLite
188188-189189-3. Run setup script:
190190-191191- ```bash
192192- perl scripts/setup_database.pl
193193- ```
194194-195195-4. Import data (requires conversion from MySQL to SQLite format)
196196-197197-See [docs/database_setup.md](docs/database_setup.md) for detailed instructions.
198198-199199-## Debugging and Logging
117117+| Command | Description |
118118+|---------|-------------|
119119+| `make build` | Build the binary |
120120+| `make restart` | Build and restart the application |
121121+| `make kill` | Stop the running application |
122122+| `make test` | Run unit tests |
123123+| `make test-api` | Run API integration tests |
124124+| `make test-db` | Create a fresh test database with fixtures |
125125+| `make run-test` | Run the application with the test database |
126126+| `make backup` | Backup the current SQLite database |
127127+| `make restore` | Restore the SQLite database from backup |
128128+| `make fmt` | Run `go fmt` and clean up whitespace |
129129+| `make build-linux` | Cross-compile for Linux amd64 |
130130+| `make help` | Show all available targets |
200131201201-Tumble provides comprehensive database logging to help troubleshoot connection issues and diagnose problems.
132132+### Testing
202133203203-### Automatic Database Diagnostics
134134+- `make test-api` runs integration tests against a test instance.
135135+- `make test-db` creates a fresh SQLite test database with sample fixtures.
136136+- `make run-test` starts the application using `conf/config-test.yaml`.
204137205205-When the application starts, it automatically:
206206-207207-- **Logs connection attempts** with database details (host, database name, file paths)
208208-- **Verifies database health** by checking for expected tables (`ircLink`, `image`, `quote`)
209209-- **Reports table statistics** including row counts for each table
210210-- **Warns about issues** such as:
211211- - Missing database files (SQLite)
212212- - Empty databases (no tables)
213213- - Missing expected tables
214214- - Empty tables (no data)
215215- - File permission problems (SQLite)
216216- - Connection failures with troubleshooting suggestions
217217-218218-### Debug Mode
219219-220220-For verbose logging of all database operations, enable debug mode:
138138+To test against MySQL, edit `conf/config-test-mysql.yaml` and run:
221139222140```bash
223223-export TUMBLE_DEBUG=1
141141+bin/tumble conf/config-test-mysql.yaml
224142```
225143226226-With debug mode enabled, you'll see:
144144+### Logging
227145228228-- All SQL queries being executed
229229-- Row counts for each query result
230230-- Detailed query execution information
146146+Set `TUMBLE_MODE=development` or `TUMBLE_LOGGING_LEVEL=debug` for verbose output including SQL queries. Logs are written to `tumble.log` by default.
231147232232-**Example:**
148148+## Deployment
233149234234-```bash
235235-# Enable debug mode
236236-export TUMBLE_DEBUG=1
150150+See the systemd unit files in the `contrib/` directory for production deployment examples.
237151238238-# Start your web server or run the application
239239-perl -I htdocs/lib htdocs/index.cgi
240240-```
152152+The recommended approach:
241153242242-### Log Output Examples
154154+1. Build the binary with `make build` (or `make build-linux` for Linux targets).
155155+2. Create a `config.yaml` with your production settings.
156156+3. Run the binary: `bin/tumble config.yaml`
243157244244-**Successful MySQL connection:**
158158+With `embed_assets: true` (the default), only the binary and config file are needed on the server.
245159246246-```
247247-[MySQL] Attempting to connect to database 'tumble' on host 'localhost' as user 'tumble'
248248-[MySQL] Connection SUCCESSFUL
249249-[MySQL] Database contains 3 table(s)
250250-[MySQL] Tables: ircLink, image, quote
251251-```
160160+The production systemd units in `contrib/` use Flox to manage the MySQL service (`tumble-db.service` runs `flox activate --start-services`). See `contrib/README` for details.
252161253253-**SQLite with missing database file:**
162162+## API
254163255255-```
256256-[SQLite] Attempting to connect to database file: tumble.db
257257-[SQLite] Database file DOES NOT EXIST
258258-[SQLite] - SQLite will create a new empty database file
259259-[SQLite] - You will need to run the database setup script to create tables
260260-[SQLite] Connection SUCCESSFUL
261261-[SQLite] WARNING: Database appears to be empty (no tables found)
262262-[SQLite] - You need to run the database setup script
263263-```
164164+Full API documentation is available at `/api/docs` on a running instance, generated from `internal/assets/openapi.json`.
264165265265-**Connection failure with diagnostics:**
166166+### Link Submission
266167267267-```
268268-[MySQL] Connection FAILED: Access denied for user 'tumble'@'localhost'
269269-[MySQL] Diagnostics:
270270-[MySQL] - DSN: dbi:mysql:tumble;host=localhost
271271-[MySQL] - Username: tumble
272272-[MySQL] Troubleshooting suggestions:
273273-[MySQL] 1. Verify MySQL server is running: systemctl status mysql
274274-[MySQL] 2. Check credentials in config.yaml are correct
275275-[MySQL] 3. Verify user has permissions: GRANT ALL ON tumble.* TO 'tumble'@'localhost'
276276-```
168168+Submit links via `POST /link/`. Duplicate detection is automatic:
277169278278-### API Features
279279-280280-#### Link Submission with Duplicate Detection
281281-282282-When submitting links via `/link/`, the API automatically detects if a URL has been previously posted and provides contextual information:
283283-284284-- **Behavior**: Links are always added to the database, even if duplicates exist
285170- **JSON API** (`Accept: application/json` or `source=api`):
286286- - **201 Created**: New link (first time posted)
287287- - **208 Already Reported**: Duplicate detected, includes details of all previous submissions
171171+ - **201 Created**: New link
172172+ - **208 Already Reported**: Duplicate, includes previous submission details
288173- **IRC Source** (`source=irc`): Returns ID with duplicate marker if applicable
289289- - New: `"123"`
290290- - Duplicate: `"123 (duplicate, previously posted by alice)"`
291291-- **HTML Response**: Shows duplicate notification with original poster and timestamp
292292-293293-**Example JSON Response (Duplicate):**
294294-295295-```json
296296-{
297297- "link_id": 456,
298298- "is_duplicate": true,
299299- "previous_submissions": [
300300- {
301301- "link_id": 123,
302302- "user": "alice",
303303- "timestamp": "2026-01-15T10:30:00Z",
304304- "title": "Example Page"
305305- }
306306- ]
307307-}
308308-```
174174+- **HTML**: Shows duplicate notification with original poster and timestamp
309175310310-For complete API documentation, visit `/docs` on your running instance or see `internal/assets/openapi.json`.
176176+The legacy `/irclink/` endpoint is still supported but `/link/` is preferred.
311177312312-> **Note:** The legacy `/irclink/` endpoint is still supported for backwards compatibility but `/link/` is preferred.
178178+### Link and Quote Deletion
313179314314-#### Link Deletion
315315-316316-Links can be deleted via the API using the `DELETE` method on `/link/123` (where `123` is the link ID). This requires authentication using an admin secret.
317317-318318-**Configuration:**
319319-320320-Add an `admin_secret` to your `config.yaml`:
321321-322322-```yaml
323323-admin_secret: "your-random-secret-string"
324324-```
325325-326326-You can also set it via environment variable: `TUMBLE_ADMIN_SECRET=your-secret`
327327-328328-**Usage:**
180180+Delete links or quotes via `DELETE /link/{id}` or `DELETE /quote/{id}`. Requires authentication:
329181330182```bash
331331-# Using X-Admin-Secret header (recommended)
183183+# Using header (recommended)
332184curl -X DELETE -H "X-Admin-Secret: your-secret" https://your-server/link/123
333185334186# Using query parameter
335187curl -X DELETE "https://your-server/link/123?secret=your-secret"
336188```
337189338338-**Responses:**
339339-340340-- **200 OK**: Link deleted successfully
341341-- **400 Bad Request**: Missing or invalid ID
342342-- **403 Forbidden**: Missing or invalid admin secret
343343-- **404 Not Found**: Link does not exist
344344-345345-If no `admin_secret` is configured, deletion falls back to localhost-only access for backwards compatibility.
346346-347347-#### Quote Deletion
348348-349349-Quotes can be deleted via the API using the `DELETE` method on `/quote/123` (where `123` is the quote ID). This uses the same authentication mechanism as link deletion.
350350-351351-**Usage:**
352352-353353-```bash
354354-# Using X-Admin-Secret header (recommended)
355355-curl -X DELETE -H "X-Admin-Secret: your-secret" https://your-server/quote/123
190190+Configure `admin_secret` in your config file or via `TUMBLE_ADMIN_SECRET`. Without it, deletion falls back to localhost-only access.
356191357357-# Using query parameter
358358-curl -X DELETE "https://your-server/quote/123?secret=your-secret"
359359-```
192192+### Click Signature Tracking
360193361361-**Responses:**
194194+Optional HMAC-signed URLs for verified click tracking. When `click_signing_key` is configured, links render as `/link/123?sig=abc123...` and the server validates signatures to distinguish real clicks from bots.
362195363363-- **200 OK**: Quote deleted successfully
364364-- **400 Bad Request**: Missing or invalid ID
365365-- **403 Forbidden**: Missing or invalid admin secret
366366-- **404 Not Found**: Quote does not exist
196196+### Dead Link Detection
367197368368-#### Click Signature Tracking
198198+Tumble automatically detects dead links (4xx/5xx responses) and checks the [Wayback Machine](https://archive.org) for archived snapshots. A background job runs on startup and daily. Archive.org requests are rate-limited to 5 req/s.
369199370370-Tumble supports signed URLs for verified click tracking. When enabled, links include an HMAC signature that validates clicks came from the rendered page rather than bots or direct URL access.
200200+### Caching
371201372372-**Configuration:**
202202+Link previews are cached in the database. Disable with `caching.enabled: false` in config. Invalidate manually via `GET /api/caching/invalidate?url=<encoded_url>`.
373203374374-Add a `click_signing_key` to your `config.yaml`:
204204+## Project Structure
375205376376-```yaml
377377-click_signing_key: "your-random-secret-string"
378206```
379379-380380-Or set via environment variable: `TUMBLE_CLICK_SIGNING_KEY=your-secret`
381381-382382-**How it works:**
383383-384384-- When configured, links render as `/link/123?sig=abc123...` instead of `/link/123`
385385-- The signature is an HMAC-SHA256 hash of the link ID using your secret key
386386-- On redirect, the server validates the signature to distinguish verified clicks from unverified access
387387-- This helps track genuine user engagement vs. crawler/bot traffic
388388-389389-**Note:** If no `click_signing_key` is configured, links work normally without signatures. This feature is optional and doesn't affect basic functionality.
390390-391391-#### Dead Link Detection and Archive.org Integration
392392-393393-With ~20 years of links, many URLs have gone dead over time. Tumble
394394-automatically detects dead links and checks the [Wayback Machine](https://archive.org)
395395-for archived snapshots.
396396-397397-- **Detection**: Links returning 4xx/5xx errors are flagged with an HTTP
398398- status badge and excluded from search results.
399399-- **Archive.org lookups**: A background job checks dead links against the
400400- Wayback Machine Availability API on startup and daily. Newly detected
401401- dead links are also checked on-demand.
402402-- **UI**: When an archived snapshot exists, a "View on Archive.org" link
403403- appears next to the error badge with the snapshot date, giving users
404404- access to the original content. Works in both the default theme and
405405- Scott Mode.
406406-- **Rate limiting**: Archive.org requests are limited to 5 req/s to stay
407407- well within API limits.
408408-409409-#### Caching
410410-411411-Link previews are cached in the database to reduce external requests.
207207+cmd/tumble/ Entry point
208208+internal/
209209+ assets/ Static assets and OpenAPI spec
210210+ config/ Configuration loading
211211+ data/ Database models and queries
212212+ handler/ HTTP handlers
213213+ templates/ HTML templates
214214+ service/ Business logic
215215+ scheduler/ Background jobs
216216+ archive/ Archive.org integration
217217+conf/ Configuration files
218218+contrib/ Systemd unit files
219219+sql/ SQL setup scripts
220220+tests/ Integration tests and fixtures
221221+```
412222413413-- `caching.enabled`: Set to `false` to disable server-side caching.
414414-- To invalidate a cache entry manually:
415415- `GET /api/caching/invalidate?url=<encoded_url>`
223223+## License
416224417417-## Bugs
418418-419419- * fix user-agent being hardy for link verification
420420- * abstract quantity of items to be in 'hot shit' category
421421- * Fix odd encoding bugs for web site titles
422422- * Probably lots of others, but it has been in production for 10 years.
225225+This project is licensed under the MIT License. See the [LICENSE](LICENSE) file for details.