my website at https://anirudh.fi
Building forlater post
+110
+110
pages/blog/building-forlater.md
+110
pages/blog/building-forlater.md
···
1
+
---
2
+
template:
3
+
slug: building-forlater
4
+
title: How I built forlater.email
5
+
subtitle: A technical breakdown of my first big side-project
6
+
date: 2021-09-25
7
+
---
8
+
9
+
Ever since I began browsing sites like Hacker News and Lobsters, coming
10
+
across new and exciting links to check out every day, I found it hard to
11
+
keep up. On most days, I just didn't. And that's fine -- [good,
12
+
even](/blog/dont-news). But oftentimes, I'd come across a genuinely
13
+
interesting link but no time to actually read it.
14
+
15
+
I began using Pocket. It was alright -- the article view was very good;
16
+
but it stopped there. I didn't like nor use the other junk baked into
17
+
the app: discover, following/friends thing, etc. It's also proprietary,
18
+
and that irked me -- more so than the other "features".
19
+
20
+
Thus, somewhat inspired by rss2email, I began building
21
+
[forlater.email](https://forlater.email) -- a bookmarking/read-later
22
+
service that works via email. Email is the perfect tool for this
23
+
use-case: works offline; you can organize it however you like; you own
24
+
your data.
25
+
26
+
27
+
28
+
Pictured above is how forlater works. Each component is explained below.
29
+
30
+
## OpenSMTPD
31
+
32
+
Mail containing links to be saved arrive here. OpenSMTPD is beautiful
33
+
software, and its configuration is stupid simple
34
+
([smtpd.conf(5)](https://man.openbsd.org/smtpd.conf):
35
+
36
+
```conf
37
+
table blocklist file:/etc/smtpd/blocklist
38
+
39
+
action webhook mda "/home/icy/forlater/mdawh/mdawh"
40
+
match mail-from <blocklist> for any reject
41
+
match from any for rcpt-to "save@forlater.email" action webhook
42
+
```
43
+
44
+
The `filter` and `listen` directives have been snipped for brevity. The
45
+
rest, in essence, simply sends all mail to `save@forlater.email` to an
46
+
MDA program, via stdin. Any mail from an address in the blocklist file
47
+
get rejected.
48
+
49
+
[rspamd](https://rspamd.com) is used to prevent spam.
50
+
51
+
## mdawh
52
+
53
+
[mdawh](https://git.icyphox.sh/forlater/mdawh), or the MDA webhook tool.
54
+
A small Go program that processes mail coming from stdin and generates a
55
+
JSON payload that looks like so:
56
+
57
+
```json
58
+
{
59
+
"from": "foo@bar.com",
60
+
"date": "Fri, 1 Jan 2010 00:00:00 UTC",
61
+
"replyto": "...",
62
+
"body": "...",
63
+
"parts": {
64
+
"text/plain": "...",
65
+
"text/html": "...",
66
+
}
67
+
}
68
+
```
69
+
70
+
This is POSTed to a configured HTTP endpoint -- which in this case, is
71
+
navani.
72
+
73
+
## navani
74
+
75
+
[navani](https://git.icyphox.sh/forlater/navani) is forlater's primary
76
+
mail processing service[^1]. Listens for webhooks from mdawh, processes
77
+
them, and sends mail using a configured SMTP server. URLs are cached in
78
+
Redis along with the HTML content.
79
+
80
+
For the readable HTML,
81
+
[go-readability](https://github.com/go-shiori/go-readability) is used;
82
+
the output of which is rendered into a minimal [HTML email
83
+
template](https://git.icyphox.sh/forlater/navani/tree/templates/html.tpl)
84
+
-- something that I never want to write again.
85
+
86
+
The plaintext part is currently generated using `lynx -image_links -dump
87
+
-stdin`. The `-image_links` flag is handy because it generates footnote
88
+
links for images as well, instead of simply ignoring images altogether.
89
+
I plan to rewrite this; possibly using a blend of HTML-to-plaintext
90
+
libraries and handwritten rules.
91
+
92
+
## future improvements
93
+
94
+
I plan to implement some kind of `settings@` address to configure and
95
+
store user settings (dark theme? fonts?). However, this introduces state
96
+
in an otherwise mostly stateless system.
97
+
98
+
The other thing I've been thinking of is making your own newsletter of
99
+
sorts. For example: save a bunch of links during the week, and have them
100
+
all delivered over the weekend.
101
+
102
+
Neither of these "features" are confirmed to happen, primarily because
103
+
forlater is feature-complete for my use. That said, I'm happy to
104
+
consider any improvements or suggestions that you might have -- please
105
+
[shoot them my way](mailto:x@icyphox.sh).
106
+
107
+
Finally, thanks to everyone who tossed a few bucks my way -- mighty kind
108
+
of you.
109
+
110
+
[^1]: Named after [Navani Kholin](https://coppermind.net/wiki/Navani_Kholin).