Enable LLMs to handle webhooks with plaintext files
1# Lure
2
3Lure is a library for processing webhook events into LLM-consumable prompts. It
4looks something like this:
5
61. An HTTP request is received at a path like `/webhooks/tangled`
72. Lure strips the configured base path and matches the remainder to a template
8 file on disk, e.g. with base path `/webhooks`, the path `/webhooks/tangled`
9 matches `./lures/tangled.lure`. Nested paths are supported: `/webhooks/github/push`
10 matches `./lures/github/push.lure`. The `.lure` file is part config, part
11 template (more on this later).
123. According to the config, Lure validates the webhook according to the
13 specified strategy (e.g. API key or HMAC verification)
144. If validation succeeds, Lure executes some callback with the string result of
15 evaluating the template contents with the webhook payload.
16
17The goal is to trigger LLM executions in response to webhook events, but without
18the requirement for Zapier/IFTTT and with as little HTTP endpoint exposure as
19possible. Consumers of the Lure library provide their own HTTP server--no server
20is provided by Lure.
21
22## `.lure` file format
23
24Lures are intended to be written by LLMs, so a `.lure` file is essentially a
25Markdown file with frontmatter. Here is a contrived example:
26
27```md
28---
29verify:
30 hmac:
31 header: X-My-Header-Signature
32 prefix: "sha256=" # optional: stripped before comparing the digest
33 secret: $MY_WEBHOOK_SECRET
34payload:
35 contentType: json
36config:
37 arbitrary: true
38 someValue: 3
39---
40
41You have received information about a {{ payload.event }} event on My
42Service. Read the following payload and respond according to your skills:
43
44{{ payload.body }}
45```
46
47Different verification methods can be supported, for generic implementations or
48vendor-specific requirements. Only one verification method can be specified per
49lure.
50
51> **Note:** A lure without a `verify` block will accept requests from any
52> sender. Unverified lures should only be used on trusted internal networks;
53> any publicly-exposed lure endpoint should specify a verification method. Set
54> `allowUnverified: false` at handler creation time to reject unverified lures
55> at startup.
56
57### Template scope
58
59Templates are evaluated using [Liquid](https://liquidjs.com). The following
60variables are available:
61
62- `payload`: The request body. For `contentType: json`, this is the parsed
63 JSON value.
64- `headers`: The request headers as a plain object with lowercase keys
65 (e.g. `{{ headers["x-my-header"] }}`).
66- `query`: The query string parameters as a plain object
67 (e.g. `{{ query.foo }}`).
68
69Use `{{ expression }}` to interpolate values and `{% if %}...{% endif %}` for
70conditionals.
71
72## Usage
73
74Use either the `@lure-hooks/fetch` or `@lure-hooks/express` packages to construct an
75endpoint handler that suits your HTTP server of choice.
76
77Both handler constructors take the following parameters:
78
79- `basePath`: The URL path prefix under which all lure endpoints are mounted,
80 e.g. `/webhooks`. Lure only handles requests whose path begins with this
81 prefix; all other requests are passed through.
82- `configSchema`: A Standard Schema for validating any extra config you would
83 like to allow in the `config` frontmatter key
84- `luresDir`: A path to a directory of lures
85- `callback`: A function that you want to run in response to incoming webhooks.
86 It will be called with the templated prompt `prompt` and the value of the
87 `config` frontmatter value.
88- `maxAttempts`: How many times to attempt the `callback` before giving up.
89 Defaults to `1` (no retries). If all attempts fail, the webhook is dropped.
90- `allowUnverified`: If `false`, lures without a `verify` block will be
91 rejected at startup. Defaults to `true`.
92- `watch`: If `true`, Lure watches `luresDir` for changes and reloads lures as
93 they are added, modified, or removed. Defaults to `false`.
94
95## Generating lures
96
97Since `.lure` files follow a structured format, they are well-suited to be
98generated by an LLM. A `create-lure` skill is available in
99[SKILL.md](./SKILL.md) at the root of this repository.
100
101## Lifecycle
102
103### At Startup
104
1051. The parent program creates either a fetch or an Express lure handler, as
106 described above.
1072. Lure traverses the specified directory and discovers any `.lure` files.
1083. Each `.lure` file has their frontmatter validated. The parsed config and
109 template content are cached.
1104. If `watch` is enabled, a filesystem watcher is started on `luresDir`. When
111 a `.lure` file is added or modified, it is re-validated and its cache entry
112 updated. If validation fails, the previous cached version is retained and an
113 error is logged. When a `.lure` file is removed, its cache entry is
114 discarded. Changes take effect immediately — queue processing always uses the
115 current cache, so a reload applies to any items already in the queue as well.
116
117### Per Request
118
1191. The requested path is checked against registered lure paths.
1202. On a hit, we immediately return a 204 response, to keep the response
121 time as low as possible.
1223. Webhook requests are copied and added to an in-memory queue for processing.
123 Requests in the queue will be lost if the process exits.
1244. The queue processor removes requests from the queue FIFO. If verification
125 fails, the request is dropped.
1265. On successful verification, the lure template is evaluated using the
127 request.
1286. The provided `callback` is executed with the fully-formed prompt and the
129 config object from the original `.lure` frontmatter. If the callback throws,
130 it will be retried up to `maxAttempts` times before the webhook is dropped.