Enable LLMs to handle webhooks with plaintext files
1# Lure
2
3Lure is a library for processing webhook events into LLM-consumable prompts. It
4looks something like this:
5
61. An HTTP request is received at a path like `/webhooks/tangled`
72. Lure strips the configured base path and matches the remainder to a template
8 file on disk, e.g. with base path `/webhooks`, the path `/webhooks/tangled`
9 matches `./lures/tangled.lure`. Nested paths are supported: `/webhooks/github/push`
10 matches `./lures/github/push.lure`. The `.lure` file is part config, part
11 template (more on this later).
123. According to the config, Lure validates the webhook according to the
13 specified strategy (e.g. API key or HMAC verification)
144. If validation succeeds, Lure executes some callback with the string result of
15 evaluating the template contents with the webhook payload.
16
17The goal is to trigger LLM executions in response to webhook events, but without
18the requirement for Zapier/IFTTT and with as little HTTP endpoint exposure as
19possible. Consumers of the Lure library provide their own HTTP server--no server
20is provided by Lure.
21
22## `.lure` file format
23
24Lures are intended to be written by LLMs, so a `.lure` file is essentially a
25Markdown file with frontmatter. Here is a contrived example:
26
27```md
28---
29verify:
30 hmac:
31 location: header
32 name: X-My-Header-Signature
33 secret: $MY_WEBHOOK_SECRET
34payload:
35 contentType: json
36config:
37 arbitrary: true
38 someValue: 3
39---
40
41You have received information about a {{ payload.event }} event on My
42Service. Read the following payload and respond according to your skills:
43
44{{ payload.body }}
45```
46
47Different verification methods can be supported, for generic implementations or
48vendor-specific requirements. Only one verification method can be specified per
49lure.
50
51> **Note:** A lure without a `verify` block will accept requests from any
52> sender. Unverified lures should only be used on trusted internal networks;
53> any publicly-exposed lure endpoint should specify a verification method. Set
54> `allowUnverified: false` at handler creation time to reject unverified lures
55> at startup.
56
57### Template scope
58
59Templates are evaluated using [Liquid](https://liquidjs.com). The following
60variables are available:
61
62- `payload`: The request body. For `contentType: json`, this is the parsed
63 JSON value.
64- `headers`: The request headers as a plain object with lowercase keys
65 (e.g. `{{ headers["x-my-header"] }}`).
66- `query`: The query string parameters as a plain object
67 (e.g. `{{ query.foo }}`).
68
69Use `{{ expression }}` to interpolate values and `{% if %}...{% endif %}` for
70conditionals.
71
72## Usage
73
74Use either the `@lure/fetch` or `@lure/express` packages to construct an
75endpoint handler that suits your HTTP server of choice.
76
77Both handler constructors take the following parameters:
78
79- `basePath`: The URL path prefix under which all lure endpoints are mounted,
80 e.g. `/webhooks`. Lure only handles requests whose path begins with this
81 prefix; all other requests are passed through.
82- `configSchema`: A Standard Schema for validating any extra config you would
83 like to allow in the `config` frontmatter key
84- `luresDir`: A path to a directory of lures
85- `callback`: A function that you want to run in response to incoming webhooks.
86 It will be called with the templated prompt `prompt` and the value of the
87 `config` frontmatter value.
88- `maxAttempts`: How many times to attempt the `callback` before giving up.
89 Defaults to `1` (no retries). If all attempts fail, the webhook is dropped.
90- `allowUnverified`: If `false`, lures without a `verify` block will be
91 rejected at startup. Defaults to `true`.
92- `watch`: If `true`, Lure watches `luresDir` for changes and reloads lures as
93 they are added, modified, or removed. Defaults to `false`.
94
95## Generating lures
96
97Since `.lure` files follow a structured format, they are well-suited to be
98generated by an LLM. Copy the prompt below into any LLM conversation, replace
99the placeholder at the end with a description of your webhook, and the LLM will
100produce a ready-to-use `.lure` file.
101
102````
103You are generating a .lure file for the Lure webhook library. A .lure file is
104a Markdown file with YAML frontmatter. The filename without the .lure extension
105determines the webhook path relative to the configured base path: `push.lure`
106handles `<basePath>/push`, and `github/push.lure` handles
107`<basePath>/github/push`.
108
109## Frontmatter
110
111### `verify` (optional)
112
113Specifies how to authenticate incoming webhook requests. Omit if the provider
114does not sign requests. Only one strategy may be specified.
115
116HMAC:
117
118```yaml
119verify:
120 hmac:
121 location: header # or "query"
122 name: X-Hub-Signature-256 # header or query parameter name
123 secret: $ENV_VAR_NAME # must be an environment variable reference
124```
125
126### `payload` (optional)
127
128```yaml
129payload:
130 contentType: json # currently the only supported value
131```
132
133### `config` (optional)
134
135An arbitrary object passed as-is to the application callback. Use this for any
136application-specific values you want to associate with this lure.
137
138```yaml
139config:
140 key: value
141```
142
143## Template body
144
145Below the frontmatter is a Liquid template (https://liquidjs.com). Write it as
146a natural language prompt for the LLM that will process the webhook. The
147following variables are available:
148
149- `payload` — the request body (parsed JSON for `contentType: json`)
150- `headers` — request headers as a plain object with lowercase keys (e.g. `{{ headers["x-my-header"] }}`)
151- `query` — query string as a plain object (e.g. `{{ query.foo }}`)
152
153Use `{{ expression }}` to interpolate values and `{% if %}...{% endif %}` for
154conditionals.
155
156## Task
157
158Generate a .lure file for the following webhook:
159
160[DESCRIBE THE WEBHOOK SOURCE, EVENT TYPE, PAYLOAD SHAPE, VERIFICATION METHOD,
161AND WHAT THE LLM RECEIVING THE PROMPT SHOULD DO IN RESPONSE]
162````
163
164## Lifecycle
165
166### At Startup
167
1681. The parent program creates either a fetch or an Express lure handler, as
169 described above.
1702. Lure traverses the specified directory and discovers any `.lure` files.
1713. Each `.lure` file has their frontmatter validated. The parsed config and
172 template content are cached.
1734. If `watch` is enabled, a filesystem watcher is started on `luresDir`. When
174 a `.lure` file is added or modified, it is re-validated and its cache entry
175 updated. If validation fails, the previous cached version is retained and an
176 error is logged. When a `.lure` file is removed, its cache entry is
177 discarded. Changes take effect immediately — queue processing always uses the
178 current cache, so a reload applies to any items already in the queue as well.
179
180### Per Request
181
1821. The requested path is checked against registered lure paths.
1832. On a hit, we immediately return a 204 response, to keep the response
184 time as low as possible.
1853. Webhook requests are copied and added to an in-memory queue for processing.
186 Requests in the queue will be lost if the process exits.
1874. The queue processor removes requests from the queue FIFO. If verification
188 fails, the request is dropped.
1895. On successful verification, the lure template is evaluated using the
190 request.
1916. The provided `callback` is executed with the fully-formed prompt and the
192 config object from the original `.lure` frontmatter. If the callback throws,
193 it will be retried up to `maxAttempts` times before the webhook is dropped.