# Lure

Lure is a library for processing webhook events into LLM-consumable prompts. It
looks something like this:

1. An HTTP request is received at a path like `/webhooks/tangled`
2. Lure strips the configured base path and matches the remainder to a template
   file on disk, e.g. with base path `/webhooks`, the path `/webhooks/tangled`
   matches `./lures/tangled.lure`. Nested paths are supported: `/webhooks/github/push`
   matches `./lures/github/push.lure`. The `.lure` file is part config, part
   template (more on this later).
3. According to the config, Lure validates the webhook according to the
   specified strategy (e.g. API key or HMAC verification)
4. If validation succeeds, Lure executes some callback with the string result of
   evaluating the template contents with the webhook payload.

The goal is to trigger LLM executions in response to webhook events, but without
the requirement for Zapier/IFTTT and with as little HTTP endpoint exposure as
possible. Consumers of the Lure library provide their own HTTP server--no server
is provided by Lure.

## `.lure` file format

Lures are intended to be written by LLMs, so a `.lure` file is essentially a
Markdown file with frontmatter. Here is a contrived example:

```md
---
verify:
  hmac:
    header: X-My-Header-Signature
    prefix: "sha256=" # optional: stripped before comparing the digest
    secret: $MY_WEBHOOK_SECRET
payload:
  contentType: json
config:
  arbitrary: true
  someValue: 3
---

You have received information about a {{ payload.event }} event on My
Service. Read the following payload and respond according to your skills:

{{ payload.body }}
```

Different verification methods can be supported, for generic implementations or
vendor-specific requirements. Only one verification method can be specified per
lure.

> **Note:** A lure without a `verify` block will accept requests from any
> sender. Unverified lures should only be used on trusted internal networks;
> any publicly-exposed lure endpoint should specify a verification method. Set
> `allowUnverified: false` at handler creation time to reject unverified lures
> at startup.

### Template scope

Templates are evaluated using [Liquid](https://liquidjs.com). The following
variables are available:

- `payload`: The request body. For `contentType: json`, this is the parsed
  JSON value.
- `headers`: The request headers as a plain object with lowercase keys
  (e.g. `{{ headers["x-my-header"] }}`).
- `query`: The query string parameters as a plain object
  (e.g. `{{ query.foo }}`).

Use `{{ expression }}` to interpolate values and `{% if %}...{% endif %}` for
conditionals.

## Usage

Use either the `@lure-hooks/fetch` or `@lure-hooks/express` packages to construct an
endpoint handler that suits your HTTP server of choice.

Both handler constructors take the following parameters:

- `basePath`: The URL path prefix under which all lure endpoints are mounted,
  e.g. `/webhooks`. Lure only handles requests whose path begins with this
  prefix; all other requests are passed through.
- `configSchema`: A Standard Schema for validating any extra config you would
  like to allow in the `config` frontmatter key
- `luresDir`: A path to a directory of lures
- `callback`: A function that you want to run in response to incoming webhooks.
  It will be called with the templated prompt `prompt` and the value of the
  `config` frontmatter value.
- `maxAttempts`: How many times to attempt the `callback` before giving up.
  Defaults to `1` (no retries). If all attempts fail, the webhook is dropped.
- `allowUnverified`: If `false`, lures without a `verify` block will be
  rejected at startup. Defaults to `true`.
- `watch`: If `true`, Lure watches `luresDir` for changes and reloads lures as
  they are added, modified, or removed. Defaults to `false`.

## Generating lures

Since `.lure` files follow a structured format, they are well-suited to be
generated by an LLM. A `create-lure` skill is available in
[SKILL.md](./SKILL.md) at the root of this repository.

## Lifecycle

### At Startup

1. The parent program creates either a fetch or an Express lure handler, as
   described above.
2. Lure traverses the specified directory and discovers any `.lure` files.
3. Each `.lure` file has their frontmatter validated. The parsed config and
   template content are cached.
4. If `watch` is enabled, a filesystem watcher is started on `luresDir`. When
   a `.lure` file is added or modified, it is re-validated and its cache entry
   updated. If validation fails, the previous cached version is retained and an
   error is logged. When a `.lure` file is removed, its cache entry is
   discarded. Changes take effect immediately — queue processing always uses the
   current cache, so a reload applies to any items already in the queue as well.

### Per Request

1. The requested path is checked against registered lure paths.
2. On a hit, we immediately return a 204 response, to keep the response
   time as low as possible.
3. Webhook requests are copied and added to an in-memory queue for processing.
   Requests in the queue will be lost if the process exits.
4. The queue processor removes requests from the queue FIFO. If verification
   fails, the request is dropped.
5. On successful verification, the lure template is evaluated using the
   request.
6. The provided `callback` is executed with the fully-formed prompt and the
   config object from the original `.lure` frontmatter. If the callback throws,
   it will be retried up to `maxAttempts` times before the webhook is dropped.