personal memory agent
Expand PROVIDERS.md with complete implementation guide
Add comprehensive documentation for new provider implementers:
- Full generate()/agenerate() function signature
- Parameter details table with provider-specific nuances (vision, thinking, timeouts)
- Token usage format examples for OpenAI, Anthropic, Google
- Finish event format with usage field
- Batch processing note (automatic via agenerate)
- OpenAI-compatible providers section for DO/Azure/local LLMs
- Expanded 15-item checklist including get_model_provider(), fixtures, and doc updates
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
+126
-28
+126
-28
docs/PROVIDERS.md
+126
-28
docs/PROVIDERS.md
···
55
55
56
56
These functions handle direct LLM text generation. The unified API in `muse/models.py` routes requests to provider-specific implementations.
57
57
58
+
**Function signature:**
59
+
```python
60
+
def generate(
61
+
contents: Union[str, List[Any]],
62
+
model: str,
63
+
temperature: float = 0.3,
64
+
max_output_tokens: int = 8192 * 2,
65
+
system_instruction: Optional[str] = None,
66
+
json_output: bool = False,
67
+
thinking_budget: Optional[int] = None,
68
+
timeout_s: Optional[float] = None,
69
+
context: Optional[str] = None,
70
+
**kwargs: Any,
71
+
) -> str:
72
+
```
73
+
74
+
The `agenerate()` function has the same signature but is `async`.
75
+
76
+
**Parameter details:**
77
+
78
+
| Parameter | Notes |
79
+
|-----------|-------|
80
+
| `contents` | String, list of strings, or list with mixed content. For vision-capable providers (currently Google only), can include PIL Image objects. Other providers stringify non-text content. |
81
+
| `model` | Already resolved by routing - providers don't need to handle model selection. |
82
+
| `max_output_tokens` | Response token limit. Note: Google internally adds `thinking_budget` to this for total budget calculation. |
83
+
| `system_instruction` | System prompt. Providers handle this per their API (separate field, prepended message, etc.). |
84
+
| `json_output` | Request JSON response. Google uses `response_mime_type`, Anthropic/OpenAI use response format or system instruction. |
85
+
| `thinking_budget` | Token budget for reasoning/thinking. Only Google and Anthropic (certain models) support this - others should silently ignore. |
86
+
| `timeout_s` | Request timeout in seconds. Convert to provider's expected format (e.g., Google uses milliseconds internally). |
87
+
| `context` | For token logging attribution only - not used in generation. |
88
+
| `**kwargs` | Absorb unknown kwargs for forward compatibility. Provider-specific options (e.g., `cached_content` for Google) pass through here. |
89
+
58
90
**Key responsibilities:**
59
-
- Accept the common parameter set (see `muse/models.py` `generate()` signature)
91
+
- Accept the common parameter set shown above
60
92
- Return the response text as a string
61
93
- Call `log_token_usage()` after successful generation
62
94
- Handle provider-specific response parsing and validation
63
95
64
-
**Important notes:**
65
-
- The `model` parameter arrives already resolved - providers don't do routing
66
-
- The `context` parameter is for token logging attribution only
67
-
- Absorb unknown kwargs via `**kwargs` to maintain forward compatibility
68
-
- Provider-specific features (e.g., `cached_content` for Google) are passed through kwargs
69
-
70
-
**Handling optional features:**
71
-
72
-
Some parameters are provider-specific:
73
-
- `thinking_budget`: Supported by Google (via ThinkingConfig) and Anthropic (certain models)
74
-
- `json_output`: Google uses `response_mime_type`, Anthropic adds system instruction
75
-
- `cached_content`: Google-only content caching
76
-
77
-
Providers should gracefully ignore unsupported parameters.
96
+
**Important:** Providers should gracefully ignore unsupported parameters rather than raising errors.
78
97
79
98
## run_agent()
80
99
···
114
133
115
134
Use `JSONEventCallback` from `muse/agents.py` to wrap the callback and auto-add timestamps.
116
135
136
+
**Finish event format:**
137
+
138
+
The `finish` event must include the result text and should include usage for token tracking:
139
+
```python
140
+
callback.emit({
141
+
"event": "finish",
142
+
"result": final_text,
143
+
"usage": usage_dict, # Same format as token logging
144
+
"ts": int(time.time() * 1000),
145
+
})
146
+
```
147
+
117
148
**Error handling pattern:**
118
149
119
150
All providers must follow this pattern to prevent duplicate error reporting:
···
157
188
```python
158
189
from muse.models import log_token_usage
159
190
191
+
log_token_usage(model=model, usage=usage_dict, context=context)
192
+
```
193
+
194
+
**Usage dict format:**
195
+
196
+
Providers should build a usage dict from their response. The normalized format is:
197
+
```python
198
+
usage_dict = {
199
+
"input_tokens": 1500, # Required
200
+
"output_tokens": 500, # Required
201
+
"total_tokens": 2000, # Required
202
+
"cached_tokens": 800, # Optional: cache hits
203
+
"reasoning_tokens": 200, # Optional: thinking/reasoning tokens
204
+
}
205
+
```
206
+
207
+
Provider-specific extraction examples:
208
+
```python
209
+
# OpenAI / OpenAI-compatible
210
+
usage_dict = {
211
+
"input_tokens": response.usage.prompt_tokens,
212
+
"output_tokens": response.usage.completion_tokens,
213
+
"total_tokens": response.usage.total_tokens,
214
+
}
215
+
216
+
# Anthropic
217
+
usage_dict = {
218
+
"input_tokens": response.usage.input_tokens,
219
+
"output_tokens": response.usage.output_tokens,
220
+
"total_tokens": response.usage.input_tokens + response.usage.output_tokens,
221
+
}
222
+
223
+
# Google (can pass response object directly)
160
224
log_token_usage(model=model, usage=response, context=context)
161
225
```
162
226
163
227
**Key points:**
164
228
- Call after successful generation
165
-
- `usage` can be provider-specific format or response object - `log_token_usage()` normalizes it
229
+
- `log_token_usage()` normalizes various formats automatically
166
230
- `context` enables attribution to specific features/operations
167
-
- See `muse/models.py` `log_token_usage()` for supported formats
168
231
169
232
## Context & Routing
170
233
···
227
290
- `tests/integration/test_google_backend.py`, etc.
228
291
229
292
Run integration tests with: `make test-integration`
293
+
294
+
## Batch Processing
295
+
296
+
The `Batch` class in `muse/batch.py` automatically works with all providers via the unified `agenerate()` API. No provider-specific batch implementation is needed - just ensure your `agenerate()` works correctly.
297
+
298
+
## OpenAI-Compatible Providers
299
+
300
+
For providers with OpenAI-compatible APIs (e.g., DigitalOcean, Azure OpenAI, local LLMs), you can leverage the OpenAI SDK with a custom base URL:
301
+
302
+
```python
303
+
from openai import OpenAI
304
+
305
+
client = OpenAI(
306
+
api_key=os.getenv("MYPROVIDER_API_KEY"),
307
+
base_url="https://api.myprovider.com/v1",
308
+
)
309
+
```
310
+
311
+
This allows reusing much of the OpenAI provider's patterns for request/response handling.
230
312
231
313
## Checklist for New Providers
232
314
233
-
1. Create `muse/providers/<name>.py` with `__all__` exports
234
-
2. Implement `generate()`, `agenerate()`, `run_agent()`
235
-
3. Add model constants to `muse/models.py` (e.g., `MYPROVIDER_PRO`)
236
-
4. Add provider to `PROVIDER_DEFAULTS` in `muse/models.py`
237
-
5. Add routing cases in `muse/models.py`:
315
+
**Core implementation:**
316
+
1. Create `muse/providers/<name>.py` with `__all__ = ["generate", "agenerate", "run_agent"]`
317
+
2. Implement `generate()`, `agenerate()`, `run_agent()` following signatures above
318
+
319
+
**Model constants** in `muse/models.py`:
320
+
3. Add model constants using the pattern `{PROVIDER}_{TIER}` (e.g., `DO_LLAMA_70B`, `DO_MISTRAL_NEMO`)
321
+
- Existing examples: `GEMINI_FLASH`, `GPT_5`, `CLAUDE_SONNET_4`
322
+
4. Add provider tier mappings to `PROVIDER_DEFAULTS` dict
323
+
5. Update `get_model_provider()` to detect your models by prefix (critical for cost tracking)
324
+
325
+
**Routing:**
326
+
6. Add `elif provider == "<name>"` cases in `muse/models.py`:
238
327
- `generate()` function (around line 622)
239
328
- `agenerate()` function (around line 700)
240
-
6. Add routing case in `muse/agents.py` `main_async()`
241
-
7. Add API key to `apps/settings/routes.py` `PROVIDER_API_KEYS`
242
-
8. Add API key UI field in `apps/settings/workspace.html`
243
-
9. Create unit tests in `tests/test_<name>.py`
244
-
10. Create integration tests in `tests/integration/test_<name>_backend.py`
245
-
11. Update `muse/providers/__init__.py` docstring
329
+
7. Add routing case in `muse/agents.py` `main_async()` (around line 331)
330
+
331
+
**Settings UI:**
332
+
8. Add API key to `apps/settings/routes.py` `PROVIDER_API_KEYS` dict
333
+
9. Add API key UI field in `apps/settings/workspace.html`
334
+
335
+
**Testing:**
336
+
10. Create unit tests in `tests/test_<name>.py`
337
+
11. Create integration tests in `tests/integration/test_<name>_backend.py`
338
+
12. Add test contexts to `fixtures/journal/config/journal.json`
339
+
340
+
**Documentation:**
341
+
13. Update `muse/providers/__init__.py` docstring
342
+
14. Update `docs/MUSE.md` providers table
343
+
15. Update `docs/CORTEX.md` valid provider values