personal memory agent
Add dynamic context registry for tier/label/group discovery
Replace static CONTEXT_DEFAULTS with dynamic discovery system that reads
tier/label/group from category and agent JSON configs at runtime.
- Add tier/label/group field support to observe/categories/*.json
- Add tier/label/group field support to muse/agents/*.json and apps/*/agents/*.json
- Add _discover_agent_contexts() to scan agent JSONs for metadata
- Add _build_context_registry() to merge static defaults with discovered configs
- Add get_context_registry() with lazy initialization and caching
- Update _resolve_tier() and resolve_provider() to use dynamic registry
- Add app-specific groups: "Todos" for todos agents, "Entities" for entities agents
- Add tests for registry discovery and structure validation
- Update docs/PROVIDERS.md with dynamic discovery documentation
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
+271
-24
+2
-1
apps/entities/agents/entities.json
+2
-1
apps/entities/agents/entities.json
+2
-1
apps/entities/agents/entities_review.json
+2
-1
apps/entities/agents/entities_review.json
+2
-1
apps/entities/agents/entity_assist.json
+2
-1
apps/entities/agents/entity_assist.json
+2
-1
apps/entities/agents/entity_describe.json
+2
-1
apps/entities/agents/entity_describe.json
+2
-1
apps/todos/agents/review.json
+2
-1
apps/todos/agents/review.json
+2
-1
apps/todos/agents/todo.json
+2
-1
apps/todos/agents/todo.json
+2
-1
apps/todos/agents/weekly.json
+2
-1
apps/todos/agents/weekly.json
+10
-2
docs/PROVIDERS.md
+10
-2
docs/PROVIDERS.md
···
234
234
Context strings determine provider and model selection. Providers receive already-resolved models, but understanding the system helps:
235
235
236
236
**Context naming convention:** `{module}.{feature}[.{operation}]`
237
-
- Examples: `observe.enrich`, `insight.meetings`, `app.chat.title`
237
+
238
+
**Dynamic discovery:** Categories and agents can express their own tier/label/group in their JSON configs:
239
+
- Categories: `observe/categories/*.json` - add `tier`, `label`, `group` fields
240
+
- System agents: `muse/agents/*.json` - add `tier`, `label`, `group` fields
241
+
- App agents: `apps/*/agents/*.json` - add `tier`, `label`, `group` fields
242
+
243
+
These are discovered at runtime and merged with static defaults. Use `get_context_registry()` to get the complete context map including discovered entries.
244
+
245
+
See `CONTEXT_DEFAULTS` in `muse/models.py` for static context patterns (non-discoverable contexts like `observe.detect.*`, `insight.*`).
238
246
239
247
**Resolution** (handled by `muse/models.py` `resolve_provider()`):
240
248
1. Exact match in journal.json `providers.contexts`
241
249
2. Glob pattern match (fnmatch) with specificity ranking
242
-
3. Built-in `CONTEXT_DEFAULTS` in `muse/models.py`
250
+
3. Dynamic context registry (static defaults + discovered categories/agents)
243
251
4. Default provider/tier from config
244
252
245
253
Providers don't implement routing - they receive the resolved model.
+148
-14
muse/models.py
+148
-14
muse/models.py
···
81
81
# - insight.* -> insight module, all features (wildcard)
82
82
# - app.chat.title -> apps module, chat app, title operation
83
83
#
84
+
# DYNAMIC DISCOVERY:
85
+
# Categories (observe/categories/*.json) and agents (muse/agents/*.json,
86
+
# apps/*/agents/*.json) can express tier/label/group in their JSON configs.
87
+
# These are discovered at runtime and merged with the static defaults below.
88
+
#
84
89
# When adding new contexts:
85
90
# 1. Use module prefix matching the package (observe, think, app, muse)
86
91
# 2. Add specific operations as suffixes when granular control is needed
87
92
# 3. Use wildcards sparingly - prefer explicit entries for clarity
88
93
# 4. If not listed here, context falls back to DEFAULT_TIER (FLASH)
94
+
# 5. For categories/agents, prefer adding tier/label/group to JSON configs
89
95
# ---------------------------------------------------------------------------
90
96
97
+
# Static context defaults - non-discoverable contexts only
98
+
# Categories and agents express their own tier/label/group in JSON configs
91
99
CONTEXT_DEFAULTS: Dict[str, Dict[str, Any]] = {
92
100
# Observe pipeline - screen and audio capture processing
93
101
"observe.describe.frame": {
···
95
103
"label": "Screen Categorization",
96
104
"group": "Observe",
97
105
},
106
+
# Fallback for categories without explicit tier in their JSON
98
107
"observe.describe.*": {
99
108
"tier": TIER_FLASH,
100
109
"label": "Screen Transcription",
···
143
152
"label": "Chat Title Generation",
144
153
"group": "Apps",
145
154
},
146
-
# Agent runs - AI agent execution via Cortex
155
+
# Fallback for agents without explicit tier in their JSON
147
156
"agent.*": {
148
157
"tier": TIER_FLASH,
149
-
"label": "Agent Runs",
150
-
"group": "Muse",
158
+
"label": "Other Agents",
159
+
"group": "Agents",
151
160
},
152
161
}
153
162
154
163
164
+
# ---------------------------------------------------------------------------
165
+
# Dynamic context discovery
166
+
# ---------------------------------------------------------------------------
167
+
168
+
# Cached context registry (built lazily on first use)
169
+
_context_registry: Optional[Dict[str, Dict[str, Any]]] = None
170
+
171
+
172
+
def _discover_agent_contexts() -> Dict[str, Dict[str, Any]]:
173
+
"""Discover agent context defaults from JSON config files.
174
+
175
+
Scans system agents (muse/agents/*.json) and app agents (apps/*/agents/*.json)
176
+
for tier/label/group metadata. This is a lightweight scan that only reads
177
+
the JSON metadata, not the full agent configuration.
178
+
179
+
Returns
180
+
-------
181
+
Dict[str, Dict[str, Any]]
182
+
Mapping of context patterns to {tier, label, group} dicts.
183
+
Context patterns are: agent.system.{name} or agent.{app}.{name}
184
+
"""
185
+
contexts = {}
186
+
187
+
# System agents from muse/agents/
188
+
agents_dir = Path(__file__).parent / "agents"
189
+
if agents_dir.exists():
190
+
for json_path in agents_dir.glob("*.json"):
191
+
agent_name = json_path.stem
192
+
try:
193
+
with open(json_path, "r", encoding="utf-8") as f:
194
+
config = json.load(f)
195
+
196
+
context = f"agent.system.{agent_name}"
197
+
contexts[context] = {
198
+
"tier": config.get("tier", TIER_FLASH),
199
+
"label": config.get("label", config.get("title", agent_name)),
200
+
"group": config.get("group", "Agents"),
201
+
}
202
+
except Exception:
203
+
pass # Skip agents that can't be loaded
204
+
205
+
# App agents from apps/*/agents/
206
+
apps_dir = Path(__file__).parent.parent / "apps"
207
+
if apps_dir.is_dir():
208
+
for app_path in apps_dir.iterdir():
209
+
if not app_path.is_dir() or app_path.name.startswith("_"):
210
+
continue
211
+
agents_subdir = app_path / "agents"
212
+
if not agents_subdir.is_dir():
213
+
continue
214
+
app_name = app_path.name
215
+
for json_path in agents_subdir.glob("*.json"):
216
+
agent_name = json_path.stem
217
+
try:
218
+
with open(json_path, "r", encoding="utf-8") as f:
219
+
config = json.load(f)
220
+
221
+
context = f"agent.{app_name}.{agent_name}"
222
+
contexts[context] = {
223
+
"tier": config.get("tier", TIER_FLASH),
224
+
"label": config.get("label", config.get("title", agent_name)),
225
+
"group": config.get("group", "Agents"),
226
+
}
227
+
except Exception:
228
+
pass # Skip agents that can't be loaded
229
+
230
+
return contexts
231
+
232
+
233
+
def _build_context_registry() -> Dict[str, Dict[str, Any]]:
234
+
"""Build complete context registry from static defaults and discovered configs.
235
+
236
+
Merges:
237
+
1. Static CONTEXT_DEFAULTS (non-discoverable contexts)
238
+
2. Category contexts from observe/describe.py CATEGORIES
239
+
3. Agent contexts from _discover_agent_contexts()
240
+
241
+
Returns
242
+
-------
243
+
Dict[str, Dict[str, Any]]
244
+
Complete context registry mapping patterns to {tier, label, group}.
245
+
"""
246
+
# Start with static defaults
247
+
registry = dict(CONTEXT_DEFAULTS)
248
+
249
+
# Merge category contexts (lazy import to avoid circular dependency)
250
+
try:
251
+
from observe.describe import CATEGORIES
252
+
253
+
for category, metadata in CATEGORIES.items():
254
+
context = metadata.get("context", f"observe.describe.{category}")
255
+
registry[context] = {
256
+
"tier": metadata.get("tier", TIER_FLASH),
257
+
"label": metadata.get("label", category.replace("_", " ").title()),
258
+
"group": metadata.get("group", "Screen Analysis"),
259
+
}
260
+
except ImportError:
261
+
pass # observe module not available
262
+
263
+
# Merge agent contexts
264
+
agent_contexts = _discover_agent_contexts()
265
+
registry.update(agent_contexts)
266
+
267
+
return registry
268
+
269
+
270
+
def get_context_registry() -> Dict[str, Dict[str, Any]]:
271
+
"""Get the complete context registry, building it lazily on first use.
272
+
273
+
Returns
274
+
-------
275
+
Dict[str, Dict[str, Any]]
276
+
Complete context registry mapping patterns to {tier, label, group}.
277
+
"""
278
+
global _context_registry
279
+
if _context_registry is None:
280
+
_context_registry = _build_context_registry()
281
+
return _context_registry
282
+
283
+
155
284
def _resolve_tier(context: str) -> int:
156
285
"""Resolve context to tier number.
157
286
158
-
Checks journal config contexts first, then CONTEXT_DEFAULTS with glob matching.
287
+
Checks journal config contexts first, then dynamic context registry with glob matching.
159
288
160
289
Parameters
161
290
----------
···
167
296
int
168
297
Tier number (1=pro, 2=flash, 3=lite).
169
298
"""
170
-
import fnmatch
171
-
172
299
from think.utils import get_config
173
300
174
301
journal_config = get_config()
175
302
providers_config = journal_config.get("providers", {})
176
303
contexts = providers_config.get("contexts", {})
304
+
305
+
# Get dynamic context registry (includes static defaults + discovered categories/agents)
306
+
registry = get_context_registry()
177
307
178
308
# Check journal config contexts first (exact match)
179
309
if context in contexts:
180
310
return contexts[context].get("tier", DEFAULT_TIER)
181
311
182
-
# Check CONTEXT_DEFAULTS (exact match)
183
-
if context in CONTEXT_DEFAULTS:
184
-
return CONTEXT_DEFAULTS[context]["tier"]
312
+
# Check context registry (exact match)
313
+
if context in registry:
314
+
return registry[context]["tier"]
185
315
186
316
# Check glob patterns in both
187
317
for pattern, ctx_config in contexts.items():
188
318
if fnmatch.fnmatch(context, pattern):
189
319
return ctx_config.get("tier", DEFAULT_TIER)
190
320
191
-
for pattern, ctx_default in CONTEXT_DEFAULTS.items():
321
+
for pattern, ctx_default in registry.items():
192
322
if fnmatch.fnmatch(context, pattern):
193
323
return ctx_default["tier"]
194
324
···
330
460
matches.sort(key=lambda x: x[0], reverse=True)
331
461
_, _, match_config = matches[0]
332
462
333
-
# No context match - check CONTEXT_DEFAULTS for this context
463
+
# No context match - check dynamic context registry for this context
334
464
if match_config is None:
465
+
# Get dynamic context registry (includes static defaults + discovered categories/agents)
466
+
registry = get_context_registry()
467
+
335
468
# Check for matching context default (exact match first, then glob)
336
469
context_tier = None
337
470
if context:
338
-
if context in CONTEXT_DEFAULTS:
339
-
context_tier = CONTEXT_DEFAULTS[context]["tier"]
471
+
if context in registry:
472
+
context_tier = registry[context]["tier"]
340
473
else:
341
474
# Check glob patterns
342
475
matches = []
343
-
for pattern, ctx_default in CONTEXT_DEFAULTS.items():
476
+
for pattern, ctx_default in registry.items():
344
477
if fnmatch.fnmatch(context, pattern):
345
478
specificity = len(pattern.split("*")[0])
346
479
matches.append((specificity, ctx_default["tier"]))
···
794
927
"DEFAULT_TIER",
795
928
"DEFAULT_PROVIDER",
796
929
"CONTEXT_DEFAULTS",
930
+
"get_context_registry",
797
931
# Model constants (used by muse backends for defaults)
798
932
"GEMINI_FLASH",
799
933
"GPT_5",
+12
-1
observe/describe.py
+12
-1
observe/describe.py
···
53
53
- description (required): Single-line description for categorization prompt
54
54
- followup (optional, default: false): Whether to run follow-up analysis
55
55
- output (optional, default: "markdown"): Response format if followup=true
56
+
- tier (optional, default: 2): Model tier for this category (1=pro, 2=flash, 3=lite)
57
+
- label (optional): Human-readable name for settings UI
58
+
- group (optional, default: "Screen Analysis"): Category for grouping in settings UI
56
59
57
60
If followup=true, a matching .txt file contains the follow-up prompt.
58
61
···
79
82
logger.warning(f"Category {category} missing 'description' field")
80
83
continue
81
84
82
-
# Apply defaults
85
+
# Apply defaults for observation settings
83
86
metadata.setdefault("followup", False)
84
87
metadata.setdefault("output", "markdown")
88
+
89
+
# Apply defaults for tier routing
90
+
# tier: 1=pro, 2=flash, 3=lite (default: flash)
91
+
metadata.setdefault("tier", 2)
92
+
# label: Human-readable name (default: title-cased category name)
93
+
metadata.setdefault("label", category.replace("_", " ").title())
94
+
# group: Settings UI grouping (default: Screen Analysis)
95
+
metadata.setdefault("group", "Screen Analysis")
85
96
86
97
# Store the category context for later resolution
87
98
# The model will be resolved at runtime via generate()
+87
tests/test_models.py
+87
tests/test_models.py
···
23
23
TIER_LITE,
24
24
TIER_PRO,
25
25
calc_token_cost,
26
+
get_context_registry,
26
27
resolve_provider,
27
28
)
28
29
···
354
355
provider, model = resolve_provider("test.string")
355
356
assert provider == "google"
356
357
assert model == GEMINI_FLASH
358
+
359
+
360
+
# ---------------------------------------------------------------------------
361
+
# Dynamic context registry tests
362
+
# ---------------------------------------------------------------------------
363
+
364
+
365
+
def test_context_registry_includes_static_defaults():
366
+
"""Test that registry includes all static CONTEXT_DEFAULTS entries."""
367
+
registry = get_context_registry()
368
+
369
+
# All static defaults should be in registry
370
+
for context in CONTEXT_DEFAULTS:
371
+
assert context in registry, f"Static default {context} not in registry"
372
+
assert registry[context]["tier"] == CONTEXT_DEFAULTS[context]["tier"]
373
+
374
+
375
+
def test_context_registry_includes_categories():
376
+
"""Test that registry includes discovered category contexts."""
377
+
registry = get_context_registry()
378
+
379
+
# Should have category entries (from observe/categories/*.json)
380
+
category_contexts = [k for k in registry if k.startswith("observe.describe.")]
381
+
382
+
# Should have more than just the static entries (frame and wildcard)
383
+
assert len(category_contexts) > 2, "Should discover category contexts"
384
+
385
+
# Each category context should have required fields
386
+
for context in category_contexts:
387
+
if context == "observe.describe.*":
388
+
continue # Skip wildcard
389
+
assert "tier" in registry[context]
390
+
assert "label" in registry[context]
391
+
assert "group" in registry[context]
392
+
assert registry[context]["tier"] in (TIER_PRO, TIER_FLASH, TIER_LITE)
393
+
394
+
395
+
def test_context_registry_includes_agents():
396
+
"""Test that registry includes discovered agent contexts."""
397
+
registry = get_context_registry()
398
+
399
+
# Should have agent entries (from muse/agents/*.json and apps/*/agents/*.json)
400
+
agent_contexts = [k for k in registry if k.startswith("agent.")]
401
+
402
+
# Should have more than just the fallback pattern
403
+
assert len(agent_contexts) > 1, "Should discover agent contexts"
404
+
405
+
# Should have system agents
406
+
system_agents = [k for k in agent_contexts if k.startswith("agent.system.")]
407
+
assert len(system_agents) > 0, "Should discover system agents"
408
+
409
+
# Should have app agents
410
+
app_agents = [
411
+
k
412
+
for k in agent_contexts
413
+
if k.startswith("agent.")
414
+
and not k.startswith("agent.system.")
415
+
and k != "agent.*"
416
+
]
417
+
assert len(app_agents) > 0, "Should discover app agents"
418
+
419
+
420
+
def test_context_registry_structure():
421
+
"""Test that all registry entries have required fields."""
422
+
registry = get_context_registry()
423
+
required_keys = {"tier", "label", "group"}
424
+
425
+
for context, config in registry.items():
426
+
assert isinstance(config, dict), f"{context} should be a dict"
427
+
assert required_keys <= set(
428
+
config.keys()
429
+
), f"{context} missing keys: {required_keys - set(config.keys())}"
430
+
assert config["tier"] in (
431
+
TIER_PRO,
432
+
TIER_FLASH,
433
+
TIER_LITE,
434
+
), f"{context} has invalid tier: {config['tier']}"
435
+
436
+
437
+
def test_context_registry_is_cached():
438
+
"""Test that registry is built once and cached."""
439
+
registry1 = get_context_registry()
440
+
registry2 = get_context_registry()
441
+
442
+
# Should return the same object (cached)
443
+
assert registry1 is registry2