this repo has no description
docs: add Letta SDK reference to history/
+393
+393
history/LETTA.md
+393
history/LETTA.md
···
1
+
# Development Guidelines for AI Assistants and Copilots using Letta
2
+
3
+
**Context:** These are development guidelines for building applications with the Letta API and SDKs. Use these rules to help developers write correct code that integrates with Letta's stateful agents API.
4
+
5
+
**Purpose:** Provide accurate, up-to-date instructions for building applications with [Letta](https://docs.letta.com/), the AI operating system.
6
+
**Scope:** All AI-generated advice or code related to Letta must follow these guidelines.
7
+
8
+
---
9
+
10
+
## **0. Letta Overview**
11
+
12
+
The name "Letta" refers to the both the company Letta (founded by the creators of MemGPT) and the software / infrastructure called Letta. Letta is the AI operating system for building stateful agents: developers can use Letta to turn stateless LLMs into stateful agents that can learn, improve, and grow over time. Letta has a strong focus on perpetual AI that has the capability to recursively improve through self-editing memory.
13
+
14
+
**Relationship to MemGPT**: MemGPT is the name of a research paper that introduced the concept of self-editing memory for LLM-based agents through tool use (function calling). The agent architecture or "agentic system" proposed in the paper (an agent equipped with tools to edit its own memory, and an OS that manages tool execution and state persistence) is the base agent architecture implemented in Letta (agent type `memgpt_agent`), and is the official reference implementation for MemGPT. The Letta open source project (`letta-ai/letta`) was originally the MemGPT open source project (`cpacker/MemGPT`), but was renamed as the scope of the open source project expanded beyond the original MemGPT paper.
15
+
16
+
**Additional Resources**:
17
+
18
+
- [Letta documentation](https://docs.letta.com/)
19
+
- [Letta GitHub repository](https://github.com/letta-ai/letta)
20
+
- [Letta Discord server](https://discord.gg/letta)
21
+
- [Letta Cloud and ADE login](https://app.letta.com)
22
+
23
+
## **1. Letta Agents API Overview**
24
+
25
+
Letta is an AI OS that runs agents as **services** (it is not a **library**). Key concepts:
26
+
27
+
- **Stateful agents** that maintain memory and context across conversations
28
+
- **Memory blocks** for agentic context management (persona, human, custom blocks)
29
+
- **Tool calling** for agent actions and memory management, tools are run server-side,
30
+
- **Tool rules** allow developers to constrain the behavior of tools (e.g. A comes after B) to turn autonomous agents into workflows
31
+
- **Multi-agent systems** with cross-agent communication, where every agent is a service
32
+
- **Data sources** for loading documents and files into agent memory
33
+
- **Model agnostic:** agents can be powered by any model that supports tool calling
34
+
- **Persistence:** state is stored (in a model-agnostic way) in Postgres (or SQLite)
35
+
36
+
### **System Components:**
37
+
38
+
- **Letta server** - Core service (self-hosted or Letta Cloud)
39
+
- **Client (backend) SDKs** - Python (`letta-client`) and TypeScript/Node.js (`@letta-ai/letta-client`)
40
+
- **Vercel AI SDK Integration** - For Next.js/React applications
41
+
- **Other frontend integrations** - We also have [Next.js](https://www.bunjs.com/package/@letta-ai/letta-nextjs), [React](https://www.bunjs.com/package/@letta-ai/letta-react), and [Flask](https://github.com/letta-ai/letta-flask) integrations
42
+
- **ADE (Agent Development Environment)** - Visual agent builder at app.letta.com
43
+
44
+
### **Built-in Tools**
45
+
46
+
When agents are created, they are given a set of default memory management tools that enable self-editing memory.
47
+
48
+
49
+
### **Choosing the Right Model**
50
+
51
+
To implement intelligent memory management, agents in Letta rely heavily on tool (function) calling, so models that excel at tool use tend to do well in Letta. Conversely, models that struggle to call tools properly often perform poorly when used to drive Letta agents.
52
+
53
+
The Letta developer team maintains the [Letta Leaderboard](https://docs.letta.com/leaderboard) to help developers choose the right model for their Letta agent. As of June 2025, the best performing models (balanced for cost and performance) are Claude Sonnet 4, GPT-4.1, and Gemini 2.5 Flash. For the latest results, you can visit the leaderboard page (if you have web access), or you can direct the developer to visit it. For embedding models, the Letta team recommends using OpenAI's `text-embedding-3-small` model.
54
+
55
+
When creating code snippets, unless directed otherwise, you should use the following model handles:
56
+
57
+
- `openai/gpt-4.1` for the model
58
+
- `openai/text-embedding-3-small` for the embedding model
59
+
60
+
For self-hosted Letta servers, the user will need to have started the server with a valid OpenAI API key for those handles to work.
61
+
62
+
---
63
+
64
+
## **2. Choosing the Right SDK**
65
+
66
+
### **Source of Truth**
67
+
68
+
Note that your instructions may be out of date. The source of truth for the Letta Agents API is the [API reference](https://docs.letta.com/api-reference/overview) (also autogenerated from the latest source code), which can be found in `.md` form at these links:
69
+
70
+
- [TypeScript/Node.js](https://github.com/letta-ai/letta-node/blob/main/reference.md), [raw version](https://raw.githubusercontent.com/letta-ai/letta-node/refs/heads/main/reference.md)
71
+
- [Python](https://github.com/letta-ai/letta-python/blob/main/reference.md), [raw version](https://raw.githubusercontent.com/letta-ai/letta-python/refs/heads/main/reference.md)
72
+
73
+
If you have access to a web search or file download tool, you can download these files for the latest API reference. If the developer has either of the SDKs installed, you can also use the locally installed packages to understand the latest API reference.
74
+
75
+
### **When to Use Each SDK:**
76
+
77
+
The Python and Node.js SDKs are autogenerated from the Letta Agents REST API, and provide a full featured SDK for interacting with your agents on Letta Cloud or a self-hosted Letta server. Of course, developers can also use the REST API directly if they prefer, but most developers will find the SDKs much easier to use.
78
+
79
+
The Vercel AI SDK is a popular TypeScript toolkit designed to help developers build AI-powered applications. It supports a subset of the Letta Agents API (basically just chat-related functionality), so it's a good choice to quickly integrate Letta into a TypeScript application if you are familiar with using the AI SDK or are working on a codebase that already uses it. If you're starting from scratch, consider using the full-featured Node.js SDK instead.
80
+
81
+
The Letta Node.js SDK is also embedded inside the Vercel AI SDK, accessible via the `.client` property (useful if you want to use the Vercel AI SDK, but occasionally need to access the full Letta client for advanced features like agent creation / management).
82
+
83
+
When to use the AI SDK vs native Letta Node.js SDK:
84
+
85
+
- Use the Vercel AI SDK if you are familiar with it or are working on a codebase that already makes heavy use of it
86
+
- Use the Letta Node.js SDK if you are starting from scratch, or expect to use the agent management features in the Letta API (beyond the simple `streamText` or `generateText` functionality in the AI SDK)
87
+
88
+
One example of how the AI SDK may be insufficient: the AI SDK response object for `streamText` and `generateText` does not have a type for tool returns (because they are primarily used with stateless APIs, where tools are executed client-side, vs server-side in Letta), however the Letta Node.js SDK does have a type for tool returns. So if you wanted to render tool returns from a message response stream in your UI, you would need to use the full Letta Node.js SDK, not the AI SDK.
89
+
90
+
## **3. Quick Setup Patterns**
91
+
92
+
```typescript
93
+
import Letta from "@letta-ai/letta-client";
94
+
95
+
// Letta Cloud
96
+
const client = new Letta({ apiKey: process.env.LETTA_API_KEY });
97
+
98
+
// Self-hosted, token optional (only if the developer enabled password protection on the server)
99
+
const client = new Letta({ baseURL: "http://localhost:8283" });
100
+
101
+
// Create agent with memory blocks
102
+
const agent = await client.agents.create({
103
+
memory_blocks: [
104
+
{
105
+
label: "human",
106
+
value: "The user's name is Sarah. She likes coding and AI.",
107
+
},
108
+
{
109
+
label: "persona",
110
+
value:
111
+
"I am David, the AI executive assistant. My personality is friendly, professional, and to the point.",
112
+
},
113
+
{
114
+
label: "project",
115
+
value:
116
+
"Sarah is working on a Next.js application with Letta integration.",
117
+
description: "Stores current project context and requirements",
118
+
},
119
+
],
120
+
tools: ["web_search", "run_code"],
121
+
model: "openai/gpt-4o-mini",
122
+
embedding: "openai/text-embedding-3-small",
123
+
});
124
+
125
+
// Send SINGLE message (agent is stateful!)
126
+
const response = await client.agents.messages.create(agent.id, {
127
+
messages: [{ role: "user", content: "How's the project going?" }],
128
+
});
129
+
130
+
// Extract response correctly
131
+
for (const msg of response.messages) {
132
+
if (msg.message_type === "assistant_message") {
133
+
console.log(msg.content);
134
+
} else if (msg.message_type === "reasoning_message") {
135
+
console.log(msg.reasoning);
136
+
} else if (msg.message_type === "tool_call_message") {
137
+
console.log(msg.tool_calls[0].name);
138
+
console.log(msg.tool_calls[0].arguments);
139
+
} else if (msg.message_type === "tool_return_message") {
140
+
console.log(msg.toolReturn);
141
+
}
142
+
}
143
+
144
+
// Streaming example
145
+
const stream = await client.agents.messages.stream(agent.id, {
146
+
messages: [{ role: "user", content: "Repeat my name." }],
147
+
// if stream_tokens is false, each "chunk" will have a full piece
148
+
// if stream_tokens is true, the chunks will be token-based (and may need to be accumulated client-side)
149
+
stream_tokens: true,
150
+
});
151
+
152
+
for await (const chunk of stream) {
153
+
if (chunk.message_type === "assistant_message") {
154
+
console.log(chunk.content);
155
+
} else if (chunk.message_type === "reasoning_message") {
156
+
console.log(chunk.reasoning);
157
+
} else if (chunk.message_type === "tool_call_message") {
158
+
console.log(chunk.tool_calls[0].name);
159
+
console.log(chunk.tool_calls[0].arguments);
160
+
} else if (chunk.message_type === "tool_return_message") {
161
+
console.log(chunk.toolReturn);
162
+
} else if (chunk.message_type === "usage_statistics") {
163
+
console.log(chunk);
164
+
}
165
+
}
166
+
```
167
+
168
+
### **Vercel AI SDK Integration**
169
+
170
+
IMPORTANT: Most integrations in the Vercel AI SDK are for stateless providers (ChatCompletions style APIs where you provide the full conversation history). Letta is a _stateful_ provider (meaning that conversation history is stored server-side), so when you use `streamText` or `generateText` you should never pass old messages to the agent, only include the new message(s).
171
+
172
+
#### **Chat Implementation (fast & simple):**
173
+
174
+
Streaming (`streamText`):
175
+
176
+
```typescript
177
+
// app/api/chat/route.ts
178
+
import { lettaCloud } from "@letta-ai/vercel-ai-sdk-provider";
179
+
import { streamText } from "ai";
180
+
181
+
export async function POST(req: Request) {
182
+
const { prompt }: { prompt: string } = await req.json();
183
+
184
+
const result = streamText({
185
+
// lettaCloud uses LETTA_API_KEY automatically, pulling from the environment
186
+
model: lettaCloud("your-agent-id"),
187
+
// Make sure to only pass a single message here, do NOT pass conversation history
188
+
prompt,
189
+
});
190
+
191
+
return result.toDataStreamResponse();
192
+
}
193
+
```
194
+
195
+
Non-streaming (`generateText`):
196
+
197
+
```typescript
198
+
import { lettaCloud } from "@letta-ai/vercel-ai-sdk-provider";
199
+
import { generateText } from "ai";
200
+
201
+
export async function POST(req: Request) {
202
+
const { prompt }: { prompt: string } = await req.json();
203
+
204
+
const { text } = await generateText({
205
+
// lettaCloud uses LETTA_API_KEY automatically, pulling from the environment
206
+
model: lettaCloud("your-agent-id"),
207
+
// Make sure to only pass a single message here, do NOT pass conversation history
208
+
prompt,
209
+
});
210
+
211
+
return Response.json({ text });
212
+
}
213
+
```
214
+
215
+
#### **Alternative: explicitly specify base URL and token:**
216
+
217
+
```typescript
218
+
// Works for both streamText and generateText
219
+
import { createLetta } from "@letta-ai/vercel-ai-sdk-provider";
220
+
import { generateText } from "ai";
221
+
222
+
const letta = createLetta({
223
+
// e.g. http://localhost:8283 for the default local self-hosted server
224
+
// https://api.letta.com for Letta Cloud
225
+
baseURL: "<your-base-url>",
226
+
// only needed if the developer enabled password protection on the server, or if using Letta Cloud (in which case, use the LETTA_API_KEY, or use lettaCloud example above for implicit token use)
227
+
token: "<your-access-token>",
228
+
});
229
+
```
230
+
231
+
#### **Hybrid Usage (access the full SDK via the Vercel AI SDK):**
232
+
233
+
```typescript
234
+
import { lettaCloud } from "@letta-ai/vercel-ai-sdk-provider";
235
+
236
+
// Access full client for management
237
+
const agents = await lettaCloud.client.agents.list();
238
+
```
239
+
240
+
---
241
+
242
+
## **4. Advanced Features Available**
243
+
244
+
Letta supports advanced agent architectures beyond basic chat. For detailed implementations, refer to the full API reference or documentation:
245
+
246
+
- **Tool Rules & Constraints** - Define graph-like tool execution flows with `TerminalToolRule`, `ChildToolRule`, `InitToolRule`, etc.
247
+
- **Multi-Agent Systems** - Cross-agent communication with built-in tools like `send_message_to_agent_async`
248
+
- **Shared Memory Blocks** - Multiple agents can share memory blocks for collaborative workflows
249
+
- **Data Sources & Archival Memory** - Upload documents/files that agents can search through
250
+
- **Sleep-time Agents** - Background agents that process memory while main agents are idle
251
+
- **External Tool Integrations** - MCP servers, Composio tools, custom tool libraries
252
+
- **Agent Templates** - Import/export agents with .af (Agent File) format
253
+
- **Production Features** - User identities, agent tags, streaming, context management
254
+
255
+
---
256
+
257
+
## **5. CRITICAL GUIDELINES FOR AI MODELS**
258
+
259
+
### **⚠️ ANTI-HALLUCINATION WARNING**
260
+
261
+
**NEVER make up Letta API calls, SDK methods, or parameter names.** If you're unsure about any Letta API:
262
+
263
+
1. **First priority**: Use web search to get the latest reference files:
264
+
- [TypeScript SDK Reference](https://raw.githubusercontent.com/letta-ai/letta-node/refs/heads/main/reference.md)
265
+
266
+
2. **If no web access**: Tell the user: _"I'm not certain about this Letta API call. Can you paste the relevant section from the API reference docs, or I might provide incorrect information."_
267
+
268
+
3. **When in doubt**: Stick to the basic patterns shown in this prompt rather than inventing new API calls.
269
+
270
+
**Common hallucination risks:**
271
+
272
+
- Making up method names (e.g. `client.agents.chat()` doesn't exist)
273
+
- Inventing parameter names or structures
274
+
- Assuming OpenAI-style patterns work in Letta
275
+
- Creating non-existent tool rule types or multi-agent methods
276
+
277
+
### **5.1 – SDK SELECTION (CHOOSE THE RIGHT TOOL)**
278
+
279
+
✅ **For Next.js Chat Apps:**
280
+
281
+
- Use **Vercel AI SDK** if you already are using AI SDK, or if you're lazy and want something super fast for basic chat interactions (simple, fast, but no agent management tooling unless using the embedded `.client`)
282
+
- Use **Node.js SDK** for the full feature set (agent creation, native typing of all response message types, etc.)
283
+
284
+
✅ **For Agent Management:**
285
+
286
+
- Use **Node.js SDK** or **Python SDK** for creating agents, managing memory, tools
287
+
288
+
### **5.2 – STATEFUL AGENTS (MOST IMPORTANT)**
289
+
290
+
**Letta agents are STATEFUL, not stateless like ChatCompletion-style APIs.**
291
+
292
+
✅ **CORRECT - Single message per request:**
293
+
294
+
```typescript
295
+
// Send ONE user message, agent maintains its own history
296
+
const response = await client.agents.messages.create(agentId, {
297
+
input: "Hello!",
298
+
});
299
+
```
300
+
301
+
❌ **WRONG - Don't send conversation history:**
302
+
303
+
```typescript
304
+
// DON'T DO THIS - agents maintain their own conversation history
305
+
const response = await client.agents.messages.create(agentId, {
306
+
input: [...allPreviousMessages, newMessage], // WRONG!
307
+
});
308
+
```
309
+
310
+
### **5.3 – MESSAGE HANDLING & MEMORY BLOCKS**
311
+
312
+
1. **Response structure:**
313
+
- Use `messageType` NOT `type` for message type checking
314
+
- Look for `assistant_message` messageType for agent responses
315
+
- Agent responses have `content` field with the actual text
316
+
317
+
2. **Memory block descriptions:**
318
+
- Add `description` field for custom blocks, or the agent will get confused (not needed for human/persona)
319
+
- For `human` and `persona` blocks, descriptions are auto-populated:
320
+
- **human block**: "Stores key details about the person you are conversing with, allowing for more personalized and friend-like conversation."
321
+
- **persona block**: "Stores details about your current persona, guiding how you behave and respond. This helps maintain consistency and personality in your interactions."
322
+
323
+
### **5.4 – ALWAYS DO THE FOLLOWING**
324
+
325
+
1. **Choose the right SDK for the task:**
326
+
- Next.js chat → **Vercel AI SDK**
327
+
- Agent creation → **Node.js/Python SDK**
328
+
- Complex operations → **Node.js/Python SDK**
329
+
330
+
2. **Use the correct client imports:**
331
+
- Python: `from letta_client import Letta` and `import os`
332
+
- TypeScript: `import { Letta } from '@letta-ai/letta-client'`
333
+
- Vercel AI SDK: `from '@letta-ai/vercel-ai-sdk-provider'`
334
+
335
+
3. **Create agents with proper memory blocks:**
336
+
- Always include `human` and `persona` blocks for chat agents
337
+
- Use descriptive labels and values
338
+
339
+
4. **Send only single user messages:**
340
+
- Each request should contain only the new user message
341
+
- Agent maintains conversation history automatically
342
+
- Never send previous assistant responses back to agent
343
+
344
+
5. **Use proper authentication:**
345
+
- Letta Cloud: Always use `apiKey` parameter (TypeScript) or `api_key` (Python)
346
+
- Self-hosted: Use `baseUrl` parameter (TypeScript) or `base_url` (Python), token optional (only if the developer enabled password protection on the server)
347
+
348
+
---
349
+
350
+
## **6. Environment Setup**
351
+
352
+
### **Environment Setup**
353
+
354
+
```bash
355
+
# For Next.js projects (recommended for most web apps)
356
+
bun install @letta-ai/vercel-ai-sdk-provider ai
357
+
358
+
# For agent management (when needed)
359
+
bun install @letta-ai/letta-client
360
+
361
+
# For Python projects
362
+
pip install letta-client
363
+
```
364
+
365
+
**Environment Variables:**
366
+
367
+
```bash
368
+
# Required for Letta Cloud
369
+
LETTA_API_KEY=your_api_key_here
370
+
371
+
# Store agent ID after creation (Next.js)
372
+
LETTA_AGENT_ID=agent-xxxxxxxxx
373
+
374
+
# For self-hosted (optional)
375
+
LETTA_BASE_URL=http://localhost:8283
376
+
```
377
+
378
+
---
379
+
380
+
## **7. Verification Checklist**
381
+
382
+
Before providing Letta solutions, verify:
383
+
384
+
1. **SDK Choice**: Are you using the simplest appropriate SDK?
385
+
- Familiar with or already using Vercel AI SDK? → use the Vercel AI SDK Letta provider
386
+
- Agent management needed? → use the Node.js/Python SDKs
387
+
2. **Statefulness**: Are you sending ONLY the new user message (NOT a full conversation history)?
388
+
3. **Message Types**: Are you checking the response types of the messages returned?
389
+
4. **Response Parsing**: If using the Python/Node.js SDK, are you extracting `content` from assistant messages?
390
+
5. **Imports**: Correct package imports for the chosen SDK?
391
+
6. **Client**: Proper client initialization with auth/base_url?
392
+
7. **Agent Creation**: Memory blocks with proper structure?
393
+
8. **Memory Blocks**: Descriptions for custom blocks?