A human-friendly DSL for ATProto Lexicons
0
fork

Configure Feed

Select the types of activity you want to include in your feed.

Update docs

+1752 -354
-354
website/content/docs/cli.md
··· 1 - +++ 2 - title = "CLI Reference" 3 - description = "Command-line tool documentation" 4 - weight = 3 5 - +++ 6 - 7 - ## Installation 8 - 9 - Clone and build from source: 10 - 11 - ```bash 12 - git clone https://tangled.org/@stavola.xyz/mlf 13 - cd mlf 14 - cargo build --release 15 - ``` 16 - 17 - The binary will be at `target/release/mlf`. 18 - 19 - ### Install with cargo 20 - 21 - ```bash 22 - # Install with all code generators (default: TypeScript, Go, Rust + JSON) 23 - cargo install --path mlf-cli --all-features 24 - 25 - # Install with only specific generators 26 - cargo install --path mlf-cli --no-default-features --features typescript 27 - cargo install --path mlf-cli --no-default-features --features typescript,go 28 - 29 - # Install with JSON generation only (minimal) 30 - cargo install --path mlf-cli --no-default-features 31 - ``` 32 - 33 - ### Available Features 34 - 35 - The CLI uses cargo features for optional code generators: 36 - 37 - - `typescript` - TypeScript type definitions and interfaces 38 - - `go` - Go structs with JSON tags 39 - - `rust` - Rust structs with serde derive macros 40 - - **default** = `["typescript", "go", "rust"]` - All generators enabled 41 - 42 - The JSON lexicon generator is always available (built into `mlf-codegen`). 43 - 44 - ## Commands 45 - 46 - ### `mlf check` 47 - 48 - Validate MLF lexicon files for syntax and type errors. 49 - 50 - ```bash 51 - mlf check [INPUT]... 52 - ``` 53 - 54 - **Arguments:** 55 - - `[INPUT]...` - MLF lexicon file(s) to validate (glob patterns supported) 56 - 57 - **Examples:** 58 - 59 - ```bash 60 - # Check a single file 61 - mlf check thread.mlf 62 - 63 - # Check multiple files 64 - mlf check thread.mlf profile.mlf reply.mlf 65 - 66 - # Check with glob patterns 67 - mlf check "lexicons/**/*.mlf" 68 - ``` 69 - 70 - **Output:** 71 - - ✓ Success message if valid 72 - - Detailed error messages with source context if invalid 73 - 74 - --- 75 - 76 - ### `mlf validate` 77 - 78 - Validate a JSON record against an MLF lexicon. 79 - 80 - ```bash 81 - mlf validate <LEXICON> <RECORD> 82 - ``` 83 - 84 - **Arguments:** 85 - - `<LEXICON>` - MLF lexicon file 86 - - `<RECORD>` - JSON record file to validate against the lexicon 87 - 88 - **Example:** 89 - 90 - ```bash 91 - mlf validate thread.mlf record.json 92 - ``` 93 - 94 - **Output:** 95 - - ✓ Success if record is valid 96 - - Detailed validation errors if invalid 97 - 98 - --- 99 - 100 - ### `mlf generate lexicon` 101 - 102 - Generate ATProto JSON lexicons from MLF files. 103 - 104 - ```bash 105 - mlf generate lexicon --output <OUTPUT> [OPTIONS] 106 - ``` 107 - 108 - **Options:** 109 - - `-i, --input <INPUT>` - Input MLF files (glob patterns supported, can be specified multiple times) 110 - - `-o, --output <OUTPUT>` - Output directory (required) 111 - - `--flat` - Use flat file structure (e.g., `com.example.thread.json`) 112 - 113 - **Examples:** 114 - 115 - ```bash 116 - # Generate with folder structure 117 - mlf generate lexicon -i thread.mlf -o lexicons/ 118 - # Creates: lexicons/com/example/thread.json 119 - 120 - # Generate with flat structure 121 - mlf generate lexicon -i thread.mlf -o lexicons/ --flat 122 - # Creates: lexicons/com.example.thread.json 123 - 124 - # Generate from multiple files 125 - mlf generate lexicon -i thread.mlf -i reply.mlf -o lexicons/ 126 - 127 - # Generate from glob pattern 128 - mlf generate lexicon -i "src/**/*.mlf" -o dist/lexicons/ 129 - ``` 130 - 131 - --- 132 - 133 - ### `mlf generate code` 134 - 135 - Generate code in various programming languages from MLF files. 136 - 137 - ```bash 138 - mlf generate code --generator <GENERATOR> --output <OUTPUT> [OPTIONS] 139 - ``` 140 - 141 - **Options:** 142 - - `-g, --generator <GENERATOR>` - Generator to use: `json`, `typescript`, `go`, or `rust` (required) 143 - - `-i, --input <INPUT>` - Input MLF files (glob patterns supported, can be specified multiple times) 144 - - `-o, --output <OUTPUT>` - Output directory (required) 145 - - `--flat` - Use flat file structure (e.g., `com.example.thread.ts`) 146 - 147 - **Available Generators:** 148 - 149 - | Generator | Output | Description | 150 - |-----------|--------|-------------| 151 - | `json` | `.json` | AT Protocol JSON lexicons (always available) | 152 - | `typescript` | `.ts` | TypeScript interfaces with JSDoc comments | 153 - | `go` | `.go` | Go structs with JSON tags and proper capitalization | 154 - | `rust` | `.rs` | Rust structs with serde derive macros | 155 - 156 - **TypeScript Example:** 157 - 158 - ```bash 159 - mlf generate code -g typescript -i thread.mlf -o src/types/ 160 - # Creates: src/types/com/example/thread.ts 161 - ``` 162 - 163 - Generated TypeScript: 164 - ```typescript 165 - /** 166 - * Generated from com.example.thread 167 - * Do not edit manually 168 - */ 169 - 170 - export interface Thread { 171 - /** Thread title */ 172 - title: string; 173 - /** Creation timestamp */ 174 - createdAt: string; 175 - posts: Post[]; 176 - } 177 - ``` 178 - 179 - **Go Example:** 180 - 181 - ```bash 182 - mlf generate code -g go -i thread.mlf -o pkg/models/ 183 - # Creates: pkg/models/com/example/thread.go 184 - ``` 185 - 186 - Generated Go: 187 - ```go 188 - // Generated from com.example.thread 189 - // Do not edit manually 190 - 191 - package thread 192 - 193 - // Thread represents a discussion thread 194 - type Thread struct { 195 - // Thread title 196 - Title string `json:"title"` 197 - // Creation timestamp 198 - CreatedAt string `json:"createdAt"` 199 - Posts []Post `json:"posts"` 200 - } 201 - ``` 202 - 203 - **Rust Example:** 204 - 205 - ```bash 206 - mlf generate code -g rust -i thread.mlf -o src/models/ 207 - # Creates: src/models/com/example/thread.rs 208 - ``` 209 - 210 - Generated Rust: 211 - ```rust 212 - // Generated from com.example.thread 213 - // Do not edit manually 214 - 215 - use serde::{Deserialize, Serialize}; 216 - 217 - /// Thread represents a discussion thread 218 - #[derive(Debug, Clone, Serialize, Deserialize)] 219 - pub struct Thread { 220 - /// Thread title 221 - pub title: String, 222 - /// Creation timestamp 223 - #[serde(rename = "createdAt")] 224 - pub created_at: String, 225 - pub posts: Vec<Post>, 226 - } 227 - ``` 228 - 229 - **Notes:** 230 - 231 - - TypeScript: Uses interfaces for records, type aliases for defs, optional fields use `?` 232 - - Go: Uses structs with JSON tags, pointer types for optional fields, PascalCase for exports 233 - - Rust: Uses structs with serde, `Option<T>` for optional fields, snake_case with `#[serde(rename)]` 234 - - All generators handle doc comments, constraints are preserved in generated code where applicable 235 - 236 - --- 237 - 238 - ### `mlf generate mlf` 239 - 240 - Convert ATProto JSON lexicons back to MLF format. 241 - 242 - ```bash 243 - mlf generate mlf --output <OUTPUT> [OPTIONS] 244 - ``` 245 - 246 - **Options:** 247 - - `-i, --input <INPUT>` - Input JSON lexicon files (glob patterns supported, can be specified multiple times) 248 - - `-o, --output <OUTPUT>` - Output directory (required) 249 - 250 - **Examples:** 251 - 252 - ```bash 253 - # Convert a single JSON lexicon to MLF 254 - mlf generate mlf -i com.example.thread.json -o ./lexicons/ 255 - # Creates: lexicons/com/example/thread.mlf 256 - 257 - # Convert multiple JSON lexicons 258 - mlf generate mlf -i lexicon1.json -i lexicon2.json -o ./mlf/ 259 - 260 - # Convert using glob pattern 261 - mlf generate mlf -i "dist/lexicons/**/*.json" -o ./src/ 262 - ``` 263 - 264 - **Features:** 265 - 266 - - **Smart type conversion** - Automatically converts format strings (did, datetime, handle) to prelude types (Did, Datetime, Handle) 267 - - **Proper formatting** - Generates clean, properly indented MLF with correct syntax 268 - - **Reference conversion** - Converts `namespace#name` refs to `namespace.name` notation 269 - - **Complete coverage** - Supports records, queries, procedures, subscriptions, tokens, and type definitions 270 - 271 - **Input format:** 272 - 273 - The JSON lexicon must be a valid ATProto lexicon with: 274 - - `lexicon: 1` version field 275 - - `id` field containing the namespace 276 - - `defs` object with type definitions 277 - 278 - **Example:** 279 - 280 - Given a JSON lexicon: 281 - ```json 282 - { 283 - "lexicon": 1, 284 - "id": "com.example.thread", 285 - "defs": { 286 - "main": { 287 - "type": "record", 288 - "key": "tid", 289 - "record": { 290 - "type": "object", 291 - "required": ["title", "createdAt"], 292 - "properties": { 293 - "title": { 294 - "type": "string", 295 - "maxLength": 200 296 - }, 297 - "createdAt": { 298 - "type": "string", 299 - "format": "datetime" 300 - } 301 - } 302 - } 303 - } 304 - } 305 - } 306 - ``` 307 - 308 - Generates MLF: 309 - ```mlf 310 - record main { 311 - title!: string constrained { 312 - maxLength: 200, 313 - }, 314 - createdAt!: Datetime, 315 - }; 316 - ``` 317 - 318 - **Use cases:** 319 - 320 - - **Migration** - Convert existing JSON lexicons to MLF format 321 - - **Interoperability** - Work with lexicons from other tools or repositories 322 - - **Comparison** - Generate MLF from JSON to compare with hand-written MLF 323 - - **Learning** - See how JSON lexicons translate to MLF syntax 324 - 325 - --- 326 - 327 - ## Error Messages 328 - 329 - MLF provides rich error messages with: 330 - 331 - - Source code context 332 - - Labeled spans showing exact error locations 333 - - Helpful suggestions for fixing errors 334 - - Error codes for categorization 335 - 336 - **Example error:** 337 - 338 - ``` 339 - × Undefined reference to 'ProfileView' 340 - ╭─[profile.mlf:5:12] 341 - 5 │ author: ProfileView, 342 - · ^^^^^^^^^^^ 'ProfileView' is not defined 343 - ╰──── 344 - help: Make sure this type is defined in the same file or imported via 'use'. 345 - ``` 346 - 347 - ## Environment Variables 348 - 349 - None currently used. 350 - 351 - ## Exit Codes 352 - 353 - - `0` - Success 354 - - `1` - Error (syntax, validation, or runtime error)
+77
website/content/docs/cli/01-installation.md
··· 1 + +++ 2 + title = "Installation" 3 + description = "How to install the MLF CLI" 4 + weight = 1 5 + +++ 6 + 7 + ## Building from Source 8 + 9 + Clone and build from source: 10 + 11 + ```bash 12 + git clone https://tangled.org/@stavola.xyz/mlf 13 + cd mlf 14 + cargo build --release 15 + ``` 16 + 17 + The binary will be at `target/release/mlf`. 18 + 19 + ## Installing with Cargo 20 + 21 + ### All Features (Recommended) 22 + 23 + Install with all code generators enabled: 24 + 25 + ```bash 26 + cargo install --path mlf-cli --all-features 27 + ``` 28 + 29 + This includes TypeScript, Go, Rust, and JSON lexicon generators. 30 + 31 + ### Selective Installation 32 + 33 + Install only specific generators to reduce compilation time and binary size: 34 + 35 + ```bash 36 + # TypeScript only 37 + cargo install --path mlf-cli --no-default-features --features typescript 38 + 39 + # TypeScript and Go 40 + cargo install --path mlf-cli --no-default-features --features typescript,go 41 + 42 + # Minimal (JSON lexicon generation only) 43 + cargo install --path mlf-cli --no-default-features 44 + ``` 45 + 46 + ## Available Features 47 + 48 + The CLI uses cargo features for optional code generators: 49 + 50 + | Feature | Description | Output Format | 51 + |---------|-------------|---------------| 52 + | `typescript` | TypeScript type definitions and interfaces | `.ts` | 53 + | `go` | Go structs with JSON tags | `.go` | 54 + | `rust` | Rust structs with serde derive macros | `.rs` | 55 + | (built-in) | JSON lexicon generator | `.json` | 56 + 57 + **Default features:** `["typescript", "go", "rust"]` - All generators enabled. 58 + 59 + The JSON lexicon generator is always available (built into `mlf-codegen` core). 60 + 61 + ## Verify Installation 62 + 63 + After installation, verify the CLI is working: 64 + 65 + ```bash 66 + mlf --version 67 + mlf --help 68 + ``` 69 + 70 + ## Next Steps 71 + 72 + Once installed, you can: 73 + 74 + 1. [Configure your project](02-configuration.md) with an `mlf.toml` file 75 + 2. [Check MLF files](03-check.md) for syntax and type errors 76 + 3. [Generate code](05-generate.md) in your preferred language 77 + 4. [Fetch remote lexicons](06-fetch.md) from ATProto repositories
+216
website/content/docs/cli/02-configuration.md
··· 1 + +++ 2 + title = "Configuration" 3 + description = "Project configuration with mlf.toml" 4 + weight = 2 5 + +++ 6 + 7 + MLF projects can be configured with an `mlf.toml` file that specifies source directories, output configurations, and dependencies. This allows you to run commands without repeatedly specifying arguments. 8 + 9 + ## Basic Structure 10 + 11 + Create an `mlf.toml` file in your project root: 12 + 13 + ```toml 14 + [source] 15 + directory = "./lexicons" 16 + 17 + [[output]] 18 + type = "lexicon" 19 + directory = "./dist/lexicons" 20 + 21 + [[output]] 22 + type = "typescript" 23 + directory = "./src/types" 24 + 25 + [dependencies] 26 + dependencies = ["stream.place", "app.bsky"] 27 + ``` 28 + 29 + ## Configuration Sections 30 + 31 + ### Source Directory 32 + 33 + The `[source]` section specifies where your MLF files are located: 34 + 35 + ```toml 36 + [source] 37 + directory = "./lexicons" 38 + ``` 39 + 40 + **Default:** `"./lexicons"` 41 + 42 + This directory is used by: 43 + - `mlf check` (when run without arguments) 44 + - `mlf generate` (when run without arguments) 45 + 46 + ### Output Configurations 47 + 48 + The `[[output]]` section defines code generation targets. You can have multiple output blocks: 49 + 50 + ```toml 51 + [[output]] 52 + type = "lexicon" 53 + directory = "./dist/lexicons" 54 + 55 + [[output]] 56 + type = "typescript" 57 + directory = "./src/types" 58 + 59 + [[output]] 60 + type = "go" 61 + directory = "./pkg/lexicons" 62 + 63 + [[output]] 64 + type = "rust" 65 + directory = "./src/lexicons" 66 + ``` 67 + 68 + **Supported types:** 69 + - `"lexicon"` - Generate JSON lexicons 70 + - `"typescript"` - Generate TypeScript types 71 + - `"go"` - Generate Go structs 72 + - `"rust"` - Generate Rust structs 73 + 74 + When you run `mlf generate` without arguments, it will generate all configured outputs. 75 + 76 + ### Dependencies 77 + 78 + The `[dependencies]` section lists external lexicon namespaces to fetch: 79 + 80 + ```toml 81 + [dependencies] 82 + dependencies = [ 83 + "stream.place", 84 + "app.bsky", 85 + "com.atproto" 86 + ] 87 + ``` 88 + 89 + These dependencies are fetched when you run `mlf fetch` without arguments. 90 + 91 + ## Commands Using Configuration 92 + 93 + ### `mlf check` 94 + 95 + When run without arguments, checks all files in the source directory: 96 + 97 + ```bash 98 + mlf check 99 + # Equivalent to: mlf check "./lexicons/**/*.mlf" 100 + ``` 101 + 102 + ### `mlf generate` 103 + 104 + When run without a subcommand, generates all configured outputs: 105 + 106 + ```bash 107 + mlf generate 108 + # Runs all [[output]] configurations 109 + ``` 110 + 111 + ### `mlf fetch` 112 + 113 + When run without arguments, fetches all dependencies: 114 + 115 + ```bash 116 + mlf fetch 117 + # Fetches: stream.place, app.bsky, com.atproto 118 + ``` 119 + 120 + Add `--save` flag to automatically update `mlf.toml`: 121 + 122 + ```bash 123 + mlf fetch stream.place --save 124 + # Downloads lexicons AND adds to dependencies array 125 + ``` 126 + 127 + ## Complete Example 128 + 129 + Here's a complete `mlf.toml` for a TypeScript project using ATProto lexicons: 130 + 131 + ```toml 132 + [source] 133 + directory = "./lexicons" 134 + 135 + [[output]] 136 + type = "lexicon" 137 + directory = "./dist/lexicons" 138 + 139 + [[output]] 140 + type = "typescript" 141 + directory = "./src/types" 142 + 143 + [dependencies] 144 + dependencies = [ 145 + "app.bsky", 146 + "com.atproto", 147 + "chat.bsky" 148 + ] 149 + ``` 150 + 151 + ### Workflow 152 + 153 + 1. **Fetch dependencies:** 154 + ```bash 155 + mlf fetch 156 + ``` 157 + Downloads `app.bsky`, `com.atproto`, and `chat.bsky` lexicons to `.mlf/lexicons/` 158 + 159 + 2. **Write your lexicons:** 160 + Create `.mlf` files in `./lexicons/` that reference the fetched lexicons 161 + 162 + 3. **Check for errors:** 163 + ```bash 164 + mlf check 165 + ``` 166 + Validates all MLF files in `./lexicons/` 167 + 168 + 4. **Generate outputs:** 169 + ```bash 170 + mlf generate 171 + ``` 172 + Creates: 173 + - JSON lexicons in `./dist/lexicons/` 174 + - TypeScript types in `./src/types/` 175 + 176 + ## Fetched Lexicons Cache 177 + 178 + When you fetch lexicons, they're stored in `.mlf/lexicons/`: 179 + 180 + ``` 181 + .mlf/ 182 + ├── .gitignore # Automatically created 183 + ├── .lexicon-cache.toml # Metadata about fetched lexicons 184 + └── lexicons/ 185 + ├── json/ # Original JSON lexicons 186 + │ ├── app.bsky.actor.profile.json 187 + │ └── ... 188 + └── mlf/ # Converted MLF format 189 + ├── app.bsky.actor.profile.mlf 190 + └── ... 191 + ``` 192 + 193 + The `.mlf` directory is automatically added to `.gitignore`, so fetched lexicons won't be committed to your repository. 194 + 195 + ## Best Practices 196 + 197 + 1. **Commit `mlf.toml`** - Version control your configuration 198 + 2. **Don't commit `.mlf/`** - Let each developer fetch dependencies 199 + 3. **Use semantic namespaces** - Organize lexicons by domain 200 + 4. **Multiple outputs** - Generate both lexicons and code simultaneously 201 + 5. **CI/CD integration** - Run `mlf check` in your CI pipeline 202 + 203 + ## Override Configuration 204 + 205 + You can always override configuration with explicit arguments: 206 + 207 + ```bash 208 + # Override source directory 209 + mlf check "./other-lexicons/**/*.mlf" 210 + 211 + # Override output 212 + mlf generate lexicon -i custom.mlf -o ./custom-output/ 213 + 214 + # Fetch specific namespace (ignoring dependencies list) 215 + mlf fetch stream.place 216 + ```
+191
website/content/docs/cli/03-check.md
··· 1 + +++ 2 + title = "Check Command" 3 + description = "Validate MLF lexicon files" 4 + weight = 3 5 + +++ 6 + 7 + The `mlf check` command validates MLF lexicon files for syntax and type errors. 8 + 9 + ## Usage 10 + 11 + ```bash 12 + mlf check [INPUT]... 13 + ``` 14 + 15 + **Arguments:** 16 + - `[INPUT]...` - MLF lexicon file(s) to validate (glob patterns supported) 17 + 18 + If no input files are provided, `mlf check` will use the source directory from your `mlf.toml` configuration. 19 + 20 + ## Examples 21 + 22 + ### Check with Configuration 23 + 24 + If you have an `mlf.toml` file: 25 + 26 + ```toml 27 + [source] 28 + directory = "./lexicons" 29 + ``` 30 + 31 + Simply run: 32 + 33 + ```bash 34 + mlf check 35 + ``` 36 + 37 + This automatically checks all `.mlf` files in `./lexicons/`. 38 + 39 + ### Check Specific Files 40 + 41 + ```bash 42 + # Check a single file 43 + mlf check thread.mlf 44 + 45 + # Check multiple files 46 + mlf check thread.mlf profile.mlf reply.mlf 47 + ``` 48 + 49 + ### Check with Glob Patterns 50 + 51 + ```bash 52 + # Check all MLF files in a directory 53 + mlf check "lexicons/**/*.mlf" 54 + 55 + # Check files matching a pattern 56 + mlf check "src/*/schema.mlf" 57 + 58 + # Check all MLF files recursively 59 + mlf check "**/*.mlf" 60 + ``` 61 + 62 + ## Validation Checks 63 + 64 + The check command performs comprehensive validation: 65 + 66 + ### 1. **Syntax Validation** 67 + - Correct MLF syntax 68 + - Proper use of keywords 69 + - Valid identifiers and namespaces 70 + - Correct constraint syntax 71 + 72 + ### 2. **Type Validation** 73 + - All referenced types exist 74 + - Type constraints are valid 75 + - Required fields are properly marked 76 + - Array and object structures are correct 77 + 78 + ### 3. **Semantic Validation** 79 + - No duplicate definitions 80 + - Valid record keys 81 + - Proper XRPC method signatures 82 + - Correct union type usage 83 + 84 + ### 4. **Cross-Reference Validation** 85 + - External references resolve correctly 86 + - Import statements are valid 87 + - Prelude types are accessible 88 + - Fetched dependencies are available 89 + 90 + ## Output 91 + 92 + ### Success 93 + 94 + When all files are valid: 95 + 96 + ``` 97 + ✓ thread.mlf: Parsed successfully 98 + ✓ profile.mlf: Parsed successfully 99 + ✓ reply.mlf: Parsed successfully 100 + 101 + ✓ All lexicons are valid 102 + ``` 103 + 104 + ### Errors 105 + 106 + When errors are found, detailed diagnostics are shown: 107 + 108 + ``` 109 + × Undefined reference to 'ProfileView' 110 + ╭─[profile.mlf:5:12] 111 + 5 │ author: ProfileView, 112 + · ^^^^^^^^^^^ 'ProfileView' is not defined 113 + ╰──── 114 + help: Make sure this type is defined in the same file or imported via 'use'. 115 + ``` 116 + 117 + Each error includes: 118 + - Error message with context 119 + - Source code snippet 120 + - Exact location (line and column) 121 + - Helpful suggestions for fixing 122 + 123 + ## Working with Dependencies 124 + 125 + If your lexicons reference external types (e.g., from `app.bsky` or `com.atproto`), make sure to fetch them first: 126 + 127 + ```bash 128 + # Fetch dependencies 129 + mlf fetch 130 + 131 + # Then check your lexicons 132 + mlf check 133 + ``` 134 + 135 + The check command automatically loads lexicons from `.mlf/lexicons/mlf/` if they exist. 136 + 137 + ## Exit Codes 138 + 139 + - `0` - All files are valid 140 + - `1` - Validation errors found 141 + 142 + ## Common Issues 143 + 144 + ### Undefined Reference 145 + 146 + ``` 147 + × Undefined reference to 'SomeType' 148 + ``` 149 + 150 + **Solution:** 151 + - Define `SomeType` in the same file, or 152 + - Fetch the lexicon containing `SomeType` using `mlf fetch`, or 153 + - Add an import statement if needed 154 + 155 + ### Parse Error 156 + 157 + ``` 158 + × Expected 'constrained', found 'contsrained' 159 + ``` 160 + 161 + **Solution:** Fix the typo in your MLF file 162 + 163 + ### Type Mismatch 164 + 165 + ``` 166 + × Field 'count' expects integer, found string 167 + ``` 168 + 169 + **Solution:** Correct the field type to match the schema 170 + 171 + ## Integration with CI/CD 172 + 173 + Use `mlf check` in your continuous integration pipeline: 174 + 175 + ```yaml 176 + # GitHub Actions example 177 + - name: Validate MLF Lexicons 178 + run: | 179 + mlf fetch 180 + mlf check 181 + ``` 182 + 183 + This ensures all lexicons remain valid as your project evolves. 184 + 185 + ## Tips 186 + 187 + 1. **Use configuration** - Set up `mlf.toml` to avoid typing paths repeatedly 188 + 2. **Check often** - Run `mlf check` frequently during development 189 + 3. **Version control** - Commit valid lexicons only 190 + 4. **Fetch first** - Always fetch dependencies before checking 191 + 5. **Read errors carefully** - MLF provides detailed error messages with helpful suggestions
+211
website/content/docs/cli/04-validate.md
··· 1 + +++ 2 + title = "Validate Command" 3 + description = "Validate JSON records against lexicons" 4 + weight = 4 5 + +++ 6 + 7 + The `mlf validate` command validates JSON record data against an MLF lexicon schema. 8 + 9 + ## Usage 10 + 11 + ```bash 12 + mlf validate <LEXICON> <RECORD> 13 + ``` 14 + 15 + **Arguments:** 16 + - `<LEXICON>` - MLF lexicon file defining the schema 17 + - `<RECORD>` - JSON file containing the record to validate 18 + 19 + ## Example 20 + 21 + Given a lexicon file `thread.mlf`: 22 + 23 + ```mlf 24 + record main { 25 + title!: string constrained { 26 + maxLength: 200, 27 + }, 28 + createdAt!: Datetime, 29 + posts: Post[], 30 + }; 31 + 32 + def Post = { 33 + text!: string, 34 + createdAt!: Datetime, 35 + }; 36 + ``` 37 + 38 + And a JSON record file `my-thread.json`: 39 + 40 + ```json 41 + { 42 + "title": "My Discussion Thread", 43 + "createdAt": "2024-01-15T10:30:00Z", 44 + "posts": [ 45 + { 46 + "text": "First post!", 47 + "createdAt": "2024-01-15T10:30:00Z" 48 + } 49 + ] 50 + } 51 + ``` 52 + 53 + Validate the record: 54 + 55 + ```bash 56 + mlf validate thread.mlf my-thread.json 57 + ``` 58 + 59 + ## Output 60 + 61 + ### Valid Record 62 + 63 + ``` 64 + ✓ Lexicon parsed successfully 65 + ✓ JSON record parsed successfully 66 + ✓ Record is valid according to the lexicon schema 67 + ``` 68 + 69 + ### Invalid Record 70 + 71 + If the record doesn't match the schema: 72 + 73 + ``` 74 + ✗ Record validation failed with 2 error(s): 75 + • Field 'title' is required but missing 76 + • Field 'createdAt': expected string in datetime format, found "not-a-date" 77 + ``` 78 + 79 + ## Validation Rules 80 + 81 + The validate command checks: 82 + 83 + ### 1. **Required Fields** 84 + - All fields marked with `!` must be present 85 + - Optional fields can be omitted 86 + 87 + ### 2. **Type Matching** 88 + - String fields must be strings 89 + - Integer fields must be numbers 90 + - Arrays must be arrays 91 + - Objects must have correct structure 92 + 93 + ### 3. **Constraints** 94 + - String length constraints (`maxLength`, `minLength`) 95 + - String grapheme constraints (`maxGraphemes`, `minGraphemes`) 96 + - Integer range constraints (`minimum`, `maximum`) 97 + - Array length constraints 98 + - Format validation (`datetime`, `uri`, `at-uri`, etc.) 99 + 100 + ### 4. **Nested Structures** 101 + - Object fields are recursively validated 102 + - Array items match their item schema 103 + - References to other defs are resolved 104 + 105 + ## Use Cases 106 + 107 + ### 1. **Test Data Validation** 108 + 109 + Validate test fixtures before using them: 110 + 111 + ```bash 112 + mlf validate profile.mlf test-data/profile-1.json 113 + ``` 114 + 115 + ### 2. **API Response Validation** 116 + 117 + Check that API responses match your schema: 118 + 119 + ```bash 120 + curl https://api.example.com/profile > response.json 121 + mlf validate profile.mlf response.json 122 + ``` 123 + 124 + ### 3. **Schema Migration** 125 + 126 + When updating schemas, validate existing records against the new schema: 127 + 128 + ```bash 129 + # Validate all records 130 + for file in records/*.json; do 131 + mlf validate schema.mlf "$file" || echo "Failed: $file" 132 + done 133 + ``` 134 + 135 + ### 4. **Development Workflow** 136 + 137 + Validate records during development: 138 + 139 + ```bash 140 + # Write some test data 141 + echo '{"title": "Test", "createdAt": "2024-01-15T10:30:00Z"}' > test.json 142 + 143 + # Validate it 144 + mlf validate thread.mlf test.json 145 + ``` 146 + 147 + ## Common Validation Errors 148 + 149 + ### Missing Required Field 150 + 151 + ``` 152 + ✗ Field 'title' is required but missing 153 + ``` 154 + 155 + **Solution:** Add the required field to your JSON 156 + 157 + ### Type Mismatch 158 + 159 + ``` 160 + ✗ Field 'count': expected integer, found "123" 161 + ``` 162 + 163 + **Solution:** Use a number instead of a string: `"count": 123` 164 + 165 + ### Constraint Violation 166 + 167 + ``` 168 + ✗ Field 'title': string length 250 exceeds maxLength 200 169 + ``` 170 + 171 + **Solution:** Shorten the string to meet the constraint 172 + 173 + ### Invalid Format 174 + 175 + ``` 176 + ✗ Field 'createdAt': expected datetime format, found "2024-01-15" 177 + ``` 178 + 179 + **Solution:** Use full ISO 8601 datetime format: `"2024-01-15T10:30:00Z"` 180 + 181 + ### Invalid Array Item 182 + 183 + ``` 184 + ✗ Array item at index 0: Field 'text' is required but missing 185 + ``` 186 + 187 + **Solution:** Ensure all array items match the schema 188 + 189 + ## Exit Codes 190 + 191 + - `0` - Record is valid 192 + - `1` - Validation failed or error occurred 193 + 194 + ## Limitations 195 + 196 + The validate command currently validates against the lexicon structure but does not: 197 + 198 + - Validate CID references 199 + - Validate blob content types 200 + - Check external references 201 + - Verify cryptographic signatures 202 + 203 + These checks would typically be performed by the PDS or other ATProto infrastructure. 204 + 205 + ## Tips 206 + 207 + 1. **Start simple** - Validate basic records first, then add complexity 208 + 2. **Use test data** - Create JSON fixtures for your schemas 209 + 3. **Automate validation** - Add validation to your test suite 210 + 4. **Check constraints** - Pay attention to string lengths and ranges 211 + 5. **Format matters** - Use proper datetime/URI formats as specified by ATProto
+355
website/content/docs/cli/05-generate.md
··· 1 + +++ 2 + title = "Generate Commands" 3 + description = "Generate code and lexicons from MLF" 4 + weight = 5 5 + +++ 6 + 7 + The `mlf generate` command converts MLF files to various output formats including JSON lexicons and code in multiple programming languages. 8 + 9 + ## Overview 10 + 11 + ```bash 12 + # Generate all configured outputs 13 + mlf generate 14 + 15 + # Generate JSON lexicons 16 + mlf generate lexicon -i <INPUT> -o <OUTPUT> 17 + 18 + # Generate code in a specific language 19 + mlf generate code -g <GENERATOR> -i <INPUT> -o <OUTPUT> 20 + 21 + # Convert JSON lexicons to MLF 22 + mlf generate mlf -i <INPUT> -o <OUTPUT> 23 + ``` 24 + 25 + ## Generate All Outputs 26 + 27 + When run without a subcommand, `mlf generate` uses your `mlf.toml` configuration to generate all specified outputs: 28 + 29 + ```toml 30 + [[output]] 31 + type = "lexicon" 32 + directory = "./dist/lexicons" 33 + 34 + [[output]] 35 + type = "typescript" 36 + directory = "./src/types" 37 + ``` 38 + 39 + ```bash 40 + mlf generate 41 + ``` 42 + 43 + This will: 44 + 1. Read the source directory from configuration 45 + 2. Process all `.mlf` files 46 + 3. Generate each configured output type 47 + 48 + **Output:** 49 + ``` 50 + Running 2 output configuration(s)... 51 + 52 + Generating lexicon output to ./dist/lexicons... 53 + Generated: ./dist/lexicons/com/example/thread.json 54 + ✓ Generated lexicon output successfully 55 + 56 + Generating typescript output to ./src/types... 57 + Generated: ./src/types/com/example/thread.ts 58 + ✓ Generated typescript output successfully 59 + 60 + ✓ Successfully generated all 2 output(s) 61 + ``` 62 + 63 + --- 64 + 65 + ## Generate Lexicon (JSON) 66 + 67 + Generate ATProto JSON lexicons from MLF files. 68 + 69 + ```bash 70 + mlf generate lexicon -i <INPUT> -o <OUTPUT> [OPTIONS] 71 + ``` 72 + 73 + **Options:** 74 + - `-i, --input <INPUT>` - Input MLF files (glob patterns supported, can be specified multiple times) 75 + - `-o, --output <OUTPUT>` - Output directory (required) 76 + - `--flat` - Use flat file structure (e.g., `com.example.thread.json`) 77 + 78 + **Examples:** 79 + 80 + ```bash 81 + # Generate with folder structure 82 + mlf generate lexicon -i thread.mlf -o lexicons/ 83 + # Creates: lexicons/com/example/thread.json 84 + 85 + # Generate with flat structure 86 + mlf generate lexicon -i thread.mlf -o lexicons/ --flat 87 + # Creates: lexicons/com.example.thread.json 88 + 89 + # Generate from multiple files 90 + mlf generate lexicon -i thread.mlf -i reply.mlf -o lexicons/ 91 + 92 + # Generate from glob pattern 93 + mlf generate lexicon -i "src/**/*.mlf" -o dist/lexicons/ 94 + ``` 95 + 96 + --- 97 + 98 + ## Generate Code 99 + 100 + Generate code in various programming languages from MLF files. 101 + 102 + ```bash 103 + mlf generate code -g <GENERATOR> -i <INPUT> -o <OUTPUT> [OPTIONS] 104 + ``` 105 + 106 + **Options:** 107 + - `-g, --generator <GENERATOR>` - Generator to use (required): `json`, `typescript`, `go`, or `rust` 108 + - `-i, --input <INPUT>` - Input MLF files (glob patterns supported, can be specified multiple times) 109 + - `-o, --output <OUTPUT>` - Output directory (required) 110 + - `--flat` - Use flat file structure 111 + 112 + **Available Generators:** 113 + 114 + | Generator | Output | Features | 115 + |-----------|--------|----------| 116 + | `json` | `.json` | AT Protocol JSON lexicons (always available) | 117 + | `typescript` | `.ts` | TypeScript interfaces with JSDoc, optional fields with `?` | 118 + | `go` | `.go` | Go structs with JSON tags, proper capitalization | 119 + | `rust` | `.rs` | Rust structs with serde, `Option<T>` for optional fields | 120 + 121 + ### TypeScript Example 122 + 123 + ```bash 124 + mlf generate code -g typescript -i thread.mlf -o src/types/ 125 + ``` 126 + 127 + **Input MLF:** 128 + ```mlf 129 + /// A discussion thread 130 + record main { 131 + /// Thread title 132 + title!: string constrained { 133 + maxLength: 200, 134 + }, 135 + /// Creation timestamp 136 + createdAt!: Datetime, 137 + posts: Post[], 138 + }; 139 + ``` 140 + 141 + **Generated TypeScript:** 142 + ```typescript 143 + /** 144 + * Generated from com.example.thread 145 + * Do not edit manually 146 + */ 147 + 148 + /** 149 + * A discussion thread 150 + */ 151 + export interface Main { 152 + /** Thread title */ 153 + title: string; 154 + /** Creation timestamp */ 155 + createdAt: string; 156 + posts?: Post[]; 157 + } 158 + ``` 159 + 160 + ### Go Example 161 + 162 + ```bash 163 + mlf generate code -g go -i thread.mlf -o pkg/models/ 164 + ``` 165 + 166 + **Generated Go:** 167 + ```go 168 + // Generated from com.example.thread 169 + // Do not edit manually 170 + 171 + package thread 172 + 173 + // Main represents a discussion thread 174 + type Main struct { 175 + // Thread title 176 + Title string `json:"title"` 177 + // Creation timestamp 178 + CreatedAt string `json:"createdAt"` 179 + // Posts (optional) 180 + Posts []Post `json:"posts,omitempty"` 181 + } 182 + ``` 183 + 184 + ### Rust Example 185 + 186 + ```bash 187 + mlf generate code -g rust -i thread.mlf -o src/models/ 188 + ``` 189 + 190 + **Generated Rust:** 191 + ```rust 192 + // Generated from com.example.thread 193 + // Do not edit manually 194 + 195 + use serde::{Deserialize, Serialize}; 196 + 197 + /// A discussion thread 198 + #[derive(Debug, Clone, Serialize, Deserialize)] 199 + pub struct Main { 200 + /// Thread title 201 + pub title: String, 202 + /// Creation timestamp 203 + #[serde(rename = "createdAt")] 204 + pub created_at: String, 205 + /// Posts (optional) 206 + #[serde(skip_serializing_if = "Option::is_none")] 207 + pub posts: Option<Vec<Post>>, 208 + } 209 + ``` 210 + 211 + --- 212 + 213 + ## Generate MLF 214 + 215 + Convert ATProto JSON lexicons back to MLF format. 216 + 217 + ```bash 218 + mlf generate mlf -i <INPUT> -o <OUTPUT> 219 + ``` 220 + 221 + **Options:** 222 + - `-i, --input <INPUT>` - Input JSON lexicon files (glob patterns supported, can be specified multiple times) 223 + - `-o, --output <OUTPUT>` - Output directory (required) 224 + 225 + **Examples:** 226 + 227 + ```bash 228 + # Convert a single JSON lexicon to MLF 229 + mlf generate mlf -i com.example.thread.json -o ./lexicons/ 230 + # Creates: lexicons/com/example/thread.mlf 231 + 232 + # Convert multiple JSON lexicons 233 + mlf generate mlf -i lexicon1.json -i lexicon2.json -o ./mlf/ 234 + 235 + # Convert using glob pattern 236 + mlf generate mlf -i "dist/lexicons/**/*.json" -o ./src/ 237 + ``` 238 + 239 + **Features:** 240 + 241 + - **Smart type conversion** - Automatically converts format strings to prelude types 242 + - `"format": "did"` → `Did` 243 + - `"format": "datetime"` → `Datetime` 244 + - `"format": "handle"` → `Handle` 245 + - **Proper formatting** - Generates clean, properly indented MLF 246 + - **Reference conversion** - Converts `namespace#name` to `namespace.name` 247 + - **Complete coverage** - Supports all ATProto lexicon types 248 + 249 + **Input JSON:** 250 + ```json 251 + { 252 + "lexicon": 1, 253 + "id": "com.example.thread", 254 + "defs": { 255 + "main": { 256 + "type": "record", 257 + "record": { 258 + "type": "object", 259 + "required": ["title", "createdAt"], 260 + "properties": { 261 + "title": { 262 + "type": "string", 263 + "maxLength": 200 264 + }, 265 + "createdAt": { 266 + "type": "string", 267 + "format": "datetime" 268 + } 269 + } 270 + } 271 + } 272 + } 273 + } 274 + ``` 275 + 276 + **Generated MLF:** 277 + ```mlf 278 + record main { 279 + title!: string constrained { 280 + maxLength: 200, 281 + }, 282 + createdAt!: Datetime, 283 + }; 284 + ``` 285 + 286 + --- 287 + 288 + ## Code Generator Features 289 + 290 + ### Documentation Comments 291 + 292 + All generators preserve documentation comments from MLF: 293 + 294 + ```mlf 295 + /// This is a user profile 296 + def Profile = { 297 + /// The user's display name 298 + displayName: string, 299 + }; 300 + ``` 301 + 302 + Generates appropriate doc comments for each language (JSDoc, Go comments, Rust doc comments). 303 + 304 + ### Optional Fields 305 + 306 + Optional fields are handled idiomatically in each language: 307 + 308 + - **TypeScript**: `field?: Type` 309 + - **Go**: `Field *Type` or `json:",omitempty"` 310 + - **Rust**: `field: Option<Type>` with `#[serde(skip_serializing_if = "Option::is_none")]` 311 + 312 + ### Field Naming Conventions 313 + 314 + Each generator follows language conventions: 315 + 316 + - **TypeScript**: camelCase (matches JSON) 317 + - **Go**: PascalCase with JSON tags 318 + - **Rust**: snake_case with `#[serde(rename)]` attributes 319 + 320 + ### Type Mapping 321 + 322 + MLF types are mapped to appropriate language types: 323 + 324 + | MLF Type | TypeScript | Go | Rust | 325 + |----------|------------|-------|------| 326 + | `string` | `string` | `string` | `String` | 327 + | `integer` | `number` | `int64` | `i64` | 328 + | `boolean` | `boolean` | `bool` | `bool` | 329 + | `bytes` | `Uint8Array` | `[]byte` | `Vec<u8>` | 330 + | `array` | `T[]` | `[]T` | `Vec<T>` | 331 + | `Did` | `string` | `string` | `String` | 332 + | `Datetime` | `string` | `string` | `String` | 333 + 334 + --- 335 + 336 + ## Tips 337 + 338 + 1. **Use configuration** - Set up `mlf.toml` for multi-output generation 339 + 2. **Commit generated code** - If it's part of your build artifacts 340 + 3. **Regenerate often** - Run `mlf generate` after any lexicon changes 341 + 4. **Use flat mode** - For simpler directory structures 342 + 5. **Multiple generators** - Generate multiple languages from the same MLF files 343 + 6. **Version control** - Track both MLF source and generated code 344 + 345 + ## Error Handling 346 + 347 + If generation fails for some files, you'll see detailed errors: 348 + 349 + ``` 350 + 3 file(s) generated successfully, 1 error(s) encountered: 351 + 352 + thread.mlf - Type resolution error: undefined reference to 'Post' 353 + ``` 354 + 355 + The command exits with status `1` if any errors occur.
+307
website/content/docs/cli/06-fetch.md
··· 1 + +++ 2 + title = "Fetch Command" 3 + description = "Download lexicons from remote repositories" 4 + weight = 6 5 + +++ 6 + 7 + The `mlf fetch` command downloads ATProto lexicons from remote repositories and converts them to MLF format. 8 + 9 + ## Usage 10 + 11 + ```bash 12 + # Fetch all dependencies from mlf.toml 13 + mlf fetch 14 + 15 + # Fetch a specific namespace 16 + mlf fetch <NAMESPACE> 17 + 18 + # Fetch and save to dependencies 19 + mlf fetch <NAMESPACE> --save 20 + ``` 21 + 22 + **Arguments:** 23 + - `[NAMESPACE]` - Optional namespace to fetch (e.g., `stream.place`, `app.bsky`) 24 + 25 + **Options:** 26 + - `--save` - Add the namespace to dependencies in `mlf.toml` 27 + 28 + ## How It Works 29 + 30 + The fetch command follows the ATProto lexicon discovery protocol: 31 + 32 + 1. **DNS Lookup** - Queries `_lexicon.<reversed-authority>` TXT record 33 + 2. **DID Resolution** - Resolves the DID to a PDS endpoint 34 + 3. **Fetch Records** - Queries `com.atproto.repo.listRecords` for lexicon schemas 35 + 4. **Save & Convert** - Saves JSON and converts to MLF format 36 + 37 + ## Examples 38 + 39 + ### Fetch All Dependencies 40 + 41 + With an `mlf.toml` file: 42 + 43 + ```toml 44 + [dependencies] 45 + dependencies = ["stream.place", "app.bsky"] 46 + ``` 47 + 48 + Run: 49 + 50 + ```bash 51 + mlf fetch 52 + ``` 53 + 54 + **Output:** 55 + ``` 56 + Fetching 2 dependencies... 57 + 58 + Fetching: stream.place 59 + Fetching lexicons for authority: stream.place 60 + → Resolved DID: did:web:stream.place 61 + → Using PDS: https://stream.place 62 + → Found 5 lexicon record(s) 63 + Processing: stream.place.thread 64 + → Saved JSON to .mlf/lexicons/json/stream.place.thread.json 65 + → Converted to MLF at .mlf/lexicons/mlf/stream.place.thread.mlf 66 + ✓ Successfully fetched lexicons for stream.place 67 + 68 + Fetching: app.bsky 69 + ... 70 + 71 + ✓ Successfully fetched all 2 dependencies 72 + ``` 73 + 74 + ### Fetch Specific Namespace 75 + 76 + ```bash 77 + mlf fetch stream.place 78 + ``` 79 + 80 + This downloads all lexicons under the `stream.place` authority. 81 + 82 + ### Fetch and Save 83 + 84 + ```bash 85 + mlf fetch stream.place --save 86 + ``` 87 + 88 + This: 89 + 1. Downloads the lexicons 90 + 2. Adds `"stream.place"` to the dependencies array in `mlf.toml` 91 + 3. Creates `mlf.toml` if it doesn't exist 92 + 93 + ## Storage Structure 94 + 95 + Fetched lexicons are stored in `.mlf/lexicons/`: 96 + 97 + ``` 98 + .mlf/ 99 + ├── .gitignore # Auto-generated 100 + ├── .lexicon-cache.toml # Cache metadata 101 + └── lexicons/ 102 + ├── json/ # Original JSON lexicons 103 + │ ├── stream.place.thread.json 104 + │ └── app.bsky.actor.profile.json 105 + └── mlf/ # Converted MLF format 106 + ├── stream.place.thread.mlf 107 + └── app.bsky.actor.profile.mlf 108 + ``` 109 + 110 + ### Cache File 111 + 112 + The `.lexicon-cache.toml` tracks what's been fetched: 113 + 114 + ```toml 115 + [[lexicons.stream.place.thread]] 116 + nsid = "stream.place.thread" 117 + fetched_at = "2024-01-15T10:30:00Z" 118 + did = "did:web:stream.place" 119 + ``` 120 + 121 + ## DNS Resolution 122 + 123 + For a namespace like `stream.place.thread`: 124 + 125 + 1. Extract authority: `stream.place` 126 + 2. Reverse for DNS: `place.stream` 127 + 3. Query TXT record: `_lexicon.place.stream` 128 + 4. Parse `did=did:web:...` or `did=did:plc:...` 129 + 130 + **Example DNS record:** 131 + ``` 132 + _lexicon.place.stream. 300 IN TXT "did=did:web:stream.place" 133 + ``` 134 + 135 + ## DID Resolution 136 + 137 + ### did:web 138 + 139 + For `did:web:stream.place`, the PDS is `https://stream.place` 140 + 141 + ### did:plc 142 + 143 + For `did:plc:abc123...`, query `https://plc.directory/did:plc:abc123...` to get the PDS endpoint from the DID document. 144 + 145 + ## Fetched Lexicons in Your Code 146 + 147 + Once fetched, lexicons in `.mlf/lexicons/mlf/` are automatically available for: 148 + 149 + ### Type References 150 + 151 + ```mlf 152 + use stream.place.thread; 153 + 154 + def Reply = { 155 + thread!: stream.place.thread, 156 + text!: string, 157 + }; 158 + ``` 159 + 160 + ### Code Generation 161 + 162 + ```bash 163 + mlf generate code -g typescript -i my-lexicon.mlf -o src/types/ 164 + ``` 165 + 166 + The generator can resolve references to fetched lexicons. 167 + 168 + ### Validation 169 + 170 + ```bash 171 + mlf check my-lexicon.mlf 172 + ``` 173 + 174 + The check command loads fetched lexicons for type resolution. 175 + 176 + ## Working Without mlf.toml 177 + 178 + If you don't have an `mlf.toml`, the fetch command will offer to create one: 179 + 180 + ```bash 181 + $ mlf fetch stream.place 182 + No mlf.toml found in current or parent directories. 183 + Would you like to create one in the current directory? (y/n) 184 + y 185 + Created mlf.toml in /path/to/current/dir 186 + ... 187 + ``` 188 + 189 + ## Re-fetching 190 + 191 + If a namespace is already cached, fetch skips it: 192 + 193 + ```bash 194 + $ mlf fetch stream.place 195 + Lexicon 'stream.place.thread' is already cached. Skipping fetch. 196 + (Use --force to re-fetch) 197 + ``` 198 + 199 + To re-fetch: 200 + 201 + ```bash 202 + mlf fetch stream.place --force # Not yet implemented 203 + ``` 204 + 205 + ## Error Handling 206 + 207 + ### DNS Errors 208 + 209 + ``` 210 + ✗ DNS lookup failed: No TXT record found for _lexicon.place.stream 211 + ``` 212 + 213 + **Causes:** 214 + - Domain doesn't have a lexicon TXT record 215 + - DNS propagation delay 216 + - Network issues 217 + 218 + ### DID Resolution Errors 219 + 220 + ``` 221 + ✗ Failed to resolve DID: No PDS endpoint found in DID document 222 + ``` 223 + 224 + **Causes:** 225 + - Invalid DID format 226 + - PLC directory unreachable 227 + - DID document missing PDS service 228 + 229 + ### No Records Found 230 + 231 + ``` 232 + ✗ No lexicon records found for stream.place 233 + ``` 234 + 235 + **Causes:** 236 + - Namespace exists but has no published lexicons 237 + - Wrong namespace (typo) 238 + - PDS doesn't support lexicon publishing 239 + 240 + ## Best Practices 241 + 242 + 1. **Fetch before work** - Always fetch dependencies before coding 243 + 2. **Use --save** - Keep `mlf.toml` up to date with dependencies 244 + 3. **Don't commit `.mlf/`** - Let each developer fetch independently 245 + 4. **Check DNS** - Verify TXT records before fetching 246 + 5. **Version dependencies** - Consider tracking lexicon versions (future feature) 247 + 248 + ## CI/CD Integration 249 + 250 + In your CI pipeline: 251 + 252 + ```yaml 253 + # GitHub Actions example 254 + - name: Fetch ATProto Lexicons 255 + run: | 256 + mlf fetch 257 + 258 + - name: Generate Code 259 + run: | 260 + mlf generate 261 + ``` 262 + 263 + This ensures builds have access to the latest lexicons. 264 + 265 + ## Comparison with npm/cargo 266 + 267 + The fetch command is similar to package managers: 268 + 269 + | Command | npm | cargo | mlf | 270 + |---------|-----|-------|-----| 271 + | Install deps | `npm install` | `cargo fetch` | `mlf fetch` | 272 + | Add dep | `npm install pkg --save` | `cargo add pkg` | `mlf fetch ns --save` | 273 + | Config file | `package.json` | `Cargo.toml` | `mlf.toml` | 274 + | Cache | `node_modules/` | `~/.cargo/` | `.mlf/` | 275 + 276 + ## Troubleshooting 277 + 278 + ### Network Issues 279 + 280 + ```bash 281 + # Check DNS resolution 282 + dig TXT _lexicon.place.stream 283 + 284 + # Test DID resolution 285 + curl https://plc.directory/did:plc:abc123 286 + ``` 287 + 288 + ### Invalid Namespace 289 + 290 + Make sure you're using the correct namespace format: 291 + - ✓ `stream.place` 292 + - ✓ `app.bsky` 293 + - ✗ `stream.place.thread` (too specific) 294 + 295 + ### Permission Errors 296 + 297 + Ensure you have write permissions for the project directory to create `.mlf/`. 298 + 299 + ## Future Features 300 + 301 + Planned enhancements: 302 + 303 + - `--force` flag to re-fetch cached lexicons 304 + - Version pinning (fetch specific lexicon versions) 305 + - Private/authenticated repositories 306 + - Offline mode with local cache 307 + - Fetch from local directories
+335
website/content/docs/cli/07-errors.md
··· 1 + +++ 2 + title = "Error Messages" 3 + description = "Understanding MLF error diagnostics" 4 + weight = 7 5 + +++ 6 + 7 + MLF provides rich, helpful error messages with source code context and suggestions for fixing issues. 8 + 9 + ## Error Format 10 + 11 + All MLF errors follow this format: 12 + 13 + ``` 14 + × Error title 15 + ╭─[file.mlf:5:12] 16 + 5 │ author: ProfileView, 17 + · ^^^^^^^^^^^ error message 18 + ╰──── 19 + help: Suggestion for fixing the issue 20 + ``` 21 + 22 + **Components:** 23 + - **× Error title** - Brief description of the error 24 + - **Source location** - File name, line, and column 25 + - **Source context** - The relevant code snippet 26 + - **Error span** - Highlighted location with `^^^` 27 + - **help:** - Actionable suggestion for fixing 28 + 29 + ## Common Errors 30 + 31 + ### Parse Errors 32 + 33 + #### Expected Token 34 + 35 + ``` 36 + × Expected '}', found ',' 37 + ╭─[thread.mlf:10:5] 38 + 10 │ title: string, 39 + · ^ expected '}' 40 + ╰──── 41 + help: Check for missing closing braces or extra commas 42 + ``` 43 + 44 + **Cause:** Syntax error in MLF code 45 + 46 + **Fix:** Correct the syntax according to MLF grammar 47 + 48 + #### Invalid Identifier 49 + 50 + ``` 51 + × Invalid identifier: '123invalid' 52 + ╭─[thread.mlf:5:5] 53 + 5 │ 123invalid: string, 54 + · ^^^^^^^^^^ identifiers cannot start with numbers 55 + ╰──── 56 + help: Identifiers must start with a letter or underscore 57 + ``` 58 + 59 + **Cause:** Identifier doesn't follow naming rules 60 + 61 + **Fix:** Start identifiers with letters or underscores 62 + 63 + ### Type Errors 64 + 65 + #### Undefined Reference 66 + 67 + ``` 68 + × Undefined reference to 'ProfileView' 69 + ╭─[thread.mlf:8:12] 70 + 8 │ author: ProfileView, 71 + · ^^^^^^^^^^^ 'ProfileView' is not defined 72 + ╰──── 73 + help: Make sure this type is defined in the same file or imported via 'use'. 74 + ``` 75 + 76 + **Causes:** 77 + - Type is not defined 78 + - Type is in another file and not imported 79 + - Typo in type name 80 + 81 + **Fixes:** 82 + - Define the type in the same file 83 + - Use `mlf fetch` to download external lexicons 84 + - Add a `use` statement (if MLF supports imports) 85 + - Check spelling 86 + 87 + #### Type Mismatch 88 + 89 + ``` 90 + × Type mismatch: expected integer, found string 91 + ╭─[thread.mlf:12:15] 92 + 12 │ count: string, 93 + · ^^^^^^ expected integer type 94 + ╰──── 95 + help: Change this to 'integer' or update the constraint 96 + ``` 97 + 98 + **Cause:** Field type doesn't match its definition 99 + 100 + **Fix:** Update the type to match the schema 101 + 102 + ### Constraint Errors 103 + 104 + #### Invalid Constraint 105 + 106 + ``` 107 + × Invalid constraint for type 'integer': 'maxLength' 108 + ╭─[thread.mlf:15:9] 109 + 15 │ maxLength: 100, 110 + · ^^^^^^^^^ 'maxLength' is only valid for string types 111 + ╰──── 112 + help: Use 'maximum' for integer constraints 113 + ``` 114 + 115 + **Cause:** Constraint doesn't apply to the type 116 + 117 + **Fix:** Use the correct constraint for the type: 118 + - String: `maxLength`, `minLength`, `maxGraphemes`, `minGraphemes` 119 + - Integer: `minimum`, `maximum` 120 + - Array: `maxLength`, `minLength` 121 + 122 + #### Constraint Value Error 123 + 124 + ``` 125 + × Constraint value must be positive 126 + ╭─[thread.mlf:18:20] 127 + 18 │ maxLength: -10, 128 + · ^^^ negative value not allowed 129 + ╰──── 130 + help: Use a positive integer value 131 + ``` 132 + 133 + **Cause:** Constraint value is invalid 134 + 135 + **Fix:** Use a valid value according to the constraint rules 136 + 137 + ### Record Errors 138 + 139 + #### Missing Record Key 140 + 141 + ``` 142 + × Record definition must specify a key type 143 + ╭─[thread.mlf:3:1] 144 + 3 │ record main { 145 + · ^^^^^^^^^^^ missing key specification 146 + ╰──── 147 + help: Add 'key: "tid"' or another valid key type to the record 148 + ``` 149 + 150 + **Cause:** Record doesn't specify how records are keyed 151 + 152 + **Fix:** Add a key specification to the record definition 153 + 154 + ### XRPC Errors 155 + 156 + #### Invalid Response Code 157 + 158 + ``` 159 + × Invalid HTTP response code: 999 160 + ╭─[api.mlf:25:5] 161 + 25 │ 999: error, 162 + · ^^^ response code must be between 200-599 163 + ╰──── 164 + help: Use a valid HTTP status code 165 + ``` 166 + 167 + **Cause:** Invalid HTTP status code in query/procedure 168 + 169 + **Fix:** Use standard HTTP status codes (200, 400, 401, 404, 500, etc.) 170 + 171 + #### Missing Required Response 172 + 173 + ``` 174 + × Query/procedure must define at least one success response 175 + ╭─[api.mlf:20:1] 176 + 20 │ query getProfile(...) { 177 + · ^^^^^^^^^^^^^^^^^^^^^ no 200-level responses defined 178 + ╰──── 179 + help: Add at least one 2xx response (typically 200) 180 + ``` 181 + 182 + **Cause:** XRPC method has no success responses 183 + 184 + **Fix:** Add a 200-level response 185 + 186 + ### Union Errors 187 + 188 + #### Empty Union 189 + 190 + ``` 191 + × Union must have at least one type 192 + ╭─[thread.mlf:30:15] 193 + 30 │ data: unit | , 194 + · ^ empty union 195 + ╰──── 196 + help: Add at least one type to the union 197 + ``` 198 + 199 + **Cause:** Union has no types 200 + 201 + **Fix:** Add types to the union: `string | integer` 202 + 203 + #### Duplicate Union Types 204 + 205 + ``` 206 + × Duplicate type in union: 'string' 207 + ╭─[thread.mlf:32:20] 208 + 32 │ data: string | string, 209 + · ^^^^^^ duplicate type 210 + ╰──── 211 + help: Remove duplicate types from the union 212 + ``` 213 + 214 + **Cause:** Same type appears multiple times in union 215 + 216 + **Fix:** Remove duplicates 217 + 218 + ## Validation Errors 219 + 220 + ### Record Validation 221 + 222 + ``` 223 + ✗ Record validation failed with 2 error(s): 224 + • Field 'title' is required but missing 225 + • Field 'count': expected integer, found "123" 226 + ``` 227 + 228 + **Cause:** JSON record doesn't match lexicon schema 229 + 230 + **Fix:** Update the JSON to match the schema 231 + 232 + ## Generation Errors 233 + 234 + ### File Read Error 235 + 236 + ``` 237 + Failed to read file: permission denied 238 + ``` 239 + 240 + **Cause:** Cannot read input file 241 + 242 + **Fix:** Check file permissions and path 243 + 244 + ### Type Resolution Error 245 + 246 + ``` 247 + Type resolution error: circular reference detected 248 + ``` 249 + 250 + **Cause:** Types reference each other in a cycle 251 + 252 + **Fix:** Break the circular dependency 253 + 254 + ### Code Generation Error 255 + 256 + ``` 257 + Generator 'typescript' not found 258 + ``` 259 + 260 + **Cause:** Generator feature not enabled 261 + 262 + **Fix:** Install with `--features typescript` or use `--all-features` 263 + 264 + ## Fetch Errors 265 + 266 + ### DNS Error 267 + 268 + ``` 269 + ✗ DNS lookup failed: No TXT record found for _lexicon.place.stream 270 + ``` 271 + 272 + **Cause:** Domain has no lexicon TXT record 273 + 274 + **Fix:** Verify the namespace is correct and has published lexicons 275 + 276 + ### Network Error 277 + 278 + ``` 279 + ✗ Failed to fetch lexicon records: connection timeout 280 + ``` 281 + 282 + **Cause:** Network connectivity issues 283 + 284 + **Fix:** Check internet connection and try again 285 + 286 + ### Parse Error 287 + 288 + ``` 289 + ✗ Failed to parse lexicon JSON: unexpected end of input 290 + ``` 291 + 292 + **Cause:** Malformed JSON from remote repository 293 + 294 + **Fix:** Report issue to namespace maintainer 295 + 296 + ## Exit Codes 297 + 298 + All MLF commands use standard exit codes: 299 + 300 + - `0` - Success 301 + - `1` - Error occurred 302 + 303 + Use in scripts: 304 + 305 + ```bash 306 + if mlf check; then 307 + echo "✓ Valid" 308 + else 309 + echo "✗ Invalid" 310 + exit 1 311 + fi 312 + ``` 313 + 314 + ## Tips for Reading Errors 315 + 316 + 1. **Read the title** - Gives you the high-level issue 317 + 2. **Check the location** - Find the exact line and column 318 + 3. **Examine the span** - See what code is problematic 319 + 4. **Follow the help** - Actionable suggestions for fixes 320 + 5. **Look for patterns** - Similar errors often have similar fixes 321 + 322 + ## Reporting Bugs 323 + 324 + If you encounter an error that seems like a bug: 325 + 326 + 1. Note the exact error message 327 + 2. Create a minimal reproduction case 328 + 3. Check if it's a known issue 329 + 4. Report at: https://github.com/anthropics/claude-code/issues 330 + 331 + Include: 332 + - MLF version (`mlf --version`) 333 + - Complete error message 334 + - Minimal MLF code that reproduces the issue 335 + - Expected behavior vs actual behavior
+60
website/content/docs/cli/_index.md
··· 1 + +++ 2 + title = "CLI Reference" 3 + description = "Command-line tool documentation" 4 + weight = 3 5 + sort_by = "weight" 6 + template = "section.html" 7 + +++ 8 + 9 + The MLF CLI is a powerful command-line tool for working with Matt's Lexicon Format and ATProto lexicons. 10 + 11 + ## Overview 12 + 13 + The `mlf` command-line tool provides: 14 + 15 + - **Validation** - Check MLF files for syntax and type errors 16 + - **Code Generation** - Generate TypeScript, Go, Rust, and JSON from MLF 17 + - **Lexicon Fetching** - Download lexicons from remote ATProto repositories 18 + - **Format Conversion** - Convert between MLF and JSON lexicon formats 19 + - **Project Management** - Configure projects with `mlf.toml` 20 + 21 + ## Quick Start 22 + 23 + ```bash 24 + # Check MLF files 25 + mlf check 26 + 27 + # Generate TypeScript types 28 + mlf generate code -g typescript -i "lexicons/**/*.mlf" -o src/types/ 29 + 30 + # Fetch remote lexicons 31 + mlf fetch stream.place --save 32 + 33 + # Generate all configured outputs 34 + mlf generate 35 + ``` 36 + 37 + ## Project Configuration 38 + 39 + Projects can be configured with an `mlf.toml` file that specifies source directories, output configurations, and dependencies. This allows you to run commands without repeatedly specifying arguments. 40 + 41 + See the [Configuration](02-configuration.md) section for details. 42 + 43 + ## Command Categories 44 + 45 + ### Validation Commands 46 + - `mlf check` - Validate MLF syntax and types 47 + - `mlf validate` - Validate JSON records against lexicons 48 + 49 + ### Generation Commands 50 + - `mlf generate lexicon` - Generate JSON lexicons 51 + - `mlf generate code` - Generate code in various languages 52 + - `mlf generate mlf` - Convert JSON lexicons to MLF 53 + - `mlf generate` (no subcommand) - Generate all configured outputs 54 + 55 + ### Fetching Commands 56 + - `mlf fetch` - Download lexicons from remote repositories 57 + - `mlf fetch <namespace>` - Fetch specific namespace 58 + - `mlf fetch <namespace> --save` - Fetch and add to dependencies 59 + 60 + This guide walks through each command in detail, with examples and usage patterns.