An experimental TypeSpec syntax for Lexicon
more
+140
-190
-117
IMPROVEMENTS.md
-117
IMPROVEMENTS.md
···
1
-
# tlex Emitter Improvement Plan
2
-
3
-
## Philosophy
4
-
Keep TypeSpec-first design. Use native TypeSpec features where possible, add Lexicon-specific decorators only where necessary. Ensure all schemas can potentially work with other emitters.
5
-
6
-
## Immediate Improvements (High Impact)
7
-
8
-
### 1. Use TypeSpec stdlib `@minItems`/`@maxItems` ✅
9
-
**Current:** Using custom `@maxItems` import from compiler
10
-
**Change:** Import and use from TypeSpec standard library consistently
11
-
**Files:**
12
-
- `src/emitter.ts` - Update imports
13
-
- Remove custom implementation if any
14
-
**Test Impact:** None (already working)
15
-
16
-
### 2. Add Default Value Support
17
-
**Current:** TypeSpec default syntax (`name?: string = "Rex"`) not mapped
18
-
**Change:** Emit Lexicon `default` field from TypeSpec default values
19
-
**Files:**
20
-
- `src/emitter.ts` - Read default values from ModelProperty
21
-
- Check for `prop.default` and emit to lexicon
22
-
**Test Impact:** Need tests with defaults (but won't match existing lexicons yet)
23
-
**Lexicon Support:** boolean, integer, string have `default` field
24
-
25
-
### 3. Add Enum Constraint Support
26
-
**Current:** No way to express closed enums on primitives
27
-
**Change:** Detect TypeSpec enum types and emit Lexicon `enum` field
28
-
**Files:**
29
-
- `src/emitter.ts` - Check if type is Enum, extract values
30
-
**Test Impact:** Need new test scenarios
31
-
**Lexicon Support:** integer, string have `enum` field (array of values)
32
-
33
-
### 4. Add Nullable Support
34
-
**Current:** Not handling `T | null` unions
35
-
**Change:** Detect unions with null, emit to `nullable` array on objects
36
-
**Files:**
37
-
- `src/emitter.ts` - Check union variants for null type
38
-
- Add `nullable` field to object schema
39
-
**Test Impact:** Need new test scenarios
40
-
**Lexicon Support:** objects have `nullable: string[]` field
41
-
42
-
## High Value Improvements
43
-
44
-
### 5. Provide Pre-defined Format Scalars
45
-
**Status:** ✅ DONE
46
-
**Implementation:** Pre-defined format scalars added (did, handle, atUri, datetime, cid, cidLink, etc.)
47
-
**Files:**
48
-
- `lib/main.tsp` - Scalar definitions
49
-
- `src/emitter.ts` - Detects and maps to format automatically
50
-
51
-
### 6. Add Const Support for String/Integer
52
-
**Current:** Only `@lexConst` for boolean
53
-
**Change:** Extend to support string and integer const values
54
-
**Files:**
55
-
- `lib/decorators.tsp` - Make `@lexConst` accept any value
56
-
- `src/decorators.ts` - Handle string/integer
57
-
- `src/emitter.ts` - Emit const on string/integer primitives
58
-
**Test Impact:** Need new test scenarios
59
-
**Lexicon Support:** string, integer, boolean all support `const`
60
-
61
-
## Nice to Have (Lower Priority)
62
-
63
-
### 7. Native DateTime Type Mapping
64
-
**Status:** ✅ DONE (via `datetime` scalar)
65
-
**Implementation:** Use `datetime` scalar instead of `@lexFormat("datetime")`
66
-
67
-
### 8. Token Type Support
68
-
**When needed:** Implement token pattern
69
-
**Likely approach:** `@token` decorator on model or scalar
70
-
**Not implementing yet:** Wait until we port a lexicon that uses tokens
71
-
72
-
## Future Major Features
73
-
74
-
### 9. XRPC Query/Procedure/Subscription Support
75
-
**Approach:** Use TypeSpec operations with `@typespec/http` decorators
76
-
**Example:**
77
-
```typespec
78
-
import "@typespec/http";
79
-
80
-
@route("/xrpc/app.bsky.actor.getProfile")
81
-
@get
82
-
op getProfile(@query actor: atIdentifier): ProfileViewDetailed;
83
-
```
84
-
**Not implementing yet:** Focus on record types first
85
-
86
-
## Implementation Order
87
-
88
-
### Phase 1: Core Improvements (Keep tests green)
89
-
1. ✅ Stdlib `@minItems`/`@maxItems` - Quick win, no test changes
90
-
2. Default values - Add support, test separately
91
-
3. Enum constraints - Add support, test separately
92
-
4. Nullable support - Add support, test separately
93
-
94
-
### Phase 2: Developer Experience
95
-
5. Pre-defined format scalars - Big DX improvement
96
-
6. Extend const to string/integer - Complete the feature
97
-
98
-
### Phase 3: Future (when needed)
99
-
7. Native datetime mapping
100
-
8. Token types
101
-
9. XRPC operations
102
-
103
-
## Testing Strategy
104
-
105
-
- Keep ALL existing tests green
106
-
- Don't modify output fixtures
107
-
- Add NEW test scenarios for new features
108
-
- Test features in isolation first
109
-
- Combine features once all work independently
110
-
111
-
## Success Criteria
112
-
113
-
- All 23 existing tests remain green
114
-
- New features have test coverage
115
-
- Code is more idiomatic TypeSpec
116
-
- Better developer experience with type-safe scalars
117
-
- Can still emit identical JSON to atproto/lexicons for ported schemas
+54
SYNTAX.md
+54
SYNTAX.md
···
307
307
</td></tr>
308
308
</table>
309
309
310
+
### Constant Values
311
+
312
+
Use `@readOnly` with a default value to create constant fields:
313
+
314
+
<table>
315
+
<tr><th>TypeSpec</th><th>Lexicon JSON</th></tr>
316
+
<tr><td>
317
+
318
+
```typespec
319
+
model NotFoundPost {
320
+
@required uri: atUri;
321
+
322
+
@readOnly
323
+
@required
324
+
notFound: boolean = true;
325
+
}
326
+
327
+
model Config {
328
+
@readOnly
329
+
version: string = "1.0";
330
+
331
+
@readOnly
332
+
maxRetries: integer = 3;
333
+
}
334
+
```
335
+
336
+
</td><td>
337
+
338
+
```json
339
+
{
340
+
"notFoundPost": {
341
+
"type": "object",
342
+
"required": ["uri", "notFound"],
343
+
"properties": {
344
+
"uri": { "type": "string", "format": "at-uri" },
345
+
"notFound": { "type": "boolean", "const": true }
346
+
}
347
+
},
348
+
"config": {
349
+
"type": "object",
350
+
"properties": {
351
+
"version": { "type": "string", "const": "1.0" },
352
+
"maxRetries": { "type": "integer", "const": 3 }
353
+
}
354
+
}
355
+
}
356
+
```
357
+
358
+
</td></tr>
359
+
</table>
360
+
361
+
**Important:** `@readOnly` emits `const` (not `default`). Only valid for `string`, `boolean`, and `integer` types.
362
+
310
363
## Arrays
311
364
312
365
<table>
···
1686
1739
| `"minimum": N` / `"maximum": N` | `@minValue(N)` / `@maxValue(N)` | Number constraints |
1687
1740
| `"minLength": N` / `"maxLength": N` (array) | `@minItems(N)` / `@maxItems(N)` | Array constraints |
1688
1741
| `"default": V` | `field?: T = V` | Default value |
1742
+
| `"const": V` | `@readOnly field: T = V` | Constant value (string/boolean/integer only) |
+14
-4
packages/emitter/lib/decorators.tsp
+14
-4
packages/emitter/lib/decorators.tsp
···
17
17
extern dec minBytes(target: unknown, value: valueof int32);
18
18
19
19
/**
20
-
* Marks a field as having a constant value.
21
-
* Maps to lexicon's "const" field.
22
-
* Supports boolean, string, and integer values.
20
+
* Marks a field as read-only with a constant value.
21
+
* Must be used with a default value assignment.
22
+
* Maps to lexicon's "const" field (does NOT emit "default").
23
+
* Only valid for string, boolean, and integer types.
24
+
*
25
+
* @example
26
+
* ```typespec
27
+
* @readOnly
28
+
* foo: string = "fixed-value";
29
+
*
30
+
* @readOnly
31
+
* bar: boolean = true;
32
+
* ```
23
33
*/
24
-
extern dec lexConst(target: unknown, value: unknown);
34
+
extern dec readOnly(target: unknown);
25
35
26
36
/**
27
37
* Specifies an enum of allowed values for a field.
+14
-30
packages/emitter/src/decorators.ts
+14
-30
packages/emitter/src/decorators.ts
···
3
3
4
4
const maxGraphemesKey = Symbol("maxGraphemes");
5
5
const minGraphemesKey = Symbol("minGraphemes");
6
-
const constKey = Symbol("const");
7
6
const enumKey = Symbol("enum");
8
7
const recordKey = Symbol("record");
9
8
const blobKey = Symbol("blob");
10
9
const requiredKey = Symbol("required");
10
+
const readOnlyKey = Symbol("readOnly");
11
11
const tokenKey = Symbol("token");
12
12
const closedKey = Symbol("closed");
13
13
const queryKey = Symbol("query");
···
124
124
target: Type,
125
125
): number | undefined {
126
126
return program.stateMap(minGraphemesKey).get(target);
127
-
}
128
-
129
-
/**
130
-
* @lexConst decorator for constant values (boolean, string, or integer)
131
-
*/
132
-
export function $lexConst(
133
-
context: DecoratorContext,
134
-
target: Type,
135
-
value: Type,
136
-
) {
137
-
const valueAny = value as any;
138
-
139
-
// Handle different value types
140
-
if (valueAny.kind === "Boolean") {
141
-
context.program.stateMap(constKey).set(target, valueAny.value);
142
-
} else if (valueAny.kind === "String") {
143
-
context.program.stateMap(constKey).set(target, valueAny.value);
144
-
} else if (valueAny.kind === "Number" || valueAny.kind === "Numeric") {
145
-
context.program.stateMap(constKey).set(target, valueAny.value);
146
-
} else {
147
-
context.program.stateMap(constKey).set(target, value);
148
-
}
149
-
}
150
-
151
-
export function getLexConst(
152
-
program: Program,
153
-
target: Type,
154
-
): boolean | string | number | undefined {
155
-
return program.stateMap(constKey).get(target);
156
127
}
157
128
158
129
/**
···
355
326
export function isInline(program: Program, target: Type): boolean {
356
327
return program.stateSet(inlineKey).has(target);
357
328
}
329
+
330
+
/**
331
+
* @readOnly decorator for marking fields with constant values
332
+
* Used with default values to represent lexicon "const" field.
333
+
* Only valid for string, boolean, and integer types.
334
+
*/
335
+
export function $readOnly(context: DecoratorContext, target: Type) {
336
+
context.program.stateSet(readOnlyKey).add(target);
337
+
}
338
+
339
+
export function isReadOnly(program: Program, target: Type): boolean {
340
+
return program.stateSet(readOnlyKey).has(target);
341
+
}
+35
-16
packages/emitter/src/emitter.ts
+35
-16
packages/emitter/src/emitter.ts
···
32
32
import {
33
33
getMaxGraphemes,
34
34
getMinGraphemes,
35
-
getLexConst,
36
35
getLexEnum,
37
36
getRecordKey,
38
37
isBlob,
39
38
isRequired,
39
+
isReadOnly,
40
40
isToken,
41
41
isClosed,
42
42
isQuery,
···
1079
1079
}
1080
1080
1081
1081
private applyPropertyMetadata(primitive: any, prop: ModelProperty) {
1082
-
// Apply const value
1083
-
const constValue = getLexConst(this.program, prop);
1084
-
if (
1085
-
constValue !== undefined &&
1086
-
this.isValidConstForType(primitive.type, constValue)
1082
+
// Check if @readOnly is present
1083
+
const hasReadOnly = isReadOnly(this.program, prop);
1084
+
1085
+
// Apply default value as const if @readOnly is present
1086
+
const defaultValue = (prop as any).default?.value;
1087
+
if (hasReadOnly) {
1088
+
// Validate that readOnly is only used on string, boolean, or integer
1089
+
if (!this.isValidConstForType(primitive.type, defaultValue)) {
1090
+
this.program.reportDiagnostic({
1091
+
code: "invalid-readonly-type",
1092
+
severity: "error",
1093
+
message: `@readOnly is only valid for string, boolean, and integer types, but found type: ${primitive.type}`,
1094
+
target: prop,
1095
+
});
1096
+
return;
1097
+
}
1098
+
1099
+
if (defaultValue === undefined) {
1100
+
this.program.reportDiagnostic({
1101
+
code: "readonly-missing-default",
1102
+
severity: "error",
1103
+
message: "@readOnly requires a default value assignment",
1104
+
target: prop,
1105
+
});
1106
+
return;
1107
+
}
1108
+
1109
+
// Set const value from default, don't emit default field
1110
+
primitive.const = defaultValue;
1111
+
} else if (
1112
+
defaultValue !== undefined &&
1113
+
this.isValidDefaultForType(primitive.type, defaultValue)
1087
1114
) {
1088
-
primitive.const = constValue;
1115
+
// Normal default value (no @readOnly)
1116
+
primitive.default = defaultValue;
1089
1117
}
1090
1118
1091
1119
// Apply enum values
1092
1120
const enumValues = getLexEnum(this.program, prop);
1093
1121
if (enumValues !== undefined && enumValues.length > 0) {
1094
1122
primitive.enum = enumValues;
1095
-
}
1096
-
1097
-
// Apply default value
1098
-
const defaultValue = (prop as any).default?.value;
1099
-
if (
1100
-
defaultValue !== undefined &&
1101
-
this.isValidDefaultForType(primitive.type, defaultValue)
1102
-
) {
1103
-
primitive.default = defaultValue;
1104
1123
}
1105
1124
}
1106
1125
+1
-1
packages/emitter/src/index.ts
+1
-1
packages/emitter/src/index.ts
+2
-2
packages/emitter/src/tsp-index.ts
+2
-2
packages/emitter/src/tsp-index.ts
···
1
1
import {
2
2
$maxGraphemes,
3
3
$minGraphemes,
4
-
$lexConst,
5
4
$lexEnum,
6
5
$record,
7
6
$blob,
8
7
$required,
8
+
$readOnly,
9
9
$token,
10
10
$closed,
11
11
$query,
···
23
23
"": {
24
24
maxGraphemes: $maxGraphemes,
25
25
minGraphemes: $minGraphemes,
26
-
lexConst: $lexConst,
27
26
lexEnum: $lexEnum,
28
27
record: $record,
29
28
required: $required,
29
+
readOnly: $readOnly,
30
30
token: $token,
31
31
closed: $closed,
32
32
query: $query,
+6
-6
packages/emitter/test/integration/atproto/input/app/bsky/embed/record.tsp
+6
-6
packages/emitter/test/integration/atproto/input/app/bsky/embed/record.tsp
···
51
51
model ViewNotFound {
52
52
@required uri: atUri;
53
53
54
-
@lexConst(true)
54
+
@readOnly
55
55
@required
56
-
notFound: boolean;
56
+
notFound: boolean = true;
57
57
}
58
58
59
59
model ViewBlocked {
60
60
@required uri: atUri;
61
61
62
-
@lexConst(true)
62
+
@readOnly
63
63
@required
64
-
blocked: boolean;
64
+
blocked: boolean = true;
65
65
66
66
@required author: app.bsky.feed.defs.BlockedAuthor;
67
67
}
···
69
69
model ViewDetached {
70
70
@required uri: atUri;
71
71
72
-
@lexConst(true)
72
+
@readOnly
73
73
@required
74
-
detached: boolean;
74
+
detached: boolean = true;
75
75
}
76
76
}
+4
-4
packages/emitter/test/integration/atproto/input/app/bsky/feed/defs.tsp
+4
-4
packages/emitter/test/integration/atproto/input/app/bsky/feed/defs.tsp
···
92
92
model NotFoundPost {
93
93
@required uri: atUri;
94
94
95
-
@lexConst(true)
95
+
@readOnly
96
96
@required
97
-
notFound: boolean;
97
+
notFound: boolean = true;
98
98
}
99
99
100
100
model BlockedPost {
101
101
@required uri: atUri;
102
102
103
-
@lexConst(true)
103
+
@readOnly
104
104
@required
105
-
blocked: boolean;
105
+
blocked: boolean = true;
106
106
107
107
@required author: BlockedAuthor;
108
108
}
+2
-2
packages/emitter/test/integration/atproto/input/app/bsky/graph/defs.tsp
+2
-2
packages/emitter/test/integration/atproto/input/app/bsky/graph/defs.tsp
···
124
124
model NotFoundActor {
125
125
@required actor: atIdentifier;
126
126
127
-
@lexConst(true)
127
+
@readOnly
128
128
@required
129
-
notFound: boolean;
129
+
notFound: boolean = true;
130
130
}
131
131
132
132
@doc("lists the bi-directional graph relationships between one actor (not indicated in the object), and the target actors (the DID included in the object)")
+4
-4
packages/emitter/test/spec/basic/input/com/example/booleanConstraints.tsp
+4
-4
packages/emitter/test/spec/basic/input/com/example/booleanConstraints.tsp
···
10
10
defaultFalse?: boolean = false;
11
11
12
12
@doc("Constant true value")
13
-
@lexConst(true)
14
-
constTrue?: boolean;
13
+
@readOnly
14
+
constTrue?: boolean = true;
15
15
16
16
@doc("Constant false value")
17
-
@lexConst(false)
18
-
constFalse?: boolean;
17
+
@readOnly
18
+
constFalse?: boolean = false;
19
19
}
20
20
}
+2
-2
packages/emitter/test/spec/basic/input/com/example/integerConstraints.tsp
+2
-2
packages/emitter/test/spec/basic/input/com/example/integerConstraints.tsp