internal/cuetxtar: document new @test directives

+200 -10

2 changed files

expand all

cue

testdata

readme.md

doc

specs

inline-test-attributes

spec.md

+93 -3

cue/testdata/readme.md

··· 142 142 | `contains="s"` | error message must contain substring `s` | 143 143 | `any` | at least one *descendant* has the error (requires `code=`) | 144 144 | `path=(p\|q)` | error exists at one of the listed paths | 145 + | `pos=[...]` | error positions must match (see below) | 146 + | `args=[...]` | Msg() args must contain the listed values (see below) | 147 + | `suberr=(...)` | sub-error spec for multi-error values (see below) | 148 + 149 + #### `pos=[...]` — error positions 150 + 151 + Specifies expected error positions. Each position is written as `deltaLine:col` 152 + relative to the `@test` attribute line, or `file:line:col` for positions in 153 + other files: 154 + 155 + ```cue 156 + bad: x & y @test(err, code=eval, pos=[0:5 0:9]) 157 + ``` 158 + 159 + `pos=[]` is a fill-in placeholder. Running with `CUE_UPDATE=1` writes the 160 + actual positions. Running with `CUE_UPDATE=force` overwrites existing 161 + non-empty `pos=` specs too. 162 + 163 + #### `args=[v1, v2, ...]` — Msg() argument check 164 + 165 + Checks that the values returned by the error's `Msg()` method include 166 + all listed strings (matched via `fmt.Sprint`, **order-independent**, 167 + **subset check**): 168 + 169 + ```cue 170 + e: [] & 4 @test(err, code=eval, contains="conflicting values", args=[list, int]) 171 + ``` 172 + 173 + Design note: `args=` is a **subset check** — extra actual arguments not 174 + listed in `args=` are allowed. This is intentional: `args=` is for 175 + verifying the arguments that matter (e.g. type names) without having to 176 + enumerate every argument. Arguments that happen to vary in order across 177 + implementations can be checked without also repeating arguments already 178 + covered by `contains=`. 179 + 180 + If you need to verify the exact set of arguments, list all of them: 181 + 182 + ```cue 183 + e: [3][-1] @test(err, code=eval, args=[-1]) // only one arg — effectively exact 184 + ``` 185 + 186 + #### `suberr=(...)` — sub-error specs 187 + 188 + For errors composed of multiple sub-errors (typically failed disjunctions), 189 + each `suberr=(...)` spec matches one sub-error **order-independently**. 190 + The body accepts the same options as `@test(err, ...)`: 191 + 192 + ```cue 193 + x: null | {n: 3} 194 + x: #empty & {n: 3} @test(err, code=eval, 195 + suberr=(contains="conflicting values", args=[struct, null]), 196 + suberr=(contains="not allowed")) 197 + ``` 198 + 199 + Matching is two-pass: 200 + 1. Specs with non-empty `pos=` are matched first (position is a stronger 201 + discriminator than `contains=`). 202 + 2. Remaining specs are matched by `contains=` against unmatched actual 203 + sub-errors. 204 + 205 + `pos=[]` placeholders inside `suberr=(...)` trigger write-back on 206 + `CUE_UPDATE=1`. All position updates for the same `@test` attribute are 207 + applied atomically. 145 208 146 209 ### `kind` — value kind 147 210 ··· 213 276 214 277 --- 215 278 279 + ## `shareID` — vertex sharing assertion 280 + 281 + `@test(shareID=name)` asserts that all fields annotated with the same 282 + `name` share the same underlying `*adt.Vertex` (pointer identity after 283 + following indirections): 284 + 285 + ```cue 286 + a: {x: 1} 287 + b: a 288 + @test(eq, { 289 + a: {x: 1} @test(shareID=A) // first occurrence: eq check runs normally 290 + b: a @test(shareID=A) // subsequent occurrences: eq check skipped 291 + }) 292 + ``` 293 + 294 + Rules: 295 + - The **first** field with a given `shareID` name has its eq check run 296 + normally (so every value-expr pair is validated at least once). 297 + - **Subsequent** fields with the same name skip the eq check — their 298 + expression is treated as documentation only. 299 + - The sharing assertion itself runs after all eq checks. 300 + 301 + `@test(shareID=name)` may only appear inside a `@test(eq, {...})` body. 302 + 303 + --- 304 + 216 305 ## Empty placeholder and `CUE_UPDATE` 217 306 218 307 `@test()` (empty body) is a fill-in placeholder. Running with `CUE_UPDATE=1` ··· 223 312 | Value | `@test(eq, <value>)` | 224 313 | Error | `@test(err, code=<code>, contains="<msg>")` | 225 314 226 - `CUE_UPDATE=diff` records a failing assertion as 227 - `@test(..., skip:v3, diff="got …; want …")` rather than overwriting the 228 - expected value. `CUE_UPDATE=force` overwrites unconditionally. 315 + `CUE_UPDATE=diff` shows a unified diff of what `CUE_UPDATE=1` *would* write, 316 + without modifying any files. Documentary sections (e.g. `out/errors.txt`) 317 + are also validated in this mode. `CUE_UPDATE=force` overwrites unconditionally, 318 + including non-empty `pos=` specs that would normally require manual review. 229 319 230 320 --- 231 321

+107 -7

doc/specs/inline-test-attributes/spec.md

··· 224 224 - `code=<code>` — the error code must match. Valid codes include `cycle`, `eval`, `incomplete`, `structural`, `reference`, and any other code defined in `cuelang.org/go/cue/errors`. 225 225 - `contains="substring"` — the error message must contain the given substring. 226 226 - `any` (bare flag) — at least one **descendant** of the annotated field must have an error. The annotated field itself is not required to be an error. `code` MUST be specified when `any` is used. 227 + - `args=[v1, v2, ...]` — the values returned by the error's `Msg()` method must include all listed strings (matched via `fmt.Sprint`, order-independent, subset check). See `Requirement: err directive — args= sub-option` for details. 228 + - `suberr=(...)` — matches one sub-error of a multi-error value (e.g. a failed disjunction). Multiple `suberr=` entries match sub-errors order-independently. The body accepts the same sub-options as `@test(err, ...)`. See `Requirement: err directive — suberr=(...)` for details. 227 229 - `pos=[spec ...]` — asserts the exact set of source positions reported by the error (as returned by `cuelang.org/go/cue/errors.Positions`). Each whitespace-delimited spec takes one of two forms: 228 230 - `deltaLine:col` — position in the **same file** as the `@test` attribute, expressed as a signed line offset from an *anchor line* and a 1-indexed column. The anchor depends on where the `@test` attribute appears: 229 231 - **Field attribute** (`field: value @test(...)`): anchor is the field's line in the stripped output (`deltaLine=0` = same line as the field). ··· 267 269 #### Scenario: pos= mismatch fails test 268 270 - **WHEN** `@test(err, pos=[0:5])` is declared but the error has two positions 269 271 - **THEN** the test fails reporting the count mismatch 272 + 273 + --- 274 + 275 + ### Requirement: `err` directive — `args=` sub-option 276 + The `args=[v1, v2, ...]` sub-option of `@test(err, ...)` asserts that the error's `Msg()` method returns arguments that include all listed strings. The check is **order-independent** and a **subset check**: every expected string must appear in the actual `Msg()` args (matched by stringifying each actual arg with `fmt.Sprint`), but extra actual args not listed in `args=` are allowed. 277 + 278 + Design rationale: `args=` targets the structured arguments of an error message (e.g. type names, field names), not the rendered text. Internationalized error messages may change wording while keeping the same structured args. `contains=` checks rendered text; `args=` checks structured data. The subset-check design lets authors list only the args they care about (e.g. the two type names in a type-conflict error) without repeating args already covered by `contains=`, and allows checking args whose order may vary across evaluator implementations. 279 + 280 + ```cue 281 + e: [] & 4 @test(err, code=eval, contains="conflicting values", args=[list, int]) 282 + e3: [3][-1] @test(err, code=eval, args=[-1]) 283 + ``` 284 + 285 + #### Scenario: args= subset matches 286 + - **WHEN** `@test(err, args=[list, int])` is declared and `Msg()` returns args `["conflicting values []", "list", "int"]` (or any order) 287 + - **THEN** the test passes because both `"list"` and `"int"` are found in the actual args 288 + 289 + #### Scenario: args= missing arg fails 290 + - **WHEN** `@test(err, args=[list, int])` is declared but `Msg()` returns only `["list"]` 291 + - **THEN** the test fails reporting that `"int"` was not found 292 + 293 + #### Scenario: args= extra actual args are allowed 294 + - **WHEN** `@test(err, args=[int])` is declared and `Msg()` returns `["list", "int"]` 295 + - **THEN** the test passes; the extra `"list"` arg is allowed under the subset-check design 296 + 297 + #### Scenario: args= is order-independent 298 + - **WHEN** `@test(err, args=[int, list])` is declared and `Msg()` returns args in order `["list", "int"]` 299 + - **THEN** the test passes regardless of argument order 300 + 301 + --- 302 + 303 + ### Requirement: `err` directive — `suberr=(...)` sub-option 304 + The `suberr=(...)` sub-option of `@test(err, ...)` asserts properties of individual sub-errors within a multi-error value (e.g., a failed disjunction). Each `suberr=(...)` entry matches one sub-error; multiple entries match sub-errors order-independently. The body of `suberr=(...)` accepts the same sub-options as `@test(err, ...)`: `code=`, `contains=`, `pos=`, `args=`. 305 + 306 + Matching is two-pass: 307 + 1. Specs with non-empty `pos=` are matched first against sub-errors with matching positions (position is a stronger discriminator than text content). 308 + 2. Remaining specs are matched by `contains=` against unmatched actual sub-errors. 309 + 310 + `pos=[]` placeholders inside `suberr=(...)` trigger write-back on `CUE_UPDATE=1`. All position updates for the same `@test` attribute are applied atomically. 311 + 312 + ```cue 313 + x: null | {n: 3} 314 + x: #empty & {n: 3} @test(err, code=eval, 315 + suberr=(contains="conflicting values", args=[struct, null]), 316 + suberr=(contains="not allowed")) 317 + ``` 318 + 319 + #### Scenario: suberr= matches all sub-errors 320 + - **WHEN** `@test(err, suberr=(contains="A"), suberr=(contains="B"))` is declared and the error has exactly two sub-errors whose messages contain `"A"` and `"B"` respectively 321 + - **THEN** the test passes 322 + 323 + #### Scenario: suberr= order-independent matching 324 + - **WHEN** two `suberr=` specs are declared and the actual sub-errors appear in reverse order relative to the specs 325 + - **THEN** the test passes; matching is order-independent 326 + 327 + #### Scenario: suberr= unmatched spec fails 328 + - **WHEN** `@test(err, suberr=(contains="A"), suberr=(contains="B"))` is declared but only one sub-error exists (containing "A") 329 + - **THEN** the test fails reporting that the `suberr=(contains="B")` spec was unmatched 330 + 331 + #### Scenario: suberr= pos= placeholder fills on CUE_UPDATE 332 + - **WHEN** `suberr=(pos=[])` appears inside a `@test(err, ...)` and `CUE_UPDATE=1` is set 333 + - **THEN** the runner fills in the actual positions within the `suberr=(...)` body 334 + 335 + --- 336 + 337 + ### Requirement: `shareID` directive 338 + The `shareID=name` argument, used inside the expected struct of a `@test(eq, {...})` body, asserts that all fields annotated with the same `name` share the same underlying `*adt.Vertex` (pointer identity after following indirections). This verifies that the evaluator's vertex-sharing optimization is working correctly. 339 + 340 + Rules: 341 + - The **first** field carrying a given `shareID` name has its `eq` check run normally — so every value-expression pair is validated at least once. 342 + - **Subsequent** fields with the same `shareID` name skip the `eq` check — their expression is treated as documentation only. 343 + - The pointer-identity assertion runs after all `eq` checks complete. 344 + - `@test(shareID=name)` may only appear inside a `@test(eq, {...})` body; it is not valid as a standalone field attribute. 345 + 346 + ```cue 347 + a: {x: 1} 348 + b: a 349 + @test(eq, { 350 + a: {x: 1} @test(shareID=A) // first occurrence: eq check runs normally 351 + b: a @test(shareID=A) // subsequent occurrence: eq check skipped 352 + }) 353 + ``` 354 + 355 + #### Scenario: shareID first occurrence runs eq check 356 + - **WHEN** `a: {x: 1} @test(shareID=A)` is the first field with `shareID=A` and `a` does not equal `{x: 1}` 357 + - **THEN** the test fails on the eq check 358 + 359 + #### Scenario: shareID subsequent occurrence skips eq check 360 + - **WHEN** `b: a @test(shareID=A)` is the second field with `shareID=A` and the expression `a` would not be valid in this scope 361 + - **THEN** the eq check is skipped; only the pointer-identity assertion runs 362 + 363 + #### Scenario: shareID pointer identity fails 364 + - **WHEN** `a` and `b` are annotated with the same `shareID=A` but point to different `*adt.Vertex` objects 365 + - **THEN** the test fails reporting that the vertices are not shared 366 + 367 + #### Scenario: shareID pointer identity passes 368 + - **WHEN** `b: a` (a reference) and `a` are annotated with `shareID=A` and the evaluator has shared the vertex 369 + - **THEN** the test passes 270 370 271 371 --- 272 372 ··· 530 630 531 631 Behavior matrix: 532 632 533 - | Section present? | `CUE_UPDATE=1` | Normal run | `CUE_CHECK=1` | 534 - |-----------------|---------------|------------|---------------| 633 + | Section present? | `CUE_UPDATE=1` | Normal run | `CUE_UPDATE=diff` | 634 + |-----------------|---------------|------------|-------------------| 535 635 | No | skip (don't create) | skip | skip | 536 - | Yes | update silently | skip (no fail) | fail if stale | 636 + | Yes | update silently | skip (no fail) | show diff (no fail) | 537 637 538 638 Key points: 539 639 - The section is **never auto-created** by `CUE_UPDATE=1`. It must be added manually. 540 640 - A normal test run silently ignores differences in this section. 541 - - `CUE_CHECK=1` enables strict mode: the section fails if its content is stale. 641 + - `CUE_UPDATE=diff` shows a unified diff of what `CUE_UPDATE=1` would write, without modifying files. 542 642 - `CUE_UPDATE=1` updates the section's content when it already exists. 543 643 544 644 Both child errors (incomplete, eval, cycle, etc.) and propagated-from-child error markers are included. Child-only marker errors (which don't carry their own message) are excluded. ··· 555 655 - **WHEN** the archive has an `-- out/errors.txt --` section and `CUE_UPDATE=1` is set 556 656 - **THEN** the section is updated with the current error output 557 657 558 - #### Scenario: Section present — CUE_CHECK=1 fails on stale content 559 - - **WHEN** the archive has an `-- out/errors.txt --` section with stale content and `CUE_CHECK=1` is set 560 - - **THEN** the test fails with a message showing the expected and actual error output 658 + #### Scenario: Section present — CUE_UPDATE=diff shows diff on stale content 659 + - **WHEN** the archive has an `-- out/errors.txt --` section with stale content and `CUE_UPDATE=diff` is set 660 + - **THEN** the runner shows a unified diff of the expected and actual error output but does not fail the test

Configure Feed

Configure Feed