Monorepo for Aesthetic.Computer
aesthetic.computer
1# Documentation Testing & Validation
2
3*Testing framework for ensuring documentation accuracy and completeness*
4
5## Overview
6
7A comprehensive testing system to validate that documentation matches implementation, examples work correctly, and coverage is maintained across the Aesthetic Computer codebase.
8
9## Test Categories
10
11### 1. API Documentation Tests
12
13#### Function Signature Validation
14- **Purpose**: Ensure documented functions exist and have correct signatures
15- **Method**: Compare `docs.js` entries with actual `disk.mjs` exports
16- **Tests**:
17 ```javascript
18 describe('API Function Signatures', () => {
19 it('should have all documented functions exported from disk.mjs', () => {
20 const documented = extractDocumentedFunctions('docs.js');
21 const exported = extractExportedFunctions('disk.mjs');
22
23 documented.forEach(func => {
24 expect(exported).toContain(func.name);
25 });
26 });
27 });
28 ```
29
30#### Parameter Type Validation
31- **Purpose**: Verify parameter types match documentation
32- **Method**: Runtime analysis with mock inputs
33- **Implementation**: Mock BIOS system for testing API functions
34
35#### Example Code Validation
36- **Purpose**: Ensure all code examples in documentation work
37- **Method**: Execute examples in controlled environment
38- **Tests**:
39 ```javascript
40 describe('Documentation Examples', () => {
41 it('should execute graphics examples without errors', async () => {
42 const examples = extractExamplesFromDocs('graphics');
43
44 for (const example of examples) {
45 const mockAPI = createMockAPI();
46 await expect(executeExample(example, mockAPI)).resolves.not.toThrow();
47 }
48 });
49 });
50 ```
51
52### 2. KidLisp Documentation Tests
53
54#### Syntax Validation
55- **Purpose**: Verify KidLisp syntax examples are valid
56- **Method**: Parse examples through KidLisp interpreter
57- **Tests**:
58 ```javascript
59 describe('KidLisp Syntax Examples', () => {
60 it('should parse all syntax examples successfully', () => {
61 const syntaxExamples = extractKidlispExamples('syntax');
62
63 syntaxExamples.forEach(example => {
64 expect(() => parseKidlisp(example.code)).not.toThrow();
65 });
66 });
67 });
68 ```
69
70#### Function Documentation Coverage
71- **Purpose**: Ensure all KidLisp functions are documented
72- **Method**: Compare implemented functions with documented functions
73- **Implementation**: Extract from `kidlisp.mjs` function registry
74
75#### Example Program Execution
76- **Purpose**: Verify KidLisp example programs run correctly
77- **Method**: Execute in sandboxed KidLisp environment
78- **Tests**: Validate visual output matches expected results
79
80### 3. Piece Documentation Tests
81
82#### Lifecycle Function Coverage
83- **Purpose**: Ensure all piece lifecycle functions are documented
84- **Method**: Scan existing pieces for function usage patterns
85- **Tests**:
86 ```javascript
87 describe('Piece Lifecycle Documentation', () => {
88 it('should document all lifecycle functions used in pieces', () => {
89 const usedFunctions = scanPiecesForLifecycleFunctions();
90 const documentedFunctions = extractDocumentedLifecycleFunctions();
91
92 usedFunctions.forEach(func => {
93 expect(documentedFunctions).toContain(func);
94 });
95 });
96 });
97 ```
98
99#### Template Validation
100- **Purpose**: Verify piece templates are valid and functional
101- **Method**: Test templates as actual pieces
102- **Implementation**: Load templates in mock AC environment
103
104#### Cross-Reference Accuracy
105- **Purpose**: Ensure piece documentation references are accurate
106- **Method**: Validate all links and function references
107- **Tests**: Check that referenced API functions exist
108
109### 4. Link and Reference Tests
110
111#### Internal Link Validation
112- **Purpose**: Ensure all internal documentation links work
113- **Method**: Parse markdown files and validate references
114- **Tests**:
115 ```javascript
116 describe('Documentation Links', () => {
117 it('should have valid internal references', () => {
118 const links = extractInternalLinks(documentationFiles);
119
120 links.forEach(link => {
121 expect(fileExists(link.target)).toBe(true);
122 });
123 });
124 });
125 ```
126
127#### External Link Validation
128- **Purpose**: Check external links are accessible
129- **Method**: HTTP requests to validate URLs
130- **Implementation**: Periodic link checking with retry logic
131
132#### Cross-Reference Consistency
133- **Purpose**: Ensure references between docs are bidirectional
134- **Method**: Build reference graph and check completeness
135
136## Testing Framework
137
138### Mock API System
139
140#### Mock BIOS Implementation
141```javascript
142class MockBIOS {
143 constructor() {
144 this.screen = { width: 800, height: 600 };
145 this.canvas = createMockCanvas();
146 this.state = {};
147 }
148
149 // Implement minimal API for testing
150 wipe(color) { /* mock implementation */ }
151 ink(color) { /* mock implementation */ }
152 box(x, y, w, h) { /* mock implementation */ }
153 // ... other API functions
154}
155```
156
157#### Sandboxed Execution Environment
158- **Isolated context** for running examples
159- **Resource limits** to prevent infinite loops
160- **State capture** for validating results
161- **Error handling** with detailed reporting
162
163### Test Data Generation
164
165#### API Function Discovery
166```javascript
167function extractAPIFunctions(diskPath) {
168 const source = readFileSync(diskPath, 'utf8');
169
170 // Parse AST to find exported functions
171 const ast = parse(source, { sourceType: 'module' });
172
173 return ast.body
174 .filter(node => node.type === 'ExportNamedDeclaration')
175 .map(node => extractFunctionInfo(node));
176}
177```
178
179#### Documentation Parsing
180```javascript
181function extractDocumentationExamples(docsPath) {
182 const content = readFileSync(docsPath, 'utf8');
183 const tokens = marked.lexer(content);
184
185 return tokens
186 .filter(token => token.type === 'code')
187 .filter(token => token.lang === 'javascript' || token.lang === 'lisp')
188 .map(token => ({ code: token.text, lang: token.lang }));
189}
190```
191
192### Continuous Integration
193
194#### GitHub Actions Workflow
195```yaml
196name: Documentation Tests
197
198on: [push, pull_request]
199
200jobs:
201 docs-tests:
202 runs-on: ubuntu-latest
203
204 steps:
205 - uses: actions/checkout@v2
206
207 - name: Setup Node.js
208 uses: actions/setup-node@v2
209 with:
210 node-version: '18'
211
212 - name: Install dependencies
213 run: npm ci
214
215 - name: Run API documentation tests
216 run: npm run test:docs:api
217
218 - name: Run KidLisp documentation tests
219 run: npm run test:docs:kidlisp
220
221 - name: Run piece documentation tests
222 run: npm run test:docs:pieces
223
224 - name: Generate coverage report
225 run: npm run docs:coverage
226
227 - name: Upload coverage
228 uses: codecov/codecov-action@v1
229```
230
231#### Local Development Workflow
232```bash
233# Run all documentation tests
234npm run test:docs
235
236# Run specific test suites
237npm run test:docs:api
238npm run test:docs:kidlisp
239npm run test:docs:pieces
240
241# Generate coverage report
242npm run docs:coverage
243
244# Update documentation cache
245npm run docs:update
246
247# Validate examples only
248npm run test:examples
249```
250
251## Coverage Metrics
252
253### API Coverage
254- **Function Coverage**: % of API functions documented
255- **Parameter Coverage**: % of parameters with type documentation
256- **Example Coverage**: % of functions with working examples
257- **Usage Coverage**: % of functions with real-world usage examples
258
259### KidLisp Coverage
260- **Syntax Coverage**: % of language features documented
261- **Function Coverage**: % of built-in functions documented
262- **Example Coverage**: % of concepts with examples
263- **Tutorial Coverage**: % of language progression covered
264
265### Piece Coverage
266- **Lifecycle Coverage**: % of lifecycle functions documented
267- **Pattern Coverage**: % of common patterns documented
268- **Template Coverage**: % of use cases with templates
269- **Integration Coverage**: % of API integration documented
270
271## Reporting
272
273### Test Results Dashboard
274```
275📊 Documentation Test Results
276
277API Documentation:
278 ✅ Function Signatures: 95% (142/150 functions)
279 ✅ Examples: 78% (117/150 functions)
280 ❌ Parameter Types: 65% (98/150 functions)
281
282KidLisp Documentation:
283 ✅ Syntax Examples: 100% (25/25 examples)
284 ✅ Function Coverage: 88% (44/50 functions)
285 ✅ Tutorial Progression: 92% (23/25 topics)
286
287Piece Documentation:
288 ✅ Lifecycle Functions: 100% (7/7 functions)
289 ⚠️ Templates: 60% (6/10 use cases)
290 ❌ Integration Examples: 45% (18/40 patterns)
291
292Overall Score: 82% (456/556 total items)
293```
294
295### Actionable Reports
296- **Missing Documentation**: List of undocumented functions
297- **Broken Examples**: Examples that fail to execute
298- **Outdated References**: Links and references that need updates
299- **Coverage Gaps**: Areas needing attention
300
301## Tools & Scripts
302
303### Test Runner
304```bash
305#!/bin/bash
306# test-documentation.sh
307
308echo "🧪 Running Documentation Tests..."
309
310# API Tests
311echo "📚 Testing API Documentation..."
312node tools/test-api-docs.mjs
313
314# KidLisp Tests
315echo "🔤 Testing KidLisp Documentation..."
316node tools/test-kidlisp-docs.mjs
317
318# Piece Tests
319echo "🧩 Testing Piece Documentation..."
320node tools/test-piece-docs.mjs
321
322# Link Tests
323echo "🔗 Testing Documentation Links..."
324node tools/test-links.mjs
325
326echo "✅ Documentation tests complete!"
327```
328
329### Example Validator
330```javascript
331// validate-examples.mjs
332import { executeMockPiece } from './mock-bios.mjs';
333import { extractExamples } from './docs-parser.mjs';
334
335async function validateExamples() {
336 const examples = await extractExamples('/reference');
337 const results = [];
338
339 for (const example of examples) {
340 try {
341 await executeMockPiece(example.code);
342 results.push({ ...example, status: 'pass' });
343 } catch (error) {
344 results.push({ ...example, status: 'fail', error: error.message });
345 }
346 }
347
348 return results;
349}
350```
351
352## Integration Points
353
354- **CI/CD Pipeline**: Automated testing on every commit
355- **Pre-commit Hooks**: Quick validation before commits
356- **Documentation Updates**: Automatic test updates when docs change
357- **Development Workflow**: Tests as part of development cycle
358- **Release Process**: Documentation validation before releases
359
360---
361
362*This testing framework ensures documentation stays accurate, complete, and useful as the Aesthetic Computer system evolves.*