docs: document ConfigProvider implementation in layering.md

+392

1 changed file

expand all

layering.md

+392

layering.md

··· 435 435 4. **Async providers**: How to handle slow providers (Vault, remote config) gracefully? 436 436 437 437 5. **Caching**: Should registry cache loaded configs for repeated `.load()` calls? 438 + 439 + --- 440 + 441 + ## Implemented: ConfigProvider System 442 + 443 + Phase 2 of the roadmap has been implemented. The c12-layer fork now includes a fully functional pluggable provider system that allows users to customize the configuration loading pipeline while maintaining complete backward compatibility. 444 + 445 + ### Architecture Overview 446 + 447 + The provider system introduces a `ConfigProvider` interface that abstracts each configuration source. Instead of hardcoded loading logic, `loadConfig()` now iterates through an ordered list of providers, each responsible for loading one source of configuration. 448 + 449 + ```mermaid 450 + flowchart LR 451 + subgraph providers["Provider Pipeline"] 452 + direction TB 453 + p1[overrides priority: 100] 454 + p2[main priority: 200] 455 + p3[rc priority: 300] 456 + p4[packageJson priority: 400] 457 + p5[defaultConfig priority: 500] 458 + end 459 + 460 + loadConfig --> sort[Sort by priority] 461 + sort --> providers 462 + providers --> merge[defu merge all configs] 463 + merge --> extends[Process extends] 464 + extends --> result[ResolvedConfig] 465 + ``` 466 + 467 + ### Core Types 468 + 469 + The implementation adds these types in [`src/providers.ts`](src/providers.ts): 470 + 471 + ```typescript 472 + /** 473 + * Context passed to config providers during loading 474 + */ 475 + interface ProviderContext<T, MT> { 476 + /** Normalized load options */ 477 + options: LoadConfigOptions<T, MT>; 478 + /** Merger function (defu or custom) */ 479 + merger: (...sources: Array<T | null | undefined>) => T; 480 + /** Resolve a config file (used by main provider) */ 481 + resolveConfig: (source: string, options: LoadConfigOptions<T, MT>) => Promise<ResolvedConfig<T, MT>>; 482 + /** Results from previously loaded providers (by name) */ 483 + loadedConfigs: Map<string, T | null | undefined>; 484 + } 485 + 486 + /** 487 + * Result returned by a config provider 488 + */ 489 + interface ProviderResult<T, MT> { 490 + /** The loaded configuration (or null/undefined if not found) */ 491 + config: T | null | undefined; 492 + /** Layer metadata for introspection */ 493 + layer?: Partial<ConfigLayer<T, MT>>; 494 + /** Additional metadata to merge into ResolvedConfig */ 495 + meta?: MT; 496 + /** Resolved config file path (for main provider) */ 497 + configFile?: string; 498 + /** Internal config file path */ 499 + _configFile?: string; 500 + } 501 + 502 + /** 503 + * A pluggable configuration source provider 504 + */ 505 + interface ConfigProvider<T, MT> { 506 + /** Unique name for this provider */ 507 + name: string; 508 + /** 509 + * Priority determines merge order. 510 + * Lower numbers = higher priority (merged first, so they "win"). 511 + */ 512 + priority: number; 513 + /** 514 + * Load configuration from this provider. 515 + * Return null/undefined if this provider has no config to contribute. 516 + */ 517 + load(ctx: ProviderContext<T, MT>): Promise<ProviderResult<T, MT> | null | undefined>; 518 + } 519 + ``` 520 + 521 + ### Built-in Providers 522 + 523 + Five built-in providers replicate the original c12 behavior: 524 + 525 + | Provider | Priority | Factory Function | Description | 526 + |----------|----------|------------------|-------------| 527 + | overrides | 100 | `createOverridesProvider()` | Returns `options.overrides` | 528 + | main | 200 | `createMainProvider()` | Loads `<name>.config.ts` via `resolveConfig()` | 529 + | rc | 300 | `createRcProvider()` | Loads `.namerc` files (cwd, workspace, home) | 530 + | packageJson | 400 | `createPackageJsonProvider()` | Loads from `package.json[name]` | 531 + | defaultConfig | 500 | `createDefaultConfigProvider()` | Returns `options.defaultConfig` | 532 + 533 + Each provider is a standalone factory function that can be individually imported and customized. 534 + 535 + ### Exports 536 + 537 + The following are exported from the main `c12` module: 538 + 539 + ```typescript 540 + // Types 541 + export type { ConfigProvider, ProviderContext, ProviderResult } from "./providers.ts"; 542 + 543 + // Factory functions for built-in providers 544 + export { 545 + getDefaultProviders, // Returns all 5 built-in providers 546 + sortProviders, // Sort providers by priority 547 + createOverridesProvider, 548 + createMainProvider, 549 + createRcProvider, 550 + createPackageJsonProvider, 551 + createDefaultConfigProvider, 552 + } from "./providers.ts"; 553 + ``` 554 + 555 + ### Usage Examples 556 + 557 + #### Basic: No Changes Required 558 + 559 + Existing code works without modification: 560 + 561 + ```typescript 562 + import { loadConfig } from 'c12'; 563 + 564 + // Works exactly as before 565 + const { config } = await loadConfig({ name: 'myapp' }); 566 + ``` 567 + 568 + #### Adding a Custom Provider 569 + 570 + Inject a custom source between existing ones: 571 + 572 + ```typescript 573 + import { loadConfig, getDefaultProviders, ConfigProvider } from 'c12'; 574 + 575 + // Create a provider that loads from environment variables 576 + const envProvider: ConfigProvider = { 577 + name: 'env', 578 + priority: 150, // Between overrides (100) and main (200) 579 + async load(ctx) { 580 + const config: Record<string, any> = {}; 581 + const prefix = `${ctx.options.name?.toUpperCase()}_`; 582 + 583 + for (const [key, value] of Object.entries(process.env)) { 584 + if (key.startsWith(prefix)) { 585 + const configKey = key.slice(prefix.length).toLowerCase(); 586 + config[configKey] = value; 587 + } 588 + } 589 + 590 + if (Object.keys(config).length === 0) return null; 591 + 592 + return { 593 + config, 594 + layer: { 595 + config, 596 + configFile: 'environment', 597 + }, 598 + }; 599 + }, 600 + }; 601 + 602 + const { config, layers } = await loadConfig({ 603 + name: 'myapp', 604 + providers: [...getDefaultProviders(), envProvider], 605 + }); 606 + 607 + // layers will include { configFile: 'environment', config: {...} } 608 + ``` 609 + 610 + #### Replacing Default Providers 611 + 612 + Use only specific sources: 613 + 614 + ```typescript 615 + import { loadConfig, createMainProvider, createOverridesProvider } from 'c12'; 616 + 617 + // Only load from overrides and main config file (no RC, no package.json) 618 + const { config } = await loadConfig({ 619 + name: 'myapp', 620 + providers: [ 621 + createOverridesProvider(), 622 + createMainProvider(), 623 + ], 624 + overrides: { debug: true }, 625 + }); 626 + ``` 627 + 628 + #### Reordering Providers 629 + 630 + Change the default priority order: 631 + 632 + ```typescript 633 + import { loadConfig, getDefaultProviders } from 'c12'; 634 + 635 + // Make package.json take precedence over RC files 636 + const providers = getDefaultProviders().map(p => { 637 + if (p.name === 'packageJson') return { ...p, priority: 250 }; 638 + if (p.name === 'rc') return { ...p, priority: 350 }; 639 + return p; 640 + }); 641 + 642 + const { config } = await loadConfig({ 643 + name: 'myapp', 644 + providers, 645 + }); 646 + ``` 647 + 648 + #### Provider Dependencies 649 + 650 + Access previously loaded configs in your provider: 651 + 652 + ```typescript 653 + const conditionalProvider: ConfigProvider = { 654 + name: 'conditional', 655 + priority: 250, 656 + async load(ctx) { 657 + // Check what the main config loaded 658 + const mainConfig = ctx.loadedConfigs.get('main'); 659 + 660 + if (mainConfig?.featureFlags?.enableAdvanced) { 661 + return { 662 + config: { advancedSetting: 'enabled' }, 663 + }; 664 + } 665 + 666 + return null; // Don't contribute if feature not enabled 667 + }, 668 + }; 669 + ``` 670 + 671 + #### Remote Configuration Provider 672 + 673 + Load config from a remote source: 674 + 675 + ```typescript 676 + const remoteProvider: ConfigProvider = { 677 + name: 'remote', 678 + priority: 175, // After env, before main 679 + async load(ctx) { 680 + const endpoint = process.env.CONFIG_ENDPOINT; 681 + if (!endpoint) return null; 682 + 683 + try { 684 + const response = await fetch(`${endpoint}/${ctx.options.name}`); 685 + if (!response.ok) return null; 686 + 687 + const config = await response.json(); 688 + return { 689 + config, 690 + layer: { 691 + config, 692 + configFile: endpoint, 693 + }, 694 + }; 695 + } catch { 696 + // Network error - silently skip 697 + return null; 698 + } 699 + }, 700 + }; 701 + ``` 702 + 703 + ### How the Provider System Works 704 + 705 + 1. **Initialization**: `loadConfig()` normalizes options and sets up dotenv (unchanged) 706 + 707 + 2. **Provider Selection**: 708 + - If `options.providers` is specified, use those providers 709 + - Otherwise, call `getDefaultProviders()` to get the built-in set 710 + 711 + 3. **Sorting**: Providers are sorted by `priority` (ascending). Lower priority numbers are processed first and "win" in the merge. 712 + 713 + 4. **Sequential Loading**: Each provider's `load()` method is called in order. The `ProviderContext` includes: 714 + - The normalized `options` 715 + - The `merger` function (defu or custom) 716 + - A `resolveConfig` helper for loading files 717 + - A `loadedConfigs` map with results from previous providers 718 + 719 + 5. **Result Collection**: Non-null results are collected. Each result includes: 720 + - `config`: The configuration object 721 + - `layer`: Optional metadata for the `layers` array 722 + - `configFile`, `_configFile`, `meta`: For the main provider 723 + 724 + 6. **Merging**: All collected configs are merged using the merger function, in priority order 725 + 726 + 7. **Extends Processing**: The `extends` mechanism works as before, after the main merge 727 + 728 + 8. **Layer Preservation**: Provider results with `layer` data are added to `ResolvedConfig.layers` 729 + 730 + ### Impact on Existing c12 Users 731 + 732 + #### Zero Breaking Changes 733 + 734 + The provider system is **fully backward compatible**. Existing code continues to work without any modifications: 735 + 736 + ```typescript 737 + // This code works identically before and after the change 738 + const { config } = await loadConfig({ 739 + name: 'myapp', 740 + overrides: { debug: true }, 741 + rcFile: '.myapprc', 742 + packageJson: true, 743 + defaults: { port: 3000 }, 744 + }); 745 + ``` 746 + 747 + #### Behavioral Equivalence 748 + 749 + When `options.providers` is not specified, the system behaves identically to the original c12: 750 + 751 + 1. Same source loading order (overrides → main → rc → packageJson → defaultConfig) 752 + 2. Same merge semantics (defu, or custom merger) 753 + 3. Same `extends` processing 754 + 4. Same `layers` array structure 755 + 5. Same handling of `$development`, `$production`, etc. 756 + 757 + #### Minor Differences 758 + 759 + There is one minor difference in error messages: 760 + 761 + | Scenario | Original | With Providers | 762 + |----------|----------|----------------| 763 + | `configFileRequired: true` with missing file | `Required config (CUSTOM) cannot be resolved.` | `Required config (/full/path/CUSTOM) cannot be resolved.` | 764 + 765 + The new message includes the full resolved path, which is more helpful for debugging. 766 + 767 + #### New Option 768 + 769 + One new option is added to `LoadConfigOptions`: 770 + 771 + ```typescript 772 + interface LoadConfigOptions<T, MT> { 773 + // ... existing options ... 774 + 775 + /** 776 + * Custom configuration providers. 777 + * When specified, replaces the built-in source loading. 778 + * Use getDefaultProviders() to include defaults. 779 + */ 780 + providers?: ConfigProvider<T, MT>[]; 781 + } 782 + ``` 783 + 784 + ### Testing the Provider System 785 + 786 + The implementation includes comprehensive tests in [`test/loader.test.ts`](test/loader.test.ts): 787 + 788 + ```typescript 789 + describe("providers", () => { 790 + it("uses default providers when none specified"); 791 + it("allows custom providers to inject config"); 792 + it("respects provider priority order"); 793 + it("allows removing default providers"); 794 + it("provider can access previously loaded configs"); 795 + }); 796 + ``` 797 + 798 + Run tests with: 799 + ```bash 800 + pnpm test 801 + ``` 802 + 803 + ### Future Enhancements 804 + 805 + The provider system creates a foundation for additional features: 806 + 807 + 1. **Drop-in Directory Provider**: A `createConfigDirProvider()` that scans `*.config.d/` directories 808 + 809 + 2. **Provenance Tracking**: Providers already return layer metadata; a tracking merger could record which provider contributed each key 810 + 811 + 3. **Validation Phase**: A `validateProviders()` function that checks all sources exist before loading 812 + 813 + 4. **Parallel Loading**: Independent providers could load concurrently for better performance 814 + 815 + 5. **Provider Plugins**: A registry of community providers (Vault, AWS SSM, Consul, etc.) 816 + 817 + ### Files Changed 818 + 819 + | File | Change | 820 + |------|--------| 821 + | [`src/providers.ts`](src/providers.ts) | New file: Provider types and built-in implementations | 822 + | [`src/types.ts`](src/types.ts) | Added `providers` option to `LoadConfigOptions` | 823 + | [`src/loader.ts`](src/loader.ts) | Refactored to use provider pipeline | 824 + | [`src/index.ts`](src/index.ts) | Export provider types and functions | 825 + | [`test/loader.test.ts`](test/loader.test.ts) | Added provider system tests | 826 + 827 + ### Summary 828 + 829 + The ConfigProvider system transforms c12 from a fixed-pipeline loader into an extensible, customizable configuration platform while maintaining complete backward compatibility. Users who don't need customization see no changes; power users gain full control over the loading pipeline.

Configure Feed

Configure Feed