Keep using Photos.app like you always do. Attic quietly backs up your originals and edits to an S3 bucket you control. One-way, append-only.
Rewrite Attic CLI in Swift, remove legacy Deno/TypeScript implementation
Replace the Deno/TypeScript CLI with a native Swift implementation using
a shared AtticCore library (for future menu bar app reuse). LadderKit is
now called directly as a library instead of shelling out to a subprocess.
Swift implementation includes:
- AtticCore: S3 provider, manifest, config, keychain, metadata,
backup/verify/refresh/rebuild pipelines (all behind protocols)
- AtticCLI: ArgumentParser commands with live-updating terminal dashboard
- 75 tests using Swift Testing framework with mock implementations
Also: include iCloud Shared Photo Library assets in PhotoKit enumeration,
add macOS permission dialog guidance to init and README.
+4075
-6836
+7
-1
.gitignore
+7
-1
.gitignore
+44
-43
CLAUDE.md
+44
-43
CLAUDE.md
···
5
5
6
6
## What This Is
7
7
8
-
Deno/TypeScript CLI for backing up iCloud Photos to S3-compatible storage. Part
9
-
of the photo-cloud system (companion: [ladder](https://github.com/tijs/ladder)).
8
+
Swift CLI and shared library for backing up iCloud Photos to S3-compatible
9
+
storage. Part of the photo-cloud system (companion:
10
+
[ladder](https://github.com/tijs/ladder)).
10
11
11
12
## Commands
12
13
13
14
```bash
14
-
deno task check # Type check (cli/mod.ts)
15
-
deno task test # Run all tests
16
-
deno task lint # Lint
17
-
deno task fmt # Format
18
-
deno task fmt:check # Check formatting
19
-
```
20
-
21
-
Run a single test file:
22
-
23
-
```bash
24
-
deno test --allow-read --allow-write --allow-env --allow-ffi --allow-net cli/src/commands/backup.test.ts
15
+
swift build 2>&1 | xcsift # Build
16
+
swift test 2>&1 | xcsift # Run all tests
17
+
swift build -c release 2>&1 | xcsift # Release build
25
18
```
26
19
27
20
Run a single test by name:
28
21
29
22
```bash
30
-
deno test --allow-read --allow-write --allow-env --allow-ffi --allow-net --filter "test name" cli/src/commands/backup.test.ts
23
+
swift test --filter "testName" 2>&1 | xcsift
31
24
```
32
25
33
-
## Workspace Structure
26
+
## Package Structure
34
27
35
-
Deno workspace with two members:
28
+
Swift package with three targets:
36
29
37
-
- `shared/` — `@attic/shared` — `PhotoAsset` type, `AssetKind`/`CloudLocalState`
38
-
constants, S3 path helpers
39
-
- `cli/` — `@attic/cli` — all commands, config, storage, manifest, export logic
30
+
- `AtticCore` — shared library: S3 provider, manifest, config, keychain,
31
+
metadata, backup/verify/refresh pipelines. Used by both CLI and menu bar app.
32
+
- `AtticCLI` — executable: ArgumentParser commands, terminal renderer
33
+
- `AtticCoreTests` — tests using Swift Testing framework
40
34
41
-
Import shared code as `@attic/shared` (mapped in `cli/deno.json`).
35
+
Dependencies: `aws-sdk-swift` (AWSS3), `swift-argument-parser`, `LadderKit`
36
+
(path dependency from `../ladder`).
42
37
43
-
Key dependencies: `@aws-sdk/client-s3`, `@cliffy/command` (CLI framework),
44
-
`@db/sqlite` (Photos.sqlite reader), `@std/crypto` (SHA-256).
38
+
Platform: macOS 14+, Swift 6.x, Apple Silicon only.
45
39
46
40
## Architecture
47
41
48
42
The backup pipeline:
49
-
`Photos.sqlite → reader.ts → PhotoAsset[] → backup.ts → ladder export → S3 upload → manifest update`
43
+
`Photos Library → LadderKit (PhotoKit + enrichment) → AssetInfo[] → BackupPipeline → S3 upload → manifest update`
50
44
51
-
- **reader.ts** (`cli/src/photos-db/`) — reads Photos.sqlite read-only. Main
52
-
query joins `ZASSET` + `ZADDITIONALASSETATTRIBUTES`, then six enrichment
53
-
queries (albums, keywords, people, descriptions, edits, rendered resources)
54
-
each return a `Map` keyed by Z_PK. Uses `safeQuery()` for resilience across
55
-
macOS versions.
56
-
- **backup.ts** (`cli/src/commands/`) — orchestrates filter → batch → export →
57
-
upload → manifest. Batches of 50 assets sent to ladder subprocess via JSON
58
-
stdin/stdout.
45
+
- **LadderKit** provides `PhotoLibrary` (PhotoKit), `PhotosDatabase`
46
+
(Photos.sqlite enrichment), and `PhotoExporter` (export with SHA-256 hashing +
47
+
AppleScript fallback for iCloud-only assets). Called directly as a library.
59
48
- **S3 key format** — originals: `originals/{year}/{month}/{uuid}.{ext}`,
60
49
metadata: `metadata/assets/{uuid}.json`
61
50
- **Manifest** (`manifest.json` on S3) — maps UUID →
62
51
`{ s3Key, checksum, backedUpAt }`. S3 is the single source of truth. Saved to
63
-
S3 every 50 assets during backup. No local manifest file. Existing local
64
-
manifests at `~/.attic/manifest.json` are migrated to S3 on first run.
52
+
S3 every 50 assets during backup.
65
53
66
-
All external dependencies are behind interfaces (`S3Provider`, `Exporter`,
67
-
`ManifestStore`, `PhotosDbReader`) for testability.
54
+
All external dependencies are behind protocols (`S3Providing`, `ManifestStoring`,
55
+
`ConfigProviding`, `KeychainProviding`, `ExportProviding`) for testability.
56
+
57
+
## CLI Commands
58
+
59
+
| Command | Description |
60
+
|---------|-------------|
61
+
| `scan` | Scan Photos library, show summary |
62
+
| `status` | Show backup progress vs manifest |
63
+
| `backup` | Back up photos/videos to S3 |
64
+
| `verify` | Verify S3 objects against manifest |
65
+
| `refresh-metadata` | Re-upload metadata JSON |
66
+
| `rebuild` | Rebuild manifest from S3 metadata |
67
+
| `init` | Interactive S3 setup |
68
68
69
69
## Testing Patterns
70
70
71
71
Tests use mock implementations — never external services or credentials:
72
72
73
-
- `createMockS3Provider()` — in-memory `Map<string, Uint8Array>`
74
-
- `createMockExporter()` — returns pre-configured assets from a `Map`
75
-
- `makeAsset()` helper in test files creates `PhotoAsset` with sensible defaults
76
-
and partial overrides
73
+
- `MockS3Provider` — in-memory `[String: Data]`
74
+
- `MockExportProvider` — returns canned export results
75
+
- `TimeoutExportProvider` — simulates batch timeouts + deferred retry
76
+
77
+
Uses Swift Testing framework (`@Test`, `#expect`, `@Suite`).
77
78
78
79
## Reference Docs
79
80
80
-
- [Architecture](docs/architecture.md) — pipeline, reader, ladder protocol,
81
-
manifest, interfaces
81
+
- [Architecture](docs/architecture.md) — pipeline, reader, manifest, interfaces
82
82
- [Asset Metadata](docs/metadata.md) — per-asset JSON schema uploaded to S3
83
83
84
84
## Conventions
85
85
86
86
- Files should stay under 500 lines
87
-
- Use `AssetKind.PHOTO` / `AssetKind.VIDEO` constants, not magic numbers
87
+
- Use LadderKit's `AssetKind` constants, not magic numbers
88
88
- S3 keys and UUIDs are validated with regex before interpolation (path
89
89
traversal prevention)
90
-
- `removeStagedFile()` constrains deletion to the staging directory
90
+
- All dependencies injected via protocols
91
+
- Swift 6 strict concurrency — all types are `Sendable` where needed
+39
Package.swift
+39
Package.swift
···
1
+
// swift-tools-version: 6.1
2
+
3
+
import PackageDescription
4
+
5
+
let package = Package(
6
+
name: "attic",
7
+
platforms: [.macOS(.v14)],
8
+
products: [
9
+
.library(name: "AtticCore", targets: ["AtticCore"]),
10
+
],
11
+
dependencies: [
12
+
.package(url: "https://github.com/awslabs/aws-sdk-swift.git", from: "1.6.0"),
13
+
.package(url: "https://github.com/apple/swift-argument-parser.git", from: "1.7.0"),
14
+
.package(path: "../ladder"),
15
+
],
16
+
targets: [
17
+
.target(
18
+
name: "AtticCore",
19
+
dependencies: [
20
+
.product(name: "AWSS3", package: "aws-sdk-swift"),
21
+
.product(name: "LadderKit", package: "ladder"),
22
+
],
23
+
path: "Sources/AtticCore"
24
+
),
25
+
.executableTarget(
26
+
name: "AtticCLI",
27
+
dependencies: [
28
+
"AtticCore",
29
+
.product(name: "ArgumentParser", package: "swift-argument-parser"),
30
+
],
31
+
path: "Sources/AtticCLI"
32
+
),
33
+
.testTarget(
34
+
name: "AtticCoreTests",
35
+
dependencies: ["AtticCore"],
36
+
path: "Tests/AtticCoreTests"
37
+
),
38
+
]
39
+
)
+80
-80
README.md
+80
-80
README.md
···
6
6
7
7
Back up your iCloud Photos library to S3-compatible storage.
8
8
9
-
Attic reads the Photos.sqlite database directly, exports originals via a
10
-
companion Swift tool called [ladder](https://github.com/tijs/ladder), and
11
-
uploads them to an S3-compatible bucket. A local manifest tracks what has
12
-
already been backed up so subsequent runs only upload new assets.
9
+
Attic reads your Photos library via PhotoKit, enriches metadata from
10
+
Photos.sqlite, exports originals with SHA-256 hashing, and uploads them to an
11
+
S3-compatible bucket. A manifest on S3 tracks what has already been backed up so
12
+
subsequent runs only upload new assets.
13
+
14
+
Uses [LadderKit](https://github.com/tijs/ladder) for PhotoKit access and
15
+
AppleScript fallback for iCloud-only assets.
13
16
14
17
Works with any S3-compatible provider. EU-friendly options include
15
18
[Scaleway](https://www.scaleway.com/en/object-storage/),
···
24
27
brew install tijs/tap/attic
25
28
```
26
29
27
-
### From source (requires Deno v2+)
30
+
### From source (requires Swift 6.x, macOS 14+)
28
31
29
32
```bash
30
33
git clone https://github.com/tijs/attic.git
31
34
cd attic
32
-
deno task install
35
+
swift build -c release
36
+
sudo cp .build/release/AtticCLI /usr/local/bin/attic
33
37
```
34
-
35
-
This installs `attic` to `~/.deno/bin/`. Make sure that's on your PATH.
36
38
37
39
## Prerequisites
38
40
39
-
- The [ladder](https://github.com/tijs/ladder) binary. Ladder is a separate
40
-
Swift tool that uses PhotoKit to export original photo/video files from the
41
-
Photos library.
41
+
- macOS 14+ (Sonoma), Apple Silicon
42
42
- An S3-compatible storage bucket and API credentials
43
-
- macOS (Photos.sqlite access and Keychain are macOS-only)
43
+
44
+
## Permissions
45
+
46
+
On first run, macOS will show permission dialogs for:
47
+
48
+
- **Photos library access** — required to read your photo/video assets
49
+
- **Keychain access** — required to read stored S3 credentials
50
+
51
+
Both are one-time prompts. Click "Allow" or "Always Allow" to proceed. These
52
+
permissions can be reviewed in System Settings → Privacy & Security.
44
53
45
54
## Setup
46
55
···
54
63
is saved to `~/.attic/config.json` and credentials are stored in the macOS
55
64
Keychain.
56
65
57
-
Build the ladder binary and add it to your PATH (see
58
-
[ladder](https://github.com/tijs/ladder) for details):
59
-
60
-
```bash
61
-
git clone https://github.com/tijs/ladder.git
62
-
cd ladder
63
-
swift build -c release
64
-
sudo cp .build/release/ladder /usr/local/bin/
65
-
```
66
-
67
-
Alternatively, pass `--ladder <path>` to the backup command or set the
68
-
`LADDER_PATH` environment variable.
69
-
70
66
## Commands
71
67
72
68
### init
···
79
75
80
76
### scan
81
77
82
-
Scan the Photos library and print statistics (asset counts, sizes, types, local
83
-
vs iCloud-only).
78
+
Scan the Photos library and print statistics (asset counts, types, favorites,
79
+
edits).
84
80
85
81
```bash
86
82
attic scan
···
88
84
89
85
### status
90
86
91
-
Compare the Photos database against the local backup manifest to show how many
92
-
assets are backed up vs pending.
87
+
Compare the Photos library against the S3 manifest to show how many assets are
88
+
backed up vs pending.
93
89
94
90
```bash
95
91
attic status
···
97
93
98
94
### backup
99
95
100
-
Export pending assets via ladder and upload originals + metadata JSON to S3.
96
+
Export pending assets and upload originals + metadata JSON to S3.
101
97
102
98
```bash
103
99
attic backup
104
100
```
105
101
106
-
| Flag | Description |
107
-
| --------------------- | -------------------------------------------------------- |
108
-
| `--dry-run` | Show what would be uploaded without uploading |
109
-
| `--limit N` | Stop after N assets (useful for test runs) |
110
-
| `--batch-size N` | Assets per export batch (default: 50) |
111
-
| `--type photo\|video` | Only back up photos or videos |
112
-
| `--bucket NAME` | Override bucket from config |
113
-
| `--ladder PATH` | Path to the ladder binary (or set `LADDER_PATH` env var) |
114
-
| `--db PATH` | Path to Photos.sqlite |
115
-
| `-q, --quiet` | Suppress progress output (for unattended use) |
116
-
| `--log PATH` | Append structured JSONL log to file |
117
-
| `--notify` | Send macOS notification on completion |
102
+
| Flag | Description |
103
+
| --------------------- | ------------------------------------------ |
104
+
| `--dry-run` | Show what would be uploaded without uploading |
105
+
| `--limit N` | Stop after N assets (useful for test runs) |
106
+
| `--batch-size N` | Assets per export batch (default: 50) |
107
+
| `--type photo\|video` | Only back up photos or videos |
108
+
109
+
During a backup, a live-updating terminal dashboard shows progress, speed,
110
+
current file, and elapsed time. Non-TTY output (pipes, CI) falls back to
111
+
line-by-line progress.
118
112
119
113
### verify
120
114
121
-
Verify backup integrity by checking S3 objects against the manifest.
115
+
Verify backup integrity by confirming every manifest entry exists in S3.
122
116
123
117
```bash
124
118
attic verify
125
119
```
126
120
127
-
| Flag | Description |
128
-
| -------------------- | ---------------------------------------------------------- |
129
-
| `--deep` | Download each object and re-verify SHA-256 checksum (slow) |
130
-
| `--rebuild-manifest` | Reconstruct the local manifest from S3 metadata files |
131
-
| `--bucket NAME` | Override bucket from config |
121
+
| Flag | Description |
122
+
| ----------------- | --------------------------------- |
123
+
| `--concurrency N` | Concurrent requests (default: 20) |
132
124
133
125
### refresh-metadata
134
126
135
127
Re-upload metadata JSON for already backed-up assets without re-uploading the
136
-
original files. Useful after adding new metadata fields or enrichments.
128
+
original files. Useful after adding new metadata fields.
137
129
138
130
```bash
139
131
attic refresh-metadata
···
143
135
| ----------------- | -------------------------------- |
144
136
| `--dry-run` | Show what would be uploaded |
145
137
| `--concurrency N` | Concurrent uploads (default: 20) |
146
-
| `--bucket NAME` | Override bucket from config |
147
-
| `--db PATH` | Path to Photos.sqlite |
138
+
139
+
### rebuild
140
+
141
+
Rebuild the manifest from S3 metadata files (disaster recovery).
142
+
143
+
```bash
144
+
attic rebuild
145
+
```
148
146
149
147
## Configuration
150
148
151
-
Attic stores its configuration at `~/.attic/config.json` (see
152
-
`config.example.json` for a template):
149
+
Attic stores its configuration at `~/.attic/config.json`:
153
150
154
151
```json
155
152
{
···
168
165
above. Credentials are always stored in the macOS Keychain, never in config
169
166
files or environment variables.
170
167
171
-
`scan` and `status` work without config (they only read Photos.sqlite). `backup`
172
-
and `verify` require config and will tell you to run `attic init` if it's
173
-
missing.
168
+
`scan` works without config (it only reads the Photos library). All other
169
+
commands require config and S3 credentials — run `attic init` if missing.
174
170
175
-
## Development
176
-
177
-
If you're working on attic itself, use `deno task` to run commands from source:
171
+
## Architecture
178
172
179
-
```bash
180
-
deno task check # Type check
181
-
deno task test # Run tests
182
-
deno task lint # Lint
183
-
deno task fmt # Format
184
-
deno task compile # Build standalone binary
173
+
```
174
+
Photos Library → PhotoKit (LadderKit) → AssetInfo[]
175
+
↓
176
+
BackupPipeline (AtticCore)
177
+
↓ ↓
178
+
S3 upload Manifest update
185
179
```
186
180
187
-
## Testing
181
+
The project is a Swift package with three targets:
182
+
183
+
- **AtticCore** — shared library: S3 provider, manifest, config, keychain,
184
+
metadata, backup/verify/refresh pipelines. Designed for reuse by both the CLI
185
+
and a planned macOS menu bar app.
186
+
- **AtticCLI** — executable: ArgumentParser commands, terminal renderer
187
+
- **AtticCoreTests** — tests using Swift Testing framework
188
+
189
+
All external dependencies are behind protocols (`S3Providing`, `ManifestStoring`,
190
+
`ConfigProviding`, `KeychainProviding`, `ExportProviding`) for testability.
191
+
192
+
## Development
188
193
189
194
```bash
190
-
deno task test
195
+
swift build # Build
196
+
swift test # Run tests
197
+
swift build -c release # Release build
198
+
swift test --filter "testName" # Run single test
191
199
```
192
200
193
-
Tests use dependency injection with mock implementations for the S3 client and
194
-
exporter, so no external services or credentials are needed.
201
+
Tests use dependency injection with mock implementations (MockS3Provider,
202
+
MockExportProvider) — no external services or credentials needed.
195
203
196
204
## Documentation
197
205
198
-
- [Architecture](docs/architecture.md) -- How attic works: the backup pipeline,
199
-
Photos.sqlite reader, ladder protocol, manifest lifecycle, and design
200
-
boundaries
201
-
- [Asset Metadata](docs/metadata.md) -- Schema reference for the per-asset JSON
206
+
- [Architecture](docs/architecture.md) — How attic works: the backup pipeline,
207
+
photo library access, manifest lifecycle, and design boundaries
208
+
- [Asset Metadata](docs/metadata.md) — Schema reference for the per-asset JSON
202
209
uploaded to S3
203
-
- [Unattended Backups](docs/unattended-backups.md) -- Set up daily scheduled
204
-
backups via launchd on a dedicated Mac
205
-
206
-
## Future Plans
207
-
208
-
- **Rendered edit backup** -- Detect and upload edited versions alongside
209
-
originals (see `docs/plans/`)
+20
Sources/AtticCLI/AtticCLI.swift
+20
Sources/AtticCLI/AtticCLI.swift
···
1
+
import ArgumentParser
2
+
import AtticCore
3
+
4
+
@main
5
+
struct AtticCLI: AsyncParsableCommand {
6
+
static let configuration = CommandConfiguration(
7
+
commandName: "attic",
8
+
abstract: "Back up iCloud Photos to S3-compatible storage.",
9
+
version: AtticCore.version,
10
+
subcommands: [
11
+
ScanCommand.self,
12
+
StatusCommand.self,
13
+
BackupCommand.self,
14
+
VerifyCommand.self,
15
+
RefreshMetadataCommand.self,
16
+
RebuildCommand.self,
17
+
InitCommand.self,
18
+
]
19
+
)
20
+
}
+93
Sources/AtticCLI/BackupCommand.swift
+93
Sources/AtticCLI/BackupCommand.swift
···
1
+
import ArgumentParser
2
+
import Foundation
3
+
import AtticCore
4
+
import LadderKit
5
+
6
+
struct BackupCommand: AsyncParsableCommand {
7
+
static let configuration = CommandConfiguration(
8
+
commandName: "backup",
9
+
abstract: "Back up photos and videos to S3."
10
+
)
11
+
12
+
@Option(name: .long, help: "Number of assets per export batch.")
13
+
var batchSize: Int = 50
14
+
15
+
@Option(name: .long, help: "Maximum number of assets to back up (0 = unlimited).")
16
+
var limit: Int = 0
17
+
18
+
@Option(name: .long, help: "Only back up assets of this type (photo or video).")
19
+
var type: String?
20
+
21
+
@Flag(name: .long, help: "Show what would be backed up without uploading.")
22
+
var dryRun: Bool = false
23
+
24
+
func run() async throws {
25
+
let (config, s3, manifestStore) = try Dependencies.makeBackupDeps()
26
+
var manifest = try await Dependencies.loadManifest(store: manifestStore)
27
+
let assets = Dependencies.loadAssets()
28
+
29
+
let assetKind: AssetKind? = switch type?.lowercased() {
30
+
case "photo": .photo
31
+
case "video": .video
32
+
default: nil
33
+
}
34
+
35
+
let stagingDir = FileManager.default.temporaryDirectory
36
+
.appendingPathComponent("attic-staging-\(UUID().uuidString)")
37
+
try FileManager.default.createDirectory(at: stagingDir, withIntermediateDirectories: true)
38
+
defer { try? FileManager.default.removeItem(at: stagingDir) }
39
+
40
+
let exporter = LadderKitExportProvider(stagingDir: stagingDir)
41
+
42
+
// Pre-flight permission check
43
+
try await exporter.checkPermissions()
44
+
45
+
let isTTY = isatty(STDOUT_FILENO) != 0
46
+
let progress: any BackupProgressDelegate = isTTY
47
+
? TerminalRenderer()
48
+
: LogProgressDelegate()
49
+
50
+
let options = BackupOptions(
51
+
batchSize: batchSize,
52
+
limit: limit,
53
+
type: assetKind,
54
+
dryRun: dryRun
55
+
)
56
+
57
+
let report = try await runBackup(
58
+
assets: assets,
59
+
manifest: &manifest,
60
+
manifestStore: manifestStore,
61
+
exporter: exporter,
62
+
s3: s3,
63
+
options: options,
64
+
progress: progress
65
+
)
66
+
67
+
if !isTTY {
68
+
debugPrint("Backup complete: \(report.uploaded) uploaded, \(report.failed) failed (\(formatBytes(report.totalBytes)))")
69
+
}
70
+
}
71
+
}
72
+
73
+
/// Simple line-by-line progress for non-TTY output (CI, pipes).
74
+
struct LogProgressDelegate: BackupProgressDelegate {
75
+
func backupStarted(pending: Int, photos: Int, videos: Int) {
76
+
debugPrint("Starting backup: \(pending) assets (\(photos) photos, \(videos) videos)")
77
+
}
78
+
func batchStarted(batchNumber: Int, totalBatches: Int, assetCount: Int) {
79
+
debugPrint("Batch \(batchNumber)/\(totalBatches) (\(assetCount) assets)")
80
+
}
81
+
func assetUploaded(uuid: String, filename: String, type: AssetKind, size: Int) {
82
+
debugPrint(" ✓ \(filename) (\(formatBytes(size)))")
83
+
}
84
+
func assetFailed(uuid: String, filename: String, message: String) {
85
+
debugPrint(" ✗ \(filename): \(message)")
86
+
}
87
+
func manifestSaved(entriesCount: Int) {
88
+
debugPrint(" Manifest saved (\(entriesCount) entries)")
89
+
}
90
+
func backupCompleted(uploaded: Int, failed: Int, totalBytes: Int) {
91
+
debugPrint("Done: \(uploaded) uploaded, \(failed) failed (\(formatBytes(totalBytes)))")
92
+
}
93
+
}
+88
Sources/AtticCLI/Dependencies.swift
+88
Sources/AtticCLI/Dependencies.swift
···
1
+
import Foundation
2
+
import AtticCore
3
+
import LadderKit
4
+
5
+
/// Builds the real dependencies for CLI commands from config + keychain.
6
+
enum Dependencies {
7
+
/// Load config from default location, or throw with a helpful message.
8
+
static func loadConfig() throws -> AtticConfig {
9
+
let provider = FileConfigProvider()
10
+
guard let config = try provider.load() else {
11
+
throw CLIError.notInitialized
12
+
}
13
+
return config
14
+
}
15
+
16
+
/// Load credentials from the macOS Keychain.
17
+
static func loadCredentials(config: AtticConfig) throws -> S3Credentials {
18
+
let keychain = SecurityKeychain()
19
+
return try keychain.loadCredentials(
20
+
accessKeyService: config.keychain.accessKeyService,
21
+
secretKeyService: config.keychain.secretKeyService
22
+
)
23
+
}
24
+
25
+
/// Create an S3 client from config + credentials.
26
+
static func makeS3Client(config: AtticConfig, credentials: S3Credentials) throws -> AWSS3Client {
27
+
try AWSS3Client(
28
+
credentials: credentials,
29
+
bucket: config.bucket,
30
+
endpoint: config.endpoint,
31
+
region: config.region,
32
+
pathStyle: config.pathStyle
33
+
)
34
+
}
35
+
36
+
/// Create the full set of backup dependencies.
37
+
static func makeBackupDeps() throws -> (
38
+
config: AtticConfig,
39
+
s3: AWSS3Client,
40
+
manifestStore: S3ManifestStore
41
+
) {
42
+
let config = try loadConfig()
43
+
let creds = try loadCredentials(config: config)
44
+
let s3 = try makeS3Client(config: config, credentials: creds)
45
+
let manifestStore = S3ManifestStore(s3: s3)
46
+
return (config, s3, manifestStore)
47
+
}
48
+
49
+
/// Load manifest with automatic migration from local file if needed.
50
+
///
51
+
/// On first run after upgrading from the Deno CLI, this detects a local
52
+
/// `~/.attic/manifest.json`, uploads it to S3, and returns it. Subsequent
53
+
/// runs use the S3 manifest directly.
54
+
static func loadManifest(store: S3ManifestStore) async throws -> Manifest {
55
+
try await loadManifestWithMigration(s3Store: store)
56
+
}
57
+
58
+
/// Default system Photos library location.
59
+
static var defaultLibraryURL: URL {
60
+
FileManager.default.homeDirectoryForCurrentUser
61
+
.appendingPathComponent("Pictures")
62
+
.appendingPathComponent("Photos Library.photoslibrary")
63
+
}
64
+
65
+
/// Load assets from PhotoKit + enrich from Photos.sqlite.
66
+
static func loadAssets() -> [AssetInfo] {
67
+
let library = PhotoKitLibrary()
68
+
var assets = library.enumerateAssets()
69
+
70
+
if let dbPath = PhotosLibraryPath.databasePath(for: defaultLibraryURL) {
71
+
let enrichment = PhotosDatabase.readEnrichment(dbPath: dbPath)
72
+
PhotosDatabase.enrich(&assets, with: enrichment)
73
+
}
74
+
75
+
return assets
76
+
}
77
+
}
78
+
79
+
enum CLIError: Error, CustomStringConvertible {
80
+
case notInitialized
81
+
82
+
var description: String {
83
+
switch self {
84
+
case .notInitialized:
85
+
"Attic is not configured. Run 'attic init' first."
86
+
}
87
+
}
88
+
}
+81
Sources/AtticCLI/InitCommand.swift
+81
Sources/AtticCLI/InitCommand.swift
···
1
+
import ArgumentParser
2
+
import Foundation
3
+
import AtticCore
4
+
5
+
struct InitCommand: AsyncParsableCommand {
6
+
static let configuration = CommandConfiguration(
7
+
commandName: "init",
8
+
abstract: "Set up Attic with S3 endpoint, bucket, and credentials."
9
+
)
10
+
11
+
func run() async throws {
12
+
debugPrint("Attic Setup")
13
+
debugPrint("===========")
14
+
debugPrint("")
15
+
16
+
let endpoint = prompt("S3 endpoint (https://...): ")
17
+
guard endpoint.hasPrefix("https://") else {
18
+
throw ValidationError("Endpoint must start with https://")
19
+
}
20
+
21
+
let region = prompt("Region: ")
22
+
let bucket = prompt("Bucket name: ")
23
+
let pathStyleInput = prompt("Use path-style URLs? (Y/n): ")
24
+
let pathStyle = pathStyleInput.lowercased() != "n"
25
+
26
+
let accessKey = prompt("Access key ID: ")
27
+
let secretKey = promptSecret("Secret access key: ")
28
+
29
+
let config = AtticConfig(
30
+
endpoint: endpoint,
31
+
region: region,
32
+
bucket: bucket,
33
+
pathStyle: pathStyle
34
+
)
35
+
// Save config
36
+
let configProvider = FileConfigProvider()
37
+
try configProvider.write(config)
38
+
39
+
// Save credentials to Keychain
40
+
let keychain = SecurityKeychain()
41
+
try keychain.store(service: config.keychain.accessKeyService, value: accessKey)
42
+
try keychain.store(service: config.keychain.secretKeyService, value: secretKey)
43
+
44
+
debugPrint("")
45
+
debugPrint("Configuration saved.")
46
+
debugPrint("Credentials stored in macOS Keychain.")
47
+
debugPrint("")
48
+
debugPrint("Next steps:")
49
+
debugPrint(" attic scan Scan your Photos library")
50
+
debugPrint(" attic status Check backup progress")
51
+
debugPrint(" attic backup Start backing up")
52
+
debugPrint("")
53
+
debugPrint("macOS will ask for permission to access Photos and")
54
+
debugPrint("Keychain on first run — both are expected and required.")
55
+
}
56
+
}
57
+
58
+
// MARK: - Interactive prompts
59
+
60
+
private func prompt(_ message: String) -> String {
61
+
print(message, terminator: "")
62
+
return readLine(strippingNewline: true) ?? ""
63
+
}
64
+
65
+
private func promptSecret(_ message: String) -> String {
66
+
print(message, terminator: "")
67
+
68
+
// Disable echo for secret input
69
+
var oldTermios = termios()
70
+
tcgetattr(STDIN_FILENO, &oldTermios)
71
+
var newTermios = oldTermios
72
+
newTermios.c_lflag &= ~UInt(ECHO)
73
+
tcsetattr(STDIN_FILENO, TCSANOW, &newTermios)
74
+
75
+
let value = readLine(strippingNewline: true) ?? ""
76
+
77
+
// Restore echo
78
+
tcsetattr(STDIN_FILENO, TCSANOW, &oldTermios)
79
+
print("") // newline after hidden input
80
+
return value
81
+
}
+24
Sources/AtticCLI/LadderKitExportProvider.swift
+24
Sources/AtticCLI/LadderKitExportProvider.swift
···
1
+
import Foundation
2
+
import AtticCore
3
+
import LadderKit
4
+
5
+
/// Bridges LadderKit's PhotoExporter to AtticCore's ExportProviding protocol.
6
+
struct LadderKitExportProvider: ExportProviding {
7
+
private let exporter: PhotoExporter
8
+
9
+
init(stagingDir: URL, library: PhotoLibrary = PhotoKitLibrary()) {
10
+
self.exporter = PhotoExporter(
11
+
stagingDir: stagingDir,
12
+
library: library,
13
+
scriptExporter: AppleScriptRunner()
14
+
)
15
+
}
16
+
17
+
func exportBatch(uuids: [String]) async throws -> ExportResponse {
18
+
try await exporter.export(uuids: uuids)
19
+
}
20
+
21
+
func checkPermissions() async throws {
22
+
try await exporter.checkPermissions()
23
+
}
24
+
}
+35
Sources/AtticCLI/RebuildCommand.swift
+35
Sources/AtticCLI/RebuildCommand.swift
···
1
+
import ArgumentParser
2
+
import AtticCore
3
+
4
+
struct RebuildCommand: AsyncParsableCommand {
5
+
static let configuration = CommandConfiguration(
6
+
commandName: "rebuild",
7
+
abstract: "Rebuild manifest from S3 metadata files (disaster recovery)."
8
+
)
9
+
10
+
func run() async throws {
11
+
let (_, s3, manifestStore) = try Dependencies.makeBackupDeps()
12
+
13
+
debugPrint("Rebuilding manifest from S3 metadata...")
14
+
15
+
let (manifest, report) = try await runRebuildManifest(
16
+
s3: s3, manifestStore: manifestStore
17
+
)
18
+
19
+
debugPrint("")
20
+
debugPrint("Rebuild Results")
21
+
debugPrint("===============")
22
+
debugPrint("Recovered: \(report.recovered)")
23
+
debugPrint("Skipped: \(report.skipped)")
24
+
debugPrint("Errors: \(report.errors.count)")
25
+
debugPrint("")
26
+
debugPrint("Manifest saved with \(manifest.entries.count) entries.")
27
+
28
+
if !report.errors.isEmpty {
29
+
debugPrint("")
30
+
for err in report.errors.prefix(10) {
31
+
debugPrint(" ✗ \(err.key): \(err.message)")
32
+
}
33
+
}
34
+
}
35
+
}
+49
Sources/AtticCLI/RefreshMetadataCommand.swift
+49
Sources/AtticCLI/RefreshMetadataCommand.swift
···
1
+
import ArgumentParser
2
+
import AtticCore
3
+
4
+
struct RefreshMetadataCommand: AsyncParsableCommand {
5
+
static let configuration = CommandConfiguration(
6
+
commandName: "refresh-metadata",
7
+
abstract: "Re-generate and upload metadata JSON for all backed-up assets."
8
+
)
9
+
10
+
@Option(name: .long, help: "Number of concurrent uploads.")
11
+
var concurrency: Int = 20
12
+
13
+
@Flag(name: .long, help: "Show what would be refreshed without uploading.")
14
+
var dryRun: Bool = false
15
+
16
+
func run() async throws {
17
+
let (_, s3, manifestStore) = try Dependencies.makeBackupDeps()
18
+
let manifest = try await Dependencies.loadManifest(store: manifestStore)
19
+
let assets = Dependencies.loadAssets()
20
+
21
+
let options = RefreshMetadataOptions(concurrency: concurrency, dryRun: dryRun)
22
+
23
+
if dryRun {
24
+
let backedUpCount = assets.filter { manifest.isBackedUp($0.uuid) }.count
25
+
debugPrint("Dry run: would refresh metadata for \(backedUpCount) assets.")
26
+
return
27
+
}
28
+
29
+
debugPrint("Refreshing metadata for backed-up assets...")
30
+
31
+
let report = try await runRefreshMetadata(
32
+
assets: assets, manifest: manifest, s3: s3, options: options
33
+
)
34
+
35
+
debugPrint("")
36
+
debugPrint("Refresh Results")
37
+
debugPrint("===============")
38
+
debugPrint("Updated: \(report.updated)")
39
+
debugPrint("Failed: \(report.failed)")
40
+
debugPrint("Bytes: \(formatBytes(report.totalBytes))")
41
+
42
+
if !report.errors.isEmpty {
43
+
debugPrint("")
44
+
for err in report.errors.prefix(10) {
45
+
debugPrint(" ✗ \(err.uuid): \(err.message)")
46
+
}
47
+
}
48
+
}
49
+
}
+49
Sources/AtticCLI/ScanCommand.swift
+49
Sources/AtticCLI/ScanCommand.swift
···
1
+
import ArgumentParser
2
+
import AtticCore
3
+
import LadderKit
4
+
5
+
struct ScanCommand: AsyncParsableCommand {
6
+
static let configuration = CommandConfiguration(
7
+
commandName: "scan",
8
+
abstract: "Scan your Photos library and show a summary."
9
+
)
10
+
11
+
func run() async throws {
12
+
let assets = Dependencies.loadAssets()
13
+
14
+
if assets.isEmpty {
15
+
debugPrint("No assets found in Photos library.")
16
+
return
17
+
}
18
+
19
+
let photos = assets.filter { $0.kind == .photo }
20
+
let videos = assets.filter { $0.kind == .video }
21
+
22
+
debugPrint("Photos Library Scan")
23
+
debugPrint("===================")
24
+
debugPrint("Total assets: \(assets.count)")
25
+
debugPrint(" Photos: \(photos.count)")
26
+
debugPrint(" Videos: \(videos.count)")
27
+
28
+
// Group by UTI
29
+
var utiCounts: [String: Int] = [:]
30
+
for asset in assets {
31
+
let uti = asset.uniformTypeIdentifier ?? "unknown"
32
+
utiCounts[uti, default: 0] += 1
33
+
}
34
+
let topUTIs = utiCounts.sorted { $0.value > $1.value }.prefix(10)
35
+
36
+
debugPrint("")
37
+
debugPrint("Top file types:")
38
+
for (uti, count) in topUTIs {
39
+
debugPrint(" \(uti): \(count)")
40
+
}
41
+
42
+
// Favorites + edited
43
+
let favorites = assets.filter(\.isFavorite).count
44
+
let edited = assets.filter(\.hasEdit).count
45
+
debugPrint("")
46
+
debugPrint("Favorites: \(favorites)")
47
+
debugPrint("Edited: \(edited)")
48
+
}
49
+
}
+47
Sources/AtticCLI/StatusCommand.swift
+47
Sources/AtticCLI/StatusCommand.swift
···
1
+
import ArgumentParser
2
+
import AtticCore
3
+
import LadderKit
4
+
5
+
struct StatusCommand: AsyncParsableCommand {
6
+
static let configuration = CommandConfiguration(
7
+
commandName: "status",
8
+
abstract: "Show backup progress — how many assets are backed up vs pending."
9
+
)
10
+
11
+
func run() async throws {
12
+
let (config, _, manifestStore) = try Dependencies.makeBackupDeps()
13
+
let manifest = try await Dependencies.loadManifest(store: manifestStore)
14
+
let assets = Dependencies.loadAssets()
15
+
16
+
// Single-pass counting
17
+
var backedUpPhotos = 0, backedUpVideos = 0, pendingPhotos = 0, pendingVideos = 0
18
+
var backedUpBytes = 0
19
+
for asset in assets {
20
+
if manifest.isBackedUp(asset.uuid) {
21
+
if asset.kind == .photo { backedUpPhotos += 1 } else { backedUpVideos += 1 }
22
+
backedUpBytes += manifest.entries[asset.uuid]?.size ?? 0
23
+
} else {
24
+
if asset.kind == .photo { pendingPhotos += 1 } else { pendingVideos += 1 }
25
+
}
26
+
}
27
+
let backedUpCount = backedUpPhotos + backedUpVideos
28
+
let pendingCount = pendingPhotos + pendingVideos
29
+
30
+
let pct = assets.isEmpty ? 100.0 : Double(backedUpCount) / Double(assets.count) * 100
31
+
32
+
debugPrint("Attic Backup Status")
33
+
debugPrint("====================")
34
+
debugPrint("Bucket: \(config.bucket)")
35
+
debugPrint("Completion: \(String(format: "%.1f", pct))%")
36
+
debugPrint("")
37
+
debugPrint("Backed up: \(backedUpCount) (\(formatBytes(backedUpBytes)))")
38
+
debugPrint(" Photos: \(backedUpPhotos)")
39
+
debugPrint(" Videos: \(backedUpVideos)")
40
+
debugPrint("")
41
+
debugPrint("Pending: \(pendingCount)")
42
+
debugPrint(" Photos: \(pendingPhotos)")
43
+
debugPrint(" Videos: \(pendingVideos)")
44
+
debugPrint("")
45
+
debugPrint("Manifest: \(manifest.entries.count) entries")
46
+
}
47
+
}
+167
Sources/AtticCLI/TerminalRenderer.swift
+167
Sources/AtticCLI/TerminalRenderer.swift
···
1
+
import Foundation
2
+
import AtticCore
3
+
import LadderKit
4
+
5
+
/// ANSI live-updating dashboard for backup progress.
6
+
///
7
+
/// Redraws a fixed-height block in-place using ANSI escape codes.
8
+
/// Shows progress bar, counts, speed, current file, and elapsed time.
9
+
final class TerminalRenderer: BackupProgressDelegate, @unchecked Sendable {
10
+
private let lock = NSLock()
11
+
private var state = RenderState()
12
+
private var startTime: Date?
13
+
private var lastRenderTime: Date?
14
+
15
+
private struct RenderState {
16
+
var total: Int = 0
17
+
var photos: Int = 0
18
+
var videos: Int = 0
19
+
var uploaded: Int = 0
20
+
var failed: Int = 0
21
+
var totalBytes: Int = 0
22
+
var currentFile: String = ""
23
+
var currentBatch: Int = 0
24
+
var totalBatches: Int = 0
25
+
var uploadedPhotos: Int = 0
26
+
var uploadedVideos: Int = 0
27
+
var headerPrinted: Bool = false
28
+
}
29
+
30
+
// MARK: - BackupProgressDelegate
31
+
32
+
func backupStarted(pending: Int, photos: Int, videos: Int) {
33
+
lock.withLock {
34
+
state.total = pending
35
+
state.photos = photos
36
+
state.videos = videos
37
+
startTime = Date()
38
+
}
39
+
render()
40
+
}
41
+
42
+
func batchStarted(batchNumber: Int, totalBatches: Int, assetCount: Int) {
43
+
lock.withLock {
44
+
state.currentBatch = batchNumber
45
+
state.totalBatches = totalBatches
46
+
}
47
+
render()
48
+
}
49
+
50
+
func assetUploaded(uuid: String, filename: String, type: AssetKind, size: Int) {
51
+
lock.withLock {
52
+
state.uploaded += 1
53
+
state.totalBytes += size
54
+
state.currentFile = "\(filename) (\(formatBytes(size)))"
55
+
if type == .photo { state.uploadedPhotos += 1 }
56
+
else { state.uploadedVideos += 1 }
57
+
}
58
+
render()
59
+
}
60
+
61
+
func assetFailed(uuid: String, filename: String, message: String) {
62
+
lock.withLock {
63
+
state.failed += 1
64
+
state.currentFile = "\(filename) — \(message)"
65
+
}
66
+
render()
67
+
}
68
+
69
+
func manifestSaved(entriesCount: Int) {
70
+
// No visual update needed
71
+
}
72
+
73
+
func backupCompleted(uploaded: Int, failed: Int, totalBytes: Int) {
74
+
lock.withLock {
75
+
state.uploaded = uploaded
76
+
state.failed = failed
77
+
state.totalBytes = totalBytes
78
+
state.currentFile = ""
79
+
}
80
+
renderFinal()
81
+
}
82
+
83
+
84
+
// MARK: - Rendering
85
+
86
+
private func render() {
87
+
let (s, elapsed): (RenderState, TimeInterval) = lock.withLock {
88
+
let e = startTime.map { Date().timeIntervalSince($0) } ?? 0
89
+
return (state, e)
90
+
}
91
+
let speed = elapsed > 0 ? Double(s.totalBytes) / elapsed : 0
92
+
93
+
let completed = s.uploaded + s.failed
94
+
let barWidth = 30
95
+
let filled = s.total > 0 ? (completed * barWidth) / s.total : 0
96
+
let bar = String(repeating: "█", count: filled) + String(repeating: "░", count: barWidth - filled)
97
+
98
+
var lines: [String] = []
99
+
100
+
if !s.headerPrinted {
101
+
lines.append("")
102
+
lock.withLock { state.headerPrinted = true }
103
+
}
104
+
105
+
lines.append(" Progress [\(bar)] \(completed)/\(s.total)")
106
+
lines.append(" Photos \(s.uploadedPhotos) uploaded")
107
+
lines.append(" Videos \(s.uploadedVideos) uploaded")
108
+
lines.append(" Speed \(formatBytes(Int(speed)))/s")
109
+
lines.append(" Errors \(s.failed)")
110
+
lines.append("")
111
+
lines.append(" Current \(s.currentFile)")
112
+
lines.append(" Elapsed \(formatDuration(elapsed))")
113
+
114
+
// Move cursor up to overwrite previous render (8 lines of content)
115
+
let now = Date()
116
+
let shouldMoveCursor: Bool = lock.withLock {
117
+
if lastRenderTime != nil {
118
+
return true
119
+
}
120
+
lastRenderTime = now
121
+
return false
122
+
}
123
+
124
+
if shouldMoveCursor {
125
+
// Move up 8 lines and clear each
126
+
print("\u{1b}[\(lines.count)A", terminator: "")
127
+
}
128
+
lock.withLock { lastRenderTime = now }
129
+
130
+
for line in lines {
131
+
print("\u{1b}[2K\(line)")
132
+
}
133
+
134
+
fflush(stdout)
135
+
}
136
+
137
+
private func renderFinal() {
138
+
let s: RenderState = lock.withLock { state }
139
+
let elapsed = lock.withLock { startTime.map { Date().timeIntervalSince($0) } ?? 0 }
140
+
141
+
// Clear the live display
142
+
let lineCount = 8
143
+
print("\u{1b}[\(lineCount)A", terminator: "")
144
+
for _ in 0..<lineCount {
145
+
print("\u{1b}[2K")
146
+
}
147
+
print("\u{1b}[\(lineCount)A", terminator: "")
148
+
149
+
print("Backup complete in \(formatDuration(elapsed))")
150
+
print(" Uploaded: \(s.uploaded) (\(formatBytes(s.totalBytes)))")
151
+
if s.failed > 0 {
152
+
print(" Failed: \(s.failed)")
153
+
}
154
+
155
+
fflush(stdout)
156
+
}
157
+
}
158
+
159
+
// MARK: - Formatting
160
+
161
+
private func formatDuration(_ seconds: TimeInterval) -> String {
162
+
let total = Int(seconds)
163
+
let h = total / 3600
164
+
let m = (total % 3600) / 60
165
+
let s = total % 60
166
+
return String(format: "%02d:%02d:%02d", h, m, s)
167
+
}
+44
Sources/AtticCLI/VerifyCommand.swift
+44
Sources/AtticCLI/VerifyCommand.swift
···
1
+
import ArgumentParser
2
+
import AtticCore
3
+
4
+
struct VerifyCommand: AsyncParsableCommand {
5
+
static let configuration = CommandConfiguration(
6
+
commandName: "verify",
7
+
abstract: "Verify backed-up assets exist in S3."
8
+
)
9
+
10
+
@Option(name: .long, help: "Number of concurrent verification requests.")
11
+
var concurrency: Int = 20
12
+
13
+
func run() async throws {
14
+
let (_, s3, manifestStore) = try Dependencies.makeBackupDeps()
15
+
let manifest = try await Dependencies.loadManifest(store: manifestStore)
16
+
17
+
guard !manifest.entries.isEmpty else {
18
+
debugPrint("Manifest is empty — nothing to verify.")
19
+
return
20
+
}
21
+
22
+
debugPrint("Verifying \(manifest.entries.count) assets...")
23
+
24
+
let report = try await runVerify(manifest: manifest, s3: s3, concurrency: concurrency)
25
+
26
+
debugPrint("")
27
+
debugPrint("Verify Results")
28
+
debugPrint("==============")
29
+
debugPrint("OK: \(report.ok)")
30
+
debugPrint("Missing: \(report.missing)")
31
+
debugPrint("Errors: \(report.failed)")
32
+
33
+
if !report.errors.isEmpty {
34
+
debugPrint("")
35
+
debugPrint("Issues:")
36
+
for err in report.errors.prefix(20) {
37
+
debugPrint(" \(err.uuid): \(err.message)")
38
+
}
39
+
if report.errors.count > 20 {
40
+
debugPrint(" ... and \(report.errors.count - 20) more")
41
+
}
42
+
}
43
+
}
44
+
}
+123
Sources/AtticCore/AWSS3Client.swift
+123
Sources/AtticCore/AWSS3Client.swift
···
1
+
import Foundation
2
+
import AWSS3
3
+
import SmithyIdentity
4
+
import Smithy
5
+
6
+
/// S3 provider using the official AWS SDK for Swift.
7
+
public struct AWSS3Client: S3Providing {
8
+
private let client: S3Client
9
+
private let bucket: String
10
+
11
+
public init(
12
+
credentials: S3Credentials,
13
+
bucket: String,
14
+
endpoint: String,
15
+
region: String,
16
+
pathStyle: Bool
17
+
) throws {
18
+
let awsCredentials = AWSCredentialIdentity(
19
+
accessKey: credentials.accessKeyId,
20
+
secret: credentials.secretAccessKey
21
+
)
22
+
let config = try S3Client.S3ClientConfig(
23
+
awsCredentialIdentityResolver: StaticAWSCredentialIdentityResolver(awsCredentials),
24
+
region: region,
25
+
forcePathStyle: pathStyle,
26
+
endpoint: endpoint
27
+
)
28
+
self.client = S3Client(config: config)
29
+
self.bucket = bucket
30
+
}
31
+
32
+
public func putObject(key: String, body: Data, contentType: String?) async throws {
33
+
let input = PutObjectInput(
34
+
body: .data(body),
35
+
bucket: bucket,
36
+
contentType: contentType,
37
+
key: key
38
+
)
39
+
_ = try await client.putObject(input: input)
40
+
}
41
+
42
+
public func putObject(key: String, fileURL: URL, contentType: String?) async throws {
43
+
// Use memory-mapped I/O to avoid loading the entire file into heap.
44
+
// The kernel pages in only what the network layer reads.
45
+
let data = try Data(contentsOf: fileURL, options: .mappedIfSafe)
46
+
let input = PutObjectInput(
47
+
body: .data(data),
48
+
bucket: bucket,
49
+
contentType: contentType,
50
+
key: key
51
+
)
52
+
_ = try await client.putObject(input: input)
53
+
}
54
+
55
+
public func getObject(key: String) async throws -> Data {
56
+
let input = GetObjectInput(bucket: bucket, key: key)
57
+
let output = try await client.getObject(input: input)
58
+
guard let body = output.body else {
59
+
throw S3ClientError.emptyResponse(key)
60
+
}
61
+
return try await body.readData() ?? Data()
62
+
}
63
+
64
+
public func headObject(key: String) async throws -> S3ObjectMeta? {
65
+
do {
66
+
let input = HeadObjectInput(bucket: bucket, key: key)
67
+
let output = try await client.headObject(input: input)
68
+
return S3ObjectMeta(
69
+
contentLength: Int(output.contentLength ?? 0),
70
+
contentType: output.contentType
71
+
)
72
+
} catch is AWSS3.NotFound {
73
+
return nil
74
+
} catch {
75
+
// Some S3-compatible providers return different error types for 404
76
+
let description = String(describing: error)
77
+
if description.contains("NotFound") || description.contains("NoSuchKey")
78
+
|| description.contains("404") {
79
+
return nil
80
+
}
81
+
throw error
82
+
}
83
+
}
84
+
85
+
public func listObjects(prefix: String) async throws -> [S3ListObject] {
86
+
var results: [S3ListObject] = []
87
+
var continuationToken: String?
88
+
89
+
repeat {
90
+
let input = ListObjectsV2Input(
91
+
bucket: bucket,
92
+
continuationToken: continuationToken,
93
+
prefix: prefix
94
+
)
95
+
let output = try await client.listObjectsV2(input: input)
96
+
97
+
for object in output.contents ?? [] {
98
+
if let key = object.key {
99
+
results.append(S3ListObject(
100
+
key: key,
101
+
size: Int(object.size ?? 0)
102
+
))
103
+
}
104
+
}
105
+
106
+
continuationToken = output.nextContinuationToken
107
+
} while continuationToken != nil
108
+
109
+
return results
110
+
}
111
+
}
112
+
113
+
/// Errors from the S3 client wrapper.
114
+
public enum S3ClientError: Error, CustomStringConvertible {
115
+
case emptyResponse(String)
116
+
117
+
public var description: String {
118
+
switch self {
119
+
case .emptyResponse(let key):
120
+
"Empty response body for S3 key: \(key)"
121
+
}
122
+
}
123
+
}
+185
Sources/AtticCore/AtticConfig.swift
+185
Sources/AtticCore/AtticConfig.swift
···
1
+
import Foundation
2
+
3
+
/// S3 connection configuration stored at ~/.attic/config.json.
4
+
public struct AtticConfig: Codable, Equatable, Sendable {
5
+
public var endpoint: String
6
+
public var region: String
7
+
public var bucket: String
8
+
public var pathStyle: Bool
9
+
public var keychain: KeychainConfig
10
+
11
+
public struct KeychainConfig: Codable, Equatable, Sendable {
12
+
public var accessKeyService: String
13
+
public var secretKeyService: String
14
+
15
+
public init(
16
+
accessKeyService: String = "attic-s3-access-key",
17
+
secretKeyService: String = "attic-s3-secret-key"
18
+
) {
19
+
self.accessKeyService = accessKeyService
20
+
self.secretKeyService = secretKeyService
21
+
}
22
+
}
23
+
24
+
public init(
25
+
endpoint: String,
26
+
region: String,
27
+
bucket: String,
28
+
pathStyle: Bool = true,
29
+
keychain: KeychainConfig = KeychainConfig()
30
+
) {
31
+
self.endpoint = endpoint
32
+
self.region = region
33
+
self.bucket = bucket
34
+
self.pathStyle = pathStyle
35
+
self.keychain = keychain
36
+
}
37
+
}
38
+
39
+
/// Protocol for loading and writing Attic configuration.
40
+
public protocol ConfigProviding: Sendable {
41
+
func load() throws -> AtticConfig?
42
+
func require() throws -> AtticConfig
43
+
func write(_ config: AtticConfig) throws
44
+
}
45
+
46
+
/// File-based config provider reading from ~/.attic/config.json.
47
+
public struct FileConfigProvider: ConfigProviding {
48
+
private let directory: URL
49
+
50
+
public init(directory: URL? = nil) {
51
+
self.directory = directory ?? Self.defaultDirectory
52
+
}
53
+
54
+
public static var defaultDirectory: URL {
55
+
FileManager.default.homeDirectoryForCurrentUser
56
+
.appendingPathComponent(".attic")
57
+
}
58
+
59
+
public func load() throws -> AtticConfig? {
60
+
let path = directory.appendingPathComponent("config.json")
61
+
guard FileManager.default.fileExists(atPath: path.path) else {
62
+
return nil
63
+
}
64
+
let data = try Data(contentsOf: path)
65
+
let raw = try JSONSerialization.jsonObject(with: data)
66
+
return try AtticConfig.validate(raw)
67
+
}
68
+
69
+
public func require() throws -> AtticConfig {
70
+
guard let config = try load() else {
71
+
let path = directory.appendingPathComponent("config.json").path
72
+
throw ConfigError.notFound(path)
73
+
}
74
+
return config
75
+
}
76
+
77
+
public func write(_ config: AtticConfig) throws {
78
+
let fm = FileManager.default
79
+
if !fm.fileExists(atPath: directory.path) {
80
+
try fm.createDirectory(at: directory, withIntermediateDirectories: true)
81
+
try fm.setAttributes(
82
+
[.posixPermissions: 0o700],
83
+
ofItemAtPath: directory.path
84
+
)
85
+
}
86
+
87
+
let encoder = JSONEncoder()
88
+
encoder.outputFormatting = [.prettyPrinted, .sortedKeys]
89
+
var data = try encoder.encode(config)
90
+
data.append(contentsOf: "\n".utf8)
91
+
92
+
let path = directory.appendingPathComponent("config.json")
93
+
let tempPath = directory.appendingPathComponent("config.json.tmp")
94
+
try data.write(to: tempPath, options: .atomic)
95
+
// Move temp file into place (atomic on same filesystem)
96
+
if fm.fileExists(atPath: path.path) {
97
+
try fm.removeItem(at: path)
98
+
}
99
+
try fm.moveItem(at: tempPath, to: path)
100
+
try fm.setAttributes(
101
+
[.posixPermissions: 0o600],
102
+
ofItemAtPath: path.path
103
+
)
104
+
}
105
+
}
106
+
107
+
// MARK: - Validation
108
+
109
+
private nonisolated(unsafe) let bucketPattern = /^[a-z0-9][a-z0-9.\-]{1,61}[a-z0-9]$/
110
+
111
+
extension AtticConfig {
112
+
/// Validate a raw JSON object into an AtticConfig.
113
+
static func validate(_ raw: Any) throws -> AtticConfig {
114
+
guard let obj = raw as? [String: Any] else {
115
+
throw ConfigError.invalid("Config must be a JSON object")
116
+
}
117
+
118
+
guard let endpoint = obj["endpoint"] as? String, !endpoint.isEmpty else {
119
+
throw ConfigError.invalid(
120
+
#"Config: "endpoint" is required (e.g. "https://s3.fr-par.scw.cloud")"#
121
+
)
122
+
}
123
+
guard endpoint.hasPrefix("https://") else {
124
+
throw ConfigError.invalid(
125
+
#"Config: "endpoint" must start with https://"#
126
+
)
127
+
}
128
+
129
+
guard let region = obj["region"] as? String, !region.isEmpty else {
130
+
throw ConfigError.invalid(
131
+
#"Config: "region" is required (e.g. "fr-par")"#
132
+
)
133
+
}
134
+
135
+
guard let bucket = obj["bucket"] as? String, !bucket.isEmpty else {
136
+
throw ConfigError.invalid(#"Config: "bucket" is required"#)
137
+
}
138
+
guard bucket.wholeMatch(of: bucketPattern) != nil else {
139
+
throw ConfigError.invalid(
140
+
"Config: \"bucket\" name \"\(bucket)\" is invalid. "
141
+
+ "Use lowercase letters, numbers, dots, and hyphens (3-63 chars)."
142
+
)
143
+
}
144
+
145
+
let pathStyle: Bool
146
+
if let ps = obj["pathStyle"] {
147
+
pathStyle = (ps as? Bool) ?? true
148
+
} else {
149
+
pathStyle = true
150
+
}
151
+
152
+
let keychainObj = obj["keychain"] as? [String: Any] ?? [:]
153
+
let accessKeyService = (keychainObj["accessKeyService"] as? String)
154
+
.flatMap { $0.isEmpty ? nil : $0 } ?? "attic-s3-access-key"
155
+
let secretKeyService = (keychainObj["secretKeyService"] as? String)
156
+
.flatMap { $0.isEmpty ? nil : $0 } ?? "attic-s3-secret-key"
157
+
158
+
return AtticConfig(
159
+
endpoint: endpoint,
160
+
region: region,
161
+
bucket: bucket,
162
+
pathStyle: pathStyle,
163
+
keychain: KeychainConfig(
164
+
accessKeyService: accessKeyService,
165
+
secretKeyService: secretKeyService
166
+
)
167
+
)
168
+
}
169
+
}
170
+
171
+
/// Configuration errors.
172
+
public enum ConfigError: Error, CustomStringConvertible {
173
+
case notFound(String)
174
+
case invalid(String)
175
+
176
+
public var description: String {
177
+
switch self {
178
+
case .notFound(let path):
179
+
"No config file found at \(path)\n"
180
+
+ "Run \"attic init\" to set up your S3 connection, or create the file manually."
181
+
case .invalid(let message):
182
+
message
183
+
}
184
+
}
185
+
}
+6
Sources/AtticCore/AtticCore.swift
+6
Sources/AtticCore/AtticCore.swift
+329
Sources/AtticCore/BackupPipeline.swift
+329
Sources/AtticCore/BackupPipeline.swift
···
1
+
import Foundation
2
+
import LadderKit
3
+
4
+
/// Shared ISO8601 formatter — reused across all pipeline operations.
5
+
nonisolated(unsafe) let isoFormatter = ISO8601DateFormatter()
6
+
7
+
/// Shared JSON encoder for metadata uploads.
8
+
let metadataEncoder: JSONEncoder = {
9
+
let encoder = JSONEncoder()
10
+
encoder.outputFormatting = [.prettyPrinted, .sortedKeys]
11
+
return encoder
12
+
}()
13
+
14
+
/// Maximum number of errors to keep in a report (prevents unbounded growth).
15
+
let maxReportErrors = 1000
16
+
17
+
/// Options controlling backup behavior.
18
+
public struct BackupOptions: Sendable {
19
+
public var batchSize: Int
20
+
public var limit: Int
21
+
public var type: AssetKind?
22
+
public var dryRun: Bool
23
+
public var saveInterval: Int
24
+
25
+
public init(
26
+
batchSize: Int = 50,
27
+
limit: Int = 0,
28
+
type: AssetKind? = nil,
29
+
dryRun: Bool = false,
30
+
saveInterval: Int = 50
31
+
) {
32
+
self.batchSize = batchSize
33
+
self.limit = limit
34
+
self.type = type
35
+
self.dryRun = dryRun
36
+
self.saveInterval = saveInterval
37
+
}
38
+
}
39
+
40
+
/// Result of a backup run.
41
+
public struct BackupReport: Sendable {
42
+
public var uploaded: Int = 0
43
+
public var failed: Int = 0
44
+
public var skipped: Int = 0
45
+
public var totalBytes: Int = 0
46
+
public var errors: [(uuid: String, message: String)] = []
47
+
48
+
mutating func appendError(uuid: String, message: String) {
49
+
if errors.count < maxReportErrors {
50
+
errors.append((uuid: uuid, message: message))
51
+
}
52
+
}
53
+
}
54
+
55
+
/// Run the backup pipeline: filter → batch → export → upload → manifest.
56
+
public func runBackup(
57
+
assets: [AssetInfo],
58
+
manifest: inout Manifest,
59
+
manifestStore: any ManifestStoring,
60
+
exporter: any ExportProviding,
61
+
s3: any S3Providing,
62
+
options: BackupOptions = BackupOptions(),
63
+
progress: any BackupProgressDelegate = NullProgressDelegate()
64
+
) async throws -> BackupReport {
65
+
// Filter to pending assets, optionally by type
66
+
var pending = assets.filter { asset in
67
+
if manifest.isBackedUp(asset.uuid) { return false }
68
+
if let type = options.type, asset.kind != type { return false }
69
+
return true
70
+
}
71
+
72
+
// Apply limit
73
+
if options.limit > 0 {
74
+
pending = Array(pending.prefix(options.limit))
75
+
}
76
+
77
+
if pending.isEmpty {
78
+
return BackupReport()
79
+
}
80
+
81
+
var photoCount = 0
82
+
var videoCount = 0
83
+
for asset in pending {
84
+
if asset.kind == .photo { photoCount += 1 }
85
+
else { videoCount += 1 }
86
+
}
87
+
88
+
progress.backupStarted(pending: pending.count, photos: photoCount, videos: videoCount)
89
+
90
+
if options.dryRun {
91
+
var report = BackupReport()
92
+
report.skipped = pending.count
93
+
return report
94
+
}
95
+
96
+
// Build UUID-to-asset lookup
97
+
let assetByUUID = Dictionary(uniqueKeysWithValues: pending.map { ($0.uuid, $0) })
98
+
99
+
var report = BackupReport()
100
+
var sinceLastSave = 0
101
+
var deferred: [String] = []
102
+
103
+
// Process in batches
104
+
let totalBatches = (pending.count + options.batchSize - 1) / options.batchSize
105
+
106
+
for batchIndex in 0..<totalBatches {
107
+
try Task.checkCancellation()
108
+
109
+
let start = batchIndex * options.batchSize
110
+
let end = min(start + options.batchSize, pending.count)
111
+
let batch = Array(pending[start..<end])
112
+
let batchUUIDs = batch.map(\.uuid)
113
+
114
+
progress.batchStarted(
115
+
batchNumber: batchIndex + 1,
116
+
totalBatches: totalBatches,
117
+
assetCount: batch.count
118
+
)
119
+
120
+
// 1. Export via LadderKit
121
+
let batchResult: ExportResponse
122
+
do {
123
+
batchResult = try await exporter.exportBatch(uuids: batchUUIDs)
124
+
} catch let error as ExportProviderError where error.isTimeout {
125
+
// Batch timeout: retry each asset individually
126
+
var combinedResults: [ExportResult] = []
127
+
var combinedErrors: [LadderKit.ExportError] = []
128
+
for uuid in batchUUIDs {
129
+
try Task.checkCancellation()
130
+
do {
131
+
let result = try await exporter.exportBatch(uuids: [uuid])
132
+
combinedResults.append(contentsOf: result.results)
133
+
combinedErrors.append(contentsOf: result.errors)
134
+
} catch let innerError as ExportProviderError where innerError.isTimeout {
135
+
deferred.append(uuid)
136
+
} catch {
137
+
let msg = String(describing: error)
138
+
report.appendError(uuid: uuid, message: msg)
139
+
report.failed += 1
140
+
let filename = assetByUUID[uuid]?.originalFilename ?? uuid
141
+
progress.assetFailed(uuid: uuid, filename: filename, message: msg)
142
+
}
143
+
}
144
+
let combined = ExportResponse(results: combinedResults, errors: combinedErrors)
145
+
try await uploadExported(
146
+
combined, assetByUUID: assetByUUID, s3: s3,
147
+
manifest: &manifest, manifestStore: manifestStore,
148
+
report: &report, sinceLastSave: &sinceLastSave,
149
+
saveInterval: options.saveInterval,
150
+
progress: progress
151
+
)
152
+
continue
153
+
} catch let error as ExportProviderError where error.isPermission {
154
+
// Permission error: abort all remaining batches
155
+
let msg = String(describing: error)
156
+
for uuid in batchUUIDs {
157
+
report.appendError(uuid: uuid, message: msg)
158
+
report.failed += 1
159
+
}
160
+
for asset in pending[end...] {
161
+
report.appendError(uuid: asset.uuid, message: msg)
162
+
report.failed += 1
163
+
}
164
+
break
165
+
} catch {
166
+
// Non-timeout error: fail the whole batch
167
+
let msg = String(describing: error)
168
+
for uuid in batchUUIDs {
169
+
report.appendError(uuid: uuid, message: msg)
170
+
report.failed += 1
171
+
let filename = assetByUUID[uuid]?.originalFilename ?? uuid
172
+
progress.assetFailed(uuid: uuid, filename: filename, message: msg)
173
+
}
174
+
continue
175
+
}
176
+
177
+
// 2. Upload exported assets
178
+
try await uploadExported(
179
+
batchResult, assetByUUID: assetByUUID, s3: s3,
180
+
manifest: &manifest, manifestStore: manifestStore,
181
+
report: &report, sinceLastSave: &sinceLastSave,
182
+
saveInterval: options.saveInterval,
183
+
progress: progress
184
+
)
185
+
}
186
+
187
+
// Retry deferred assets
188
+
if !deferred.isEmpty {
189
+
for uuid in deferred {
190
+
try Task.checkCancellation()
191
+
do {
192
+
let result = try await exporter.exportBatch(uuids: [uuid])
193
+
try await uploadExported(
194
+
result, assetByUUID: assetByUUID, s3: s3,
195
+
manifest: &manifest, manifestStore: manifestStore,
196
+
report: &report, sinceLastSave: &sinceLastSave,
197
+
saveInterval: options.saveInterval,
198
+
progress: progress
199
+
)
200
+
} catch {
201
+
let msg = String(describing: error)
202
+
report.appendError(uuid: uuid, message: msg)
203
+
report.failed += 1
204
+
let filename = assetByUUID[uuid]?.originalFilename ?? uuid
205
+
progress.assetFailed(uuid: uuid, filename: filename, message: msg)
206
+
}
207
+
}
208
+
}
209
+
210
+
// Final save
211
+
if sinceLastSave > 0 {
212
+
try await manifestStore.save(manifest)
213
+
progress.manifestSaved(entriesCount: manifest.entries.count)
214
+
}
215
+
216
+
progress.backupCompleted(
217
+
uploaded: report.uploaded,
218
+
failed: report.failed,
219
+
totalBytes: report.totalBytes
220
+
)
221
+
222
+
return report
223
+
}
224
+
225
+
// MARK: - Upload helper
226
+
227
+
private func uploadExported(
228
+
_ batchResult: ExportResponse,
229
+
assetByUUID: [String: AssetInfo],
230
+
s3: any S3Providing,
231
+
manifest: inout Manifest,
232
+
manifestStore: any ManifestStoring,
233
+
report: inout BackupReport,
234
+
sinceLastSave: inout Int,
235
+
saveInterval: Int,
236
+
progress: any BackupProgressDelegate
237
+
) async throws {
238
+
// Record export errors
239
+
for err in batchResult.errors {
240
+
let filename = assetByUUID[err.uuid]?.originalFilename ?? err.uuid
241
+
progress.assetFailed(uuid: err.uuid, filename: filename, message: err.message)
242
+
report.appendError(uuid: err.uuid, message: err.message)
243
+
report.failed += 1
244
+
}
245
+
246
+
// Upload successful exports
247
+
for exported in batchResult.results {
248
+
try Task.checkCancellation()
249
+
250
+
guard let asset = assetByUUID[exported.uuid] else { continue }
251
+
252
+
let ext = S3Paths.extensionFromUTIOrFilename(
253
+
uti: asset.uniformTypeIdentifier,
254
+
filename: asset.originalFilename ?? "unknown"
255
+
)
256
+
let s3Key = try S3Paths.originalKey(
257
+
uuid: asset.uuid,
258
+
dateCreated: asset.creationDate,
259
+
extension: ext
260
+
)
261
+
262
+
do {
263
+
// Upload original via file URL (avoids loading into memory)
264
+
let fileURL = URL(fileURLWithPath: exported.path)
265
+
try await withRetry {
266
+
try await s3.putObject(
267
+
key: s3Key,
268
+
fileURL: fileURL,
269
+
contentType: contentTypeForExtension(ext)
270
+
)
271
+
}
272
+
273
+
// Build and upload metadata
274
+
let isoNow = isoFormatter.string(from: Date())
275
+
let meta = buildMetadataJSON(
276
+
asset: asset,
277
+
s3Key: s3Key,
278
+
checksum: "sha256:\(exported.sha256)",
279
+
backedUpAt: isoNow
280
+
)
281
+
let metaData = try metadataEncoder.encode(meta)
282
+
let metaKey = try S3Paths.metadataKey(uuid: asset.uuid)
283
+
try await withRetry {
284
+
try await s3.putObject(
285
+
key: metaKey,
286
+
body: metaData,
287
+
contentType: "application/json"
288
+
)
289
+
}
290
+
291
+
// Update manifest
292
+
manifest.markBackedUp(
293
+
uuid: asset.uuid,
294
+
s3Key: s3Key,
295
+
checksum: "sha256:\(exported.sha256)",
296
+
size: Int(exported.size)
297
+
)
298
+
sinceLastSave += 1
299
+
report.uploaded += 1
300
+
report.totalBytes += Int(exported.size)
301
+
302
+
let filename = asset.originalFilename ?? "unknown"
303
+
progress.assetUploaded(
304
+
uuid: asset.uuid,
305
+
filename: filename,
306
+
type: asset.kind,
307
+
size: Int(exported.size)
308
+
)
309
+
310
+
// Periodic manifest save (skip sortedKeys for speed)
311
+
if sinceLastSave >= saveInterval {
312
+
try await manifestStore.save(manifest)
313
+
progress.manifestSaved(entriesCount: manifest.entries.count)
314
+
sinceLastSave = 0
315
+
}
316
+
} catch is CancellationError {
317
+
throw CancellationError()
318
+
} catch {
319
+
let msg = String(describing: error)
320
+
let filename = asset.originalFilename ?? exported.uuid
321
+
progress.assetFailed(uuid: exported.uuid, filename: filename, message: msg)
322
+
report.appendError(uuid: exported.uuid, message: msg)
323
+
report.failed += 1
324
+
}
325
+
326
+
// Clean up staged file
327
+
try? FileManager.default.removeItem(atPath: exported.path)
328
+
}
329
+
}
+36
Sources/AtticCore/BackupProgress.swift
+36
Sources/AtticCore/BackupProgress.swift
···
1
+
import Foundation
2
+
import LadderKit
3
+
4
+
/// Progress events emitted during backup.
5
+
///
6
+
/// Implemented by the CLI (terminal renderer) and the menu bar app (AppState).
7
+
public protocol BackupProgressDelegate: Sendable {
8
+
/// Backup is starting with this many pending assets.
9
+
func backupStarted(pending: Int, photos: Int, videos: Int)
10
+
11
+
/// A batch is starting.
12
+
func batchStarted(batchNumber: Int, totalBatches: Int, assetCount: Int)
13
+
14
+
/// A single asset was uploaded successfully.
15
+
func assetUploaded(uuid: String, filename: String, type: AssetKind, size: Int)
16
+
17
+
/// A single asset failed.
18
+
func assetFailed(uuid: String, filename: String, message: String)
19
+
20
+
/// Manifest was saved to S3.
21
+
func manifestSaved(entriesCount: Int)
22
+
23
+
/// Backup completed.
24
+
func backupCompleted(uploaded: Int, failed: Int, totalBytes: Int)
25
+
}
26
+
27
+
/// No-op delegate for quiet/test runs.
28
+
public struct NullProgressDelegate: BackupProgressDelegate {
29
+
public init() {}
30
+
public func backupStarted(pending: Int, photos: Int, videos: Int) {}
31
+
public func batchStarted(batchNumber: Int, totalBatches: Int, assetCount: Int) {}
32
+
public func assetUploaded(uuid: String, filename: String, type: AssetKind, size: Int) {}
33
+
public func assetFailed(uuid: String, filename: String, message: String) {}
34
+
public func manifestSaved(entriesCount: Int) {}
35
+
public func backupCompleted(uploaded: Int, failed: Int, totalBytes: Int) {}
36
+
}
+60
Sources/AtticCore/ExportProviding.swift
+60
Sources/AtticCore/ExportProviding.swift
···
1
+
import Foundation
2
+
import LadderKit
3
+
4
+
/// Abstraction over the photo export mechanism for testability.
5
+
///
6
+
/// The real implementation calls LadderKit's PhotoExporter directly.
7
+
/// Tests use MockExportProvider with pre-configured results.
8
+
public protocol ExportProviding: Sendable {
9
+
/// Export a batch of assets by UUID. Returns results and errors.
10
+
func exportBatch(uuids: [String]) async throws -> ExportResponse
11
+
12
+
/// Pre-flight check: verify required permissions (Photos, Automation).
13
+
/// Throws if permissions are missing.
14
+
func checkPermissions() async throws
15
+
}
16
+
17
+
/// Errors from the export layer.
18
+
public enum ExportProviderError: Error, CustomStringConvertible {
19
+
case timeout(seconds: Int)
20
+
case permissionDenied(String)
21
+
22
+
public var description: String {
23
+
switch self {
24
+
case .timeout(let seconds):
25
+
"Export timed out after \(seconds)s"
26
+
case .permissionDenied(let message):
27
+
message
28
+
}
29
+
}
30
+
31
+
public var isTimeout: Bool {
32
+
if case .timeout = self { return true }
33
+
return false
34
+
}
35
+
36
+
public var isPermission: Bool {
37
+
if case .permissionDenied = self { return true }
38
+
return false
39
+
}
40
+
}
41
+
42
+
/// Extension-to-content-type lookup table.
43
+
private let contentTypeMap: [String: String] = [
44
+
"jpg": "image/jpeg",
45
+
"jpeg": "image/jpeg",
46
+
"heic": "image/heic",
47
+
"png": "image/png",
48
+
"tiff": "image/tiff",
49
+
"gif": "image/gif",
50
+
"mp4": "video/mp4",
51
+
"mov": "video/quicktime",
52
+
"m4v": "video/x-m4v",
53
+
"avi": "video/x-msvideo",
54
+
"orf": "image/x-olympus-orf",
55
+
]
56
+
57
+
/// Map a file extension to its MIME content type.
58
+
public func contentTypeForExtension(_ ext: String) -> String {
59
+
contentTypeMap[ext] ?? "application/octet-stream"
60
+
}
+10
Sources/AtticCore/FormatBytes.swift
+10
Sources/AtticCore/FormatBytes.swift
···
1
+
import Foundation
2
+
3
+
/// Format a byte count as a human-readable string.
4
+
public func formatBytes(_ bytes: Int) -> String {
5
+
if bytes == 0 { return "0 B" }
6
+
let units = ["B", "KB", "MB", "GB", "TB"]
7
+
let index = min(Int(log(Double(bytes)) / log(1024)), units.count - 1)
8
+
let value = Double(bytes) / pow(1024, Double(index))
9
+
return String(format: "%.1f %@", value, units[index])
10
+
}
+104
Sources/AtticCore/KeychainCredentials.swift
+104
Sources/AtticCore/KeychainCredentials.swift
···
1
+
import Foundation
2
+
import Security
3
+
4
+
/// S3 credentials read from macOS Keychain.
5
+
public struct S3Credentials: Sendable {
6
+
public let accessKeyId: String
7
+
public let secretAccessKey: String
8
+
9
+
public init(accessKeyId: String, secretAccessKey: String) {
10
+
self.accessKeyId = accessKeyId
11
+
self.secretAccessKey = secretAccessKey
12
+
}
13
+
}
14
+
15
+
/// Protocol for reading and writing Keychain credentials.
16
+
public protocol KeychainProviding: Sendable {
17
+
func loadCredentials(
18
+
accessKeyService: String,
19
+
secretKeyService: String
20
+
) throws -> S3Credentials
21
+
22
+
func store(service: String, value: String) throws
23
+
}
24
+
25
+
/// Keychain provider using the macOS Security framework directly.
26
+
public struct SecurityKeychain: KeychainProviding {
27
+
private static let account = "attic"
28
+
29
+
public init() {}
30
+
31
+
public func loadCredentials(
32
+
accessKeyService: String = "attic-s3-access-key",
33
+
secretKeyService: String = "attic-s3-secret-key"
34
+
) throws -> S3Credentials {
35
+
let accessKeyId = try get(service: accessKeyService)
36
+
let secretAccessKey = try get(service: secretKeyService)
37
+
return S3Credentials(
38
+
accessKeyId: accessKeyId,
39
+
secretAccessKey: secretAccessKey
40
+
)
41
+
}
42
+
43
+
public func store(service: String, value: String) throws {
44
+
let query: [String: Any] = [
45
+
kSecClass as String: kSecClassGenericPassword,
46
+
kSecAttrService as String: service,
47
+
kSecAttrAccount as String: Self.account,
48
+
]
49
+
50
+
// Delete existing item first (SecItemUpdate doesn't create)
51
+
SecItemDelete(query as CFDictionary)
52
+
53
+
var addQuery = query
54
+
addQuery[kSecValueData as String] = value.data(using: .utf8)!
55
+
addQuery[kSecAttrAccessible as String] = kSecAttrAccessibleWhenUnlocked
56
+
57
+
let status = SecItemAdd(addQuery as CFDictionary, nil)
58
+
guard status == errSecSuccess else {
59
+
throw KeychainError.storeFailed(
60
+
service: service,
61
+
status: status
62
+
)
63
+
}
64
+
}
65
+
66
+
private func get(service: String) throws -> String {
67
+
let query: [String: Any] = [
68
+
kSecClass as String: kSecClassGenericPassword,
69
+
kSecAttrService as String: service,
70
+
kSecReturnData as String: true,
71
+
kSecMatchLimit as String: kSecMatchLimitOne,
72
+
]
73
+
74
+
var result: AnyObject?
75
+
let status = SecItemCopyMatching(query as CFDictionary, &result)
76
+
77
+
guard status == errSecSuccess, let data = result as? Data,
78
+
let value = String(data: data, encoding: .utf8)
79
+
else {
80
+
throw KeychainError.readFailed(
81
+
service: service,
82
+
status: status
83
+
)
84
+
}
85
+
86
+
return value
87
+
}
88
+
}
89
+
90
+
/// Keychain errors.
91
+
public enum KeychainError: Error, CustomStringConvertible {
92
+
case readFailed(service: String, status: OSStatus)
93
+
case storeFailed(service: String, status: OSStatus)
94
+
95
+
public var description: String {
96
+
switch self {
97
+
case .readFailed(let service, let status):
98
+
"Failed to read keychain item \"\(service)\" (status: \(status)). "
99
+
+ "Store it with: security add-generic-password -s \(service) -a attic -w \"<value>\""
100
+
case .storeFailed(let service, let status):
101
+
"Failed to store credential in Keychain for service \"\(service)\" (status: \(status))"
102
+
}
103
+
}
104
+
}
+86
Sources/AtticCore/Manifest.swift
+86
Sources/AtticCore/Manifest.swift
···
1
+
import Foundation
2
+
3
+
/// A single backed-up asset's record in the manifest.
4
+
public struct ManifestEntry: Codable, Sendable, Equatable {
5
+
public var uuid: String
6
+
public var s3Key: String
7
+
public var checksum: String
8
+
public var backedUpAt: String
9
+
public var size: Int?
10
+
11
+
public init(
12
+
uuid: String,
13
+
s3Key: String,
14
+
checksum: String,
15
+
backedUpAt: String,
16
+
size: Int? = nil
17
+
) {
18
+
self.uuid = uuid
19
+
self.s3Key = s3Key
20
+
self.checksum = checksum
21
+
self.backedUpAt = backedUpAt
22
+
self.size = size
23
+
}
24
+
}
25
+
26
+
/// The backup manifest — maps UUID to ManifestEntry.
27
+
public struct Manifest: Codable, Sendable {
28
+
public var entries: [String: ManifestEntry]
29
+
30
+
public init(entries: [String: ManifestEntry] = [:]) {
31
+
self.entries = entries
32
+
}
33
+
34
+
/// Check whether an asset has been backed up.
35
+
public func isBackedUp(_ uuid: String) -> Bool {
36
+
entries[uuid] != nil
37
+
}
38
+
39
+
/// Mark an asset as backed up (mutates in place).
40
+
public mutating func markBackedUp(
41
+
uuid: String,
42
+
s3Key: String,
43
+
checksum: String,
44
+
size: Int? = nil,
45
+
backedUpAt: String? = nil
46
+
) {
47
+
entries[uuid] = ManifestEntry(
48
+
uuid: uuid,
49
+
s3Key: s3Key,
50
+
checksum: checksum,
51
+
backedUpAt: backedUpAt ?? isoFormatter.string(from: Date()),
52
+
size: size
53
+
)
54
+
}
55
+
}
56
+
57
+
/// S3 key where the shared manifest is stored.
58
+
public let manifestS3Key = "manifest.json"
59
+
60
+
/// Protocol for loading and saving the manifest.
61
+
public protocol ManifestStoring: Sendable {
62
+
func load() async throws -> Manifest
63
+
func save(_ manifest: Manifest) async throws
64
+
}
65
+
66
+
// MARK: - Validation
67
+
68
+
extension Manifest {
69
+
/// Parse and validate manifest data.
70
+
public static func parse(from data: Data) throws -> Manifest {
71
+
let decoded = try JSONDecoder().decode(Manifest.self, from: data)
72
+
return decoded
73
+
}
74
+
75
+
/// Encode manifest to JSON data.
76
+
///
77
+
/// - Parameter sortedKeys: Use sorted keys for human readability (slower for large manifests).
78
+
/// Defaults to false for periodic saves; callers should pass true for final saves.
79
+
public func encoded(sortedKeys: Bool = false) throws -> Data {
80
+
let encoder = JSONEncoder()
81
+
encoder.outputFormatting = sortedKeys ? [.prettyPrinted, .sortedKeys] : []
82
+
var data = try encoder.encode(self)
83
+
data.append(contentsOf: "\n".utf8)
84
+
return data
85
+
}
86
+
}
+81
Sources/AtticCore/MetadataBuilder.swift
+81
Sources/AtticCore/MetadataBuilder.swift
···
1
+
import Foundation
2
+
import LadderKit
3
+
4
+
/// Per-asset metadata JSON uploaded to S3 at metadata/assets/{uuid}.json.
5
+
public struct AssetMetadata: Codable, Sendable {
6
+
public var uuid: String
7
+
public var originalFilename: String
8
+
public var dateCreated: String?
9
+
public var width: Int
10
+
public var height: Int
11
+
public var latitude: Double?
12
+
public var longitude: Double?
13
+
public var fileSize: Int?
14
+
public var type: String?
15
+
public var favorite: Bool
16
+
public var title: String?
17
+
public var description: String?
18
+
public var albums: [AlbumRef]
19
+
public var keywords: [String]
20
+
public var people: [PersonRef]
21
+
public var hasEdit: Bool
22
+
public var editedAt: String?
23
+
public var editor: String?
24
+
public var s3Key: String
25
+
public var checksum: String
26
+
public var backedUpAt: String
27
+
}
28
+
29
+
/// Album reference for metadata JSON (matches Deno schema).
30
+
public struct AlbumRef: Codable, Sendable, Equatable {
31
+
public var uuid: String
32
+
public var title: String
33
+
34
+
public init(uuid: String, title: String) {
35
+
self.uuid = uuid
36
+
self.title = title
37
+
}
38
+
}
39
+
40
+
/// Person reference for metadata JSON (matches Deno schema).
41
+
public struct PersonRef: Codable, Sendable, Equatable {
42
+
public var uuid: String
43
+
public var displayName: String
44
+
45
+
public init(uuid: String, displayName: String) {
46
+
self.uuid = uuid
47
+
self.displayName = displayName
48
+
}
49
+
}
50
+
51
+
/// Build a metadata JSON object for upload to S3.
52
+
public func buildMetadataJSON(
53
+
asset: AssetInfo,
54
+
s3Key: String,
55
+
checksum: String,
56
+
backedUpAt: String
57
+
) -> AssetMetadata {
58
+
AssetMetadata(
59
+
uuid: asset.uuid,
60
+
originalFilename: asset.originalFilename ?? "unknown",
61
+
dateCreated: asset.creationDate.map { isoFormatter.string(from: $0) },
62
+
width: asset.pixelWidth,
63
+
height: asset.pixelHeight,
64
+
latitude: asset.latitude,
65
+
longitude: asset.longitude,
66
+
fileSize: nil, // TODO: add when LadderKit has originalFileSize
67
+
type: asset.uniformTypeIdentifier,
68
+
favorite: asset.isFavorite,
69
+
title: nil, // LadderKit AssetInfo doesn't have title
70
+
description: asset.assetDescription,
71
+
albums: asset.albums.map { AlbumRef(uuid: $0.identifier, title: $0.title) },
72
+
keywords: asset.keywords,
73
+
people: asset.people.map { PersonRef(uuid: $0.uuid, displayName: $0.displayName) },
74
+
hasEdit: asset.hasEdit,
75
+
editedAt: asset.editedAt.map { isoFormatter.string(from: $0) },
76
+
editor: asset.editor,
77
+
s3Key: s3Key,
78
+
checksum: checksum,
79
+
backedUpAt: backedUpAt
80
+
)
81
+
}
+68
Sources/AtticCore/MockS3Provider.swift
+68
Sources/AtticCore/MockS3Provider.swift
···
1
+
import Foundation
2
+
3
+
/// In-memory S3 mock for tests. Stores objects as [String: StoredObject].
4
+
public actor MockS3Provider: S3Providing {
5
+
public struct StoredObject: Sendable {
6
+
public var body: Data
7
+
public var contentType: String?
8
+
9
+
public init(body: Data, contentType: String? = nil) {
10
+
self.body = body
11
+
self.contentType = contentType
12
+
}
13
+
}
14
+
15
+
public private(set) var objects: [String: StoredObject] = [:]
16
+
public private(set) var putCount = 0
17
+
public private(set) var getCount = 0
18
+
19
+
public init() {}
20
+
21
+
/// Pre-populate with test data.
22
+
public init(objects: [String: Data]) {
23
+
self.objects = objects.mapValues { StoredObject(body: $0) }
24
+
}
25
+
26
+
public func putObject(key: String, body: Data, contentType: String?) async throws {
27
+
objects[key] = StoredObject(body: body, contentType: contentType)
28
+
putCount += 1
29
+
}
30
+
31
+
public func putObject(key: String, fileURL: URL, contentType: String?) async throws {
32
+
let data = try Data(contentsOf: fileURL)
33
+
objects[key] = StoredObject(body: data, contentType: contentType)
34
+
putCount += 1
35
+
}
36
+
37
+
public func getObject(key: String) async throws -> Data {
38
+
getCount += 1
39
+
guard let obj = objects[key] else {
40
+
throw MockS3Error.notFound(key)
41
+
}
42
+
return obj.body
43
+
}
44
+
45
+
public func headObject(key: String) async throws -> S3ObjectMeta? {
46
+
guard let obj = objects[key] else { return nil }
47
+
return S3ObjectMeta(
48
+
contentLength: obj.body.count,
49
+
contentType: obj.contentType
50
+
)
51
+
}
52
+
53
+
public func listObjects(prefix: String) async throws -> [S3ListObject] {
54
+
objects.keys
55
+
.filter { $0.hasPrefix(prefix) }
56
+
.sorted()
57
+
.map { key in
58
+
S3ListObject(
59
+
key: key,
60
+
size: objects[key]?.body.count ?? 0
61
+
)
62
+
}
63
+
}
64
+
}
65
+
66
+
enum MockS3Error: Error {
67
+
case notFound(String)
68
+
}
+70
Sources/AtticCore/RebuildManifest.swift
+70
Sources/AtticCore/RebuildManifest.swift
···
1
+
import Foundation
2
+
3
+
/// Result of a rebuild-manifest run.
4
+
public struct RebuildManifestReport: Sendable {
5
+
public var recovered: Int = 0
6
+
public var skipped: Int = 0
7
+
public var errors: [(key: String, message: String)] = []
8
+
}
9
+
10
+
/// Rebuild manifest from S3 metadata JSON files.
11
+
///
12
+
/// Scans `metadata/assets/` in S3, parses each JSON file, and reconstructs
13
+
/// manifest entries. Used as a disaster recovery mechanism when the manifest
14
+
/// is lost or corrupted.
15
+
public func runRebuildManifest(
16
+
s3: any S3Providing,
17
+
manifestStore: any ManifestStoring
18
+
) async throws -> (Manifest, RebuildManifestReport) {
19
+
let objects = try await s3.listObjects(prefix: "metadata/assets/")
20
+
var manifest = Manifest()
21
+
var report = RebuildManifestReport()
22
+
23
+
for obj in objects {
24
+
guard obj.key.hasSuffix(".json") else {
25
+
report.skipped += 1
26
+
continue
27
+
}
28
+
29
+
do {
30
+
let data = try await s3.getObject(key: obj.key)
31
+
let parsed = try JSONDecoder().decode(MetadataForRebuild.self, from: data)
32
+
33
+
guard S3Paths.isValidUUID(parsed.uuid),
34
+
S3Paths.isValidS3Key(parsed.s3Key),
35
+
isValidChecksum(parsed.checksum) else {
36
+
report.errors.append((key: obj.key, message: "Validation failed"))
37
+
continue
38
+
}
39
+
40
+
manifest.markBackedUp(
41
+
uuid: parsed.uuid,
42
+
s3Key: parsed.s3Key,
43
+
checksum: parsed.checksum,
44
+
backedUpAt: parsed.backedUpAt ?? isoFormatter.string(from: Date())
45
+
)
46
+
report.recovered += 1
47
+
} catch {
48
+
report.errors.append((key: obj.key, message: String(describing: error)))
49
+
}
50
+
}
51
+
52
+
try await manifestStore.save(manifest)
53
+
return (manifest, report)
54
+
}
55
+
56
+
// MARK: - Internals
57
+
58
+
/// Minimal struct for parsing metadata JSON during rebuild.
59
+
private struct MetadataForRebuild: Decodable {
60
+
let uuid: String
61
+
let s3Key: String
62
+
let checksum: String
63
+
let backedUpAt: String?
64
+
}
65
+
66
+
nonisolated(unsafe) private let checksumValidation = /^sha256:[a-f0-9]+$/
67
+
68
+
private func isValidChecksum(_ value: String) -> Bool {
69
+
value.wholeMatch(of: checksumValidation) != nil
70
+
}
+156
Sources/AtticCore/RefreshMetadata.swift
+156
Sources/AtticCore/RefreshMetadata.swift
···
1
+
import Foundation
2
+
import LadderKit
3
+
4
+
/// Options controlling refresh-metadata behavior.
5
+
public struct RefreshMetadataOptions: Sendable {
6
+
public var concurrency: Int
7
+
public var dryRun: Bool
8
+
9
+
public init(concurrency: Int = 20, dryRun: Bool = false) {
10
+
self.concurrency = concurrency
11
+
self.dryRun = dryRun
12
+
}
13
+
}
14
+
15
+
/// Result of a refresh-metadata run.
16
+
public struct RefreshMetadataReport: Sendable {
17
+
public var updated: Int = 0
18
+
public var skipped: Int = 0
19
+
public var failed: Int = 0
20
+
public var totalBytes: Int = 0
21
+
public var errors: [(uuid: String, message: String)] = []
22
+
}
23
+
24
+
/// Progress events emitted during metadata refresh.
25
+
public protocol RefreshMetadataProgressDelegate: Sendable {
26
+
func refreshStarted(total: Int)
27
+
func assetRefreshed(uuid: String, filename: String)
28
+
func assetFailed(uuid: String, message: String)
29
+
func refreshCompleted(report: RefreshMetadataReport)
30
+
}
31
+
32
+
/// No-op refresh progress delegate.
33
+
public struct NullRefreshMetadataProgressDelegate: RefreshMetadataProgressDelegate {
34
+
public init() {}
35
+
public func refreshStarted(total: Int) {}
36
+
public func assetRefreshed(uuid: String, filename: String) {}
37
+
public func assetFailed(uuid: String, message: String) {}
38
+
public func refreshCompleted(report: RefreshMetadataReport) {}
39
+
}
40
+
41
+
/// Re-generate and upload metadata JSON for all backed-up assets.
42
+
///
43
+
/// This is useful when the metadata schema changes or new fields are added.
44
+
/// Only processes assets that exist in both the manifest and the asset list.
45
+
public func runRefreshMetadata(
46
+
assets: [AssetInfo],
47
+
manifest: Manifest,
48
+
s3: any S3Providing,
49
+
options: RefreshMetadataOptions = RefreshMetadataOptions(),
50
+
progress: any RefreshMetadataProgressDelegate = NullRefreshMetadataProgressDelegate()
51
+
) async throws -> RefreshMetadataReport {
52
+
// Only refresh assets that are in the manifest
53
+
let backedUp = assets.filter { manifest.isBackedUp($0.uuid) }
54
+
55
+
guard !backedUp.isEmpty else {
56
+
let report = RefreshMetadataReport()
57
+
progress.refreshCompleted(report: report)
58
+
return report
59
+
}
60
+
61
+
progress.refreshStarted(total: backedUp.count)
62
+
63
+
if options.dryRun {
64
+
var report = RefreshMetadataReport()
65
+
report.skipped = backedUp.count
66
+
progress.refreshCompleted(report: report)
67
+
return report
68
+
}
69
+
70
+
let report = RefreshReport()
71
+
72
+
await withTaskGroup(of: Void.self) { group in
73
+
var cursor = 0
74
+
75
+
for _ in 0..<min(options.concurrency, backedUp.count) {
76
+
let asset = backedUp[cursor]
77
+
cursor += 1
78
+
group.addTask {
79
+
await refreshSingle(asset: asset, manifest: manifest, s3: s3,
80
+
report: report, progress: progress)
81
+
}
82
+
}
83
+
84
+
for await _ in group {
85
+
if cursor < backedUp.count {
86
+
let asset = backedUp[cursor]
87
+
cursor += 1
88
+
group.addTask {
89
+
await refreshSingle(asset: asset, manifest: manifest, s3: s3,
90
+
report: report, progress: progress)
91
+
}
92
+
}
93
+
}
94
+
}
95
+
96
+
let finalReport = await report.snapshot()
97
+
progress.refreshCompleted(report: finalReport)
98
+
return finalReport
99
+
}
100
+
101
+
// MARK: - Internals
102
+
103
+
private actor RefreshReport {
104
+
var updated = 0
105
+
var failed = 0
106
+
var totalBytes = 0
107
+
var errors: [(uuid: String, message: String)] = []
108
+
109
+
func markUpdated(bytes: Int) { updated += 1; totalBytes += bytes }
110
+
func markFailed(_ uuid: String, _ message: String) {
111
+
failed += 1
112
+
if errors.count < maxReportErrors {
113
+
errors.append((uuid: uuid, message: message))
114
+
}
115
+
}
116
+
117
+
func snapshot() -> RefreshMetadataReport {
118
+
RefreshMetadataReport(
119
+
updated: updated, skipped: 0, failed: failed,
120
+
totalBytes: totalBytes, errors: errors
121
+
)
122
+
}
123
+
}
124
+
125
+
private func refreshSingle(
126
+
asset: AssetInfo,
127
+
manifest: Manifest,
128
+
s3: any S3Providing,
129
+
report: RefreshReport,
130
+
progress: any RefreshMetadataProgressDelegate
131
+
) async {
132
+
guard let entry = manifest.entries[asset.uuid] else { return }
133
+
134
+
do {
135
+
let meta = buildMetadataJSON(
136
+
asset: asset,
137
+
s3Key: entry.s3Key,
138
+
checksum: entry.checksum,
139
+
backedUpAt: entry.backedUpAt
140
+
)
141
+
let data = try metadataEncoder.encode(meta)
142
+
let metaKey = try S3Paths.metadataKey(uuid: asset.uuid)
143
+
144
+
try await withRetry {
145
+
try await s3.putObject(key: metaKey, body: data, contentType: "application/json")
146
+
}
147
+
148
+
await report.markUpdated(bytes: data.count)
149
+
let filename = asset.originalFilename ?? asset.uuid
150
+
progress.assetRefreshed(uuid: asset.uuid, filename: filename)
151
+
} catch {
152
+
let msg = String(describing: error)
153
+
await report.markFailed(asset.uuid, msg)
154
+
progress.assetFailed(uuid: asset.uuid, message: msg)
155
+
}
156
+
}
+39
Sources/AtticCore/RetryPolicy.swift
+39
Sources/AtticCore/RetryPolicy.swift
···
1
+
import Foundation
2
+
3
+
/// Retry an async operation with exponential backoff.
4
+
///
5
+
/// Handles transient network failures (e.g. after sleep/wake).
6
+
/// Respects Task cancellation to bail out immediately.
7
+
public func withRetry<T: Sendable>(
8
+
maxAttempts: Int = 3,
9
+
baseDelay: Duration = .seconds(1),
10
+
operation: @Sendable () async throws -> T
11
+
) async throws -> T {
12
+
for attempt in 1...maxAttempts {
13
+
do {
14
+
return try await operation()
15
+
} catch {
16
+
if attempt == maxAttempts { throw error }
17
+
18
+
// Don't retry if cancelled
19
+
try Task.checkCancellation()
20
+
21
+
// Only retry on transient/network errors
22
+
guard isTransient(error) else { throw error }
23
+
24
+
let delay = baseDelay * Int(pow(2.0, Double(attempt - 1)))
25
+
try await Task.sleep(for: delay)
26
+
}
27
+
}
28
+
fatalError("unreachable")
29
+
}
30
+
31
+
/// Determine whether an error is transient and worth retrying.
32
+
private func isTransient(_ error: Error) -> Bool {
33
+
let message = String(describing: error).lowercased()
34
+
let transientPatterns = [
35
+
"timeout", "econnreset", "econnrefused", "epipe",
36
+
"socket", "network", "fetch failed",
37
+
]
38
+
return transientPatterns.contains { message.contains($0) }
39
+
}
+82
Sources/AtticCore/S3ManifestStore.swift
+82
Sources/AtticCore/S3ManifestStore.swift
···
1
+
import Foundation
2
+
3
+
/// Manifest store backed by S3. This is the primary store.
4
+
public struct S3ManifestStore: ManifestStoring {
5
+
private let s3: any S3Providing
6
+
private let key: String
7
+
8
+
public init(s3: any S3Providing, key: String = manifestS3Key) {
9
+
self.s3 = s3
10
+
self.key = key
11
+
}
12
+
13
+
public func load() async throws -> Manifest {
14
+
do {
15
+
let data = try await s3.getObject(key: key)
16
+
return try Manifest.parse(from: data)
17
+
} catch {
18
+
if isNotFoundError(error) {
19
+
return Manifest()
20
+
}
21
+
throw error
22
+
}
23
+
}
24
+
25
+
public func save(_ manifest: Manifest) async throws {
26
+
let data = try manifest.encoded()
27
+
try await s3.putObject(key: key, body: data, contentType: "application/json")
28
+
}
29
+
}
30
+
31
+
/// Load manifest from S3, migrating from local file if needed.
32
+
///
33
+
/// TODO: Remove this migration path once all users have upgraded from the
34
+
/// Deno CLI (v0.2.x) to the Swift CLI. At that point everyone's manifest
35
+
/// will already be on S3 and the local-file fallback is dead code.
36
+
///
37
+
/// Migration flow (one-time):
38
+
/// 1. If S3 has a manifest with entries, use it.
39
+
/// 2. If S3 is empty, check for a local manifest at ~/.attic/manifest.json.
40
+
/// 3. If local exists, upload it to S3 and return it.
41
+
/// 4. If neither exists, return empty manifest.
42
+
public func loadManifestWithMigration(
43
+
s3Store: ManifestStoring,
44
+
localDirectory: URL? = nil
45
+
) async throws -> Manifest {
46
+
let s3Manifest = try await s3Store.load()
47
+
48
+
if !s3Manifest.entries.isEmpty {
49
+
return s3Manifest
50
+
}
51
+
52
+
// Check for local manifest to migrate
53
+
let dir = localDirectory
54
+
?? FileManager.default.homeDirectoryForCurrentUser.appendingPathComponent(".attic")
55
+
let localPath = dir.appendingPathComponent("manifest.json")
56
+
57
+
guard FileManager.default.fileExists(atPath: localPath.path) else {
58
+
return s3Manifest
59
+
}
60
+
61
+
do {
62
+
let data = try Data(contentsOf: localPath)
63
+
let localManifest = try Manifest.parse(from: data)
64
+
65
+
if !localManifest.entries.isEmpty {
66
+
debugPrint(" Migrating local manifest (\(localManifest.entries.count) entries) to S3...")
67
+
try await s3Store.save(localManifest)
68
+
debugPrint(" Migration complete.\n")
69
+
return localManifest
70
+
}
71
+
} catch {
72
+
// No local manifest or unreadable — that's fine
73
+
}
74
+
75
+
return s3Manifest
76
+
}
77
+
78
+
private func isNotFoundError(_ error: Error) -> Bool {
79
+
let description = String(describing: error)
80
+
return description.contains("NotFound") || description.contains("NoSuchKey")
81
+
|| description.contains("notFound")
82
+
}
+122
Sources/AtticCore/S3Paths.swift
+122
Sources/AtticCore/S3Paths.swift
···
1
+
import Foundation
2
+
3
+
/// S3 key generation and path safety for photo backup storage.
4
+
public enum S3Paths {
5
+
// MARK: - Validation patterns
6
+
7
+
private static nonisolated(unsafe) let uuidPattern = /^[A-Za-z0-9._\-]+$/
8
+
private static nonisolated(unsafe) let s3KeyPattern = /^[A-Za-z0-9\/._\-]+$/
9
+
private static nonisolated(unsafe) let extPattern = /^[a-z0-9]+$/
10
+
11
+
/// UTI-to-extension lookup table.
12
+
private static let utiMap: [String: String] = [
13
+
"public.jpeg": "jpg",
14
+
"public.heic": "heic",
15
+
"public.png": "png",
16
+
"public.tiff": "tiff",
17
+
"com.compuserve.gif": "gif",
18
+
"public.mpeg-4": "mp4",
19
+
"com.apple.quicktime-movie": "mov",
20
+
"com.apple.m4v-video": "m4v",
21
+
"public.avi": "avi",
22
+
"com.olympus.raw-image": "orf",
23
+
]
24
+
25
+
// MARK: - Key generation
26
+
27
+
/// UTC calendar for date component extraction — hoisted to avoid per-call allocation.
28
+
private static let utcCalendar: Calendar = {
29
+
var cal = Calendar(identifier: .gregorian)
30
+
cal.timeZone = TimeZone(identifier: "UTC")!
31
+
return cal
32
+
}()
33
+
34
+
/// Generate S3 key for an original photo/video file.
35
+
public static func originalKey(
36
+
uuid: String,
37
+
dateCreated: Date?,
38
+
extension ext: String
39
+
) throws -> String {
40
+
try assertSafeUUID(uuid)
41
+
let cleanExt = ext.lowercased().trimmingPrefix(".")
42
+
let extString = String(cleanExt)
43
+
try assertSafeExtension(extString)
44
+
45
+
let year: String
46
+
let month: String
47
+
if let date = dateCreated {
48
+
let components = utcCalendar.dateComponents([.year, .month], from: date)
49
+
year = String(components.year!)
50
+
month = String(format: "%02d", components.month!)
51
+
} else {
52
+
year = "unknown"
53
+
month = "00"
54
+
}
55
+
56
+
return "originals/\(year)/\(month)/\(uuid).\(extString)"
57
+
}
58
+
59
+
/// Generate S3 key for an asset's metadata JSON.
60
+
public static func metadataKey(uuid: String) throws -> String {
61
+
try assertSafeUUID(uuid)
62
+
return "metadata/assets/\(uuid).json"
63
+
}
64
+
65
+
/// Extract file extension from a UTI or filename.
66
+
public static func extensionFromUTIOrFilename(
67
+
uti: String?,
68
+
filename: String
69
+
) -> String {
70
+
if let uti, let mapped = utiMap[uti] {
71
+
return mapped
72
+
}
73
+
74
+
if let dotIndex = filename.lastIndex(of: ".") {
75
+
let afterDot = filename[filename.index(after: dotIndex)...]
76
+
if !afterDot.isEmpty {
77
+
return afterDot.lowercased()
78
+
}
79
+
}
80
+
81
+
return "bin"
82
+
}
83
+
84
+
// MARK: - Validation
85
+
86
+
/// Whether a string looks like a valid asset UUID (alphanumeric, dots, hyphens, underscores).
87
+
public static func isValidUUID(_ value: String) -> Bool {
88
+
value.wholeMatch(of: uuidPattern) != nil
89
+
}
90
+
91
+
/// Whether a string looks like a valid S3 key (alphanumeric, slashes, dots, hyphens, underscores).
92
+
public static func isValidS3Key(_ value: String) -> Bool {
93
+
value.wholeMatch(of: s3KeyPattern) != nil
94
+
}
95
+
96
+
private static func assertSafeUUID(_ uuid: String) throws {
97
+
guard isValidUUID(uuid) else {
98
+
throw S3PathError.unsafeUUID(uuid)
99
+
}
100
+
}
101
+
102
+
private static func assertSafeExtension(_ ext: String) throws {
103
+
guard ext.wholeMatch(of: extPattern) != nil else {
104
+
throw S3PathError.unsafeExtension(ext)
105
+
}
106
+
}
107
+
}
108
+
109
+
/// Errors from S3 path generation.
110
+
public enum S3PathError: Error, CustomStringConvertible {
111
+
case unsafeUUID(String)
112
+
case unsafeExtension(String)
113
+
114
+
public var description: String {
115
+
switch self {
116
+
case .unsafeUUID(let value):
117
+
"Unsafe UUID for S3 key: \(value)"
118
+
case .unsafeExtension(let value):
119
+
"Unsafe extension for S3 key: \(value)"
120
+
}
121
+
}
122
+
}
+54
Sources/AtticCore/S3Providing.swift
+54
Sources/AtticCore/S3Providing.swift
···
1
+
import Foundation
2
+
3
+
/// Metadata returned by a HEAD request.
4
+
public struct S3ObjectMeta: Sendable {
5
+
public var contentLength: Int
6
+
public var contentType: String?
7
+
8
+
public init(contentLength: Int, contentType: String? = nil) {
9
+
self.contentLength = contentLength
10
+
self.contentType = contentType
11
+
}
12
+
}
13
+
14
+
/// An object listing entry from listObjects.
15
+
public struct S3ListObject: Sendable {
16
+
public var key: String
17
+
public var size: Int
18
+
19
+
public init(key: String, size: Int) {
20
+
self.key = key
21
+
self.size = size
22
+
}
23
+
}
24
+
25
+
/// Protocol for S3-compatible storage operations.
26
+
public protocol S3Providing: Sendable {
27
+
/// Upload an object from data.
28
+
func putObject(key: String, body: Data, contentType: String?) async throws
29
+
30
+
/// Upload an object by streaming from a file URL (avoids loading into memory).
31
+
func putObject(key: String, fileURL: URL, contentType: String?) async throws
32
+
33
+
/// Download an object's contents.
34
+
func getObject(key: String) async throws -> Data
35
+
36
+
/// Get an object's metadata, or nil if it doesn't exist.
37
+
func headObject(key: String) async throws -> S3ObjectMeta?
38
+
39
+
/// List objects with a given prefix.
40
+
func listObjects(prefix: String) async throws -> [S3ListObject]
41
+
}
42
+
43
+
// Convenience overloads.
44
+
extension S3Providing {
45
+
public func putObject(key: String, body: Data) async throws {
46
+
try await putObject(key: key, body: body, contentType: nil)
47
+
}
48
+
49
+
/// Default file upload: uses memory-mapped I/O. Real implementations may override.
50
+
public func putObject(key: String, fileURL: URL, contentType: String?) async throws {
51
+
let data = try Data(contentsOf: fileURL, options: .mappedIfSafe)
52
+
try await putObject(key: key, body: data, contentType: contentType)
53
+
}
54
+
}
+94
Sources/AtticCore/VerifyPipeline.swift
+94
Sources/AtticCore/VerifyPipeline.swift
···
1
+
import Foundation
2
+
3
+
/// Result of a verify run.
4
+
public struct VerifyReport: Sendable {
5
+
public var ok: Int = 0
6
+
public var missing: Int = 0
7
+
public var failed: Int = 0
8
+
public var errors: [(uuid: String, message: String)] = []
9
+
}
10
+
11
+
/// Verify backed-up assets exist in S3 via HEAD requests.
12
+
public func runVerify(
13
+
manifest: Manifest,
14
+
s3: any S3Providing,
15
+
concurrency: Int = 20
16
+
) async throws -> VerifyReport {
17
+
let entries = Array(manifest.entries.values)
18
+
19
+
guard !entries.isEmpty else {
20
+
return VerifyReport()
21
+
}
22
+
23
+
let report = VerifyReportAccumulator()
24
+
25
+
await withTaskGroup(of: Void.self) { group in
26
+
var cursor = 0
27
+
28
+
// Seed initial tasks up to concurrency limit
29
+
for _ in 0..<min(concurrency, entries.count) {
30
+
let entry = entries[cursor]
31
+
cursor += 1
32
+
group.addTask {
33
+
await verifySingle(entry: entry, s3: s3, report: report)
34
+
}
35
+
}
36
+
37
+
// As each task completes, enqueue the next
38
+
for await _ in group {
39
+
if cursor < entries.count {
40
+
let entry = entries[cursor]
41
+
cursor += 1
42
+
group.addTask {
43
+
await verifySingle(entry: entry, s3: s3, report: report)
44
+
}
45
+
}
46
+
}
47
+
}
48
+
49
+
return await report.snapshot()
50
+
}
51
+
52
+
// MARK: - Internals
53
+
54
+
/// Actor to accumulate verify results safely.
55
+
private actor VerifyReportAccumulator {
56
+
var ok = 0
57
+
var missing = 0
58
+
var failed = 0
59
+
var errors: [(uuid: String, message: String)] = []
60
+
61
+
func markOK() { ok += 1 }
62
+
func markMissing(_ uuid: String) {
63
+
missing += 1
64
+
if errors.count < maxReportErrors {
65
+
errors.append((uuid: uuid, message: "Missing from S3"))
66
+
}
67
+
}
68
+
func markFailed(_ uuid: String, _ message: String) {
69
+
failed += 1
70
+
if errors.count < maxReportErrors {
71
+
errors.append((uuid: uuid, message: message))
72
+
}
73
+
}
74
+
75
+
func snapshot() -> VerifyReport {
76
+
VerifyReport(ok: ok, missing: missing, failed: failed, errors: errors)
77
+
}
78
+
}
79
+
80
+
private func verifySingle(
81
+
entry: ManifestEntry,
82
+
s3: any S3Providing,
83
+
report: VerifyReportAccumulator
84
+
) async {
85
+
do {
86
+
if try await s3.headObject(key: entry.s3Key) != nil {
87
+
await report.markOK()
88
+
} else {
89
+
await report.markMissing(entry.uuid)
90
+
}
91
+
} catch {
92
+
await report.markFailed(entry.uuid, String(describing: error))
93
+
}
94
+
}
+117
Tests/AtticCoreTests/AtticConfigTests.swift
+117
Tests/AtticCoreTests/AtticConfigTests.swift
···
1
+
import Testing
2
+
import Foundation
3
+
@testable import AtticCore
4
+
5
+
@Suite("AtticConfig")
6
+
struct AtticConfigTests {
7
+
@Test func validateAcceptsValidConfigWithAllFields() throws {
8
+
let config = try AtticConfig.validate([
9
+
"endpoint": "https://s3.fr-par.scw.cloud",
10
+
"region": "fr-par",
11
+
"bucket": "my-photo-backup",
12
+
"pathStyle": false,
13
+
"keychain": [
14
+
"accessKeyService": "custom-access",
15
+
"secretKeyService": "custom-secret",
16
+
],
17
+
] as [String: Any])
18
+
19
+
#expect(config.endpoint == "https://s3.fr-par.scw.cloud")
20
+
#expect(config.region == "fr-par")
21
+
#expect(config.bucket == "my-photo-backup")
22
+
#expect(config.pathStyle == false)
23
+
#expect(config.keychain.accessKeyService == "custom-access")
24
+
#expect(config.keychain.secretKeyService == "custom-secret")
25
+
}
26
+
27
+
@Test func validateAppliesDefaultsForOptionalFields() throws {
28
+
let config = try AtticConfig.validate([
29
+
"endpoint": "https://s3.fr-par.scw.cloud",
30
+
"region": "fr-par",
31
+
"bucket": "my-photo-backup",
32
+
] as [String: Any])
33
+
34
+
#expect(config.pathStyle == true)
35
+
#expect(config.keychain.accessKeyService == "attic-s3-access-key")
36
+
#expect(config.keychain.secretKeyService == "attic-s3-secret-key")
37
+
}
38
+
39
+
@Test func validateRejectsMissingEndpoint() {
40
+
#expect(throws: ConfigError.self) {
41
+
try AtticConfig.validate(["region": "fr-par", "bucket": "b"] as [String: Any])
42
+
}
43
+
}
44
+
45
+
@Test func validateRejectsNonHTTPSEndpoint() {
46
+
#expect(throws: ConfigError.self) {
47
+
try AtticConfig.validate([
48
+
"endpoint": "http://s3.example.com",
49
+
"region": "us-east-1",
50
+
"bucket": "bbb",
51
+
] as [String: Any])
52
+
}
53
+
}
54
+
55
+
@Test func validateRejectsMissingRegion() {
56
+
#expect(throws: ConfigError.self) {
57
+
try AtticConfig.validate([
58
+
"endpoint": "https://s3.example.com",
59
+
"bucket": "bbb",
60
+
] as [String: Any])
61
+
}
62
+
}
63
+
64
+
@Test func validateRejectsMissingBucket() {
65
+
#expect(throws: ConfigError.self) {
66
+
try AtticConfig.validate([
67
+
"endpoint": "https://s3.example.com",
68
+
"region": "us-east-1",
69
+
] as [String: Any])
70
+
}
71
+
}
72
+
73
+
@Test func validateRejectsInvalidBucketName() {
74
+
#expect(throws: ConfigError.self) {
75
+
try AtticConfig.validate([
76
+
"endpoint": "https://s3.example.com",
77
+
"region": "us-east-1",
78
+
"bucket": "A",
79
+
] as [String: Any])
80
+
}
81
+
}
82
+
83
+
@Test func validateRejectsNonObjectInput() {
84
+
#expect(throws: ConfigError.self) {
85
+
try AtticConfig.validate("not an object" as Any)
86
+
}
87
+
#expect(throws: ConfigError.self) {
88
+
try AtticConfig.validate([] as [Any] as Any)
89
+
}
90
+
}
91
+
92
+
@Test func writeAndLoadConfigRoundTrip() throws {
93
+
let dir = FileManager.default.temporaryDirectory
94
+
.appendingPathComponent(UUID().uuidString)
95
+
defer { try? FileManager.default.removeItem(at: dir) }
96
+
97
+
let provider = FileConfigProvider(directory: dir)
98
+
let config = AtticConfig(
99
+
endpoint: "https://s3.fr-par.scw.cloud",
100
+
region: "fr-par",
101
+
bucket: "test-bucket"
102
+
)
103
+
104
+
try provider.write(config)
105
+
106
+
let loaded = try provider.load()
107
+
#expect(loaded == config)
108
+
}
109
+
110
+
@Test func loadReturnsNilWhenFileDoesNotExist() throws {
111
+
let dir = FileManager.default.temporaryDirectory
112
+
.appendingPathComponent(UUID().uuidString)
113
+
let provider = FileConfigProvider(directory: dir)
114
+
let result = try provider.load()
115
+
#expect(result == nil)
116
+
}
117
+
}
+346
Tests/AtticCoreTests/BackupPipelineTests.swift
+346
Tests/AtticCoreTests/BackupPipelineTests.swift
···
1
+
import Testing
2
+
import Foundation
3
+
import LadderKit
4
+
@testable import AtticCore
5
+
6
+
// MARK: - Test helpers
7
+
8
+
/// Mock exporter that returns pre-configured results from an in-memory map.
9
+
struct MockExportProvider: ExportProviding {
10
+
/// Map of UUID → (filename, data). UUIDs not in the map produce errors.
11
+
let availableAssets: [String: (filename: String, data: Data)]
12
+
let stagingDir: URL
13
+
14
+
init(
15
+
assets: [String: (filename: String, data: Data)] = [:],
16
+
stagingDir: URL? = nil
17
+
) {
18
+
self.availableAssets = assets
19
+
self.stagingDir = stagingDir ?? FileManager.default.temporaryDirectory
20
+
.appendingPathComponent("attic-test-staging-\(UUID().uuidString)")
21
+
}
22
+
23
+
func exportBatch(uuids: [String]) async throws -> ExportResponse {
24
+
let fm = FileManager.default
25
+
if !fm.fileExists(atPath: stagingDir.path) {
26
+
try fm.createDirectory(at: stagingDir, withIntermediateDirectories: true)
27
+
}
28
+
29
+
var results: [ExportResult] = []
30
+
var errors: [LadderKit.ExportError] = []
31
+
32
+
for uuid in uuids {
33
+
if let asset = availableAssets[uuid] {
34
+
let filePath = stagingDir.appendingPathComponent(asset.filename)
35
+
try asset.data.write(to: filePath)
36
+
results.append(ExportResult(
37
+
uuid: uuid,
38
+
path: filePath.path,
39
+
size: Int64(asset.data.count),
40
+
sha256: "fakehash_\(uuid)"
41
+
))
42
+
} else {
43
+
errors.append(LadderKit.ExportError(
44
+
uuid: uuid,
45
+
message: "Asset not found in mock library"
46
+
))
47
+
}
48
+
}
49
+
50
+
return ExportResponse(results: results, errors: errors)
51
+
}
52
+
53
+
func checkPermissions() async throws {}
54
+
}
55
+
56
+
/// Mock exporter that throws timeout on batches containing specific UUIDs.
57
+
struct TimeoutExportProvider: ExportProviding {
58
+
let inner: MockExportProvider
59
+
let slowUUIDs: Set<String>
60
+
let deferredRetrySucceeds: Bool
61
+
private let retryCounter = RetryCounter()
62
+
63
+
actor RetryCounter {
64
+
var counts: [String: Int] = [:]
65
+
func increment(_ uuid: String) -> Int {
66
+
counts[uuid, default: 0] += 1
67
+
return counts[uuid]!
68
+
}
69
+
}
70
+
71
+
func exportBatch(uuids: [String]) async throws -> ExportResponse {
72
+
let containsSlow = uuids.contains { slowUUIDs.contains($0) }
73
+
74
+
if containsSlow && uuids.count > 1 {
75
+
throw ExportProviderError.timeout(seconds: 300)
76
+
}
77
+
78
+
if uuids.count == 1, let uuid = uuids.first, slowUUIDs.contains(uuid) {
79
+
let count = await retryCounter.increment(uuid)
80
+
if count == 1 {
81
+
throw ExportProviderError.timeout(seconds: 300)
82
+
}
83
+
}
84
+
85
+
return try await inner.exportBatch(uuids: uuids)
86
+
}
87
+
88
+
func checkPermissions() async throws {}
89
+
}
90
+
91
+
func makeTestAsset(
92
+
uuid: String,
93
+
kind: AssetKind = .photo,
94
+
filename: String = "IMG_0001.HEIC",
95
+
uti: String = "public.heic"
96
+
) -> AssetInfo {
97
+
AssetInfo(
98
+
identifier: "\(uuid)/L0/001",
99
+
creationDate: ISO8601DateFormatter().date(from: "2024-01-15T12:00:00Z"),
100
+
kind: kind,
101
+
pixelWidth: 4032,
102
+
pixelHeight: 3024,
103
+
latitude: 52.09,
104
+
longitude: 4.34,
105
+
isFavorite: false,
106
+
originalFilename: filename,
107
+
uniformTypeIdentifier: uti,
108
+
hasEdit: false
109
+
)
110
+
}
111
+
112
+
func createTestContext() async throws -> (MockS3Provider, S3ManifestStore) {
113
+
let s3 = MockS3Provider()
114
+
let store = S3ManifestStore(s3: s3)
115
+
return (s3, store)
116
+
}
117
+
118
+
// MARK: - Tests
119
+
120
+
@Suite("BackupPipeline")
121
+
struct BackupPipelineTests {
122
+
@Test func uploadsPendingAssetsAndUpdatesManifest() async throws {
123
+
let assets = [makeTestAsset(uuid: "uuid-1"), makeTestAsset(uuid: "uuid-2")]
124
+
125
+
let exporter = MockExportProvider(assets: [
126
+
"uuid-1": ("IMG_0001.HEIC", Data("photo1".utf8)),
127
+
"uuid-2": ("IMG_0002.HEIC", Data("photo2".utf8)),
128
+
])
129
+
let (s3, manifestStore) = try await createTestContext()
130
+
var manifest = try await manifestStore.load()
131
+
132
+
let report = try await runBackup(
133
+
assets: assets,
134
+
manifest: &manifest,
135
+
manifestStore: manifestStore,
136
+
exporter: exporter,
137
+
s3: s3,
138
+
options: BackupOptions(batchSize: 10)
139
+
)
140
+
141
+
#expect(report.uploaded == 2)
142
+
#expect(report.failed == 0)
143
+
#expect(report.errors.isEmpty)
144
+
#expect(manifest.isBackedUp("uuid-1"))
145
+
#expect(manifest.isBackedUp("uuid-2"))
146
+
147
+
// S3 should have originals + metadata + manifest
148
+
let manifestExists = try await s3.headObject(key: "manifest.json")
149
+
#expect(manifestExists != nil)
150
+
}
151
+
152
+
@Test func skipsAlreadyBackedUpAssets() async throws {
153
+
let assets = [makeTestAsset(uuid: "uuid-1"), makeTestAsset(uuid: "uuid-2")]
154
+
155
+
let exporter = MockExportProvider(assets: [
156
+
"uuid-2": ("IMG_0002.HEIC", Data("photo2".utf8)),
157
+
])
158
+
let (s3, manifestStore) = try await createTestContext()
159
+
var manifest = try await manifestStore.load()
160
+
161
+
// Pre-mark uuid-1 as backed up
162
+
manifest.markBackedUp(
163
+
uuid: "uuid-1",
164
+
s3Key: "originals/2024/01/uuid-1.heic",
165
+
checksum: "sha256:abc",
166
+
backedUpAt: "2024-01-15T00:00:00Z"
167
+
)
168
+
169
+
let report = try await runBackup(
170
+
assets: assets,
171
+
manifest: &manifest,
172
+
manifestStore: manifestStore,
173
+
exporter: exporter,
174
+
s3: s3,
175
+
options: BackupOptions(batchSize: 10)
176
+
)
177
+
178
+
#expect(report.uploaded == 1)
179
+
#expect(report.failed == 0)
180
+
}
181
+
182
+
@Test func respectsLimitFlag() async throws {
183
+
let assets = [
184
+
makeTestAsset(uuid: "uuid-1"),
185
+
makeTestAsset(uuid: "uuid-2"),
186
+
makeTestAsset(uuid: "uuid-3"),
187
+
]
188
+
189
+
let exporter = MockExportProvider(assets: [
190
+
"uuid-1": ("IMG_0001.HEIC", Data("p1".utf8)),
191
+
])
192
+
let (s3, manifestStore) = try await createTestContext()
193
+
var manifest = try await manifestStore.load()
194
+
195
+
let report = try await runBackup(
196
+
assets: assets,
197
+
manifest: &manifest,
198
+
manifestStore: manifestStore,
199
+
exporter: exporter,
200
+
s3: s3,
201
+
options: BackupOptions(batchSize: 10, limit: 1)
202
+
)
203
+
204
+
#expect(report.uploaded == 1)
205
+
}
206
+
207
+
@Test func dryRunSkipsUploads() async throws {
208
+
let assets = [makeTestAsset(uuid: "uuid-1")]
209
+
210
+
let exporter = MockExportProvider()
211
+
let (s3, manifestStore) = try await createTestContext()
212
+
var manifest = try await manifestStore.load()
213
+
214
+
let report = try await runBackup(
215
+
assets: assets,
216
+
manifest: &manifest,
217
+
manifestStore: manifestStore,
218
+
exporter: exporter,
219
+
s3: s3,
220
+
options: BackupOptions(dryRun: true)
221
+
)
222
+
223
+
#expect(report.uploaded == 0)
224
+
#expect(report.skipped == 1)
225
+
let objects = try await s3.listObjects(prefix: "")
226
+
#expect(objects.isEmpty)
227
+
#expect(!manifest.isBackedUp("uuid-1"))
228
+
}
229
+
230
+
@Test func handlesExportErrorsGracefully() async throws {
231
+
let assets = [
232
+
makeTestAsset(uuid: "uuid-1"),
233
+
makeTestAsset(uuid: "uuid-missing"),
234
+
]
235
+
236
+
let exporter = MockExportProvider(assets: [
237
+
"uuid-1": ("IMG_0001.HEIC", Data("data".utf8)),
238
+
])
239
+
let (s3, manifestStore) = try await createTestContext()
240
+
var manifest = try await manifestStore.load()
241
+
242
+
let report = try await runBackup(
243
+
assets: assets,
244
+
manifest: &manifest,
245
+
manifestStore: manifestStore,
246
+
exporter: exporter,
247
+
s3: s3,
248
+
options: BackupOptions(batchSize: 10)
249
+
)
250
+
251
+
#expect(report.uploaded == 1)
252
+
#expect(report.failed == 1)
253
+
#expect(report.errors.count == 1)
254
+
#expect(report.errors[0].uuid == "uuid-missing")
255
+
}
256
+
257
+
@Test func filtersByType() async throws {
258
+
let assets = [
259
+
makeTestAsset(uuid: "photo-1", kind: .photo),
260
+
makeTestAsset(
261
+
uuid: "video-1",
262
+
kind: .video,
263
+
filename: "VID.MOV",
264
+
uti: "com.apple.quicktime-movie"
265
+
),
266
+
]
267
+
268
+
let exporter = MockExportProvider(assets: [
269
+
"video-1": ("VID.MOV", Data("video".utf8)),
270
+
])
271
+
let (s3, manifestStore) = try await createTestContext()
272
+
var manifest = try await manifestStore.load()
273
+
274
+
let report = try await runBackup(
275
+
assets: assets,
276
+
manifest: &manifest,
277
+
manifestStore: manifestStore,
278
+
exporter: exporter,
279
+
s3: s3,
280
+
options: BackupOptions(batchSize: 10, type: .video)
281
+
)
282
+
283
+
#expect(report.uploaded == 1)
284
+
#expect(manifest.isBackedUp("video-1"))
285
+
#expect(!manifest.isBackedUp("photo-1"))
286
+
}
287
+
288
+
@Test func defersTimedOutAssetsAndRetriesAfterBatches() async throws {
289
+
let assets = [
290
+
makeTestAsset(uuid: "fast-1"),
291
+
makeTestAsset(uuid: "slow-1"),
292
+
makeTestAsset(uuid: "fast-2"),
293
+
]
294
+
295
+
let inner = MockExportProvider(assets: [
296
+
"fast-1": ("IMG_0001.HEIC", Data("f1".utf8)),
297
+
"slow-1": ("BIG_VIDEO.MOV", Data("s1".utf8)),
298
+
"fast-2": ("IMG_0003.HEIC", Data("f2".utf8)),
299
+
])
300
+
let exporter = TimeoutExportProvider(
301
+
inner: inner,
302
+
slowUUIDs: ["slow-1"],
303
+
deferredRetrySucceeds: true
304
+
)
305
+
let (s3, manifestStore) = try await createTestContext()
306
+
var manifest = try await manifestStore.load()
307
+
308
+
let report = try await runBackup(
309
+
assets: assets,
310
+
manifest: &manifest,
311
+
manifestStore: manifestStore,
312
+
exporter: exporter,
313
+
s3: s3,
314
+
options: BackupOptions(batchSize: 3)
315
+
)
316
+
317
+
#expect(report.uploaded == 3)
318
+
#expect(report.failed == 0)
319
+
#expect(manifest.isBackedUp("fast-1"))
320
+
#expect(manifest.isBackedUp("fast-2"))
321
+
#expect(manifest.isBackedUp("slow-1"))
322
+
}
323
+
324
+
@Test func savesManifestToS3() async throws {
325
+
let assets = [makeTestAsset(uuid: "uuid-1")]
326
+
327
+
let exporter = MockExportProvider(assets: [
328
+
"uuid-1": ("IMG_0001.HEIC", Data("data".utf8)),
329
+
])
330
+
let (s3, manifestStore) = try await createTestContext()
331
+
var manifest = try await manifestStore.load()
332
+
333
+
_ = try await runBackup(
334
+
assets: assets,
335
+
manifest: &manifest,
336
+
manifestStore: manifestStore,
337
+
exporter: exporter,
338
+
s3: s3,
339
+
options: BackupOptions(batchSize: 10)
340
+
)
341
+
342
+
// Load from S3 — should persist
343
+
let loaded = try await manifestStore.load()
344
+
#expect(loaded.isBackedUp("uuid-1"))
345
+
}
346
+
}
+27
Tests/AtticCoreTests/FormatBytesTests.swift
+27
Tests/AtticCoreTests/FormatBytesTests.swift
···
1
+
import Testing
2
+
@testable import AtticCore
3
+
4
+
@Suite("FormatBytes")
5
+
struct FormatBytesTests {
6
+
@Test func zero() {
7
+
#expect(formatBytes(0) == "0 B")
8
+
}
9
+
10
+
@Test func bytes() {
11
+
#expect(formatBytes(500) == "500.0 B")
12
+
}
13
+
14
+
@Test func kilobytes() {
15
+
#expect(formatBytes(1024) == "1.0 KB")
16
+
#expect(formatBytes(1536) == "1.5 KB")
17
+
}
18
+
19
+
@Test func megabytes() {
20
+
#expect(formatBytes(1_048_576) == "1.0 MB")
21
+
#expect(formatBytes(2_500_000) == "2.4 MB")
22
+
}
23
+
24
+
@Test func gigabytes() {
25
+
#expect(formatBytes(1_073_741_824) == "1.0 GB")
26
+
}
27
+
}
+111
Tests/AtticCoreTests/ManifestCompatibilityTests.swift
+111
Tests/AtticCoreTests/ManifestCompatibilityTests.swift
···
1
+
import Testing
2
+
import Foundation
3
+
@testable import AtticCore
4
+
5
+
@Suite("Manifest cross-version compatibility")
6
+
struct ManifestCompatibilityTests {
7
+
/// A manifest JSON produced by the Deno CLI (v0.2.x format).
8
+
/// The Swift version must be able to parse this exactly.
9
+
static let denoManifestJSON = """
10
+
{
11
+
"entries": {
12
+
"A1B2C3D4-E5F6-7890-ABCD-EF1234567890": {
13
+
"uuid": "A1B2C3D4-E5F6-7890-ABCD-EF1234567890",
14
+
"s3Key": "originals/2024/01/A1B2C3D4-E5F6-7890-ABCD-EF1234567890.heic",
15
+
"checksum": "sha256:9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08",
16
+
"backedUpAt": "2024-01-15T12:34:56Z",
17
+
"size": 4194304
18
+
},
19
+
"DEADBEEF-CAFE-4321-BABE-FEEDFACE1234": {
20
+
"uuid": "DEADBEEF-CAFE-4321-BABE-FEEDFACE1234",
21
+
"s3Key": "originals/2023/12/DEADBEEF-CAFE-4321-BABE-FEEDFACE1234.mov",
22
+
"checksum": "sha256:60303ae22b998861bce3b28f33eec1be758a213c86c93c076dbe9f558c11c752",
23
+
"backedUpAt": "2024-02-20T08:15:00Z"
24
+
}
25
+
}
26
+
}
27
+
"""
28
+
29
+
@Test func parsesDenoManifestFormat() throws {
30
+
let data = Data(Self.denoManifestJSON.utf8)
31
+
let manifest = try Manifest.parse(from: data)
32
+
33
+
#expect(manifest.entries.count == 2)
34
+
35
+
let entry1 = manifest.entries["A1B2C3D4-E5F6-7890-ABCD-EF1234567890"]
36
+
#expect(entry1 != nil)
37
+
#expect(entry1?.s3Key == "originals/2024/01/A1B2C3D4-E5F6-7890-ABCD-EF1234567890.heic")
38
+
#expect(entry1?.checksum == "sha256:9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08")
39
+
#expect(entry1?.backedUpAt == "2024-01-15T12:34:56Z")
40
+
#expect(entry1?.size == 4194304)
41
+
42
+
let entry2 = manifest.entries["DEADBEEF-CAFE-4321-BABE-FEEDFACE1234"]
43
+
#expect(entry2 != nil)
44
+
#expect(entry2?.size == nil) // Optional field, not present
45
+
}
46
+
47
+
@Test func swiftManifestRoundTripsToDenoCompatibleFormat() throws {
48
+
var manifest = Manifest()
49
+
manifest.markBackedUp(
50
+
uuid: "uuid-1",
51
+
s3Key: "originals/2024/06/uuid-1.heic",
52
+
checksum: "sha256:abc123def456",
53
+
size: 1024,
54
+
backedUpAt: "2024-06-15T10:00:00Z"
55
+
)
56
+
57
+
// Encode and re-parse
58
+
let data = try manifest.encoded()
59
+
let parsed = try JSONSerialization.jsonObject(with: data) as! [String: Any]
60
+
61
+
// Verify structure matches Deno format
62
+
let entries = parsed["entries"] as! [String: Any]
63
+
let entry = entries["uuid-1"] as! [String: Any]
64
+
65
+
#expect(entry["uuid"] as? String == "uuid-1")
66
+
#expect(entry["s3Key"] as? String == "originals/2024/06/uuid-1.heic")
67
+
#expect(entry["checksum"] as? String == "sha256:abc123def456")
68
+
#expect(entry["backedUpAt"] as? String == "2024-06-15T10:00:00Z")
69
+
#expect(entry["size"] as? Int == 1024)
70
+
}
71
+
72
+
@Test func swiftCanContinueDenoBackup() throws {
73
+
// Parse a Deno manifest, add a Swift entry, re-encode, re-parse
74
+
let data = Data(Self.denoManifestJSON.utf8)
75
+
var manifest = try Manifest.parse(from: data)
76
+
77
+
#expect(manifest.isBackedUp("A1B2C3D4-E5F6-7890-ABCD-EF1234567890"))
78
+
#expect(!manifest.isBackedUp("NEW-UUID"))
79
+
80
+
// Swift adds a new entry
81
+
manifest.markBackedUp(
82
+
uuid: "NEW-UUID",
83
+
s3Key: "originals/2025/03/NEW-UUID.png",
84
+
checksum: "sha256:newchecksum",
85
+
size: 2048,
86
+
backedUpAt: "2025-03-21T00:00:00Z"
87
+
)
88
+
89
+
// Round-trip
90
+
let encoded = try manifest.encoded()
91
+
let reloaded = try Manifest.parse(from: encoded)
92
+
93
+
#expect(reloaded.entries.count == 3)
94
+
#expect(reloaded.isBackedUp("A1B2C3D4-E5F6-7890-ABCD-EF1234567890"))
95
+
#expect(reloaded.isBackedUp("DEADBEEF-CAFE-4321-BABE-FEEDFACE1234"))
96
+
#expect(reloaded.isBackedUp("NEW-UUID"))
97
+
98
+
// Original entries preserved exactly
99
+
let original = reloaded.entries["A1B2C3D4-E5F6-7890-ABCD-EF1234567890"]
100
+
#expect(original?.size == 4194304)
101
+
#expect(original?.backedUpAt == "2024-01-15T12:34:56Z")
102
+
}
103
+
104
+
@Test func handlesEmptyManifestFromDeno() throws {
105
+
let json = """
106
+
{ "entries": {} }
107
+
"""
108
+
let manifest = try Manifest.parse(from: Data(json.utf8))
109
+
#expect(manifest.entries.isEmpty)
110
+
}
111
+
}
+70
Tests/AtticCoreTests/ManifestTests.swift
+70
Tests/AtticCoreTests/ManifestTests.swift
···
1
+
import Testing
2
+
import Foundation
3
+
@testable import AtticCore
4
+
5
+
@Suite("Manifest")
6
+
struct ManifestTests {
7
+
@Test func emptyManifest() {
8
+
let manifest = Manifest()
9
+
#expect(manifest.entries.isEmpty)
10
+
#expect(!manifest.isBackedUp("any-uuid"))
11
+
}
12
+
13
+
@Test func markBackedUp() {
14
+
var manifest = Manifest()
15
+
manifest.markBackedUp(
16
+
uuid: "test-uuid",
17
+
s3Key: "originals/2024/01/test-uuid.heic",
18
+
checksum: "sha256:abc123",
19
+
size: 1024,
20
+
backedUpAt: "2024-01-15T12:00:00Z"
21
+
)
22
+
23
+
#expect(manifest.isBackedUp("test-uuid"))
24
+
#expect(!manifest.isBackedUp("other-uuid"))
25
+
26
+
let entry = manifest.entries["test-uuid"]!
27
+
#expect(entry.uuid == "test-uuid")
28
+
#expect(entry.s3Key == "originals/2024/01/test-uuid.heic")
29
+
#expect(entry.checksum == "sha256:abc123")
30
+
#expect(entry.size == 1024)
31
+
#expect(entry.backedUpAt == "2024-01-15T12:00:00Z")
32
+
}
33
+
34
+
@Test func markBackedUpDefaultsTimestamp() {
35
+
var manifest = Manifest()
36
+
manifest.markBackedUp(
37
+
uuid: "test-uuid",
38
+
s3Key: "originals/2024/01/test-uuid.heic",
39
+
checksum: "sha256:abc123"
40
+
)
41
+
42
+
let entry = manifest.entries["test-uuid"]!
43
+
#expect(!entry.backedUpAt.isEmpty)
44
+
}
45
+
46
+
@Test func encodeAndParseRoundTrip() throws {
47
+
var manifest = Manifest()
48
+
manifest.markBackedUp(
49
+
uuid: "uuid-1",
50
+
s3Key: "originals/2024/01/uuid-1.heic",
51
+
checksum: "sha256:aaa",
52
+
size: 500,
53
+
backedUpAt: "2024-01-01T00:00:00Z"
54
+
)
55
+
manifest.markBackedUp(
56
+
uuid: "uuid-2",
57
+
s3Key: "originals/2024/02/uuid-2.jpg",
58
+
checksum: "sha256:bbb",
59
+
backedUpAt: "2024-02-01T00:00:00Z"
60
+
)
61
+
62
+
let data = try manifest.encoded()
63
+
let parsed = try Manifest.parse(from: data)
64
+
65
+
#expect(parsed.entries.count == 2)
66
+
#expect(parsed.entries["uuid-1"]?.s3Key == "originals/2024/01/uuid-1.heic")
67
+
#expect(parsed.entries["uuid-2"]?.checksum == "sha256:bbb")
68
+
#expect(parsed.entries["uuid-2"]?.size == nil)
69
+
}
70
+
}
+84
Tests/AtticCoreTests/MetadataBuilderTests.swift
+84
Tests/AtticCoreTests/MetadataBuilderTests.swift
···
1
+
import Testing
2
+
import Foundation
3
+
import LadderKit
4
+
@testable import AtticCore
5
+
6
+
@Suite("MetadataBuilder")
7
+
struct MetadataBuilderTests {
8
+
@Test func buildsMetadataFromAssetInfo() {
9
+
let date = ISO8601DateFormatter().date(from: "2024-06-15T10:30:00Z")!
10
+
let asset = AssetInfo(
11
+
identifier: "ABC-123/L0/001",
12
+
creationDate: date,
13
+
kind: .photo,
14
+
pixelWidth: 4032,
15
+
pixelHeight: 3024,
16
+
latitude: 52.3676,
17
+
longitude: 4.9041,
18
+
isFavorite: true,
19
+
originalFilename: "IMG_001.HEIC",
20
+
uniformTypeIdentifier: "public.heic",
21
+
hasEdit: false,
22
+
albums: [AlbumInfo(identifier: "album-1", title: "Vacation")],
23
+
keywords: ["beach", "summer"],
24
+
people: [PersonInfo(uuid: "person-1", displayName: "Alice")],
25
+
assetDescription: "A sunny beach photo"
26
+
)
27
+
28
+
let metadata = buildMetadataJSON(
29
+
asset: asset,
30
+
s3Key: "originals/2024/06/ABC-123.heic",
31
+
checksum: "sha256:abc123",
32
+
backedUpAt: "2024-06-15T12:00:00Z"
33
+
)
34
+
35
+
#expect(metadata.uuid == "ABC-123")
36
+
#expect(metadata.originalFilename == "IMG_001.HEIC")
37
+
#expect(metadata.dateCreated == "2024-06-15T10:30:00Z")
38
+
#expect(metadata.width == 4032)
39
+
#expect(metadata.height == 3024)
40
+
#expect(metadata.latitude == 52.3676)
41
+
#expect(metadata.longitude == 4.9041)
42
+
#expect(metadata.favorite == true)
43
+
#expect(metadata.description == "A sunny beach photo")
44
+
#expect(metadata.albums.count == 1)
45
+
#expect(metadata.albums[0].title == "Vacation")
46
+
#expect(metadata.keywords == ["beach", "summer"])
47
+
#expect(metadata.people.count == 1)
48
+
#expect(metadata.people[0].displayName == "Alice")
49
+
#expect(metadata.hasEdit == false)
50
+
#expect(metadata.s3Key == "originals/2024/06/ABC-123.heic")
51
+
#expect(metadata.checksum == "sha256:abc123")
52
+
#expect(metadata.backedUpAt == "2024-06-15T12:00:00Z")
53
+
}
54
+
55
+
@Test func handlesNilOptionalFields() {
56
+
let asset = AssetInfo(
57
+
identifier: "XYZ-789/L0/001",
58
+
creationDate: nil,
59
+
kind: .video,
60
+
pixelWidth: 1920,
61
+
pixelHeight: 1080,
62
+
latitude: nil,
63
+
longitude: nil,
64
+
isFavorite: false
65
+
)
66
+
67
+
let metadata = buildMetadataJSON(
68
+
asset: asset,
69
+
s3Key: "originals/unknown/00/XYZ-789.mov",
70
+
checksum: "sha256:def456",
71
+
backedUpAt: "2024-01-01T00:00:00Z"
72
+
)
73
+
74
+
#expect(metadata.uuid == "XYZ-789")
75
+
#expect(metadata.originalFilename == "unknown")
76
+
#expect(metadata.dateCreated == nil)
77
+
#expect(metadata.latitude == nil)
78
+
#expect(metadata.longitude == nil)
79
+
#expect(metadata.description == nil)
80
+
#expect(metadata.albums.isEmpty)
81
+
#expect(metadata.keywords.isEmpty)
82
+
#expect(metadata.people.isEmpty)
83
+
}
84
+
}
+131
Tests/AtticCoreTests/RebuildManifestTests.swift
+131
Tests/AtticCoreTests/RebuildManifestTests.swift
···
1
+
import Testing
2
+
import Foundation
3
+
@testable import AtticCore
4
+
5
+
@Suite("RebuildManifest")
6
+
struct RebuildManifestTests {
7
+
@Test func rebuildsManifestFromMetadataFiles() async throws {
8
+
let s3 = MockS3Provider()
9
+
let store = S3ManifestStore(s3: s3)
10
+
11
+
// Upload a metadata JSON file to S3
12
+
let metaJSON = """
13
+
{
14
+
"uuid": "uuid-1",
15
+
"s3Key": "originals/2024/01/uuid-1.heic",
16
+
"checksum": "sha256:abc123",
17
+
"backedUpAt": "2024-01-15T00:00:00Z",
18
+
"originalFilename": "IMG_0001.HEIC",
19
+
"width": 4032,
20
+
"height": 3024,
21
+
"favorite": false,
22
+
"hasEdit": false,
23
+
"albums": [],
24
+
"keywords": [],
25
+
"people": []
26
+
}
27
+
"""
28
+
try await s3.putObject(
29
+
key: "metadata/assets/uuid-1.json",
30
+
body: Data(metaJSON.utf8),
31
+
contentType: "application/json"
32
+
)
33
+
34
+
let (manifest, report) = try await runRebuildManifest(s3: s3, manifestStore: store)
35
+
36
+
#expect(report.recovered == 1)
37
+
#expect(report.errors.isEmpty)
38
+
#expect(manifest.isBackedUp("uuid-1"))
39
+
#expect(manifest.entries["uuid-1"]?.s3Key == "originals/2024/01/uuid-1.heic")
40
+
#expect(manifest.entries["uuid-1"]?.checksum == "sha256:abc123")
41
+
}
42
+
43
+
@Test func skipsNonJSONFiles() async throws {
44
+
let s3 = MockS3Provider()
45
+
let store = S3ManifestStore(s3: s3)
46
+
47
+
try await s3.putObject(
48
+
key: "metadata/assets/readme.txt",
49
+
body: Data("not json".utf8)
50
+
)
51
+
52
+
let (manifest, report) = try await runRebuildManifest(s3: s3, manifestStore: store)
53
+
54
+
#expect(report.skipped == 1)
55
+
#expect(report.recovered == 0)
56
+
#expect(manifest.entries.isEmpty)
57
+
}
58
+
59
+
@Test func handlesInvalidJSON() async throws {
60
+
let s3 = MockS3Provider()
61
+
let store = S3ManifestStore(s3: s3)
62
+
63
+
try await s3.putObject(
64
+
key: "metadata/assets/broken.json",
65
+
body: Data("not valid json".utf8)
66
+
)
67
+
68
+
let (manifest, report) = try await runRebuildManifest(s3: s3, manifestStore: store)
69
+
70
+
#expect(report.errors.count == 1)
71
+
#expect(report.recovered == 0)
72
+
#expect(manifest.entries.isEmpty)
73
+
}
74
+
75
+
@Test func rejectsInvalidUUIDs() async throws {
76
+
let s3 = MockS3Provider()
77
+
let store = S3ManifestStore(s3: s3)
78
+
79
+
let metaJSON = """
80
+
{
81
+
"uuid": "../evil",
82
+
"s3Key": "originals/2024/01/evil.heic",
83
+
"checksum": "sha256:abc123"
84
+
}
85
+
"""
86
+
try await s3.putObject(
87
+
key: "metadata/assets/evil.json",
88
+
body: Data(metaJSON.utf8)
89
+
)
90
+
91
+
let (manifest, report) = try await runRebuildManifest(s3: s3, manifestStore: store)
92
+
93
+
#expect(report.errors.count == 1)
94
+
#expect(report.recovered == 0)
95
+
#expect(manifest.entries.isEmpty)
96
+
}
97
+
98
+
@Test func savesRebuiltManifestToS3() async throws {
99
+
let s3 = MockS3Provider()
100
+
let store = S3ManifestStore(s3: s3)
101
+
102
+
let metaJSON = """
103
+
{
104
+
"uuid": "uuid-1",
105
+
"s3Key": "originals/2024/01/uuid-1.heic",
106
+
"checksum": "sha256:abc123",
107
+
"backedUpAt": "2024-01-15T00:00:00Z"
108
+
}
109
+
"""
110
+
try await s3.putObject(
111
+
key: "metadata/assets/uuid-1.json",
112
+
body: Data(metaJSON.utf8)
113
+
)
114
+
115
+
_ = try await runRebuildManifest(s3: s3, manifestStore: store)
116
+
117
+
// Verify manifest was persisted
118
+
let loaded = try await store.load()
119
+
#expect(loaded.isBackedUp("uuid-1"))
120
+
}
121
+
122
+
@Test func emptyS3ReturnsEmptyManifest() async throws {
123
+
let s3 = MockS3Provider()
124
+
let store = S3ManifestStore(s3: s3)
125
+
126
+
let (manifest, report) = try await runRebuildManifest(s3: s3, manifestStore: store)
127
+
128
+
#expect(report.recovered == 0)
129
+
#expect(manifest.entries.isEmpty)
130
+
}
131
+
}
+82
Tests/AtticCoreTests/RefreshMetadataTests.swift
+82
Tests/AtticCoreTests/RefreshMetadataTests.swift
···
1
+
import Testing
2
+
import Foundation
3
+
import LadderKit
4
+
@testable import AtticCore
5
+
6
+
@Suite("RefreshMetadata")
7
+
struct RefreshMetadataTests {
8
+
@Test func refreshesMetadataForBackedUpAssets() async throws {
9
+
let s3 = MockS3Provider()
10
+
var manifest = Manifest()
11
+
manifest.markBackedUp(
12
+
uuid: "uuid-1",
13
+
s3Key: "originals/2024/01/uuid-1.heic",
14
+
checksum: "sha256:abc",
15
+
backedUpAt: "2024-01-15T00:00:00Z"
16
+
)
17
+
18
+
let assets = [makeTestAsset(uuid: "uuid-1")]
19
+
20
+
let report = try await runRefreshMetadata(
21
+
assets: assets, manifest: manifest, s3: s3
22
+
)
23
+
24
+
#expect(report.updated == 1)
25
+
#expect(report.failed == 0)
26
+
27
+
// Verify metadata was uploaded
28
+
let metaKey = "metadata/assets/uuid-1.json"
29
+
let data = try await s3.getObject(key: metaKey)
30
+
let meta = try JSONDecoder().decode(AssetMetadata.self, from: data)
31
+
#expect(meta.uuid == "uuid-1")
32
+
#expect(meta.s3Key == "originals/2024/01/uuid-1.heic")
33
+
}
34
+
35
+
@Test func skipsAssetsNotInManifest() async throws {
36
+
let s3 = MockS3Provider()
37
+
let manifest = Manifest()
38
+
let assets = [makeTestAsset(uuid: "uuid-1")]
39
+
40
+
let report = try await runRefreshMetadata(
41
+
assets: assets, manifest: manifest, s3: s3
42
+
)
43
+
44
+
#expect(report.updated == 0)
45
+
let objects = try await s3.listObjects(prefix: "metadata/")
46
+
#expect(objects.isEmpty)
47
+
}
48
+
49
+
@Test func dryRunSkipsUpload() async throws {
50
+
let s3 = MockS3Provider()
51
+
var manifest = Manifest()
52
+
manifest.markBackedUp(
53
+
uuid: "uuid-1",
54
+
s3Key: "originals/2024/01/uuid-1.heic",
55
+
checksum: "sha256:abc"
56
+
)
57
+
58
+
let assets = [makeTestAsset(uuid: "uuid-1")]
59
+
let options = RefreshMetadataOptions(dryRun: true)
60
+
61
+
let report = try await runRefreshMetadata(
62
+
assets: assets, manifest: manifest, s3: s3, options: options
63
+
)
64
+
65
+
#expect(report.skipped == 1)
66
+
#expect(report.updated == 0)
67
+
let objects = try await s3.listObjects(prefix: "metadata/")
68
+
#expect(objects.isEmpty)
69
+
}
70
+
71
+
@Test func emptyAssetsReturnsEmptyReport() async throws {
72
+
let s3 = MockS3Provider()
73
+
let manifest = Manifest()
74
+
75
+
let report = try await runRefreshMetadata(
76
+
assets: [], manifest: manifest, s3: s3
77
+
)
78
+
79
+
#expect(report.updated == 0)
80
+
#expect(report.failed == 0)
81
+
}
82
+
}
+96
Tests/AtticCoreTests/RetryPolicyTests.swift
+96
Tests/AtticCoreTests/RetryPolicyTests.swift
···
1
+
import Testing
2
+
import Foundation
3
+
@testable import AtticCore
4
+
5
+
@Suite("RetryPolicy")
6
+
struct RetryPolicyTests {
7
+
@Test func returnsResultOnFirstSuccess() async throws {
8
+
let result = try await withRetry { 42 }
9
+
#expect(result == 42)
10
+
}
11
+
12
+
@Test func retriesOnTransientErrorThenSucceeds() async throws {
13
+
let counter = Counter()
14
+
let result: String = try await withRetry(baseDelay: .milliseconds(10)) {
15
+
let attempt = await counter.increment()
16
+
if attempt == 1 { throw TransientError("fetch failed") }
17
+
return "ok"
18
+
}
19
+
#expect(result == "ok")
20
+
#expect(await counter.value == 2)
21
+
}
22
+
23
+
@Test func doesNotRetryOnNonTransientError() async {
24
+
let counter = Counter()
25
+
do {
26
+
let _: Int = try await withRetry(baseDelay: .milliseconds(10)) {
27
+
await counter.increment()
28
+
throw NonTransientError("Access denied")
29
+
}
30
+
} catch {
31
+
#expect(error is NonTransientError)
32
+
}
33
+
#expect(await counter.value == 1)
34
+
}
35
+
36
+
@Test func throwsAfterMaxAttempts() async {
37
+
let counter = Counter()
38
+
do {
39
+
let _: Int = try await withRetry(
40
+
maxAttempts: 3,
41
+
baseDelay: .milliseconds(10)
42
+
) {
43
+
await counter.increment()
44
+
throw TransientError("ECONNRESET")
45
+
}
46
+
} catch {
47
+
#expect(error is TransientError)
48
+
}
49
+
#expect(await counter.value == 3)
50
+
}
51
+
52
+
@Test func respectsCancellation() async {
53
+
let counter = Counter()
54
+
let task = Task {
55
+
try await withRetry(
56
+
maxAttempts: 5,
57
+
baseDelay: .milliseconds(100)
58
+
) {
59
+
await counter.increment()
60
+
throw TransientError("timeout")
61
+
} as Int
62
+
}
63
+
64
+
// Give it time to fail once and enter backoff
65
+
try? await Task.sleep(for: .milliseconds(50))
66
+
task.cancel()
67
+
68
+
do {
69
+
_ = try await task.value
70
+
} catch {
71
+
#expect(error is CancellationError)
72
+
}
73
+
#expect(await counter.value == 1)
74
+
}
75
+
}
76
+
77
+
// MARK: - Test helpers
78
+
79
+
private struct TransientError: Error, CustomStringConvertible {
80
+
let description: String
81
+
init(_ description: String) { self.description = description }
82
+
}
83
+
84
+
private struct NonTransientError: Error, CustomStringConvertible {
85
+
let description: String
86
+
init(_ description: String) { self.description = description }
87
+
}
88
+
89
+
private actor Counter {
90
+
var value = 0
91
+
@discardableResult
92
+
func increment() -> Int {
93
+
value += 1
94
+
return value
95
+
}
96
+
}
+216
Tests/AtticCoreTests/S3ManifestStoreTests.swift
+216
Tests/AtticCoreTests/S3ManifestStoreTests.swift
···
1
+
import Testing
2
+
import Foundation
3
+
@testable import AtticCore
4
+
5
+
@Suite("MockS3Provider")
6
+
struct MockS3ProviderTests {
7
+
@Test func putAndGetRoundTrip() async throws {
8
+
let s3 = MockS3Provider()
9
+
let data = Data("hello".utf8)
10
+
try await s3.putObject(key: "test.txt", body: data, contentType: "text/plain")
11
+
let result = try await s3.getObject(key: "test.txt")
12
+
#expect(result == data)
13
+
}
14
+
15
+
@Test func getThrowsOnMissing() async {
16
+
let s3 = MockS3Provider()
17
+
do {
18
+
_ = try await s3.getObject(key: "missing")
19
+
Issue.record("Expected error")
20
+
} catch {
21
+
#expect(String(describing: error).contains("notFound"))
22
+
}
23
+
}
24
+
25
+
@Test func headReturnsNilOnMissing() async throws {
26
+
let s3 = MockS3Provider()
27
+
let meta = try await s3.headObject(key: "missing")
28
+
#expect(meta == nil)
29
+
}
30
+
31
+
@Test func headReturnsMetaForExistingObject() async throws {
32
+
let s3 = MockS3Provider()
33
+
let data = Data("hello world".utf8)
34
+
try await s3.putObject(key: "test.txt", body: data, contentType: "text/plain")
35
+
let meta = try await s3.headObject(key: "test.txt")
36
+
#expect(meta?.contentLength == 11)
37
+
#expect(meta?.contentType == "text/plain")
38
+
}
39
+
40
+
@Test func listObjectsWithPrefix() async throws {
41
+
let s3 = MockS3Provider()
42
+
try await s3.putObject(key: "photos/a.jpg", body: Data("a".utf8))
43
+
try await s3.putObject(key: "photos/b.jpg", body: Data("b".utf8))
44
+
try await s3.putObject(key: "videos/c.mp4", body: Data("c".utf8))
45
+
46
+
let photos = try await s3.listObjects(prefix: "photos/")
47
+
#expect(photos.count == 2)
48
+
#expect(photos.map(\.key).contains("photos/a.jpg"))
49
+
50
+
let videos = try await s3.listObjects(prefix: "videos/")
51
+
#expect(videos.count == 1)
52
+
}
53
+
54
+
@Test func tracksCallCounts() async throws {
55
+
let s3 = MockS3Provider()
56
+
try await s3.putObject(key: "a", body: Data())
57
+
try await s3.putObject(key: "b", body: Data())
58
+
_ = try await s3.getObject(key: "a")
59
+
60
+
#expect(await s3.putCount == 2)
61
+
#expect(await s3.getCount == 1)
62
+
}
63
+
}
64
+
65
+
@Suite("S3ManifestStore")
66
+
struct S3ManifestStoreTests {
67
+
@Test func loadReturnsEmptyWhenKeyMissing() async throws {
68
+
let s3 = MockS3Provider()
69
+
let store = S3ManifestStore(s3: s3)
70
+
let manifest = try await store.load()
71
+
#expect(manifest.entries.isEmpty)
72
+
}
73
+
74
+
@Test func saveAndLoadRoundTrip() async throws {
75
+
let s3 = MockS3Provider()
76
+
let store = S3ManifestStore(s3: s3)
77
+
var manifest = Manifest()
78
+
manifest.markBackedUp(
79
+
uuid: "uuid-1",
80
+
s3Key: "originals/2024/01/uuid-1.heic",
81
+
checksum: "sha256:abc",
82
+
backedUpAt: "2024-01-15T00:00:00Z"
83
+
)
84
+
try await store.save(manifest)
85
+
86
+
let loaded = try await store.load()
87
+
#expect(loaded.isBackedUp("uuid-1"))
88
+
#expect(loaded.entries["uuid-1"]?.s3Key == "originals/2024/01/uuid-1.heic")
89
+
}
90
+
91
+
@Test func savesWithCorrectContentType() async throws {
92
+
let s3 = MockS3Provider()
93
+
let store = S3ManifestStore(s3: s3)
94
+
try await store.save(Manifest())
95
+
96
+
let obj = await s3.objects["manifest.json"]
97
+
#expect(obj?.contentType == "application/json")
98
+
}
99
+
}
100
+
101
+
@Suite("Manifest migration")
102
+
struct ManifestMigrationTests {
103
+
@Test func usesS3ManifestWhenPresent() async throws {
104
+
let s3 = MockS3Provider()
105
+
let store = S3ManifestStore(s3: s3)
106
+
var existing = Manifest()
107
+
existing.markBackedUp(
108
+
uuid: "s3-uuid",
109
+
s3Key: "originals/2024/01/s3.heic",
110
+
checksum: "sha256:s3",
111
+
backedUpAt: "2024-01-15T00:00:00Z"
112
+
)
113
+
try await store.save(existing)
114
+
115
+
let manifest = try await loadManifestWithMigration(
116
+
s3Store: store,
117
+
localDirectory: URL(fileURLWithPath: "/nonexistent")
118
+
)
119
+
#expect(manifest.isBackedUp("s3-uuid"))
120
+
}
121
+
122
+
@Test func migratesLocalManifestToS3() async throws {
123
+
let dir = FileManager.default.temporaryDirectory
124
+
.appendingPathComponent(UUID().uuidString)
125
+
try FileManager.default.createDirectory(at: dir, withIntermediateDirectories: true)
126
+
defer { try? FileManager.default.removeItem(at: dir) }
127
+
128
+
// Write a local manifest
129
+
let localJSON = """
130
+
{
131
+
"entries": {
132
+
"local-uuid": {
133
+
"uuid": "local-uuid",
134
+
"s3Key": "originals/2024/01/local.heic",
135
+
"checksum": "sha256:local",
136
+
"backedUpAt": "2024-01-15T00:00:00Z"
137
+
}
138
+
}
139
+
}
140
+
"""
141
+
try localJSON.write(
142
+
to: dir.appendingPathComponent("manifest.json"),
143
+
atomically: true,
144
+
encoding: .utf8
145
+
)
146
+
147
+
let s3 = MockS3Provider()
148
+
let store = S3ManifestStore(s3: s3)
149
+
150
+
let manifest = try await loadManifestWithMigration(
151
+
s3Store: store,
152
+
localDirectory: dir
153
+
)
154
+
#expect(manifest.isBackedUp("local-uuid"))
155
+
156
+
// Verify it was uploaded to S3
157
+
let s3Manifest = try await store.load()
158
+
#expect(s3Manifest.isBackedUp("local-uuid"))
159
+
}
160
+
161
+
@Test func returnsEmptyWhenNeitherExists() async throws {
162
+
let s3 = MockS3Provider()
163
+
let store = S3ManifestStore(s3: s3)
164
+
let manifest = try await loadManifestWithMigration(
165
+
s3Store: store,
166
+
localDirectory: URL(fileURLWithPath: "/nonexistent")
167
+
)
168
+
#expect(manifest.entries.isEmpty)
169
+
}
170
+
171
+
@Test func s3TakesPrecedenceOverLocal() async throws {
172
+
let dir = FileManager.default.temporaryDirectory
173
+
.appendingPathComponent(UUID().uuidString)
174
+
try FileManager.default.createDirectory(at: dir, withIntermediateDirectories: true)
175
+
defer { try? FileManager.default.removeItem(at: dir) }
176
+
177
+
// Local manifest has one entry
178
+
let localJSON = """
179
+
{
180
+
"entries": {
181
+
"local-uuid": {
182
+
"uuid": "local-uuid",
183
+
"s3Key": "originals/2024/01/local.heic",
184
+
"checksum": "sha256:local",
185
+
"backedUpAt": "2024-01-15T00:00:00Z"
186
+
}
187
+
}
188
+
}
189
+
"""
190
+
try localJSON.write(
191
+
to: dir.appendingPathComponent("manifest.json"),
192
+
atomically: true,
193
+
encoding: .utf8
194
+
)
195
+
196
+
// S3 has a different entry
197
+
let s3 = MockS3Provider()
198
+
let store = S3ManifestStore(s3: s3)
199
+
var s3Manifest = Manifest()
200
+
s3Manifest.markBackedUp(
201
+
uuid: "s3-uuid",
202
+
s3Key: "originals/2024/01/s3.heic",
203
+
checksum: "sha256:s3",
204
+
backedUpAt: "2024-01-15T00:00:00Z"
205
+
)
206
+
try await store.save(s3Manifest)
207
+
208
+
// S3 should win — local is not consulted
209
+
let manifest = try await loadManifestWithMigration(
210
+
s3Store: store,
211
+
localDirectory: dir
212
+
)
213
+
#expect(manifest.isBackedUp("s3-uuid"))
214
+
#expect(!manifest.isBackedUp("local-uuid"))
215
+
}
216
+
}
+65
Tests/AtticCoreTests/S3PathsTests.swift
+65
Tests/AtticCoreTests/S3PathsTests.swift
···
1
+
import Testing
2
+
import Foundation
3
+
@testable import AtticCore
4
+
5
+
@Suite("S3Paths")
6
+
struct S3PathsTests {
7
+
@Test func originalKeyGeneratesCorrectPath() throws {
8
+
let date = ISO8601DateFormatter().date(from: "2024-01-15T12:00:00Z")!
9
+
let key = try S3Paths.originalKey(uuid: "abc-uuid", dateCreated: date, extension: "heic")
10
+
#expect(key == "originals/2024/01/abc-uuid.heic")
11
+
}
12
+
13
+
@Test func originalKeyHandlesNilDate() throws {
14
+
let key = try S3Paths.originalKey(uuid: "abc-uuid", dateCreated: nil, extension: "jpg")
15
+
#expect(key == "originals/unknown/00/abc-uuid.jpg")
16
+
}
17
+
18
+
@Test func originalKeyStripsLeadingDot() throws {
19
+
let date = ISO8601DateFormatter().date(from: "2024-03-01T00:00:00Z")!
20
+
let key = try S3Paths.originalKey(uuid: "x", dateCreated: date, extension: ".HEIC")
21
+
#expect(key == "originals/2024/03/x.heic")
22
+
}
23
+
24
+
@Test func originalKeyRejectsUnsafeUUID() {
25
+
let date = ISO8601DateFormatter().date(from: "2024-01-15T12:00:00Z")!
26
+
#expect(throws: S3PathError.self) {
27
+
try S3Paths.originalKey(uuid: "../../../etc", dateCreated: date, extension: "heic")
28
+
}
29
+
#expect(throws: S3PathError.self) {
30
+
try S3Paths.originalKey(uuid: "uuid/with/slashes", dateCreated: date, extension: "heic")
31
+
}
32
+
}
33
+
34
+
@Test func originalKeyRejectsUnsafeExtension() {
35
+
let date = ISO8601DateFormatter().date(from: "2024-01-15T12:00:00Z")!
36
+
#expect(throws: S3PathError.self) {
37
+
try S3Paths.originalKey(uuid: "abc", dateCreated: date, extension: "h/e")
38
+
}
39
+
}
40
+
41
+
@Test func metadataKeyGeneratesCorrectPath() throws {
42
+
let key = try S3Paths.metadataKey(uuid: "abc-uuid")
43
+
#expect(key == "metadata/assets/abc-uuid.json")
44
+
}
45
+
46
+
@Test func metadataKeyRejectsUnsafeUUID() {
47
+
#expect(throws: S3PathError.self) {
48
+
try S3Paths.metadataKey(uuid: "../escape")
49
+
}
50
+
}
51
+
52
+
@Test func extensionFromUTIMapsKnownTypes() {
53
+
#expect(S3Paths.extensionFromUTIOrFilename(uti: "public.heic", filename: "IMG.HEIC") == "heic")
54
+
#expect(S3Paths.extensionFromUTIOrFilename(uti: "public.jpeg", filename: "IMG.JPG") == "jpg")
55
+
#expect(S3Paths.extensionFromUTIOrFilename(uti: "com.apple.quicktime-movie", filename: "VID.MOV") == "mov")
56
+
}
57
+
58
+
@Test func extensionFromUTIFallsBackToFilename() {
59
+
#expect(S3Paths.extensionFromUTIOrFilename(uti: "unknown.uti", filename: "photo.webp") == "webp")
60
+
}
61
+
62
+
@Test func extensionFromUTIReturnsBinAsLastResort() {
63
+
#expect(S3Paths.extensionFromUTIOrFilename(uti: nil, filename: "noext") == "bin")
64
+
}
65
+
}
+87
Tests/AtticCoreTests/VerifyPipelineTests.swift
+87
Tests/AtticCoreTests/VerifyPipelineTests.swift
···
1
+
import Testing
2
+
import Foundation
3
+
@testable import AtticCore
4
+
5
+
@Suite("VerifyPipeline")
6
+
struct VerifyPipelineTests {
7
+
private func makeManifest(entries: [(uuid: String, s3Key: String, checksum: String)]) -> Manifest {
8
+
var manifest = Manifest()
9
+
for e in entries {
10
+
manifest.markBackedUp(uuid: e.uuid, s3Key: e.s3Key, checksum: e.checksum)
11
+
}
12
+
return manifest
13
+
}
14
+
15
+
@Test func verifyReportsOKForExistingObjects() async throws {
16
+
let s3 = MockS3Provider()
17
+
try await s3.putObject(key: "originals/2024/01/uuid-1.heic", body: Data("photo".utf8))
18
+
try await s3.putObject(key: "originals/2024/01/uuid-2.heic", body: Data("photo2".utf8))
19
+
20
+
let manifest = makeManifest(entries: [
21
+
(uuid: "uuid-1", s3Key: "originals/2024/01/uuid-1.heic", checksum: "sha256:abc"),
22
+
(uuid: "uuid-2", s3Key: "originals/2024/01/uuid-2.heic", checksum: "sha256:def"),
23
+
])
24
+
25
+
let report = try await runVerify(manifest: manifest, s3: s3)
26
+
27
+
#expect(report.ok == 2)
28
+
#expect(report.missing == 0)
29
+
#expect(report.failed == 0)
30
+
}
31
+
32
+
@Test func verifyDetectsMissingObjects() async throws {
33
+
let s3 = MockS3Provider()
34
+
// Only put one of the two objects
35
+
try await s3.putObject(key: "originals/2024/01/uuid-1.heic", body: Data("photo".utf8))
36
+
37
+
let manifest = makeManifest(entries: [
38
+
(uuid: "uuid-1", s3Key: "originals/2024/01/uuid-1.heic", checksum: "sha256:abc"),
39
+
(uuid: "uuid-2", s3Key: "originals/2024/01/uuid-2.heic", checksum: "sha256:def"),
40
+
])
41
+
42
+
let report = try await runVerify(manifest: manifest, s3: s3)
43
+
44
+
#expect(report.ok == 1)
45
+
#expect(report.missing == 1)
46
+
}
47
+
48
+
@Test func emptyManifestReturnsEmptyReport() async throws {
49
+
let s3 = MockS3Provider()
50
+
let manifest = Manifest()
51
+
52
+
let report = try await runVerify(manifest: manifest, s3: s3)
53
+
54
+
#expect(report.ok == 0)
55
+
#expect(report.missing == 0)
56
+
#expect(report.failed == 0)
57
+
}
58
+
59
+
@Test func verifyReportsErrorsForS3Failures() async throws {
60
+
let s3 = FailingS3Provider()
61
+
62
+
var manifest = Manifest()
63
+
manifest.markBackedUp(uuid: "uuid-1", s3Key: "originals/2024/01/uuid-1.heic", checksum: "sha256:abc")
64
+
65
+
let report = try await runVerify(manifest: manifest, s3: s3)
66
+
67
+
#expect(report.ok == 0)
68
+
#expect(report.missing == 0)
69
+
#expect(report.failed == 1)
70
+
#expect(report.errors.count == 1)
71
+
#expect(report.errors.first?.uuid == "uuid-1")
72
+
}
73
+
}
74
+
75
+
/// S3 provider that throws on headObject to test the error path.
76
+
private actor FailingS3Provider: S3Providing {
77
+
func putObject(key: String, body: Data, contentType: String?) async throws {}
78
+
func getObject(key: String) async throws -> Data { Data() }
79
+
func headObject(key: String) async throws -> S3ObjectMeta? {
80
+
throw FailingS3Error.networkError
81
+
}
82
+
func listObjects(prefix: String) async throws -> [S3ListObject] { [] }
83
+
}
84
+
85
+
private enum FailingS3Error: Error {
86
+
case networkError
87
+
}
-16
cli/deno.json
-16
cli/deno.json
···
1
-
{
2
-
"name": "@attic/cli",
3
-
"version": "0.1.0",
4
-
"exports": "./mod.ts",
5
-
"imports": {
6
-
"@attic/shared": "../shared/mod.ts",
7
-
"@aws-sdk/client-s3": "npm:@aws-sdk/client-s3@^3",
8
-
"@cliffy/command": "jsr:@cliffy/command@^1.0",
9
-
"@cliffy/prompt": "jsr:@cliffy/prompt@^1.0",
10
-
"@cliffy/ansi": "jsr:@cliffy/ansi@^1.0",
11
-
"@db/sqlite": "jsr:@db/sqlite@^0.12",
12
-
"@std/assert": "jsr:@std/assert@^1.0",
13
-
"@std/crypto": "jsr:@std/crypto@^1.0",
14
-
"@std/path": "jsr:@std/path@^1.0"
15
-
}
16
-
}
-343
cli/mod.ts
-343
cli/mod.ts
···
1
-
import { Command, EnumType } from "@cliffy/command";
2
-
import { type AtticConfig, requireConfig } from "./src/config/config.ts";
3
-
import type { S3ConnectionConfig } from "./src/storage/s3-client.ts";
4
-
5
-
const assetType = new EnumType(["photo", "video"]);
6
-
7
-
function s3ConnectionFromConfig(config: AtticConfig): S3ConnectionConfig {
8
-
return {
9
-
endpoint: config.endpoint,
10
-
region: config.region,
11
-
pathStyle: config.pathStyle,
12
-
};
13
-
}
14
-
15
-
const main = new Command()
16
-
.name("attic")
17
-
.version("0.2.6")
18
-
.description("Back up your iCloud Photos library to S3-compatible storage")
19
-
.action(function (this: Command) {
20
-
this.showHelp();
21
-
});
22
-
23
-
main.command("scan", "Scan Photos library and show statistics")
24
-
.option("--db <path:string>", "Path to Photos.sqlite")
25
-
.action(async ({ db }: { db?: string }) => {
26
-
const { openPhotosDb } = await import("./src/photos-db/reader.ts");
27
-
const { printScanReport } = await import("./src/commands/scan.ts");
28
-
29
-
const reader = openPhotosDb(db);
30
-
try {
31
-
const assets = reader.readAssets();
32
-
printScanReport(assets);
33
-
} finally {
34
-
reader.close();
35
-
}
36
-
});
37
-
38
-
main.command("status", "Compare Photos DB vs backup manifest")
39
-
.option("--db <path:string>", "Path to Photos.sqlite")
40
-
.option("--bucket <name:string>", "Override bucket from config")
41
-
.action(async ({ db, bucket }: { db?: string; bucket?: string }) => {
42
-
const { openPhotosDb } = await import("./src/photos-db/reader.ts");
43
-
const { printStatusReport } = await import("./src/commands/status.ts");
44
-
const { createS3ManifestStore } = await import(
45
-
"./src/manifest/manifest.ts"
46
-
);
47
-
const { createS3Provider } = await import("./src/storage/s3-client.ts");
48
-
const { loadKeychainCredentials } = await import(
49
-
"./src/keychain/keychain.ts"
50
-
);
51
-
52
-
const config = requireConfig();
53
-
const reader = openPhotosDb(db);
54
-
const credentials = await loadKeychainCredentials(
55
-
config.keychain.accessKeyService,
56
-
config.keychain.secretKeyService,
57
-
);
58
-
const s3 = createS3Provider(
59
-
credentials,
60
-
bucket ?? config.bucket,
61
-
s3ConnectionFromConfig(config),
62
-
);
63
-
try {
64
-
const assets = reader.readAssets();
65
-
const manifestStore = createS3ManifestStore(s3);
66
-
const manifest = await manifestStore.load();
67
-
printStatusReport(assets, manifest);
68
-
} finally {
69
-
s3.destroy();
70
-
reader.close();
71
-
}
72
-
});
73
-
74
-
main.command("backup", "Back up pending assets to S3")
75
-
.option("--dry-run", "Show what would be uploaded")
76
-
.option("--limit <n:integer>", "Stop after N assets (useful for test runs)")
77
-
.option("--batch-size <n:integer>", "Assets per export batch", {
78
-
default: 50,
79
-
})
80
-
.type("asset-type", assetType)
81
-
.option("--type <type:asset-type>", "Only back up photos or videos")
82
-
.option("--bucket <name:string>", "Override bucket from config")
83
-
.option("--ladder <path:string>", "Path to ladder binary")
84
-
.option("--db <path:string>", "Path to Photos.sqlite")
85
-
.option("-q, --quiet", "Suppress progress output (for unattended use)")
86
-
.option("--log <path:string>", "Append structured JSONL log to file")
87
-
.option("--notify", "Send macOS notification on completion")
88
-
.action(async (options: {
89
-
dryRun?: boolean;
90
-
limit?: number;
91
-
batchSize: number;
92
-
type?: "photo" | "video";
93
-
bucket?: string;
94
-
ladder?: string;
95
-
db?: string;
96
-
quiet?: boolean;
97
-
log?: string;
98
-
notify?: boolean;
99
-
}) => {
100
-
const { openPhotosDb } = await import("./src/photos-db/reader.ts");
101
-
const { runBackup } = await import("./src/commands/backup.ts");
102
-
const {
103
-
createS3ManifestStore,
104
-
loadManifestWithMigration,
105
-
} = await import("./src/manifest/manifest.ts");
106
-
const { createS3Provider } = await import("./src/storage/s3-client.ts");
107
-
const { loadKeychainCredentials } = await import(
108
-
"./src/keychain/keychain.ts"
109
-
);
110
-
const { createLadderExporter } = await import("./src/export/exporter.ts");
111
-
const { createFileLogger, createNullLogger } = await import(
112
-
"./src/logger.ts"
113
-
);
114
-
115
-
const config = requireConfig();
116
-
const reader = openPhotosDb(options.db);
117
-
const logger = options.log
118
-
? createFileLogger(options.log)
119
-
: createNullLogger();
120
-
const credentials = await loadKeychainCredentials(
121
-
config.keychain.accessKeyService,
122
-
config.keychain.secretKeyService,
123
-
);
124
-
const s3 = createS3Provider(
125
-
credentials,
126
-
options.bucket ?? config.bucket,
127
-
s3ConnectionFromConfig(config),
128
-
);
129
-
130
-
try {
131
-
const assets = reader.readAssets();
132
-
133
-
const manifestStore = createS3ManifestStore(s3);
134
-
const manifest = await loadManifestWithMigration(manifestStore);
135
-
136
-
const ladderPath = options.ladder ??
137
-
Deno.env.get("LADDER_PATH") ??
138
-
"ladder";
139
-
140
-
const exporter = createLadderExporter(ladderPath);
141
-
142
-
await runBackup(assets, manifest, manifestStore, exporter, s3, {
143
-
batchSize: options.batchSize,
144
-
limit: options.limit ?? 0,
145
-
type: options.type ?? null,
146
-
dryRun: options.dryRun ?? false,
147
-
quiet: options.quiet ?? false,
148
-
logger,
149
-
notifyOnComplete: options.notify ?? false,
150
-
});
151
-
} finally {
152
-
s3.destroy();
153
-
logger.close();
154
-
reader.close();
155
-
}
156
-
});
157
-
158
-
main
159
-
.command("refresh-metadata", "Re-upload metadata JSON for backed-up assets")
160
-
.option("--dry-run", "Show what would be uploaded")
161
-
.option("--concurrency <n:integer>", "Concurrent uploads", { default: 20 })
162
-
.option("--bucket <name:string>", "Override bucket from config")
163
-
.option("--db <path:string>", "Path to Photos.sqlite")
164
-
.action(async (options: {
165
-
dryRun?: boolean;
166
-
concurrency: number;
167
-
bucket?: string;
168
-
db?: string;
169
-
}) => {
170
-
const { openPhotosDb } = await import("./src/photos-db/reader.ts");
171
-
const { refreshMetadata } = await import(
172
-
"./src/commands/refresh-metadata.ts"
173
-
);
174
-
const { createS3ManifestStore } = await import(
175
-
"./src/manifest/manifest.ts"
176
-
);
177
-
const { createS3Provider } = await import("./src/storage/s3-client.ts");
178
-
const { loadKeychainCredentials } = await import(
179
-
"./src/keychain/keychain.ts"
180
-
);
181
-
182
-
const config = requireConfig();
183
-
const reader = openPhotosDb(options.db);
184
-
const credentials = await loadKeychainCredentials(
185
-
config.keychain.accessKeyService,
186
-
config.keychain.secretKeyService,
187
-
);
188
-
const s3 = createS3Provider(
189
-
credentials,
190
-
options.bucket ?? config.bucket,
191
-
s3ConnectionFromConfig(config),
192
-
);
193
-
try {
194
-
const assets = reader.readAssets();
195
-
196
-
const manifestStore = createS3ManifestStore(s3);
197
-
const manifest = await manifestStore.load();
198
-
199
-
const report = await refreshMetadata(assets, manifest, s3, {
200
-
concurrency: options.concurrency,
201
-
dryRun: options.dryRun ?? false,
202
-
});
203
-
if (report.failed > 0) Deno.exit(2);
204
-
} finally {
205
-
s3.destroy();
206
-
reader.close();
207
-
}
208
-
});
209
-
210
-
main.command("init", "Set up attic configuration")
211
-
.action(async () => {
212
-
const { runInit } = await import("./src/commands/init.ts");
213
-
await runInit();
214
-
});
215
-
216
-
main.command("verify", "Verify backup integrity against S3")
217
-
.option("--deep", "Download and re-checksum each object")
218
-
.option("--rebuild-manifest", "Reconstruct manifest from S3 metadata")
219
-
.option("--bucket <name:string>", "Override bucket from config")
220
-
.action(async (options: {
221
-
deep?: boolean;
222
-
rebuildManifest?: boolean;
223
-
bucket?: string;
224
-
}) => {
225
-
const { runVerify } = await import("./src/commands/verify.ts");
226
-
const { rebuildManifest } = await import("./src/commands/rebuild.ts");
227
-
const { createS3ManifestStore } = await import(
228
-
"./src/manifest/manifest.ts"
229
-
);
230
-
const { createS3Provider } = await import("./src/storage/s3-client.ts");
231
-
const { loadKeychainCredentials } = await import(
232
-
"./src/keychain/keychain.ts"
233
-
);
234
-
235
-
const config = requireConfig();
236
-
237
-
const credentials = await loadKeychainCredentials(
238
-
config.keychain.accessKeyService,
239
-
config.keychain.secretKeyService,
240
-
);
241
-
const s3 = createS3Provider(
242
-
credentials,
243
-
options.bucket ?? config.bucket,
244
-
s3ConnectionFromConfig(config),
245
-
);
246
-
const manifestStore = createS3ManifestStore(s3);
247
-
248
-
try {
249
-
if (options.rebuildManifest) {
250
-
await rebuildManifest(s3, manifestStore);
251
-
} else {
252
-
const manifest = await manifestStore.load();
253
-
await runVerify(manifest, s3, {
254
-
deep: options.deep ?? false,
255
-
});
256
-
}
257
-
} finally {
258
-
s3.destroy();
259
-
}
260
-
});
261
-
262
-
try {
263
-
await main.parse(Deno.args);
264
-
} catch (error: unknown) {
265
-
handleError(error);
266
-
Deno.exit(1);
267
-
}
268
-
269
-
function handleError(error: unknown): void {
270
-
if (!(error instanceof Error)) {
271
-
console.error("An unexpected error occurred.");
272
-
return;
273
-
}
274
-
275
-
const msg = error.message;
276
-
277
-
// Keychain not found
278
-
if (
279
-
msg.includes("find-generic-password") ||
280
-
msg.includes("SecKeychainSearchCopyNext")
281
-
) {
282
-
console.error("Could not read credentials from macOS Keychain.");
283
-
console.error('Run "attic init" to set up your credentials.\n');
284
-
return;
285
-
}
286
-
287
-
// Config missing (thrown by requireConfig)
288
-
if (msg.includes("No config file found")) {
289
-
console.error(msg);
290
-
return;
291
-
}
292
-
293
-
// Config validation error
294
-
if (msg.startsWith("Config:")) {
295
-
console.error(msg);
296
-
console.error(
297
-
'Run "attic init" to reconfigure, or edit ~/.attic/config.json.\n',
298
-
);
299
-
return;
300
-
}
301
-
302
-
// S3 access denied
303
-
if (msg.includes("AccessDenied") || msg.includes("403")) {
304
-
console.error(
305
-
"S3 access denied. Check your credentials and bucket permissions.",
306
-
);
307
-
console.error('Run "attic init" to update credentials.\n');
308
-
return;
309
-
}
310
-
311
-
// S3 bucket not found
312
-
if (msg.includes("NoSuchBucket")) {
313
-
console.error(
314
-
"S3 bucket not found. Check the bucket name in ~/.attic/config.json.",
315
-
);
316
-
return;
317
-
}
318
-
319
-
// Network error
320
-
if (
321
-
msg.includes("ECONNREFUSED") || msg.includes("ETIMEDOUT") ||
322
-
msg.includes("fetch failed")
323
-
) {
324
-
console.error(
325
-
"Could not connect to S3 endpoint. Check your network and endpoint URL in ~/.attic/config.json.",
326
-
);
327
-
return;
328
-
}
329
-
330
-
// Photos.sqlite not found
331
-
if (
332
-
msg.includes("Photos.sqlite") || msg.includes("unable to open database")
333
-
) {
334
-
console.error("Could not open Photos database.");
335
-
console.error(
336
-
"Make sure Photos is set up on this Mac and the database exists.\n",
337
-
);
338
-
return;
339
-
}
340
-
341
-
// Fallback
342
-
console.error(`Error: ${msg}`);
343
-
}
-7
cli/src/abort-error.ts
-7
cli/src/abort-error.ts
-399
cli/src/commands/backup.test.ts
-399
cli/src/commands/backup.test.ts
···
1
-
import { assertEquals } from "@std/assert";
2
-
import type { PhotoAsset } from "@attic/shared";
3
-
import { AssetKind, CloudLocalState } from "@attic/shared";
4
-
import { runBackup } from "./backup.ts";
5
-
import { createS3ManifestStore, isBackedUp } from "../manifest/manifest.ts";
6
-
import { createMockExporter } from "../export/exporter.mock.ts";
7
-
import { createMockS3Provider } from "../storage/s3-client.mock.ts";
8
-
import {
9
-
type ExportBatchResult,
10
-
type Exporter,
11
-
LadderTimeoutError,
12
-
} from "../export/exporter.ts";
13
-
14
-
function makeAsset(
15
-
uuid: string,
16
-
overrides: Partial<PhotoAsset> = {},
17
-
): PhotoAsset {
18
-
return {
19
-
uuid,
20
-
filename: "IMG_0001.HEIC",
21
-
originalFilename: "IMG_0001.HEIC",
22
-
directory: "/some/dir",
23
-
dateCreated: new Date("2024-01-15T12:00:00Z"),
24
-
kind: AssetKind.PHOTO,
25
-
uniformTypeIdentifier: "public.heic",
26
-
width: 4032,
27
-
height: 3024,
28
-
latitude: 52.09,
29
-
longitude: 4.34,
30
-
favorite: false,
31
-
cloudLocalState: CloudLocalState.LOCAL,
32
-
originalFileSize: 3000,
33
-
originalStableHash: "abc123",
34
-
title: null,
35
-
description: null,
36
-
albums: [],
37
-
keywords: [],
38
-
people: [],
39
-
hasEdit: false,
40
-
editedAt: null,
41
-
editor: null,
42
-
...overrides,
43
-
};
44
-
}
45
-
46
-
function createTestContext() {
47
-
const s3 = createMockS3Provider();
48
-
const manifestStore = createS3ManifestStore(s3);
49
-
return { s3, manifestStore };
50
-
}
51
-
52
-
Deno.test("backup: uploads pending assets and updates manifest", async () => {
53
-
const tmpDir = await Deno.makeTempDir();
54
-
const stagingDir = `${tmpDir}/staging`;
55
-
try {
56
-
const assets = [makeAsset("uuid-1"), makeAsset("uuid-2")];
57
-
58
-
const mockAssets = new Map([
59
-
[
60
-
"uuid-1",
61
-
{ filename: "IMG_0001.HEIC", data: new TextEncoder().encode("photo1") },
62
-
],
63
-
[
64
-
"uuid-2",
65
-
{ filename: "IMG_0002.HEIC", data: new TextEncoder().encode("photo2") },
66
-
],
67
-
]);
68
-
69
-
const exporter = createMockExporter(mockAssets, stagingDir);
70
-
const { s3, manifestStore } = createTestContext();
71
-
const manifest = await manifestStore.load();
72
-
73
-
const report = await runBackup(
74
-
assets,
75
-
manifest,
76
-
manifestStore,
77
-
exporter,
78
-
s3,
79
-
{ batchSize: 10 },
80
-
stagingDir,
81
-
);
82
-
83
-
assertEquals(report.uploaded, 2);
84
-
assertEquals(report.failed, 0);
85
-
assertEquals(report.errors.length, 0);
86
-
87
-
// Manifest should have both entries
88
-
assertEquals(isBackedUp(manifest, "uuid-1"), true);
89
-
assertEquals(isBackedUp(manifest, "uuid-2"), true);
90
-
91
-
// S3 should have originals + metadata + manifest
92
-
// 2 originals + 2 metadata + 1 manifest.json
93
-
assertEquals(s3.objects.has("manifest.json"), true);
94
-
} finally {
95
-
await Deno.remove(tmpDir, { recursive: true });
96
-
}
97
-
});
98
-
99
-
Deno.test("backup: skips already backed-up assets", async () => {
100
-
const tmpDir = await Deno.makeTempDir();
101
-
const stagingDir = `${tmpDir}/staging`;
102
-
try {
103
-
const assets = [makeAsset("uuid-1"), makeAsset("uuid-2")];
104
-
105
-
const mockAssets = new Map([
106
-
[
107
-
"uuid-2",
108
-
{ filename: "IMG_0002.HEIC", data: new TextEncoder().encode("photo2") },
109
-
],
110
-
]);
111
-
112
-
const exporter = createMockExporter(mockAssets, stagingDir);
113
-
const { s3, manifestStore } = createTestContext();
114
-
const manifest = await manifestStore.load();
115
-
116
-
// Pre-mark uuid-1 as backed up
117
-
manifest.entries["uuid-1"] = {
118
-
uuid: "uuid-1",
119
-
s3Key: "originals/2024/01/uuid-1.heic",
120
-
checksum: "sha256:abc",
121
-
backedUpAt: "2024-01-15T00:00:00Z",
122
-
};
123
-
124
-
const report = await runBackup(
125
-
assets,
126
-
manifest,
127
-
manifestStore,
128
-
exporter,
129
-
s3,
130
-
{ batchSize: 10 },
131
-
stagingDir,
132
-
);
133
-
134
-
assertEquals(report.uploaded, 1);
135
-
assertEquals(report.failed, 0);
136
-
137
-
// Only uuid-2's files should be in S3 (plus manifest)
138
-
assertEquals(s3.objects.has("manifest.json"), true);
139
-
} finally {
140
-
await Deno.remove(tmpDir, { recursive: true });
141
-
}
142
-
});
143
-
144
-
Deno.test("backup: respects --limit flag", async () => {
145
-
const tmpDir = await Deno.makeTempDir();
146
-
const stagingDir = `${tmpDir}/staging`;
147
-
try {
148
-
const assets = [
149
-
makeAsset("uuid-1"),
150
-
makeAsset("uuid-2"),
151
-
makeAsset("uuid-3"),
152
-
];
153
-
154
-
const mockAssets = new Map([
155
-
[
156
-
"uuid-1",
157
-
{ filename: "IMG_0001.HEIC", data: new TextEncoder().encode("p1") },
158
-
],
159
-
]);
160
-
161
-
const exporter = createMockExporter(mockAssets, stagingDir);
162
-
const { s3, manifestStore } = createTestContext();
163
-
const manifest = await manifestStore.load();
164
-
165
-
const report = await runBackup(
166
-
assets,
167
-
manifest,
168
-
manifestStore,
169
-
exporter,
170
-
s3,
171
-
{ limit: 1, batchSize: 10 },
172
-
stagingDir,
173
-
);
174
-
175
-
assertEquals(report.uploaded, 1);
176
-
} finally {
177
-
await Deno.remove(tmpDir, { recursive: true });
178
-
}
179
-
});
180
-
181
-
Deno.test("backup: dry run skips uploads", async () => {
182
-
const tmpDir = await Deno.makeTempDir();
183
-
const stagingDir = `${tmpDir}/staging`;
184
-
try {
185
-
const assets = [makeAsset("uuid-1")];
186
-
187
-
const exporter = createMockExporter(new Map(), stagingDir);
188
-
const { s3, manifestStore } = createTestContext();
189
-
const manifest = await manifestStore.load();
190
-
191
-
const report = await runBackup(
192
-
assets,
193
-
manifest,
194
-
manifestStore,
195
-
exporter,
196
-
s3,
197
-
{ dryRun: true },
198
-
stagingDir,
199
-
);
200
-
201
-
assertEquals(report.uploaded, 0);
202
-
assertEquals(report.skipped, 1);
203
-
assertEquals(s3.objects.size, 0);
204
-
assertEquals(isBackedUp(manifest, "uuid-1"), false);
205
-
} finally {
206
-
await Deno.remove(tmpDir, { recursive: true });
207
-
}
208
-
});
209
-
210
-
Deno.test("backup: handles export errors gracefully", async () => {
211
-
const tmpDir = await Deno.makeTempDir();
212
-
const stagingDir = `${tmpDir}/staging`;
213
-
try {
214
-
const assets = [makeAsset("uuid-1"), makeAsset("uuid-missing")];
215
-
216
-
// Only uuid-1 is known to the exporter
217
-
const mockAssets = new Map([
218
-
[
219
-
"uuid-1",
220
-
{ filename: "IMG_0001.HEIC", data: new TextEncoder().encode("data") },
221
-
],
222
-
]);
223
-
224
-
const exporter = createMockExporter(mockAssets, stagingDir);
225
-
const { s3, manifestStore } = createTestContext();
226
-
const manifest = await manifestStore.load();
227
-
228
-
const report = await runBackup(
229
-
assets,
230
-
manifest,
231
-
manifestStore,
232
-
exporter,
233
-
s3,
234
-
{ batchSize: 10 },
235
-
stagingDir,
236
-
);
237
-
238
-
assertEquals(report.uploaded, 1);
239
-
assertEquals(report.failed, 1);
240
-
assertEquals(report.errors.length, 1);
241
-
assertEquals(report.errors[0].uuid, "uuid-missing");
242
-
} finally {
243
-
await Deno.remove(tmpDir, { recursive: true });
244
-
}
245
-
});
246
-
247
-
Deno.test("backup: filters by type", async () => {
248
-
const tmpDir = await Deno.makeTempDir();
249
-
const stagingDir = `${tmpDir}/staging`;
250
-
try {
251
-
const assets = [
252
-
makeAsset("photo-1", { kind: AssetKind.PHOTO }),
253
-
makeAsset("video-1", {
254
-
kind: AssetKind.VIDEO,
255
-
uniformTypeIdentifier: "com.apple.quicktime-movie",
256
-
filename: "VID.MOV",
257
-
originalFilename: "VID.MOV",
258
-
}),
259
-
];
260
-
261
-
const mockAssets = new Map([
262
-
[
263
-
"video-1",
264
-
{ filename: "VID.MOV", data: new TextEncoder().encode("video") },
265
-
],
266
-
]);
267
-
268
-
const exporter = createMockExporter(mockAssets, stagingDir);
269
-
const { s3, manifestStore } = createTestContext();
270
-
const manifest = await manifestStore.load();
271
-
272
-
const report = await runBackup(
273
-
assets,
274
-
manifest,
275
-
manifestStore,
276
-
exporter,
277
-
s3,
278
-
{ type: "video", batchSize: 10 },
279
-
stagingDir,
280
-
);
281
-
282
-
assertEquals(report.uploaded, 1);
283
-
assertEquals(isBackedUp(manifest, "video-1"), true);
284
-
assertEquals(isBackedUp(manifest, "photo-1"), false);
285
-
} finally {
286
-
await Deno.remove(tmpDir, { recursive: true });
287
-
}
288
-
});
289
-
290
-
Deno.test("backup: defers timed-out assets and retries after remaining batches", async () => {
291
-
const tmpDir = await Deno.makeTempDir();
292
-
const stagingDir = `${tmpDir}/staging`;
293
-
try {
294
-
const assets = [
295
-
makeAsset("fast-1"),
296
-
makeAsset("slow-1"),
297
-
makeAsset("fast-2"),
298
-
];
299
-
300
-
const mockData = new Map([
301
-
[
302
-
"fast-1",
303
-
{ filename: "IMG_0001.HEIC", data: new TextEncoder().encode("f1") },
304
-
],
305
-
[
306
-
"slow-1",
307
-
{ filename: "BIG_VIDEO.MOV", data: new TextEncoder().encode("s1") },
308
-
],
309
-
[
310
-
"fast-2",
311
-
{ filename: "IMG_0003.HEIC", data: new TextEncoder().encode("f2") },
312
-
],
313
-
]);
314
-
315
-
// Exporter that times out on batch calls containing "slow-1",
316
-
// but succeeds when "slow-1" is called individually (deferred retry)
317
-
let slowRetryCount = 0;
318
-
const innerExporter = createMockExporter(mockData, stagingDir);
319
-
const timeoutExporter: Exporter & { stagingDir: string } = {
320
-
stagingDir,
321
-
exportBatch(
322
-
uuids: string[],
323
-
signal?: AbortSignal,
324
-
): Promise<ExportBatchResult> {
325
-
if (uuids.includes("slow-1") && uuids.length > 1) {
326
-
// Batch with slow asset: timeout
327
-
throw new LadderTimeoutError(300_000);
328
-
}
329
-
if (uuids.length === 1 && uuids[0] === "slow-1") {
330
-
slowRetryCount++;
331
-
if (slowRetryCount === 1) {
332
-
// First individual retry: also times out (gets deferred)
333
-
throw new LadderTimeoutError(300_000);
334
-
}
335
-
// Second retry (deferred): succeeds
336
-
}
337
-
return innerExporter.exportBatch(uuids, signal);
338
-
},
339
-
};
340
-
341
-
const { s3, manifestStore } = createTestContext();
342
-
const manifest = await manifestStore.load();
343
-
344
-
const report = await runBackup(
345
-
assets,
346
-
manifest,
347
-
manifestStore,
348
-
timeoutExporter,
349
-
s3,
350
-
{ batchSize: 3, quiet: true },
351
-
stagingDir,
352
-
);
353
-
354
-
// All 3 should be uploaded: fast-1 and fast-2 via individual retry,
355
-
// slow-1 via deferred retry
356
-
assertEquals(report.uploaded, 3);
357
-
assertEquals(report.failed, 0);
358
-
assertEquals(isBackedUp(manifest, "fast-1"), true);
359
-
assertEquals(isBackedUp(manifest, "fast-2"), true);
360
-
assertEquals(isBackedUp(manifest, "slow-1"), true);
361
-
} finally {
362
-
await Deno.remove(tmpDir, { recursive: true });
363
-
}
364
-
});
365
-
366
-
Deno.test("backup: saves manifest to S3", async () => {
367
-
const tmpDir = await Deno.makeTempDir();
368
-
const stagingDir = `${tmpDir}/staging`;
369
-
try {
370
-
const assets = [makeAsset("uuid-1")];
371
-
372
-
const mockAssets = new Map([
373
-
[
374
-
"uuid-1",
375
-
{ filename: "IMG_0001.HEIC", data: new TextEncoder().encode("data") },
376
-
],
377
-
]);
378
-
379
-
const exporter = createMockExporter(mockAssets, stagingDir);
380
-
const { s3, manifestStore } = createTestContext();
381
-
const manifest = await manifestStore.load();
382
-
383
-
await runBackup(
384
-
assets,
385
-
manifest,
386
-
manifestStore,
387
-
exporter,
388
-
s3,
389
-
{ batchSize: 10 },
390
-
stagingDir,
391
-
);
392
-
393
-
// Load from S3 — should persist
394
-
const loaded = await manifestStore.load();
395
-
assertEquals(isBackedUp(loaded, "uuid-1"), true);
396
-
} finally {
397
-
await Deno.remove(tmpDir, { recursive: true });
398
-
}
399
-
});
-537
cli/src/commands/backup.ts
-537
cli/src/commands/backup.ts
···
1
-
import type { PhotoAsset } from "@attic/shared";
2
-
import {
3
-
AssetKind,
4
-
buildMetadataJson,
5
-
CloudLocalState,
6
-
extensionFromUtiOrFilename,
7
-
metadataKey,
8
-
originalKey,
9
-
} from "@attic/shared";
10
-
import type { Manifest, ManifestStore } from "../manifest/manifest.ts";
11
-
import { isBackedUp, markBackedUp } from "../manifest/manifest.ts";
12
-
import type { ExportBatchResult, Exporter } from "../export/exporter.ts";
13
-
import {
14
-
isPermissionError,
15
-
isTimeoutError,
16
-
removeStagedFile,
17
-
} from "../export/exporter.ts";
18
-
import type { S3Provider } from "../storage/s3-client.ts";
19
-
import { formatBytes } from "../format.ts";
20
-
import { startSpinner } from "../spinner.ts";
21
-
import type { BackupLogger } from "../logger.ts";
22
-
import { createNullLogger } from "../logger.ts";
23
-
import { notify } from "../notify.ts";
24
-
import { withRetry } from "../retry.ts";
25
-
26
-
export interface BackupOptions {
27
-
/** Maximum assets per ladder batch. */
28
-
batchSize: number;
29
-
/** Stop after this many assets total (0 = unlimited). */
30
-
limit: number;
31
-
/** Only process assets of this type (null = all). */
32
-
type: "photo" | "video" | null;
33
-
/** Skip uploads, just show what would happen. */
34
-
dryRun: boolean;
35
-
/** Save manifest every N assets. */
36
-
saveInterval: number;
37
-
/** Suppress progress output (for unattended/scripted use). */
38
-
quiet: boolean;
39
-
/** Structured JSONL logger (null logger if --log not given). */
40
-
logger: BackupLogger;
41
-
/** Send macOS notification on completion. */
42
-
notifyOnComplete: boolean;
43
-
}
44
-
45
-
const DEFAULT_OPTIONS: BackupOptions = {
46
-
batchSize: 50,
47
-
limit: 0,
48
-
type: null,
49
-
dryRun: false,
50
-
saveInterval: 50,
51
-
quiet: false,
52
-
logger: createNullLogger(),
53
-
notifyOnComplete: false,
54
-
};
55
-
56
-
export interface BackupReport {
57
-
uploaded: number;
58
-
failed: number;
59
-
skipped: number;
60
-
totalBytes: number;
61
-
errors: Array<{ uuid: string; message: string }>;
62
-
}
63
-
64
-
/** Run the backup pipeline: scan -> filter -> export -> upload -> manifest. */
65
-
export async function runBackup(
66
-
assets: PhotoAsset[],
67
-
manifest: Manifest,
68
-
manifestStore: ManifestStore,
69
-
exporter: Exporter,
70
-
s3: S3Provider,
71
-
opts: Partial<BackupOptions> = {},
72
-
stagingDir?: string,
73
-
): Promise<BackupReport> {
74
-
const options = { ...DEFAULT_OPTIONS, ...opts };
75
-
const log = options.quiet ? () => {} : console.log.bind(console);
76
-
const logger = options.logger;
77
-
78
-
// Pre-flight: verify ladder has required permissions before doing any work
79
-
if (exporter.checkPermissions) {
80
-
try {
81
-
await exporter.checkPermissions();
82
-
} catch (error: unknown) {
83
-
if (isPermissionError(error)) {
84
-
const msg = error instanceof Error ? error.message : String(error);
85
-
console.error(`\n ${msg}`);
86
-
console.error(
87
-
`\n Fix the permission issue and run \`attic backup\` again.\n`,
88
-
);
89
-
return {
90
-
uploaded: 0,
91
-
failed: assets.length,
92
-
skipped: 0,
93
-
totalBytes: 0,
94
-
errors: [],
95
-
};
96
-
}
97
-
// Non-permission errors from the probe are non-fatal — ladder may
98
-
// not be installed yet, or Photos may not be set up. Let the normal
99
-
// batch flow surface these errors with full context.
100
-
}
101
-
}
102
-
103
-
// Filter to pending assets
104
-
let pending = assets.filter((a) => !isBackedUp(manifest, a.uuid));
105
-
106
-
// Filter by type if requested
107
-
if (options.type === "photo") {
108
-
pending = pending.filter((a) => a.kind === AssetKind.PHOTO);
109
-
} else if (options.type === "video") {
110
-
pending = pending.filter((a) => a.kind === AssetKind.VIDEO);
111
-
}
112
-
113
-
// Apply limit
114
-
if (options.limit > 0) {
115
-
pending = pending.slice(0, options.limit);
116
-
}
117
-
118
-
if (pending.length === 0) {
119
-
log("\n Nothing to back up — all assets are current.\n");
120
-
return { uploaded: 0, failed: 0, skipped: 0, totalBytes: 0, errors: [] };
121
-
}
122
-
123
-
const pendingSize = pending.reduce(
124
-
(sum, a) => sum + (a.originalFileSize ?? 0),
125
-
0,
126
-
);
127
-
128
-
const photoCount = pending.filter((a) => a.kind === AssetKind.PHOTO).length;
129
-
const videoCount = pending.filter((a) => a.kind === AssetKind.VIDEO).length;
130
-
const typeSummary = [
131
-
photoCount > 0 ? `${photoCount} photos` : "",
132
-
videoCount > 0 ? `${videoCount} videos` : "",
133
-
].filter(Boolean).join(", ");
134
-
135
-
log(`\n Attic — Backup`);
136
-
log(` ══════════════\n`);
137
-
log(
138
-
` Pending: ${pending.length.toLocaleString()} assets (${typeSummary})`,
139
-
);
140
-
log(` Est. size: ${formatBytes(pendingSize)}`);
141
-
if (options.dryRun) log(` Mode: DRY RUN`);
142
-
log();
143
-
144
-
logger.start(pending.length, photoCount, videoCount);
145
-
146
-
if (options.dryRun) {
147
-
return {
148
-
uploaded: 0,
149
-
failed: 0,
150
-
skipped: pending.length,
151
-
totalBytes: 0,
152
-
errors: [],
153
-
};
154
-
}
155
-
156
-
// Build UUID-to-asset lookup for metadata
157
-
const assetByUuid = new Map<string, PhotoAsset>();
158
-
for (const a of pending) {
159
-
assetByUuid.set(a.uuid, a);
160
-
}
161
-
162
-
// Resolve staging directory for safe file cleanup
163
-
const resolvedStagingDir = stagingDir ??
164
-
(exporter as { stagingDir?: string }).stagingDir ??
165
-
`${Deno.env.get("HOME") ?? "~"}/.attic/staging`;
166
-
167
-
const report: BackupReport = {
168
-
uploaded: 0,
169
-
failed: 0,
170
-
skipped: 0,
171
-
totalBytes: 0,
172
-
errors: [],
173
-
};
174
-
175
-
let sinceLastSave = 0;
176
-
177
-
// AbortController for cancelling in-flight operations (subprocess, S3 uploads)
178
-
const abortController = new AbortController();
179
-
const { signal } = abortController;
180
-
181
-
let interruptCount = 0;
182
-
const onInterrupt = () => {
183
-
interruptCount++;
184
-
185
-
if (interruptCount === 1) {
186
-
// First Ctrl+C: graceful — cancel in-flight operations, save manifest
187
-
abortController.abort();
188
-
manifestStore.save(manifest).catch(() => {});
189
-
} else {
190
-
// Second Ctrl+C: force exit. Manifest was last saved to S3
191
-
// at most saveInterval assets ago. Any unsaved progress will
192
-
// be re-uploaded on the next run (uploads are idempotent).
193
-
console.error("\n Force quit — progress saved up to last checkpoint.");
194
-
Deno.exit(130);
195
-
}
196
-
};
197
-
Deno.addSignalListener("SIGINT", onInterrupt);
198
-
199
-
// Helper: format asset name for log messages
200
-
function assetLabel(uuid: string): string {
201
-
const a = assetByUuid.get(uuid);
202
-
if (!a) return uuid.substring(0, 8);
203
-
const name = a.originalFilename ?? a.filename ?? uuid.substring(0, 8);
204
-
const size = a.originalFileSize ? formatBytes(a.originalFileSize) : "?";
205
-
const type = a.kind === AssetKind.PHOTO ? "photo" : "video";
206
-
return `${name} (${type}, ${size})`;
207
-
}
208
-
209
-
// Helper: upload exported assets to S3, update manifest and report
210
-
async function uploadExported(
211
-
batchResult: ExportBatchResult,
212
-
): Promise<void> {
213
-
// Record export errors with asset context
214
-
for (const err of batchResult.errors) {
215
-
const asset = assetByUuid.get(err.uuid);
216
-
const detail = asset
217
-
? exportErrorDetail(asset, err.message)
218
-
: err.message;
219
-
console.error(` Export error: ${assetLabel(err.uuid)} — ${detail}`);
220
-
report.errors.push(err);
221
-
report.failed++;
222
-
logger.error(err.uuid, detail);
223
-
}
224
-
225
-
for (const exported of batchResult.results) {
226
-
if (signal.aborted) break;
227
-
const asset = assetByUuid.get(exported.uuid);
228
-
if (!asset) continue;
229
-
230
-
const ext = extensionFromUtiOrFilename(
231
-
asset.uniformTypeIdentifier,
232
-
asset.originalFilename ?? asset.filename,
233
-
);
234
-
const s3Key = originalKey(asset.uuid, asset.dateCreated, ext);
235
-
236
-
try {
237
-
let fileData: Uint8Array | null = await Deno.readFile(exported.path);
238
-
await withRetry(
239
-
() => s3.putObject(s3Key, fileData!, contentTypeFor(ext)),
240
-
{ signal },
241
-
);
242
-
fileData = null;
243
-
244
-
const meta = buildMetadataJson(
245
-
asset,
246
-
s3Key,
247
-
`sha256:${exported.sha256}`,
248
-
new Date().toISOString(),
249
-
);
250
-
const metaData = new TextEncoder().encode(
251
-
JSON.stringify(meta, null, 2),
252
-
);
253
-
await withRetry(
254
-
() =>
255
-
s3.putObject(
256
-
metadataKey(asset.uuid),
257
-
metaData,
258
-
"application/json",
259
-
),
260
-
{ signal },
261
-
);
262
-
263
-
markBackedUp(
264
-
manifest,
265
-
asset.uuid,
266
-
`sha256:${exported.sha256}`,
267
-
s3Key,
268
-
exported.size,
269
-
);
270
-
sinceLastSave++;
271
-
report.uploaded++;
272
-
report.totalBytes += exported.size;
273
-
274
-
const name = asset.originalFilename ?? asset.filename ?? "unknown";
275
-
const typeLabel = asset.kind === AssetKind.PHOTO ? "photo" : "video";
276
-
logger.uploaded(asset.uuid, name, typeLabel, exported.size);
277
-
278
-
if (!options.quiet) {
279
-
const done = report.uploaded + report.failed;
280
-
const pct = ((done / pending.length) * 100).toFixed(0);
281
-
log(
282
-
` [${done}/${pending.length}] ${pct}% ${name} (${typeLabel}, ${
283
-
formatBytes(exported.size)
284
-
})`,
285
-
);
286
-
}
287
-
288
-
if (sinceLastSave >= options.saveInterval) {
289
-
await manifestStore.save(manifest);
290
-
sinceLastSave = 0;
291
-
}
292
-
} catch (error: unknown) {
293
-
if (signal.aborted) break;
294
-
const msg = error instanceof Error ? error.message : String(error);
295
-
console.error(` Upload error: ${exported.uuid} — ${msg}`);
296
-
report.errors.push({ uuid: exported.uuid, message: msg });
297
-
report.failed++;
298
-
logger.error(exported.uuid, msg);
299
-
} finally {
300
-
await removeStagedFile(exported.path, resolvedStagingDir);
301
-
}
302
-
}
303
-
}
304
-
305
-
// Assets deferred due to individual timeout — retried after all batches
306
-
const deferred: string[] = [];
307
-
308
-
// Process in batches
309
-
for (let i = 0; i < pending.length; i += options.batchSize) {
310
-
if (signal.aborted) break;
311
-
312
-
const batch = pending.slice(i, i + options.batchSize);
313
-
const batchUuids = batch.map((a) => a.uuid);
314
-
const batchNum = Math.floor(i / options.batchSize) + 1;
315
-
const totalBatches = Math.ceil(pending.length / options.batchSize);
316
-
317
-
const batchPhotos = batch.filter((a) => a.kind === AssetKind.PHOTO).length;
318
-
const batchVideos = batch.length - batchPhotos;
319
-
const batchBytes = batch.reduce(
320
-
(sum, a) => sum + (a.originalFileSize ?? 0),
321
-
0,
322
-
);
323
-
324
-
if (totalBatches > 1) {
325
-
const parts = [
326
-
batchPhotos > 0 ? `${batchPhotos} photos` : "",
327
-
batchVideos > 0 ? `${batchVideos} videos` : "",
328
-
].filter(Boolean).join(", ");
329
-
log(
330
-
` Batch ${batchNum}/${totalBatches} (${parts}, ~${
331
-
formatBytes(batchBytes)
332
-
})`,
333
-
);
334
-
}
335
-
336
-
exporter.setEstimatedBatchBytes?.(batchBytes);
337
-
338
-
// 1. Export via ladder
339
-
const spinner = options.quiet
340
-
? { stop() {} }
341
-
: startSpinner(`Exporting ${batch.length} assets from Photos library...`);
342
-
let batchResult: ExportBatchResult;
343
-
try {
344
-
batchResult = await exporter.exportBatch(batchUuids, signal);
345
-
} catch (error: unknown) {
346
-
spinner.stop();
347
-
if (signal.aborted) break;
348
-
349
-
// On timeout: retry each asset individually to find the slow ones
350
-
if (isTimeoutError(error)) {
351
-
log(
352
-
` Batch timed out — retrying ${batch.length} assets individually...`,
353
-
);
354
-
const combined: ExportBatchResult = { results: [], errors: [] };
355
-
356
-
for (const uuid of batchUuids) {
357
-
if (signal.aborted) break;
358
-
const assetBytes = assetByUuid.get(uuid)?.originalFileSize ?? 0;
359
-
exporter.setEstimatedBatchBytes?.(assetBytes);
360
-
try {
361
-
const result = await exporter.exportBatch([uuid], signal);
362
-
combined.results.push(...result.results);
363
-
combined.errors.push(...result.errors);
364
-
} catch (innerError: unknown) {
365
-
if (signal.aborted) break;
366
-
if (isTimeoutError(innerError)) {
367
-
log(
368
-
` Deferring ${
369
-
assetLabel(uuid)
370
-
} — timed out, will retry after remaining batches`,
371
-
);
372
-
deferred.push(uuid);
373
-
} else {
374
-
const msg = innerError instanceof Error
375
-
? innerError.message
376
-
: String(innerError);
377
-
report.errors.push({ uuid, message: msg });
378
-
report.failed++;
379
-
logger.error(uuid, msg);
380
-
}
381
-
}
382
-
}
383
-
384
-
// Upload whatever succeeded from individual retries
385
-
await uploadExported(combined);
386
-
} else if (isPermissionError(error)) {
387
-
// Permission error: abort all remaining batches — retrying won't help
388
-
const msg = error instanceof Error ? error.message : String(error);
389
-
console.error(`\n ${msg}`);
390
-
console.error(
391
-
`\n Fix the permission issue and run \`attic backup\` again.\n`,
392
-
);
393
-
for (const uuid of batchUuids) {
394
-
report.errors.push({ uuid, message: msg });
395
-
report.failed++;
396
-
logger.error(uuid, msg);
397
-
}
398
-
// Mark all remaining assets as failed too
399
-
for (
400
-
const uuid of pending.slice(i + options.batchSize).map((a) => a.uuid)
401
-
) {
402
-
report.errors.push({ uuid, message: msg });
403
-
report.failed++;
404
-
}
405
-
break;
406
-
} else {
407
-
// Non-timeout error: fail the whole batch
408
-
const msg = error instanceof Error ? error.message : String(error);
409
-
console.error(` Export failed: ${msg}`);
410
-
for (const uuid of batchUuids) {
411
-
report.errors.push({ uuid, message: msg });
412
-
report.failed++;
413
-
logger.error(uuid, msg);
414
-
}
415
-
}
416
-
if (totalBatches > 1 && batchNum < totalBatches) log();
417
-
continue;
418
-
}
419
-
spinner.stop();
420
-
421
-
// 2. Upload exported assets
422
-
await uploadExported(batchResult);
423
-
424
-
// Batch separator when multiple batches
425
-
if (totalBatches > 1 && batchNum < totalBatches) {
426
-
log();
427
-
}
428
-
}
429
-
430
-
// Retry deferred assets
431
-
if (deferred.length > 0 && !signal.aborted) {
432
-
log();
433
-
log(` Retrying ${deferred.length} deferred assets...`);
434
-
for (const uuid of deferred) {
435
-
if (signal.aborted) break;
436
-
const assetBytes = assetByUuid.get(uuid)?.originalFileSize ?? 0;
437
-
exporter.setEstimatedBatchBytes?.(assetBytes);
438
-
try {
439
-
const result = await exporter.exportBatch([uuid], signal);
440
-
await uploadExported(result);
441
-
} catch (retryError: unknown) {
442
-
if (signal.aborted) break;
443
-
const msg = retryError instanceof Error
444
-
? retryError.message
445
-
: String(retryError);
446
-
log(` Failed: ${assetLabel(uuid)} — ${msg}`);
447
-
report.errors.push({ uuid, message: msg });
448
-
report.failed++;
449
-
logger.error(uuid, msg);
450
-
}
451
-
}
452
-
}
453
-
454
-
// Final save
455
-
if (sinceLastSave > 0) {
456
-
await manifestStore.save(manifest);
457
-
}
458
-
459
-
// Summary
460
-
Deno.removeSignalListener("SIGINT", onInterrupt);
461
-
462
-
if (signal.aborted) {
463
-
log(`\n\n ── Interrupted ──`);
464
-
log(
465
-
` Uploaded: ${report.uploaded.toLocaleString()} of ${pending.length.toLocaleString()}`,
466
-
);
467
-
log(` Total: ${formatBytes(report.totalBytes)}`);
468
-
log(` Manifest saved — will resume from here next run.\n`);
469
-
logger.interrupted(report.uploaded, pending.length, report.totalBytes);
470
-
} else {
471
-
log(`\n ── Complete ──`);
472
-
log(` Uploaded: ${report.uploaded.toLocaleString()}`);
473
-
log(` Failed: ${report.failed.toLocaleString()}`);
474
-
log(` Total: ${formatBytes(report.totalBytes)}`);
475
-
if (report.failed > 0) {
476
-
log(`\n Run \`attic backup\` again to retry failed assets.`);
477
-
}
478
-
log();
479
-
logger.complete(report.uploaded, report.failed, report.totalBytes);
480
-
}
481
-
482
-
// macOS notification
483
-
if (options.notifyOnComplete) {
484
-
if (report.failed > 0) {
485
-
await notify(
486
-
"Attic Backup",
487
-
`Done with errors: ${report.uploaded} uploaded, ${report.failed} failed`,
488
-
"Basso",
489
-
);
490
-
} else if (signal.aborted) {
491
-
await notify(
492
-
"Attic Backup",
493
-
`Interrupted: ${report.uploaded} of ${pending.length} uploaded`,
494
-
"Basso",
495
-
);
496
-
} else {
497
-
await notify(
498
-
"Attic Backup",
499
-
`Complete: ${report.uploaded} assets (${
500
-
formatBytes(report.totalBytes)
501
-
})`,
502
-
);
503
-
}
504
-
}
505
-
506
-
return report;
507
-
}
508
-
509
-
/** Add context to a ladder export error based on what we know about the asset. */
510
-
function exportErrorDetail(asset: PhotoAsset, message: string): string {
511
-
const hints: string[] = [];
512
-
if (asset.cloudLocalState === CloudLocalState.ICLOUD_ONLY) {
513
-
hints.push("iCloud-only asset");
514
-
}
515
-
if (!asset.originalFileSize) {
516
-
hints.push("no original file size recorded");
517
-
}
518
-
if (hints.length === 0) return message;
519
-
return `${message} — ${hints.join(", ")}`;
520
-
}
521
-
522
-
function contentTypeFor(ext: string): string {
523
-
const map: Record<string, string> = {
524
-
jpg: "image/jpeg",
525
-
jpeg: "image/jpeg",
526
-
heic: "image/heic",
527
-
png: "image/png",
528
-
tiff: "image/tiff",
529
-
gif: "image/gif",
530
-
mp4: "video/mp4",
531
-
mov: "video/quicktime",
532
-
m4v: "video/x-m4v",
533
-
avi: "video/x-msvideo",
534
-
orf: "image/x-olympus-orf",
535
-
};
536
-
return map[ext] ?? "application/octet-stream";
537
-
}
-122
cli/src/commands/init.ts
-122
cli/src/commands/init.ts
···
1
-
import { Confirm, Input, Secret } from "@cliffy/prompt";
2
-
import {
3
-
type AtticConfig,
4
-
configPath,
5
-
loadConfig,
6
-
writeConfig,
7
-
} from "../config/config.ts";
8
-
import { storeKeychainCredential } from "../keychain/keychain.ts";
9
-
10
-
const BUCKET_PATTERN = /^[a-z0-9][a-z0-9.\-]{1,61}[a-z0-9]$/;
11
-
12
-
const EU_PROVIDER_EXAMPLES = [
13
-
" Scaleway (EU): https://s3.fr-par.scw.cloud",
14
-
" Hetzner (EU): https://fsn1.your-objectstorage.com",
15
-
" OVH (EU): https://s3.gra.io.cloud.ovh.net",
16
-
" AWS: https://s3.eu-west-1.amazonaws.com",
17
-
];
18
-
19
-
export async function runInit(): Promise<void> {
20
-
console.log("\n attic — iCloud Photos backup to S3-compatible storage\n");
21
-
22
-
const existing = loadConfig();
23
-
if (existing) {
24
-
const overwrite = await Confirm.prompt({
25
-
message: `Config already exists at ${configPath()}. Overwrite?`,
26
-
default: false,
27
-
});
28
-
if (!overwrite) {
29
-
console.log(" Cancelled.\n");
30
-
return;
31
-
}
32
-
}
33
-
34
-
console.log(" S3 Connection");
35
-
console.log(" " + "─".repeat(40) + "\n");
36
-
console.log(" Provider examples:");
37
-
for (const line of EU_PROVIDER_EXAMPLES) {
38
-
console.log(line);
39
-
}
40
-
console.log();
41
-
42
-
const endpoint = await Input.prompt({
43
-
message: "Endpoint URL",
44
-
validate: (v) => {
45
-
if (!v.startsWith("https://")) return "Must start with https://";
46
-
return true;
47
-
},
48
-
});
49
-
50
-
const region = await Input.prompt({
51
-
message: "Region",
52
-
hint: "e.g. fr-par, eu-central-1, fsn1",
53
-
validate: (v) => {
54
-
if (v.trim() === "") return "Region is required";
55
-
return true;
56
-
},
57
-
});
58
-
59
-
const bucket = await Input.prompt({
60
-
message: "Bucket name",
61
-
validate: (v) => {
62
-
if (v.trim() === "") return "Bucket name is required";
63
-
if (!BUCKET_PATTERN.test(v)) {
64
-
return "Use lowercase letters, numbers, dots, and hyphens (3-63 chars)";
65
-
}
66
-
return true;
67
-
},
68
-
});
69
-
70
-
const pathStyle = await Confirm.prompt({
71
-
message: "Use path-style URLs? (most S3-compatible providers need this)",
72
-
default: true,
73
-
});
74
-
75
-
console.log("\n Credentials");
76
-
console.log(" " + "─".repeat(40) + "\n");
77
-
78
-
const accessKey = await Input.prompt({
79
-
message: "Access key",
80
-
});
81
-
82
-
const secretKey = await Secret.prompt({
83
-
message: "Secret key",
84
-
});
85
-
86
-
const config: AtticConfig = {
87
-
endpoint,
88
-
region,
89
-
bucket,
90
-
pathStyle,
91
-
keychain: {
92
-
accessKeyService: "attic-s3-access-key",
93
-
secretKeyService: "attic-s3-secret-key",
94
-
},
95
-
};
96
-
97
-
// Write config file
98
-
console.log(`\n Writing config to ${configPath()}...`);
99
-
writeConfig(config);
100
-
console.log(" Done.");
101
-
102
-
// Store credentials in Keychain (-U flag: update if exists, create if not)
103
-
console.log(" Storing credentials in macOS Keychain...");
104
-
await storeKeychainCredential(
105
-
config.keychain.accessKeyService,
106
-
accessKey,
107
-
);
108
-
await storeKeychainCredential(
109
-
config.keychain.secretKeyService,
110
-
secretKey,
111
-
);
112
-
console.log(" Done.");
113
-
114
-
console.log("\n Setup complete.\n");
115
-
console.log(" Required permissions (System Settings > Privacy & Security):");
116
-
console.log(" - Photos: grant access to ladder");
117
-
console.log(" - Full Disk Access: enable for attic and ladder");
118
-
console.log(
119
-
" - Automation: grant ladder access to Photos (for iCloud-only assets)",
120
-
);
121
-
console.log('\n Run "attic scan" to see your Photos library.\n');
122
-
}
-130
cli/src/commands/rebuild.test.ts
-130
cli/src/commands/rebuild.test.ts
···
1
-
import { assertEquals } from "@std/assert";
2
-
import { rebuildManifest } from "./rebuild.ts";
3
-
import { createS3ManifestStore, isBackedUp } from "../manifest/manifest.ts";
4
-
import { createMockS3Provider } from "../storage/s3-client.mock.ts";
5
-
6
-
Deno.test("rebuildManifest: reconstructs from S3 metadata", async () => {
7
-
const s3 = createMockS3Provider();
8
-
9
-
const meta1 = {
10
-
uuid: "uuid-1",
11
-
s3Key: "originals/2024/01/uuid-1.heic",
12
-
checksum:
13
-
"sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
14
-
backedUpAt: "2024-01-15T10:00:00Z",
15
-
};
16
-
const meta2 = {
17
-
uuid: "uuid-2",
18
-
s3Key: "originals/2024/02/uuid-2.jpg",
19
-
checksum:
20
-
"sha256:bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb",
21
-
backedUpAt: "2024-02-20T14:00:00Z",
22
-
};
23
-
24
-
await s3.putObject(
25
-
"metadata/assets/uuid-1.json",
26
-
new TextEncoder().encode(JSON.stringify(meta1)),
27
-
"application/json",
28
-
);
29
-
await s3.putObject(
30
-
"metadata/assets/uuid-2.json",
31
-
new TextEncoder().encode(JSON.stringify(meta2)),
32
-
"application/json",
33
-
);
34
-
35
-
const manifestStore = createS3ManifestStore(s3);
36
-
const rebuilt = await rebuildManifest(s3, manifestStore);
37
-
38
-
assertEquals(isBackedUp(rebuilt, "uuid-1"), true);
39
-
assertEquals(isBackedUp(rebuilt, "uuid-2"), true);
40
-
assertEquals(
41
-
rebuilt.entries["uuid-1"].s3Key,
42
-
"originals/2024/01/uuid-1.heic",
43
-
);
44
-
assertEquals(
45
-
rebuilt.entries["uuid-1"].checksum,
46
-
"sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
47
-
);
48
-
assertEquals(rebuilt.entries["uuid-1"].backedUpAt, "2024-01-15T10:00:00Z");
49
-
assertEquals(rebuilt.entries["uuid-2"].backedUpAt, "2024-02-20T14:00:00Z");
50
-
51
-
// Verify it was saved to S3
52
-
const loaded = await manifestStore.load();
53
-
assertEquals(isBackedUp(loaded, "uuid-1"), true);
54
-
assertEquals(isBackedUp(loaded, "uuid-2"), true);
55
-
});
56
-
57
-
Deno.test("rebuildManifest: skips invalid metadata files", async () => {
58
-
const s3 = createMockS3Provider();
59
-
60
-
// Valid metadata
61
-
await s3.putObject(
62
-
"metadata/assets/uuid-1.json",
63
-
new TextEncoder().encode(
64
-
JSON.stringify({
65
-
uuid: "uuid-1",
66
-
s3Key: "originals/2024/01/uuid-1.heic",
67
-
checksum:
68
-
"sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
69
-
}),
70
-
),
71
-
"application/json",
72
-
);
73
-
74
-
// Invalid metadata (missing required fields)
75
-
await s3.putObject(
76
-
"metadata/assets/bad.json",
77
-
new TextEncoder().encode('{"foo": "bar"}'),
78
-
"application/json",
79
-
);
80
-
81
-
// Not JSON at all
82
-
await s3.putObject(
83
-
"metadata/assets/garbage.json",
84
-
new TextEncoder().encode("not json {{{"),
85
-
"application/json",
86
-
);
87
-
88
-
const manifestStore = createS3ManifestStore(s3);
89
-
const rebuilt = await rebuildManifest(s3, manifestStore);
90
-
91
-
assertEquals(Object.keys(rebuilt.entries).length, 1);
92
-
assertEquals(isBackedUp(rebuilt, "uuid-1"), true);
93
-
});
94
-
95
-
Deno.test("rebuildManifest: rejects path-traversal s3Keys", async () => {
96
-
const s3 = createMockS3Provider();
97
-
98
-
// s3Key with path traversal
99
-
await s3.putObject(
100
-
"metadata/assets/evil.json",
101
-
new TextEncoder().encode(
102
-
JSON.stringify({
103
-
uuid: "uuid-evil",
104
-
s3Key: "../../etc/passwd",
105
-
checksum:
106
-
"sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
107
-
}),
108
-
),
109
-
"application/json",
110
-
);
111
-
112
-
// Invalid checksum format
113
-
await s3.putObject(
114
-
"metadata/assets/bad-checksum.json",
115
-
new TextEncoder().encode(
116
-
JSON.stringify({
117
-
uuid: "uuid-bad",
118
-
s3Key: "originals/2024/01/uuid-bad.heic",
119
-
checksum: "md5:abc123",
120
-
}),
121
-
),
122
-
"application/json",
123
-
);
124
-
125
-
const manifestStore = createS3ManifestStore(s3);
126
-
const rebuilt = await rebuildManifest(s3, manifestStore);
127
-
128
-
// Neither should be accepted
129
-
assertEquals(Object.keys(rebuilt.entries).length, 0);
130
-
});
-88
cli/src/commands/rebuild.ts
-88
cli/src/commands/rebuild.ts
···
1
-
import type {
2
-
Manifest,
3
-
ManifestEntry,
4
-
ManifestStore,
5
-
} from "../manifest/manifest.ts";
6
-
import { markBackedUp } from "../manifest/manifest.ts";
7
-
import type { S3Provider } from "../storage/s3-client.ts";
8
-
9
-
const UUID_PATTERN = /^[A-Za-z0-9._-]+$/;
10
-
const S3_KEY_PATTERN = /^originals\/\d{4}\/\d{2}\/[A-Za-z0-9._-]+\.[a-z0-9]+$/;
11
-
const CHECKSUM_PATTERN = /^sha256:[a-f0-9]{64}$/;
12
-
13
-
/** Rebuild manifest from S3 metadata JSON files. */
14
-
export async function rebuildManifest(
15
-
s3: S3Provider,
16
-
manifestStore: ManifestStore,
17
-
): Promise<Manifest> {
18
-
console.log(`\n Attic — Rebuild Manifest`);
19
-
console.log(` ═══════════════════════\n`);
20
-
console.log(` Scanning S3 for metadata files...\n`);
21
-
22
-
const manifest: Manifest = { entries: {} };
23
-
let count = 0;
24
-
25
-
for await (const obj of s3.listObjects("metadata/assets/")) {
26
-
if (!obj.key.endsWith(".json")) continue;
27
-
28
-
try {
29
-
const data = await s3.getObject(obj.key);
30
-
const text = new TextDecoder().decode(data);
31
-
const meta: unknown = JSON.parse(text);
32
-
const entry = parseMetadataToEntry(meta);
33
-
if (entry) {
34
-
markBackedUp(
35
-
manifest,
36
-
entry.uuid,
37
-
entry.checksum,
38
-
entry.s3Key,
39
-
entry.size,
40
-
entry.backedUpAt,
41
-
);
42
-
count++;
43
-
}
44
-
} catch {
45
-
console.error(` Warning: failed to parse ${obj.key}`);
46
-
}
47
-
48
-
if (count % 100 === 0 && count > 0) {
49
-
console.log(` Recovered ${count} entries...`);
50
-
}
51
-
}
52
-
53
-
await manifestStore.save(manifest);
54
-
55
-
console.log(`\n Rebuilt manifest with ${count.toLocaleString()} entries.\n`);
56
-
57
-
return manifest;
58
-
}
59
-
60
-
/** Extract a ManifestEntry from an S3 metadata JSON object. Validates format. */
61
-
function parseMetadataToEntry(
62
-
data: unknown,
63
-
): ManifestEntry | null {
64
-
if (typeof data !== "object" || data === null) return null;
65
-
const obj = data as Record<string, unknown>;
66
-
if (
67
-
typeof obj.uuid !== "string" ||
68
-
typeof obj.s3Key !== "string" ||
69
-
typeof obj.checksum !== "string"
70
-
) {
71
-
return null;
72
-
}
73
-
74
-
// Validate field formats
75
-
if (!UUID_PATTERN.test(obj.uuid)) return null;
76
-
if (!S3_KEY_PATTERN.test(obj.s3Key)) return null;
77
-
if (!CHECKSUM_PATTERN.test(obj.checksum)) return null;
78
-
79
-
return {
80
-
uuid: obj.uuid,
81
-
s3Key: obj.s3Key,
82
-
checksum: obj.checksum,
83
-
backedUpAt: typeof obj.backedUpAt === "string"
84
-
? obj.backedUpAt
85
-
: new Date().toISOString(),
86
-
size: typeof obj.fileSize === "number" ? obj.fileSize : undefined,
87
-
};
88
-
}
-167
cli/src/commands/refresh-metadata.test.ts
-167
cli/src/commands/refresh-metadata.test.ts
···
1
-
import { assertEquals, assertExists } from "@std/assert";
2
-
import type { PhotoAsset } from "@attic/shared";
3
-
import { AssetKind, CloudLocalState } from "@attic/shared";
4
-
import { refreshMetadata } from "./refresh-metadata.ts";
5
-
import type { Manifest } from "../manifest/manifest.ts";
6
-
import { createMockS3Provider } from "../storage/s3-client.mock.ts";
7
-
8
-
function makeAsset(
9
-
uuid: string,
10
-
overrides: Partial<PhotoAsset> = {},
11
-
): PhotoAsset {
12
-
return {
13
-
uuid,
14
-
filename: "IMG_0001.HEIC",
15
-
originalFilename: "IMG_0001.HEIC",
16
-
directory: "/some/dir",
17
-
dateCreated: new Date("2024-01-15T12:00:00Z"),
18
-
kind: AssetKind.PHOTO,
19
-
uniformTypeIdentifier: "public.heic",
20
-
width: 4032,
21
-
height: 3024,
22
-
latitude: 52.09,
23
-
longitude: 4.34,
24
-
favorite: false,
25
-
cloudLocalState: CloudLocalState.LOCAL,
26
-
originalFileSize: 3000,
27
-
originalStableHash: "abc123",
28
-
title: null,
29
-
description: null,
30
-
albums: [],
31
-
keywords: [],
32
-
people: [],
33
-
hasEdit: false,
34
-
editedAt: null,
35
-
editor: null,
36
-
...overrides,
37
-
};
38
-
}
39
-
40
-
function makeManifest(
41
-
entries: Record<
42
-
string,
43
-
{ s3Key: string; checksum: string; backedUpAt: string }
44
-
>,
45
-
): Manifest {
46
-
const manifest: Manifest = { entries: {} };
47
-
for (const [uuid, e] of Object.entries(entries)) {
48
-
manifest.entries[uuid] = { uuid, ...e };
49
-
}
50
-
return manifest;
51
-
}
52
-
53
-
Deno.test("refresh-metadata: updates metadata for backed-up assets", async () => {
54
-
const assets = [
55
-
makeAsset("uuid-1", {
56
-
title: "Sunset",
57
-
description: "Beautiful sunset",
58
-
albums: [{ uuid: "album-1", title: "Vacation" }],
59
-
keywords: ["nature"],
60
-
people: [{ uuid: "person-1", displayName: "Alice" }],
61
-
}),
62
-
makeAsset("uuid-2"),
63
-
];
64
-
65
-
const manifest = makeManifest({
66
-
"uuid-1": {
67
-
s3Key: "originals/2024/01/uuid-1.heic",
68
-
checksum: "sha256:abc",
69
-
backedUpAt: "2024-02-01T00:00:00Z",
70
-
},
71
-
"uuid-2": {
72
-
s3Key: "originals/2024/01/uuid-2.heic",
73
-
checksum: "sha256:def",
74
-
backedUpAt: "2024-02-01T00:00:00Z",
75
-
},
76
-
});
77
-
78
-
const s3 = createMockS3Provider();
79
-
const report = await refreshMetadata(assets, manifest, s3);
80
-
81
-
assertEquals(report.updated, 2);
82
-
assertEquals(report.failed, 0);
83
-
84
-
// Verify metadata was uploaded
85
-
const meta1Obj = s3.objects.get("metadata/assets/uuid-1.json");
86
-
assertExists(meta1Obj);
87
-
const meta1 = JSON.parse(new TextDecoder().decode(meta1Obj.body));
88
-
assertEquals(meta1.title, "Sunset");
89
-
assertEquals(meta1.description, "Beautiful sunset");
90
-
assertEquals(meta1.albums, [{ uuid: "album-1", title: "Vacation" }]);
91
-
assertEquals(meta1.keywords, ["nature"]);
92
-
assertEquals(meta1.people, [{ uuid: "person-1", displayName: "Alice" }]);
93
-
assertEquals(meta1.s3Key, "originals/2024/01/uuid-1.heic");
94
-
assertEquals(meta1.checksum, "sha256:abc");
95
-
assertEquals(meta1.backedUpAt, "2024-02-01T00:00:00Z");
96
-
97
-
// uuid-2 also updated with defaults
98
-
const meta2Obj = s3.objects.get("metadata/assets/uuid-2.json");
99
-
assertExists(meta2Obj);
100
-
const meta2 = JSON.parse(new TextDecoder().decode(meta2Obj.body));
101
-
assertEquals(meta2.title, null);
102
-
assertEquals(meta2.albums, []);
103
-
});
104
-
105
-
Deno.test("refresh-metadata: skips assets not in manifest", async () => {
106
-
const assets = [makeAsset("uuid-1"), makeAsset("uuid-not-backed-up")];
107
-
108
-
const manifest = makeManifest({
109
-
"uuid-1": {
110
-
s3Key: "originals/2024/01/uuid-1.heic",
111
-
checksum: "sha256:abc",
112
-
backedUpAt: "2024-02-01T00:00:00Z",
113
-
},
114
-
});
115
-
116
-
const s3 = createMockS3Provider();
117
-
const report = await refreshMetadata(assets, manifest, s3);
118
-
119
-
assertEquals(report.updated, 1);
120
-
assertEquals(
121
-
s3.objects.has("metadata/assets/uuid-not-backed-up.json"),
122
-
false,
123
-
);
124
-
});
125
-
126
-
Deno.test("refresh-metadata: dry run uploads nothing", async () => {
127
-
const assets = [makeAsset("uuid-1")];
128
-
129
-
const manifest = makeManifest({
130
-
"uuid-1": {
131
-
s3Key: "originals/2024/01/uuid-1.heic",
132
-
checksum: "sha256:abc",
133
-
backedUpAt: "2024-02-01T00:00:00Z",
134
-
},
135
-
});
136
-
137
-
const s3 = createMockS3Provider();
138
-
const report = await refreshMetadata(assets, manifest, s3, { dryRun: true });
139
-
140
-
assertEquals(report.updated, 0);
141
-
assertEquals(report.skipped, 1);
142
-
assertEquals(s3.objects.size, 0);
143
-
});
144
-
145
-
Deno.test("refresh-metadata: records failures when S3 upload throws", async () => {
146
-
const assets = [makeAsset("uuid-1")];
147
-
148
-
const manifest = makeManifest({
149
-
"uuid-1": {
150
-
s3Key: "originals/2024/01/uuid-1.heic",
151
-
checksum: "sha256:abc",
152
-
backedUpAt: "2024-02-01T00:00:00Z",
153
-
},
154
-
});
155
-
156
-
const s3 = createMockS3Provider();
157
-
s3.putObject = () => {
158
-
throw new Error("S3 unavailable");
159
-
};
160
-
161
-
const report = await refreshMetadata(assets, manifest, s3);
162
-
163
-
assertEquals(report.updated, 0);
164
-
assertEquals(report.failed, 1);
165
-
assertEquals(report.errors.length, 1);
166
-
assertEquals(report.errors[0].message, "S3 unavailable");
167
-
});
-147
cli/src/commands/refresh-metadata.ts
-147
cli/src/commands/refresh-metadata.ts
···
1
-
import type { PhotoAsset } from "@attic/shared";
2
-
import { buildMetadataJson, metadataKey } from "@attic/shared";
3
-
import type { Manifest } from "../manifest/manifest.ts";
4
-
import type { S3Provider } from "../storage/s3-client.ts";
5
-
import { formatBytes } from "../format.ts";
6
-
import { withRetry } from "../retry.ts";
7
-
8
-
export interface RefreshMetadataOptions {
9
-
/** Maximum concurrent uploads. */
10
-
concurrency: number;
11
-
/** Show what would be uploaded without uploading. */
12
-
dryRun: boolean;
13
-
}
14
-
15
-
const DEFAULT_OPTIONS: RefreshMetadataOptions = {
16
-
concurrency: 20,
17
-
dryRun: false,
18
-
};
19
-
20
-
export interface RefreshMetadataReport {
21
-
updated: number;
22
-
skipped: number;
23
-
failed: number;
24
-
totalBytes: number;
25
-
errors: Array<{ uuid: string; message: string }>;
26
-
}
27
-
28
-
/**
29
-
* Re-upload metadata JSON for already-backed-up assets.
30
-
* Original files and manifest are left untouched.
31
-
*/
32
-
export async function refreshMetadata(
33
-
assets: PhotoAsset[],
34
-
manifest: Manifest,
35
-
s3: S3Provider,
36
-
opts: Partial<RefreshMetadataOptions> = {},
37
-
): Promise<RefreshMetadataReport> {
38
-
const options = { ...DEFAULT_OPTIONS, ...opts };
39
-
options.concurrency = Math.max(1, options.concurrency);
40
-
41
-
// Only refresh assets that are in the manifest
42
-
const assetByUuid = new Map<string, PhotoAsset>();
43
-
for (const a of assets) {
44
-
assetByUuid.set(a.uuid, a);
45
-
}
46
-
47
-
const toRefresh: Array<
48
-
{ asset: PhotoAsset; s3Key: string; checksum: string; backedUpAt: string }
49
-
> = [];
50
-
for (const [uuid, entry] of Object.entries(manifest.entries)) {
51
-
const asset = assetByUuid.get(uuid);
52
-
if (asset) {
53
-
toRefresh.push({
54
-
asset,
55
-
s3Key: entry.s3Key,
56
-
checksum: entry.checksum,
57
-
backedUpAt: entry.backedUpAt,
58
-
});
59
-
}
60
-
}
61
-
62
-
console.log(`\n Attic — Refresh Metadata`);
63
-
console.log(` ════════════════════════\n`);
64
-
console.log(
65
-
` Backed-up assets in DB: ${toRefresh.length.toLocaleString()}`,
66
-
);
67
-
if (options.dryRun) console.log(` Mode: DRY RUN`);
68
-
console.log();
69
-
70
-
if (toRefresh.length === 0) {
71
-
console.log(" Nothing to refresh — no backed-up assets found in DB.\n");
72
-
return { updated: 0, skipped: 0, failed: 0, totalBytes: 0, errors: [] };
73
-
}
74
-
75
-
if (options.dryRun) {
76
-
return {
77
-
updated: 0,
78
-
skipped: toRefresh.length,
79
-
failed: 0,
80
-
totalBytes: 0,
81
-
errors: [],
82
-
};
83
-
}
84
-
85
-
const report: RefreshMetadataReport = {
86
-
updated: 0,
87
-
skipped: 0,
88
-
failed: 0,
89
-
totalBytes: 0,
90
-
errors: [],
91
-
};
92
-
93
-
// Process with bounded concurrency using an index counter (O(1) per item).
94
-
// Mutations to `report` are safe: Deno is single-threaded, and all
95
-
// increments happen synchronously between await points.
96
-
let cursor = 0;
97
-
const workers = Array.from(
98
-
{ length: Math.min(options.concurrency, toRefresh.length) },
99
-
async () => {
100
-
while (cursor < toRefresh.length) {
101
-
const item = toRefresh[cursor++];
102
-
try {
103
-
const meta = buildMetadataJson(
104
-
item.asset,
105
-
item.s3Key,
106
-
item.checksum,
107
-
item.backedUpAt,
108
-
);
109
-
const data = new TextEncoder().encode(
110
-
JSON.stringify(meta, null, 2),
111
-
);
112
-
await withRetry(() =>
113
-
s3.putObject(
114
-
metadataKey(item.asset.uuid),
115
-
data,
116
-
"application/json",
117
-
)
118
-
);
119
-
report.updated++;
120
-
report.totalBytes += data.byteLength;
121
-
} catch (error: unknown) {
122
-
const msg = error instanceof Error ? error.message : "Unknown error";
123
-
report.errors.push({ uuid: item.asset.uuid, message: msg });
124
-
report.failed++;
125
-
}
126
-
127
-
const done = report.updated + report.failed;
128
-
if (done % 50 === 0 || done === toRefresh.length) {
129
-
const pct = ((done / toRefresh.length) * 100).toFixed(1);
130
-
console.log(
131
-
` Progress: ${done}/${toRefresh.length} (${pct}%) ` +
132
-
`Uploaded: ${formatBytes(report.totalBytes)}`,
133
-
);
134
-
}
135
-
}
136
-
},
137
-
);
138
-
139
-
await Promise.all(workers);
140
-
141
-
console.log(`\n ── Complete ──`);
142
-
console.log(` Updated: ${report.updated.toLocaleString()}`);
143
-
console.log(` Failed: ${report.failed.toLocaleString()}`);
144
-
console.log(` Total: ${formatBytes(report.totalBytes)}\n`);
145
-
146
-
return report;
147
-
}
-72
cli/src/commands/scan.ts
-72
cli/src/commands/scan.ts
···
1
-
import type { PhotoAsset } from "@attic/shared";
2
-
import { AssetKind, CloudLocalState } from "@attic/shared";
3
-
import { formatBytes } from "../format.ts";
4
-
5
-
export function printScanReport(assets: PhotoAsset[]): void {
6
-
const totalSize = assets.reduce(
7
-
(sum, a) => sum + (a.originalFileSize ?? 0),
8
-
0,
9
-
);
10
-
11
-
const photos = assets.filter((a) => a.kind === AssetKind.PHOTO);
12
-
const videos = assets.filter((a) => a.kind === AssetKind.VIDEO);
13
-
const photoSize = photos.reduce(
14
-
(sum, a) => sum + (a.originalFileSize ?? 0),
15
-
0,
16
-
);
17
-
const videoSize = videos.reduce(
18
-
(sum, a) => sum + (a.originalFileSize ?? 0),
19
-
0,
20
-
);
21
-
22
-
const local = assets.filter(
23
-
(a) => a.cloudLocalState === CloudLocalState.LOCAL,
24
-
);
25
-
const icloudOnly = assets.filter(
26
-
(a) => a.cloudLocalState === CloudLocalState.ICLOUD_ONLY,
27
-
);
28
-
29
-
const favorites = assets.filter((a) => a.favorite);
30
-
31
-
// Type breakdown
32
-
const typeGroups = new Map<string, number>();
33
-
for (const asset of assets) {
34
-
const type = asset.uniformTypeIdentifier ?? "unknown";
35
-
typeGroups.set(type, (typeGroups.get(type) ?? 0) + 1);
36
-
}
37
-
const sortedTypes = [...typeGroups.entries()].sort((a, b) => b[1] - a[1]);
38
-
39
-
console.log(`\n Attic — Library Scan`);
40
-
console.log(` ════════════════════\n`);
41
-
console.log(` Total assets: ${assets.length.toLocaleString()}`);
42
-
console.log(` Total size: ${formatBytes(totalSize)}\n`);
43
-
44
-
console.log(
45
-
` Photos: ${photos.length.toLocaleString()} (${
46
-
formatBytes(photoSize)
47
-
})`,
48
-
);
49
-
console.log(
50
-
` Videos: ${videos.length.toLocaleString()} (${
51
-
formatBytes(videoSize)
52
-
})\n`,
53
-
);
54
-
55
-
console.log(` Local originals: ${local.length.toLocaleString()}`);
56
-
console.log(
57
-
` iCloud only: ${icloudOnly.length.toLocaleString()}\n`,
58
-
);
59
-
60
-
const edited = assets.filter((a) => a.hasEdit);
61
-
console.log(` Favorites: ${favorites.length.toLocaleString()}`);
62
-
console.log(` Edited: ${edited.length.toLocaleString()}\n`);
63
-
64
-
console.log(` Types:`);
65
-
for (const [type, count] of sortedTypes.slice(0, 10)) {
66
-
console.log(` ${type.padEnd(40)} ${count.toLocaleString()}`);
67
-
}
68
-
if (sortedTypes.length > 10) {
69
-
console.log(` ... and ${sortedTypes.length - 10} more types`);
70
-
}
71
-
console.log();
72
-
}
-56
cli/src/commands/status.ts
-56
cli/src/commands/status.ts
···
1
-
import type { PhotoAsset } from "@attic/shared";
2
-
import type { Manifest } from "../manifest/manifest.ts";
3
-
import { isBackedUp } from "../manifest/manifest.ts";
4
-
import { formatBytes } from "../format.ts";
5
-
6
-
/** Get the best-known size for an asset: Photos DB first, manifest fallback. */
7
-
function assetSize(asset: PhotoAsset, manifest: Manifest): number {
8
-
if (asset.originalFileSize && asset.originalFileSize > 0) {
9
-
return asset.originalFileSize;
10
-
}
11
-
return manifest.entries[asset.uuid]?.size ?? 0;
12
-
}
13
-
14
-
export function printStatusReport(
15
-
assets: PhotoAsset[],
16
-
manifest: Manifest,
17
-
): void {
18
-
const backedUp = assets.filter((a) => isBackedUp(manifest, a.uuid));
19
-
const pending = assets.filter((a) => !isBackedUp(manifest, a.uuid));
20
-
21
-
const totalSize = assets.reduce(
22
-
(sum, a) => sum + assetSize(a, manifest),
23
-
0,
24
-
);
25
-
const backedUpSize = backedUp.reduce(
26
-
(sum, a) => sum + assetSize(a, manifest),
27
-
0,
28
-
);
29
-
const pendingSize = pending.reduce(
30
-
(sum, a) => sum + assetSize(a, manifest),
31
-
0,
32
-
);
33
-
34
-
console.log(`\n Attic — Backup Status`);
35
-
console.log(` ═════════════════════\n`);
36
-
console.log(
37
-
` Total assets: ${assets.length.toLocaleString()} (${
38
-
formatBytes(totalSize)
39
-
})`,
40
-
);
41
-
console.log(
42
-
` Backed up: ${backedUp.length.toLocaleString()} (${
43
-
formatBytes(backedUpSize)
44
-
})`,
45
-
);
46
-
console.log(
47
-
` Pending: ${pending.length.toLocaleString()} (${
48
-
formatBytes(pendingSize)
49
-
})`,
50
-
);
51
-
52
-
const pct = assets.length > 0
53
-
? ((backedUp.length / assets.length) * 100).toFixed(1)
54
-
: "0.0";
55
-
console.log(`\n Progress: ${pct}%\n`);
56
-
}
-166
cli/src/commands/verify.test.ts
-166
cli/src/commands/verify.test.ts
···
1
-
import { assertEquals } from "@std/assert";
2
-
import { runVerify } from "./verify.ts";
3
-
import type { Manifest } from "../manifest/manifest.ts";
4
-
import { createMockS3Provider } from "../storage/s3-client.mock.ts";
5
-
6
-
function makeManifest(
7
-
entries: Record<string, { s3Key: string; checksum: string }>,
8
-
): Manifest {
9
-
const manifest: Manifest = { entries: {} };
10
-
for (const [uuid, { s3Key, checksum }] of Object.entries(entries)) {
11
-
manifest.entries[uuid] = {
12
-
uuid,
13
-
s3Key,
14
-
checksum,
15
-
backedUpAt: "2024-01-15T00:00:00Z",
16
-
};
17
-
}
18
-
return manifest;
19
-
}
20
-
21
-
Deno.test("verify quick: all objects present", async () => {
22
-
const s3 = createMockS3Provider();
23
-
const data = new TextEncoder().encode("photo-data");
24
-
await s3.putObject("originals/2024/01/uuid-1.heic", data);
25
-
await s3.putObject("originals/2024/01/uuid-2.heic", data);
26
-
27
-
const manifest = makeManifest({
28
-
"uuid-1": {
29
-
s3Key: "originals/2024/01/uuid-1.heic",
30
-
checksum: "sha256:abc",
31
-
},
32
-
"uuid-2": {
33
-
s3Key: "originals/2024/01/uuid-2.heic",
34
-
checksum: "sha256:def",
35
-
},
36
-
});
37
-
38
-
const report = await runVerify(manifest, s3, { concurrency: 1 });
39
-
40
-
assertEquals(report.total, 2);
41
-
assertEquals(report.ok, 2);
42
-
assertEquals(report.missing, 0);
43
-
assertEquals(report.corrupted, 0);
44
-
assertEquals(report.errors.length, 0);
45
-
});
46
-
47
-
Deno.test("verify quick: detects missing objects", async () => {
48
-
const s3 = createMockS3Provider();
49
-
await s3.putObject(
50
-
"originals/2024/01/uuid-1.heic",
51
-
new TextEncoder().encode("data"),
52
-
);
53
-
54
-
const manifest = makeManifest({
55
-
"uuid-1": {
56
-
s3Key: "originals/2024/01/uuid-1.heic",
57
-
checksum: "sha256:abc",
58
-
},
59
-
"uuid-2": {
60
-
s3Key: "originals/2024/01/uuid-2.heic",
61
-
checksum: "sha256:def",
62
-
},
63
-
});
64
-
65
-
const report = await runVerify(manifest, s3, { concurrency: 1 });
66
-
67
-
assertEquals(report.total, 2);
68
-
assertEquals(report.ok, 1);
69
-
assertEquals(report.missing, 1);
70
-
assertEquals(report.errors.length, 1);
71
-
assertEquals(report.errors[0].uuid, "uuid-2");
72
-
});
73
-
74
-
Deno.test("verify deep: checksum match passes", async () => {
75
-
const s3 = createMockS3Provider();
76
-
const data = new TextEncoder().encode("hello");
77
-
78
-
const hashBuffer = await crypto.subtle.digest("SHA-256", data);
79
-
const hashHex = Array.from(
80
-
new Uint8Array(hashBuffer),
81
-
(b) => b.toString(16).padStart(2, "0"),
82
-
).join("");
83
-
84
-
await s3.putObject("originals/2024/01/uuid-1.heic", data);
85
-
86
-
const manifest = makeManifest({
87
-
"uuid-1": {
88
-
s3Key: "originals/2024/01/uuid-1.heic",
89
-
checksum: `sha256:${hashHex}`,
90
-
},
91
-
});
92
-
93
-
const report = await runVerify(manifest, s3, {
94
-
deep: true,
95
-
concurrency: 1,
96
-
});
97
-
98
-
assertEquals(report.total, 1);
99
-
assertEquals(report.ok, 1);
100
-
assertEquals(report.corrupted, 0);
101
-
});
102
-
103
-
Deno.test("verify deep: checksum mismatch detected", async () => {
104
-
const s3 = createMockS3Provider();
105
-
await s3.putObject(
106
-
"originals/2024/01/uuid-1.heic",
107
-
new TextEncoder().encode("actual-data"),
108
-
);
109
-
110
-
const manifest = makeManifest({
111
-
"uuid-1": {
112
-
s3Key: "originals/2024/01/uuid-1.heic",
113
-
checksum:
114
-
"sha256:0000000000000000000000000000000000000000000000000000000000000000",
115
-
},
116
-
});
117
-
118
-
const report = await runVerify(manifest, s3, {
119
-
deep: true,
120
-
concurrency: 1,
121
-
});
122
-
123
-
assertEquals(report.total, 1);
124
-
assertEquals(report.ok, 0);
125
-
assertEquals(report.corrupted, 1);
126
-
assertEquals(report.errors.length, 1);
127
-
assertEquals(report.errors[0].uuid, "uuid-1");
128
-
});
129
-
130
-
Deno.test("verify: empty manifest reports nothing to verify", async () => {
131
-
const s3 = createMockS3Provider();
132
-
const manifest: Manifest = { entries: {} };
133
-
134
-
const report = await runVerify(manifest, s3);
135
-
136
-
assertEquals(report.total, 0);
137
-
assertEquals(report.ok, 0);
138
-
});
139
-
140
-
Deno.test("verify: concurrent verification produces correct counts", async () => {
141
-
const s3 = createMockS3Provider();
142
-
143
-
// Create 10 objects, leave 3 missing
144
-
for (let i = 0; i < 7; i++) {
145
-
await s3.putObject(
146
-
`originals/2024/01/uuid-${i}.heic`,
147
-
new TextEncoder().encode(`data-${i}`),
148
-
);
149
-
}
150
-
151
-
const entries: Record<string, { s3Key: string; checksum: string }> = {};
152
-
for (let i = 0; i < 10; i++) {
153
-
entries[`uuid-${i}`] = {
154
-
s3Key: `originals/2024/01/uuid-${i}.heic`,
155
-
checksum: "sha256:abc",
156
-
};
157
-
}
158
-
159
-
const manifest = makeManifest(entries);
160
-
const report = await runVerify(manifest, s3, { concurrency: 5 });
161
-
162
-
assertEquals(report.total, 10);
163
-
assertEquals(report.ok, 7);
164
-
assertEquals(report.missing, 3);
165
-
assertEquals(report.errors.length, 3);
166
-
});
-207
cli/src/commands/verify.ts
-207
cli/src/commands/verify.ts
···
1
-
import type { Manifest, ManifestEntry } from "../manifest/manifest.ts";
2
-
import type { S3Provider } from "../storage/s3-client.ts";
3
-
import { withRetry } from "../retry.ts";
4
-
5
-
export interface VerifyOptions {
6
-
/** Download each object and re-checksum (slow but thorough). */
7
-
deep: boolean;
8
-
/** Max concurrent S3 requests. */
9
-
concurrency: number;
10
-
}
11
-
12
-
const DEFAULT_OPTIONS: VerifyOptions = {
13
-
deep: false,
14
-
concurrency: 50,
15
-
};
16
-
17
-
const MAX_ERRORS = 1000;
18
-
19
-
export interface VerifyReport {
20
-
total: number;
21
-
ok: number;
22
-
missing: number;
23
-
corrupted: number;
24
-
errors: Array<{ uuid: string; message: string }>;
25
-
errorsOverflow: number;
26
-
}
27
-
28
-
/** Verify backup integrity by checking S3 objects against the manifest. */
29
-
export async function runVerify(
30
-
manifest: Manifest,
31
-
s3: S3Provider,
32
-
opts: Partial<VerifyOptions> = {},
33
-
): Promise<VerifyReport> {
34
-
const options = { ...DEFAULT_OPTIONS, ...opts };
35
-
const entries = Object.values(manifest.entries);
36
-
37
-
if (entries.length === 0) {
38
-
console.log("\n Nothing to verify — manifest is empty.\n");
39
-
return {
40
-
total: 0,
41
-
ok: 0,
42
-
missing: 0,
43
-
corrupted: 0,
44
-
errors: [],
45
-
errorsOverflow: 0,
46
-
};
47
-
}
48
-
49
-
console.log(`\n Attic — Verify`);
50
-
console.log(` ══════════════\n`);
51
-
console.log(` Manifest entries: ${entries.length.toLocaleString()}`);
52
-
console.log(
53
-
` Mode: ${options.deep ? "deep (checksum)" : "quick (HEAD)"}`,
54
-
);
55
-
console.log(` Concurrency: ${options.concurrency}`);
56
-
console.log();
57
-
58
-
const report: VerifyReport = {
59
-
total: entries.length,
60
-
ok: 0,
61
-
missing: 0,
62
-
corrupted: 0,
63
-
errors: [],
64
-
errorsOverflow: 0,
65
-
};
66
-
67
-
// Bounded concurrency pool
68
-
let cursor = 0;
69
-
let completed = 0;
70
-
71
-
function recordResult(entry: ManifestEntry, result: VerifyResult): void {
72
-
switch (result.status) {
73
-
case "ok":
74
-
report.ok++;
75
-
break;
76
-
case "missing":
77
-
report.missing++;
78
-
pushError(report, entry.uuid, `Missing from S3: ${entry.s3Key}`);
79
-
break;
80
-
case "corrupted":
81
-
report.corrupted++;
82
-
pushError(report, entry.uuid, result.message);
83
-
break;
84
-
case "error":
85
-
pushError(report, entry.uuid, result.message);
86
-
break;
87
-
}
88
-
completed++;
89
-
}
90
-
91
-
async function worker(): Promise<void> {
92
-
while (cursor < entries.length) {
93
-
const i = cursor++;
94
-
const entry = entries[i];
95
-
const result = options.deep
96
-
? await verifyDeep(entry, s3)
97
-
: await verifyQuick(entry, s3);
98
-
recordResult(entry, result);
99
-
100
-
// Progress every 100 completions
101
-
if (completed % 100 === 0 || completed === entries.length) {
102
-
const pct = ((completed / entries.length) * 100).toFixed(1);
103
-
console.log(
104
-
` Checked ${completed}/${entries.length} (${pct}%) ` +
105
-
`OK: ${report.ok} Missing: ${report.missing} Corrupted: ${report.corrupted}`,
106
-
);
107
-
}
108
-
}
109
-
}
110
-
111
-
const workerCount = Math.min(options.concurrency, entries.length);
112
-
await Promise.all(Array.from({ length: workerCount }, () => worker()));
113
-
114
-
// Summary
115
-
console.log(`\n ── Verify Complete ──`);
116
-
console.log(` Total: ${report.total.toLocaleString()}`);
117
-
console.log(` OK: ${report.ok.toLocaleString()}`);
118
-
console.log(` Missing: ${report.missing.toLocaleString()}`);
119
-
console.log(` Corrupted: ${report.corrupted.toLocaleString()}`);
120
-
if (report.errorsOverflow > 0) {
121
-
console.log(
122
-
` (${report.errorsOverflow.toLocaleString()} additional errors not shown)`,
123
-
);
124
-
}
125
-
console.log();
126
-
127
-
return report;
128
-
}
129
-
130
-
function pushError(
131
-
report: VerifyReport,
132
-
uuid: string,
133
-
message: string,
134
-
): void {
135
-
if (report.errors.length < MAX_ERRORS) {
136
-
report.errors.push({ uuid, message });
137
-
} else {
138
-
report.errorsOverflow++;
139
-
}
140
-
}
141
-
142
-
interface VerifyResult {
143
-
status: "ok" | "missing" | "corrupted" | "error";
144
-
message: string;
145
-
}
146
-
147
-
/** Quick verify: HEAD the S3 object, confirm it exists. */
148
-
async function verifyQuick(
149
-
entry: ManifestEntry,
150
-
s3: S3Provider,
151
-
): Promise<VerifyResult> {
152
-
try {
153
-
const meta = await withRetry(() => s3.headObject(entry.s3Key));
154
-
if (!meta) {
155
-
return { status: "missing", message: `Not found: ${entry.s3Key}` };
156
-
}
157
-
return { status: "ok", message: "" };
158
-
} catch (error: unknown) {
159
-
const msg = error instanceof Error ? error.message : "Unknown error";
160
-
return { status: "error", message: msg };
161
-
}
162
-
}
163
-
164
-
/** Deep verify: download the S3 object, compute SHA-256, compare to manifest checksum. */
165
-
async function verifyDeep(
166
-
entry: ManifestEntry,
167
-
s3: S3Provider,
168
-
): Promise<VerifyResult> {
169
-
try {
170
-
let data: Uint8Array;
171
-
try {
172
-
data = await withRetry(() => s3.getObject(entry.s3Key));
173
-
} catch (error: unknown) {
174
-
if (
175
-
error instanceof Error && error.message.includes("not found") ||
176
-
error instanceof Error && error.message.includes("Not found") ||
177
-
error instanceof Error && error.message.includes("NoSuchKey")
178
-
) {
179
-
return { status: "missing", message: `Not found: ${entry.s3Key}` };
180
-
}
181
-
throw error;
182
-
}
183
-
184
-
const hashBuffer = await crypto.subtle.digest(
185
-
"SHA-256",
186
-
data.buffer as ArrayBuffer,
187
-
);
188
-
const hashHex = Array.from(
189
-
new Uint8Array(hashBuffer),
190
-
(b) => b.toString(16).padStart(2, "0"),
191
-
).join("");
192
-
const actual = `sha256:${hashHex}`;
193
-
194
-
if (actual !== entry.checksum) {
195
-
return {
196
-
status: "corrupted",
197
-
message:
198
-
`Checksum mismatch for ${entry.s3Key}: expected ${entry.checksum}, got ${actual}`,
199
-
};
200
-
}
201
-
202
-
return { status: "ok", message: "" };
203
-
} catch (error: unknown) {
204
-
const msg = error instanceof Error ? error.message : "Unknown error";
205
-
return { status: "error", message: msg };
206
-
}
207
-
}
-133
cli/src/config/config.test.ts
-133
cli/src/config/config.test.ts
···
1
-
import { assertEquals, assertThrows } from "@std/assert";
2
-
import { loadConfig, validateConfig, writeConfig } from "./config.ts";
3
-
import { join } from "@std/path/join";
4
-
5
-
Deno.test("validateConfig accepts valid config with all fields", () => {
6
-
const config = validateConfig({
7
-
endpoint: "https://s3.fr-par.scw.cloud",
8
-
region: "fr-par",
9
-
bucket: "my-photo-backup",
10
-
pathStyle: false,
11
-
keychain: {
12
-
accessKeyService: "custom-access",
13
-
secretKeyService: "custom-secret",
14
-
},
15
-
});
16
-
17
-
assertEquals(config.endpoint, "https://s3.fr-par.scw.cloud");
18
-
assertEquals(config.region, "fr-par");
19
-
assertEquals(config.bucket, "my-photo-backup");
20
-
assertEquals(config.pathStyle, false);
21
-
assertEquals(config.keychain.accessKeyService, "custom-access");
22
-
assertEquals(config.keychain.secretKeyService, "custom-secret");
23
-
});
24
-
25
-
Deno.test("validateConfig applies defaults for optional fields", () => {
26
-
const config = validateConfig({
27
-
endpoint: "https://s3.fr-par.scw.cloud",
28
-
region: "fr-par",
29
-
bucket: "my-photo-backup",
30
-
});
31
-
32
-
assertEquals(config.pathStyle, true);
33
-
assertEquals(config.keychain.accessKeyService, "attic-s3-access-key");
34
-
assertEquals(config.keychain.secretKeyService, "attic-s3-secret-key");
35
-
});
36
-
37
-
Deno.test("validateConfig rejects missing endpoint", () => {
38
-
assertThrows(
39
-
() => validateConfig({ region: "fr-par", bucket: "b" }),
40
-
Error,
41
-
'"endpoint" is required',
42
-
);
43
-
});
44
-
45
-
Deno.test("validateConfig rejects non-https endpoint", () => {
46
-
assertThrows(
47
-
() =>
48
-
validateConfig({
49
-
endpoint: "http://s3.example.com",
50
-
region: "us-east-1",
51
-
bucket: "bbb",
52
-
}),
53
-
Error,
54
-
"must start with https://",
55
-
);
56
-
});
57
-
58
-
Deno.test("validateConfig rejects missing region", () => {
59
-
assertThrows(
60
-
() => validateConfig({ endpoint: "https://s3.example.com", bucket: "bbb" }),
61
-
Error,
62
-
'"region" is required',
63
-
);
64
-
});
65
-
66
-
Deno.test("validateConfig rejects missing bucket", () => {
67
-
assertThrows(
68
-
() =>
69
-
validateConfig({
70
-
endpoint: "https://s3.example.com",
71
-
region: "us-east-1",
72
-
}),
73
-
Error,
74
-
'"bucket" is required',
75
-
);
76
-
});
77
-
78
-
Deno.test("validateConfig rejects invalid bucket name", () => {
79
-
assertThrows(
80
-
() =>
81
-
validateConfig({
82
-
endpoint: "https://s3.example.com",
83
-
region: "us-east-1",
84
-
bucket: "A",
85
-
}),
86
-
Error,
87
-
"is invalid",
88
-
);
89
-
});
90
-
91
-
Deno.test("validateConfig rejects non-object input", () => {
92
-
assertThrows(
93
-
() => validateConfig("not an object"),
94
-
Error,
95
-
"must be a JSON object",
96
-
);
97
-
assertThrows(
98
-
() => validateConfig(null),
99
-
Error,
100
-
"must be a JSON object",
101
-
);
102
-
});
103
-
104
-
Deno.test("writeConfig and loadConfig round-trip", () => {
105
-
const dir = Deno.makeTempDirSync();
106
-
const config = {
107
-
endpoint: "https://s3.fr-par.scw.cloud",
108
-
region: "fr-par",
109
-
bucket: "test-bucket",
110
-
pathStyle: true,
111
-
keychain: {
112
-
accessKeyService: "attic-s3-access-key",
113
-
secretKeyService: "attic-s3-secret-key",
114
-
},
115
-
};
116
-
117
-
writeConfig(config, dir);
118
-
119
-
// Verify file exists
120
-
const text = Deno.readTextFileSync(join(dir, "config.json"));
121
-
const parsed = JSON.parse(text);
122
-
assertEquals(parsed.endpoint, "https://s3.fr-par.scw.cloud");
123
-
124
-
// Round-trip through loadConfig
125
-
const loaded = loadConfig(dir);
126
-
assertEquals(loaded, config);
127
-
});
128
-
129
-
Deno.test("loadConfig returns null when file does not exist", () => {
130
-
const dir = Deno.makeTempDirSync();
131
-
const result = loadConfig(dir);
132
-
assertEquals(result, null);
133
-
});
-129
cli/src/config/config.ts
-129
cli/src/config/config.ts
···
1
-
import { join } from "@std/path/join";
2
-
3
-
export interface AtticConfig {
4
-
endpoint: string;
5
-
region: string;
6
-
bucket: string;
7
-
pathStyle: boolean;
8
-
keychain: {
9
-
accessKeyService: string;
10
-
secretKeyService: string;
11
-
};
12
-
}
13
-
14
-
const CONFIG_DIR = join(
15
-
Deno.env.get("HOME") ?? "~",
16
-
".attic",
17
-
);
18
-
19
-
const CONFIG_PATH = join(CONFIG_DIR, "config.json");
20
-
21
-
const BUCKET_PATTERN = /^[a-z0-9][a-z0-9.\-]{1,61}[a-z0-9]$/;
22
-
23
-
/** Load config from ~/.attic/config.json. Returns null if file doesn't exist. */
24
-
export function loadConfig(dir: string = CONFIG_DIR): AtticConfig | null {
25
-
const path = join(dir, "config.json");
26
-
let text: string;
27
-
try {
28
-
text = Deno.readTextFileSync(path);
29
-
} catch (error: unknown) {
30
-
if (error instanceof Deno.errors.NotFound) {
31
-
return null;
32
-
}
33
-
throw error;
34
-
}
35
-
const raw: unknown = JSON.parse(text);
36
-
return validateConfig(raw);
37
-
}
38
-
39
-
/** Validate a raw parsed config object. Throws with specific messages on invalid fields. */
40
-
export function validateConfig(raw: unknown): AtticConfig {
41
-
if (typeof raw !== "object" || raw === null || Array.isArray(raw)) {
42
-
throw new Error("Config must be a JSON object");
43
-
}
44
-
45
-
const obj = raw as Record<string, unknown>;
46
-
47
-
if (typeof obj.endpoint !== "string" || obj.endpoint === "") {
48
-
throw new Error(
49
-
'Config: "endpoint" is required (e.g. "https://s3.fr-par.scw.cloud")',
50
-
);
51
-
}
52
-
if (!obj.endpoint.startsWith("https://")) {
53
-
throw new Error('Config: "endpoint" must start with https://');
54
-
}
55
-
56
-
if (typeof obj.region !== "string" || obj.region === "") {
57
-
throw new Error('Config: "region" is required (e.g. "fr-par")');
58
-
}
59
-
60
-
if (typeof obj.bucket !== "string" || obj.bucket === "") {
61
-
throw new Error('Config: "bucket" is required');
62
-
}
63
-
if (!BUCKET_PATTERN.test(obj.bucket)) {
64
-
throw new Error(
65
-
`Config: "bucket" name "${obj.bucket}" is invalid. ` +
66
-
"Use lowercase letters, numbers, dots, and hyphens (3-63 chars).",
67
-
);
68
-
}
69
-
70
-
const pathStyle = obj.pathStyle !== undefined ? Boolean(obj.pathStyle) : true;
71
-
72
-
const keychain = typeof obj.keychain === "object" && obj.keychain !== null &&
73
-
!Array.isArray(obj.keychain)
74
-
? obj.keychain as Record<string, unknown>
75
-
: {};
76
-
77
-
const accessKeyService = typeof keychain.accessKeyService === "string" &&
78
-
keychain.accessKeyService !== ""
79
-
? keychain.accessKeyService
80
-
: "attic-s3-access-key";
81
-
82
-
const secretKeyService = typeof keychain.secretKeyService === "string" &&
83
-
keychain.secretKeyService !== ""
84
-
? keychain.secretKeyService
85
-
: "attic-s3-secret-key";
86
-
87
-
return {
88
-
endpoint: obj.endpoint,
89
-
region: obj.region,
90
-
bucket: obj.bucket,
91
-
pathStyle,
92
-
keychain: { accessKeyService, secretKeyService },
93
-
};
94
-
}
95
-
96
-
/** Write config to disk, creating ~/.attic/ if needed. */
97
-
export function writeConfig(
98
-
config: AtticConfig,
99
-
dir: string = CONFIG_DIR,
100
-
): void {
101
-
Deno.mkdirSync(dir, { recursive: true, mode: 0o700 });
102
-
const path = join(dir, "config.json");
103
-
Deno.writeTextFileSync(
104
-
path,
105
-
JSON.stringify(config, null, 2) + "\n",
106
-
{ mode: 0o600 },
107
-
);
108
-
}
109
-
110
-
/** Resolve the default config file path. */
111
-
export function configPath(): string {
112
-
return CONFIG_PATH;
113
-
}
114
-
115
-
/**
116
-
* Load and validate config, throwing a user-friendly error if missing.
117
-
* Use this for commands that require S3 (backup, verify).
118
-
*/
119
-
export function requireConfig(dir?: string): AtticConfig {
120
-
const config = loadConfig(dir);
121
-
if (config === null) {
122
-
const path = dir ? join(dir, "config.json") : configPath();
123
-
throw new Error(
124
-
`No config file found at ${path}\n` +
125
-
'Run "attic init" to set up your S3 connection, or create the file manually.',
126
-
);
127
-
}
128
-
return config;
129
-
}
-102
cli/src/export/exporter-race.test.ts
-102
cli/src/export/exporter-race.test.ts
···
1
-
import { assertEquals, assertRejects } from "@std/assert";
2
-
import { LadderTimeoutError, raceSubprocess } from "./exporter.ts";
3
-
import { AbortError } from "../abort-error.ts";
4
-
5
-
/** Create a fake ChildProcess that resolves/rejects after a delay. */
6
-
function fakeProcess(
7
-
delayMs: number,
8
-
code = 0,
9
-
): Deno.ChildProcess {
10
-
let killed = false;
11
-
// deno-lint-ignore no-explicit-any
12
-
const fake: any = {
13
-
output(): Promise<Deno.CommandOutput> {
14
-
return new Promise((resolve, reject) => {
15
-
const id = setTimeout(() => {
16
-
if (killed) {
17
-
resolve({
18
-
code: 137,
19
-
signal: "SIGTERM",
20
-
success: false,
21
-
stdout: new Uint8Array(),
22
-
stderr: new Uint8Array(),
23
-
});
24
-
} else {
25
-
resolve({
26
-
code,
27
-
signal: null,
28
-
success: code === 0,
29
-
stdout: new Uint8Array(),
30
-
stderr: new Uint8Array(),
31
-
});
32
-
}
33
-
}, delayMs);
34
-
// Store for cleanup
35
-
fake._timerId = id;
36
-
fake._reject = reject;
37
-
});
38
-
},
39
-
kill(_signal: string) {
40
-
killed = true;
41
-
// Resolve the output promise quickly after kill
42
-
if (fake._timerId) {
43
-
clearTimeout(fake._timerId);
44
-
// The process.output() will resolve on next tick with killed state
45
-
}
46
-
},
47
-
pid: 999,
48
-
status: Promise.resolve({ code, signal: null, success: code === 0 }),
49
-
stdin: null,
50
-
stdout: null,
51
-
stderr: null,
52
-
ref() {},
53
-
unref() {},
54
-
[Symbol.dispose]() {},
55
-
};
56
-
return fake as Deno.ChildProcess;
57
-
}
58
-
59
-
Deno.test("raceSubprocess: returns output when subprocess completes before timeout", async () => {
60
-
const process = fakeProcess(10);
61
-
const result = await raceSubprocess(process, 5000);
62
-
assertEquals(result.code, 0);
63
-
});
64
-
65
-
Deno.test("raceSubprocess: rejects on timeout with LadderTimeoutError", async () => {
66
-
const process = fakeProcess(5000);
67
-
await assertRejects(
68
-
() => raceSubprocess(process, 50),
69
-
LadderTimeoutError,
70
-
);
71
-
});
72
-
73
-
Deno.test("raceSubprocess: rejects on abort signal", async () => {
74
-
const controller = new AbortController();
75
-
const process = fakeProcess(5000);
76
-
77
-
setTimeout(() => controller.abort(), 20);
78
-
79
-
await assertRejects(
80
-
() => raceSubprocess(process, 30000, controller.signal),
81
-
AbortError,
82
-
);
83
-
});
84
-
85
-
Deno.test("raceSubprocess: rejects immediately if signal already aborted", async () => {
86
-
const controller = new AbortController();
87
-
controller.abort();
88
-
89
-
const process = fakeProcess(10);
90
-
await assertRejects(
91
-
() => raceSubprocess(process, 5000, controller.signal),
92
-
Error,
93
-
);
94
-
});
95
-
96
-
Deno.test("raceSubprocess: cleans up timer on normal completion", async () => {
97
-
// If the timer leaked without cleanup, the test sanitizer would flag it.
98
-
// The unrefTimer prevents blocking, and the finally block clears it.
99
-
const process = fakeProcess(10);
100
-
const result = await raceSubprocess(process, 60000);
101
-
assertEquals(result.code, 0);
102
-
});
-24
cli/src/export/exporter-timeout.test.ts
-24
cli/src/export/exporter-timeout.test.ts
···
1
-
import { assertEquals } from "@std/assert";
2
-
import {
3
-
isTimeoutError,
4
-
LadderTimeoutError,
5
-
timeoutForBytes,
6
-
} from "./exporter.ts";
7
-
8
-
Deno.test("timeoutForBytes: scales with size", () => {
9
-
// Small batch: base timeout (10 min) + 1 min for < 100 MB
10
-
assertEquals(timeoutForBytes(50 * 1024 * 1024), 10 * 60_000 + 60_000);
11
-
// 500 MB batch: base + 5 min
12
-
assertEquals(timeoutForBytes(500 * 1024 * 1024), 10 * 60_000 + 5 * 60_000);
13
-
// 0 bytes: just base
14
-
assertEquals(timeoutForBytes(0), 10 * 60_000);
15
-
// Negative bytes: treated as 0
16
-
assertEquals(timeoutForBytes(-100), 10 * 60_000);
17
-
});
18
-
19
-
Deno.test("isTimeoutError: detects LadderTimeoutError", () => {
20
-
assertEquals(isTimeoutError(new LadderTimeoutError(300_000)), true);
21
-
assertEquals(isTimeoutError(new Error("connection reset")), false);
22
-
assertEquals(isTimeoutError(new Error("timed out")), false); // plain Error, not LadderTimeoutError
23
-
assertEquals(isTimeoutError("not an error"), false);
24
-
});
-50
cli/src/export/exporter.mock.ts
-50
cli/src/export/exporter.mock.ts
···
1
-
import { join } from "@std/path/join";
2
-
import type { ExportBatchResult, ExportedAsset, Exporter } from "./exporter.ts";
3
-
4
-
/** In-memory mock exporter for testing. */
5
-
export function createMockExporter(
6
-
knownAssets: Map<string, { filename: string; data: Uint8Array }>,
7
-
stagingDir: string,
8
-
): Exporter & { stagingDir: string } {
9
-
return {
10
-
stagingDir,
11
-
12
-
async exportBatch(
13
-
uuids: string[],
14
-
_signal?: AbortSignal,
15
-
): Promise<ExportBatchResult> {
16
-
await Deno.mkdir(stagingDir, { recursive: true });
17
-
18
-
const results: ExportedAsset[] = [];
19
-
const errors: ExportBatchResult["errors"] = [];
20
-
21
-
for (const uuid of uuids) {
22
-
const asset = knownAssets.get(uuid);
23
-
if (!asset) {
24
-
errors.push({ uuid, message: "Asset not found in Photos library" });
25
-
continue;
26
-
}
27
-
28
-
const path = join(stagingDir, `${uuid}_${asset.filename}`);
29
-
await Deno.writeFile(path, asset.data);
30
-
31
-
// Deterministic hash for testing (hex of byte values, zero-padded to 64 chars)
32
-
const hex = Array.from(
33
-
asset.data,
34
-
(b) => b.toString(16).padStart(2, "0"),
35
-
)
36
-
.join("");
37
-
const hash = hex.padEnd(64, "0").slice(0, 64);
38
-
39
-
results.push({
40
-
uuid,
41
-
path,
42
-
size: asset.data.length,
43
-
sha256: hash,
44
-
});
45
-
}
46
-
47
-
return { results, errors };
48
-
},
49
-
};
50
-
}
-89
cli/src/export/exporter.test.ts
-89
cli/src/export/exporter.test.ts
···
1
-
import { assertEquals, assertRejects } from "@std/assert";
2
-
import { createMockExporter } from "./exporter.mock.ts";
3
-
import { removeStagedFile } from "./exporter.ts";
4
-
5
-
Deno.test("mock exporter: exports known assets", async () => {
6
-
const dir = await Deno.makeTempDir();
7
-
try {
8
-
const assets = new Map([
9
-
["uuid-1", { filename: "IMG_001.HEIC", data: new Uint8Array([1, 2, 3]) }],
10
-
]);
11
-
const exporter = createMockExporter(assets, dir);
12
-
const result = await exporter.exportBatch(["uuid-1"]);
13
-
14
-
assertEquals(result.results.length, 1);
15
-
assertEquals(result.errors.length, 0);
16
-
assertEquals(result.results[0].uuid, "uuid-1");
17
-
assertEquals(result.results[0].size, 3);
18
-
19
-
// Verify file was written
20
-
const data = await Deno.readFile(result.results[0].path);
21
-
assertEquals(data, new Uint8Array([1, 2, 3]));
22
-
} finally {
23
-
await Deno.remove(dir, { recursive: true });
24
-
}
25
-
});
26
-
27
-
Deno.test("mock exporter: reports missing assets as errors", async () => {
28
-
const dir = await Deno.makeTempDir();
29
-
try {
30
-
const exporter = createMockExporter(new Map(), dir);
31
-
const result = await exporter.exportBatch(["missing-uuid"]);
32
-
33
-
assertEquals(result.results.length, 0);
34
-
assertEquals(result.errors.length, 1);
35
-
assertEquals(result.errors[0].uuid, "missing-uuid");
36
-
} finally {
37
-
await Deno.remove(dir, { recursive: true });
38
-
}
39
-
});
40
-
41
-
Deno.test("removeStagedFile: cleans up file in staging dir", async () => {
42
-
const dir = await Deno.makeTempDir();
43
-
try {
44
-
const path = `${dir}/testfile`;
45
-
await Deno.writeTextFile(path, "data");
46
-
47
-
await removeStagedFile(path, dir);
48
-
49
-
let exists = true;
50
-
try {
51
-
await Deno.stat(path);
52
-
} catch {
53
-
exists = false;
54
-
}
55
-
assertEquals(exists, false);
56
-
} finally {
57
-
await Deno.remove(dir, { recursive: true });
58
-
}
59
-
});
60
-
61
-
Deno.test("removeStagedFile: ignores missing file", async () => {
62
-
const dir = await Deno.makeTempDir();
63
-
try {
64
-
// Should not throw
65
-
await removeStagedFile(`${dir}/nonexistent`, dir);
66
-
} finally {
67
-
await Deno.remove(dir, { recursive: true });
68
-
}
69
-
});
70
-
71
-
Deno.test("removeStagedFile: rejects path outside staging dir", async () => {
72
-
const stagingDir = await Deno.makeTempDir();
73
-
const outsideDir = await Deno.makeTempDir();
74
-
const outsidePath = `${outsideDir}/secret`;
75
-
await Deno.writeTextFile(outsidePath, "sensitive");
76
-
try {
77
-
await assertRejects(
78
-
() => removeStagedFile(outsidePath, stagingDir),
79
-
Error,
80
-
"Refusing to delete file outside staging directory",
81
-
);
82
-
// File should still exist
83
-
const data = await Deno.readTextFile(outsidePath);
84
-
assertEquals(data, "sensitive");
85
-
} finally {
86
-
await Deno.remove(stagingDir, { recursive: true });
87
-
await Deno.remove(outsideDir, { recursive: true });
88
-
}
89
-
});
-321
cli/src/export/exporter.ts
-321
cli/src/export/exporter.ts
···
1
-
import { join } from "@std/path/join";
2
-
import { AbortError } from "../abort-error.ts";
3
-
4
-
/** Result of exporting a single asset via ladder. */
5
-
export interface ExportedAsset {
6
-
uuid: string;
7
-
path: string;
8
-
size: number;
9
-
sha256: string;
10
-
}
11
-
12
-
/** Error from ladder for a single asset. */
13
-
export interface ExportError {
14
-
uuid: string;
15
-
message: string;
16
-
}
17
-
18
-
/** Combined result from a ladder export batch. */
19
-
export interface ExportBatchResult {
20
-
results: ExportedAsset[];
21
-
errors: ExportError[];
22
-
}
23
-
24
-
/** Abstraction over the ladder binary for testability. */
25
-
export interface Exporter {
26
-
/** Export a batch of assets by UUID to the staging directory. */
27
-
exportBatch(
28
-
uuids: string[],
29
-
signal?: AbortSignal,
30
-
): Promise<ExportBatchResult>;
31
-
/** Hint to scale timeout for the next exportBatch call. Implementations may ignore. */
32
-
setEstimatedBatchBytes?(estimatedBytes: number): void;
33
-
/** Pre-flight check: verify ladder has required permissions (Photos, Automation). */
34
-
checkPermissions?(): Promise<void>;
35
-
}
36
-
37
-
/** Thrown when the ladder subprocess exceeds its timeout. */
38
-
export class LadderTimeoutError extends Error {
39
-
constructor(timeoutMs: number) {
40
-
super(`Ladder subprocess timed out after ${timeoutMs / 1000}s`);
41
-
this.name = "LadderTimeoutError";
42
-
}
43
-
}
44
-
45
-
/** Thrown when ladder reports a missing macOS permission (e.g. Automation). */
46
-
export class LadderPermissionError extends Error {
47
-
constructor(message: string) {
48
-
super(message);
49
-
this.name = "LadderPermissionError";
50
-
}
51
-
}
52
-
53
-
const DEFAULT_STAGING_DIR = join(
54
-
Deno.env.get("HOME") ?? "~",
55
-
".attic",
56
-
"staging",
57
-
);
58
-
59
-
/** Remove a staged file, ignoring NotFound errors. Path must be inside stagingDir. */
60
-
export async function removeStagedFile(
61
-
path: string,
62
-
stagingDir: string,
63
-
): Promise<void> {
64
-
// Resolve symlinks (e.g. /var -> /private/var on macOS) for both paths.
65
-
// For the file path, resolve its parent dir if the file itself doesn't exist.
66
-
const parentDir = path.substring(0, path.lastIndexOf("/"));
67
-
const fileName = path.substring(path.lastIndexOf("/") + 1);
68
-
const resolvedParent = await Deno.realPath(parentDir).catch(() => parentDir);
69
-
const resolvedPath = `${resolvedParent}/${fileName}`;
70
-
const resolvedDir = await Deno.realPath(stagingDir).catch(() => stagingDir);
71
-
if (
72
-
!resolvedPath.startsWith(resolvedDir + "/") && resolvedPath !== resolvedDir
73
-
) {
74
-
throw new Error(
75
-
`Refusing to delete file outside staging directory: ${path}`,
76
-
);
77
-
}
78
-
try {
79
-
await Deno.remove(path);
80
-
} catch (error: unknown) {
81
-
if (!(error instanceof Deno.errors.NotFound)) {
82
-
throw error;
83
-
}
84
-
}
85
-
}
86
-
87
-
/** Validate that ladder output conforms to ExportBatchResult shape. */
88
-
function assertExportBatchResult(
89
-
data: unknown,
90
-
): asserts data is ExportBatchResult {
91
-
if (data == null || typeof data !== "object") {
92
-
throw new Error("Ladder output is not an object");
93
-
}
94
-
const obj = data as Record<string, unknown>;
95
-
if (!Array.isArray(obj.results)) {
96
-
throw new Error("Ladder output missing 'results' array");
97
-
}
98
-
if (!Array.isArray(obj.errors)) {
99
-
throw new Error("Ladder output missing 'errors' array");
100
-
}
101
-
for (const r of obj.results) {
102
-
if (
103
-
typeof r !== "object" || r == null ||
104
-
typeof (r as Record<string, unknown>).uuid !== "string" ||
105
-
typeof (r as Record<string, unknown>).path !== "string" ||
106
-
typeof (r as Record<string, unknown>).size !== "number" ||
107
-
typeof (r as Record<string, unknown>).sha256 !== "string"
108
-
) {
109
-
throw new Error(
110
-
`Invalid result entry in ladder output: ${JSON.stringify(r)}`,
111
-
);
112
-
}
113
-
}
114
-
for (const e of obj.errors) {
115
-
if (
116
-
typeof e !== "object" || e == null ||
117
-
typeof (e as Record<string, unknown>).uuid !== "string" ||
118
-
typeof (e as Record<string, unknown>).message !== "string"
119
-
) {
120
-
throw new Error(
121
-
`Invalid error entry in ladder output: ${JSON.stringify(e)}`,
122
-
);
123
-
}
124
-
}
125
-
}
126
-
127
-
/** Strip the "/L0/001" suffix from a PhotoKit local identifier, returning the bare UUID. */
128
-
function stripLocalIdSuffix(id: string): string {
129
-
const slashIndex = id.indexOf("/");
130
-
return slashIndex === -1 ? id : id.substring(0, slashIndex);
131
-
}
132
-
133
-
/** Base timeout for the ladder subprocess (10 minutes).
134
-
* Matches ladder's per-asset AppleScript timeout so iCloud downloads
135
-
* don't get killed while still in progress. */
136
-
const LADDER_BASE_TIMEOUT_MS = 10 * 60 * 1000;
137
-
138
-
/** Extra timeout per 100 MB of estimated batch size (~1 min per 100 MB). */
139
-
const TIMEOUT_PER_100MB_MS = 60 * 1000;
140
-
141
-
/** Calculate timeout based on estimated batch size in bytes. */
142
-
export function timeoutForBytes(estimatedBytes: number): number {
143
-
const bytes = Math.max(0, estimatedBytes);
144
-
const extra = Math.ceil(bytes / (100 * 1024 * 1024)) * TIMEOUT_PER_100MB_MS;
145
-
return LADDER_BASE_TIMEOUT_MS + extra;
146
-
}
147
-
148
-
/** Options for creating a ladder exporter. */
149
-
export interface LadderExporterOptions {
150
-
stagingDir?: string;
151
-
/** Base timeout in ms (before size scaling). Defaults to 5 min. */
152
-
baseTimeoutMs?: number;
153
-
}
154
-
155
-
/** Spawn a single ladder process and return the parsed result. */
156
-
async function spawnLadder(
157
-
ladderPath: string,
158
-
uuids: string[],
159
-
stagingDir: string,
160
-
timeoutMs: number,
161
-
signal?: AbortSignal,
162
-
): Promise<ExportBatchResult> {
163
-
// PhotoKit expects local identifiers in "UUID/L0/001" format
164
-
const photoKitIds = uuids.map((uuid) => `${uuid}/L0/001`);
165
-
166
-
const request = JSON.stringify({
167
-
uuids: photoKitIds,
168
-
stagingDir,
169
-
});
170
-
171
-
const cmd = new Deno.Command(ladderPath, {
172
-
stdin: "piped",
173
-
stdout: "piped",
174
-
stderr: "piped",
175
-
});
176
-
177
-
const process = cmd.spawn();
178
-
179
-
const writer = process.stdin.getWriter();
180
-
await writer.write(new TextEncoder().encode(request));
181
-
await writer.close();
182
-
183
-
// Race the subprocess against timeout and abort signal
184
-
const result = await raceSubprocess(process, timeoutMs, signal);
185
-
186
-
if (result.code !== 0) {
187
-
const err = new TextDecoder().decode(result.stderr).trim();
188
-
// Exit code 77 = permission error (ladder convention)
189
-
if (result.code === 77) {
190
-
throw new LadderPermissionError(
191
-
err.replace(/^ladder:\s*/, ""),
192
-
);
193
-
}
194
-
throw new Error(`ladder exited with code ${result.code}: ${err}`);
195
-
}
196
-
197
-
const output = new TextDecoder().decode(result.stdout);
198
-
const parsed: unknown = JSON.parse(output);
199
-
assertExportBatchResult(parsed);
200
-
201
-
// Map PhotoKit identifiers ("UUID/L0/001") back to bare UUIDs
202
-
for (const r of parsed.results) {
203
-
r.uuid = stripLocalIdSuffix(r.uuid);
204
-
}
205
-
for (const e of parsed.errors) {
206
-
e.uuid = stripLocalIdSuffix(e.uuid);
207
-
}
208
-
209
-
return parsed;
210
-
}
211
-
212
-
/** Check whether an error is a ladder timeout. */
213
-
export function isTimeoutError(error: unknown): boolean {
214
-
return error instanceof LadderTimeoutError;
215
-
}
216
-
217
-
/** Check whether an error is a ladder permission issue. */
218
-
export function isPermissionError(error: unknown): boolean {
219
-
return error instanceof LadderPermissionError;
220
-
}
221
-
222
-
/** Create an exporter that shells out to the ladder binary. */
223
-
export function createLadderExporter(
224
-
ladderPath: string,
225
-
opts: LadderExporterOptions = {},
226
-
): Exporter & { stagingDir: string; setEstimatedBatchBytes(n: number): void } {
227
-
const stagingDir = opts.stagingDir ?? DEFAULT_STAGING_DIR;
228
-
const baseTimeoutMs = opts.baseTimeoutMs ?? LADDER_BASE_TIMEOUT_MS;
229
-
let currentTimeoutMs = baseTimeoutMs;
230
-
let stagingDirCreated = false;
231
-
232
-
return {
233
-
stagingDir,
234
-
235
-
setEstimatedBatchBytes(estimatedBytes: number) {
236
-
currentTimeoutMs = Math.max(
237
-
baseTimeoutMs,
238
-
timeoutForBytes(estimatedBytes),
239
-
);
240
-
},
241
-
242
-
async checkPermissions(): Promise<void> {
243
-
if (!stagingDirCreated) {
244
-
await Deno.mkdir(stagingDir, { recursive: true });
245
-
stagingDirCreated = true;
246
-
}
247
-
// Spawn ladder with an empty UUID list — triggers pre-flight
248
-
// permission checks without exporting anything.
249
-
await spawnLadder(ladderPath, [], stagingDir, 30_000);
250
-
},
251
-
252
-
async exportBatch(
253
-
uuids: string[],
254
-
signal?: AbortSignal,
255
-
): Promise<ExportBatchResult> {
256
-
if (!stagingDirCreated) {
257
-
await Deno.mkdir(stagingDir, { recursive: true });
258
-
stagingDirCreated = true;
259
-
}
260
-
return spawnLadder(
261
-
ladderPath,
262
-
uuids,
263
-
stagingDir,
264
-
currentTimeoutMs,
265
-
signal,
266
-
);
267
-
},
268
-
};
269
-
}
270
-
271
-
/** Race a subprocess against a timeout and optional abort signal.
272
-
* Kills the process on timeout or abort.
273
-
* Cleans up timer and listeners on completion to prevent leaks. */
274
-
export async function raceSubprocess(
275
-
process: Deno.ChildProcess,
276
-
timeoutMs: number,
277
-
signal?: AbortSignal,
278
-
): Promise<Deno.CommandOutput> {
279
-
signal?.throwIfAborted();
280
-
281
-
const outputPromise = process.output();
282
-
283
-
let removeAbortListener: (() => void) | undefined;
284
-
const abortPromise = signal
285
-
? new Promise<never>((_resolve, reject) => {
286
-
const onAbort = () => reject(new AbortError("Backup interrupted"));
287
-
if (signal.aborted) {
288
-
onAbort();
289
-
return;
290
-
}
291
-
signal.addEventListener("abort", onAbort, { once: true });
292
-
removeAbortListener = () => signal.removeEventListener("abort", onAbort);
293
-
})
294
-
: null;
295
-
296
-
let timeoutId: number | undefined;
297
-
const timeoutPromise = new Promise<never>((_resolve, reject) => {
298
-
timeoutId = setTimeout(() => {
299
-
reject(new LadderTimeoutError(timeoutMs));
300
-
}, timeoutMs);
301
-
Deno.unrefTimer(timeoutId);
302
-
});
303
-
304
-
const racers: Promise<Deno.CommandOutput>[] = [outputPromise, timeoutPromise];
305
-
if (abortPromise) racers.push(abortPromise);
306
-
307
-
try {
308
-
return await Promise.race(racers);
309
-
} catch (err) {
310
-
// Kill the subprocess on timeout or abort
311
-
try {
312
-
process.kill("SIGTERM");
313
-
} catch {
314
-
// Process may have already exited
315
-
}
316
-
throw err;
317
-
} finally {
318
-
clearTimeout(timeoutId);
319
-
removeAbortListener?.();
320
-
}
321
-
}
-7
cli/src/format.ts
-7
cli/src/format.ts
···
1
-
/** Format a byte count as a human-readable string. */
2
-
export function formatBytes(bytes: number): string {
3
-
if (bytes === 0) return "0 B";
4
-
const units = ["B", "KB", "MB", "GB", "TB"];
5
-
const i = Math.floor(Math.log(bytes) / Math.log(1024));
6
-
return `${(bytes / Math.pow(1024, i)).toFixed(1)} ${units[i]}`;
7
-
}
-65
cli/src/keychain/keychain.ts
-65
cli/src/keychain/keychain.ts
···
1
-
const ACCOUNT = "attic";
2
-
3
-
export interface KeychainCredentials {
4
-
accessKeyId: string;
5
-
secretAccessKey: string;
6
-
}
7
-
8
-
/** Read S3 credentials from macOS Keychain. */
9
-
export async function loadKeychainCredentials(
10
-
accessKeyService = "attic-s3-access-key",
11
-
secretKeyService = "attic-s3-secret-key",
12
-
): Promise<KeychainCredentials> {
13
-
const accessKeyId = await keychainGet(accessKeyService);
14
-
const secretAccessKey = await keychainGet(secretKeyService);
15
-
return { accessKeyId, secretAccessKey };
16
-
}
17
-
18
-
/** Store a credential in macOS Keychain. Uses -U to update if it already exists. */
19
-
export async function storeKeychainCredential(
20
-
service: string,
21
-
value: string,
22
-
): Promise<void> {
23
-
const cmd = new Deno.Command("security", {
24
-
args: [
25
-
"add-generic-password",
26
-
"-U",
27
-
"-s",
28
-
service,
29
-
"-a",
30
-
ACCOUNT,
31
-
"-w",
32
-
value,
33
-
],
34
-
stderr: "piped",
35
-
});
36
-
const { code, stderr } = await cmd.output();
37
-
if (code !== 0) {
38
-
const err = new TextDecoder().decode(stderr);
39
-
throw new Error(
40
-
`Failed to store credential in Keychain for service "${service}": ${err.trim()}`,
41
-
);
42
-
}
43
-
}
44
-
45
-
async function keychainGet(service: string): Promise<string> {
46
-
const cmd = new Deno.Command("security", {
47
-
args: [
48
-
"find-generic-password",
49
-
"-s",
50
-
service,
51
-
"-w",
52
-
],
53
-
stdout: "piped",
54
-
stderr: "piped",
55
-
});
56
-
const { code, stdout, stderr } = await cmd.output();
57
-
if (code !== 0) {
58
-
const err = new TextDecoder().decode(stderr);
59
-
throw new Error(
60
-
`Failed to read keychain item "${service}": ${err.trim()}. ` +
61
-
`Store it with: security add-generic-password -s ${service} -a ${ACCOUNT} -w "<value>"`,
62
-
);
63
-
}
64
-
return new TextDecoder().decode(stdout).trim();
65
-
}
-64
cli/src/logger.ts
-64
cli/src/logger.ts
···
1
-
/**
2
-
* Structured JSONL logger for unattended backup runs.
3
-
* Each line is a self-contained JSON object with an event type and timestamp.
4
-
*/
5
-
6
-
export interface BackupLogger {
7
-
start(pending: number, photos: number, videos: number): void;
8
-
uploaded(uuid: string, filename: string, type: string, size: number): void;
9
-
error(uuid: string, message: string): void;
10
-
complete(uploaded: number, failed: number, totalBytes: number): void;
11
-
interrupted(uploaded: number, pending: number, totalBytes: number): void;
12
-
close(): void;
13
-
}
14
-
15
-
function makeEntry(event: string, data: Record<string, unknown>): string {
16
-
return JSON.stringify({
17
-
event,
18
-
...data,
19
-
timestamp: new Date().toISOString(),
20
-
});
21
-
}
22
-
23
-
/** Create a logger that appends JSONL to the given file path. */
24
-
export function createFileLogger(path: string): BackupLogger {
25
-
const file = Deno.openSync(path, { write: true, create: true, append: true });
26
-
const encoder = new TextEncoder();
27
-
28
-
const write = (line: string) => {
29
-
file.writeSync(encoder.encode(line + "\n"));
30
-
};
31
-
32
-
return {
33
-
start(pending, photos, videos) {
34
-
write(makeEntry("start", { pending, photos, videos }));
35
-
},
36
-
uploaded(uuid, filename, type, size) {
37
-
write(makeEntry("uploaded", { uuid, filename, type, size }));
38
-
},
39
-
error(uuid, message) {
40
-
write(makeEntry("error", { uuid, message }));
41
-
},
42
-
complete(uploaded, failed, totalBytes) {
43
-
write(makeEntry("complete", { uploaded, failed, totalBytes }));
44
-
},
45
-
interrupted(uploaded, pending, totalBytes) {
46
-
write(makeEntry("interrupted", { uploaded, pending, totalBytes }));
47
-
},
48
-
close() {
49
-
file.close();
50
-
},
51
-
};
52
-
}
53
-
54
-
/** No-op logger for when --log is not specified. */
55
-
export function createNullLogger(): BackupLogger {
56
-
return {
57
-
start() {},
58
-
uploaded() {},
59
-
error() {},
60
-
complete() {},
61
-
interrupted() {},
62
-
close() {},
63
-
};
64
-
}
-173
cli/src/manifest/manifest.test.ts
-173
cli/src/manifest/manifest.test.ts
···
1
-
import { assertEquals, assertRejects } from "@std/assert";
2
-
import type { Manifest } from "./manifest.ts";
3
-
import {
4
-
createS3ManifestStore,
5
-
isBackedUp,
6
-
loadManifestWithMigration,
7
-
markBackedUp,
8
-
} from "./manifest.ts";
9
-
import { createMockS3Provider } from "../storage/s3-client.mock.ts";
10
-
11
-
// --- Core functions ---
12
-
13
-
Deno.test("isBackedUp checks correctly", () => {
14
-
const manifest: Manifest = { entries: {} };
15
-
16
-
assertEquals(isBackedUp(manifest, "uuid-1"), false);
17
-
18
-
markBackedUp(
19
-
manifest,
20
-
"uuid-1",
21
-
"sha256:abc",
22
-
"originals/2024/01/uuid-1.heic",
23
-
);
24
-
25
-
assertEquals(isBackedUp(manifest, "uuid-1"), true);
26
-
assertEquals(isBackedUp(manifest, "uuid-2"), false);
27
-
});
28
-
29
-
// --- S3 manifest store ---
30
-
31
-
Deno.test("S3 store: load returns empty manifest when key missing", async () => {
32
-
const s3 = createMockS3Provider();
33
-
const store = createS3ManifestStore(s3);
34
-
const manifest = await store.load();
35
-
assertEquals(manifest.entries, {});
36
-
});
37
-
38
-
Deno.test("S3 store: save and load round-trip", async () => {
39
-
const s3 = createMockS3Provider();
40
-
const store = createS3ManifestStore(s3);
41
-
const manifest = { entries: {} } as Manifest;
42
-
markBackedUp(
43
-
manifest,
44
-
"uuid-1",
45
-
"sha256:abc",
46
-
"originals/2024/01/uuid-1.heic",
47
-
);
48
-
await store.save(manifest);
49
-
50
-
const loaded = await store.load();
51
-
assertEquals(isBackedUp(loaded, "uuid-1"), true);
52
-
assertEquals(
53
-
loaded.entries["uuid-1"].s3Key,
54
-
"originals/2024/01/uuid-1.heic",
55
-
);
56
-
});
57
-
58
-
Deno.test("S3 store: load rejects invalid JSON", async () => {
59
-
const s3 = createMockS3Provider();
60
-
await s3.putObject(
61
-
"manifest.json",
62
-
new TextEncoder().encode('{"bad": true}'),
63
-
);
64
-
const store = createS3ManifestStore(s3);
65
-
await assertRejects(
66
-
() => store.load(),
67
-
Error,
68
-
"missing or invalid 'entries'",
69
-
);
70
-
});
71
-
72
-
Deno.test("S3 store: saves with correct content type", async () => {
73
-
const s3 = createMockS3Provider();
74
-
const store = createS3ManifestStore(s3);
75
-
await store.save({ entries: {} });
76
-
77
-
const obj = s3.objects.get("manifest.json");
78
-
assertEquals(obj?.contentType, "application/json");
79
-
});
80
-
81
-
// --- Migration ---
82
-
83
-
Deno.test("migration: uses S3 manifest when present", async () => {
84
-
const s3 = createMockS3Provider();
85
-
const store = createS3ManifestStore(s3);
86
-
const existing = { entries: {} } as Manifest;
87
-
markBackedUp(existing, "s3-uuid", "sha256:s3", "originals/2024/01/s3.heic");
88
-
await store.save(existing);
89
-
90
-
const manifest = await loadManifestWithMigration(store, "/nonexistent");
91
-
assertEquals(isBackedUp(manifest, "s3-uuid"), true);
92
-
});
93
-
94
-
Deno.test("migration: migrates local manifest to S3", async () => {
95
-
const dir = await Deno.makeTempDir();
96
-
try {
97
-
// Write a local manifest file directly
98
-
const localManifest = {
99
-
entries: {
100
-
"local-uuid": {
101
-
uuid: "local-uuid",
102
-
s3Key: "originals/2024/01/local.heic",
103
-
checksum: "sha256:local",
104
-
backedUpAt: "2024-01-15T00:00:00Z",
105
-
},
106
-
},
107
-
};
108
-
await Deno.writeTextFile(
109
-
`${dir}/manifest.json`,
110
-
JSON.stringify(localManifest, null, 2),
111
-
);
112
-
113
-
// S3 is empty
114
-
const s3 = createMockS3Provider();
115
-
const s3Store = createS3ManifestStore(s3);
116
-
117
-
const manifest = await loadManifestWithMigration(s3Store, dir);
118
-
assertEquals(isBackedUp(manifest, "local-uuid"), true);
119
-
120
-
// Verify it was uploaded to S3
121
-
const s3Manifest = await s3Store.load();
122
-
assertEquals(isBackedUp(s3Manifest, "local-uuid"), true);
123
-
} finally {
124
-
await Deno.remove(dir, { recursive: true });
125
-
}
126
-
});
127
-
128
-
Deno.test("migration: returns empty when neither exists", async () => {
129
-
const s3 = createMockS3Provider();
130
-
const store = createS3ManifestStore(s3);
131
-
const manifest = await loadManifestWithMigration(store, "/nonexistent");
132
-
assertEquals(Object.keys(manifest.entries).length, 0);
133
-
});
134
-
135
-
Deno.test("migration: S3 takes precedence over local", async () => {
136
-
const dir = await Deno.makeTempDir();
137
-
try {
138
-
// Local manifest has one entry
139
-
const localManifest = {
140
-
entries: {
141
-
"local-uuid": {
142
-
uuid: "local-uuid",
143
-
s3Key: "originals/2024/01/local.heic",
144
-
checksum: "sha256:local",
145
-
backedUpAt: "2024-01-15T00:00:00Z",
146
-
},
147
-
},
148
-
};
149
-
await Deno.writeTextFile(
150
-
`${dir}/manifest.json`,
151
-
JSON.stringify(localManifest, null, 2),
152
-
);
153
-
154
-
// S3 has a different entry
155
-
const s3 = createMockS3Provider();
156
-
const s3Store = createS3ManifestStore(s3);
157
-
const s3Manifest = { entries: {} } as Manifest;
158
-
markBackedUp(
159
-
s3Manifest,
160
-
"s3-uuid",
161
-
"sha256:s3",
162
-
"originals/2024/01/s3.heic",
163
-
);
164
-
await s3Store.save(s3Manifest);
165
-
166
-
// S3 should win — local is not consulted
167
-
const manifest = await loadManifestWithMigration(s3Store, dir);
168
-
assertEquals(isBackedUp(manifest, "s3-uuid"), true);
169
-
assertEquals(isBackedUp(manifest, "local-uuid"), false);
170
-
} finally {
171
-
await Deno.remove(dir, { recursive: true });
172
-
}
173
-
});
-134
cli/src/manifest/manifest.ts
-134
cli/src/manifest/manifest.ts
···
1
-
import { join } from "@std/path/join";
2
-
import type { S3Provider } from "../storage/s3-client.ts";
3
-
4
-
export interface ManifestEntry {
5
-
uuid: string;
6
-
s3Key: string;
7
-
checksum: string;
8
-
backedUpAt: string;
9
-
size?: number;
10
-
}
11
-
12
-
export interface Manifest {
13
-
entries: Record<string, ManifestEntry>;
14
-
}
15
-
16
-
export interface ManifestStore {
17
-
load(): Promise<Manifest>;
18
-
save(manifest: Manifest): Promise<void>;
19
-
}
20
-
21
-
/** S3 key where the shared manifest is stored. */
22
-
export const MANIFEST_S3_KEY = "manifest.json";
23
-
24
-
/** Check whether an asset has been backed up. */
25
-
export function isBackedUp(manifest: Manifest, uuid: string): boolean {
26
-
return uuid in manifest.entries;
27
-
}
28
-
29
-
/** Mark an asset as backed up (mutates in place). */
30
-
export function markBackedUp(
31
-
manifest: Manifest,
32
-
uuid: string,
33
-
checksum: string,
34
-
s3Key: string,
35
-
size?: number,
36
-
backedUpAt?: string,
37
-
): void {
38
-
manifest.entries[uuid] = {
39
-
uuid,
40
-
s3Key,
41
-
checksum,
42
-
backedUpAt: backedUpAt ?? new Date().toISOString(),
43
-
...(size != null ? { size } : {}),
44
-
};
45
-
}
46
-
47
-
function assertManifest(data: unknown): asserts data is Manifest {
48
-
if (typeof data !== "object" || data === null) {
49
-
throw new Error("Invalid manifest file: expected a JSON object");
50
-
}
51
-
const obj = data as Record<string, unknown>;
52
-
if (typeof obj.entries !== "object" || obj.entries === null) {
53
-
throw new Error("Invalid manifest file: missing or invalid 'entries'");
54
-
}
55
-
}
56
-
57
-
/** Create a manifest store backed by S3. This is the primary store. */
58
-
export function createS3ManifestStore(
59
-
s3: S3Provider,
60
-
key: string = MANIFEST_S3_KEY,
61
-
): ManifestStore {
62
-
return {
63
-
async load(): Promise<Manifest> {
64
-
try {
65
-
const data = await s3.getObject(key);
66
-
const text = new TextDecoder().decode(data);
67
-
const parsed: unknown = JSON.parse(text);
68
-
assertManifest(parsed);
69
-
return parsed;
70
-
} catch (error: unknown) {
71
-
if (isNotFoundError(error)) {
72
-
return { entries: {} };
73
-
}
74
-
throw error;
75
-
}
76
-
},
77
-
78
-
async save(manifest: Manifest): Promise<void> {
79
-
const json = JSON.stringify(manifest, null, 2) + "\n";
80
-
const data = new TextEncoder().encode(json);
81
-
await s3.putObject(key, data, "application/json");
82
-
},
83
-
};
84
-
}
85
-
86
-
function isNotFoundError(error: unknown): boolean {
87
-
if (!(error instanceof Error)) return false;
88
-
return error.name === "NoSuchKey" || error.name === "NotFound";
89
-
}
90
-
91
-
/** Load manifest from S3, migrating from local file if needed.
92
-
*
93
-
* Migration flow (one-time):
94
-
* 1. If S3 has a manifest with entries, use it.
95
-
* 2. If S3 is empty, check for a local manifest at ~/.attic/manifest.json.
96
-
* 3. If local exists, upload it to S3 and return it.
97
-
* 4. If neither exists, return empty manifest.
98
-
*/
99
-
export async function loadManifestWithMigration(
100
-
s3Store: ManifestStore,
101
-
localDir?: string,
102
-
): Promise<Manifest> {
103
-
const s3Manifest = await s3Store.load();
104
-
105
-
if (Object.keys(s3Manifest.entries).length > 0) {
106
-
return s3Manifest;
107
-
}
108
-
109
-
// Check for local manifest to migrate
110
-
const dir = localDir ??
111
-
join(Deno.env.get("HOME") ?? "~", ".attic");
112
-
const localPath = join(dir, "manifest.json");
113
-
114
-
try {
115
-
const text = await Deno.readTextFile(localPath);
116
-
const data: unknown = JSON.parse(text);
117
-
assertManifest(data);
118
-
119
-
if (Object.keys(data.entries).length > 0) {
120
-
console.log(
121
-
` Migrating local manifest (${
122
-
Object.keys(data.entries).length
123
-
} entries) to S3...`,
124
-
);
125
-
await s3Store.save(data);
126
-
console.log(` Migration complete.\n`);
127
-
return data;
128
-
}
129
-
} catch {
130
-
// No local manifest or unreadable — that's fine
131
-
}
132
-
133
-
return s3Manifest;
134
-
}
-37
cli/src/notify.ts
-37
cli/src/notify.ts
···
1
-
/**
2
-
* macOS notifications via osascript.
3
-
* Falls back silently if osascript is unavailable.
4
-
*/
5
-
6
-
export async function notify(
7
-
title: string,
8
-
message: string,
9
-
sound: string = "default",
10
-
): Promise<void> {
11
-
try {
12
-
const cmd = new Deno.Command("osascript", {
13
-
args: [
14
-
"-e",
15
-
`display notification "${escapeAppleScript(message)}" with title "${
16
-
escapeAppleScript(title)
17
-
}" sound name "${escapeAppleScript(sound)}"`,
18
-
],
19
-
stdout: "null",
20
-
stderr: "null",
21
-
});
22
-
const { success } = await cmd.output();
23
-
if (!success) {
24
-
// Silently ignore — notifications are best-effort
25
-
}
26
-
} catch {
27
-
// osascript not available or other error — skip silently
28
-
}
29
-
}
30
-
31
-
function escapeAppleScript(s: string): string {
32
-
return s
33
-
.replace(/\\/g, "\\\\")
34
-
.replace(/"/g, '\\"')
35
-
.replace(/\n/g, " ")
36
-
.replace(/\r/g, "");
37
-
}
-331
cli/src/photos-db/reader.test.ts
-331
cli/src/photos-db/reader.test.ts
···
1
-
import { assertEquals } from "@std/assert";
2
-
import { Database } from "@db/sqlite";
3
-
import { coreDataTimestampToDate, openPhotosDb } from "./reader.ts";
4
-
import { AssetKind, CloudLocalState } from "@attic/shared";
5
-
6
-
function createTestDb(opts?: { withEnrichment?: boolean }): string {
7
-
const path = Deno.makeTempFileSync({ suffix: ".sqlite" });
8
-
const db = new Database(path);
9
-
10
-
db.exec(`
11
-
CREATE TABLE ZASSET (
12
-
Z_PK INTEGER PRIMARY KEY,
13
-
ZUUID TEXT,
14
-
ZFILENAME TEXT,
15
-
ZDIRECTORY TEXT,
16
-
ZDATECREATED REAL,
17
-
ZKIND INTEGER,
18
-
ZUNIFORMTYPEIDENTIFIER TEXT,
19
-
ZWIDTH INTEGER,
20
-
ZHEIGHT INTEGER,
21
-
ZLATITUDE REAL,
22
-
ZLONGITUDE REAL,
23
-
ZFAVORITE INTEGER,
24
-
ZCLOUDLOCALSTATE INTEGER,
25
-
ZTRASHEDSTATE INTEGER DEFAULT 0
26
-
);
27
-
28
-
CREATE TABLE ZADDITIONALASSETATTRIBUTES (
29
-
Z_PK INTEGER PRIMARY KEY,
30
-
ZASSET INTEGER,
31
-
ZORIGINALFILESIZE INTEGER,
32
-
ZORIGINALFILENAME TEXT,
33
-
ZORIGINALSTABLEHASH TEXT,
34
-
ZTITLE TEXT,
35
-
ZUNMANAGEDADJUSTMENT INTEGER
36
-
);
37
-
`);
38
-
39
-
// 2024-01-15 12:00:00 UTC as CoreData timestamp
40
-
// Unix: 1705320000 - CoreData epoch offset 978307200 = 727012800
41
-
const coreDataTs = 727012800;
42
-
43
-
db.exec(`
44
-
INSERT INTO ZASSET (Z_PK, ZUUID, ZFILENAME, ZDIRECTORY, ZDATECREATED, ZKIND,
45
-
ZUNIFORMTYPEIDENTIFIER, ZWIDTH, ZHEIGHT, ZLATITUDE, ZLONGITUDE, ZFAVORITE,
46
-
ZCLOUDLOCALSTATE, ZTRASHEDSTATE)
47
-
VALUES
48
-
(1, 'uuid-photo-1', 'IMG_0001.HEIC', '/some/dir', ${coreDataTs}, ${AssetKind.PHOTO},
49
-
'public.heic', 4032, 3024, 52.09, 4.34, 1, ${CloudLocalState.LOCAL}, 0),
50
-
(2, 'uuid-video-1', 'IMG_0002.MOV', '/some/dir', ${
51
-
coreDataTs + 3600
52
-
}, ${AssetKind.VIDEO},
53
-
'com.apple.quicktime-movie', 1920, 1080, NULL, NULL, 0, ${CloudLocalState.ICLOUD_ONLY}, 0),
54
-
(3, 'uuid-trashed', 'IMG_0003.HEIC', '/some/dir', ${coreDataTs}, ${AssetKind.PHOTO},
55
-
'public.heic', 4032, 3024, NULL, NULL, 0, ${CloudLocalState.LOCAL}, 1);
56
-
57
-
INSERT INTO ZADDITIONALASSETATTRIBUTES (Z_PK, ZASSET, ZORIGINALFILESIZE, ZORIGINALFILENAME, ZORIGINALSTABLEHASH, ZTITLE)
58
-
VALUES
59
-
(1, 1, 3158112, 'IMG_0001.HEIC', 'abc123', 'Sunset at the beach'),
60
-
(2, 2, 52428800, 'IMG_0002.MOV', 'def456', NULL),
61
-
(3, 3, 1000000, 'IMG_0003.HEIC', 'ghi789', NULL);
62
-
`);
63
-
64
-
if (opts?.withEnrichment !== false) {
65
-
db.exec(`
66
-
CREATE TABLE ZASSETDESCRIPTION (
67
-
Z_PK INTEGER PRIMARY KEY,
68
-
ZASSETATTRIBUTES INTEGER,
69
-
ZLONGDESCRIPTION TEXT
70
-
);
71
-
72
-
CREATE TABLE ZGENERICALBUM (
73
-
Z_PK INTEGER PRIMARY KEY,
74
-
ZUUID TEXT,
75
-
ZTITLE TEXT
76
-
);
77
-
78
-
CREATE TABLE Z_33ASSETS (
79
-
Z_3ASSETS INTEGER,
80
-
Z_33ALBUMS INTEGER
81
-
);
82
-
83
-
CREATE TABLE ZKEYWORD (
84
-
Z_PK INTEGER PRIMARY KEY,
85
-
ZTITLE TEXT
86
-
);
87
-
88
-
CREATE TABLE Z_1KEYWORDS (
89
-
Z_1ASSETATTRIBUTES INTEGER,
90
-
Z_52KEYWORDS INTEGER
91
-
);
92
-
93
-
CREATE TABLE ZPERSON (
94
-
Z_PK INTEGER PRIMARY KEY,
95
-
ZPERSONUUID TEXT,
96
-
ZDISPLAYNAME TEXT
97
-
);
98
-
99
-
CREATE TABLE ZDETECTEDFACE (
100
-
Z_PK INTEGER PRIMARY KEY,
101
-
ZASSETFORFACE INTEGER,
102
-
ZPERSONFORFACE INTEGER,
103
-
ZHIDDEN INTEGER DEFAULT 0,
104
-
ZASSETVISIBLE INTEGER DEFAULT 1
105
-
);
106
-
107
-
CREATE TABLE ZUNMANAGEDADJUSTMENT (
108
-
Z_PK INTEGER PRIMARY KEY,
109
-
ZADJUSTMENTTIMESTAMP REAL,
110
-
ZADJUSTMENTFORMATIDENTIFIER TEXT
111
-
);
112
-
113
-
CREATE TABLE ZINTERNALRESOURCE (
114
-
Z_PK INTEGER PRIMARY KEY,
115
-
ZASSET INTEGER,
116
-
ZRESOURCETYPE INTEGER,
117
-
ZTRASHEDSTATE INTEGER DEFAULT 0,
118
-
ZVERSION INTEGER DEFAULT 0,
119
-
ZDATALENGTH INTEGER,
120
-
ZCOMPACTUTI TEXT,
121
-
ZLOCALAVAILABILITY INTEGER DEFAULT 1
122
-
);
123
-
`);
124
-
125
-
if (opts?.withEnrichment) {
126
-
// Description for photo asset (aa.Z_PK = 1, aa.ZASSET = 1)
127
-
db.exec(`
128
-
INSERT INTO ZASSETDESCRIPTION (Z_PK, ZASSETATTRIBUTES, ZLONGDESCRIPTION)
129
-
VALUES (1, 1, 'A beautiful sunset over the ocean');
130
-
`);
131
-
132
-
// Albums
133
-
db.exec(`
134
-
INSERT INTO ZGENERICALBUM (Z_PK, ZUUID, ZTITLE)
135
-
VALUES (1, 'album-uuid-1', 'Vacation 2024'),
136
-
(2, 'album-uuid-2', 'Favorites');
137
-
138
-
INSERT INTO Z_33ASSETS (Z_3ASSETS, Z_33ALBUMS)
139
-
VALUES (1, 1), (1, 2);
140
-
`);
141
-
142
-
// Keywords
143
-
db.exec(`
144
-
INSERT INTO ZKEYWORD (Z_PK, ZTITLE)
145
-
VALUES (1, 'sunset'), (2, 'ocean');
146
-
147
-
INSERT INTO Z_1KEYWORDS (Z_1ASSETATTRIBUTES, Z_52KEYWORDS)
148
-
VALUES (1, 1), (1, 2);
149
-
`);
150
-
151
-
// People
152
-
db.exec(`
153
-
INSERT INTO ZPERSON (Z_PK, ZPERSONUUID, ZDISPLAYNAME)
154
-
VALUES (1, 'person-uuid-1', 'Alice');
155
-
156
-
INSERT INTO ZDETECTEDFACE (Z_PK, ZASSETFORFACE, ZPERSONFORFACE, ZHIDDEN, ZASSETVISIBLE)
157
-
VALUES (1, 1, 1, 0, 1);
158
-
`);
159
-
160
-
// Edit data: photo asset (Z_PK=1) has an edit
161
-
const editTs = 727012800 + 7200; // 2 hours after creation
162
-
db.exec(`
163
-
INSERT INTO ZUNMANAGEDADJUSTMENT (Z_PK, ZADJUSTMENTTIMESTAMP, ZADJUSTMENTFORMATIDENTIFIER)
164
-
VALUES (1, ${editTs}, 'com.apple.photo');
165
-
166
-
UPDATE ZADDITIONALASSETATTRIBUTES SET ZUNMANAGEDADJUSTMENT = 1 WHERE Z_PK = 1;
167
-
168
-
INSERT INTO ZINTERNALRESOURCE (Z_PK, ZASSET, ZRESOURCETYPE, ZTRASHEDSTATE, ZVERSION, ZDATALENGTH, ZCOMPACTUTI, ZLOCALAVAILABILITY)
169
-
VALUES (1, 1, 1, 0, 1, 2048000, 'public.heic', 1);
170
-
`);
171
-
}
172
-
}
173
-
174
-
db.close();
175
-
return path;
176
-
}
177
-
178
-
Deno.test("coreDataTimestampToDate converts correctly", () => {
179
-
// 2024-01-15 12:00:00 UTC = unix 1705320000 - 978307200 = 727012800
180
-
const date = coreDataTimestampToDate(727012800);
181
-
assertEquals(date?.toISOString(), "2024-01-15T12:00:00.000Z");
182
-
});
183
-
184
-
Deno.test("coreDataTimestampToDate returns null for null", () => {
185
-
assertEquals(coreDataTimestampToDate(null), null);
186
-
});
187
-
188
-
Deno.test("readAssets returns non-trashed assets with correct fields", () => {
189
-
const dbPath = createTestDb({ withEnrichment: true });
190
-
try {
191
-
const reader = openPhotosDb(dbPath);
192
-
const assets = reader.readAssets();
193
-
reader.close();
194
-
195
-
assertEquals(assets.length, 2, "should exclude trashed asset");
196
-
197
-
const photo = assets.find((a) => a.uuid === "uuid-photo-1")!;
198
-
assertEquals(photo.filename, "IMG_0001.HEIC");
199
-
assertEquals(photo.originalFilename, "IMG_0001.HEIC");
200
-
assertEquals(photo.kind, AssetKind.PHOTO);
201
-
assertEquals(photo.uniformTypeIdentifier, "public.heic");
202
-
assertEquals(photo.width, 4032);
203
-
assertEquals(photo.height, 3024);
204
-
assertEquals(photo.latitude, 52.09);
205
-
assertEquals(photo.longitude, 4.34);
206
-
assertEquals(photo.favorite, true);
207
-
assertEquals(photo.cloudLocalState, CloudLocalState.LOCAL);
208
-
assertEquals(photo.originalFileSize, 3158112);
209
-
assertEquals(photo.originalStableHash, "abc123");
210
-
assertEquals(
211
-
photo.dateCreated?.toISOString(),
212
-
"2024-01-15T12:00:00.000Z",
213
-
);
214
-
215
-
// Enrichment fields
216
-
assertEquals(photo.title, "Sunset at the beach");
217
-
assertEquals(photo.description, "A beautiful sunset over the ocean");
218
-
assertEquals(photo.albums.length, 2);
219
-
assertEquals(photo.albums[0].title, "Vacation 2024");
220
-
assertEquals(photo.albums[1].title, "Favorites");
221
-
assertEquals(photo.keywords, ["sunset", "ocean"]);
222
-
assertEquals(photo.people.length, 1);
223
-
assertEquals(photo.people[0].displayName, "Alice");
224
-
225
-
// Edit fields
226
-
assertEquals(photo.hasEdit, true);
227
-
assertEquals(photo.editedAt?.toISOString(), "2024-01-15T14:00:00.000Z");
228
-
assertEquals(photo.editor, "com.apple.photo");
229
-
230
-
const video = assets.find((a) => a.uuid === "uuid-video-1")!;
231
-
assertEquals(video.kind, AssetKind.VIDEO);
232
-
assertEquals(video.latitude, null);
233
-
assertEquals(video.longitude, null);
234
-
assertEquals(video.favorite, false);
235
-
assertEquals(video.cloudLocalState, CloudLocalState.ICLOUD_ONLY);
236
-
assertEquals(video.originalFileSize, 52428800);
237
-
238
-
// Video has no enrichment data
239
-
assertEquals(video.title, null);
240
-
assertEquals(video.description, null);
241
-
assertEquals(video.albums, []);
242
-
assertEquals(video.keywords, []);
243
-
assertEquals(video.people, []);
244
-
245
-
// Video has no edit
246
-
assertEquals(video.hasEdit, false);
247
-
assertEquals(video.editedAt, null);
248
-
assertEquals(video.editor, null);
249
-
} finally {
250
-
Deno.removeSync(dbPath);
251
-
}
252
-
});
253
-
254
-
Deno.test("readAssets works without enrichment tables (schema resilience)", () => {
255
-
const dbPath = createTestDb({ withEnrichment: false });
256
-
try {
257
-
const reader = openPhotosDb(dbPath);
258
-
const assets = reader.readAssets();
259
-
reader.close();
260
-
261
-
assertEquals(assets.length, 2, "should return both non-trashed assets");
262
-
263
-
const photo = assets.find((a) => a.uuid === "uuid-photo-1")!;
264
-
assertEquals(photo.title, "Sunset at the beach");
265
-
assertEquals(photo.description, null);
266
-
assertEquals(photo.albums, []);
267
-
assertEquals(photo.keywords, []);
268
-
assertEquals(photo.people, []);
269
-
assertEquals(photo.hasEdit, false);
270
-
assertEquals(photo.editedAt, null);
271
-
assertEquals(photo.editor, null);
272
-
} finally {
273
-
Deno.removeSync(dbPath);
274
-
}
275
-
});
276
-
277
-
Deno.test("readAssets: adjustment without rendered resource yields hasEdit false", () => {
278
-
const path = Deno.makeTempFileSync({ suffix: ".sqlite" });
279
-
const db = new Database(path);
280
-
281
-
const coreDataTs = 727012800;
282
-
283
-
db.exec(`
284
-
CREATE TABLE ZASSET (
285
-
Z_PK INTEGER PRIMARY KEY, ZUUID TEXT, ZFILENAME TEXT, ZDIRECTORY TEXT,
286
-
ZDATECREATED REAL, ZKIND INTEGER, ZUNIFORMTYPEIDENTIFIER TEXT,
287
-
ZWIDTH INTEGER, ZHEIGHT INTEGER, ZLATITUDE REAL, ZLONGITUDE REAL,
288
-
ZFAVORITE INTEGER, ZCLOUDLOCALSTATE INTEGER, ZTRASHEDSTATE INTEGER DEFAULT 0
289
-
);
290
-
CREATE TABLE ZADDITIONALASSETATTRIBUTES (
291
-
Z_PK INTEGER PRIMARY KEY, ZASSET INTEGER, ZORIGINALFILESIZE INTEGER,
292
-
ZORIGINALFILENAME TEXT, ZORIGINALSTABLEHASH TEXT, ZTITLE TEXT,
293
-
ZUNMANAGEDADJUSTMENT INTEGER
294
-
);
295
-
CREATE TABLE ZUNMANAGEDADJUSTMENT (
296
-
Z_PK INTEGER PRIMARY KEY, ZADJUSTMENTTIMESTAMP REAL,
297
-
ZADJUSTMENTFORMATIDENTIFIER TEXT
298
-
);
299
-
CREATE TABLE ZINTERNALRESOURCE (
300
-
Z_PK INTEGER PRIMARY KEY, ZASSET INTEGER, ZRESOURCETYPE INTEGER,
301
-
ZTRASHEDSTATE INTEGER DEFAULT 0, ZVERSION INTEGER DEFAULT 0,
302
-
ZDATALENGTH INTEGER, ZCOMPACTUTI TEXT, ZLOCALAVAILABILITY INTEGER DEFAULT 1
303
-
);
304
-
305
-
INSERT INTO ZASSET VALUES (1, 'uuid-adj-only', 'IMG.HEIC', '/dir', ${coreDataTs},
306
-
${AssetKind.PHOTO}, 'public.heic', 4032, 3024, NULL, NULL, 0, ${CloudLocalState.LOCAL}, 0);
307
-
INSERT INTO ZADDITIONALASSETATTRIBUTES VALUES (1, 1, 1000, 'IMG.HEIC', 'hash', NULL, 1);
308
-
INSERT INTO ZUNMANAGEDADJUSTMENT VALUES (1, ${
309
-
coreDataTs + 100
310
-
}, 'com.apple.photo');
311
-
`);
312
-
// No ZINTERNALRESOURCE row — adjustment exists but no rendered file
313
-
db.close();
314
-
315
-
try {
316
-
const reader = openPhotosDb(path);
317
-
const assets = reader.readAssets();
318
-
reader.close();
319
-
320
-
assertEquals(assets.length, 1);
321
-
assertEquals(
322
-
assets[0].hasEdit,
323
-
false,
324
-
"no rendered resource = hasEdit false",
325
-
);
326
-
assertEquals(assets[0].editedAt, null);
327
-
assertEquals(assets[0].editor, null);
328
-
} finally {
329
-
Deno.removeSync(path);
330
-
}
331
-
});
-328
cli/src/photos-db/reader.ts
-328
cli/src/photos-db/reader.ts
···
1
-
import { Database } from "@db/sqlite";
2
-
import type {
3
-
AlbumRef,
4
-
AssetKindValue,
5
-
CloudLocalStateValue,
6
-
PersonRef,
7
-
PhotoAsset,
8
-
} from "@attic/shared";
9
-
import { AssetKind, CloudLocalState } from "@attic/shared";
10
-
11
-
const DEFAULT_DB_PATH = `${
12
-
Deno.env.get("HOME")
13
-
}/Pictures/Photos Library.photoslibrary/database/Photos.sqlite`;
14
-
15
-
const ASSETS_QUERY = `
16
-
SELECT
17
-
a.Z_PK,
18
-
a.ZUUID,
19
-
a.ZFILENAME,
20
-
a.ZDIRECTORY,
21
-
a.ZDATECREATED,
22
-
a.ZKIND,
23
-
a.ZUNIFORMTYPEIDENTIFIER,
24
-
a.ZWIDTH,
25
-
a.ZHEIGHT,
26
-
a.ZLATITUDE,
27
-
a.ZLONGITUDE,
28
-
a.ZFAVORITE,
29
-
a.ZCLOUDLOCALSTATE,
30
-
aa.ZORIGINALFILESIZE,
31
-
aa.ZORIGINALFILENAME,
32
-
aa.ZORIGINALSTABLEHASH,
33
-
aa.ZTITLE
34
-
FROM ZASSET a
35
-
JOIN ZADDITIONALASSETATTRIBUTES aa ON aa.ZASSET = a.Z_PK
36
-
WHERE a.ZTRASHEDSTATE = 0
37
-
`;
38
-
39
-
/** CoreData epoch (2001-01-01) offset from Unix epoch in seconds. */
40
-
const CORE_DATA_EPOCH_OFFSET = 978307200;
41
-
42
-
/** Convert CoreData timestamp to JS Date. */
43
-
export function coreDataTimestampToDate(
44
-
timestamp: number | null | undefined,
45
-
): Date | null {
46
-
if (timestamp == null) return null;
47
-
return new Date((timestamp + CORE_DATA_EPOCH_OFFSET) * 1000);
48
-
}
49
-
50
-
interface RawRow {
51
-
Z_PK: number;
52
-
ZUUID: string;
53
-
ZFILENAME: string | null;
54
-
ZDIRECTORY: string | null;
55
-
ZDATECREATED: number | null;
56
-
ZKIND: number;
57
-
ZUNIFORMTYPEIDENTIFIER: string | null;
58
-
ZWIDTH: number;
59
-
ZHEIGHT: number;
60
-
ZLATITUDE: number | null;
61
-
ZLONGITUDE: number | null;
62
-
ZFAVORITE: number;
63
-
ZCLOUDLOCALSTATE: number;
64
-
ZORIGINALFILESIZE: number | null;
65
-
ZORIGINALFILENAME: string | null;
66
-
ZORIGINALSTABLEHASH: string | null;
67
-
ZTITLE: string | null;
68
-
}
69
-
70
-
const REQUIRED_COLUMNS = [
71
-
"ZUUID",
72
-
"ZFILENAME",
73
-
"ZKIND",
74
-
"ZWIDTH",
75
-
"ZHEIGHT",
76
-
"ZCLOUDLOCALSTATE",
77
-
];
78
-
79
-
function assertValidRow(row: Record<string, unknown>): void {
80
-
for (const key of REQUIRED_COLUMNS) {
81
-
if (!(key in row)) {
82
-
throw new Error(
83
-
`Photos database schema mismatch: missing column '${key}'. ` +
84
-
`This may indicate an unsupported macOS version.`,
85
-
);
86
-
}
87
-
}
88
-
}
89
-
90
-
const VALID_ASSET_KINDS = new Set<number>(Object.values(AssetKind));
91
-
const VALID_CLOUD_STATES = new Set<number>(Object.values(CloudLocalState));
92
-
93
-
function assertAssetKind(value: number): AssetKindValue {
94
-
if (!VALID_ASSET_KINDS.has(value)) {
95
-
throw new Error(`Unknown asset kind: ${value}`);
96
-
}
97
-
return value as AssetKindValue;
98
-
}
99
-
100
-
function assertCloudLocalState(value: number): CloudLocalStateValue {
101
-
if (!VALID_CLOUD_STATES.has(value)) {
102
-
throw new Error(`Unknown cloud local state: ${value}`);
103
-
}
104
-
return value as CloudLocalStateValue;
105
-
}
106
-
107
-
interface EditInfo {
108
-
editedAt: Date | null;
109
-
editor: string;
110
-
}
111
-
112
-
interface EnrichmentMaps {
113
-
descriptions: Map<number, string>;
114
-
albums: Map<number, AlbumRef[]>;
115
-
keywords: Map<number, string[]>;
116
-
people: Map<number, PersonRef[]>;
117
-
edits: Map<number, EditInfo>;
118
-
renderedAssets: Set<number>;
119
-
}
120
-
121
-
/** hasEdit requires both an adjustment record and a rendered resource. */
122
-
function editFields(
123
-
enrichment: EnrichmentMaps,
124
-
pk: number,
125
-
): Pick<PhotoAsset, "hasEdit" | "editedAt" | "editor"> {
126
-
const editInfo = enrichment.edits.get(pk);
127
-
const hasEdit = editInfo != null && enrichment.renderedAssets.has(pk);
128
-
return {
129
-
hasEdit,
130
-
editedAt: hasEdit ? editInfo.editedAt : null,
131
-
editor: hasEdit ? editInfo.editor : null,
132
-
};
133
-
}
134
-
135
-
function rowToAsset(row: RawRow, enrichment: EnrichmentMaps): PhotoAsset {
136
-
const pk = row.Z_PK;
137
-
return {
138
-
uuid: row.ZUUID,
139
-
filename: row.ZFILENAME ?? "",
140
-
directory: row.ZDIRECTORY,
141
-
dateCreated: coreDataTimestampToDate(row.ZDATECREATED),
142
-
kind: assertAssetKind(row.ZKIND),
143
-
uniformTypeIdentifier: row.ZUNIFORMTYPEIDENTIFIER,
144
-
width: row.ZWIDTH,
145
-
height: row.ZHEIGHT,
146
-
latitude: row.ZLATITUDE,
147
-
longitude: row.ZLONGITUDE,
148
-
favorite: row.ZFAVORITE === 1,
149
-
cloudLocalState: assertCloudLocalState(row.ZCLOUDLOCALSTATE),
150
-
originalFileSize: row.ZORIGINALFILESIZE,
151
-
originalFilename: row.ZORIGINALFILENAME,
152
-
originalStableHash: row.ZORIGINALSTABLEHASH,
153
-
title: row.ZTITLE ?? null,
154
-
description: enrichment.descriptions.get(pk) ?? null,
155
-
albums: enrichment.albums.get(pk) ?? [],
156
-
keywords: enrichment.keywords.get(pk) ?? [],
157
-
people: enrichment.people.get(pk) ?? [],
158
-
...editFields(enrichment, pk),
159
-
};
160
-
}
161
-
162
-
function safeQuery<T>(db: Database, sql: string, label: string): T[] {
163
-
try {
164
-
return db.prepare(sql).all() as unknown as T[];
165
-
} catch (err: unknown) {
166
-
const msg = err instanceof Error ? err.message : String(err);
167
-
if (!msg.includes("no such table")) {
168
-
console.error(`Enrichment query failed (${label}): ${msg}`);
169
-
}
170
-
return [];
171
-
}
172
-
}
173
-
174
-
function buildDescriptionMap(db: Database): Map<number, string> {
175
-
const rows = safeQuery<{ ZASSET: number; ZLONGDESCRIPTION: string }>(
176
-
db,
177
-
`SELECT aa.ZASSET, d.ZLONGDESCRIPTION
178
-
FROM ZASSETDESCRIPTION d
179
-
JOIN ZADDITIONALASSETATTRIBUTES aa ON d.ZASSETATTRIBUTES = aa.Z_PK
180
-
WHERE d.ZLONGDESCRIPTION IS NOT NULL AND d.ZLONGDESCRIPTION != ''`,
181
-
"descriptions",
182
-
);
183
-
const map = new Map<number, string>();
184
-
for (const r of rows) map.set(r.ZASSET, r.ZLONGDESCRIPTION);
185
-
return map;
186
-
}
187
-
188
-
function buildAlbumMap(db: Database): Map<number, AlbumRef[]> {
189
-
const rows = safeQuery<
190
-
{ Z_3ASSETS: number; ZUUID: string; ZTITLE: string }
191
-
>(
192
-
db,
193
-
`SELECT ja.Z_3ASSETS, g.ZUUID, g.ZTITLE
194
-
FROM Z_33ASSETS ja
195
-
JOIN ZGENERICALBUM g ON ja.Z_33ALBUMS = g.Z_PK
196
-
WHERE g.ZTITLE IS NOT NULL`,
197
-
"albums",
198
-
);
199
-
const map = new Map<number, AlbumRef[]>();
200
-
for (const r of rows) {
201
-
const list = map.get(r.Z_3ASSETS) ?? [];
202
-
list.push({ uuid: r.ZUUID, title: r.ZTITLE });
203
-
map.set(r.Z_3ASSETS, list);
204
-
}
205
-
return map;
206
-
}
207
-
208
-
function buildKeywordMap(db: Database): Map<number, string[]> {
209
-
const rows = safeQuery<{ ZASSET: number; ZTITLE: string }>(
210
-
db,
211
-
`SELECT aa.ZASSET, k.ZTITLE
212
-
FROM Z_1KEYWORDS jk
213
-
JOIN ZKEYWORD k ON jk.Z_52KEYWORDS = k.Z_PK
214
-
JOIN ZADDITIONALASSETATTRIBUTES aa ON jk.Z_1ASSETATTRIBUTES = aa.Z_PK
215
-
WHERE k.ZTITLE IS NOT NULL`,
216
-
"keywords",
217
-
);
218
-
const map = new Map<number, string[]>();
219
-
for (const r of rows) {
220
-
const list = map.get(r.ZASSET) ?? [];
221
-
list.push(r.ZTITLE);
222
-
map.set(r.ZASSET, list);
223
-
}
224
-
return map;
225
-
}
226
-
227
-
function buildEditMap(db: Database): Map<number, EditInfo> {
228
-
const rows = safeQuery<{
229
-
ZASSET: number;
230
-
ZADJUSTMENTTIMESTAMP: number | null;
231
-
ZADJUSTMENTFORMATIDENTIFIER: string;
232
-
}>(
233
-
db,
234
-
`SELECT aa.ZASSET, ua.ZADJUSTMENTTIMESTAMP, ua.ZADJUSTMENTFORMATIDENTIFIER
235
-
FROM ZADDITIONALASSETATTRIBUTES aa
236
-
JOIN ZUNMANAGEDADJUSTMENT ua ON aa.ZUNMANAGEDADJUSTMENT = ua.Z_PK
237
-
WHERE aa.ZUNMANAGEDADJUSTMENT IS NOT NULL
238
-
AND ua.ZADJUSTMENTFORMATIDENTIFIER IS NOT NULL`,
239
-
"edits",
240
-
);
241
-
const map = new Map<number, EditInfo>();
242
-
for (const r of rows) {
243
-
map.set(r.ZASSET, {
244
-
editedAt: coreDataTimestampToDate(r.ZADJUSTMENTTIMESTAMP),
245
-
editor: r.ZADJUSTMENTFORMATIDENTIFIER,
246
-
});
247
-
}
248
-
return map;
249
-
}
250
-
251
-
function buildRenderedAssetSet(db: Database): Set<number> {
252
-
const rows = safeQuery<{ ZASSET: number }>(
253
-
db,
254
-
`SELECT DISTINCT ir.ZASSET
255
-
FROM ZINTERNALRESOURCE ir
256
-
WHERE ir.ZRESOURCETYPE = 1
257
-
AND ir.ZTRASHEDSTATE = 0
258
-
AND ir.ZVERSION != 0`,
259
-
"rendered resources",
260
-
);
261
-
return new Set(rows.map((r) => r.ZASSET));
262
-
}
263
-
264
-
function buildPeopleMap(db: Database): Map<number, PersonRef[]> {
265
-
const rows = safeQuery<
266
-
{ ZASSETFORFACE: number; ZPERSONUUID: string; ZDISPLAYNAME: string }
267
-
>(
268
-
db,
269
-
`SELECT df.ZASSETFORFACE, p.ZPERSONUUID, p.ZDISPLAYNAME
270
-
FROM ZDETECTEDFACE df
271
-
JOIN ZPERSON p ON df.ZPERSONFORFACE = p.Z_PK
272
-
WHERE p.ZDISPLAYNAME IS NOT NULL AND p.ZDISPLAYNAME != ''
273
-
AND df.ZHIDDEN = 0 AND df.ZASSETVISIBLE = 1`,
274
-
"people",
275
-
);
276
-
const map = new Map<number, PersonRef[]>();
277
-
const seen = new Map<number, Set<string>>();
278
-
for (const r of rows) {
279
-
const assetSeen = seen.get(r.ZASSETFORFACE) ?? new Set();
280
-
if (!assetSeen.has(r.ZPERSONUUID)) {
281
-
assetSeen.add(r.ZPERSONUUID);
282
-
seen.set(r.ZASSETFORFACE, assetSeen);
283
-
const list = map.get(r.ZASSETFORFACE) ?? [];
284
-
list.push({ uuid: r.ZPERSONUUID, displayName: r.ZDISPLAYNAME });
285
-
map.set(r.ZASSETFORFACE, list);
286
-
}
287
-
}
288
-
return map;
289
-
}
290
-
291
-
export interface PhotosDbReader {
292
-
readAssets(): PhotoAsset[];
293
-
close(): void;
294
-
}
295
-
296
-
export function openPhotosDb(
297
-
dbPath: string = DEFAULT_DB_PATH,
298
-
): PhotosDbReader {
299
-
const db = new Database(dbPath, { readonly: true });
300
-
301
-
return {
302
-
readAssets(): PhotoAsset[] {
303
-
const rows = db.prepare(ASSETS_QUERY).all() as unknown as Record<
304
-
string,
305
-
unknown
306
-
>[];
307
-
if (rows.length > 0) {
308
-
assertValidRow(rows[0]);
309
-
}
310
-
311
-
const enrichment: EnrichmentMaps = {
312
-
descriptions: buildDescriptionMap(db),
313
-
albums: buildAlbumMap(db),
314
-
keywords: buildKeywordMap(db),
315
-
people: buildPeopleMap(db),
316
-
edits: buildEditMap(db),
317
-
renderedAssets: buildRenderedAssetSet(db),
318
-
};
319
-
320
-
return (rows as unknown as RawRow[]).map((row) =>
321
-
rowToAsset(row, enrichment)
322
-
);
323
-
},
324
-
close() {
325
-
db.close();
326
-
},
327
-
};
328
-
}
-99
cli/src/retry.test.ts
-99
cli/src/retry.test.ts
···
1
-
import { assertEquals, assertRejects } from "@std/assert";
2
-
import { withRetry } from "./retry.ts";
3
-
import { AbortError } from "./abort-error.ts";
4
-
5
-
Deno.test("withRetry: returns result on first success", async () => {
6
-
const result = await withRetry(() => Promise.resolve(42));
7
-
assertEquals(result, 42);
8
-
});
9
-
10
-
Deno.test("withRetry: retries on transient error then succeeds", async () => {
11
-
let attempt = 0;
12
-
const result = await withRetry(
13
-
() => {
14
-
attempt++;
15
-
if (attempt === 1) throw new Error("fetch failed");
16
-
return Promise.resolve("ok");
17
-
},
18
-
{ baseDelayMs: 10 },
19
-
);
20
-
assertEquals(result, "ok");
21
-
assertEquals(attempt, 2);
22
-
});
23
-
24
-
Deno.test("withRetry: does not retry on non-transient error", async () => {
25
-
let attempt = 0;
26
-
await assertRejects(
27
-
() =>
28
-
withRetry(
29
-
() => {
30
-
attempt++;
31
-
throw new Error("Access denied");
32
-
},
33
-
{ baseDelayMs: 10 },
34
-
),
35
-
Error,
36
-
"Access denied",
37
-
);
38
-
assertEquals(attempt, 1);
39
-
});
40
-
41
-
Deno.test("withRetry: throws after max attempts", async () => {
42
-
let attempt = 0;
43
-
await assertRejects(
44
-
() =>
45
-
withRetry(
46
-
() => {
47
-
attempt++;
48
-
throw new Error("ECONNRESET");
49
-
},
50
-
{ maxAttempts: 3, baseDelayMs: 10 },
51
-
),
52
-
Error,
53
-
"ECONNRESET",
54
-
);
55
-
assertEquals(attempt, 3);
56
-
});
57
-
58
-
Deno.test("withRetry: respects abort signal during backoff", async () => {
59
-
const controller = new AbortController();
60
-
let attempt = 0;
61
-
62
-
// Abort after 50ms — the backoff would be 100ms so it should interrupt
63
-
setTimeout(() => controller.abort(), 50);
64
-
65
-
await assertRejects(
66
-
() =>
67
-
withRetry(
68
-
() => {
69
-
attempt++;
70
-
throw new Error("timeout");
71
-
},
72
-
{ maxAttempts: 5, baseDelayMs: 100, signal: controller.signal },
73
-
),
74
-
AbortError,
75
-
);
76
-
// Should have attempted once, then aborted during the first backoff delay
77
-
assertEquals(attempt, 1);
78
-
});
79
-
80
-
Deno.test("withRetry: stops retrying when signal already aborted", async () => {
81
-
const controller = new AbortController();
82
-
controller.abort();
83
-
84
-
let attempt = 0;
85
-
await assertRejects(
86
-
() =>
87
-
withRetry(
88
-
() => {
89
-
attempt++;
90
-
throw new Error("timeout");
91
-
},
92
-
{ signal: controller.signal, baseDelayMs: 10 },
93
-
),
94
-
Error,
95
-
"timeout",
96
-
);
97
-
// Should attempt once, then see signal is aborted and not retry
98
-
assertEquals(attempt, 1);
99
-
});
-61
cli/src/retry.ts
-61
cli/src/retry.ts
···
1
-
import { AbortError } from "./abort-error.ts";
2
-
3
-
export interface RetryOptions {
4
-
maxAttempts?: number;
5
-
baseDelayMs?: number;
6
-
signal?: AbortSignal;
7
-
}
8
-
9
-
/** Retry an async operation with exponential backoff.
10
-
* Handles transient network failures (e.g. after sleep/wake).
11
-
* Respects an optional AbortSignal to bail out immediately. */
12
-
export async function withRetry<T>(
13
-
fn: () => Promise<T>,
14
-
opts: RetryOptions = {},
15
-
): Promise<T> {
16
-
const { maxAttempts = 3, baseDelayMs = 1000, signal } = opts;
17
-
18
-
for (let attempt = 1; attempt <= maxAttempts; attempt++) {
19
-
try {
20
-
return await fn();
21
-
} catch (error: unknown) {
22
-
if (attempt === maxAttempts) throw error;
23
-
24
-
// Don't retry if we've been aborted
25
-
if (signal?.aborted) throw error;
26
-
27
-
// Only retry on transient/network errors, not permission or validation errors
28
-
const msg = error instanceof Error ? error.message : String(error);
29
-
const isTransient =
30
-
/timeout|ECONNRESET|ECONNREFUSED|EPIPE|socket|network|fetch failed/i
31
-
.test(msg);
32
-
if (!isTransient) throw error;
33
-
34
-
const delay = baseDelayMs * Math.pow(2, attempt - 1);
35
-
await abortableDelay(delay, signal);
36
-
}
37
-
}
38
-
throw new Error("unreachable");
39
-
}
40
-
41
-
/** Sleep that can be interrupted by an AbortSignal. */
42
-
function abortableDelay(ms: number, signal?: AbortSignal): Promise<void> {
43
-
if (!signal) {
44
-
return new Promise((resolve) => setTimeout(resolve, ms));
45
-
}
46
-
return new Promise((resolve, reject) => {
47
-
if (signal.aborted) {
48
-
reject(new AbortError("Retry aborted"));
49
-
return;
50
-
}
51
-
const id = setTimeout(() => {
52
-
signal.removeEventListener("abort", onAbort);
53
-
resolve();
54
-
}, ms);
55
-
function onAbort() {
56
-
clearTimeout(id);
57
-
reject(new AbortError("Retry aborted"));
58
-
}
59
-
signal.addEventListener("abort", onAbort, { once: true });
60
-
});
61
-
}
-48
cli/src/spinner.ts
-48
cli/src/spinner.ts
···
1
-
const FRAMES = ["⠋", "⠙", "⠹", "⠸", "⠼", "⠴", "⠦", "⠧", "⠇", "⠏"];
2
-
3
-
export interface Spinner {
4
-
update(message: string): void;
5
-
stop(): void;
6
-
}
7
-
8
-
/** Start a spinner with an animated indicator and message. */
9
-
export function startSpinner(message: string): Spinner {
10
-
let frame = 0;
11
-
let current = message;
12
-
const isTTY = Deno.stdout.isTerminal();
13
-
14
-
if (!isTTY) {
15
-
// Non-interactive: just print the message once
16
-
console.log(` ${message}`);
17
-
return {
18
-
update(msg: string) {
19
-
console.log(` ${msg}`);
20
-
},
21
-
stop() {},
22
-
};
23
-
}
24
-
25
-
const encoder = new TextEncoder();
26
-
const write = (text: string) => Deno.stdout.writeSync(encoder.encode(text));
27
-
28
-
const render = () => {
29
-
write(`\r ${FRAMES[frame]} ${current}`);
30
-
frame = (frame + 1) % FRAMES.length;
31
-
};
32
-
33
-
render();
34
-
const interval = setInterval(render, 80);
35
-
36
-
return {
37
-
update(msg: string) {
38
-
// Clear current line and show new message
39
-
write(`\r\x1b[2K`);
40
-
current = msg;
41
-
render();
42
-
},
43
-
stop() {
44
-
clearInterval(interval);
45
-
write(`\r\x1b[2K`);
46
-
},
47
-
};
48
-
}
-54
cli/src/storage/s3-client.mock.ts
-54
cli/src/storage/s3-client.mock.ts
···
1
-
import type { S3Object, S3ObjectMeta, S3Provider } from "./s3-client.ts";
2
-
3
-
/** In-memory mock S3 provider for testing. */
4
-
export function createMockS3Provider(): S3Provider & {
5
-
objects: Map<string, { body: Uint8Array; contentType?: string }>;
6
-
} {
7
-
const objects = new Map<
8
-
string,
9
-
{ body: Uint8Array; contentType?: string }
10
-
>();
11
-
12
-
return {
13
-
objects,
14
-
15
-
putObject(
16
-
key: string,
17
-
body: Uint8Array,
18
-
contentType?: string,
19
-
): Promise<void> {
20
-
objects.set(key, { body, contentType });
21
-
return Promise.resolve();
22
-
},
23
-
24
-
getObject(key: string): Promise<Uint8Array> {
25
-
const obj = objects.get(key);
26
-
if (!obj) {
27
-
const err = new Error(`NoSuchKey: ${key}`);
28
-
err.name = "NoSuchKey";
29
-
throw err;
30
-
}
31
-
return Promise.resolve(obj.body);
32
-
},
33
-
34
-
headObject(key: string): Promise<S3ObjectMeta | null> {
35
-
const obj = objects.get(key);
36
-
if (!obj) return Promise.resolve(null);
37
-
return Promise.resolve({
38
-
contentLength: obj.body.length,
39
-
contentType: obj.contentType ?? null,
40
-
etag: null,
41
-
});
42
-
},
43
-
44
-
destroy() {},
45
-
46
-
async *listObjects(prefix: string): AsyncIterable<S3Object> {
47
-
for (const [key, obj] of objects) {
48
-
if (key.startsWith(prefix)) {
49
-
yield { key, size: obj.body.length, lastModified: null };
50
-
}
51
-
}
52
-
},
53
-
};
54
-
}
-37
cli/src/storage/s3-client.test.ts
-37
cli/src/storage/s3-client.test.ts
···
1
-
import { assertEquals } from "@std/assert";
2
-
import { createMockS3Provider } from "./s3-client.mock.ts";
3
-
4
-
Deno.test("mock S3: put and get round-trip", async () => {
5
-
const s3 = createMockS3Provider();
6
-
const data = new TextEncoder().encode("hello");
7
-
await s3.putObject("test/file.txt", data, "text/plain");
8
-
9
-
const retrieved = await s3.getObject("test/file.txt");
10
-
assertEquals(new TextDecoder().decode(retrieved), "hello");
11
-
});
12
-
13
-
Deno.test("mock S3: headObject returns null for missing", async () => {
14
-
const s3 = createMockS3Provider();
15
-
const result = await s3.headObject("nonexistent");
16
-
assertEquals(result, null);
17
-
});
18
-
19
-
Deno.test("mock S3: headObject returns metadata for existing", async () => {
20
-
const s3 = createMockS3Provider();
21
-
await s3.putObject("key", new Uint8Array(42));
22
-
const head = await s3.headObject("key");
23
-
assertEquals(head?.contentLength, 42);
24
-
});
25
-
26
-
Deno.test("mock S3: listObjects filters by prefix", async () => {
27
-
const s3 = createMockS3Provider();
28
-
await s3.putObject("originals/2024/01/a.heic", new Uint8Array(1));
29
-
await s3.putObject("originals/2024/02/b.heic", new Uint8Array(2));
30
-
await s3.putObject("metadata/c.json", new Uint8Array(3));
31
-
32
-
const keys: string[] = [];
33
-
for await (const obj of s3.listObjects("originals/")) {
34
-
keys.push(obj.key);
35
-
}
36
-
assertEquals(keys.length, 2);
37
-
});
-139
cli/src/storage/s3-client.ts
-139
cli/src/storage/s3-client.ts
···
1
-
import {
2
-
GetObjectCommand,
3
-
HeadObjectCommand,
4
-
ListObjectsV2Command,
5
-
PutObjectCommand,
6
-
S3Client,
7
-
} from "@aws-sdk/client-s3";
8
-
9
-
export interface S3Object {
10
-
key: string;
11
-
size: number;
12
-
lastModified: Date | null;
13
-
}
14
-
15
-
export interface S3ObjectMeta {
16
-
contentLength: number;
17
-
contentType: string | null;
18
-
etag: string | null;
19
-
}
20
-
21
-
export interface S3Provider {
22
-
putObject(
23
-
key: string,
24
-
body: Uint8Array,
25
-
contentType?: string,
26
-
): Promise<void>;
27
-
getObject(key: string): Promise<Uint8Array>;
28
-
headObject(key: string): Promise<S3ObjectMeta | null>;
29
-
listObjects(prefix: string): AsyncIterable<S3Object>;
30
-
/** Close the underlying HTTP connection pool so the process can exit. */
31
-
destroy(): void;
32
-
}
33
-
34
-
export interface S3ConnectionConfig {
35
-
endpoint: string;
36
-
region: string;
37
-
pathStyle: boolean;
38
-
}
39
-
40
-
/** Base timeout for S3 requests (2 minutes).
41
-
* For putObject, this is extended based on body size to allow large uploads. */
42
-
const BASE_TIMEOUT_MS = 120_000;
43
-
44
-
/** Minimum assumed upload speed for timeout calculation (~500 KB/s). */
45
-
const MIN_BYTES_PER_MS = 500;
46
-
47
-
export function createS3Provider(
48
-
credentials: { accessKeyId: string; secretAccessKey: string },
49
-
bucket: string,
50
-
connection: S3ConnectionConfig,
51
-
): S3Provider {
52
-
const client = new S3Client({
53
-
endpoint: connection.endpoint,
54
-
region: connection.region,
55
-
credentials: {
56
-
accessKeyId: credentials.accessKeyId,
57
-
secretAccessKey: credentials.secretAccessKey,
58
-
},
59
-
forcePathStyle: connection.pathStyle,
60
-
});
61
-
62
-
return {
63
-
async putObject(
64
-
key: string,
65
-
body: Uint8Array,
66
-
contentType?: string,
67
-
): Promise<void> {
68
-
// Scale timeout with body size: base + time at ~500KB/s
69
-
const timeoutMs = BASE_TIMEOUT_MS +
70
-
Math.ceil(body.byteLength / MIN_BYTES_PER_MS);
71
-
await client.send(
72
-
new PutObjectCommand({
73
-
Bucket: bucket,
74
-
Key: key,
75
-
Body: body,
76
-
ContentType: contentType,
77
-
}),
78
-
{ abortSignal: AbortSignal.timeout(timeoutMs) },
79
-
);
80
-
},
81
-
82
-
async getObject(key: string): Promise<Uint8Array> {
83
-
const result = await client.send(
84
-
new GetObjectCommand({ Bucket: bucket, Key: key }),
85
-
{ abortSignal: AbortSignal.timeout(BASE_TIMEOUT_MS) },
86
-
);
87
-
const stream = result.Body;
88
-
if (!stream) throw new Error(`Empty response for ${key}`);
89
-
return new Uint8Array(await stream.transformToByteArray());
90
-
},
91
-
92
-
async headObject(key: string): Promise<S3ObjectMeta | null> {
93
-
try {
94
-
const result = await client.send(
95
-
new HeadObjectCommand({ Bucket: bucket, Key: key }),
96
-
{ abortSignal: AbortSignal.timeout(BASE_TIMEOUT_MS) },
97
-
);
98
-
return {
99
-
contentLength: result.ContentLength ?? 0,
100
-
contentType: result.ContentType ?? null,
101
-
etag: result.ETag ?? null,
102
-
};
103
-
} catch (error: unknown) {
104
-
if (error instanceof Error && error.name === "NotFound") {
105
-
return null;
106
-
}
107
-
throw error;
108
-
}
109
-
},
110
-
111
-
destroy() {
112
-
client.destroy();
113
-
},
114
-
115
-
async *listObjects(prefix: string): AsyncIterable<S3Object> {
116
-
let continuationToken: string | undefined;
117
-
do {
118
-
const result = await client.send(
119
-
new ListObjectsV2Command({
120
-
Bucket: bucket,
121
-
Prefix: prefix,
122
-
ContinuationToken: continuationToken,
123
-
}),
124
-
{ abortSignal: AbortSignal.timeout(BASE_TIMEOUT_MS) },
125
-
);
126
-
for (const obj of result.Contents ?? []) {
127
-
if (obj.Key) {
128
-
yield {
129
-
key: obj.Key,
130
-
size: obj.Size ?? 0,
131
-
lastModified: obj.LastModified ?? null,
132
-
};
133
-
}
134
-
}
135
-
continuationToken = result.NextContinuationToken;
136
-
} while (continuationToken);
137
-
},
138
-
};
139
-
}
-10
config.example.json
-10
config.example.json
-18
deno.json
-18
deno.json
···
1
-
{
2
-
"workspace": ["./shared", "./cli"],
3
-
"tasks": {
4
-
"check": "deno check cli/mod.ts",
5
-
"test": "deno test --allow-read --allow-write --allow-env --allow-ffi --allow-net",
6
-
"lint": "deno lint",
7
-
"fmt": "deno fmt",
8
-
"fmt:check": "deno fmt --check",
9
-
"init": "deno run --allow-read --allow-write --allow-env --allow-run=security cli/mod.ts init",
10
-
"scan": "deno run --allow-read --allow-env --allow-ffi cli/mod.ts scan",
11
-
"status": "deno run --allow-read --allow-write --allow-env --allow-ffi cli/mod.ts status",
12
-
"backup": "deno run --allow-read --allow-write --allow-env --allow-sys --allow-ffi --allow-net --allow-run cli/mod.ts backup",
13
-
"verify": "deno run --allow-read --allow-write --allow-env --allow-sys --allow-net --allow-run=security cli/mod.ts verify",
14
-
"refresh-metadata": "deno run --allow-read --allow-write --allow-env --allow-sys --allow-ffi --allow-net --allow-run=security cli/mod.ts refresh-metadata",
15
-
"install": "deno install -g -A --name attic cli/mod.ts",
16
-
"compile": "deno compile -A --output attic cli/mod.ts"
17
-
}
18
-
}
-1153
deno.lock
-1153
deno.lock
···
1
-
{
2
-
"version": "5",
3
-
"specifiers": {
4
-
"jsr:@cliffy/ansi@1.0.0": "1.0.0",
5
-
"jsr:@cliffy/command@1": "1.0.0",
6
-
"jsr:@cliffy/flags@1.0.0": "1.0.0",
7
-
"jsr:@cliffy/internal@1.0.0": "1.0.0",
8
-
"jsr:@cliffy/keycode@1.0.0": "1.0.0",
9
-
"jsr:@cliffy/prompt@1": "1.0.0",
10
-
"jsr:@cliffy/table@1.0.0": "1.0.0",
11
-
"jsr:@db/sqlite@0.12": "0.12.0",
12
-
"jsr:@denosaurs/plug@1": "1.1.0",
13
-
"jsr:@std/assert@0.217": "0.217.0",
14
-
"jsr:@std/assert@1": "1.0.18",
15
-
"jsr:@std/assert@^1.0.18": "1.0.18",
16
-
"jsr:@std/encoding@1": "1.0.10",
17
-
"jsr:@std/encoding@^1.0.10": "1.0.10",
18
-
"jsr:@std/fmt@1": "1.0.9",
19
-
"jsr:@std/fmt@^1.0.9": "1.0.9",
20
-
"jsr:@std/fs@1": "1.0.22",
21
-
"jsr:@std/internal@^1.0.12": "1.0.12",
22
-
"jsr:@std/io@~0.225.3": "0.225.3",
23
-
"jsr:@std/path@0.217": "0.217.0",
24
-
"jsr:@std/path@1": "1.1.4",
25
-
"jsr:@std/path@^1.1.4": "1.1.4",
26
-
"jsr:@std/text@^1.0.17": "1.0.17",
27
-
"npm:@aws-sdk/client-s3@3": "3.1008.0",
28
-
"npm:@types/node@*": "24.2.0"
29
-
},
30
-
"jsr": {
31
-
"@cliffy/ansi@1.0.0": {
32
-
"integrity": "987008f74e50aa72cc1517ffccc769711734a14927bc4599e052efe1b9a840e2",
33
-
"dependencies": [
34
-
"jsr:@cliffy/internal",
35
-
"jsr:@std/encoding@^1.0.10",
36
-
"jsr:@std/io"
37
-
]
38
-
},
39
-
"@cliffy/command@1.0.0": {
40
-
"integrity": "c52a241ea68857fcdaff4f3173eb404f8017d7bc35553b6f533c592b89dde7d2",
41
-
"dependencies": [
42
-
"jsr:@cliffy/flags",
43
-
"jsr:@cliffy/internal",
44
-
"jsr:@cliffy/table",
45
-
"jsr:@std/fmt@^1.0.9",
46
-
"jsr:@std/text"
47
-
]
48
-
},
49
-
"@cliffy/flags@1.0.0": {
50
-
"integrity": "8b57698adc644da8f90422d58976362d41a4ebca39c312ca1c101585d0148feb",
51
-
"dependencies": [
52
-
"jsr:@cliffy/internal",
53
-
"jsr:@std/text"
54
-
]
55
-
},
56
-
"@cliffy/internal@1.0.0": {
57
-
"integrity": "1e17ccbcd5420093c0a93e5b3827bbdc9abac5195bacf187edc44665e54bdde6"
58
-
},
59
-
"@cliffy/keycode@1.0.0": {
60
-
"integrity": "755dbf007be110dcb5625f87eb61b362b6a0ca6835453af03ebd3b34d399cf14"
61
-
},
62
-
"@cliffy/prompt@1.0.0": {
63
-
"integrity": "48b4cd35199fda7832f35e1fe0a3e8bc2b1ea49ba57b4ec0e29e22db44e8ca9f",
64
-
"dependencies": [
65
-
"jsr:@cliffy/ansi",
66
-
"jsr:@cliffy/internal",
67
-
"jsr:@cliffy/keycode",
68
-
"jsr:@std/assert@^1.0.18",
69
-
"jsr:@std/fmt@^1.0.9",
70
-
"jsr:@std/io",
71
-
"jsr:@std/path@^1.1.4",
72
-
"jsr:@std/text"
73
-
]
74
-
},
75
-
"@cliffy/table@1.0.0": {
76
-
"integrity": "3fdaa9e1ef1ea62022108adabd826932bdea8dd05497079896febcd41322907f",
77
-
"dependencies": [
78
-
"jsr:@std/fmt@^1.0.9"
79
-
]
80
-
},
81
-
"@db/sqlite@0.12.0": {
82
-
"integrity": "dd1ef7f621ad50fc1e073a1c3609c4470bd51edc0994139c5bf9851de7a6d85f",
83
-
"dependencies": [
84
-
"jsr:@denosaurs/plug",
85
-
"jsr:@std/path@0.217"
86
-
]
87
-
},
88
-
"@denosaurs/plug@1.1.0": {
89
-
"integrity": "eb2f0b7546c7bca2000d8b0282c54d50d91cf6d75cb26a80df25a6de8c4bc044",
90
-
"dependencies": [
91
-
"jsr:@std/encoding@1",
92
-
"jsr:@std/fmt@1",
93
-
"jsr:@std/fs",
94
-
"jsr:@std/path@1"
95
-
]
96
-
},
97
-
"@std/assert@0.217.0": {
98
-
"integrity": "c98e279362ca6982d5285c3b89517b757c1e3477ee9f14eb2fdf80a45aaa9642"
99
-
},
100
-
"@std/assert@1.0.18": {
101
-
"integrity": "270245e9c2c13b446286de475131dc688ca9abcd94fc5db41d43a219b34d1c78",
102
-
"dependencies": [
103
-
"jsr:@std/internal"
104
-
]
105
-
},
106
-
"@std/encoding@1.0.10": {
107
-
"integrity": "8783c6384a2d13abd5e9e87a7ae0520a30e9f56aeeaa3bdf910a3eaaf5c811a1"
108
-
},
109
-
"@std/fmt@1.0.9": {
110
-
"integrity": "2487343e8899fb2be5d0e3d35013e54477ada198854e52dd05ed0422eddcabe0"
111
-
},
112
-
"@std/fs@1.0.22": {
113
-
"integrity": "de0f277a58a867147a8a01bc1b181d0dfa80bfddba8c9cf2bacd6747bcec9308",
114
-
"dependencies": [
115
-
"jsr:@std/internal",
116
-
"jsr:@std/path@^1.1.4"
117
-
]
118
-
},
119
-
"@std/internal@1.0.12": {
120
-
"integrity": "972a634fd5bc34b242024402972cd5143eac68d8dffaca5eaa4dba30ce17b027"
121
-
},
122
-
"@std/io@0.225.3": {
123
-
"integrity": "27b07b591384d12d7b568f39e61dff966b8230559122df1e9fd11cc068f7ddd1"
124
-
},
125
-
"@std/path@0.217.0": {
126
-
"integrity": "1217cc25534bca9a2f672d7fe7c6f356e4027df400c0e85c0ef3e4343bc67d11",
127
-
"dependencies": [
128
-
"jsr:@std/assert@0.217"
129
-
]
130
-
},
131
-
"@std/path@1.1.4": {
132
-
"integrity": "1d2d43f39efb1b42f0b1882a25486647cb851481862dc7313390b2bb044314b5",
133
-
"dependencies": [
134
-
"jsr:@std/internal"
135
-
]
136
-
},
137
-
"@std/text@1.0.17": {
138
-
"integrity": "4b2c4ef67ae5b6c1dfd447c81c83a43718f52e3c7e748d8b33f694aba9895f95"
139
-
}
140
-
},
141
-
"npm": {
142
-
"@aws-crypto/crc32@5.2.0": {
143
-
"integrity": "sha512-nLbCWqQNgUiwwtFsen1AdzAtvuLRsQS8rYgMuxCrdKf9kOssamGLuPwyTY9wyYblNr9+1XM8v6zoDTPPSIeANg==",
144
-
"dependencies": [
145
-
"@aws-crypto/util",
146
-
"@aws-sdk/types",
147
-
"tslib"
148
-
]
149
-
},
150
-
"@aws-crypto/crc32c@5.2.0": {
151
-
"integrity": "sha512-+iWb8qaHLYKrNvGRbiYRHSdKRWhto5XlZUEBwDjYNf+ly5SVYG6zEoYIdxvf5R3zyeP16w4PLBn3rH1xc74Rag==",
152
-
"dependencies": [
153
-
"@aws-crypto/util",
154
-
"@aws-sdk/types",
155
-
"tslib"
156
-
]
157
-
},
158
-
"@aws-crypto/sha1-browser@5.2.0": {
159
-
"integrity": "sha512-OH6lveCFfcDjX4dbAvCFSYUjJZjDr/3XJ3xHtjn3Oj5b9RjojQo8npoLeA/bNwkOkrSQ0wgrHzXk4tDRxGKJeg==",
160
-
"dependencies": [
161
-
"@aws-crypto/supports-web-crypto",
162
-
"@aws-crypto/util",
163
-
"@aws-sdk/types",
164
-
"@aws-sdk/util-locate-window",
165
-
"@smithy/util-utf8@2.3.0",
166
-
"tslib"
167
-
]
168
-
},
169
-
"@aws-crypto/sha256-browser@5.2.0": {
170
-
"integrity": "sha512-AXfN/lGotSQwu6HNcEsIASo7kWXZ5HYWvfOmSNKDsEqC4OashTp8alTmaz+F7TC2L083SFv5RdB+qU3Vs1kZqw==",
171
-
"dependencies": [
172
-
"@aws-crypto/sha256-js",
173
-
"@aws-crypto/supports-web-crypto",
174
-
"@aws-crypto/util",
175
-
"@aws-sdk/types",
176
-
"@aws-sdk/util-locate-window",
177
-
"@smithy/util-utf8@2.3.0",
178
-
"tslib"
179
-
]
180
-
},
181
-
"@aws-crypto/sha256-js@5.2.0": {
182
-
"integrity": "sha512-FFQQyu7edu4ufvIZ+OadFpHHOt+eSTBaYaki44c+akjg7qZg9oOQeLlk77F6tSYqjDAFClrHJk9tMf0HdVyOvA==",
183
-
"dependencies": [
184
-
"@aws-crypto/util",
185
-
"@aws-sdk/types",
186
-
"tslib"
187
-
]
188
-
},
189
-
"@aws-crypto/supports-web-crypto@5.2.0": {
190
-
"integrity": "sha512-iAvUotm021kM33eCdNfwIN//F77/IADDSs58i+MDaOqFrVjZo9bAal0NK7HurRuWLLpF1iLX7gbWrjHjeo+YFg==",
191
-
"dependencies": [
192
-
"tslib"
193
-
]
194
-
},
195
-
"@aws-crypto/util@5.2.0": {
196
-
"integrity": "sha512-4RkU9EsI6ZpBve5fseQlGNUWKMa1RLPQ1dnjnQoe07ldfIzcsGb5hC5W0Dm7u423KWzawlrpbjXBrXCEv9zazQ==",
197
-
"dependencies": [
198
-
"@aws-sdk/types",
199
-
"@smithy/util-utf8@2.3.0",
200
-
"tslib"
201
-
]
202
-
},
203
-
"@aws-sdk/client-s3@3.1008.0": {
204
-
"integrity": "sha512-w/SIRD25v2zVMbkn8CYIxUsac8yf5Jghkhw5j7EsNWdJhl56m/nWpUX7t1etFUW1cnzpFjZV0lXt0dNFSnbXwA==",
205
-
"dependencies": [
206
-
"@aws-crypto/sha1-browser",
207
-
"@aws-crypto/sha256-browser",
208
-
"@aws-crypto/sha256-js",
209
-
"@aws-sdk/core",
210
-
"@aws-sdk/credential-provider-node",
211
-
"@aws-sdk/middleware-bucket-endpoint",
212
-
"@aws-sdk/middleware-expect-continue",
213
-
"@aws-sdk/middleware-flexible-checksums",
214
-
"@aws-sdk/middleware-host-header",
215
-
"@aws-sdk/middleware-location-constraint",
216
-
"@aws-sdk/middleware-logger",
217
-
"@aws-sdk/middleware-recursion-detection",
218
-
"@aws-sdk/middleware-sdk-s3",
219
-
"@aws-sdk/middleware-ssec",
220
-
"@aws-sdk/middleware-user-agent",
221
-
"@aws-sdk/region-config-resolver",
222
-
"@aws-sdk/signature-v4-multi-region",
223
-
"@aws-sdk/types",
224
-
"@aws-sdk/util-endpoints",
225
-
"@aws-sdk/util-user-agent-browser",
226
-
"@aws-sdk/util-user-agent-node",
227
-
"@smithy/config-resolver",
228
-
"@smithy/core",
229
-
"@smithy/eventstream-serde-browser",
230
-
"@smithy/eventstream-serde-config-resolver",
231
-
"@smithy/eventstream-serde-node",
232
-
"@smithy/fetch-http-handler",
233
-
"@smithy/hash-blob-browser",
234
-
"@smithy/hash-node",
235
-
"@smithy/hash-stream-node",
236
-
"@smithy/invalid-dependency",
237
-
"@smithy/md5-js",
238
-
"@smithy/middleware-content-length",
239
-
"@smithy/middleware-endpoint",
240
-
"@smithy/middleware-retry",
241
-
"@smithy/middleware-serde",
242
-
"@smithy/middleware-stack",
243
-
"@smithy/node-config-provider",
244
-
"@smithy/node-http-handler",
245
-
"@smithy/protocol-http",
246
-
"@smithy/smithy-client",
247
-
"@smithy/types",
248
-
"@smithy/url-parser",
249
-
"@smithy/util-base64",
250
-
"@smithy/util-body-length-browser",
251
-
"@smithy/util-body-length-node",
252
-
"@smithy/util-defaults-mode-browser",
253
-
"@smithy/util-defaults-mode-node",
254
-
"@smithy/util-endpoints",
255
-
"@smithy/util-middleware",
256
-
"@smithy/util-retry",
257
-
"@smithy/util-stream",
258
-
"@smithy/util-utf8@4.2.2",
259
-
"@smithy/util-waiter",
260
-
"tslib"
261
-
]
262
-
},
263
-
"@aws-sdk/core@3.973.19": {
264
-
"integrity": "sha512-56KePyOcZnKTWCd89oJS1G6j3HZ9Kc+bh/8+EbvtaCCXdP6T7O7NzCiPuHRhFLWnzXIaXX3CxAz0nI5My9spHQ==",
265
-
"dependencies": [
266
-
"@aws-sdk/types",
267
-
"@aws-sdk/xml-builder",
268
-
"@smithy/core",
269
-
"@smithy/node-config-provider",
270
-
"@smithy/property-provider",
271
-
"@smithy/protocol-http",
272
-
"@smithy/signature-v4",
273
-
"@smithy/smithy-client",
274
-
"@smithy/types",
275
-
"@smithy/util-base64",
276
-
"@smithy/util-middleware",
277
-
"@smithy/util-utf8@4.2.2",
278
-
"tslib"
279
-
]
280
-
},
281
-
"@aws-sdk/crc64-nvme@3.972.4": {
282
-
"integrity": "sha512-HKZIZLbRyvzo/bXZU7Zmk6XqU+1C9DjI56xd02vwuDIxedxBEqP17t9ExhbP9QFeNq/a3l9GOcyirFXxmbDhmw==",
283
-
"dependencies": [
284
-
"@smithy/types",
285
-
"tslib"
286
-
]
287
-
},
288
-
"@aws-sdk/credential-provider-env@3.972.17": {
289
-
"integrity": "sha512-MBAMW6YELzE1SdkOniqr51mrjapQUv8JXSGxtwRjQV0mwVDutVsn22OPAUt4RcLRvdiHQmNBDEFP9iTeSVCOlA==",
290
-
"dependencies": [
291
-
"@aws-sdk/core",
292
-
"@aws-sdk/types",
293
-
"@smithy/property-provider",
294
-
"@smithy/types",
295
-
"tslib"
296
-
]
297
-
},
298
-
"@aws-sdk/credential-provider-http@3.972.19": {
299
-
"integrity": "sha512-9EJROO8LXll5a7eUFqu48k6BChrtokbmgeMWmsH7lBb6lVbtjslUYz/ShLi+SHkYzTomiGBhmzTW7y+H4BxsnA==",
300
-
"dependencies": [
301
-
"@aws-sdk/core",
302
-
"@aws-sdk/types",
303
-
"@smithy/fetch-http-handler",
304
-
"@smithy/node-http-handler",
305
-
"@smithy/property-provider",
306
-
"@smithy/protocol-http",
307
-
"@smithy/smithy-client",
308
-
"@smithy/types",
309
-
"@smithy/util-stream",
310
-
"tslib"
311
-
]
312
-
},
313
-
"@aws-sdk/credential-provider-ini@3.972.19": {
314
-
"integrity": "sha512-pVJVjWqVrPqjpFq7o0mCmeZu1Y0c94OCHSYgivdCD2wfmYVtBbwQErakruhgOD8pcMcx9SCqRw1pzHKR7OGBcA==",
315
-
"dependencies": [
316
-
"@aws-sdk/core",
317
-
"@aws-sdk/credential-provider-env",
318
-
"@aws-sdk/credential-provider-http",
319
-
"@aws-sdk/credential-provider-login",
320
-
"@aws-sdk/credential-provider-process",
321
-
"@aws-sdk/credential-provider-sso",
322
-
"@aws-sdk/credential-provider-web-identity",
323
-
"@aws-sdk/nested-clients",
324
-
"@aws-sdk/types",
325
-
"@smithy/credential-provider-imds",
326
-
"@smithy/property-provider",
327
-
"@smithy/shared-ini-file-loader",
328
-
"@smithy/types",
329
-
"tslib"
330
-
]
331
-
},
332
-
"@aws-sdk/credential-provider-login@3.972.19": {
333
-
"integrity": "sha512-jOXdZ1o+CywQKr6gyxgxuUmnGwTTnY2Kxs1PM7fI6AYtDWDnmW/yKXayNqkF8KjP1unflqMWKVbVt5VgmE3L0g==",
334
-
"dependencies": [
335
-
"@aws-sdk/core",
336
-
"@aws-sdk/nested-clients",
337
-
"@aws-sdk/types",
338
-
"@smithy/property-provider",
339
-
"@smithy/protocol-http",
340
-
"@smithy/shared-ini-file-loader",
341
-
"@smithy/types",
342
-
"tslib"
343
-
]
344
-
},
345
-
"@aws-sdk/credential-provider-node@3.972.20": {
346
-
"integrity": "sha512-0xHca2BnPY0kzjDYPH7vk8YbfdBPpWVS67rtqQMalYDQUCBYS37cZ55K6TuFxCoIyNZgSCFrVKr9PXC5BVvQQw==",
347
-
"dependencies": [
348
-
"@aws-sdk/credential-provider-env",
349
-
"@aws-sdk/credential-provider-http",
350
-
"@aws-sdk/credential-provider-ini",
351
-
"@aws-sdk/credential-provider-process",
352
-
"@aws-sdk/credential-provider-sso",
353
-
"@aws-sdk/credential-provider-web-identity",
354
-
"@aws-sdk/types",
355
-
"@smithy/credential-provider-imds",
356
-
"@smithy/property-provider",
357
-
"@smithy/shared-ini-file-loader",
358
-
"@smithy/types",
359
-
"tslib"
360
-
]
361
-
},
362
-
"@aws-sdk/credential-provider-process@3.972.17": {
363
-
"integrity": "sha512-c8G8wT1axpJDgaP3xzcy+q8Y1fTi9A2eIQJvyhQ9xuXrUZhlCfXbC0vM9bM1CUXiZppFQ1p7g0tuUMvil/gCPg==",
364
-
"dependencies": [
365
-
"@aws-sdk/core",
366
-
"@aws-sdk/types",
367
-
"@smithy/property-provider",
368
-
"@smithy/shared-ini-file-loader",
369
-
"@smithy/types",
370
-
"tslib"
371
-
]
372
-
},
373
-
"@aws-sdk/credential-provider-sso@3.972.19": {
374
-
"integrity": "sha512-kVjQsEU3b///q7EZGrUzol9wzwJFKbEzqJKSq82A9ShrUTEO7FNylTtby3sPV19ndADZh1H3FB3+5ZrvKtEEeg==",
375
-
"dependencies": [
376
-
"@aws-sdk/core",
377
-
"@aws-sdk/nested-clients",
378
-
"@aws-sdk/token-providers",
379
-
"@aws-sdk/types",
380
-
"@smithy/property-provider",
381
-
"@smithy/shared-ini-file-loader",
382
-
"@smithy/types",
383
-
"tslib"
384
-
]
385
-
},
386
-
"@aws-sdk/credential-provider-web-identity@3.972.19": {
387
-
"integrity": "sha512-BV1BlTFdG4w4tAihxN7iXDBoNcNewXD4q8uZlNQiUrnqxwGWUhKHODIQVSPlQGxXClEj+63m+cqZskw+ESmeZg==",
388
-
"dependencies": [
389
-
"@aws-sdk/core",
390
-
"@aws-sdk/nested-clients",
391
-
"@aws-sdk/types",
392
-
"@smithy/property-provider",
393
-
"@smithy/shared-ini-file-loader",
394
-
"@smithy/types",
395
-
"tslib"
396
-
]
397
-
},
398
-
"@aws-sdk/middleware-bucket-endpoint@3.972.7": {
399
-
"integrity": "sha512-goX+axlJ6PQlRnzE2bQisZ8wVrlm6dXJfBzMJhd8LhAIBan/w1Kl73fJnalM/S+18VnpzIHumyV6DtgmvqG5IA==",
400
-
"dependencies": [
401
-
"@aws-sdk/types",
402
-
"@aws-sdk/util-arn-parser",
403
-
"@smithy/node-config-provider",
404
-
"@smithy/protocol-http",
405
-
"@smithy/types",
406
-
"@smithy/util-config-provider",
407
-
"tslib"
408
-
]
409
-
},
410
-
"@aws-sdk/middleware-expect-continue@3.972.7": {
411
-
"integrity": "sha512-mvWqvm61bmZUKmmrtl2uWbokqpenY3Mc3Jf4nXB/Hse6gWxLPaCQThmhPBDzsPSV8/Odn8V6ovWt3pZ7vy4BFQ==",
412
-
"dependencies": [
413
-
"@aws-sdk/types",
414
-
"@smithy/protocol-http",
415
-
"@smithy/types",
416
-
"tslib"
417
-
]
418
-
},
419
-
"@aws-sdk/middleware-flexible-checksums@3.973.5": {
420
-
"integrity": "sha512-Dp3hqE5W6hG8HQ3Uh+AINx9wjjqYmFHbxede54sGj3akx/haIQrkp85lNdTdC+ouNUcSYNiuGkzmyDREfHX1Gg==",
421
-
"dependencies": [
422
-
"@aws-crypto/crc32",
423
-
"@aws-crypto/crc32c",
424
-
"@aws-crypto/util",
425
-
"@aws-sdk/core",
426
-
"@aws-sdk/crc64-nvme",
427
-
"@aws-sdk/types",
428
-
"@smithy/is-array-buffer@4.2.2",
429
-
"@smithy/node-config-provider",
430
-
"@smithy/protocol-http",
431
-
"@smithy/types",
432
-
"@smithy/util-middleware",
433
-
"@smithy/util-stream",
434
-
"@smithy/util-utf8@4.2.2",
435
-
"tslib"
436
-
]
437
-
},
438
-
"@aws-sdk/middleware-host-header@3.972.7": {
439
-
"integrity": "sha512-aHQZgztBFEpDU1BB00VWCIIm85JjGjQW1OG9+98BdmaOpguJvzmXBGbnAiYcciCd+IS4e9BEq664lhzGnWJHgQ==",
440
-
"dependencies": [
441
-
"@aws-sdk/types",
442
-
"@smithy/protocol-http",
443
-
"@smithy/types",
444
-
"tslib"
445
-
]
446
-
},
447
-
"@aws-sdk/middleware-location-constraint@3.972.7": {
448
-
"integrity": "sha512-vdK1LJfffBp87Lj0Bw3WdK1rJk9OLDYdQpqoKgmpIZPe+4+HawZ6THTbvjhJt4C4MNnRrHTKHQjkwBiIpDBoig==",
449
-
"dependencies": [
450
-
"@aws-sdk/types",
451
-
"@smithy/types",
452
-
"tslib"
453
-
]
454
-
},
455
-
"@aws-sdk/middleware-logger@3.972.7": {
456
-
"integrity": "sha512-LXhiWlWb26txCU1vcI9PneESSeRp/RYY/McuM4SpdrimQR5NgwaPb4VJCadVeuGWgh6QmqZ6rAKSoL1ob16W6w==",
457
-
"dependencies": [
458
-
"@aws-sdk/types",
459
-
"@smithy/types",
460
-
"tslib"
461
-
]
462
-
},
463
-
"@aws-sdk/middleware-recursion-detection@3.972.7": {
464
-
"integrity": "sha512-l2VQdcBcYLzIzykCHtXlbpiVCZ94/xniLIkAj0jpnpjY4xlgZx7f56Ypn+uV1y3gG0tNVytJqo3K9bfMFee7SQ==",
465
-
"dependencies": [
466
-
"@aws-sdk/types",
467
-
"@aws/lambda-invoke-store",
468
-
"@smithy/protocol-http",
469
-
"@smithy/types",
470
-
"tslib"
471
-
]
472
-
},
473
-
"@aws-sdk/middleware-sdk-s3@3.972.19": {
474
-
"integrity": "sha512-/CtOHHVFg4ZuN6CnLnYkrqWgVEnbOBC4kNiKa+4fldJ9cioDt3dD/f5vpq0cWLOXwmGL2zgVrVxNhjxWpxNMkg==",
475
-
"dependencies": [
476
-
"@aws-sdk/core",
477
-
"@aws-sdk/types",
478
-
"@aws-sdk/util-arn-parser",
479
-
"@smithy/core",
480
-
"@smithy/node-config-provider",
481
-
"@smithy/protocol-http",
482
-
"@smithy/signature-v4",
483
-
"@smithy/smithy-client",
484
-
"@smithy/types",
485
-
"@smithy/util-config-provider",
486
-
"@smithy/util-middleware",
487
-
"@smithy/util-stream",
488
-
"@smithy/util-utf8@4.2.2",
489
-
"tslib"
490
-
]
491
-
},
492
-
"@aws-sdk/middleware-ssec@3.972.7": {
493
-
"integrity": "sha512-G9clGVuAml7d8DYzY6DnRi7TIIDRvZ3YpqJPz/8wnWS5fYx/FNWNmkO6iJVlVkQg9BfeMzd+bVPtPJOvC4B+nQ==",
494
-
"dependencies": [
495
-
"@aws-sdk/types",
496
-
"@smithy/types",
497
-
"tslib"
498
-
]
499
-
},
500
-
"@aws-sdk/middleware-user-agent@3.972.20": {
501
-
"integrity": "sha512-3kNTLtpUdeahxtnJRnj/oIdLAUdzTfr9N40KtxNhtdrq+Q1RPMdCJINRXq37m4t5+r3H70wgC3opW46OzFcZYA==",
502
-
"dependencies": [
503
-
"@aws-sdk/core",
504
-
"@aws-sdk/types",
505
-
"@aws-sdk/util-endpoints",
506
-
"@smithy/core",
507
-
"@smithy/protocol-http",
508
-
"@smithy/types",
509
-
"@smithy/util-retry",
510
-
"tslib"
511
-
]
512
-
},
513
-
"@aws-sdk/nested-clients@3.996.9": {
514
-
"integrity": "sha512-+RpVtpmQbbtzFOKhMlsRcXM/3f1Z49qTOHaA8gEpHOYruERmog6f2AUtf/oTRLCWjR9H2b3roqryV/hI7QMW8w==",
515
-
"dependencies": [
516
-
"@aws-crypto/sha256-browser",
517
-
"@aws-crypto/sha256-js",
518
-
"@aws-sdk/core",
519
-
"@aws-sdk/middleware-host-header",
520
-
"@aws-sdk/middleware-logger",
521
-
"@aws-sdk/middleware-recursion-detection",
522
-
"@aws-sdk/middleware-user-agent",
523
-
"@aws-sdk/region-config-resolver",
524
-
"@aws-sdk/types",
525
-
"@aws-sdk/util-endpoints",
526
-
"@aws-sdk/util-user-agent-browser",
527
-
"@aws-sdk/util-user-agent-node",
528
-
"@smithy/config-resolver",
529
-
"@smithy/core",
530
-
"@smithy/fetch-http-handler",
531
-
"@smithy/hash-node",
532
-
"@smithy/invalid-dependency",
533
-
"@smithy/middleware-content-length",
534
-
"@smithy/middleware-endpoint",
535
-
"@smithy/middleware-retry",
536
-
"@smithy/middleware-serde",
537
-
"@smithy/middleware-stack",
538
-
"@smithy/node-config-provider",
539
-
"@smithy/node-http-handler",
540
-
"@smithy/protocol-http",
541
-
"@smithy/smithy-client",
542
-
"@smithy/types",
543
-
"@smithy/url-parser",
544
-
"@smithy/util-base64",
545
-
"@smithy/util-body-length-browser",
546
-
"@smithy/util-body-length-node",
547
-
"@smithy/util-defaults-mode-browser",
548
-
"@smithy/util-defaults-mode-node",
549
-
"@smithy/util-endpoints",
550
-
"@smithy/util-middleware",
551
-
"@smithy/util-retry",
552
-
"@smithy/util-utf8@4.2.2",
553
-
"tslib"
554
-
]
555
-
},
556
-
"@aws-sdk/region-config-resolver@3.972.7": {
557
-
"integrity": "sha512-/Ev/6AI8bvt4HAAptzSjThGUMjcWaX3GX8oERkB0F0F9x2dLSBdgFDiyrRz3i0u0ZFZFQ1b28is4QhyqXTUsVA==",
558
-
"dependencies": [
559
-
"@aws-sdk/types",
560
-
"@smithy/config-resolver",
561
-
"@smithy/node-config-provider",
562
-
"@smithy/types",
563
-
"tslib"
564
-
]
565
-
},
566
-
"@aws-sdk/signature-v4-multi-region@3.996.7": {
567
-
"integrity": "sha512-mYhh7FY+7OOqjkYkd6+6GgJOsXK1xBWmuR+c5mxJPj2kr5TBNeZq+nUvE9kANWAux5UxDVrNOSiEM/wlHzC3Lg==",
568
-
"dependencies": [
569
-
"@aws-sdk/middleware-sdk-s3",
570
-
"@aws-sdk/types",
571
-
"@smithy/protocol-http",
572
-
"@smithy/signature-v4",
573
-
"@smithy/types",
574
-
"tslib"
575
-
]
576
-
},
577
-
"@aws-sdk/token-providers@3.1008.0": {
578
-
"integrity": "sha512-TulwlHQBWcJs668kNUDMZHN51DeLrDsYT59Ux4a/nbvr025gM6HjKJJ3LvnZccam7OS/ZKUVkWomCneRQKJbBg==",
579
-
"dependencies": [
580
-
"@aws-sdk/core",
581
-
"@aws-sdk/nested-clients",
582
-
"@aws-sdk/types",
583
-
"@smithy/property-provider",
584
-
"@smithy/shared-ini-file-loader",
585
-
"@smithy/types",
586
-
"tslib"
587
-
]
588
-
},
589
-
"@aws-sdk/types@3.973.5": {
590
-
"integrity": "sha512-hl7BGwDCWsjH8NkZfx+HgS7H2LyM2lTMAI7ba9c8O0KqdBLTdNJivsHpqjg9rNlAlPyREb6DeDRXUl0s8uFdmQ==",
591
-
"dependencies": [
592
-
"@smithy/types",
593
-
"tslib"
594
-
]
595
-
},
596
-
"@aws-sdk/util-arn-parser@3.972.3": {
597
-
"integrity": "sha512-HzSD8PMFrvgi2Kserxuff5VitNq2sgf3w9qxmskKDiDTThWfVteJxuCS9JXiPIPtmCrp+7N9asfIaVhBFORllA==",
598
-
"dependencies": [
599
-
"tslib"
600
-
]
601
-
},
602
-
"@aws-sdk/util-endpoints@3.996.4": {
603
-
"integrity": "sha512-Hek90FBmd4joCFj+Vc98KLJh73Zqj3s2W56gjAcTkrNLMDI5nIFkG9YpfcJiVI1YlE2Ne1uOQNe+IgQ/Vz2XRA==",
604
-
"dependencies": [
605
-
"@aws-sdk/types",
606
-
"@smithy/types",
607
-
"@smithy/url-parser",
608
-
"@smithy/util-endpoints",
609
-
"tslib"
610
-
]
611
-
},
612
-
"@aws-sdk/util-locate-window@3.965.5": {
613
-
"integrity": "sha512-WhlJNNINQB+9qtLtZJcpQdgZw3SCDCpXdUJP7cToGwHbCWCnRckGlc6Bx/OhWwIYFNAn+FIydY8SZ0QmVu3xTQ==",
614
-
"dependencies": [
615
-
"tslib"
616
-
]
617
-
},
618
-
"@aws-sdk/util-user-agent-browser@3.972.7": {
619
-
"integrity": "sha512-7SJVuvhKhMF/BkNS1n0QAJYgvEwYbK2QLKBrzDiwQGiTRU6Yf1f3nehTzm/l21xdAOtWSfp2uWSddPnP2ZtsVw==",
620
-
"dependencies": [
621
-
"@aws-sdk/types",
622
-
"@smithy/types",
623
-
"bowser",
624
-
"tslib"
625
-
]
626
-
},
627
-
"@aws-sdk/util-user-agent-node@3.973.6": {
628
-
"integrity": "sha512-iF7G0prk7AvmOK64FcLvc/fW+Ty1H+vttajL7PvJFReU8urMxfYmynTTuFKDTA76Wgpq3FzTPKwabMQIXQHiXQ==",
629
-
"dependencies": [
630
-
"@aws-sdk/middleware-user-agent",
631
-
"@aws-sdk/types",
632
-
"@smithy/node-config-provider",
633
-
"@smithy/types",
634
-
"@smithy/util-config-provider",
635
-
"tslib"
636
-
]
637
-
},
638
-
"@aws-sdk/xml-builder@3.972.10": {
639
-
"integrity": "sha512-OnejAIVD+CxzyAUrVic7lG+3QRltyja9LoNqCE/1YVs8ichoTbJlVSaZ9iSMcnHLyzrSNtvaOGjSDRP+d/ouFA==",
640
-
"dependencies": [
641
-
"@smithy/types",
642
-
"fast-xml-parser",
643
-
"tslib"
644
-
]
645
-
},
646
-
"@aws/lambda-invoke-store@0.2.4": {
647
-
"integrity": "sha512-iY8yvjE0y651BixKNPgmv1WrQc+GZ142sb0z4gYnChDDY2YqI4P/jsSopBWrKfAt7LOJAkOXt7rC/hms+WclQQ=="
648
-
},
649
-
"@smithy/abort-controller@4.2.12": {
650
-
"integrity": "sha512-xolrFw6b+2iYGl6EcOL7IJY71vvyZ0DJ3mcKtpykqPe2uscwtzDZJa1uVQXyP7w9Dd+kGwYnPbMsJrGISKiY/Q==",
651
-
"dependencies": [
652
-
"@smithy/types",
653
-
"tslib"
654
-
]
655
-
},
656
-
"@smithy/chunked-blob-reader-native@4.2.3": {
657
-
"integrity": "sha512-jA5k5Udn7Y5717L86h4EIv06wIr3xn8GM1qHRi/Nf31annXcXHJjBKvgztnbn2TxH3xWrPBfgwHsOwZf0UmQWw==",
658
-
"dependencies": [
659
-
"@smithy/util-base64",
660
-
"tslib"
661
-
]
662
-
},
663
-
"@smithy/chunked-blob-reader@5.2.2": {
664
-
"integrity": "sha512-St+kVicSyayWQca+I1rGitaOEH6uKgE8IUWoYnnEX26SWdWQcL6LvMSD19Lg+vYHKdT9B2Zuu7rd3i6Wnyb/iw==",
665
-
"dependencies": [
666
-
"tslib"
667
-
]
668
-
},
669
-
"@smithy/config-resolver@4.4.11": {
670
-
"integrity": "sha512-YxFiiG4YDAtX7WMN7RuhHZLeTmRRAOyCbr+zB8e3AQzHPnUhS8zXjB1+cniPVQI3xbWsQPM0X2aaIkO/ME0ymw==",
671
-
"dependencies": [
672
-
"@smithy/node-config-provider",
673
-
"@smithy/types",
674
-
"@smithy/util-config-provider",
675
-
"@smithy/util-endpoints",
676
-
"@smithy/util-middleware",
677
-
"tslib"
678
-
]
679
-
},
680
-
"@smithy/core@3.23.11": {
681
-
"integrity": "sha512-952rGf7hBRnhUIaeLp6q4MptKW8sPFe5VvkoZ5qIzFAtx6c/QZ/54FS3yootsyUSf9gJX/NBqEBNdNR7jMIlpQ==",
682
-
"dependencies": [
683
-
"@smithy/protocol-http",
684
-
"@smithy/types",
685
-
"@smithy/url-parser",
686
-
"@smithy/util-base64",
687
-
"@smithy/util-body-length-browser",
688
-
"@smithy/util-middleware",
689
-
"@smithy/util-stream",
690
-
"@smithy/util-utf8@4.2.2",
691
-
"@smithy/uuid",
692
-
"tslib"
693
-
]
694
-
},
695
-
"@smithy/credential-provider-imds@4.2.12": {
696
-
"integrity": "sha512-cr2lR792vNZcYMriSIj+Um3x9KWrjcu98kn234xA6reOAFMmbRpQMOv8KPgEmLLtx3eldU6c5wALKFqNOhugmg==",
697
-
"dependencies": [
698
-
"@smithy/node-config-provider",
699
-
"@smithy/property-provider",
700
-
"@smithy/types",
701
-
"@smithy/url-parser",
702
-
"tslib"
703
-
]
704
-
},
705
-
"@smithy/eventstream-codec@4.2.12": {
706
-
"integrity": "sha512-FE3bZdEl62ojmy8x4FHqxq2+BuOHlcxiH5vaZ6aqHJr3AIZzwF5jfx8dEiU/X0a8RboyNDjmXjlbr8AdEyLgiA==",
707
-
"dependencies": [
708
-
"@aws-crypto/crc32",
709
-
"@smithy/types",
710
-
"@smithy/util-hex-encoding",
711
-
"tslib"
712
-
]
713
-
},
714
-
"@smithy/eventstream-serde-browser@4.2.12": {
715
-
"integrity": "sha512-XUSuMxlTxV5pp4VpqZf6Sa3vT/Q75FVkLSpSSE3KkWBvAQWeuWt1msTv8fJfgA4/jcJhrbrbMzN1AC/hvPmm5A==",
716
-
"dependencies": [
717
-
"@smithy/eventstream-serde-universal",
718
-
"@smithy/types",
719
-
"tslib"
720
-
]
721
-
},
722
-
"@smithy/eventstream-serde-config-resolver@4.3.12": {
723
-
"integrity": "sha512-7epsAZ3QvfHkngz6RXQYseyZYHlmWXSTPOfPmXkiS+zA6TBNo1awUaMFL9vxyXlGdoELmCZyZe1nQE+imbmV+Q==",
724
-
"dependencies": [
725
-
"@smithy/types",
726
-
"tslib"
727
-
]
728
-
},
729
-
"@smithy/eventstream-serde-node@4.2.12": {
730
-
"integrity": "sha512-D1pFuExo31854eAvg89KMn9Oab/wEeJR6Buy32B49A9Ogdtx5fwZPqBHUlDzaCDpycTFk2+fSQgX689Qsk7UGA==",
731
-
"dependencies": [
732
-
"@smithy/eventstream-serde-universal",
733
-
"@smithy/types",
734
-
"tslib"
735
-
]
736
-
},
737
-
"@smithy/eventstream-serde-universal@4.2.12": {
738
-
"integrity": "sha512-+yNuTiyBACxOJUTvbsNsSOfH9G9oKbaJE1lNL3YHpGcuucl6rPZMi3nrpehpVOVR2E07YqFFmtwpImtpzlouHQ==",
739
-
"dependencies": [
740
-
"@smithy/eventstream-codec",
741
-
"@smithy/types",
742
-
"tslib"
743
-
]
744
-
},
745
-
"@smithy/fetch-http-handler@5.3.15": {
746
-
"integrity": "sha512-T4jFU5N/yiIfrtrsb9uOQn7RdELdM/7HbyLNr6uO/mpkj1ctiVs7CihVr51w4LyQlXWDpXFn4BElf1WmQvZu/A==",
747
-
"dependencies": [
748
-
"@smithy/protocol-http",
749
-
"@smithy/querystring-builder",
750
-
"@smithy/types",
751
-
"@smithy/util-base64",
752
-
"tslib"
753
-
]
754
-
},
755
-
"@smithy/hash-blob-browser@4.2.13": {
756
-
"integrity": "sha512-YrF4zWKh+ghLuquldj6e/RzE3xZYL8wIPfkt0MqCRphVICjyyjH8OwKD7LLlKpVEbk4FLizFfC1+gwK6XQdR3g==",
757
-
"dependencies": [
758
-
"@smithy/chunked-blob-reader",
759
-
"@smithy/chunked-blob-reader-native",
760
-
"@smithy/types",
761
-
"tslib"
762
-
]
763
-
},
764
-
"@smithy/hash-node@4.2.12": {
765
-
"integrity": "sha512-QhBYbGrbxTkZ43QoTPrK72DoYviDeg6YKDrHTMJbbC+A0sml3kSjzFtXP7BtbyJnXojLfTQldGdUR0RGD8dA3w==",
766
-
"dependencies": [
767
-
"@smithy/types",
768
-
"@smithy/util-buffer-from@4.2.2",
769
-
"@smithy/util-utf8@4.2.2",
770
-
"tslib"
771
-
]
772
-
},
773
-
"@smithy/hash-stream-node@4.2.12": {
774
-
"integrity": "sha512-O3YbmGExeafuM/kP7Y8r6+1y0hIh3/zn6GROx0uNlB54K9oihAL75Qtc+jFfLNliTi6pxOAYZrRKD9A7iA6UFw==",
775
-
"dependencies": [
776
-
"@smithy/types",
777
-
"@smithy/util-utf8@4.2.2",
778
-
"tslib"
779
-
]
780
-
},
781
-
"@smithy/invalid-dependency@4.2.12": {
782
-
"integrity": "sha512-/4F1zb7Z8LOu1PalTdESFHR0RbPwHd3FcaG1sI3UEIriQTWakysgJr65lc1jj6QY5ye7aFsisajotH6UhWfm/g==",
783
-
"dependencies": [
784
-
"@smithy/types",
785
-
"tslib"
786
-
]
787
-
},
788
-
"@smithy/is-array-buffer@2.2.0": {
789
-
"integrity": "sha512-GGP3O9QFD24uGeAXYUjwSTXARoqpZykHadOmA8G5vfJPK0/DC67qa//0qvqrJzL1xc8WQWX7/yc7fwudjPHPhA==",
790
-
"dependencies": [
791
-
"tslib"
792
-
]
793
-
},
794
-
"@smithy/is-array-buffer@4.2.2": {
795
-
"integrity": "sha512-n6rQ4N8Jj4YTQO3YFrlgZuwKodf4zUFs7EJIWH86pSCWBaAtAGBFfCM7Wx6D2bBJ2xqFNxGBSrUWswT3M0VJow==",
796
-
"dependencies": [
797
-
"tslib"
798
-
]
799
-
},
800
-
"@smithy/md5-js@4.2.12": {
801
-
"integrity": "sha512-W/oIpHCpWU2+iAkfZYyGWE+qkpuf3vEXHLxQQDx9FPNZTTdnul0dZ2d/gUFrtQ5je1G2kp4cjG0/24YueG2LbQ==",
802
-
"dependencies": [
803
-
"@smithy/types",
804
-
"@smithy/util-utf8@4.2.2",
805
-
"tslib"
806
-
]
807
-
},
808
-
"@smithy/middleware-content-length@4.2.12": {
809
-
"integrity": "sha512-YE58Yz+cvFInWI/wOTrB+DbvUVz/pLn5mC5MvOV4fdRUc6qGwygyngcucRQjAhiCEbmfLOXX0gntSIcgMvAjmA==",
810
-
"dependencies": [
811
-
"@smithy/protocol-http",
812
-
"@smithy/types",
813
-
"tslib"
814
-
]
815
-
},
816
-
"@smithy/middleware-endpoint@4.4.25": {
817
-
"integrity": "sha512-dqjLwZs2eBxIUG6Qtw8/YZ4DvzHGIf0DA18wrgtfP6a50UIO7e2nY0FPdcbv5tVJKqWCCU5BmGMOUwT7Puan+A==",
818
-
"dependencies": [
819
-
"@smithy/core",
820
-
"@smithy/middleware-serde",
821
-
"@smithy/node-config-provider",
822
-
"@smithy/shared-ini-file-loader",
823
-
"@smithy/types",
824
-
"@smithy/url-parser",
825
-
"@smithy/util-middleware",
826
-
"tslib"
827
-
]
828
-
},
829
-
"@smithy/middleware-retry@4.4.42": {
830
-
"integrity": "sha512-vbwyqHRIpIZutNXZpLAozakzamcINaRCpEy1MYmK6xBeW3xN+TyPRA123GjXnuxZIjc9848MRRCugVMTXxC4Eg==",
831
-
"dependencies": [
832
-
"@smithy/node-config-provider",
833
-
"@smithy/protocol-http",
834
-
"@smithy/service-error-classification",
835
-
"@smithy/smithy-client",
836
-
"@smithy/types",
837
-
"@smithy/util-middleware",
838
-
"@smithy/util-retry",
839
-
"@smithy/uuid",
840
-
"tslib"
841
-
]
842
-
},
843
-
"@smithy/middleware-serde@4.2.14": {
844
-
"integrity": "sha512-+CcaLoLa5apzSRtloOyG7lQvkUw2ZDml3hRh4QiG9WyEPfW5Ke/3tPOPiPjUneuT59Tpn8+c3RVaUvvkkwqZwg==",
845
-
"dependencies": [
846
-
"@smithy/core",
847
-
"@smithy/protocol-http",
848
-
"@smithy/types",
849
-
"tslib"
850
-
]
851
-
},
852
-
"@smithy/middleware-stack@4.2.12": {
853
-
"integrity": "sha512-kruC5gRHwsCOuyCd4ouQxYjgRAym2uDlCvQ5acuMtRrcdfg7mFBg6blaxcJ09STpt3ziEkis6bhg1uwrWU7txw==",
854
-
"dependencies": [
855
-
"@smithy/types",
856
-
"tslib"
857
-
]
858
-
},
859
-
"@smithy/node-config-provider@4.3.12": {
860
-
"integrity": "sha512-tr2oKX2xMcO+rBOjobSwVAkV05SIfUKz8iI53rzxEmgW3GOOPOv0UioSDk+J8OpRQnpnhsO3Af6IEBabQBVmiw==",
861
-
"dependencies": [
862
-
"@smithy/property-provider",
863
-
"@smithy/shared-ini-file-loader",
864
-
"@smithy/types",
865
-
"tslib"
866
-
]
867
-
},
868
-
"@smithy/node-http-handler@4.4.16": {
869
-
"integrity": "sha512-ULC8UCS/HivdCB3jhi+kLFYe4B5gxH2gi9vHBfEIiRrT2jfKiZNiETJSlzRtE6B26XbBHjPtc8iZKSNqMol9bw==",
870
-
"dependencies": [
871
-
"@smithy/abort-controller",
872
-
"@smithy/protocol-http",
873
-
"@smithy/querystring-builder",
874
-
"@smithy/types",
875
-
"tslib"
876
-
]
877
-
},
878
-
"@smithy/property-provider@4.2.12": {
879
-
"integrity": "sha512-jqve46eYU1v7pZ5BM+fmkbq3DerkSluPr5EhvOcHxygxzD05ByDRppRwRPPpFrsFo5yDtCYLKu+kreHKVrvc7A==",
880
-
"dependencies": [
881
-
"@smithy/types",
882
-
"tslib"
883
-
]
884
-
},
885
-
"@smithy/protocol-http@5.3.12": {
886
-
"integrity": "sha512-fit0GZK9I1xoRlR4jXmbLhoN0OdEpa96ul8M65XdmXnxXkuMxM0Y8HDT0Fh0Xb4I85MBvBClOzgSrV1X2s1Hxw==",
887
-
"dependencies": [
888
-
"@smithy/types",
889
-
"tslib"
890
-
]
891
-
},
892
-
"@smithy/querystring-builder@4.2.12": {
893
-
"integrity": "sha512-6wTZjGABQufekycfDGMEB84BgtdOE/rCVTov+EDXQ8NHKTUNIp/j27IliwP7tjIU9LR+sSzyGBOXjeEtVgzCHg==",
894
-
"dependencies": [
895
-
"@smithy/types",
896
-
"@smithy/util-uri-escape",
897
-
"tslib"
898
-
]
899
-
},
900
-
"@smithy/querystring-parser@4.2.12": {
901
-
"integrity": "sha512-P2OdvrgiAKpkPNKlKUtWbNZKB1XjPxM086NeVhK+W+wI46pIKdWBe5QyXvhUm3MEcyS/rkLvY8rZzyUdmyDZBw==",
902
-
"dependencies": [
903
-
"@smithy/types",
904
-
"tslib"
905
-
]
906
-
},
907
-
"@smithy/service-error-classification@4.2.12": {
908
-
"integrity": "sha512-LlP29oSQN0Tw0b6D0Xo6BIikBswuIiGYbRACy5ujw/JgWSzTdYj46U83ssf6Ux0GyNJVivs2uReU8pt7Eu9okQ==",
909
-
"dependencies": [
910
-
"@smithy/types"
911
-
]
912
-
},
913
-
"@smithy/shared-ini-file-loader@4.4.7": {
914
-
"integrity": "sha512-HrOKWsUb+otTeo1HxVWeEb99t5ER1XrBi/xka2Wv6NVmTbuCUC1dvlrksdvxFtODLBjsC+PHK+fuy2x/7Ynyiw==",
915
-
"dependencies": [
916
-
"@smithy/types",
917
-
"tslib"
918
-
]
919
-
},
920
-
"@smithy/signature-v4@5.3.12": {
921
-
"integrity": "sha512-B/FBwO3MVOL00DaRSXfXfa/TRXRheagt/q5A2NM13u7q+sHS59EOVGQNfG7DkmVtdQm5m3vOosoKAXSqn/OEgw==",
922
-
"dependencies": [
923
-
"@smithy/is-array-buffer@4.2.2",
924
-
"@smithy/protocol-http",
925
-
"@smithy/types",
926
-
"@smithy/util-hex-encoding",
927
-
"@smithy/util-middleware",
928
-
"@smithy/util-uri-escape",
929
-
"@smithy/util-utf8@4.2.2",
930
-
"tslib"
931
-
]
932
-
},
933
-
"@smithy/smithy-client@4.12.5": {
934
-
"integrity": "sha512-UqwYawyqSr/aog8mnLnfbPurS0gi4G7IYDcD28cUIBhsvWs1+rQcL2IwkUQ+QZ7dibaoRzhNF99fAQ9AUcO00w==",
935
-
"dependencies": [
936
-
"@smithy/core",
937
-
"@smithy/middleware-endpoint",
938
-
"@smithy/middleware-stack",
939
-
"@smithy/protocol-http",
940
-
"@smithy/types",
941
-
"@smithy/util-stream",
942
-
"tslib"
943
-
]
944
-
},
945
-
"@smithy/types@4.13.1": {
946
-
"integrity": "sha512-787F3yzE2UiJIQ+wYW1CVg2odHjmaWLGksnKQHUrK/lYZSEcy1msuLVvxaR/sI2/aDe9U+TBuLsXnr3vod1g0g==",
947
-
"dependencies": [
948
-
"tslib"
949
-
]
950
-
},
951
-
"@smithy/url-parser@4.2.12": {
952
-
"integrity": "sha512-wOPKPEpso+doCZGIlr+e1lVI6+9VAKfL4kZWFgzVgGWY2hZxshNKod4l2LXS3PRC9otH/JRSjtEHqQ/7eLciRA==",
953
-
"dependencies": [
954
-
"@smithy/querystring-parser",
955
-
"@smithy/types",
956
-
"tslib"
957
-
]
958
-
},
959
-
"@smithy/util-base64@4.3.2": {
960
-
"integrity": "sha512-XRH6b0H/5A3SgblmMa5ErXQ2XKhfbQB+Fm/oyLZ2O2kCUrwgg55bU0RekmzAhuwOjA9qdN5VU2BprOvGGUkOOQ==",
961
-
"dependencies": [
962
-
"@smithy/util-buffer-from@4.2.2",
963
-
"@smithy/util-utf8@4.2.2",
964
-
"tslib"
965
-
]
966
-
},
967
-
"@smithy/util-body-length-browser@4.2.2": {
968
-
"integrity": "sha512-JKCrLNOup3OOgmzeaKQwi4ZCTWlYR5H4Gm1r2uTMVBXoemo1UEghk5vtMi1xSu2ymgKVGW631e2fp9/R610ZjQ==",
969
-
"dependencies": [
970
-
"tslib"
971
-
]
972
-
},
973
-
"@smithy/util-body-length-node@4.2.3": {
974
-
"integrity": "sha512-ZkJGvqBzMHVHE7r/hcuCxlTY8pQr1kMtdsVPs7ex4mMU+EAbcXppfo5NmyxMYi2XU49eqaz56j2gsk4dHHPG/g==",
975
-
"dependencies": [
976
-
"tslib"
977
-
]
978
-
},
979
-
"@smithy/util-buffer-from@2.2.0": {
980
-
"integrity": "sha512-IJdWBbTcMQ6DA0gdNhh/BwrLkDR+ADW5Kr1aZmd4k3DIF6ezMV4R2NIAmT08wQJ3yUK82thHWmC/TnK/wpMMIA==",
981
-
"dependencies": [
982
-
"@smithy/is-array-buffer@2.2.0",
983
-
"tslib"
984
-
]
985
-
},
986
-
"@smithy/util-buffer-from@4.2.2": {
987
-
"integrity": "sha512-FDXD7cvUoFWwN6vtQfEta540Y/YBe5JneK3SoZg9bThSoOAC/eGeYEua6RkBgKjGa/sz6Y+DuBZj3+YEY21y4Q==",
988
-
"dependencies": [
989
-
"@smithy/is-array-buffer@4.2.2",
990
-
"tslib"
991
-
]
992
-
},
993
-
"@smithy/util-config-provider@4.2.2": {
994
-
"integrity": "sha512-dWU03V3XUprJwaUIFVv4iOnS1FC9HnMHDfUrlNDSh4315v0cWyaIErP8KiqGVbf5z+JupoVpNM7ZB3jFiTejvQ==",
995
-
"dependencies": [
996
-
"tslib"
997
-
]
998
-
},
999
-
"@smithy/util-defaults-mode-browser@4.3.41": {
1000
-
"integrity": "sha512-M1w1Ux0rSVvBOxIIiqbxvZvhnjQ+VUjJrugtORE90BbadSTH+jsQL279KRL3Hv0w69rE7EuYkV/4Lepz/NBW9g==",
1001
-
"dependencies": [
1002
-
"@smithy/property-provider",
1003
-
"@smithy/smithy-client",
1004
-
"@smithy/types",
1005
-
"tslib"
1006
-
]
1007
-
},
1008
-
"@smithy/util-defaults-mode-node@4.2.44": {
1009
-
"integrity": "sha512-YPze3/lD1KmWuZsl9JlfhcgGLX7AXhSoaCDtiPntUjNW5/YY0lOHjkcgxyE9x/h5vvS1fzDifMGjzqnNlNiqOQ==",
1010
-
"dependencies": [
1011
-
"@smithy/config-resolver",
1012
-
"@smithy/credential-provider-imds",
1013
-
"@smithy/node-config-provider",
1014
-
"@smithy/property-provider",
1015
-
"@smithy/smithy-client",
1016
-
"@smithy/types",
1017
-
"tslib"
1018
-
]
1019
-
},
1020
-
"@smithy/util-endpoints@3.3.3": {
1021
-
"integrity": "sha512-VACQVe50j0HZPjpwWcjyT51KUQ4AnsvEaQ2lKHOSL4mNLD0G9BjEniQ+yCt1qqfKfiAHRAts26ud7hBjamrwig==",
1022
-
"dependencies": [
1023
-
"@smithy/node-config-provider",
1024
-
"@smithy/types",
1025
-
"tslib"
1026
-
]
1027
-
},
1028
-
"@smithy/util-hex-encoding@4.2.2": {
1029
-
"integrity": "sha512-Qcz3W5vuHK4sLQdyT93k/rfrUwdJ8/HZ+nMUOyGdpeGA1Wxt65zYwi3oEl9kOM+RswvYq90fzkNDahPS8K0OIg==",
1030
-
"dependencies": [
1031
-
"tslib"
1032
-
]
1033
-
},
1034
-
"@smithy/util-middleware@4.2.12": {
1035
-
"integrity": "sha512-Er805uFUOvgc0l8nv0e0su0VFISoxhJ/AwOn3gL2NWNY2LUEldP5WtVcRYSQBcjg0y9NfG8JYrCJaYDpupBHJQ==",
1036
-
"dependencies": [
1037
-
"@smithy/types",
1038
-
"tslib"
1039
-
]
1040
-
},
1041
-
"@smithy/util-retry@4.2.12": {
1042
-
"integrity": "sha512-1zopLDUEOwumjcHdJ1mwBHddubYF8GMQvstVCLC54Y46rqoHwlIU+8ZzUeaBcD+WCJHyDGSeZ2ml9YSe9aqcoQ==",
1043
-
"dependencies": [
1044
-
"@smithy/service-error-classification",
1045
-
"@smithy/types",
1046
-
"tslib"
1047
-
]
1048
-
},
1049
-
"@smithy/util-stream@4.5.19": {
1050
-
"integrity": "sha512-v4sa+3xTweL1CLO2UP0p7tvIMH/Rq1X4KKOxd568mpe6LSLMQCnDHs4uv7m3ukpl3HvcN2JH6jiCS0SNRXKP/w==",
1051
-
"dependencies": [
1052
-
"@smithy/fetch-http-handler",
1053
-
"@smithy/node-http-handler",
1054
-
"@smithy/types",
1055
-
"@smithy/util-base64",
1056
-
"@smithy/util-buffer-from@4.2.2",
1057
-
"@smithy/util-hex-encoding",
1058
-
"@smithy/util-utf8@4.2.2",
1059
-
"tslib"
1060
-
]
1061
-
},
1062
-
"@smithy/util-uri-escape@4.2.2": {
1063
-
"integrity": "sha512-2kAStBlvq+lTXHyAZYfJRb/DfS3rsinLiwb+69SstC9Vb0s9vNWkRwpnj918Pfi85mzi42sOqdV72OLxWAISnw==",
1064
-
"dependencies": [
1065
-
"tslib"
1066
-
]
1067
-
},
1068
-
"@smithy/util-utf8@2.3.0": {
1069
-
"integrity": "sha512-R8Rdn8Hy72KKcebgLiv8jQcQkXoLMOGGv5uI1/k0l+snqkOzQ1R0ChUBCxWMlBsFMekWjq0wRudIweFs7sKT5A==",
1070
-
"dependencies": [
1071
-
"@smithy/util-buffer-from@2.2.0",
1072
-
"tslib"
1073
-
]
1074
-
},
1075
-
"@smithy/util-utf8@4.2.2": {
1076
-
"integrity": "sha512-75MeYpjdWRe8M5E3AW0O4Cx3UadweS+cwdXjwYGBW5h/gxxnbeZ877sLPX/ZJA9GVTlL/qG0dXP29JWFCD1Ayw==",
1077
-
"dependencies": [
1078
-
"@smithy/util-buffer-from@4.2.2",
1079
-
"tslib"
1080
-
]
1081
-
},
1082
-
"@smithy/util-waiter@4.2.13": {
1083
-
"integrity": "sha512-2zdZ9DTHngRtcYxJK1GUDxruNr53kv5W2Lupe0LMU+Imr6ohQg8M2T14MNkj1Y0wS3FFwpgpGQyvuaMF7CiTmQ==",
1084
-
"dependencies": [
1085
-
"@smithy/abort-controller",
1086
-
"@smithy/types",
1087
-
"tslib"
1088
-
]
1089
-
},
1090
-
"@smithy/uuid@1.1.2": {
1091
-
"integrity": "sha512-O/IEdcCUKkubz60tFbGA7ceITTAJsty+lBjNoorP4Z6XRqaFb/OjQjZODophEcuq68nKm6/0r+6/lLQ+XVpk8g==",
1092
-
"dependencies": [
1093
-
"tslib"
1094
-
]
1095
-
},
1096
-
"@types/node@24.2.0": {
1097
-
"integrity": "sha512-3xyG3pMCq3oYCNg7/ZP+E1ooTaGB4cG8JWRsqqOYQdbWNY4zbaV0Ennrd7stjiJEFZCaybcIgpTjJWHRfBSIDw==",
1098
-
"dependencies": [
1099
-
"undici-types"
1100
-
]
1101
-
},
1102
-
"bowser@2.14.1": {
1103
-
"integrity": "sha512-tzPjzCxygAKWFOJP011oxFHs57HzIhOEracIgAePE4pqB3LikALKnSzUyU4MGs9/iCEUuHlAJTjTc5M+u7YEGg=="
1104
-
},
1105
-
"fast-xml-builder@1.1.3": {
1106
-
"integrity": "sha512-1o60KoFw2+LWKQu3IdcfcFlGTW4dpqEWmjhYec6H82AYZU2TVBXep6tMl8Z1Y+wM+ZrzCwe3BZ9Vyd9N2rIvmg==",
1107
-
"dependencies": [
1108
-
"path-expression-matcher"
1109
-
]
1110
-
},
1111
-
"fast-xml-parser@5.4.1": {
1112
-
"integrity": "sha512-BQ30U1mKkvXQXXkAGcuyUA/GA26oEB7NzOtsxCDtyu62sjGw5QraKFhx2Em3WQNjPw9PG6MQ9yuIIgkSDfGu5A==",
1113
-
"dependencies": [
1114
-
"fast-xml-builder",
1115
-
"strnum"
1116
-
],
1117
-
"bin": true
1118
-
},
1119
-
"path-expression-matcher@1.1.3": {
1120
-
"integrity": "sha512-qdVgY8KXmVdJZRSS1JdEPOKPdTiEK/pi0RkcT2sw1RhXxohdujUlJFPuS1TSkevZ9vzd3ZlL7ULl1MHGTApKzQ=="
1121
-
},
1122
-
"strnum@2.2.0": {
1123
-
"integrity": "sha512-Y7Bj8XyJxnPAORMZj/xltsfo55uOiyHcU2tnAVzHUnSJR/KsEX+9RoDeXEnsXtl/CX4fAcrt64gZ13aGaWPeBg=="
1124
-
},
1125
-
"tslib@2.8.1": {
1126
-
"integrity": "sha512-oJFu94HQb+KVduSUQL7wnpmqnfmLsOA/nAh6b6EH0wCEoK0/mPeXU6c3wKDV83MkOuHPRHtSXKKU99IBazS/2w=="
1127
-
},
1128
-
"undici-types@7.10.0": {
1129
-
"integrity": "sha512-t5Fy/nfn+14LuOc2KNYg75vZqClpAiqscVvMygNnlsHBFpSXdJaYtXMcdNLpl/Qvc3P2cB3s6lOV51nqsFq4ag=="
1130
-
}
1131
-
},
1132
-
"workspace": {
1133
-
"members": {
1134
-
"cli": {
1135
-
"dependencies": [
1136
-
"jsr:@cliffy/ansi@1",
1137
-
"jsr:@cliffy/command@1",
1138
-
"jsr:@cliffy/prompt@1",
1139
-
"jsr:@db/sqlite@0.12",
1140
-
"jsr:@std/assert@1",
1141
-
"jsr:@std/crypto@1",
1142
-
"jsr:@std/path@1",
1143
-
"npm:@aws-sdk/client-s3@3"
1144
-
]
1145
-
},
1146
-
"shared": {
1147
-
"dependencies": [
1148
-
"jsr:@std/assert@1"
1149
-
]
1150
-
}
1151
-
}
1152
-
}
1153
-
}
+71
-86
docs/architecture.md
+71
-86
docs/architecture.md
···
1
1
# Architecture
2
2
3
-
Attic reads the macOS Photos library, exports original files via a companion
4
-
Swift tool, and uploads them to S3 with rich metadata. A local manifest tracks
5
-
progress so runs are incremental.
3
+
Attic reads the macOS Photos library via PhotoKit (through LadderKit), exports
4
+
original files, and uploads them to S3 with rich metadata. A manifest on S3
5
+
tracks progress so runs are incremental.
6
6
7
7
## System overview
8
8
9
9
```
10
-
Photos.sqlite ──→ reader.ts ──→ PhotoAsset[] ──→ backup pipeline
11
-
(read-only) │
12
-
├─→ ladder (Swift subprocess)
13
-
│ exports originals to staging/
14
-
│
15
-
├─→ S3 upload (original + metadata JSON)
16
-
│
17
-
└─→ manifest.json (on S3, shared across machines)
10
+
PhotoKit ──→ LadderKit ──→ AssetInfo[] ──→ backup pipeline
11
+
(read-only) (PhotosDatabase enrichment) │
12
+
├─→ PhotoExporter (export + SHA-256)
13
+
│ originals to staging dir
14
+
│
15
+
├─→ S3 upload (original + metadata JSON)
16
+
│
17
+
└─→ manifest.json (on S3, shared across machines)
18
18
```
19
19
20
-
Attic never modifies Photos.sqlite. The database is opened read-only.
20
+
Attic never modifies the Photos library. PhotoKit access is read-only.
21
21
22
22
## Reading the Photos library
23
23
24
-
`reader.ts` queries Photos.sqlite in two stages:
24
+
LadderKit provides two components for reading the library:
25
25
26
-
**Main query** — a single SELECT joining `ZASSET` and
27
-
`ZADDITIONALASSETATTRIBUTES` returns core fields: UUID, filename, date,
28
-
dimensions, GPS, file size, UTI, favorite status, cloud state. Trashed assets
29
-
are excluded.
26
+
**PhotoKitLibrary** — enumerates assets via PhotoKit's `PHAsset` API, returning
27
+
`AssetInfo` structs with core fields: UUID, filename, date, dimensions, GPS, UTI,
28
+
favorite status, media type.
30
29
31
-
**Enrichment queries** — seven independent queries each build a `Map` keyed by
32
-
asset primary key (Z_PK). During row mapping, each asset is enriched from these
33
-
maps with a default of `null` or `[]` if no match exists.
30
+
**PhotosDatabase** — enriches assets by querying Photos.sqlite directly
31
+
(read-only). Seven independent queries each build a `Map` keyed by asset primary
32
+
key (Z_PK). During enrichment, each asset is populated from these maps with a
33
+
default of `nil` or `[]` if no match exists.
34
34
35
-
| Query | Source tables | Returns |
36
-
| ------------------ | --------------------------------------------------------- | -------------------------- |
37
-
| Descriptions | `ZASSETDESCRIPTION` → `ZADDITIONALASSETATTRIBUTES` | `Map<number, string>` |
38
-
| Albums | `ZGENERICALBUM` → `Z_33ASSETS` | `Map<number, AlbumRef[]>` |
35
+
| Query | Source tables | Returns |
36
+
| ------------------ | ---------------------------------------------------------- | -------------------------- |
37
+
| Descriptions | `ZASSETDESCRIPTION` → `ZADDITIONALASSETATTRIBUTES` | `Map<number, string>` |
38
+
| Albums | `ZGENERICALBUM` → `Z_33ASSETS` | `Map<number, AlbumRef[]>` |
39
39
| Keywords | `ZKEYWORD` → `Z_1KEYWORDS` → `ZADDITIONALASSETATTRIBUTES` | `Map<number, string[]>` |
40
-
| People | `ZPERSON` → `ZDETECTEDFACE` | `Map<number, PersonRef[]>` |
41
-
| Edits | `ZUNMANAGEDADJUSTMENT` → `ZADDITIONALASSETATTRIBUTES` | `Map<number, EditInfo>` |
42
-
| Rendered resources | `ZINTERNALRESOURCE` (resource type 1) | `Set<number>` |
40
+
| People | `ZPERSON` → `ZDETECTEDFACE` | `Map<number, PersonRef[]>` |
41
+
| Edits | `ZUNMANAGEDADJUSTMENT` → `ZADDITIONALASSETATTRIBUTES` | `Map<number, EditInfo>` |
42
+
| Rendered resources | `ZINTERNALRESOURCE` (resource type 1) | `Set<number>` |
43
43
44
44
All enrichment queries go through `safeQuery()`, which catches "no such table"
45
45
errors silently and logs other failures. This makes the reader resilient across
···
63
63
64
64
## The backup pipeline
65
65
66
-
`backup.ts` orchestrates the full flow: filter → batch → export → upload →
67
-
manifest.
66
+
`BackupPipeline.swift` orchestrates the full flow: filter → batch → export →
67
+
upload → manifest.
68
68
69
69
### 1. Filter
70
70
···
74
74
75
75
### 2. Batch and export
76
76
77
-
Pending assets are processed in batches (default 50). Each batch is sent to
78
-
**ladder**, a companion Swift binary that uses PhotoKit to export original
79
-
files. Communication is via JSON over stdin/stdout:
77
+
Pending assets are processed in batches (default 50). Each batch is exported via
78
+
LadderKit's `PhotoExporter`, which uses PhotoKit to export original files. When
79
+
PhotoKit can't find an asset (typically iCloud-only with Optimize Storage),
80
+
LadderKit falls back to AppleScript via Photos.app, which handles the iCloud
81
+
download transparently.
80
82
81
-
```
82
-
attic → stdin: { "uuids": ["UUID/L0/001", ...], "stagingDir": "/path" }
83
-
ladder → stdout: { "results": [...], "errors": [...] }
84
-
```
85
-
86
-
Each result includes the file path, size, and SHA-256 hash. PhotoKit identifiers
87
-
use the `UUID/L0/001` format; attic strips the suffix before further processing.
88
-
Ladder output is validated at the trust boundary with
89
-
`assertExportBatchResult()`.
90
-
91
-
When PhotoKit can't find an asset (typically iCloud-only with Optimize Storage),
92
-
ladder falls back to AppleScript via Photos.app. Photos.app handles the iCloud
93
-
download transparently. The AppleScript fallback runs sequentially after all
94
-
PhotoKit exports complete. The response format is identical — attic sees no
95
-
difference between PhotoKit and AppleScript exports.
83
+
Each export result includes the file path, size, and SHA-256 hash.
96
84
97
85
### 3. Upload
98
86
99
87
For each exported file, attic:
100
88
101
-
1. Reads the staged file from disk
102
-
2. Uploads the original to `originals/{year}/{month}/{uuid}.{ext}`
103
-
3. Builds and uploads a metadata JSON to `metadata/assets/{uuid}.json` (see
89
+
1. Uploads the original to `originals/{year}/{month}/{uuid}.{ext}` using
90
+
memory-mapped I/O to avoid loading entire files into heap
91
+
2. Builds and uploads a metadata JSON to `metadata/assets/{uuid}.json` (see
104
92
`docs/metadata.md`)
105
-
4. Updates the in-memory manifest
106
-
5. Cleans up the staged file
93
+
3. Updates the in-memory manifest
94
+
4. Cleans up the staged file
107
95
108
96
S3 keys are built from UUID and extension, both validated with regex
109
97
(`/^[A-Za-z0-9._-]+$/` and `/^[a-z0-9]+$/`) to prevent path traversal.
···
121
109
periodically (every 50 assets by default) for crash resilience, and always at
122
110
the end of a run.
123
111
124
-
**Migration**: existing local manifests at `~/.attic/manifest.json` are
125
-
automatically uploaded to S3 on first run via `loadManifestWithMigration()`.
112
+
**Migration**: existing local manifests at `~/.attic/manifest.json` (from the
113
+
Deno CLI) are automatically uploaded to S3 on first run via
114
+
`loadManifestWithMigration()`.
126
115
127
-
The manifest can be reconstructed from S3 via `verify --rebuild-manifest`, which
128
-
reads every `metadata/assets/*.json` file and validates UUID format, S3 key
129
-
pattern, and checksum format before accepting an entry.
116
+
The manifest can be reconstructed from S3 via `attic rebuild`, which reads every
117
+
`metadata/assets/*.json` file and validates UUID format, S3 key pattern, and
118
+
checksum format before accepting an entry.
130
119
131
120
## Verification
132
121
133
-
`verify.ts` checks backup integrity in two modes:
134
-
135
-
- **Quick** (default) — HEAD each S3 key in the manifest, confirm it exists
136
-
- **Deep** — download each object, compute SHA-256, compare to the manifest
137
-
checksum
138
-
139
-
Both modes use a bounded concurrency pool (default 50 workers). Errors are
140
-
capped at 1,000 to prevent unbounded memory growth.
122
+
`VerifyPipeline.swift` checks backup integrity by issuing a HEAD request for
123
+
each S3 key in the manifest to confirm the object exists. Uses bounded
124
+
concurrency via TaskGroup (default 20 workers). Errors are capped at 1,000 to
125
+
prevent unbounded memory growth.
141
126
142
127
## Configuration
143
128
144
129
Attic reads its configuration from `~/.attic/config.json`. The config file
145
130
specifies the S3 endpoint, region, bucket, path-style preference, and Keychain
146
-
service names. It's created by `attic init` or manually.
131
+
service names. It's created by `attic init` or manually. Config writes are
132
+
atomic (write-to-temp, then move) with `0o600` permissions.
147
133
148
-
`scan` works without config (it only reads Photos.sqlite). All other commands
149
-
(`status`, `backup`, `verify`, `refresh-metadata`) require config and S3
150
-
credentials since the manifest is stored on S3.
134
+
`scan` works without config (it only reads the Photos library). All other
135
+
commands (`status`, `backup`, `verify`, `refresh-metadata`) require config and
136
+
S3 credentials since the manifest is stored on S3.
151
137
152
138
## Credentials
153
139
154
140
S3 credentials are stored in the macOS Keychain under configurable service names
155
-
(defaults: `attic-s3-access-key` and `attic-s3-secret-key`). They are read at
156
-
runtime via `security find-generic-password` — never stored in env vars, config
157
-
files, or code.
141
+
(defaults: `attic-s3-access-key` and `attic-s3-secret-key`) with
142
+
`kSecAttrAccessibleWhenUnlocked` accessibility. They are read at runtime via the
143
+
Security framework — never stored in env vars, config files, or code.
158
144
159
145
## Interfaces and testability
160
146
161
-
All external dependencies are behind interfaces:
147
+
All external dependencies are behind protocols:
162
148
163
-
| Interface | Real implementation | Mock |
164
-
| ---------------- | --------------------------------------------- | ------------------------------------------ |
165
-
| `S3Provider` | AWS SDK client for any S3-compatible endpoint | In-memory `Map<string, Uint8Array>` |
166
-
| `Exporter` | Ladder subprocess | Returns pre-configured assets from a `Map` |
167
-
| `ManifestStore` | S3-backed JSON (`manifest.json` in bucket) | Same S3 mock used for uploads |
168
-
| `PhotosDbReader` | SQLite reader for Photos.sqlite | In-memory SQLite with test fixtures |
149
+
| Protocol | Real implementation | Mock |
150
+
| ------------------- | ------------------------- | ------------------------------------------ |
151
+
| `S3Providing` | `AWSS3Client` | `MockS3Provider` (in-memory actor) |
152
+
| `ExportProviding` | `LadderKitExportProvider` | `MockExportProvider` / `TimeoutExportProvider` |
153
+
| `ManifestStoring` | `S3ManifestStore` | Uses MockS3Provider |
154
+
| `ConfigProviding` | `FileConfigProvider` | Direct struct construction in tests |
155
+
| `KeychainProviding` | `SecurityKeychain` | Direct struct construction in tests |
169
156
170
157
Tests never hit external services, credentials, or the real Photos library.
171
158
172
159
## What attic doesn't do
173
160
174
-
- **Modify Photos.sqlite** — read-only access, always
175
-
- **Download from iCloud directly** — ladder handles iCloud-only assets via
161
+
- **Modify the Photos library** — read-only access, always
162
+
- **Download from iCloud directly** — LadderKit handles iCloud-only assets via
176
163
AppleScript fallback through Photos.app
177
164
- **Delete from S3** — the backup is append-only; there is no prune or cleanup
178
165
command
179
166
- **Back up thumbnails** — only original files and metadata
180
167
- **Back up adjustment plists** — Apple's edit recipes are not portable
181
-
- **Back up rendered edits** — detecting edits is implemented (Phase 1);
182
-
exporting and uploading rendered versions is planned (Phase 2/3, see
183
-
`docs/plans/2026-03-13-feat-backup-rendered-edits-plan.md`)
168
+
- **Back up rendered edits** — detecting edits is implemented; exporting and
169
+
uploading rendered versions is planned
184
170
- **Handle slo-mo or Live Photos specially** — these have unique resource types
185
171
that need dedicated investigation
186
-
- **Run on non-macOS** — depends on Photos.sqlite, Keychain, and PhotoKit via
187
-
ladder
172
+
- **Run on non-macOS** — depends on PhotoKit, Keychain, and Photos.sqlite
-226
docs/unattended-backups.md
-226
docs/unattended-backups.md
···
1
-
# Unattended Backups
2
-
3
-
Run attic on a schedule so your iCloud Photos library is continuously backed up
4
-
without manual intervention. This guide covers setup using macOS launchd.
5
-
6
-
## Prerequisites
7
-
8
-
Before setting up unattended backups, make sure:
9
-
10
-
1. `attic init` has been run and works interactively (`attic backup --limit 1`)
11
-
2. Both `attic` and `ladder` are installed via Homebrew
12
-
(`brew install tijs/tap/attic`)
13
-
3. The Mac is signed into iCloud with Photos enabled
14
-
15
-
## Full Disk Access
16
-
17
-
Attic and ladder need to read Photos.sqlite and access the Photos library via
18
-
PhotoKit. macOS requires Full Disk Access for this.
19
-
20
-
Open **System Settings > Privacy & Security > Full Disk Access** and enable it
21
-
for both:
22
-
23
-
- `/opt/homebrew/bin/attic`
24
-
- `/opt/homebrew/bin/ladder`
25
-
26
-
If you skip this, backups will fail with a permission error when trying to read
27
-
the Photos database.
28
-
29
-
## Automation permission (for iCloud-only assets)
30
-
31
-
When "Optimize Mac Storage" is enabled, some assets exist only in iCloud and are
32
-
invisible to PhotoKit. Ladder uses an AppleScript fallback via Photos.app to
33
-
export these. This requires **Automation permission**.
34
-
35
-
Open **System Settings > Privacy & Security > Automation** and grant
36
-
`/opt/homebrew/bin/ladder` access to **Photos**.
37
-
38
-
The easiest way to trigger the permission prompt is to run a test backup
39
-
interactively:
40
-
41
-
```bash
42
-
attic backup --limit 1
43
-
```
44
-
45
-
If the permission is missing, attic will show a clear error message and abort
46
-
before doing any work. For unattended LaunchAgent runs, the permission must be
47
-
granted interactively once beforehand.
48
-
49
-
## LaunchAgent setup
50
-
51
-
Create a LaunchAgent plist that runs `attic backup` daily.
52
-
53
-
```bash
54
-
mkdir -p ~/.attic/logs
55
-
cat > ~/Library/LaunchAgents/photos.attic.backup.plist << 'EOF'
56
-
<?xml version="1.0" encoding="UTF-8"?>
57
-
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN"
58
-
"http://www.apple.com/DTDs/PropertyList-1.0.dtd">
59
-
<plist version="1.0">
60
-
<dict>
61
-
<key>Label</key>
62
-
<string>photos.attic.backup</string>
63
-
64
-
<key>ProgramArguments</key>
65
-
<array>
66
-
<string>/opt/homebrew/bin/attic</string>
67
-
<string>backup</string>
68
-
<string>--quiet</string>
69
-
<string>--log</string>
70
-
<string>/Users/YOU/.attic/logs/backup.jsonl</string>
71
-
<string>--notify</string>
72
-
</array>
73
-
74
-
<key>StartCalendarInterval</key>
75
-
<dict>
76
-
<key>Hour</key>
77
-
<integer>3</integer>
78
-
<key>Minute</key>
79
-
<integer>0</integer>
80
-
</dict>
81
-
82
-
<key>StandardOutPath</key>
83
-
<string>/Users/YOU/.attic/logs/backup.log</string>
84
-
<key>StandardErrorPath</key>
85
-
<string>/Users/YOU/.attic/logs/backup-error.log</string>
86
-
87
-
<key>ProcessType</key>
88
-
<string>Background</string>
89
-
</dict>
90
-
</plist>
91
-
EOF
92
-
```
93
-
94
-
Replace `YOU` with your macOS username.
95
-
96
-
The flags used:
97
-
98
-
- `--quiet` suppresses interactive progress output (spinners, per-asset lines)
99
-
- `--log` appends structured JSONL to a file — one JSON object per line with
100
-
events like `start`, `uploaded`, `error`, and `complete`
101
-
- `--notify` sends a macOS notification when the backup finishes (or fails)
102
-
103
-
Load the agent:
104
-
105
-
```bash
106
-
launchctl load ~/Library/LaunchAgents/photos.attic.backup.plist
107
-
```
108
-
109
-
The backup will now run daily at 3 AM. To change the schedule, edit the
110
-
`StartCalendarInterval` section and reload:
111
-
112
-
```bash
113
-
launchctl unload ~/Library/LaunchAgents/photos.attic.backup.plist
114
-
launchctl load ~/Library/LaunchAgents/photos.attic.backup.plist
115
-
```
116
-
117
-
## Optional: weekly verification
118
-
119
-
Add a second LaunchAgent that runs `attic verify` weekly to check backup
120
-
integrity.
121
-
122
-
```bash
123
-
cat > ~/Library/LaunchAgents/photos.attic.verify.plist << 'EOF'
124
-
<?xml version="1.0" encoding="UTF-8"?>
125
-
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN"
126
-
"http://www.apple.com/DTDs/PropertyList-1.0.dtd">
127
-
<plist version="1.0">
128
-
<dict>
129
-
<key>Label</key>
130
-
<string>photos.attic.verify</string>
131
-
132
-
<key>ProgramArguments</key>
133
-
<array>
134
-
<string>/opt/homebrew/bin/attic</string>
135
-
<string>verify</string>
136
-
</array>
137
-
138
-
<key>StartCalendarInterval</key>
139
-
<dict>
140
-
<key>Weekday</key>
141
-
<integer>0</integer>
142
-
<key>Hour</key>
143
-
<integer>4</integer>
144
-
<key>Minute</key>
145
-
<integer>0</integer>
146
-
</dict>
147
-
148
-
<key>StandardOutPath</key>
149
-
<string>/Users/YOU/.attic/logs/verify.log</string>
150
-
<key>StandardErrorPath</key>
151
-
<string>/Users/YOU/.attic/logs/verify-error.log</string>
152
-
153
-
<key>ProcessType</key>
154
-
<string>Background</string>
155
-
</dict>
156
-
</plist>
157
-
EOF
158
-
```
159
-
160
-
Same drill — replace `YOU`, then `launchctl load` it.
161
-
162
-
## Checking logs
163
-
164
-
The JSONL log (`backup.jsonl`) is the best way to review backup history. Each
165
-
line is a self-contained JSON object:
166
-
167
-
```bash
168
-
# Last run summary
169
-
tail -1 ~/.attic/logs/backup.jsonl | python3 -m json.tool
170
-
171
-
# All errors
172
-
grep '"event":"error"' ~/.attic/logs/backup.jsonl
173
-
174
-
# Count uploads per run
175
-
grep '"event":"complete"' ~/.attic/logs/backup.jsonl
176
-
```
177
-
178
-
Example log entries:
179
-
180
-
```jsonl
181
-
{"event":"start","pending":100,"photos":94,"videos":6,"timestamp":"2025-03-13T03:00:01.000Z"}
182
-
{"event":"uploaded","uuid":"ABC123","filename":"IMG_0001.HEIC","type":"photo","size":1048576,"timestamp":"..."}
183
-
{"event":"error","uuid":"DEF456","message":"Upload failed","timestamp":"..."}
184
-
{"event":"complete","uploaded":99,"failed":1,"totalBytes":52428800,"timestamp":"..."}
185
-
```
186
-
187
-
The JSONL file is appended to (not overwritten), so it accumulates history
188
-
across runs.
189
-
190
-
launchd also captures stdout/stderr to `backup.log` and `backup-error.log`, but
191
-
these are overwritten each run.
192
-
193
-
## Checking status
194
-
195
-
To see if backups are running and how far along they are:
196
-
197
-
```bash
198
-
# How many assets are backed up vs pending
199
-
attic status
200
-
201
-
# Check if the LaunchAgent is loaded
202
-
launchctl list | grep attic
203
-
```
204
-
205
-
## Stopping scheduled backups
206
-
207
-
```bash
208
-
launchctl unload ~/Library/LaunchAgents/photos.attic.backup.plist
209
-
launchctl unload ~/Library/LaunchAgents/photos.attic.verify.plist
210
-
rm ~/Library/LaunchAgents/photos.attic.backup.plist
211
-
rm ~/Library/LaunchAgents/photos.attic.verify.plist
212
-
```
213
-
214
-
## Tips
215
-
216
-
- **Dedicated Mac**: A Mac mini signed into iCloud Photos makes a good always-on
217
-
backup machine. Enable "Prevent automatic sleeping" in System Settings >
218
-
Energy.
219
-
- **Network**: Backups need a stable internet connection. If uploads fail, the
220
-
next run picks up where it left off.
221
-
- **Disk space**: Attic stages files temporarily in `~/.attic/staging/` during
222
-
export. Make sure there's enough free space for a batch (default 50 assets).
223
-
- **iCloud-only assets**: Assets that only exist in iCloud are exported via
224
-
AppleScript fallback (one at a time, sequentially). This is slower than the
225
-
normal PhotoKit path since each asset is downloaded from iCloud. The first
226
-
backup of a large iCloud-only library may take a while.
-54
scripts/update-homebrew.sh
-54
scripts/update-homebrew.sh
···
1
-
#!/bin/bash
2
-
# Update the Homebrew formula after a new release.
3
-
# Usage: ./scripts/update-homebrew.sh 0.1.0
4
-
5
-
set -euo pipefail
6
-
7
-
# Strip leading 'v' if present — version field in formula should be bare (e.g. "0.1.4")
8
-
VERSION="${1:?Usage: $0 <version>}"
9
-
VERSION="${VERSION#v}"
10
-
REPO="tijs/attic"
11
-
TAP_REPO="tijs/homebrew-tap"
12
-
FORMULA="Formula/attic.rb"
13
-
14
-
echo "Fetching checksums for v${VERSION}..."
15
-
16
-
ARM_URL="https://github.com/${REPO}/releases/download/v${VERSION}/attic-${VERSION}-aarch64-apple-darwin.tar.gz"
17
-
X86_URL="https://github.com/${REPO}/releases/download/v${VERSION}/attic-${VERSION}-x86_64-apple-darwin.tar.gz"
18
-
19
-
ARM_SHA=$(curl -sL "$ARM_URL" | shasum -a 256 | cut -d' ' -f1)
20
-
X86_SHA=$(curl -sL "$X86_URL" | shasum -a 256 | cut -d' ' -f1)
21
-
22
-
echo "ARM64 SHA256: ${ARM_SHA}"
23
-
echo "x86_64 SHA256: ${X86_SHA}"
24
-
25
-
TMPDIR=$(mktemp -d)
26
-
gh repo clone "$TAP_REPO" "$TMPDIR"
27
-
28
-
cd "$TMPDIR"
29
-
30
-
# Update version
31
-
sed -i '' "s/version \".*\"/version \"${VERSION}\"/" "$FORMULA"
32
-
33
-
# Update ARM64 sha256 (first PLACEHOLDER or sha256 after arm? block)
34
-
# Use awk for precise replacement
35
-
awk -v arm="$ARM_SHA" -v x86="$X86_SHA" '
36
-
/Hardware::CPU.arm\?/ { in_arm=1 }
37
-
/else/ { in_arm=0; in_x86=1 }
38
-
/sha256/ && in_arm { sub(/sha256 ".*"/, "sha256 \"" arm "\""); in_arm=0 }
39
-
/sha256/ && in_x86 { sub(/sha256 ".*"/, "sha256 \"" x86 "\""); in_x86=0 }
40
-
{ print }
41
-
' "$FORMULA" > "$FORMULA.tmp" && mv "$FORMULA.tmp" "$FORMULA"
42
-
43
-
echo ""
44
-
echo "Updated formula:"
45
-
cat "$FORMULA"
46
-
echo ""
47
-
48
-
git add "$FORMULA"
49
-
git commit -m "attic ${VERSION}"
50
-
git push origin main
51
-
52
-
rm -rf "$TMPDIR"
53
-
54
-
echo "Done. Homebrew formula updated to v${VERSION}."