A macOS CLI tool that exports original photos and videos from the macOS Photos library using PhotoKit.
0
fork

Configure Feed

Select the types of activity you want to include in your feed.

feat: AppleScript fallback for iCloud-only assets

When "Optimize Mac Storage" is enabled, ~17% of assets are iCloud-only
and invisible to PhotoKit's fetchAssets(). Add AppleScript fallback via
Photos.app which handles iCloud download transparently.

- ScriptExporter protocol for testability and dual-use (CLI + GUI)
- AppleScriptRunner implementation using osascript subprocess
- Pre-flight Automation permission check before exports
- UUID format validation and AppleScript string escaping
- Disk space check before iCloud downloads (2 GB minimum)
- Per-asset timeout with process termination
- Live Photo handling (prefer image component)
- 57 tests (13 new), all passing without Photos access

Inspired by osxphotos (MIT) by Rhet Turnbull.

+982 -8
+24 -1
README.md
··· 78 78 } 79 79 ``` 80 80 81 - Files are streamed from PhotoKit to disk. SHA-256 is computed inline during the write — no second pass. iCloud-only assets are downloaded transparently when `networkAccessAllowed` is true (the default). 81 + Files are streamed from PhotoKit to disk. SHA-256 is computed inline during the write — no second pass. 82 + 83 + #### iCloud-only assets 84 + 85 + When "Optimize Mac Storage" is enabled, some assets exist only in iCloud and are invisible to PhotoKit's `fetchAssets()`. For these assets, `PhotoExporter` falls back to AppleScript via Photos.app, which handles the iCloud download transparently: 86 + 87 + ```swift 88 + let exporter = PhotoExporter( 89 + stagingDir: stagingDir, 90 + library: library, 91 + scriptExporter: AppleScriptRunner() // enables iCloud fallback 92 + ) 93 + ``` 94 + 95 + The fallback runs sequentially (one asset at a time) after all PhotoKit exports complete. SHA-256 is computed after export using `FileHasher`. Pass `scriptExporter: nil` to disable the fallback. 96 + 97 + This approach is inspired by [osxphotos](https://github.com/RhetTbull/osxphotos) (MIT license) by Rhet Turnbull. 98 + 99 + **Additional permission required:** The AppleScript fallback needs Automation permission (System Settings > Privacy & Security > Automation > ladder > Photos). 82 100 83 101 ### Standalone Hashing 84 102 ··· 102 120 | `PhotoLibrary` | Asset discovery and fetch by identifier | 103 121 | `AssetHandle` | Single asset's exportable resource | 104 122 | `PhotoExporter` | Concurrent export with inline hashing | 123 + | `ScriptExporter` | AppleScript fallback for iCloud-only assets | 105 124 | `PhotosDatabase` | Photos.sqlite enrichment reader | 106 125 | `PhotosLibraryPath` | Library bundle validation and path derivation | 107 126 | `StreamingHasher` | Incremental SHA-256 | ··· 155 174 156 175 - `PhotoLibrary` — inject a mock that returns pre-configured assets 157 176 - `AssetHandle` — inject a mock that writes known data 177 + - `ScriptExporter` — inject a mock for AppleScript fallback (or `nil` to disable) 158 178 159 179 Tests run without Photos library access, Photos permission, or network. See `Tests/PhotoExporterTests.swift` for examples. 160 180 ··· 207 227 208 228 - **Photos access** — grant in System Settings > Privacy & Security > Photos 209 229 - **Full Disk Access** — may be needed depending on library location 230 + - **Automation** (for iCloud-only assets) — grant in System Settings > Privacy & Security > Automation > ladder > Photos 210 231 211 232 ## Project structure 212 233 ··· 218 239 AssetInfo.swift AssetInfo, AssetKind, AlbumInfo, PersonInfo 219 240 PhotoLibrary.swift PhotoLibrary protocol + PhotoKit implementation 220 241 PhotoExporter.swift concurrent export with inline hashing 242 + AppleScriptExporter.swift iCloud-only fallback via Photos.app 221 243 PhotosDatabase.swift Photos.sqlite enrichment reader 222 244 PhotosLibraryPath.swift library bundle validation 223 245 Hasher.swift StreamingHasher + FileHasher ··· 226 248 Tests/ 227 249 AssetInfoTests.swift 228 250 PhotoExporterTests.swift 251 + AppleScriptExporterTests.swift 229 252 PhotosDatabaseTests.swift 230 253 PhotosLibraryPathTests.swift 231 254 HasherTests.swift
+12 -1
Sources/CLI/Main.swift
··· 31 31 fatalExit(error.localizedDescription) 32 32 } 33 33 34 - let exporter = PhotoExporter(stagingDir: stagingURL) 34 + let exporter = PhotoExporter( 35 + stagingDir: stagingURL, 36 + scriptExporter: AppleScriptRunner() 37 + ) 38 + 39 + // Pre-flight: verify Automation permission before starting exports 40 + do { 41 + try await exporter.checkPermissions() 42 + } catch { 43 + fatalExit(error.localizedDescription) 44 + } 45 + 35 46 let response = await exporter.export(uuids: request.uuids) 36 47 37 48 let encoder = JSONEncoder()
+242
Sources/LadderKit/AppleScriptExporter.swift
··· 1 + import Foundation 2 + 3 + /// Abstraction over AppleScript-based export for testability and dual-use (CLI/GUI). 4 + /// 5 + /// When PhotoKit can't find an asset (typically iCloud-only with Optimize Storage enabled), 6 + /// the AppleScript fallback asks Photos.app to export it. Photos.app handles the iCloud 7 + /// download transparently. 8 + /// 9 + /// Inspired by [osxphotos](https://github.com/RhetTbull/osxphotos) (MIT license). 10 + public protocol ScriptExporter: Sendable { 11 + /// Verify that required permissions are available before starting exports. 12 + /// Throws `AppleScriptError.automationPermissionDenied` if not granted. 13 + func checkPermissions() async throws 14 + 15 + /// Export an asset by its local identifier, writing the original file to `directory`. 16 + /// Returns the URL of the exported file on success. 17 + func exportAsset(identifier: String, to directory: URL, timeout: TimeInterval) async throws -> URL 18 + } 19 + 20 + /// Export via `osascript` subprocess — works from both CLI and GUI contexts. 21 + public struct AppleScriptRunner: ScriptExporter { 22 + /// Default timeout per asset (10 minutes). 23 + public static let defaultTimeout: TimeInterval = 600 24 + 25 + /// Minimum free disk space required before attempting iCloud exports (2 GB). 26 + public static let minimumFreeSpace: UInt64 = 2 * 1024 * 1024 * 1024 27 + 28 + public init() {} 29 + 30 + /// Run a lightweight AppleScript probe to verify Automation permission. 31 + public func checkPermissions() async throws { 32 + let probe = #"tell application "Photos" to return "ok""# 33 + let result = try await runOsascript(script: probe, timeout: 30) 34 + if result.exitCode != 0 { 35 + if result.stderr.contains("-1743") || result.stderr.contains("not allowed") { 36 + throw AppleScriptError.automationPermissionDenied 37 + } 38 + throw AppleScriptError.scriptFailed( 39 + "permission-check", 40 + result.stderr.trimmingCharacters(in: .whitespacesAndNewlines) 41 + ) 42 + } 43 + } 44 + 45 + public func exportAsset( 46 + identifier: String, 47 + to directory: URL, 48 + timeout: TimeInterval = defaultTimeout 49 + ) async throws -> URL { 50 + let bareUUID = stripLocalIdSuffix(identifier) 51 + 52 + guard isValidBareUUID(bareUUID) else { 53 + throw AppleScriptError.scriptFailed(bareUUID, "Invalid UUID format") 54 + } 55 + 56 + // Create per-asset subdirectory to isolate the exported filename 57 + let subdir = directory.appendingPathComponent("as_\(PathSafety.sanitizeFilename(bareUUID))") 58 + try FileManager.default.createDirectory(at: subdir, withIntermediateDirectories: true) 59 + 60 + let script = buildExportScript(uuid: bareUUID, destination: subdir.path) 61 + let result = try await runOsascript(script: script, timeout: timeout) 62 + 63 + if result.exitCode != 0 { 64 + try? FileManager.default.removeItem(at: subdir) 65 + 66 + if result.stderr.contains("-1743") || result.stderr.contains("not allowed") { 67 + throw AppleScriptError.automationPermissionDenied 68 + } 69 + if result.timedOut { 70 + throw AppleScriptError.timeout(bareUUID, timeout) 71 + } 72 + throw AppleScriptError.scriptFailed( 73 + bareUUID, 74 + result.stderr.trimmingCharacters(in: .whitespacesAndNewlines) 75 + ) 76 + } 77 + 78 + // Discover the exported file(s) 79 + let contents = try FileManager.default.contentsOfDirectory( 80 + at: subdir, 81 + includingPropertiesForKeys: nil 82 + ) 83 + let mediaFiles = contents.filter { isMediaFile($0) } 84 + 85 + guard !mediaFiles.isEmpty else { 86 + try? FileManager.default.removeItem(at: subdir) 87 + throw AppleScriptError.noFileProduced(bareUUID) 88 + } 89 + 90 + // For Live Photos (HEIC + MOV), prefer the image component 91 + if mediaFiles.count > 1, 92 + let imageFile = mediaFiles.first(where: { isImageFile($0) }) { 93 + for file in mediaFiles where file != imageFile { 94 + try? FileManager.default.removeItem(at: file) 95 + } 96 + return imageFile 97 + } 98 + 99 + return mediaFiles[0] 100 + } 101 + } 102 + 103 + // MARK: - Script execution 104 + 105 + /// Wrapper around `Process` and `Pipe` for safe cross-isolation use. 106 + /// 107 + /// ## Safety (`@unchecked Sendable`) 108 + /// The wrapped Foundation types are set up before the process runs and read 109 + /// only after it exits (via `terminationHandler`). No concurrent mutation occurs. 110 + private final class ProcessHandle: @unchecked Sendable { 111 + let process = Process() 112 + let stderrPipe = Pipe() 113 + 114 + func configure(script: String) { 115 + process.executableURL = URL(fileURLWithPath: "/usr/bin/osascript") 116 + process.arguments = ["-e", script] 117 + process.standardError = stderrPipe 118 + process.standardOutput = FileHandle.nullDevice 119 + } 120 + 121 + func readStderr() -> String { 122 + let data = stderrPipe.fileHandleForReading.readDataToEndOfFile() 123 + return String(data: data, encoding: .utf8) ?? "" 124 + } 125 + } 126 + 127 + private struct OsascriptResult: Sendable { 128 + let exitCode: Int32 129 + let stderr: String 130 + let timedOut: Bool 131 + } 132 + 133 + private func runOsascript(script: String, timeout: TimeInterval) async throws -> OsascriptResult { 134 + let handle = ProcessHandle() 135 + handle.configure(script: script) 136 + 137 + return try await withCheckedThrowingContinuation { continuation in 138 + // Timeout: terminate process after deadline 139 + DispatchQueue.global().asyncAfter(deadline: .now() + timeout) { 140 + if handle.process.isRunning { 141 + handle.process.terminate() 142 + } 143 + } 144 + 145 + handle.process.terminationHandler = { proc in 146 + let stderr = handle.readStderr() 147 + // Detect our timeout by checking for uncaught signal (SIGTERM = 15) 148 + let timedOut = proc.terminationReason == .uncaughtSignal 149 + continuation.resume(returning: OsascriptResult( 150 + exitCode: proc.terminationStatus, 151 + stderr: stderr, 152 + timedOut: timedOut 153 + )) 154 + } 155 + 156 + do { 157 + try handle.process.run() 158 + } catch { 159 + continuation.resume(throwing: error) 160 + } 161 + } 162 + } 163 + 164 + // MARK: - Helpers 165 + 166 + func buildExportScript(uuid: String, destination: String) -> String { 167 + let safeUUID = escapeForAppleScript(uuid) 168 + let safeDest = escapeForAppleScript(destination) 169 + // AppleScript export command — Photos.app handles iCloud download transparently 170 + return """ 171 + tell application "Photos" 172 + set thePic to media item id "\(safeUUID)" 173 + export {thePic} to POSIX file "\(safeDest)" with using originals 174 + end tell 175 + """ 176 + } 177 + 178 + /// Escape a string for safe interpolation into AppleScript string literals. 179 + /// Prevents injection by neutralizing `\` and `"` characters. 180 + func escapeForAppleScript(_ string: String) -> String { 181 + string 182 + .replacingOccurrences(of: "\\", with: "\\\\") 183 + .replacingOccurrences(of: "\"", with: "\\\"") 184 + } 185 + 186 + /// Validate that a string is a standard UUID (8-4-4-4-12 hex digits). 187 + func isValidBareUUID(_ string: String) -> Bool { 188 + let pattern = /^[0-9A-Fa-f]{8}-[0-9A-Fa-f]{4}-[0-9A-Fa-f]{4}-[0-9A-Fa-f]{4}-[0-9A-Fa-f]{12}$/ 189 + return string.wholeMatch(of: pattern) != nil 190 + } 191 + 192 + /// Strip "/L0/001" suffix from PhotoKit local identifier to get bare UUID. 193 + public func stripLocalIdSuffix(_ identifier: String) -> String { 194 + if let slashIndex = identifier.firstIndex(of: "/") { 195 + return String(identifier[identifier.startIndex..<slashIndex]) 196 + } 197 + return identifier 198 + } 199 + 200 + /// Check available disk space at a given path. 201 + public func availableDiskSpace(at path: URL) -> UInt64 { 202 + let attrs = try? FileManager.default.attributesOfFileSystem(forPath: path.path) 203 + return (attrs?[.systemFreeSize] as? NSNumber)?.uint64Value ?? 0 204 + } 205 + 206 + private let imageExtensions: Set<String> = [ 207 + "heic", "jpg", "jpeg", "png", "tiff", "tif", "gif", 208 + "dng", "cr2", "nef", "orf", "arw", "raw", 209 + ] 210 + private let videoExtensions: Set<String> = ["mov", "mp4", "m4v", "avi"] 211 + 212 + private func isMediaFile(_ url: URL) -> Bool { 213 + let ext = url.pathExtension.lowercased() 214 + return imageExtensions.contains(ext) || videoExtensions.contains(ext) 215 + } 216 + 217 + private func isImageFile(_ url: URL) -> Bool { 218 + imageExtensions.contains(url.pathExtension.lowercased()) 219 + } 220 + 221 + // MARK: - Errors 222 + 223 + public enum AppleScriptError: LocalizedError, Sendable { 224 + case automationPermissionDenied 225 + case timeout(String, TimeInterval) 226 + case scriptFailed(String, String) 227 + case noFileProduced(String) 228 + 229 + public var errorDescription: String? { 230 + switch self { 231 + case .automationPermissionDenied: 232 + return "Automation permission required: grant ladder access to Photos " 233 + + "in System Settings > Privacy & Security > Automation" 234 + case .timeout(let uuid, let seconds): 235 + return "AppleScript export timed out after \(Int(seconds))s for asset \(uuid)" 236 + case .scriptFailed(let uuid, let message): 237 + return "AppleScript export failed for asset \(uuid): \(message)" 238 + case .noFileProduced(let uuid): 239 + return "AppleScript export produced no file for asset \(uuid)" 240 + } 241 + } 242 + }
+94 -6
Sources/LadderKit/PhotoExporter.swift
··· 3 3 public final class PhotoExporter: Sendable { 4 4 private let stagingDir: URL 5 5 private let library: PhotoLibrary 6 + private let scriptExporter: ScriptExporter? 6 7 private let maxConcurrency: Int 7 8 8 9 public init( 9 10 stagingDir: URL, 10 11 library: PhotoLibrary = PhotoKitLibrary(), 12 + scriptExporter: ScriptExporter? = nil, 11 13 maxConcurrency: Int = 6 12 14 ) { 13 15 self.stagingDir = stagingDir 14 16 self.library = library 17 + self.scriptExporter = scriptExporter 15 18 self.maxConcurrency = maxConcurrency 16 19 } 17 20 21 + /// Pre-flight check: verify Automation permission if a script exporter is configured. 22 + /// Call before `export()` to fail fast with a clear error instead of mid-backup. 23 + public func checkPermissions() async throws { 24 + try await scriptExporter?.checkPermissions() 25 + } 26 + 18 27 public func export(uuids: [String]) async -> ExportResponse { 19 28 let assets = library.fetchAssets(identifiers: uuids) 20 29 21 - // Report missing UUIDs 22 - let errors: [ExportError] = uuids 23 - .filter { assets[$0] == nil } 24 - .map { ExportError(uuid: $0, message: "Asset not found in Photos library") } 30 + // Identify missing UUIDs (PhotoKit can't find them) 31 + let missingUUIDs = uuids.filter { assets[$0] == nil } 25 32 26 - // Export found assets with bounded concurrency 33 + // Export found assets with bounded concurrency (unchanged) 27 34 let exportResults = await withTaskGroup( 28 35 of: Result<ExportResult, ExportErrorPair>.self 29 36 ) { group in ··· 50 57 } 51 58 52 59 var allResults: [ExportResult] = [] 53 - var allErrors = errors 60 + var allErrors: [ExportError] = [] 54 61 for result in exportResults { 55 62 switch result { 56 63 case .success(let exportResult): ··· 60 67 } 61 68 } 62 69 70 + // AppleScript fallback for missing UUIDs (iCloud-only assets) 71 + if !missingUUIDs.isEmpty { 72 + let (fallbackResults, fallbackErrors) = await exportViaAppleScript(uuids: missingUUIDs) 73 + allResults.append(contentsOf: fallbackResults) 74 + allErrors.append(contentsOf: fallbackErrors) 75 + } 76 + 63 77 return ExportResponse(results: allResults, errors: allErrors) 78 + } 79 + 80 + /// Attempt AppleScript fallback for UUIDs that PhotoKit couldn't find. 81 + private func exportViaAppleScript( 82 + uuids: [String] 83 + ) async -> ([ExportResult], [ExportError]) { 84 + guard let scriptExporter else { 85 + // No script exporter: report all as "not found" (original behavior) 86 + let errors = uuids.map { ExportError(uuid: $0, message: "Asset not found in Photos library") } 87 + return ([], errors) 88 + } 89 + 90 + // Check disk space before starting iCloud downloads 91 + let freeSpace = availableDiskSpace(at: stagingDir) 92 + if freeSpace < AppleScriptRunner.minimumFreeSpace { 93 + let gbFree = Double(freeSpace) / 1_073_741_824 94 + let errors = uuids.map { uuid in 95 + ExportError( 96 + uuid: uuid, 97 + message: String( 98 + format: "Skipped iCloud download: only %.1f GB free (need 2 GB)", 99 + gbFree 100 + ) 101 + ) 102 + } 103 + return ([], errors) 104 + } 105 + 106 + var results: [ExportResult] = [] 107 + var errors: [ExportError] = [] 108 + 109 + for uuid in uuids { 110 + if Task.isCancelled { break } 111 + 112 + do { 113 + let exportedFile = try await scriptExporter.exportAsset( 114 + identifier: uuid, 115 + to: stagingDir, 116 + timeout: AppleScriptRunner.defaultTimeout 117 + ) 118 + 119 + let sha256 = try FileHasher.sha256(fileAt: exportedFile) 120 + let attrs = try FileManager.default.attributesOfItem(atPath: exportedFile.path) 121 + let size = (attrs[.size] as? NSNumber)?.int64Value ?? 0 122 + 123 + // Move from temp subdirectory to staging dir with safe naming 124 + let safeDest = try PathSafety.safeDestination( 125 + stagingDir: stagingDir, 126 + uuid: uuid, 127 + originalFilename: exportedFile.lastPathComponent 128 + ) 129 + try FileManager.default.moveItem(at: exportedFile, to: safeDest) 130 + 131 + // Clean up the per-asset temp subdirectory 132 + let tempSubdir = exportedFile.deletingLastPathComponent() 133 + if tempSubdir != stagingDir { 134 + try? FileManager.default.removeItem(at: tempSubdir) 135 + } 136 + 137 + results.append(ExportResult( 138 + uuid: uuid, 139 + path: safeDest.path, 140 + size: size, 141 + sha256: sha256 142 + )) 143 + } catch { 144 + errors.append(ExportError( 145 + uuid: uuid, 146 + message: error.localizedDescription 147 + )) 148 + } 149 + } 150 + 151 + return (results, errors) 64 152 } 65 153 66 154 private func exportAsset(
+269
Tests/AppleScriptExporterTests.swift
··· 1 + import Foundation 2 + import Photos 3 + import Testing 4 + 5 + @testable import LadderKit 6 + 7 + /// Mock script exporter that writes a known file to the target directory. 8 + struct MockScriptExporter: ScriptExporter { 9 + /// Map of UUID → (filename, data) for assets the mock can export. 10 + let assets: [String: (filename: String, data: Data)] 11 + /// If set, all exports throw this error. 12 + var error: (any Error)? 13 + 14 + func checkPermissions() async throws {} 15 + 16 + func exportAsset( 17 + identifier: String, 18 + to directory: URL, 19 + timeout: TimeInterval 20 + ) async throws -> URL { 21 + if let error { throw error } 22 + 23 + let bareUUID = stripLocalIdSuffix(identifier) 24 + guard let (filename, data) = assets[bareUUID] ?? assets[identifier] else { 25 + throw AppleScriptError.noFileProduced(bareUUID) 26 + } 27 + 28 + // Mimic AppleScript: create subdirectory and write file 29 + let subdir = directory.appendingPathComponent("as_\(bareUUID)") 30 + try FileManager.default.createDirectory(at: subdir, withIntermediateDirectories: true) 31 + let fileURL = subdir.appendingPathComponent(filename) 32 + try data.write(to: fileURL) 33 + return fileURL 34 + } 35 + } 36 + 37 + @Suite("AppleScript Exporter") 38 + struct AppleScriptExporterTests { 39 + 40 + // MARK: - stripLocalIdSuffix 41 + 42 + @Test("strips /L0/001 suffix from PhotoKit identifier") 43 + func stripSuffix() { 44 + #expect(stripLocalIdSuffix("8A3B1C2D-4E5F-6789-ABCD-EF0123456789/L0/001") 45 + == "8A3B1C2D-4E5F-6789-ABCD-EF0123456789") 46 + } 47 + 48 + @Test("returns bare UUID unchanged") 49 + func bareUUID() { 50 + #expect(stripLocalIdSuffix("8A3B1C2D-4E5F-6789-ABCD-EF0123456789") 51 + == "8A3B1C2D-4E5F-6789-ABCD-EF0123456789") 52 + } 53 + 54 + // MARK: - buildExportScript 55 + 56 + @Test("builds correct AppleScript") 57 + func exportScript() { 58 + let script = buildExportScript( 59 + uuid: "ABC-123", 60 + destination: "/tmp/staging/as_ABC-123" 61 + ) 62 + #expect(script.contains("media item id \"ABC-123\"")) 63 + #expect(script.contains("POSIX file \"/tmp/staging/as_ABC-123\"")) 64 + #expect(script.contains("with using originals")) 65 + } 66 + 67 + // MARK: - PhotoExporter with script fallback 68 + 69 + @Test("falls back to AppleScript for missing UUIDs") 70 + func fallbackForMissing() async throws { 71 + let tempDir = FileManager.default.temporaryDirectory 72 + .appendingPathComponent("ladder-test-\(UUID().uuidString)") 73 + try FileManager.default.createDirectory(at: tempDir, withIntermediateDirectories: true) 74 + defer { try? FileManager.default.removeItem(at: tempDir) } 75 + 76 + let testData = Data("icloud photo data".utf8) 77 + let scriptExporter = MockScriptExporter(assets: [ 78 + "missing-uuid": ("IMG_5678.HEIC", testData), 79 + ]) 80 + 81 + let library = MockPhotoLibrary(assets: [:]) 82 + let exporter = PhotoExporter( 83 + stagingDir: tempDir, 84 + library: library, 85 + scriptExporter: scriptExporter 86 + ) 87 + 88 + let response = await exporter.export(uuids: ["missing-uuid"]) 89 + 90 + #expect(response.results.count == 1) 91 + #expect(response.errors.isEmpty) 92 + 93 + let result = response.results[0] 94 + #expect(result.uuid == "missing-uuid") 95 + #expect(result.size == Int64(testData.count)) 96 + #expect(!result.sha256.isEmpty) 97 + } 98 + 99 + @Test("PhotoKit assets export normally, missing ones fall back to AppleScript") 100 + func mixedPhotoKitAndAppleScript() async throws { 101 + let tempDir = FileManager.default.temporaryDirectory 102 + .appendingPathComponent("ladder-test-\(UUID().uuidString)") 103 + try FileManager.default.createDirectory(at: tempDir, withIntermediateDirectories: true) 104 + defer { try? FileManager.default.removeItem(at: tempDir) } 105 + 106 + let photoKitData = Data("photokit data".utf8) 107 + let appleScriptData = Data("applescript data".utf8) 108 + 109 + let library = MockPhotoLibrary(assets: [ 110 + "local-uuid": MockAssetHandle( 111 + originalFilename: "IMG_0001.HEIC", 112 + resourceType: .photo, 113 + data: photoKitData 114 + ), 115 + ]) 116 + 117 + let scriptExporter = MockScriptExporter(assets: [ 118 + "icloud-uuid": ("IMG_9999.HEIC", appleScriptData), 119 + ]) 120 + 121 + let exporter = PhotoExporter( 122 + stagingDir: tempDir, 123 + library: library, 124 + scriptExporter: scriptExporter 125 + ) 126 + 127 + let response = await exporter.export(uuids: ["local-uuid", "icloud-uuid"]) 128 + 129 + #expect(response.results.count == 2) 130 + #expect(response.errors.isEmpty) 131 + 132 + let uuids = Set(response.results.map(\.uuid)) 133 + #expect(uuids.contains("local-uuid")) 134 + #expect(uuids.contains("icloud-uuid")) 135 + } 136 + 137 + @Test("without script exporter, missing UUIDs reported as errors (original behavior)") 138 + func noFallbackWithoutScriptExporter() async throws { 139 + let tempDir = FileManager.default.temporaryDirectory 140 + .appendingPathComponent("ladder-test-\(UUID().uuidString)") 141 + try FileManager.default.createDirectory(at: tempDir, withIntermediateDirectories: true) 142 + defer { try? FileManager.default.removeItem(at: tempDir) } 143 + 144 + let library = MockPhotoLibrary(assets: [:]) 145 + let exporter = PhotoExporter( 146 + stagingDir: tempDir, 147 + library: library, 148 + scriptExporter: nil 149 + ) 150 + 151 + let response = await exporter.export(uuids: ["missing-uuid"]) 152 + 153 + #expect(response.results.isEmpty) 154 + #expect(response.errors.count == 1) 155 + #expect(response.errors[0].uuid == "missing-uuid") 156 + #expect(response.errors[0].message == "Asset not found in Photos library") 157 + } 158 + 159 + @Test("script export error reported as ExportError") 160 + func scriptExportError() async throws { 161 + let tempDir = FileManager.default.temporaryDirectory 162 + .appendingPathComponent("ladder-test-\(UUID().uuidString)") 163 + try FileManager.default.createDirectory(at: tempDir, withIntermediateDirectories: true) 164 + defer { try? FileManager.default.removeItem(at: tempDir) } 165 + 166 + let scriptExporter = MockScriptExporter( 167 + assets: [:], 168 + error: AppleScriptError.automationPermissionDenied 169 + ) 170 + 171 + let library = MockPhotoLibrary(assets: [:]) 172 + let exporter = PhotoExporter( 173 + stagingDir: tempDir, 174 + library: library, 175 + scriptExporter: scriptExporter 176 + ) 177 + 178 + let response = await exporter.export(uuids: ["icloud-uuid"]) 179 + 180 + #expect(response.results.isEmpty) 181 + #expect(response.errors.count == 1) 182 + #expect(response.errors[0].uuid == "icloud-uuid") 183 + #expect(response.errors[0].message.contains("Automation permission")) 184 + } 185 + 186 + @Test("UUID /L0/001 suffix stripped for AppleScript export") 187 + func suffixStrippedForFallback() async throws { 188 + let tempDir = FileManager.default.temporaryDirectory 189 + .appendingPathComponent("ladder-test-\(UUID().uuidString)") 190 + try FileManager.default.createDirectory(at: tempDir, withIntermediateDirectories: true) 191 + defer { try? FileManager.default.removeItem(at: tempDir) } 192 + 193 + let testData = Data("data".utf8) 194 + // Register with bare UUID — the exporter should strip the suffix 195 + let scriptExporter = MockScriptExporter(assets: [ 196 + "ABC-123": ("IMG_0001.HEIC", testData), 197 + ]) 198 + 199 + let library = MockPhotoLibrary(assets: [:]) 200 + let exporter = PhotoExporter( 201 + stagingDir: tempDir, 202 + library: library, 203 + scriptExporter: scriptExporter 204 + ) 205 + 206 + let response = await exporter.export(uuids: ["ABC-123/L0/001"]) 207 + 208 + #expect(response.results.count == 1) 209 + #expect(response.results[0].uuid == "ABC-123/L0/001") 210 + } 211 + 212 + // MARK: - Error types 213 + 214 + @Test("AppleScriptError provides descriptive messages") 215 + func errorMessages() { 216 + let errors: [AppleScriptError] = [ 217 + .automationPermissionDenied, 218 + .timeout("UUID-123", 600), 219 + .scriptFailed("UUID-123", "some error"), 220 + .noFileProduced("UUID-123"), 221 + ] 222 + 223 + for error in errors { 224 + #expect(error.errorDescription != nil) 225 + #expect(!error.errorDescription!.isEmpty) 226 + } 227 + 228 + #expect(AppleScriptError.timeout("X", 600).errorDescription!.contains("600")) 229 + #expect(AppleScriptError.automationPermissionDenied.errorDescription!.contains("Automation")) 230 + } 231 + 232 + // MARK: - AppleScript escaping 233 + 234 + @Test("buildExportScript escapes quotes in UUID") 235 + func scriptEscapesQuotes() { 236 + let script = buildExportScript( 237 + uuid: #"ABC" & evil & ""#, 238 + destination: "/tmp/safe" 239 + ) 240 + // The quote should be escaped, not terminating the string 241 + #expect(script.contains(#"ABC\" & evil & \""#)) 242 + #expect(!script.contains(#"id "ABC" & evil"#)) 243 + } 244 + 245 + @Test("buildExportScript escapes backslashes in destination") 246 + func scriptEscapesBackslashes() { 247 + let script = buildExportScript( 248 + uuid: "ABC-123", 249 + destination: #"/tmp/path\with\backslashes"# 250 + ) 251 + #expect(script.contains(#"path\\with\\backslashes"#)) 252 + } 253 + 254 + // MARK: - UUID validation 255 + 256 + @Test("isValidBareUUID accepts standard UUIDs") 257 + func validUUIDs() { 258 + #expect(isValidBareUUID("8A3B1C2D-4E5F-6789-ABCD-EF0123456789")) 259 + #expect(isValidBareUUID("b84e8479-475c-4727-a7f4-b3d5e5d71923")) 260 + } 261 + 262 + @Test("isValidBareUUID rejects malformed strings") 263 + func invalidUUIDs() { 264 + #expect(!isValidBareUUID("not-a-uuid")) 265 + #expect(!isValidBareUUID("")) 266 + #expect(!isValidBareUUID(#"ABC" & do shell script "evil" & ""#)) 267 + #expect(!isValidBareUUID("8A3B1C2D-4E5F-6789-ABCD-EF0123456789/L0/001")) 268 + } 269 + }
+341
docs/plans/2026-03-21-001-feat-applescript-fallback-icloud-assets-plan.md
··· 1 + --- 2 + title: "feat: AppleScript fallback for iCloud-only assets" 3 + type: feat 4 + status: active 5 + date: 2026-03-21 6 + --- 7 + 8 + # AppleScript fallback for iCloud-only assets 9 + 10 + ## Overview 11 + 12 + When "Optimize Mac Storage" is enabled, ~17% of assets (6,400 of 37,305) are 13 + iCloud-only. PhotoKit's `PHAsset.fetchAssets(withLocalIdentifiers:)` cannot find 14 + these assets at all — they're invisible to the API even with 15 + `isNetworkAccessAllowed = true`. This means ladder currently reports them as 16 + "Asset not found in Photos library" errors, making it impossible to back up a 17 + complete library. 18 + 19 + The fix: when PhotoKit can't find a UUID, fall back to AppleScript via 20 + Photos.app, which handles iCloud downloads transparently. Inspired by 21 + [osxphotos](https://github.com/RhetTbull/osxphotos) (MIT license) which uses 22 + this exact mechanism. 23 + 24 + ## Problem Statement 25 + 26 + PhotoKit has two layers: **enumeration** (finding assets by identifier) and 27 + **export** (downloading data from an asset). The iCloud-only problem is at the 28 + enumeration layer — `PHAsset.fetchAssets()` simply doesn't return these assets. 29 + Setting `isNetworkAccessAllowed = true` on the export options has no effect 30 + because there's no `PHAsset` object to export from. 31 + 32 + Photos.app, however, can access ALL assets regardless of cloud state. Its 33 + AppleScript `export` command triggers an iCloud download transparently and writes 34 + the original file to disk. 35 + 36 + ## Proposed Solution 37 + 38 + Add an `AppleScriptExporter` to LadderKit that `PhotoExporter` delegates to when 39 + PhotoKit can't find a UUID. The fallback: 40 + 41 + 1. Runs only for UUIDs that `fetchAssets()` couldn't find (not for export 42 + failures) 43 + 2. Exports one asset at a time via AppleScript (network-bound, serialization 44 + avoids Photos.app contention) 45 + 3. Computes SHA-256 after export using existing `FileHasher` 46 + 4. Returns the same `ExportResult` format — attic sees no difference 47 + 48 + ```applescript 49 + tell application "Photos" 50 + set thePic to media item id "8A3B1C2D-4E5F-6789-ABCD-EF0123456789" 51 + export {thePic} to POSIX file "/path/to/staging/uuid_subdir" with using originals 52 + end tell 53 + ``` 54 + 55 + ## Technical Considerations 56 + 57 + ### CLI and GUI dual-use 58 + 59 + The fallback lives in **LadderKit** (the shared library), not in the CLI target 60 + or in attic. This means both consumers get it automatically: 61 + 62 + - **CLI** (attic subprocess): `PhotoExporter.export(uuids:)` is called from 63 + `Main.swift`. The AppleScript fallback runs in the same process via 64 + `Process()` + `osascript`. No changes needed in attic — the `ExportResponse` 65 + format is unchanged. 66 + - **Menu bar app** (direct LadderKit import): calls `PhotoExporter.export(uuids:)` 67 + directly as a Swift package. Same fallback triggers, same behavior. 68 + 69 + The `ScriptExporter` protocol allows each consumer to provide a custom 70 + implementation if needed. For example, the menu bar app could use 71 + `NSAppleScript` (in-process, no subprocess overhead) instead of the default 72 + `osascript`-via-`Process()`. But the default works for both contexts. 73 + 74 + **Automation permission** is per-binary: the CLI binary and the menu bar app 75 + each need their own grant. The menu bar app will prompt automatically on first 76 + use (standard macOS behavior for GUI apps). The CLI needs a one-time interactive 77 + grant, documented in `attic init`. 78 + 79 + ### UUID format 80 + 81 + Attic sends identifiers in PhotoKit format: `UUID/L0/001`. AppleScript's 82 + `media item id` expects the bare UUID only. Ladder must strip the `/L0/001` 83 + suffix before passing to AppleScript. The bare UUID is already extracted in 84 + `AssetInfo.swift` — reuse that logic. 85 + 86 + ### SHA-256 87 + 88 + The normal PhotoKit path uses `StreamingHasher` (inline during data stream). For 89 + AppleScript exports, the file is written by Photos.app — we hash it after using 90 + `FileHasher.sha256(fileAt:)` which already exists in `Hasher.swift`. This means 91 + an extra read pass over the file, but iCloud-only assets are already 92 + network-bound so the disk I/O overhead is negligible. 93 + 94 + ### Exported filename discovery 95 + 96 + AppleScript `export` writes files to a directory using Photos.app's internal 97 + naming (e.g., `IMG_1234.HEIC`). We don't control the filename. The approach: 98 + 99 + 1. Create a per-asset temp subdirectory inside staging dir 100 + 2. Run AppleScript export to that subdirectory 101 + 3. Find the single file that appeared (glob for `*` in the subdir) 102 + 4. Compute size and SHA-256 103 + 5. Move to final staging path using `PathSafety.safeDestination()` naming 104 + 105 + ### Live Photos 106 + 107 + AppleScript `export ... with using originals` may produce two files for Live 108 + Photos (HEIC + MOV). For the initial implementation: take the photo component 109 + only (match by image extension). The video component of Live Photos is a future 110 + enhancement. 111 + 112 + ### Concurrency 113 + 114 + AppleScript exports are serialized — one at a time. Photos.app serializes Apple 115 + Events internally, and concurrent requests cause unpredictable behavior. 116 + `PhotoExporter` continues to use `maxConcurrency: 6` for PhotoKit assets. The 117 + AppleScript fallback runs sequentially after all PhotoKit exports complete. 118 + 119 + ### Photos.app lifecycle 120 + 121 + - `tell application "Photos"` launches Photos.app if not running — this is 122 + acceptable since the user is already using their Photos library 123 + - Ladder does NOT quit Photos.app after finishing — leave lifecycle to the user 124 + - If Photos.app shows a modal dialog (update, repair), the AppleScript will 125 + time out — this is reported as an error, not a hang 126 + 127 + ### Disk space 128 + 129 + Before starting AppleScript exports, check available disk space: 130 + - Minimum threshold: 2 GB free 131 + - If below threshold: skip remaining iCloud-only assets with a clear message, 132 + continue with local-only assets 133 + - Check once before the AppleScript batch, not per-asset (disk space changes 134 + are gradual) 135 + 136 + Photos.app manages its own iCloud cache and will evict old downloads when space 137 + is low. After attic uploads and deletes staged files, the disk pressure is 138 + temporary. 139 + 140 + ### macOS permissions 141 + 142 + AppleScript to Photos.app requires **Automation permission** (System Settings > 143 + Privacy & Security > Automation > ladder > Photos). This is separate from the 144 + existing Photos library access and Full Disk Access permissions. 145 + 146 + - First AppleScript call triggers a system permission prompt 147 + - For unattended LaunchAgent runs: user must pre-grant this permission 148 + interactively once 149 + - The `attic init` command should mention this in its setup instructions 150 + - If permission is denied, the AppleScript fails with error -1743 — report this 151 + as a clear error with instructions to grant permission 152 + 153 + ### Pre-flight permission check 154 + 155 + Before starting the backup, ladder should verify that required permissions are 156 + available and give the user clear instructions if they're not. This avoids 157 + wasting time exporting local assets only to fail on iCloud-only ones later. 158 + 159 + **Check at startup (in `PhotoExporter` or caller):** 160 + 161 + 1. **Photos library access** — already checked in `Main.swift` via 162 + `PHPhotoLibrary.requestAuthorization()`. If denied, exit with instructions. 163 + 2. **Automation permission** (only when `scriptExporter` is non-nil) — run a 164 + lightweight AppleScript probe before the real export: 165 + ```applescript 166 + tell application "Photos" to return "ok" 167 + ``` 168 + If this returns error -1743, the user hasn't granted Automation permission. 169 + Report a clear, actionable error **before any export work begins**: 170 + ``` 171 + ladder: Automation permission required for iCloud-only assets. 172 + Grant access: System Settings > Privacy & Security > Automation > ladder > Photos 173 + ``` 174 + Then exit with a non-zero code so attic can surface the message. 175 + 176 + **Design:** 177 + 178 + - Add a `public func checkPermissions() async throws` method to 179 + `AppleScriptRunner` (and the `ScriptExporter` protocol as optional with a 180 + default no-op extension). 181 + - `PhotoExporter.export()` calls `scriptExporter?.checkPermissions()` before 182 + processing any assets. If it throws, all UUIDs are reported as errors with the 183 + permission message — no partial work is done. 184 + - The CLI (`Main.swift`) can also call the check before creating the exporter to 185 + fail fast with a user-friendly message. 186 + - The menu bar app can call the check during its setup flow and present a native 187 + macOS dialog guiding the user to System Settings. 188 + 189 + **Why pre-flight, not fail-on-first-use:** 190 + 191 + - Avoids exporting 40 local assets successfully, then failing on the first 192 + iCloud-only one with a confusing Apple Event error 193 + - Gives the user one clear instruction upfront instead of a mid-backup error 194 + - For unattended runs (LaunchAgent), the backup fails immediately with an 195 + actionable log message rather than producing a partial result 196 + 197 + ### Timeout 198 + 199 + Per-asset timeout for AppleScript exports: use the existing `timeoutForBytes` 200 + formula (5 min base + 1 min per 100 MB). Ladder doesn't know the file size 201 + upfront for iCloud-only assets, so use a generous default of 10 minutes per 202 + asset. If the `osascript` process exceeds this, kill it and report the asset as 203 + failed. 204 + 205 + ## Acceptance Criteria 206 + 207 + - [ ] iCloud-only assets that PhotoKit can't find are exported via AppleScript 208 + fallback 209 + - [ ] `ExportResponse` includes successful results for AppleScript-exported 210 + assets (same format as PhotoKit exports) 211 + - [ ] AppleScript exports are serialized (one at a time) 212 + - [ ] SHA-256 is computed for AppleScript-exported files 213 + - [ ] Disk space check before starting AppleScript exports (skip if < 2 GB) 214 + - [ ] Pre-flight permission check before any export work begins 215 + - [ ] Clear, actionable error message when Automation permission is missing 216 + - [ ] Per-asset timeout kills hung `osascript` processes 217 + - [ ] Existing PhotoKit export path is completely unchanged 218 + - [ ] All new code is behind protocols for testability and dual-use (CLI + GUI) 219 + - [ ] `ScriptExporter` protocol allows custom implementations per consumer 220 + - [ ] osxphotos credited in README 221 + - [ ] Tests cover: fallback triggers, script failure, timeout, disk space check, 222 + filename discovery, UUID format stripping, permission probe 223 + 224 + ## Implementation 225 + 226 + ### Files to create 227 + 228 + | File | Purpose | 229 + |------|---------| 230 + | `Sources/LadderKit/AppleScriptExporter.swift` | Protocol + real implementation for AppleScript export | 231 + | `Tests/AppleScriptExporterTests.swift` | Tests with mock script runner | 232 + 233 + ### Files to modify 234 + 235 + | File | Change | 236 + |------|--------| 237 + | `Sources/LadderKit/PhotoExporter.swift` | After PhotoKit fetch, pass missing UUIDs to AppleScriptExporter | 238 + | `Sources/LadderKit/Models.swift` | Add optional `exportMethod` field to `ExportResult` (debugging aid) | 239 + | `README.md` | Document fallback behavior, credit osxphotos | 240 + 241 + ### `AppleScriptExporter.swift` design 242 + 243 + ``` 244 + protocol ScriptExporter: Sendable 245 + func exportAsset(uuid: String, to directory: URL) async throws -> URL 246 + 247 + struct AppleScriptRunner: ScriptExporter 248 + - Strips /L0/001 suffix from UUID 249 + - Creates per-asset temp subdirectory 250 + - Runs osascript via Process() 251 + - Discovers exported file 252 + - Returns file URL 253 + 254 + - Timeout: kills Process after deadline 255 + - Disk space: checked before first export in batch 256 + ``` 257 + 258 + Key design: `ScriptExporter` protocol allows injection of a mock for testing 259 + **and** alternative implementations per consumer: 260 + - CLI: uses default `AppleScriptRunner` (runs `osascript` via `Process()`) 261 + - Menu bar app: can inject an `NSAppleScript`-based implementation (optional, 262 + the default also works) 263 + - Tests: inject a mock that returns preconfigured files 264 + 265 + `PhotoExporter` accepts an optional `ScriptExporter` parameter (default: 266 + `AppleScriptRunner()`). 267 + 268 + ### `PhotoExporter.export()` flow change 269 + 270 + ``` 271 + Current: 272 + 1. fetchAssets(uuids) → found + missing 273 + 2. Report missing as errors immediately 274 + 3. Export found via PhotoKit (concurrent) 275 + 4. Return results + errors 276 + 277 + New: 278 + 0. Pre-flight: scriptExporter?.checkPermissions() — fail fast if denied 279 + 1. fetchAssets(uuids) → found + missing 280 + 2. Export found via PhotoKit (concurrent, unchanged) 281 + 3. For missing UUIDs: try AppleScript fallback (sequential) 282 + a. Check disk space (skip all if < 2 GB) 283 + b. For each missing UUID: 284 + - Create temp subdir 285 + - Run osascript with timeout 286 + - Discover exported file 287 + - Hash with FileHasher 288 + - Move to staging path 289 + - Build ExportResult 290 + c. Report failures as errors 291 + 4. Return combined results + errors 292 + ``` 293 + 294 + PhotoKit exports run first (fast, concurrent). AppleScript exports run second 295 + (slow, sequential). This way the fast path is never blocked by iCloud downloads. 296 + 297 + ### AppleScript command 298 + 299 + ```applescript 300 + tell application "Photos" 301 + set thePic to media item id "{bare_uuid}" 302 + export {thePic} to POSIX file "{temp_subdir}" with using originals 303 + end tell 304 + ``` 305 + 306 + Executed via: 307 + ```swift 308 + let process = Process() 309 + process.executableURL = URL(fileURLWithPath: "/usr/bin/osascript") 310 + process.arguments = ["-e", script] 311 + ``` 312 + 313 + ### Error handling 314 + 315 + | Scenario | Behavior | 316 + |----------|----------| 317 + | Automation permission denied (-1743) | Error: "Grant Automation permission for ladder to control Photos in System Settings" | 318 + | Photos.app modal dialog (blocks export) | Timeout fires, kill process, report as failed | 319 + | Asset not found in Photos.app either | Error: "Asset not found in Photos library or Photos.app" | 320 + | iCloud download fails (network error) | Error from osascript, reported as export failure | 321 + | Disk space < 2 GB | Skip all remaining AppleScript exports, log warning | 322 + | Export produces no file | Error: "AppleScript export produced no file" | 323 + | Export produces multiple files (Live Photo) | Take image file, ignore video component | 324 + 325 + ## Verification 326 + 327 + 1. `swift build | xcsift` — compiles 328 + 2. `swift test | xcsift` — all existing + new tests pass 329 + 3. Manual: run ladder with a known iCloud-only UUID, verify AppleScript fallback 330 + triggers and produces correct output 331 + 4. Manual: run `attic backup --limit 5` with iCloud-only assets in the batch, 332 + verify end-to-end flow 333 + 334 + ## Sources 335 + 336 + - [osxphotos](https://github.com/RhetTbull/osxphotos) (MIT) — AppleScript 337 + export pattern for iCloud-only assets 338 + - `PhotoExporter.swift:22-24` — current "Asset not found" error generation 339 + - `PhotoLibrary.swift:39-54` — `fetchAssets()` where iCloud-only assets are 340 + invisible 341 + - `Hasher.swift:41-58` — `FileHasher.sha256(fileAt:)` for post-export hashing