this repo has no description
TODO
+280
-2
+280
-2
toru/TODO.md
+280
-2
toru/TODO.md
···
1
-
# Toru TODO: Missing Features vs Python Pooch
1
+
# Toru TODO: XDG-Eio Integration & Missing Features
2
+
3
+
This document tracks the XDG-Eio integration plan and features missing from the OCaml Toru implementation compared to the Python Pooch library.
4
+
5
+
## XDG-Eio Integration Plan
6
+
7
+
### Phase 1: Core Interface Updates
8
+
9
+
#### 1. Update Cache Module (`lib/cache.mli`)
10
+
- [ ] Change constructor to accept `Xdge.t` instead of explicit cache path
11
+
- [ ] Add `xdg` field accessor
12
+
- [ ] Keep version support for subdirectory organization
13
+
- [ ] Use `Xdge.cache_dir` as base path
14
+
- [ ] Follow XDG pretty-printing conventions
15
+
16
+
```ocaml
17
+
module Cache : sig
18
+
type t
19
+
20
+
val create : xdg:Xdge.t -> ?version:string -> unit -> t
21
+
22
+
(* Field accessors *)
23
+
val xdg : t -> Xdge.t
24
+
val base_path : t -> Eio.Fs.dir_ty Eio.Path.t
25
+
val version : t -> string option
26
+
27
+
(* Operations unchanged *)
28
+
val file_path : t -> string -> Eio.Fs.dir_ty Eio.Path.t
29
+
val exists : t -> string -> bool
30
+
val ensure_dir : t -> unit
31
+
val clear : t -> unit
32
+
val size_bytes : t -> int64
33
+
val list_files : t -> string list
34
+
35
+
(* XDG-compliant pretty printing *)
36
+
val pp : Format.formatter -> t -> unit
37
+
end
38
+
```
39
+
40
+
#### 2. Enhance Registry Module (`lib/registry.mli`)
41
+
- [ ] Add multi-source support (files, URLs, strings)
42
+
- [ ] Add registry merging capabilities (later sources override earlier)
43
+
- [ ] Add XDG integration for loading/saving registry files
44
+
- [ ] Support Pooch's multiple registry pattern
45
+
46
+
```ocaml
47
+
module Registry : sig
48
+
type t
49
+
type entry
50
+
type source =
51
+
| File of Eio.Fs.dir_ty Eio.Path.t
52
+
| Url of string
53
+
| String of string
54
+
| Xdg_file of Xdge.t * string (* Search in XDG config dirs *)
55
+
56
+
(* Entry operations unchanged *)
57
+
val create_entry : filename:string -> hash:Hash.t -> ?custom_url:string -> unit -> entry
58
+
val filename : entry -> string
59
+
val hash : entry -> Hash.t
60
+
val custom_url : entry -> string option
61
+
62
+
(* Multi-source registry support *)
63
+
val empty : t
64
+
val create : source list -> t
65
+
val load_sources : source list -> t
66
+
val add_source : t -> source -> t
67
+
val sources : t -> source list
68
+
val merge : t list -> t
69
+
70
+
(* XDG integration *)
71
+
val load_from_xdg : Xdge.t -> ?filename:string -> (t, string) result
72
+
val save_to_xdg : Xdge.t -> ?filename:string -> t -> (unit, string) result
73
+
val find_registry_file : Xdge.t -> string -> Eio.Fs.dir_ty Eio.Path.t option
74
+
75
+
(* Legacy single-file operations *)
76
+
val load : Eio.Fs.dir_ty Eio.Path.t -> t
77
+
val load_from_url : string -> t
78
+
val save : Eio.Fs.dir_ty Eio.Path.t -> t -> unit
79
+
val of_string : string -> t
80
+
val to_string : t -> string
81
+
82
+
(* Query operations unchanged *)
83
+
val find : string -> t -> entry option
84
+
val exists : string -> t -> bool
85
+
val add : entry -> t -> t
86
+
val remove : string -> t -> t
87
+
val entries : t -> entry list
88
+
val size : t -> int
89
+
90
+
(* XDG-compliant pretty printing *)
91
+
val pp : Format.formatter -> t -> unit
92
+
val pp_sources : Format.formatter -> source list -> unit
93
+
end
94
+
```
95
+
96
+
#### 3. Update Toru Main Interface (`lib/toru.mli`)
97
+
- [ ] Accept `Xdge.t` as primary parameter
98
+
- [ ] Remove `cache_path` parameter (use xdg cache directory)
99
+
- [ ] Accept `Registry.t` instead of registry file path
100
+
- [ ] Get app name from xdg context
101
+
- [ ] Add XDG-compliant pretty printing
2
102
3
-
This document tracks features missing from the OCaml Toru implementation compared to the Python Pooch library.
103
+
```ocaml
104
+
module Toru : sig
105
+
type t
106
+
107
+
val create :
108
+
xdg:Xdge.t ->
109
+
base_url:string ->
110
+
registry:Registry.t ->
111
+
?version:string ->
112
+
?downloader:(module DOWNLOADER) ->
113
+
unit -> t
114
+
115
+
(* Field accessors *)
116
+
val xdg : t -> Xdge.t
117
+
val app_name : t -> string (* derived from xdg *)
118
+
val base_url : t -> string
119
+
val cache : t -> Cache.t
120
+
val registry : t -> Registry.t
121
+
122
+
(* Operations unchanged *)
123
+
val fetch :
124
+
t ->
125
+
filename:string ->
126
+
?processor:(Eio.Fs.dir_ty Eio.Path.t -> Eio.Fs.dir_ty Eio.Path.t) ->
127
+
unit -> (Eio.Fs.dir_ty Eio.Path.t, string) result
128
+
129
+
val fetch_all :
130
+
t ->
131
+
?concurrency:int ->
132
+
unit -> (unit, string) result
133
+
134
+
val load_registry : t -> Registry.t -> t
135
+
val add_registry_entry : t -> Registry.entry -> t
136
+
val update_base_url : t -> string -> t
137
+
138
+
(* Static functions - add xdg parameter *)
139
+
val retrieve :
140
+
xdg:Xdge.t ->
141
+
url:string ->
142
+
?hash:Hash.t ->
143
+
?version:string ->
144
+
?downloader:(module DOWNLOADER) ->
145
+
unit -> (Eio.Fs.dir_ty Eio.Path.t, string) result
146
+
147
+
(* XDG-compliant pretty printing *)
148
+
val pp : Format.formatter -> t -> unit
149
+
val pp_brief : Format.formatter -> t -> unit
150
+
end
151
+
```
152
+
153
+
### Phase 2: XDG Directory Usage
154
+
155
+
#### 4. Add Configuration Support (`lib/config.mli`)
156
+
- [ ] Create new Config module for application settings
157
+
- [ ] Use `Xdge.config_dir` for storing configuration files
158
+
- [ ] Support TOML configuration format
159
+
- [ ] Use `Xdge.find_config_file` for config discovery
160
+
161
+
```ocaml
162
+
module Config : sig
163
+
type t = {
164
+
base_urls : string list;
165
+
default_downloader : string option;
166
+
timeout : float option;
167
+
concurrency : int option;
168
+
registry_sources : Registry.source list;
169
+
}
170
+
171
+
val default : t
172
+
val load : Xdge.t -> (t, string) result
173
+
val save : Xdge.t -> t -> (unit, string) result
174
+
val find_config_file : Xdge.t -> (Eio.Fs.dir_ty Eio.Path.t, string) result
175
+
176
+
(* XDG-compliant pretty printing *)
177
+
val pp : Format.formatter -> t -> unit
178
+
end
179
+
```
180
+
181
+
#### 5. Add State Management (`lib/state.mli`)
182
+
- [ ] Create State module for download history and logs
183
+
- [ ] Use `Xdge.state_dir` for persistent state
184
+
- [ ] Track download statistics and failures
185
+
- [ ] Implement download resume capability
186
+
187
+
```ocaml
188
+
module State : sig
189
+
type t
190
+
type download_entry = {
191
+
filename : string;
192
+
url : string;
193
+
hash : Hash.t option;
194
+
downloaded_at : Ptime.t;
195
+
success : bool;
196
+
error_msg : string option;
197
+
}
198
+
199
+
val create : Xdge.t -> t
200
+
val load : Xdge.t -> (t, string) result
201
+
val save : Xdge.t -> t -> (unit, string) result
202
+
203
+
val add_download : t -> download_entry -> t
204
+
val recent_downloads : t -> int -> download_entry list
205
+
val failed_downloads : t -> download_entry list
206
+
val download_count : t -> int
207
+
val success_rate : t -> float
208
+
209
+
(* XDG-compliant pretty printing *)
210
+
val pp : Format.formatter -> t -> unit
211
+
val pp_recent : Format.formatter -> t -> unit
212
+
end
213
+
```
214
+
215
+
#### 6. Add Data Directory Usage
216
+
- [ ] Use `Xdge.data_dir` for user-installed registries
217
+
- [ ] Store custom download processors in data directory
218
+
- [ ] Support user plugin/extension discovery
219
+
220
+
### Phase 3: Toru-Specific XDG Extensions
221
+
222
+
#### 7. Create Toru XDG Extensions (`lib/toru_xdg.mli`)
223
+
- [ ] Keep Toru-specific XDG functionality separate from xdg-eio
224
+
- [ ] Add archive extraction support using temp directories
225
+
- [ ] Add download lock file management
226
+
- [ ] Add cache size management utilities
227
+
228
+
```ocaml
229
+
module Toru_xdg : sig
230
+
(* Temporary directory management *)
231
+
val temp_dir : Xdge.t -> Eio.Fs.dir_ty Eio.Path.t
232
+
val with_temp_dir : Xdge.t -> (Eio.Fs.dir_ty Eio.Path.t -> 'a) -> 'a
233
+
234
+
(* Download locking *)
235
+
val with_download_lock : Xdge.t -> string -> (unit -> 'a) -> ('a, string) result
236
+
237
+
(* Cache management *)
238
+
val cache_disk_usage : Xdge.t -> int64
239
+
val cache_cleanup : Xdge.t -> ?max_size:int64 -> ?max_age:Ptime.Span.t -> unit -> int
240
+
val validate_cache_writable : Xdge.t -> (unit, string) result
241
+
242
+
(* Archive extraction *)
243
+
val extract_to_cache : Xdge.t ->
244
+
archive_path:Eio.Fs.dir_ty Eio.Path.t ->
245
+
extract_subdir:string ->
246
+
(Eio.Fs.dir_ty Eio.Path.t, string) result
247
+
end
248
+
```
249
+
250
+
### Phase 4: Examples and Documentation Updates
251
+
252
+
#### 8. Update Examples
253
+
- [ ] Update example code to use `Xdge.t`
254
+
- [ ] Show registry multi-source patterns
255
+
- [ ] Demonstrate configuration file usage
256
+
- [ ] Show state/logging integration
257
+
258
+
#### 9. Add Cmdliner Integration Example
259
+
- [ ] Create example showing `Xdge.Cmd.term` integration
260
+
- [ ] Demonstrate automatic XDG directory CLI flags
261
+
- [ ] Show environment variable precedence
262
+
263
+
### Phase 5: Testing and Migration
264
+
265
+
#### 10. Update Tests
266
+
- [ ] Test XDG directory creation and permissions
267
+
- [ ] Test multi-source registry loading
268
+
- [ ] Test configuration file discovery
269
+
- [ ] Test state persistence
270
+
- [ ] Add integration tests with real xdg-eio
271
+
272
+
#### 11. Migration Guide
273
+
- [ ] Document migration from explicit cache paths to XDG
274
+
- [ ] Provide migration utility for existing cache directories
275
+
- [ ] Document breaking changes in API
276
+
277
+
### Dependencies to Add
278
+
- [ ] Add `xdge` dependency to dune-project
279
+
- [ ] Add `ptime` for timestamp handling in state management
280
+
- [ ] Add `toml` library for configuration files (optional)
281
+
- [ ] Update OCaml-EIO-README.md reference if needed
4
282
5
283
## Major Missing Features in Toru
6
284