forked from
anil.recoil.org/monopam-myspace
My aggregated monorepo of OCaml code, automaintained
docs: design for day10 --local-repo option
Allows testing local (unpushed) changes to js_top_worker and odoc
in day10's container builds by bind-mounting local repos and pinning
opam packages from the local path.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
+169
+169
docs/plans/2026-02-21-day10-local-repo-design.md
+169
docs/plans/2026-02-21-day10-local-repo-design.md
···
1
+
# day10 --local-repo: Local Repository Override for Container Builds
2
+
3
+
## Problem
4
+
5
+
When developing changes to js_top_worker or odoc, the only way to test
6
+
them in day10's container pipeline is to push commits to the remote git
7
+
repo. This creates a frustrating cycle: edit locally, commit, push, wait
8
+
for SSH, rebuild. We need a way to use local (uncommitted) source
9
+
directly.
10
+
11
+
## Solution
12
+
13
+
A new repeatable CLI option `--local-repo PATH` that bind-mounts a local
14
+
directory into the build container and pins its opam packages from the
15
+
local path instead of from a git URL.
16
+
17
+
```
18
+
day10 run yojson --with-jtw --local-repo ~/mono/js_top_worker
19
+
day10 run yojson --with-doc --local-repo ~/mono/odoc
20
+
day10 run yojson --with-doc --with-jtw \
21
+
--local-repo ~/mono/js_top_worker \
22
+
--local-repo ~/mono/odoc
23
+
```
24
+
25
+
## Current Pin Structure
26
+
27
+
day10 pins packages from two git repos:
28
+
29
+
**js_top_worker** (`--jtw-tools-repo` / `--jtw-tools-branch`):
30
+
- js_top_worker
31
+
- js_top_worker-rpc
32
+
- js_top_worker-bin
33
+
- js_top_worker-web
34
+
- js_top_worker_rpc_def
35
+
36
+
**odoc** (`--doc-tools-repo` / `--doc-tools-branch`):
37
+
- odoc, odoc-parser (per-version odoc layer)
38
+
- odoc, odoc-parser, odoc-md, sherlodoc, odoc-driver (shared driver layer)
39
+
40
+
## Design
41
+
42
+
### Package Discovery
43
+
44
+
Each `--local-repo PATH` is scanned for `*.opam` files at the root
45
+
(maxdepth 1). The basename minus `.opam` gives the package name:
46
+
47
+
```
48
+
~/mono/js_top_worker/*.opam →
49
+
js_top_worker, js_top_worker-rpc, js_top_worker-bin,
50
+
js_top_worker-web, js_top_worker_rpc_def,
51
+
js_top_worker-client, js_top_worker-client_fut, js_top_worker-unix
52
+
```
53
+
54
+
When build scripts generate `opam pin` commands, packages found in a
55
+
local repo are pinned from the local mount path; packages not found in
56
+
any local repo are pinned from the git URL as before.
57
+
58
+
### Container Integration
59
+
60
+
The local directory is bind-mounted read-only into the container:
61
+
62
+
```
63
+
{ ty = "bind"; src = "/home/jon/mono/js_top_worker";
64
+
dst = "/home/opam/local/js_top_worker";
65
+
options = [ "ro"; "rbind"; "rprivate" ] }
66
+
```
67
+
68
+
Build scripts change from:
69
+
```
70
+
opam pin add -yn js_top_worker git+https://tangled.org/.../js_top_worker#main
71
+
```
72
+
to:
73
+
```
74
+
opam pin add -yn js_top_worker /home/opam/local/js_top_worker
75
+
```
76
+
77
+
### Caching
78
+
79
+
Layer hashes currently include `(repo, branch)`. With `--local-repo`,
80
+
the hash instead includes the local repo's git state:
81
+
82
+
- **Git repo, clean:** `local:<path>|<HEAD sha>`
83
+
- **Git repo, dirty:** `local:<path>|<HEAD sha>|dirty-<diff hash>`
84
+
- **Not a git repo:** `local:<path>|<content hash of *.opam files>`
85
+
86
+
This means unchanged local repos reuse the cached layer, while any
87
+
modification triggers a rebuild. This is important because jtw-tools
88
+
layers take several minutes to build (OCaml compiler + deps).
89
+
90
+
### Precedence
91
+
92
+
`--local-repo` takes precedence over `--jtw-tools-repo` and
93
+
`--doc-tools-repo` for any overlapping packages. The git repo/branch
94
+
flags still apply to packages not covered by any local repo.
95
+
96
+
### Config Changes
97
+
98
+
```ocaml
99
+
type t = {
100
+
...
101
+
local_repos : string list; (* paths from --local-repo *)
102
+
...
103
+
}
104
+
```
105
+
106
+
### CLI
107
+
108
+
```
109
+
--local-repo PATH Use local directory for opam pins instead of git
110
+
(repeatable)
111
+
```
112
+
113
+
Implemented as `Arg.(value & opt_all string [] & info ["local-repo"])`.
114
+
115
+
## Affected Files
116
+
117
+
1. **config.ml** -- add `local_repos : string list`
118
+
2. **main.ml** -- add `--local-repo` CLI term, wire into Config.t
119
+
3. **jtw_tools.ml** -- `build_script` checks local repos for matching
120
+
packages, `layer_hash` includes local git state
121
+
4. **doc_tools.ml** -- same pattern for `driver_build_script`,
122
+
`odoc_build_script`, and their layer hashes
123
+
5. **linux.ml** -- `ensure_jtw_tools_layer`, `ensure_driver_layer`,
124
+
`ensure_odoc_layer` add bind mounts; `run_jtw_in_container` adds
125
+
bind mount for jtw per-package builds too
126
+
127
+
## Helper Functions
128
+
129
+
A small `Local_repo` module (or functions in config.ml):
130
+
131
+
```ocaml
132
+
val discover_packages : string -> string list
133
+
(** Scan *.opam files at root of path, return package names *)
134
+
135
+
val local_repo_hash : string -> string
136
+
(** Compute cache hash for a local repo path using git state *)
137
+
138
+
val find_local_for_packages : local_repos:string list -> string list -> (string * string list) option
139
+
(** Given package names to pin, find a local_repo containing them.
140
+
Returns (path, matched_packages). *)
141
+
```
142
+
143
+
## Validation & Error Handling
144
+
145
+
- Path must exist and be a directory: error at startup if not
146
+
- Path must contain at least one `*.opam` file: warning if empty
147
+
- Two local repos providing the same package: error (ambiguous)
148
+
- Local repo packages that don't match any pin list: silently ignored
149
+
150
+
## Data Flow
151
+
152
+
```
153
+
CLI: --local-repo ~/mono/js_top_worker
154
+
↓
155
+
Config.t.local_repos = ["/home/jon/mono/js_top_worker"]
156
+
↓
157
+
Jtw_tools.layer_hash:
158
+
finds js_top_worker.opam in local repo
159
+
→ includes git HEAD + dirty state in hash
160
+
↓
161
+
Jtw_tools.build_script:
162
+
for js_top_worker, js_top_worker-rpc, etc:
163
+
→ "opam pin add -yn <pkg> /home/opam/local/js_top_worker"
164
+
↓
165
+
linux.ml ensure_jtw_tools_layer:
166
+
adds bind mount: local path → /home/opam/local/js_top_worker
167
+
↓
168
+
Container runs build_script with local source available
169
+
```