An easy-to-use platform for EEG experimentation in the classroom
packages are loading!
+185
-31
+7
.llms/learnings.md
+7
.llms/learnings.md
···
49
49
50
50
The CDN version is derived from `node_modules/pyodide/package.json` — **not** from `pyodide-lock.json`'s `info.version`, which may be a dev label like `0.28.0.dev0`.
51
51
52
+
**Packages that must be listed explicitly** (not reachable from matplotlib/scipy/pandas deps in the lock file, but required at runtime):
53
+
- `jinja2` + `markupsafe` — used by matplotlib templates and MNE HTML reports
54
+
- `decorator` — MNE core dep
55
+
- `requests` (+ `certifi`, `charset-normalizer`, `idna`, `urllib3`) — pulled in by `pooch` at MNE import time
56
+
57
+
**`micropip.install()` from JS accepts a JS array directly** — as of Pyodide 0.29.x, micropip handles the `JsProxy` conversion internally. `pyodide.toPy()` is not needed.
58
+
52
59
## Pre-existing TypeScript errors (do not treat as regressions)
53
60
54
61
- `src/renderer/epics/experimentEpics.ts` (lines 170, 205) — RxJS operator type mismatch
+176
docs/pyodide-in-electron-vite.md
+176
docs/pyodide-in-electron-vite.md
···
1
+
# Pyodide in Electron + Vite: What We Learned
2
+
3
+
This document captures everything we learned getting Pyodide (Python-in-WASM) running reliably inside an Electron + electron-vite app. It is intended as a reference for anyone maintaining or upgrading the Pyodide integration.
4
+
5
+
---
6
+
7
+
## The Core Problem: Vite's SPA Fallback
8
+
9
+
Vite's dev server runs a `historyApiFallback` middleware that returns `index.html` for **every** `fetch()` request it doesn't recognise — including `/@fs/` paths, `publicDir` paths, and anything from a web worker. This completely breaks Pyodide's package loader, which `fetch()`es `.whl` files at runtime.
10
+
11
+
This is not an obvious failure — Pyodide may partially initialise and then hang or throw cryptic errors when it tries to load packages.
12
+
13
+
### Solution: Serve Pyodide Assets Out-of-Band
14
+
15
+
We use two complementary mechanisms:
16
+
17
+
**1. Custom Vite middleware (dev only)**
18
+
19
+
In `vite.config.ts`, a plugin intercepts requests to `/pyodide/` and `/packages/` before the SPA fallback runs and streams the files directly from `src/renderer/utils/webworker/src/`:
20
+
21
+
```ts
22
+
server.middlewares.use((req, res, next) => {
23
+
const url = req.url ?? '';
24
+
if (url.startsWith('/pyodide/') || url.startsWith('/packages/')) {
25
+
const filePath = path.join(staticDir, url.split('?')[0]);
26
+
if (fs.existsSync(filePath) && fs.statSync(filePath).isFile()) {
27
+
res.setHeader('Content-Type', contentTypes[ext] ?? 'application/octet-stream');
28
+
fs.createReadStream(filePath).pipe(res);
29
+
return;
30
+
}
31
+
}
32
+
next();
33
+
});
34
+
```
35
+
36
+
**2. Electron local HTTP server on port 17173 (dev + prod)**
37
+
38
+
Web workers cannot use Vite's dev server at all — `fetch()` from a worker always hits the SPA fallback. The main process (`src/main/index.ts`) starts a plain Node.js `http` server at `http://127.0.0.1:17173` that serves `src/renderer/utils/webworker/src/` (dev) or `resources/webworker/src/` (prod).
39
+
40
+
The web worker (`webworker.js`) uses this as its `PYODIDE_ASSET_BASE`:
41
+
42
+
```js
43
+
const PYODIDE_ASSET_BASE = 'http://127.0.0.1:17173';
44
+
```
45
+
46
+
Port 17173 is hardcoded in three places that must stay in sync:
47
+
- `src/main/index.ts` — server listen port
48
+
- `src/renderer/utils/webworker/webworker.js` — `PYODIDE_ASSET_BASE`
49
+
- `src/renderer/index.html` — CSP `connect-src` directive
50
+
51
+
---
52
+
53
+
## Loading pyodide.mjs
54
+
55
+
Pyodide 0.26+ ships as an ES module (`pyodide.mjs`). You cannot `fetch()` it — Vite would intercept it. Instead, import it with Vite's `?url` suffix and use dynamic `import()`:
56
+
57
+
```js
58
+
import pyodideMjsUrl from 'pyodide/pyodide.mjs?url';
59
+
// ...
60
+
const { loadPyodide } = await import(/* @vite-ignore */ pyodideMjsUrl);
61
+
```
62
+
63
+
`import()` bypasses Vite's SPA fallback; `fetch()` does not.
64
+
65
+
Also required in `vite.config.ts`:
66
+
67
+
```ts
68
+
optimizeDeps: {
69
+
exclude: ['pyodide'], // prevent Vite from pre-bundling it
70
+
},
71
+
worker: {
72
+
format: 'es', // ES module workers required for pyodide.mjs
73
+
},
74
+
```
75
+
76
+
And workers must be created with `type: 'module'`:
77
+
78
+
```ts
79
+
new Worker(new URL('./webworker.js', import.meta.url), { type: 'module' });
80
+
```
81
+
82
+
---
83
+
84
+
## Loading the Lock File Without a Fetch
85
+
86
+
`loadPyodide` needs the lock file to resolve package names to filenames. Fetching it would hit Vite's SPA fallback. Instead, embed it at build time with Vite's `?raw` suffix and wrap it in a blob URL:
87
+
88
+
```js
89
+
import lockFileRaw from 'pyodide/pyodide-lock.json?raw';
90
+
91
+
const lockBlob = new Blob([lockFileRaw], { type: 'application/json' });
92
+
const lockFileURL = URL.createObjectURL(lockBlob);
93
+
const pyodide = await loadPyodide({ lockFileURL, packageBaseUrl });
94
+
URL.revokeObjectURL(lockFileURL);
95
+
```
96
+
97
+
---
98
+
99
+
## packageBaseUrl vs indexURL
100
+
101
+
These are easy to confuse:
102
+
103
+
| Option | Purpose |
104
+
|--------|---------|
105
+
| `indexURL` | Where Pyodide looks for its **runtime** files (WASM, stdlib). Already resolved from `node_modules` via `import.meta.url`. Do not override. |
106
+
| `packageBaseUrl` | Where `loadPackage()` fetches **package `.whl` files**. Set this to `http://127.0.0.1:17173/pyodide/`. |
107
+
108
+
---
109
+
110
+
## Package Integrity Checks
111
+
112
+
```js
113
+
await pyodide.loadPackage(['numpy', 'scipy', ...], { checkIntegrity: false });
114
+
```
115
+
116
+
`checkIntegrity: false` is required. The SHA-256 hashes in the npm package's `pyodide-lock.json` are computed against the CDN files, but we serve locally-downloaded copies that may differ (e.g. re-compressed). Integrity checks will fail without this flag.
117
+
118
+
---
119
+
120
+
## micropip.install() from JavaScript
121
+
122
+
`micropip` is a Python object loaded via `pyodide.pyimport()`. Passing a JavaScript array directly works fine in Pyodide 0.29.x — micropip handles the `JsProxy` conversion internally:
123
+
124
+
```js
125
+
const micropip = pyodide.pyimport('micropip');
126
+
await micropip.install(whlUrls); // JS array works directly
127
+
```
128
+
129
+
> Note: older guidance suggested wrapping with `pyodide.toPy(whlUrls)` — this is not necessary as of 0.29.x.
130
+
131
+
---
132
+
133
+
## Offline Package Installation (InstallMNE.mjs)
134
+
135
+
`internals/scripts/InstallMNE.mjs` runs on `postinstall` and pre-downloads all packages so the app works offline.
136
+
137
+
### Part 1 — Pyodide Binary Packages (from Pyodide CDN)
138
+
139
+
These are compiled packages bundled with Pyodide. The script reads `pyodide-lock.json`, recursively resolves transitive dependencies of the root packages, and downloads each `.whl` into `src/renderer/utils/webworker/src/pyodide/`.
140
+
141
+
**Derive the CDN version from `node_modules/pyodide/package.json`**, not from `pyodide-lock.json`'s `info.version` field — that field may be a dev label like `0.28.0.dev0` and will produce a broken CDN URL.
142
+
143
+
Current root packages and why each is listed explicitly:
144
+
145
+
| Package | Reason |
146
+
|---------|--------|
147
+
| `numpy`, `scipy`, `matplotlib`, `pandas` | Core scientific stack |
148
+
| `micropip` | Needed to install pure-Python packages at worker startup |
149
+
| `pillow` | Used by matplotlib and MNE; loaded at runtime by `loadPackage()` |
150
+
| `jinja2` | MNE dep; **not** listed in matplotlib's lock-file `depends` array despite being a runtime requirement |
151
+
| `decorator` | MNE core dep; not reachable from the scientific stack in the lock file |
152
+
| `requests` | Pulled in by `pooch` at MNE import time; brings in `certifi`, `charset-normalizer`, `idna`, `urllib3` transitively |
153
+
154
+
> **Gotcha:** The `depends` arrays in `pyodide-lock.json` are incomplete. Several packages that matplotlib, scipy, or MNE require at runtime are not listed as dependencies and will not be downloaded unless added explicitly as roots.
155
+
156
+
### Part 2 — Pure-Python Packages (from PyPI)
157
+
158
+
Packages not bundled with Pyodide must be downloaded as `py3-none-any` wheels from PyPI. They are stored in `src/renderer/utils/webworker/src/packages/` and a `manifest.json` is written so the worker knows the exact filenames.
159
+
160
+
Current PyPI packages: `mne`, `pooch`, `tqdm`, `platformdirs`, `lazy-loader`
161
+
162
+
`lazy-loader` is a core MNE dependency that does not appear in the Pyodide lock at all.
163
+
164
+
---
165
+
166
+
## Summary of File Locations
167
+
168
+
| What | Where |
169
+
|------|-------|
170
+
| Pyodide runtime + binary wheels | `src/renderer/utils/webworker/src/pyodide/` |
171
+
| Pure-Python wheels + manifest | `src/renderer/utils/webworker/src/packages/` |
172
+
| Web worker entry point | `src/renderer/utils/webworker/webworker.js` |
173
+
| JS wrappers for Python calls | `src/renderer/utils/webworker/index.ts` |
174
+
| Install script | `internals/scripts/InstallMNE.mjs` |
175
+
| Electron asset server | `src/main/index.ts` → `startPyodideAssetServer()` |
176
+
| Vite middleware | `vite.config.ts` → `serve-pyodide-assets` plugin |
+2
-2
internals/scripts/InstallMNE.mjs
+2
-2
internals/scripts/InstallMNE.mjs
···
39
39
// Root packages whose full transitive dependency tree we need from Pyodide CDN
40
40
// ---------------------------------------------------------------------------
41
41
42
-
const PYODIDE_ROOT_PACKAGES = ['numpy', 'scipy', 'matplotlib', 'pandas', 'micropip'];
42
+
const PYODIDE_ROOT_PACKAGES = ['numpy', 'scipy', 'matplotlib', 'pandas', 'micropip', 'pillow', 'jinja2', 'decorator', 'requests'];
43
43
44
44
// ---------------------------------------------------------------------------
45
45
// Pure-Python packages to download from PyPI (not bundled with Pyodide)
46
46
// ---------------------------------------------------------------------------
47
47
48
-
const PYPI_PACKAGES = ['mne', 'pooch', 'tqdm', 'platformdirs'];
48
+
const PYPI_PACKAGES = ['mne', 'pooch', 'tqdm', 'platformdirs', 'lazy-loader'];
49
49
50
50
// ---------------------------------------------------------------------------
51
51
// Shared network helpers
-29
src/renderer/utils/webworker/patches.py
-29
src/renderer/utils/webworker/patches.py
···
1
-
# patch implemented in Pyolite
2
-
# https://github.com/jupyterlite/jupyterlite/blob/0d563b9a4cca4b54411229128cb51ac4ba333c8f/packages/pyolite-kernel/py/pyolite/pyolite/patches.py
3
-
def patch_matplotlib():
4
-
import os
5
-
from io import BytesIO
6
-
7
-
# before importing matplotlib
8
-
# to avoid the wasm backend (which needs `js.document`, not available in worker)
9
-
os.environ["MPLBACKEND"] = "AGG"
10
-
11
-
import matplotlib.pyplot
12
-
from IPython.display import display
13
-
14
-
from .display import Image
15
-
16
-
_old_show = matplotlib.pyplot.show
17
-
assert _old_show, "matplotlib.pyplot.show"
18
-
19
-
def show():
20
-
buf = BytesIO()
21
-
matplotlib.pyplot.savefig(buf, format="png")
22
-
buf.seek(0)
23
-
display(Image(buf.read()))
24
-
matplotlib.pyplot.clf()
25
-
26
-
matplotlib.pyplot.show = show
27
-
28
-
29
1
def patch_pillow():
30
2
import base64
31
3
···
43
15
44
16
ALL_PATCHES = [
45
17
patch_pillow,
46
-
patch_matplotlib,
47
18
]
48
19
49
20