An easy-to-use platform for EEG experimentation in the classroom
update notes on pyodide in electron vite
also some cleanup of utils functions for Python
+61
-40
+59
-32
docs/pyodide-in-electron-vite.md
+59
-32
docs/pyodide-in-electron-vite.md
···
10
10
11
11
This is not an obvious failure — Pyodide may partially initialise and then hang or throw cryptic errors when it tries to load packages.
12
12
13
-
### Solution: Serve Pyodide Assets Out-of-Band
13
+
### Solution: Electron Custom Protocol Scheme (`pyodide://`)
14
14
15
-
We use two complementary mechanisms:
15
+
We register a privileged custom Electron protocol scheme that serves Pyodide assets directly from the filesystem — no network socket, no Vite interception, works identically in dev and production.
16
16
17
-
**1. Custom Vite middleware (dev only)**
17
+
**Registration in `src/main/index.ts` — must happen before `app.whenReady()`:**
18
18
19
-
In `vite.config.ts`, a plugin intercepts requests to `/pyodide/` and `/packages/` before the SPA fallback runs and streams the files directly from `src/renderer/utils/webworker/src/`:
19
+
```ts
20
+
protocol.registerSchemesAsPrivileged([{
21
+
scheme: 'pyodide',
22
+
privileges: {
23
+
standard: true, // treat like http for URL parsing / resolution
24
+
secure: true, // counts as a secure origin (needed for WASM, SAB)
25
+
supportFetchAPI: true, // allow fetch() from renderer and worker contexts
26
+
corsEnabled: true, // no CORS errors when Pyodide fetches its own assets
27
+
},
28
+
}]);
29
+
```
30
+
31
+
**Handler registered in `app.whenReady()`:**
20
32
21
33
```ts
22
-
server.middlewares.use((req, res, next) => {
23
-
const url = req.url ?? '';
24
-
if (url.startsWith('/pyodide/') || url.startsWith('/packages/')) {
25
-
const filePath = path.join(staticDir, url.split('?')[0]);
26
-
if (fs.existsSync(filePath) && fs.statSync(filePath).isFile()) {
27
-
res.setHeader('Content-Type', contentTypes[ext] ?? 'application/octet-stream');
28
-
fs.createReadStream(filePath).pipe(res);
29
-
return;
30
-
}
31
-
}
32
-
next();
34
+
const pyodideRoot = is.dev
35
+
? path.join(app.getAppPath(), 'src/renderer/utils/webworker/src')
36
+
: path.join(process.resourcesPath, 'webworker/src');
37
+
38
+
protocol.handle('pyodide', (request) => {
39
+
const { pathname } = new URL(request.url);
40
+
const filePath = path.join(pyodideRoot, pathname);
41
+
return net.fetch(pathToFileURL(filePath).href);
33
42
});
34
43
```
35
44
36
-
**2. Electron local HTTP server on port 17173 (dev + prod)**
45
+
The web worker uses `pyodide://host` as its asset base:
37
46
38
-
Web workers cannot use Vite's dev server at all — `fetch()` from a worker always hits the SPA fallback. The main process (`src/main/index.ts`) starts a plain Node.js `http` server at `http://127.0.0.1:17173` that serves `src/renderer/utils/webworker/src/` (dev) or `resources/webworker/src/` (prod).
47
+
```js
48
+
const PYODIDE_ASSET_BASE = 'pyodide://host';
49
+
```
39
50
40
-
The web worker (`webworker.js`) uses this as its `PYODIDE_ASSET_BASE`:
51
+
**CSP in `src/renderer/index.html`** includes `pyodide:` in `connect-src`:
41
52
42
-
```js
43
-
const PYODIDE_ASSET_BASE = 'http://127.0.0.1:17173';
53
+
```html
54
+
connect-src 'self' ws: wss: webpack: pyodide:
44
55
```
45
56
46
-
Port 17173 is hardcoded in three places that must stay in sync:
47
-
- `src/main/index.ts` — server listen port
48
-
- `src/renderer/utils/webworker/webworker.js` — `PYODIDE_ASSET_BASE`
49
-
- `src/renderer/index.html` — CSP `connect-src` directive
57
+
> **Previous approach (removed):** We previously ran a plain Node.js HTTP server on port 17173 in the main process and a custom Vite middleware that intercepted `/pyodide/` and `/packages/` requests. Both were replaced by the `pyodide://` protocol — cleaner, no open port, no hardcoded port numbers to keep in sync, and no dev-only code path.
50
58
51
59
---
52
60
···
73
81
},
74
82
```
75
83
76
-
And workers must be created with `type: 'module'`:
84
+
Workers must be created with `type: 'module'`:
77
85
78
86
```ts
79
87
new Worker(new URL('./webworker.js', import.meta.url), { type: 'module' });
···
103
111
| Option | Purpose |
104
112
|--------|---------|
105
113
| `indexURL` | Where Pyodide looks for its **runtime** files (WASM, stdlib). Already resolved from `node_modules` via `import.meta.url`. Do not override. |
106
-
| `packageBaseUrl` | Where `loadPackage()` fetches **package `.whl` files**. Set this to `http://127.0.0.1:17173/pyodide/`. |
114
+
| `packageBaseUrl` | Where `loadPackage()` fetches **package `.whl` files**. Set this to `pyodide://host/pyodide/`. |
115
+
116
+
```js
117
+
const packageBaseUrl = `${PYODIDE_ASSET_BASE}/pyodide/`;
118
+
const pyodide = await loadPyodide({ lockFileURL, packageBaseUrl });
119
+
```
107
120
108
121
---
109
122
···
117
130
118
131
---
119
132
120
-
## micropip.install() from JavaScript
133
+
## micropip and the `pyodide://` URL Limitation
121
134
122
-
`micropip` is a Python object loaded via `pyodide.pyimport()`. Passing a JavaScript array directly works fine in Pyodide 0.29.x — micropip handles the `JsProxy` conversion internally:
135
+
`micropip.install()` only accepts `http://`, `https://`, and `emfs://` URLs — it rejects custom schemes like `pyodide://`. This means we cannot install pure-Python `.whl` files (MNE and its deps) directly from the protocol.
136
+
137
+
**Workaround: fetch via JS → write to emscripten FS → install via `emfs://`**
123
138
124
139
```js
140
+
const manifest = await fetch(`${PYODIDE_ASSET_BASE}/packages/manifest.json`)
141
+
.then(r => r.json());
142
+
143
+
for (const { filename } of Object.values(manifest)) {
144
+
const buffer = await fetch(`${PYODIDE_ASSET_BASE}/packages/${filename}`)
145
+
.then(r => r.arrayBuffer());
146
+
pyodide.FS.writeFile(`/tmp/${filename}`, new Uint8Array(buffer));
147
+
}
148
+
125
149
const micropip = pyodide.pyimport('micropip');
126
-
await micropip.install(whlUrls); // JS array works directly
150
+
await micropip.install(
151
+
Object.values(manifest).map(({ filename }) => `emfs:///tmp/${filename}`)
152
+
);
127
153
```
128
154
129
-
> Note: older guidance suggested wrapping with `pyodide.toPy(whlUrls)` — this is not necessary as of 0.29.x.
155
+
The `fetch()` calls here use the `pyodide://` protocol directly (which Electron's protocol handler supports) and write the bytes into Pyodide's virtual emscripten filesystem. `micropip` then installs from `emfs://` paths, which it does accept.
156
+
157
+
> Note: In Pyodide 0.29.x, passing a JS array directly to `micropip.install()` works — no `pyodide.toPy()` wrapper needed.
130
158
131
159
---
132
160
···
304
332
| Web worker entry point | `src/renderer/utils/webworker/webworker.js` |
305
333
| JS wrappers for Python calls | `src/renderer/utils/webworker/index.ts` |
306
334
| Install script | `internals/scripts/InstallMNE.mjs` |
307
-
| Electron asset server | `src/main/index.ts` → `startPyodideAssetServer()` |
308
-
| Vite middleware | `vite.config.ts` → `serve-pyodide-assets` plugin |
335
+
| Electron protocol handler | `src/main/index.ts` → `protocol.handle('pyodide', ...)` |
309
336
| Plot widget component | `src/renderer/components/PyodidePlotWidget.tsx` |
310
337
| Plot epics (Redux-Observable) | `src/renderer/epics/pyodideEpics.ts` |
311
338
| Pyodide Redux state | `src/renderer/reducers/pyodideReducer.ts` |
+2
-8
src/renderer/utils/webworker/utils.py
+2
-8
src/renderer/utils/webworker/utils.py
···
1
-
from glob import glob
2
-
import os
3
-
from time import time, strftime, gmtime
4
1
from collections import OrderedDict
5
2
6
3
import numpy as np
7
4
from matplotlib import pyplot as plt
8
5
import pandas as pd # maybe we can remove this dependency
9
-
# import seaborn as sns
10
6
11
-
from mne import (Epochs, concatenate_raws, concatenate_epochs, create_info,
12
-
find_events, read_epochs, set_eeg_reference, viz)
7
+
from mne import (concatenate_raws, create_info, viz)
13
8
from mne.io import RawArray
14
9
from io import StringIO
15
10
16
-
11
+
# import seaborn as sns
17
12
# plt.style.use(fivethirtyeight)
18
-
19
13
# sns.set_context('talk')
20
14
# sns.set_style('white')
21
15