Python + Electron bundling — design note

Picks up where Electron Build Fix left off. That doc resolves Bugs 1–3 (renderer asar packaging) and explicitly defers Bug 4: the packaged .exe launches its window but the worker subprocess never starts because neither the Python runtime nor the backend/ tree is copied into process.resourcesPath.

This note records the bundling plan so the eventual Bug-4 PR has a clear target shape, and so we don't keep rediscovering the same trade-offs.

What "fully bundled" means here

End-state goal: a single installer per OS that runs offline with zero dependencies on the user's machine. No system Python, no pip install, no Visual C++ runtime download — the user double-clicks the installer and the worker boots.

Concretely, after install:

<install-root>/
└── resources/
    ├── app.asar              # renderer + main process (Vite + Electron)
    ├── python/               # full CPython runtime, OS-specific
    │   ├── bin/python3       # (mac/linux)
    │   └── python.exe        # (win)
    └── backend/              # EdgeWeave Python — pre-installed deps
        ├── main.py
        ├── core/
        ├── api/
        ├── ai/               # LangGraph + provider adapters (M2+)
        ├── projects/
        ├── sdk/
        └── _vendor/          # site-packages, pre-installed at build

main_electron.js already resolves both paths from process.resourcesPath, so once the installer puts them there, the existing spawn code works unchanged.

Bundling option matrix

Option	Pros	Cons	Verdict
python-build-standalone	Cross-platform parity (mac/linux/win), modern CPython, statically-linked OpenSSL, easy CI download, no system deps	~40 MB compressed per OS, license attribution	Recommended baseline.
PyInstaller --onedir	Compresses the whole backend into one folder; familiar to devs	Hides import errors at runtime; weak with optional/lazy imports (numpy, torch, langgraph extras); rebuild on every backend change	Fallback only.
Windows embeddable distribution (python-3.11-embed-amd64.zip)	Smallest payload (~10 MB), official from python.org	Windows-only; needs manual pip shim; no tkinter / SSL surprises	Use only if we ever ship Windows-first beta.
System Python + pip install	Smallest installer	Requires network on first launch; flaky on locked-down machines; matches none of EdgeWeave's UX goals	Do not ship.

We commit to python-build-standalone as the default. It's what Astral's uv uses, what pyapp and rye use under the hood, and the artifacts are re-built and re-signed on a predictable cadence.

Tarballs we'd pull in CI:

windows: cpython-3.11.*-x86_64-pc-windows-msvc-install_only.tar.gz
macos  : cpython-3.11.*-{aarch64,x86_64}-apple-darwin-install_only.tar.gz
linux  : cpython-3.11.*-x86_64-unknown-linux-gnu-install_only.tar.gz

Dependency strategy

backend/requirements.txt is satisfied at build time, not first launch. The release workflow:

1. Download the OS-matched python-build-standalone tarball, extract to frontend/python-portable/.
2. Run python-portable/bin/python -m pip install -r ../backend/requirements.txt --target ../backend/_vendor.
3. Add backend/_vendor to sys.path early in backend/main.py (or set PYTHONPATH in the spawn args).
4. electron-builder copies the prepared python-portable/ and backend/ trees as extraResources.

Why --target instead of installing into the portable Python's site-packages: keeps the bundled CPython untouched (so we can swap it without re-installing deps), and means the same _vendor works for pip wheel-built dev environments too.

// frontend/package.json — proposed extraResources
"build": {
  "extraResources": [
    {
      "from": "../backend",
      "to": "backend",
      "filter": [
        "**/*",
        "!**/__pycache__/**",
        "!**/*.pyc",
        "!tests/**",
        "!**/.pytest_cache/**"
      ]
    },
    {
      "from": "../python-portable",
      "to": "python"
    }
  ]
}

Cross-platform parity

main_electron.js already branches on process.platform:

const pythonPath = isWin
  ? path.join(process.resourcesPath, 'python', 'python.exe')
  : path.join(process.resourcesPath, 'python', 'bin', 'python3');

That matches the layout python-build-standalone ships, so nothing on the JS side has to change. The installer config does need a matrix in CI — separate release-{windows,macos,linux} jobs that each download the right tarball — but that's a release.yml change, not a code change.

Code signing / notarization

• Windows. electron-builder already wires up Authenticode signing via CSC_LINK / CSC_KEY_PASSWORD. Bundled Python binaries inside extraResources inherit the installer's signature; but the individual python.exe / pythonw.exe files retain whatever signature python-build-standalone shipped. Confirm SmartScreen doesn't bark on first launch.
• macOS. Notarization scans every Mach-O binary inside the app bundle. Every .dylib / .so shipped by python-build-standalone must be Apple-signed and hardened-runtime compatible. PBS does this upstream, but the build step needs --deep --options runtime on codesign, and the entitlements file needs com.apple.security.cs.allow-unsigned-executable-memory for numpy/torch JIT-style imports.
• Linux. No signing; ship as .AppImage or .deb. Worth testing on glibc 2.31 (Ubuntu 20.04) since python-build-standalone's linux-gnu builds target that floor.

Size / perf budget

Rough per-installer footprint with the M1–M5 backend:

Component	Size
python-build-standalone CPython 3.11	~40 MB
backend/ source	<1 MB
Pre-installed deps (FastAPI, uvicorn, numpy, pandas, langgraph, openai)	~150–200 MB
Vite renderer bundle	~5 MB
Electron itself	~80 MB
Installer total (compressed)	~150 MB

Open optimisation levers (only if needed):

• Strip numpy / pandas test directories from _vendor (*/tests/**).
• Use pip install --no-compile and skip __pycache__/.
• Lazy-import torch / sklearn behind feature flags so they only land in the "ML-heavy" build variant.

Build pipeline changes

/.github/workflows/release.yml needs three additions:

1. Cache the PBS tarball — cache key on runner.os plus the pinned PBS release tag; ~40 MB download isn't worth re-fetching per run.
2. Vendor deps step — between npm run build and npm run dist, run a python -m pip install --target backend/_vendor step that uses the freshly extracted PBS Python.
3. Per-OS jobs — fan out to windows-latest, macos-14, and ubuntu-22.04; each job picks the right tarball URL.

Pin the PBS release tag in a top-level env var (e.g. PBS_RELEASE: 20240107) so the bundled CPython is reproducible build-to-build.

Runtime considerations

• PYTHONHOME / PYTHONPATH. When main_electron.js spawns the worker, do not pass PYTHONHOME. PBS install_only is built to be relocatable via the embedded sysconfigdata module, and an externally-set PYTHONHOME overrides that and frequently breaks imports of the bundled stdlib. Instead, build a clean child env: pass through only PATH, SystemRoot, locale vars, and temp directories; explicitly delete env.PYTHONHOME to drop any value the user's shell happened to have set; and set PYTHONPATH=<backend>/_vendor so vendored deps resolve before any system site-packages. Also set PYTHONDONTWRITEBYTECODE=1 to avoid scribbling .pyc files into the read-only resource tree. See startBackend() in frontend/src/js/main_electron.js for the exact passthrough list.
• Working directory. Spawn with cwd: backendDir so relative paths in backend/core/config.py keep working as they do in dev.
• Anti-virus stalls (Windows). First launch can spend 5–10 s with Defender scanning the unpacked _vendor tree. Add a splash / progress message in the renderer if the worker handshake takes longer than 2 s.
• Updates. electron-builder's NSIS auto-updater replaces app.asar on update but leaves extraResources alone unless we bump the version dir. Either ship Python + backend inside the asar (slower startup, simpler updates) or accept that minor backend patches need a full installer rebuild. Recommended: keep them outside the asar, full installer per release — matches our current release cadence.

Future: cloud / hosted EdgeWeave

When EdgeWeave moves to a hosted multi-tenant deployment, the bundling story evaporates — the backend lives on a server, the browser only ships the renderer. But two carry-over decisions matter even on Electron:

• Backend is launched as a subprocess, not via IPC into the main process. Keeps the desktop and cloud transports identical (HTTP + WebSocket); the cloud version just talks to a remote uvicorn instead of 127.0.0.1:<random>.
• Secrets keystore is process-local. The read_env design in the chat assistant doc already treats values as worker-process-only and never lets them transit the renderer. Same code path works server-side once the worker moves off the user's machine.

Checklist for the Bug-4 PR

• Add PBS download + extract step to release.yml (Windows; mac/linux to follow).
• Add pip install --target backend/_vendor step.
• Add extraResources block to frontend/package.json.
• Set PYTHONPATH (and explicitly clear PYTHONHOME) in the spawn env in main_electron.js.
• Insert _vendor near the top of sys.path in backend/fastapi_main.py (no-op when the directory doesn't exist, so dev-from-source is unaffected).
• Add frontend/python-portable/ and backend/_vendor/ to .gitignore.
• Confirm dev-mode behaviour: startBackend() early-returns in dev (backend started externally via npm run dev:full), so no packaged-runtime fallback is needed in main_electron.js.
• Verify macOS notarization passes with the bundled CPython (hardened runtime + cs.allow-unsigned-executable-memory entitlement for numpy/torch JIT-style imports).
• Add a Linux job and a macOS job to release.yml once Windows is green end-to-end on a clean VM.
• Smoke-test on a Windows VM with no system Python installed and Defender at default settings — measure first-launch latency.