lichen.page.october
lichen.page.review from the reviewed point 0b92cb61 to stack eb5a1653, as one PR per subsystem (review set)
Star 0
Fork 0
Watch

full 10: Scripts, shortcodes and the server render, in child processes #73

Open notplants-bot.bsky.social
targeting full/09-publish-front from full/10-scripts
Open opened by and assigned to notplants-bot.bsky.social
targeting full/09-publish-front from full/10-scripts

Site scripts, all new since 09-23: script shortcodes for lichen-ssg in Lua, JavaScript and Python, the script runners they share with build hooks, the script generator and the server-side render, an output cache for shortcodes, and the built-in editor's no-JavaScript page drawn by React on the server. Since FULL-SITE-PROCESS no script runs in the site process: Lua, JavaScript, the server render and mdBook run in an engine child of their own (scripts/engine.rs), and Python in its own child, one per build. Every script is confined to its own site, and what a script starts stays in its child's process group.

AST-verified changes

  • The script runners' shared mechanics moved out of shortcodes/ (OPERATOR-REVIEW-REVISIONS R6, operator note N6; 1da80fa3, then d755f8ce): shortcodes/{runner,child,lua,js,python}.rs to scripts/, the shared items of shortcodes.rs to scripts.rs, SsrRenderer to ssr/renderer.rs, and shortcodes/ssr.rs to ssr/runtime.rs. Checked: run notify-3a70f33e, "AST cc1c8ced..1da80fa3 EQUAL with move-map-1b, cc1c8ced..d755f8ce EQUAL with move-map-2, controls RED"; adversary rerun notify-b1faa4bc, "Moves AST EQUAL (14 moved, 3018=3018) x2, my control RED". Allowed: about 16 pub(crate) visibility entries, super:: → crate::scripts:: in ssr.rs, path rewrites for SsrRenderer and in_process_call.
  • The cache moved into a module of its own (R4, operator note N2; 9334aad9): 5 cache items from shortcodes.rs to shortcodes/cache.rs. Checked: run notify-c68c90aa, "move AST EQUAL + controls"; adversary (A11498): "AST EQUAL 9/2998, control RED" (run id not recorded). Allowed: pub(super) on cached and remember.

None of this PR's 3,517 changed lines can be skipped for these moves: every file here was created after 09-23 (+3,517/−0), so in this diff all of it reads as new code. The moves only tell you that the R6 and R4 reorganisations changed no item.

What is here

  • Shortcodes (shortcodes.rs, shortcodes/syntax.rs; SSG-SCRIPT-SHORTCODES S1–S4): a site holds shortcodes/<name>.lua, .js or .py; a page calls one with Hugo's syntax {{< name key="value" >}} (paired and nested forms too, and lichen-markdown's two older forms). Each call's script gets the arguments and the inner content rendered to HTML, and returns the HTML put in the call's place. A failing call never fails the page: a published page leaves its place empty, a preview shows an error box, and the build's report names it.
  • The runners (scripts.rs, scripts/*; SCRIPT-RUNTIMES-OUT-OF-PROCESS O1, PER-SITE-PROCESS P4-5 and P4-3b): each language has a worker thread confined to the site (Landlock), and since FULL-SITE-PROCESS unit 1c every interpreter is a child process: Lua 5.4 (mlua) and JavaScript (QuickJS, rquickjs) in the engine child (scripts/engine.rs, this executable started in its engine mode), one per build, Python in its own child, one per build. Each child is started at its build's first call and driven by the one child protocol (scripts/child.rs): a time limit and kill, its own process group, address-space and CPU caps, an empty environment, descriptors 0–2 pointing nowhere. What a call starts (Lua's io.popen and os.execute) is in the child's group and ends with it. A Python or Lua script may ask for a publish or a push on its child's route socket (FULL-SITE-PROCESS 6g, #40); JavaScript has no ask.
  • The cache (shortcodes/cache.rs, N2): keyed by the script, its request and a digest of every file under shortcodes/, for the life of the process. A script can opt out (cache = false); a server can turn it off (use_shortcodes_cache = false).
  • The no-JavaScript page (ssr.rs, ssr/runtime.rs, ssr/renderer.rs; REPLACE-TERA-WITH-SSR-REACT): ?format=html is drawn by the built-in editor's own React components in one QuickJS runtime in a render child of its own (the engine child), confined to its scratch, with no site and no network, from the same views the API answers with. Its form saves as the API's replace does. It replaced the Tera-drawn page.

Files (crates/lichen-site/src/)

  • scripts.rs — the shared limits and errors (the in-process call, work_in and call_join_words are gone). Read this first.
  • scripts/engine.rs — new: the engines' child (Lua, JavaScript, the render, mdBook), started and driven as Python's is.
  • scripts/runner.rs, child.rs — the worker per language and the child protocol.
  • scripts/lua.rs, js.rs, python.rs — the three languages.
  • shortcodes.rs, shortcodes/syntax.rs — the calls in a page and their rendering.
  • shortcodes/cache.rs — the output cache.
  • ssr.rs, ssr/runtime.rs, ssr/renderer.rs — the no-JavaScript page.

Lives elsewhere

  • The build that runs shortcodes, the script generator (ssg/script.rs) and the rebuild lane: #27.
  • The confinement of worker threads, call-join and the call's process group: #17. What each spawn may do (permissions.rs): #18. Every sandbox a site process sets, the engine child's included: #39. The route socket a script's child asks on: #40.
  • The server setting use_shortcodes_cache: server_settings.rs in #31 and settings/server.rs in #37.
  • The built-in editor's React bundle the SSR runs (frontends/builtin/ssr/app-ssr.js, built) is in the base commit.
  • Tests (shortcodes/tests/, src/tests/ssr.rs, tests/call_cleanup*.rs, tests/call_runner_probes.rs) are in the base commit.

Category: Scripts, shortcodes and the server render, in child processes Size: 13 files, +3,517/−0 Rounds: round 0 at stack ae42e3c0; at 29a7c15b and at 6b650b2a none of this PR's files changed; round 1 at b66397c0 (FULL-SITE-PROCESS units 1c and 6g): Lua, JavaScript, the server render and mdBook leave the site process for an engine child (scripts/engine.rs, new), so the in-process call and its watchdog go; Python and Lua scripts get the child-routes socket to ask for a publish or a push. Read this round in its interdiff: https://tangled.org/notplants-bot.bsky.social/lichen.page.october/pulls/26/round/1/interdiff Plans and ledger: FULL-SITE-PROCESS-2026-10-25.md (part 2, units 1c and 6g); SSG-SCRIPT-SHORTCODES-2026-10-02.md with SSG-SCRIPT-SHORTCODES-0A-DESIGN-2026-10-08.md and -UNITS-DESIGN-2026-10-08.md (S1–S4); SCRIPT-RUNTIMES-OUT-OF-PROCESS-2026-10-08.md and SCRIPT-RUNTIMES-O1-DESIGN-2026-10-08.md; REPLACE-TERA-WITH-SSR-REACT-2026-10-07.md with REPLACE-TERA-R0-DESIGN-2026-10-08.md; PER-SITE-PROCESS-2026-10-09.md (P4-3b, P4-5); OPERATOR-REVIEW-REVISIONS-2026-10-13.md (N2/R4, N6/R6); SITE-FRONT-BOUNDARY-2026-10-17.md (decision 6). Ledger: lichen-builder B2765, B2776–B2777; lichen-adversary A11498, A11505. Inline #[cfg(test)] modules: none inline; 3 files point to a test file with #[path]. Bucket moves: round 1's Builds PR is split: scripts, shortcodes and the SSR here, the SSGs and the rebuild lane in #27. Round 1: #8 (Builds: scripts, shortcodes, SSGs and the server render) covered these files from 0b92cb61, except shortcodes/syntax.rs and scripts/engine.rs, which are new to the set.

Commits 0