Validation
Validation should match the risk of the change. Prose-only edits need lighter checks than changes to executable cells, helper crates, or browser-visible runtime behavior.
Common Checks
Section titled “Common Checks”Run the required lint and formatting gate for every code or configuration change:
pnpm lintpnpm format:checkpnpm lint runs ESLint for JavaScript, MJS, TypeScript, TSX, and Astro with zero warnings, verifies Prettier formatting, and runs the Rust formatting check and strict Clippy lint. Use pnpm format to apply the configured source, config, Markdown/MDX, JSON, and YAML format.
Run pnpm check for normal documentation changes:
pnpm checkThis regenerates the development runtime and runs Astro checks. It catches most MDX, frontmatter, route, import, and TypeScript issues.
Run the documentation contract checker whenever prose, navigation, package exports, command examples, or public schemas change:
pnpm test:docspnpm test:packagetest:docs validates internal files/routes/fragments, English/Japanese route parity, sidebar entries, every package export/path/CLI contract, JSON examples, package imports, and shell commands. test:package confirms the README and required public files are present in the npm archive.
Run the full suite before broad changes are considered complete when practical:
pnpm testRuntime and Browser Checks
Section titled “Runtime and Browser Checks”Use these commands when changing executable cells, cell metadata, rich output examples, Mermaid examples, media examples, or browser-visible behavior:
pnpm wasm:devpnpm test:wasmpnpm test:haskellpnpm buildpnpm test:bundlepnpm test:dev-hmrpnpm test:e2epnpm wasm:dev regenerates the runtime for local development. It requires wasm32-wasi-ghc when Haskell cells are present, as do pnpm build, pnpm check, pnpm wasm:build, and pnpm test:wasm. pnpm dev and pnpm dev:runtime are intentionally more forgiving: they keep Astro serving and show a Haskell cell error in the browser if the Haskell compiler is missing or a Haskell compile fails. pnpm test:wasm verifies generated Rust cell behavior, and pnpm test:haskell executes the generated Haskell/WASI runtime. pnpm test:e2e runs the full browser suite in Chromium, Firefox, and WebKit.
pnpm test:dev-hmr runs a standalone Chromium smoke against a temporary copy of the workspace. It installs dependencies offline from the pnpm store, builds the package, generates the initial runtime, edits an MDX cell, and verifies source and reactive output updates remain stable. Run pnpm install --frozen-lockfile first to populate the store; Rust/Wasm, wasm32-wasi-ghc, and Playwright Chromium are required. PR CI runs it once in the Linux consumer job, after the packed consumer smoke.
Production builds write dist/oxiquill/bundle-report.json and fail when an emitted client or worker JavaScript chunk exceeds 650 KiB uncompressed. Run pnpm test:bundle after pnpm build to verify the budget and confirm that ECharts and Mermaid remain behind dynamic import boundaries.
Unit and Coverage Checks
Section titled “Unit and Coverage Checks”Use unit tests for handwritten TypeScript, Preact, and Node runtime code:
pnpm test:unitpnpm test:unit:coverageCoverage requires 85% statement, branch, function, and line coverage across handwritten CLI, Astro integration, config/path, manifest, worker, generator, and runtime modules. Only generated output and type-only declarations are excluded. Add focused tests for uncovered handwritten code instead of editing generated files.
Packed Consumer Checks
Section titled “Packed Consumer Checks”Verify installation without workspace links through both supported package managers:
pnpm test:consumer:npmpnpm test:consumer:pnpmpnpm test:packed-browserThe consumer commands pack oxiquill, install the tarball in a temporary standalone project, check and build a zero-cell site with only Node on PATH, then add Rust and Python cells and verify their generated runtimes without workspace links. test:packed-browser additionally builds Python and Haskell cells from the installed package, launches Astro preview in background mode, and executes both cells through their real workers in Chromium. It also verifies that clean preserves the download cache and that offline generation can reuse it.
Pull requests require compatibility jobs on Linux, macOS, and Windows, Haskell runtime jobs on Linux and macOS, the packed Chromium job on Linux, and full browser jobs for Chromium, Firefox, and WebKit. Every supported matrix cell is merge-blocking.
The required Rust minimum job runs every generated Rust cell and the npm/pnpm packed consumers with Rust 1.85.0, and feeds the required ci-ok check. A separate, non-required canary runs the same checks with the latest stable Rust on relevant PRs, Mondays at 03:00 UTC, and manual dispatch. The packed helper fixture also exercises a registry dependency absent from the bundled lock, and tests/runtime/rust-helper-lock-smoke.mjs builds a helper workspace whose lock carries dev-only packages and alternative versions of runtime-pinned crates.
Rust Helper Checks
Section titled “Rust Helper Checks”Optional reusable Rust helper crates live under crates/* in an Oxiquill site. Validate them with:
pnpm test:rustpnpm test:rust:coveragepnpm lint:rustpnpm doc:rusttest:rust:coverage uses cargo-llvm-cov and requires 85% line/function/region coverage for helper crates. If no helper crates exist, helper commands skip cleanly.
Troubleshooting Failed Checks
Section titled “Troubleshooting Failed Checks”- Unknown Rust crate: match the cell
cratesvalue to thepackage.nameincrates/*/Cargo.toml. - Unsupported Python package: use a vendored Pyodide package or add package support before documenting it.
- Missing Python runtime: confirm
public/oxiquill/pyodideexists and runpnpm wasm:devorpnpm build. - Missing Haskell compiler: install
wasm32-wasi-ghc, source~/.ghc-wasm/envif usingghc-wasm-meta, or setOXIQUILL_HASKELL_GHCto the compiler path, then rerunpnpm wasm:devorpnpm build. - Missing Haskell runtime: confirm
public/oxiquill/haskell-wasm/status.jsonexists and rerun runtime generation. - Mermaid failure: confirm the code block language is
mermaidand runpnpm buildfor MDX errors. - Missing media: confirm the file is under
public/mediaand the MDX URL starts with/media/. - Coverage failure: add focused tests for handwritten source rather than modifying generated output.
- Documentation link/contract failure: fix the referenced page, route, public contract table, or canonical command instead of adding an exception.
Pull Request CI
Section titled “Pull Request CI”CI expands to 12 jobs: change selection, JavaScript quality, Rust quality, site build, three browsers, Linux runtime tests, three OS consumers, and the final ci-ok check. Markdown-only changes run four jobs: changes, JavaScript quality, Rust quality, and ci-ok. Other changes run the Linux site/runtime/consumer checks; macOS and Windows consumers run for package sources, package tests, root package metadata, the lockfile, and GitHub configuration. MDX is treated as executable site content. Release branches starting with release/ always run the full matrix.
Browser tests depend only on the built site. Chromium checks the bundle report and captures baseline chart screenshots only for chart sources, styles, or ECharts dependency changes. Linux exercises npm, pnpm with browser hydration, and HMR; macOS exercises pnpm and unit tests; Windows exercises npm and unit tests. Rust lint includes cargo fmt --check. The final check accepts intentional skips but rejects failed or cancelled jobs.
Declared Rust Cell Tests
Section titled “Declared Rust Cell Tests”Declare tests in a Rust cell’s leading YAML comments. Each entry selects default or an authored cases id and provides a non-empty expect list. A case may appear only once. Test metadata is never rendered or included in reader manifests.
Rust + Wasm
tested-square
if n == 0 { return Err("n must be positive".to_owned()); }
emit_json!(&(n * n));Run the cell to show its output.
Each expectation uses exactly one form:
| Form | Condition |
|---|---|
{ error: "substring" } | Execution fails with an error containing 1–256 UTF-8 bytes of text. This must be the test’s only expectation. |
{ path: "/outputs/0/value", equals: 4 } | Structural JSON equality, without type coercion. |
{ path: "/outputs/0/value", range: { min: 1, max: 10 } } | A finite number within inclusive bounds; at least one bound is required. |
{ path: "/outputs/0/value", approx: { value: 0, atol: 1.0e-8 } } | abs(actual - value) <= atol + rtol * abs(value). Tolerances default to zero, must be finite and non-negative, and at least one must be positive. |
path is an RFC 6901 JSON Pointer into the normalized { "outputs": [...] } result. The empty string selects the root; ~0 escapes ~ and ~1 escapes /. Add artifact: <id> to select an artifact by its stable id before resolving the pointer, for example { artifact: plot, path: /spec/kind, equals: line }. Rust charts, scenes, and tables take that id from metadata = ..., so emit_table_with_columns!(columns, rows, metadata = json!({ "id": "scores" })) can be checked with { artifact: scores, path: /rows/0/1, equals: 3 } wherever the table appears among the outputs. Missing paths or artifacts fail. Strings and null never become numbers.
The harness uses the browser’s output bounding and artifact validation. It tests declared conditions only: an author-reported status such as iteration-limit, or an artifact validation diagnostic, is not an execution error. Each execution uses a fresh Wasm instance and the cell’s timeoutMs; a timeout always fails, including tests expecting an error.
pnpm exec oxiquill test-wasmpnpm exec oxiquill test-wasm --page guides/validation.mdx --cell tested-square --case largerpnpm exec oxiquill test-wasm --all-cases --format jsonWithout selectors, every Rust cell runs its defaults once and every declared test runs. A declared default test provides that default run’s assertions. --all-cases adds smoke checks for untested named cases. --case only filters runnable cases, so an untested named case requires --all-cases. Python and Haskell do not yet support declared tests. Unfiltered commands retain the existing Haskell smoke step; page, cell, or case filters skip it.
Failures include page, fence line, cell, case, expectation location, condition, and a bounded actual value. See CLI for selector and NDJSON details.
Check Rendered Links
Section titled “Check Rendered Links”Check the built HTML to catch missing routes, renamed headings, and broken generated-cell or stable-reference anchors:
pnpm exec oxiquill build --check-linkspnpm exec oxiquill check-links --format jsonThe standalone command reuses the most recent build and writes no files. It follows local redirects and checks assets for existence without contacting external hosts. Failures identify the resolved target, reason, and example referring pages. Fix the source links or restore the missing target, rebuild, and check again.
For a consumer site’s CI job, run the build and link check together after installing dependencies and the toolchains needed by its cells:
- name: Build and validate local links run: pnpm exec oxiquill build --check-linksLink failures produce a nonzero exit while preserving build output for debugging. See CLI link checks for base-path behavior, failure reasons, and the NDJSON event format.