← Back to Blog

Docs and examples as executable truth

·6 min read

Two categories of code in this repo are never imported by anything that runs in CI: the twelve example applications and every fenced code block in the documentation. Both exist to be copied by users, which makes their correctness load-bearing and their rot invisible until someone pastes a broken snippet and blames themselves. Two new scripts bring both into the gate set.

Examples: import is the test

The insight behind scripts/verify_examples.py is that an example server’s whole job happens at module scope. Importing main.py executes decorators, mounts routes, and resolves imports; if any of that broke, the import fails. So each framework example gets exactly one check:

# fastapi/flask examples
python -c "import main"          # cwd = examples/<name>/server

# django examples: check imports the URLconf,
# which executes the views chain that registers @rpc
python manage.py check

Fifteen checks run per pass: eight servers plus syntax validation of the standalone scripts (which perform network work when actually run, so parse-only is deliberately the right depth). First execution caught an environment drift immediately: pyrpc-flask was missing from the dev dependency group, so plain uv sync produced an env where flask examples could not import even though CI passed. Fixed at the source.

Docs fences: parse everything, execute what earns it

Documentation snippets are mostly fragments: they reference variables defined in prose or rely on surrounding context. Blindly executing all of them produces false failures nobody trusts. The design instead layers enforcement by intent:

  • Every ```python fence must parse. ast.parse catches renamed functions, changed signatures, drift after refactors. Sixty-three fences checked today; zero exemptions needed.
  • Fences marked ```python test get executed in a fresh namespace and must be self-contained. Two complete server definitions from quickstart and installation carry the tag so far.
  • Pseudo-code opts out explicitly via ```python nocheck. Opt-outs are greppable and reviewable, unlike silent breakage.

The convention matters more than the count: when someone edits a documented API, CI tells them which doc page just went stale, in the same red X as failing tests.

Honest edges

  • Example frontends are not installed in CI yet; twelve npm installs would dominate the pipeline. The Python side covers the pyRPC contract surface.
  • TypeScript fences are counted (forty today) but not compiled. Same layering rationale: value first, noise never.

Docs that compile and examples that import stop being aspirational writing and become what they should have been all along: part of the test suite that happens to teach.