Designing the pyRPC developer console

When you run pyrpc dev app.main, what should happen? A server starts, files are watched, types are regenerated — but should there be a prompt? Should you be able to type procedures and see every RPC method registered in your running server? Should inspect add show you the parameter types and docstring without curling an endpoint?

These questions turned into a deeper architectural debate than we expected. This post walks through the three designs we considered, the one we built, and the trade-offs that shaped the final implementation.

The problem

Before this change, pyrpc dev started a uvicorn subprocess, watched files, and regenerated TypeScript types — all silently. There was no way to see what procedures were registered, inspect their signatures, or even confirm the server had picked up the latest changes. The only feedback loop was: edit a file, check the watcher output, then test via curl or the frontend.

The pyrpc shell command existed as a separate network-based REPL — it connected to a running server over HTTP, fetched the schema, and let you call procedures interactively. But it was a separate process requiring a separate terminal. Developers wanted everything in one session.

We had three design options, each with very different architectures.

Option 1: Separate CLI client

The first model is what tools like redis-cli, psql, andmongosh use. Two terminals:

# Terminal 1
pyrpc serve app.main

# Terminal 2
pyrpc shell --url http://localhost:8000

The shell is a separate process that connects over HTTP, downloads the schema, and lets you call procedures remotely. It works against any server — localhost, staging, or production — using the same protocol that frontend clients use.

Strengths: works everywhere the protocol works, no special server logic, same tool for all environments. This is how most mature infrastructure tools handle interactive access.

Weaknesses: requires a server to already be running, first call is slow (schema fetch over HTTP), can’t do compile-time validation, and is unusual for RPC frameworks (tRPC, gRPC, ConnectRPC don’t ship shells).

We already had this model working (pyrpc shell was built in the CLI overhaul). But it wasn’t what developers were asking for. They wanted something that felt like rails console or django-admin shell — a developer console attached to the running dev server, not a separate network client.

Option 2: Embedded REPL inside the server process

The second model is what Python itself uses (python),node, and irb — the REPL lives inside the same process as the application. Direct memory access, zero network calls, instant responses.

Strengths: registry is already in memory, no serialization, no HTTP overhead, instant startup.

Weaknesses: uvicorn blocks the main thread — you can’t run input() after uvicorn.run(). Server logs and REPL prompts fight over stdout — imagine kubectl logs showing[pyrpc] > prompts interleaved with request logs. Cannot connect remotely. Harder to run in Docker (no TTY in background). Doesn’t fit cloud deployments.

We rejected this model for one big reason: we don’t want production servers to have any concept of a console. The dev server should be a standard uvicorn instance that has no idea a developer is typing commands at it.

Option 3 (chosen): Dev server + interactive console

The model we built is a hybrid. The server runs as a subprocess (standard uvicorn, no modifications). The parent process imports the module directly to access the in-memory registry. A file watcher runs in a daemon thread. The main thread runs an interactive console via a simple input() loop.

$ pyrpc dev app.main

✓ Dev server for app.main
✓ Endpoint: http://127.0.0.1:8000/rpc
✓ Types: node_modules/@pyrpc/types/src/index.ts
✓ Watching 3 directories for Python changes

pyrpc> procedures
pyrpc> inspect add
pyrpc> generate
pyrpc> exit

Strengths: console doesn’t compete with server logs (separate stdout), registry access is instant (in-memory, not HTTP), works for local development where developers spend most of their time, doesn’t couple production servers to console logic, and clean shutdown viastop_event.

Weaknesses: registry can diverge from the server (more on this below), no tab completion yet, can’t connect remotely (that’s what pyrpc shell is for).

Architecture

The implementation lives in the dev command of pyrpc-codegen. Three components run concurrently:

1. Server subprocess

A standard uvicorn instance started via subprocess.Popen with--reload. The parent process generates a temporary Python file that creates PyRPCAsgiApp(default_router) and passes its path to uvicorn. The parent keeps the process handle for terminate() on shutdown and stores the startup args for the restart command.

Why subprocess and not in-process threading? uvicorn.run()blocks the main thread. Threading uvicorn is possible but the GIL, signal handling, and graceful shutdown become complex. A subprocess gives clean process isolation: kill it, restart it, ignore its stdout.

2. Watcher thread

A daemon thread running watchfiles.watch() — a Rust-backed file watcher using inotify, FSEvents, or ReadDirectoryChangesW under the hood. Passes a threading.Event as stop_event so the main thread can signal it to stop on exit. Uses yield_on_timeout=True so the loop periodically wakes up to check the stop flag.

On file changes, the watcher calls regenerate() which re-imports the module, refreshes the parent’s default_router, and writes updated TypeScript types.

Why daemon thread? Daemon threads are automatically killed when the main thread exits. No need to .join() or cleanly shut down. The thread only does file I/O and importlib.reload() — no resources that need cleanup.

3. Console loop

A _DevConsole class that runs a simple while True: input()loop in the main thread. Commands are dispatched via a dict — O(1) lookup, easy to extend. Output is formatted with Rich tables.

pyrpc> procedures
┌─────────────────────────────────────────────────────┐
│                 Procedures (3 total)                  │
├─────────┬──────────────────┬─────────┬────────────────┤
│ Name    │ Params           │ Returns │ Doc            │
├─────────┼──────────────────┼─────────┼────────────────┤
│ add     │ a: int, b: int   │ int     │ Add two nums   │
│ greet   │ name: str        │ str     │ Say hello      │
└─────────┴──────────────────┴─────────┴────────────────┘

pyrpc> inspect add
add
  Doc: Add two numbers.
  Returns: int
  Parameters (2):
    a: int
    b: int

The console reads the parent process’s default_router directly — no network calls, no serialization, instant access. This works because the parent process imports the user’s module at startup and re-imports it on every file change.

Why not cmd.Cmd or prompt_toolkit?cmd.Cmd.cmdloop() catches KeyboardInterrupt internally and raises SystemExit, which is awkward with threading.prompt_toolkit would give tab completion and history but adds a dependency. A raw input() loop is zero-dependency and works reliably across platforms.

Threading model

The three components run with different thread lifetimes:

Main Thread                 Watcher (daemon)          Subprocess
─────────────               ────────────────          ──────────
dev() starts
  ├─ import module
  ├─ regenerate()
  ├─ Popen(uvicorn) ───────────────────────────────►  uvicorn runs
  ├─ Thread(watcher) ──────►  watchfiles loop
  │                              ├─ on change
  │                              │  └─ regenerate()
  │                              └─ check stop_event
  ├─ Console.run()
  │  ├─ input() blocks
  │  ├─ user types "exit"
  │  └─ _running = False
  │
  └─ finally:
     ├─ stop_event.set() ────►  thread exits
     ├─ proc.terminate()  ─────────────────────────►  uvicorn killed
     └─ proc.wait()

Key design points:

• Non-blocking lock: regenerate() uses threading.Lock.acquire(blocking=False) so concurrent calls from the watcher and generate command don’t race on importlib.reload() (which is not thread-safe).
• Event-driven stop: threading.Event is passed to watchfiles.watch(), which checks it internally in the Rust polling loop. When set() is called from the main thread, watch() stops yielding on its next internal check.
• Daemon thread: If the main thread crashes, the watcher thread dies automatically. No orphaned threads, no resource leaks.

The reload problem (and how we fixed it)

The first version of regenerate() used a double-reload pattern:

# Old approach - smell
importlib.reload(mod)
default_router._procedures.clear()
importlib.reload(mod)

Why the double reload? Router.register() uses dict assignment, so same-named procedures are overwritten. But procedures removed from the source file persist after reload because nothing removes them. The clear() between two reloads drops the stale entries, then the second reload re-registers only the current ones.

This works but it’s fragile. If the module has a syntax error, you’ve cleared the registry and can’t populate it. The user sees zero procedures.

The fix was Router.reload_module() — a new method on the coreRouter class in pyrpc-core:

def reload_module(self, module_path: str) -> bool:
    old = dict(self._procedures)     # snapshot
    self._procedures.clear()          # clear
    try:
        importlib.reload(mod)         # single reload → @rpc fires on clean slate
    except BaseException:
        self._procedures.update(old)  # rollback on failure
        raise
    if not self._procedures:
        self._procedures.update(old)  # restore if nothing registered
        return False
    return True

One reload, atomic swap, automatic rollback on failure. If the module has a syntax error, the old registry is preserved. If the module exports no procedures, the old registry is preserved. The user never sees an empty procedure list unless they explicitly removed everything.

The registry divergence problem

The biggest open architectural concern is registry divergence. The parent process and the server subprocess each rebuild their registries through completely different mechanisms:

• Parent: importlib.reload() in the regenerate() call
• Server: fresh Python process started by uvicorn --reload

If the parent’s reload succeeds but the server’s import fails (or vice versa), the two registries diverge. The console’s procedurescommand could show procedures that the HTTP server doesn’t actually serve.

We don’t eliminate this risk in the current implementation, but we mitigate it in two ways:

1. The parent and server import the exact same module file. If the file compiles, both reloads should succeed. If it doesn’t compile, both reloads should fail.
2. The reload_module() rollback means a failed parent reload preserves the old registry, which is still consistent with whatever the server is serving.

A future improvement could verify the registries match after regeneration by calling the server’s GET /rpc endpoint and comparing the result against the parent’s default_router. A mismatch would print a warning. We may add this once the call command (in-console RPC execution) lands, since that command already needs the HTTP client.

Commands reference

Command         Description
─────────────────────────────────────────────────────
help            Show available commands and usage
procedures      List all registered RPC procedures
procs           Alias for procedures
inspect <name>  Show details for a specific procedure
generate        Manually trigger type regeneration
types           Show path to generated TypeScript types
restart         Kill and restart the dev server
exit / quit     Stop the dev server and console

Known limitations

• No tab completion. The console uses raw input() — no autocomplete for procedure names or argument hints. prompt_toolkit would solve this but adds a dependency we haven’t committed to yet.
• Redundant watch loops. Both the watcher thread and uvicorn’s --reload watch for file changes. A future optimization could unify them: the watcher signals uvicorn to restart instead of relying on a second watch loop.
• restart is no-op in --types-only mode. If the server wasn’t started, there’s nothing to restart.
• No cross-platform stdin handling. input() with Rich ANSI codes works on Windows Terminal and VS Code terminal but may have issues with PowerShell ISE or old cmd.exe.

What’s next

The immediate roadmap:

1. In-console RPC calls. Type call add(1, 2) and get the result displayed in the console. Uses HTTP to the running server.
2. Tab completion. prompt_toolkit integration for procedure names, argument hints, and command history.
3. Registry verification. Compare parent and server registries after reload and warn on mismatch.
4. Unified watch loop. One watcher that triggers both type regeneration and server restart, eliminating the redundant --reload dependency.

The full implementation is on the feat/dev-console branch. All 45 Python tests pass.