pyRPC
← Back to Blog

Windows compatibility in a Python OSS project: what we learned

·7 min read

The first time someone ran pyrpc pull on Windows, it crashed with a UnicodeEncodeError. The CLI output contained a checkmark character that Windows terminals could not render. We fixed that specific bug, but it opened a broader question: how do you maintain cross-platform compatibility in a Python OSS project when most of your team develops on macOS or Linux?

This post documents the Windows-specific issues we hit, the fixes we applied, and the policies we adopted to prevent them from recurring.

1. The Unicode problem

Windows terminals default to the system code page, which on US-English systems is cp1252 (Windows-1252). This encoding supports basic Latin characters but not the broader Unicode range. macOS and Linux terminals default to UTF-8, which covers everything.

The practical consequence: any non-ASCII character in your CLI output, template files, or log messages will crash on Windows. The characters that bit us:

  • Unicode checkmark (\u2713) — used in CLI output to indicate success. Replaced with [OK].
  • Em dash (\u2014) — used in generated TypeScript comments as section separators. Replaced with ---.
  • Right arrow (\u2192) — used in help text. Replaced with ->.
  • Bullet characters — used in Rich table rendering. Rich handles these internally, but only if the terminal supports them.

Rich (the library we use for CLI rendering) detects terminal encoding and falls back to ASCII replacements automatically — but only for its own output. Our custom strings containing Unicode characters bypass Rich's detection and go straight to stdout, where they crash on cp1252.

2. LF vs CRLF in git

Git on Windows defaults to core.autocrlf = true, which converts LF to CRLF on checkout and CRLF to LF on commit. This means every file touched on Windows produces git warnings:

warning: in the working copy of 'file.py', LF will be replaced by CRLF
the next time Git touches it

These warnings are harmless but noisy. They also cause problems if a contributor on macOS edits a file that was last touched on Windows — the diff shows every line ending changed.

The standard fix is a .gitattributes file that forces LF for Python files and other text types:

*.py      text eol=lf
*.toml    text eol=lf
*.json    text eol=lf
*.md      text eol=lf
*.ts      text eol=lf
*.tsx     text eol=lf
*.js      text eol=lf
*.css     text eol=lf
*.html    text eol=lf

We added this to the repository. It ensures that regardless of the contributor's operating system, files are stored as LF in git and checked out as LF on disk. No warnings, no spurious diffs, no CRLF contamination.

3. Path separators

Python's os.path module handles path separators transparently, but there are edge cases:

  • Module paths in CLI arguments. A user on Windows might write app\main.py. We normalize with os.path.normpath before passing to importlib, which expects dot-separated module paths.
  • Temp directory paths. tempfile.TemporaryDirectory returns paths with backslashes on Windows. These need pathlib.PurePosixPath conversion when passed to Jinja2 templates that generate TypeScript import paths.
  • Forward slashes in generated code. TypeScript import paths should always use forward slashes, even on Windows. We explicitly convert backslashes to forward slashes in generated import statements.

4. The file watcher

We use watchfiles for the pyrpc dev file watcher. On Windows, it uses ReadDirectoryChangesW under the hood (via Rust'snotify crate). This is the native Windows API for directory change notification — no polling, no CPU overhead.

The issue we hit: watchfiles on Windows has a known limitation with network drives and certain anti-virus software that intercepts file operations. If a user's project is on a network drive, the watcher might not detect changes reliably. We documented this in the CLI help text and added a --poll-interval fallback flag for watchdog-based polling.

5. The no-special-chars policy

After the Unicode crash, we codified a policy in CONTRIBUTING.md:

No emojis, no em-dashes, no Unicode special characters anywhere in the codebase. This includes CLI output, template files, log messages, and comments. These bugs are silent on macOS and Linux and only surface on Windows at the worst possible moment.

This applies to the Python codebase (packages/ and tests/). The documentation site (docs/) has no such restriction because it is rendered by Next.js and served as HTML, where Unicode is expected.

6. Testing on Windows

The hard truth is that you cannot maintain Windows compatibility without running tests on Windows. We run the full test suite on Windows via GitHub Actions:

# .github/workflows/test.yml
jobs:
  test:
    strategy:
      matrix:
        os: [ubuntu-latest, macos-latest, windows-latest]
    runs-on: ${{ matrix.os }}
    steps:
      - uses: actions/checkout@v4
      - uses: astral-sh/setup-uv@v3
      - run: uv sync
      - run: uv run pytest

Cross-platform CI is not optional. Every PR runs against all three operating systems. A macOS-only project that accepts Windows users contributions without Windows CI is going to break Windows regularly.

Lessons learned

  1. Assume cp1252, not UTF-8. Windows terminals do not render Unicode by default. Test your CLI output on a clean Windows VM before adding decorative characters.
  2. Add .gitattributes early. The LF/CRLF warning is harmless but noisy. Force LF for text files from the start. Adding it later means a one-time cleanup commit that touches every file.
  3. Use pathlib, not os.path. pathlib.Path handles separator normalization internally. os.path.join produces backslashes on Windows that may not be what you want.
  4. Document known limitations. Network drives, anti-virus, and Windows-specific performance issues should be documented so users know what to expect.
  5. CI on all three OSes. Window-specific bugs cannot be caught by macOS or Linux testing. The CI matrix must include Windows.

All 45 tests pass on Windows, macOS, and Linux. The repo is atgithub.com/pyrpc/pyrpc.