pyRPC
← Back to Blog

skip-existing and the npm guard: idempotent publishing

·6 min read

The publish workflow can run more than once for the same tag. A re-triggered run, a manual re-run after a transient failure, a retry of one job, all of these will try to upload versions that already exist. Registries reject duplicate uploads. The workflow's answer to that is idempotency: detect what is already published and skip it.

The Python side: skip-existing

- name: Publish to PyPI
  uses: pypa/gh-action-pypi-publish@release/v1
  with:
    packages-dir: dist/
    skip-existing: true
    password: ${{ secrets.PYPI_API_TOKEN }}

The action uploads five built packages. If pyrpc-core-0.12.0 already exists but pyrpc-fastapi-0.12.0 does not, skip-existing: true makes the action treat the duplicate as success and continue with the rest. Without it, the whole job would fail on the first duplicate, even though the only real problem was "this already shipped".

The npm side: check-then-publish

npm has no skip-existing flag, so each npm job implements the guard by hand:

VERSION=$(node -p "require('./package.json').version")
EXISTING=$(npm view @pyrpc/types@$VERSION version 2>/dev/null || echo "")
if [ "$EXISTING" = "$VERSION" ]; then
  echo "@pyrpc/types@$VERSION already published, skipping."
  exit 0
fi
npm publish --access public

Three lines of shell encode the idempotency contract: read the version from the manifest (the source of truth), query the registry, and skip cleanly if the exact version is already there. The 2>/dev/null || echo "" handles the "package not found" case, which would otherwise make npm view exit non-zero and trip the shell's -e flag.

Why idempotency matters here specifically

The chain topology makes partial failures likely: if react publishes but the adapters job fails, the natural recovery is to re-run the failed job, or the whole workflow. Without the guards, that re-run would collide with react's existing 0.12.0 and fail again, turning a transient blip into a permanently stuck release. With the guards, re-running is safe: everything already published is skipped, everything missing is published, and the release completes.

The boundaries of the guard

Idempotency skips exact-version matches. It does not detect a wrong build of the same version, a corrupted wheel uploaded once stays, because the version matches. This is an accepted limitation; package managers are built on the assumption that a published version is immutable. The guards make re-runs safe, not all failure modes elegant.

The pattern

Any publish pipeline that can be re-triggered should be idempotent by default. The two registries demanded two different implementations (a flag on one, a manual guard on the other) but the principle is identical: publishing is a reconciliation, not a mandate. The workflow brings the registry up to the desired state and stops, rather than demanding the registry be empty first.