Code assistants write calls their users' Python version no longer has; a release-level record of 1,872 libraries lifts four models to 68.5%–69.9% repair success
Synopsis
Code assistants often emit Python calls that belong to an older release and no longer exist in the version a user has installed. PyPI API Drift statically extracts, without importing, the public APIs of 55,565 releases across the 1,872 most-downloaded libraries, recording signatures and release-to-release changes while separating true removals from names that merely left the documented API. In a preregistered code-repair experiment, four models from one family raised API-compatible, intent-preserving migrations to 68.5%–69.9% with this API history, against 7.0%–56.6% unaided.
Interpretation
Python library public APIs keep changing across releases, and a code assistant cannot tell whether a call still exists in the version a user installed. Earlier version-aware benchmarks were small, hand-curated or frozen at a point in time, leaving no refreshable record of which public API existed in which release of which library. The record covers 1,872 libraries and 55,565 releases, with 15,258,652 release-to-release changes and 210,535 deduplicated breaking changes.
The record is built by static parsing that imports and executes nothing, reading only the needed bytes of each wheel. Extraction parses modules with Python's ast, records full function signatures, class members, imports and __all__, and diffs consecutive releases path by path. Replaying the fetch logic over 8,149 releases of 300 randomly chosen projects downloaded 8.5% of the wheels' bytes, at a median cost of 81 KB and two requests per release.
When a name disappears from the public API, it is recorded as removed only if an accessibility test on the new release proves it unreachable by attribute access. Names still reachable, or whose reachability cannot be decided statically, are recorded once as non-breaking left_public_api and carried forward, so leaving __all__ is not mistaken for deletion. In pipeline-isolated AI-agent runtime audits, 91.9% of high-confidence breaking changes were confirmed (Wilson 95% interval 86.6–95.2%), over 134 libraries never used before.
In a preregistered code-repair experiment, giving a model the dataset's API history raises how often it turns old-release code into code that runs against the new release. The same model succeeded on 56.6% of tasks closed book, 60.1% with only the new release's documentation, and 68.5% with API history. Repeated on four models from one model family, repair success with API history fell between 68.5% and 69.9%, while those models' closed-book scores ranged from 7.0% to 56.6%.
Perspective
The record targets developers and researchers working with Python libraries and coding agents, covering up to the 80 most recent final releases of each of 1,872 most-downloaded libraries, extracted statically and without imports. The checker was run on 1,823 source files of 14 widely used packages against their own installed dependency versions, raising 20 findings.
The author describes this as a high-precision, low-recall record rather than complete ground truth: compiled extensions are visible only through stubs, parameters altered by decorators are not modelled, and names created at runtime are often missing. All validation used AI agents isolated from the pipeline, without human adjudication; the clean out-of-sample precision estimates are 88.8% and 91.9%, and figures labelled in-sample were measured after fixes informed by the same samples.