Citation Telephone goes live: trace a cited sentence back through its references and see where the claim changed

Citation Telephone turns checking a citation from manual paper-by-paper lookup into an expandable citation trail: starting from a cited sentence, it opens each paper that sentence cites, finds the passage the sentence rests on, and marks whether the claim was altered in transit. Checking a citation previously meant opening the cited paper and finding the passage it was cited for, and tracing where a claim really came from meant repeating that for every paper it passed through; the 2017 citation tracing cited in the material found that the five-sentence 1980 New England Journal of Medicine letter from Porter and Jick was cited by 608 articles, of which 439 (72%) cited it as evidence that addiction was rare, and most never mentioned that its patients were hospitalized. On the held-out third set of 39 full-text citation pairs, the tool found a supporting passage in 38 (97%, 95% CI 87 to 99.5%) and reported support in 0 of 21 pairs where reviewers judged the paper did not state the claim (0%, 0 to 15%); across all three sets it found 115 of 125 (92%) and falsely reported support in 5 of 72 (7%).

The tool repeats the same steps at every paper, and only two of them, finding the passage and judging it, need a model; finding papers, parsing their structure and following references are plain code. The judgment comes from Jev, TypeSafe's System One decision model, which does not write prose but answers typed questions with probabilities; when picking a passage it can only choose among the paper's own paragraph and sentence IDs, so the evidence on screen is always text the paper contains. Open-access papers are read from the JATS XML full text publishers deposit, where every in-text citation links to its reference entry; arXiv papers are read from the HTML generated from their LaTeX. Fixed rules turn probabilities into results: an origin ends a branch, and a paper that passes the claim on is followed into its references, up to three per sentence, six hops and 24 papers per trace.

The tool can only read what is openly available: full text comes from the Europe PMC open-access subset, NIH author manuscripts in PubMed Central and arXiv, while other papers that are free to read but not openly licensed, and everything behind a paywall, are read from abstracts only; books, reports, guidelines and websites cannot be read at all. Every test pair comes from an open-access Europe PMC paper, mostly biomedical, with sentences from introductions or discussions citing at most three references, so methods citations, longer citation lists and other fields may behave differently.

The labels are AI judgments from a model family different from the tool's model, which limits shared blind spots but does not remove them, and no human expert checked them; the samples are small, so intervals are wide, especially for the third set and the false-find rate. The hop label is the weakest part, agreeing with reviewers in only 47% to 55% of pairs, with nearly all disagreements being the tool calling a full match partial; in every run of the third set, one citation the reviewers judged sound received a going-further or changed warning, while none of the three citations the reviewers found problematic was labeled a match, and three problems are far too few to tell how reliably the tool catches them. The tool checks citations, not truth, and an origin can itself be wrong.