Sy-Tuyen Ho

Starting from Llama 3.1 8B, the study first fine-tunes a reviewer on official ICLR reviews from 2018–2023, then trains four successor models on ICLR 2024 data with synthetic reviews generated by that reviewer mixed at 0%, 33%, 66%, and 100%, finding that higher synthetic exposure compresses rating distributions and monotonically reduces same-paper and corpus-level semantic diversity (about 11% and 5%), a pattern the authors call scientific-judgment collapse, and introduces TrustReviewer, which mitigates this tendency through training-time corpus curation and test-time paired activation steering.

Starting from Llama 3.1 8B, the study first fine-tunes a reviewer on official ICLR reviews from 2018–2023, then trains four successor models on ICLR 2024 data with synthetic reviews generated by that reviewer mixed at 0%, 33%, 66%, and 100%, finding that higher synthetic exposure compresses rating distributions and monotonically reduces same-paper and corpus-level semantic diversity (about 11% and 5%), a pattern the authors call scientific-judgment collapse, and introduces TrustReviewer, which mitigates this tendency through training-time corpus curation and test-time paired activation steering.