Robust aggregators scrub federated watermarks: camouflaging watermarked updates as inliers keeps one client's 3.3% effective watermark detectable
In federated LLM fine-tuning, a server using the strong Byzantine-robust aggregator RandEigen filters out existing text-watermark updates as outliers, but optimizing the watermarked samples' input embeddings to sit close to clean updates keeps the watermark detectable by black-box access from a single client's 3.3% effective watermarked data.