95% intervals calibrated on random splits fail outside the training envelope: hard-shift coverage falls to 0.40–0.75 across seven engineering datasets
Prediction intervals for engineering regression are usually calibrated on random splits, while deployment happens outside training conditions. A benchmark splits seven public engineering datasets into in-distribution, soft-shift and hard-shift rungs along physically meaningful shift variables, and compares 12 uncertainty quantification methods from five families tuned on source data only. In distribution, conformal methods cover at least 0.946 on every dataset; under hard shift, mean coverage falls to 0.40–0.75 for the 11 methods that always return an interval, and no systematic difference in method ranks is detected.