Inverse rendering stops bolting priors on as losses: shared lattice variables lift relighting PSNR by 1.5–2.5 dB
Synopsis
Physically based inverse rendering must separate materials from illumination, and the decomposition is ambiguous. Learned material predictions are usually added as auxiliary regularization losses, which change the objective and require rebalancing against photometric fitting. This work instead restricts the optimization space: multi-view material predictions are fused onto a mesh, a permutohedral lattice is built in the prior feature space, and surface points with similar predictions share material variables while the objective stays purely photometric. On Synthetic4Relight and TensoIR-Synthetic, relighting PSNR with two predictors is 1.5–2.5 dB above the strongest evaluated mesh-based baseline, and it beats a regularization loss using the same prior in controlled comparisons.
Interpretation
Materials are no longer solved independently per vertex; they are represented by shared lattice variables in a prior feature space, and the objective remains a purely photometric fit. Earlier approaches wrote learned material predictions as auxiliary regularization losses, which alter the optimization objective and require balancing the regularization strength against the photometric loss. On Synthetic4Relight and TensoIR-Synthetic, the two prior configurations improve relighting PSNR by 1.47–1.50 dB and 2.28–2.48 dB over the strongest evaluated mesh-based baseline.
Multi-view material predictions are fused onto a mesh, and a permutohedral lattice is built in the predicted feature space so that vertices with similar predictions share one set of material variables. The permutohedral lattice was designed for fast high-dimensional Gaussian filtering; here it defines how material variables are shared, and the mapping between lattice and vertices stays fixed throughout joint optimization. The evaluation runs on Synthetic4Relight, TensoIR-Synthetic, DTC-Synthetic and Stanford-ORB with two predictors, MVInverse and DiffusionRenderer, and quantitative results are means of three runs with random seeds.
Shared variables pool per-vertex photometric gradients, making material updates more consistent across sampled views, which allows the joint optimization to be shortened substantially. Without a prior, more than 90% of update energy is noise that varies with sampling; with the lattice reparameterization, the consistent share rises from 8% to 54% for albedo and from 2% to 25% for roughness. On TensoIR-Synthetic a 100-step schedule brings the update loop to 34–39 seconds while relighting quality stays above all evaluated baselines.
Perspective
The demonstrated setting is reconstructing a triangle mesh with PBR materials and an environment map from multi-view images under fixed illumination; metallic is fixed to zero in the main results and retained only as a prior feature. Quantitative results cover Synthetic4Relight, TensoIR-Synthetic, DTC-Synthetic and Stanford-ORB, with synthetic benchmarks averaged over three random seeds.
When prior predictions are blurry or inconsistent across views, similar predictions can couple vertices with different underlying materials and yield suboptimal material estimates; input-guided prior refinement restores some detail contrast but cannot create contrast from locally constant predictions, and illumination or geometry edges can still misguide selection. Optimized albedo remains 1.94–4.35 dB below the lattice oracle, leaving room for improvement within the existing representation.