Before the needles turn: a multi-scale vision transformer reads drone hyperspectral imagery to flag pine wilt at the green attack stage
Synopsis
Pine wilt disease is hardest to catch before needles visibly discolour, and RGB drone imagery lacks the cues. A multi-scale vision transformer, LMS-ViT, was built for drone hyperspectral data, combining a spectral-spatial fusion module, a Gaussian self-attention mechanism and a multi-scale feature pyramid to model spectral differences between central pixels and neighbours. In a Japanese black pine stand in Shandong, it identified green attack and early stage trees at 87.40% and 88.21% accuracy, with 87.90% overall accuracy, a Kappa of 83.25% and 2.45 M parameters.
Interpretation
Pine wilt trees in the green attack and early stages show almost no visible symptoms in drone imagery, so the hard cases are the ones with slight needle chlorosis and very small lesions. Earlier identification methods mostly targeted mid and late stages with obvious symptoms, relied on traditional machine learning classifiers and used spectral information while leaving spatial and texture features underexploited. In Japanese black pine forest on the coastal low hills of Muping, Yantai, Shandong, 23 plots of 25 m by 25 m were surveyed and confirmed by wood-chip sampling and microscopic nematode identification as 296 healthy and 113 infected trees, including 12 at green attack and 18 at early stage.
The model places pixel-level to canopy-level features in one multi-scale hierarchy and models the spectral difference between a central pixel and its neighbours separately. It pairs grouped convolution with depthwise separable convolution in a spectral-spatial fusion module, replaces the relative positional bias of multi-head attention with a Gaussian absolute bias, and halves channels layer by layer to build a feature pyramid. Comparisons were implemented uniformly in PyTorch on an NVIDIA RTX 3090 against random forest, 2DCNN, 3DCNN, HybridSN, ViT, MorphFormer and SSFIT.
On global metrics, LMS-ViT ranks first among the compared models with 87.90% overall accuracy, 87.79% average accuracy and a Kappa coefficient of 83.25%. Random forest reached 67.68% overall accuracy and a Kappa of 56.34%, with only 44.29% at green attack; MorphFormer reached 84.60% overall, 75.76% at green attack and 71.64% at early stage. LMS-ViT's overall accuracy is 1.73 percentage points above SSFIT's 86.17%, and its Kappa is 3.23 points above MorphFormer's 80.02%; it uses 2.45 M parameters, 112.42 M FLOPs, 114 minutes of training and 25 minutes of testing.
Perspective
Results come from Japanese black pine forest on the coastal low hills of Muping, Yantai, Shandong, imaged on 19 July 2024 between 11:00 and 14:00 with a DJI Matrice M300 RTK carrying a UHD185 hyperspectral imager at 4.4 cm spatial resolution; the model was trained and tested in that study area under a shared experimental setup.
Plot labels required manual region-of-interest delineation per tree from field surveys and hyperspectral data, and annotating 23 plots is time-consuming, making large-scale monitoring hard to adapt; acquisition needs clear skies and wind speed no more than 3 m/s, and the large raw data volume makes preprocessing and inference costly, so real-time monitoring is not currently achievable; the model relies on spectral features specific to the study area, adapts weakly to extreme conditions such as after heavy rain or drought, and does not account for interference from other pests and diseases such as pine caterpillars and pine brown spot needle blight.