Multimodal Railway Defect Detection with Visual and Inertial Data: Early Fusion Strategies and Misalignment Analysis

This paper investigates multimodal railway defect detection using visual and inertial data. A simple and reproducible baseline is proposed by projecting 1D accelerometer signals into a 2D representation compatible with convolutional object detectors. Two fusion strategies are evaluated: channel-wise concatenation and multiplicative modulation. Experimental results show that concatenation provides a robust improvement over image-only models, while multiplicative fusion leads to performance degradation, particularly for fine surface defects. In addition, a desynchronization study demonstrates that fusion performance is sensitive to spatial offsets, although concatenation remains relatively robust. The results also reveal that optimal performance is not always achieved at perfect alignment, highlighting the importance of signal coverage within the projection window. Overall, this work emphasizes simple early fusion and alignment robustness for multimodal railway inspection.