High-Precision Wave-Parameter Perception via Spatiotemporal Coupling of Sea-Clutter Imagery and Ship-Motion Responses
Accurate detection of near-wave field parameters is crucial for safe navigation and efficient offshore operations. However, estimating precise wave parameter remains challenging due to strong nonlinear wave dynamics and inherent measurement uncertainties. Most of existing deep learning methods relying on single modality data face limitations in insufficient accuracy. Motivated by advances in multimodal fusion algorithms, we propose a novel maritime multimodal fusion inversion model, MR-FuNet. The proposed model integrates a Convolutional Neural Network (CNN) and Bidirectional Long Short-Term Memory (BiLSTM) in parallel, enabling effective fusion of spatial information from X-band radar sea clutter images and temporal patterns from ship motion data at the feature level. A multi-dimensional attention strategy was employed to enable the model to dynamically calibrate and integrate heterogeneous information across modalities and feature domains which can significantly improve the accuracy of inversion for significant wave height and characteristic wave period. To address the lack of comprehensive and high-quality public datasets in this research area, this study constructs and releases a large-scale and multimodal dataset <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Radar Images and Ship Motion Dataset(RSD). RSD is a large-scale multimodal dataset generated via numerical simulation and covers 99 representative sea states. It provides a valuable benchmark for future research. Extensive experiments validate the effectiveness of the proposed model, demonstrating substantial performance gains across various metrics. Compared to traditional Artificial Neural Networks (ANN), the proposed multimodal fusion model achieves notable reductions in RMSE by 61.6% and 62.8% for the characteristic period and significant wave height inversion tasks, respectively. Dataset is available at https://github.com/felixfelixXu/Radar-Images-and-Ship-Motion-Dataset.