Abstract:

[Objective] Considering the limitations of traditional methods, such as insufficient classification accuracy of geomagnetic ultra-low frequency (ULF) waves in noisy environments and inferior capability in modeling long-term temporal dependencies, we propose Geoformer, a novel framework integrating depthwise separable convolution and temporal Transformer. By fusing convolutional neural networks (CNNs) with an improved Transformer architecture and optimizing positional encoding strategies, Geoformer enhances the recognition accuracy and model generalization ability for ULF waves. [Methods] This method uses CNNs to extract the local time-domain features and multi-channel spatial correlations of geomagnetic signals. Meanwhile, through time absolute positional encoding (tAPE) and efficient relative positional encoding (eRPE), it strengthens the model’s perception of the absolute positions and relative distances of signal sequences. Finally, it employs the multi-head self-attention mechanism to capture long-term temporal dependencies and multi-dimensional interactive features, achieving accurate classification. [Results] Experimental results indicate that, compared with traditional CNNs and basic Transformer models, Geoformer improves the classification accuracy by 12.2% on real world geomagnetic datasets and is significantly superior to traditional deep learning models like LSTM, GRU, and ResNet. [Limitations] The model’s computational complexity grows quadratically with the increase of signal length, necessitating the incorporation of downsampling techniques in real-time processing of ultra-long time sequences. Moreover, it relies on high-quality labeled data, so transfer learning or self-supervised pre-training should be introduced in small sample scenarios. [Conclusions] Through the innovation of the CNN-Transformer architecture and the optimization of positional encoding, this paper provides an efficient solution for the intelligent analysis of geomagnetic ultra-low frequency wave signals, holding significant promise for playing an important role in fields such as geophysical monitoring and space weather early warning.

Key words: geomagnetic ultra-low frequency waves, geoformer, signal classification