Deep learning based on self-supervised pre-training: Application on sandstone content prediction - Wang, Chong Ming; Wang, Xing Jian; Chen, Yang; Wen, Xue Mei; Zhang, Yong Heng; Li, Qing Wu (Frontiers in Earth Science, 2023)
Deep learning has been widely used in various fields and performs well in recent years. Therefore, deep learning is the future trend to realize the intelligent and automatic interpretation of seismic data. However traditional deep learning only uses labeled data to train the model which wastes a large amount of unlabeled data. Self-supervised learning, widely used in natural language processing (NLP) and computer vision, is an effective method of learning information from unlabeled data. In this paper, a pretext task is designed with reference to BERT and MAE, through the pretext task we can realize self-supervised pre-training of unlabeled seismic data. After pre-training, we fine-tune the model to the downstream task. Experiments show that the model can effectively extract information from unlabeled data through the pretext task, and the pre-trained model has better performance in downstream tasks.