数据与计算发展前沿 ›› 2026, Vol. 8 ›› Issue (3): 68-80.

doi: 10.11871/jfdc.issn.2096-742X.2026.03.007

• 技术与应用 • 上一篇    下一篇

一种基于多源数据清洗与融合的高质量海洋观测廓线数据集构建方法

原惠峰1,3(),朱雨静2,3,潘玉莹2,张荣望4,*(),金钟1,3,*()   

  1. 1 中国科学院计算机网络信息中心， 北京 100083
    2 中国科学院大气物理研究所， 北京 100029
    3 中国科学院大学， 北京 100190
    4 中国科学院南海海洋研究所， 广东 广州 510301
  • 收稿日期:2025-08-25 出版日期:2026-06-20 发布日期:2026-06-18
  • 通讯作者: 张荣望,金钟
  • 作者简介:原惠峰，中国科学院计算机网络信息中心，博士研究生，工程师，研究方向为高性能计算、软件开发与优化。
    本文主要承担的工作为：方案设计及实现。
    YUAN Huifeng, is a Ph.D. candidate and engineer at the Computer Network Information Center, Chinese Academy of Sciences. His research interests include high-performance computing and software development and optimization.
    In this paper, he is mainly responsible for scheme design and implementation.
    E-mail: hfyuan@cnic.cn|张荣望，中国科学院南海海洋研究所，博士，副研究员，研究方向为热带海气耦合观测和模拟的数据处理和应用。
    本文主要承担的工作为：方案设计指导。
    ZHANG Rongwang, Ph.D., is an associate researcher at the South China Sea Institute of Oceanology, Chinese Academy of Sciences. His research interests include data processing and applications in tropical air-sea coupling observation and modeling.
    In this paper, he is mainly responsible for scheme design guidelines.
    E-mail: rwzhang@scsio.ac.cn|金钟，中国科学院计算机网络信息中心，高性能计算技术与应用发展部主任，研究员，研究方向为高性能计算。
    本文主要承担的工作为：方案设计指导。
    JIN Zhong, is the researcher at the Computer Network Information Center, Chinese Academy of Sciences. He also serves as the director of the Department of High-Performance Computing Technology and Application Development. His research interests include high-performance computing and biomedical computing.
    In this paper, he is mainly responsible for scheme design guidelines.
    E-mail: zjin@sccas.cn
  • 基金资助:
    亚洲合作资金项目(102173250600000000010);国家重点研发计划(2023YFB3001900)

A High-Quality Ocean Observation Profile Datasets Construction Scheme Based on Multi-Source Data Cleaning and Fusion

YUAN Huifeng1,3(),ZHU Yujing2,3,PAN Yuying2,ZHANG Rongwang4,*(),JIN Zhong1,3,*()   

  1. 1 Computer Network Information Center, Chinese Academy of Sciences, Beijing 100083, China
    2 Institute of Atmospheric Physics, Chinese, Chinese Academy of Sciences, Beijing 100029, China
    3 University of Chinese Academy of Sciences, Beijing 100190, China
    4 South China Sea Institute of Oceanology, Chinese Academy of Sciences, Guangzhou, Guangdong 510301, China
  • Received:2025-08-25 Online:2026-06-20 Published:2026-06-18
  • Contact: ZHANG Rongwang,JIN Zhong

RichHTML

1

PDF

23

摘要：

【背景】随着海洋观测技术的发展，各类海洋设备、计划应运而生，海洋科学领域的研究进入以大数据为代表的“数据密集型”科研阶段。 【目的】为了结合不同来源的异构海洋观测数据，形成一套大而完整的海洋观测数据集，全面提升对海洋科学问题的科研水平，本文提出一种对多源异构海洋原位观测廓线数据的标准化、标注、清洗，构建高质量海洋观测廓线数据集的方案。 【方法】具体为：从若干海洋数据中心/观测机构获取多源异构原始海洋观测廓线数据以及数据描述信息；根据原始海洋观测廓线数据和描述信息确定的唯一标识符，依次对原始海洋观测廓线数据进行黑名单设备数据清洗、多版本数据清洗，以及基于时空联合特征的高频数据清洗，得到目标海洋观测廓线数据；将目标海洋观测廓线数据进行标准化处理后，进行数据质量标注和误差修订，构建高质量的廓线数据集。 【结论】该方案将促进多源异构廓线数据融合应用，提升数据的一致性、准确性与可用性。

关键词: 海洋大数据, 数据清洗, 海洋观测, 数据集

Abstract:

[Background] With the development of ocean observation technologies, various marine equipment and programs have emerged, propelling research in marine science into a “data-intensive”stage characterized by big data. [Objective] To integrate heterogeneous ocean observation data from diverse sources into a comprehensive and unified dataset, thereby enhancing holistic scientific capabilities in addressing marine research questions, this paper proposes a scheme for standardizing, annotating, and cleaning multi-source heterogeneous in situ ocean observation profile data to construct a high-quality ocean observation profile dataset. [Methods] Specifically, the scheme involves acquiring multi-source in-situ ocean observation profile data and corresponding metadata from several ocean data centers/agencies; applying a unique identifier derived from the raw data and descriptors to sequentially execute greylist filtering, multi-version filtering, and high-frequency observations filtering based on spatiotemporal characteristics, yielding refined ocean observation profile data; standardizing the processed data, followed by quality control and bias correction to construct a high-quality profile dataset. [Conclusions] This scheme promotes the application of multi-source heterogeneous profile data, improving data consistency, accuracy, and usability.

Key words: ocean big data, data clean, ocean observation, datasets