数据与计算发展前沿 ›› 2024, Vol. 6 ›› Issue (2): 145-155.

CSTR: 32002.14.jfdc.CN10-1649/TP.2024.02.013

doi: 10.11871/jfdc.issn.2096-742X.2024.02.013

• 技术与应用 • 上一篇    下一篇

高能同步辐射光源科学数据传输系统设计和实现

庄博*(),胡皓,齐法制,曾珊   

  1. 中国科学院高能物理研究所，北京 100049
  • 收稿日期:2022-12-28 出版日期:2024-04-20 发布日期:2024-04-26
  • 通讯作者: *庄博（E-mail: zhuangbo@ihep.ac.cn）
  • 作者简介:庄博，中国科学院高能物理研究所计算中心，硕士，工程师，负责高能同步辐射光源科学数据传输工作。
    主要研究方向为数据高效传输技术。
    本文中负责系统的设计和实现、论文整体架构和撰写。
    ZHUANG Bo, a master’s degree holder and Engineer at the Computing Center, Institute of High Energy Physics, is responsible for scientific data transfer of high energy photon source. His main research areas include efficient data transfer technologies.
    In this paper, he is responsible for designing and implementing the system, paper structure organizing and writing.
    E-mail: zhuangbo@ihep.ac.cn
  • 基金资助:
    国家自然科学基金“面向北京同步辐射装置科学元数据管理的关键技术研究”(12005247);国家自然科学基金“高能物理云网融合数据中心网络关键技术研究”(12175258);中国科学院网信专项“基于人工智能的高能物理大数据技术研究与示范”(CAS-WX2022SF-0104);高能物理科学数据传输系统关键技术研究(2023000057)

The Design and Implementation of Scientific Data Transfer System for High Energy Photon Source

ZHUANG Bo*(),HU Hao,QI Fazhi,ZENG Shan   

  1. High Energy Physics Institute, Chinese Academy of Sciences, Beijing 100049, China
  • Received:2022-12-28 Online:2024-04-20 Published:2024-04-26

RichHTML

1

PDF

5

摘要：

【目的】高能同步辐射光源HEPS（High Energy Photon Source）是我国第一台高能量同步辐射光源，预计2025年建成。HEPS建成后，首批投入使用的15条光束线站预计每天会产生200TB实验数据。为满足不同阶段对实验数据的读写需求，会对海量实验数据进行分级存储和长期备份。如何将这些数据在不同存储介质之间稳定和高效地传输是HEPS实验中需要解决的重要问题。【方法】HEPS科学数据传输系统设计了多源DAQ接口、任务调度、消息队列、集群传输、日志监控、配置管理和消息通知等功能模块。【结论】系统实现了海量数据在不同存储介质间的高效、可靠传输。【结果】系统现部署在北京同步辐射多个线站，用作HEPS数据传输的前期验证，运行稳定、效果良好，为科研人员和用户提供了良好的数据传输服务。

关键词: 高能同步辐射光源, 高效传输, 可靠传输, 传输系统

Abstract:

[Objective] High Energy Photon Source (HEPS) is the first High Energy Photon Source in China. It is expected to be completed in 2025. After completion of HEPS, it is expected to produce 200TB of experimental data every day from the 15 public beamlines and stations in the first construction phase. To meet the needs of reading and writing experimental data at different stages, massive experimental data will be stored hierarchically and backed up for a long time. Therefore, it is necessary to transfer these data stably and efficiently between the different storage. [Methods] The HEPS scientific data transfer system is designed with a multi-source DAQ interface, task scheduling, message queue, cluster transfer, log monitoring, configuration management, message notification, etc. [Conclusions] The system realizes efficient and reliable transfer of massive data between different storage media. [Results] The system is now deployed in several line stations of the Beijing Synchrotron Radiation Facility, which is used for the preliminary verification of HEPS data transfer. It runs stably and has good effects, providing terrific data transfer services for researchers and users.

Key words: high energy photon source, reliable transfer, efficient transfer, transfer system