Frontiers of Data and Computing ›› 2023, Vol. 5 ›› Issue (3): 66-91.

CSTR: 32002.14.jfdc.CN10-1649/TP.2023.03.006

doi: 10.11871/jfdc.issn.2096-742X.2023.03.006

• Special Issue: AI for Science • Previous Articles     Next Articles

Status, Challenges, and Trends of Data-Intensive Supercomputing

WEI Jia1(),CHEN Mo2,WANG Longxiang1,*(),REN Pei2,LEI Yujia1,QU Yuqi1,JIANG Qiyu1,DONG Xiaoshe1,WU Weiguo1,ZHANG Kaili2,ZHANG Xingjun1   

  1. 1. School of Computer Science and Technology, Xi’an Jiaotong University, Xi’an, Shaanxi 710049, China
    2. Huawei technologies Co. Ltd., ShenZhen, Guangdong 518129, China
  • Received:2022-06-22 Online:2023-06-20 Published:2023-06-21
  • Contact: *王龙翔(E-mail: wlx419@xjtu.edu.cn

Abstract:

[Objective] This paper is to provide a comprehensive and systematic overview of the development history, mainstream system architecture, typical applications, and computation and storage subsystems of data-intensive supercomputing, point out the future development trend, and provide references for further data-intensive supercomputing optimization. [Methods] This paper first sorts out the key concepts of data-intensive supercomputing and analyzes the support to the data-intensive applications by existing platforms. Then the real demand for data-intensive applications from the mainstream academic and industrial communities are illustrated. Finally, the future trends and potential challenges of data-intensive supercomputing are discussed and a corresponding supercomputing system evaluation model is developed. [Results] Relevant researchers and practitioners can quickly understand the key concepts and development status of supercomputing technology from this paper, and precisely capture the current and future data-intensive supercomputing research hotspots and key problems that need to be solved. [Conclusions] The problems such as the optimization on complex data type and mixed workload, and multi-protocol support and interoperability which are faced by the data-intensive supercomputing storage systems will become hot research and development issues in the coming years.

Key words: data-intensive supercomputing, I/O intensive supercomputing, high performance data analytics, parallel processing system, supercomputing storage system