The Implementation and Optimization of LICOM on GPUs

doi:10.11871/jfdc.issn.2096-742X.2020.04.008

Abstract

Abstract:

[Objective] In order to accelerate the calculation of the LICOM oceanic circulation model and reduce the cost caused by the high resolution, this paper designs and implements a GPU accelerat-ed version using CUDA C. [Methods] Based on the latest version of LICOM3, this paper analyzes the parallel algorithms of ocean grid block, and uses CUDA threads to calculate the grid points in parallel, which enables porting of the main program of LICOM to the GPU platform, and data transmission and device memory usage are optimized. [Results] Experiments show that the simulation results of GPU version program are basically same as the original CPU version program, while achieving 9.31x to 1.27x speedup on 2 to 16 NVIDIA K20 GPUs compared with the same number of Intel Xeon E5-2680 V2 CPUs. [Limitations] Because there are many boundary synchronous communications in LICOM3, which limits the scalability of the program, and it is necessary to improve the scalability of the model through boundary communications optimization and algorithm optimization. [Conclusions] This paper implements and optimizes the GPU version of the LICOM3 program, achieves some speedup and keep a good scalability, which provides experience and reference for the development of larger-scale oceanic circulation model in the future.

Key words: GPU, CUDA, parallel computing, high performance computing, LICOM, oceanic circulation model

Zhang Liuying,Wang Pengfei,Zhang Feng,Liu Hailong,Lin Pengfei,Wang Tao,Wei Junlin,Tian Shaobo,Jiang Jinrong,Chi Xuebin. The Implementation and Optimization of LICOM on GPUs[J]. Frontiers of Data and Computing, 2020, 2(4): 92-104.

Figures/Tables 22

Fig.1

Fig.2

Fig.3

Fig.4

Fig.5

Fig.6

Fig.7

Table 1

Table 2

Fig.8

Fig.9

Table 3

Fig.10

Fig.11

Fig.12

Fig.13

Fig.14

Fig.15

Fig.16

Fig.17

Fig.18

Fig.19

References 24

[1]	孙泓川, 周广庆, 曾庆存 . IAP第四代大气环流模式的气候系统模式模拟性能评估[J]. 大气科学, 2012,36(02):215-233.
[2]	张学洪, 粱信忠 . A Numerical World Ocean General Circulation Model[J]. Advances in Atmospheric Sciences, 1989(01):44-61.
[3]	刘海龙 . 高分辨率海洋环流模式和热带太平洋上层环流的模拟研究[D].北京; 中国科学院大气物理研究所, 2002.
[4]	Pengfei LIN, Zhipeng YU, Hailong LIU , et al. LICOM Model Datasets for the CMIP6 Ocean Model Intercomparison Project[J]. Advances in Atmospheric Sciences, 2020,37(03):239-249.
[5]	Farina R, Dobricic S, Storto A , et al. A revised scheme to compute horizontal covariances in an oceanographic 3D-VAR assimilation system[J]. Journal of Computational Physics, 2015,284:631-647.
[6]	Farina R, Dobricic S, Cuomo S . Some Numerical Enhancements in a Data Assimilation Scheme[J]. AIP Conference Proceedings, 2013.
[7]	王文浩, 姜金荣, 王玉柱, 迟学斌 . 海洋模式LICOM的MIC并行优化[J]. 科研信息化技术与应用, 2015,6(03):60-67.
[8]	赵晓溪 . 基于SC_Tangram框架的海洋环流模式LICOM并行实现[D]. 北京;中国科学院大学, 2017.
[9]	Jinrong Jiang, Pengfei Lin . Porting LASG/IAP Climate system Ocean Model to GPUs using OpenACC[J]. IEEE Access, 2019,PP(99):1-1.
[10]	Hoshino T, Maruyama N, Matsuoka S , et al. CUDA vs OpenACC: Performance Case Studies with Kernel Benchmarks and a Memory-Bound CFD Application [C].IEEE/ACM International Symposium on Cluster, 2013.
[11]	NVIDIA Corporation. High Performance Computing[EB/OL]. https://www.nvidia.cn/high-performance-computing/, 2020.
[12]	迟学斌, 顾蓓蓓, 武虹, 王龙, 朱鹏 . 高性能计算机系统及平台发展状况分析[J]. 计算机工程与科学, 2013,35(11):6-13.
[13]	TOP500.org.11/2019Higlights[EB/OL]. /, 2019.
[14]	Huadong Xiao, Jing Sun, Xiaofeng Bian , et al. GPU acceleration of the WSM6 cloud microphysics scheme in GRAPES model[J]. Computers and Geosciences, 2013,59(SEP.):156-162.
[15]	Mielikainen Jarno, Huang Bormin, Huang A , et al. GPU Acceleration of the Updated Goddard Shortwave Radiation Scheme in the Weather Research and Forecasting (WRF) Model[J]. IEEE Journal of Selected Topics in Applied Earth Observations And Remote Sensing, 2012,5(2):p. 555-562.
[16]	郭松, 窦勇, 雷元武 . 海洋环流数值模式POP的GPU并行优化[J]. 计算机工程与科学, 2012,34(08):147-153.
[17]	王春晖, 苗春葆, 沈飙 . 非静压海洋数值模式加速计算的CUDA实现[J]. 中国海洋大学学报(自然科学版), 2013,43(08):107-113+118.
[18]	张理论, 宋君强, 李晓梅 . LASG/IAP全球海洋环流模式的并行计算及其优化[J]. 计算机工程, 2003(14):15-17.
[19]	季旭, 武海平, 邹寅隆 , et al. 面向LICOM2的并行I/O优化[J]. 科研信息化技术与应用, 2014,5(05):37-48.
[20]	NVIDIA Corporation. CUDA C++ Programming Guide[EB/OL]. https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html, 2019.
[21]	Mark Harris . How to Optimize Data Transfers in CUDAC/C++[EB/OL].https://devblogs.nvidia.com/how-optimize-data-transfers-cuda-cc/, 2012.
[22]	扶月月, 王武, 王乔 . 基于FMM-PM方法的宇宙N体模拟在GPU上的实现和优化[J]. 数据与计算发展前沿, 2020,2(2):155-164.
[23]	周广庆, 张云泉, 姜金荣, 张贺, 吴保东, 曹杭, 王天一, 郝卉群, 朱家文, 袁良, 张明华 . 地球系统模式CAS-ESM[J]. 数据与计算发展前沿, 2020,2(1):38-54.
[24]	张云泉, 袁良, 袁国兴, 李希代 . 2019年中国高性能计算机发展现状分析与展望[J]. 数据与计算发展前沿, 2020,2(1):18-26.

硬件/软件名称	型号/版本号	个数
GPU	Nvidia Tesla K20	2
CPU	Intel Xeon E5-2680 V2 (10 core \| 2.8 GHz)	2
Intel MPI 4.1.3.0491	4.1.3.049	1
CUDA	6.0.37	1