Direct Numerical Simulation of Hypersonic Turbulence Based on CPU/GPU Heterogeneous System Architecture

doi:10.11871/jfdc.issn.2096-742X.2020.01.009

Abstract

Abstract:

[Objective] The direct numerical simulation (DNS) of hypersonic turbulence requires great many grids points and time steps. Therefore, the amount of calculation is very large. Excessively long time of calculation is an important reason that DNS cannot be applied in real applications. In order to accelerate the calculation, design of a high-performance computational fluid mechanics program OpenCFD-SCU under the CPU/GPU Heterogeneous System Architecture (HSA) is introduced in this paper. [Method] This program is based on the CPU [fortran] code OpenCFD-SC which is a high-precision finite difference solver developed by the authors. OpenCFD-SCU has the same program framework as OpenCFD-SC, and the computing part of the GPU program is programmed by CUDA to ensure that all arithmetic operations are completed on the GPU. [Results] In a same DNS task, the GPU version of OpenCFD-SCU is 60 times faster than the CPU version of OpenCFD-SC. The computing power of GPU is much higher than that of CPU. Using GPU can effectively accelerate the calculation, which is the future trend of DNS programs for hypersonic turbulence. [Conclusion] In the future, we believe that more and more hypersonic turbulence simulation can be moved to the GPU.

Key words: hypersonic turbulence, DNS, GPU

Dang Guanlin,Liu Shiwei,Hu Xiaodong,Zhang Jian,Li Xinliang. Direct Numerical Simulation of Hypersonic Turbulence Based on CPU/GPU Heterogeneous System Architecture[J]. Frontiers of Data and Computing, 2020, 2(1): 105-116.

Figures/Tables 15

Fig.1

Fig.2

Table 1

Fig.3

Fig.4

Table 2

Fig.5

Table 3

Fig.6

Fig.7

Fig.8

Fig.9

Table 4

Fig.10

Fig.11

References 22

[1]	Lesieur M, Metais O, Comte P . Large-eddy simulations of turbulence[M]. Cambridge University Press, 2005.
[2]
[3]	Takashi Amada , et al. Particle-Based Fluid Simulation on GPU. .
[4]	Krüger J, Westermann R . Linear algebra operators for GPU implementation ofnumerical algorithms[J]. ACM Trans. Graph. 2003,22(3):908-916.
[5]	Brandvik T, Pullan G . Acceleration of a two-dimensional Euler flow solver usingcommodity graphics hardware[J]. Proceedings of the Institution of MechanicalEngineers, Part C: Journal of Mechanical Engineering Science. 2007,221(12):1745-1748.
[6]	Brandvik T, Pullan G . Acceleration of a 3D Euler Solver Using Commodity Graph-ics Hardware[M]. Aerospace Sciences Meetings. Ameri-can Institute of Aeronautics and Astronautics, 2008.
[7]	Jacobsen D, Thibault J, Senocak I . An MPI-CUDA Implementation for MassivelyParallel Incompressible Flow Computations on Multi-GPU Clusters[M]. Aerospace Sciences Meetings. American Institute of Aeronautics and Astronautics, 2010.
[8]	Jacobsen D, Senocak I . Scalability of Incompressible Flow Computations onMulti-GPU Clusters Using Dual-Level and Tri-Level Parallelism[M] . Aerospace Sciences Meetings. American Institute of Aeronau-tics and Astronautics, 2011.
[9]	Jacobsen D A, Senocak I . Multi-level parallelism for incompressible flow compu-tations on GPU clusters[J]. Parallel Computing. 2013,39(1):1-20.
[10]	Khajeh-Saeed Ali, J. Blair Perot . Direct numerical simulation of turbulence using GPU accelerated supercomputers[J]. Journal of Computational Physics, 2013,235:241-257.
[11]	X. Wang, Y. Shangguan, N. Onodera, H. Kobayashi and T. Aoki . Direct numerical simulation and large eddy simulation on a turbulent wall-bounded flow using lattice Boltzmann method andmultiple GPUs[J]. Mathematical Problems in Engineering,vol. 2014.
[12]	Emelyanov, Vladislav N., A. G. Karpenko, K. N. Volkov .Development of advanced computational fluid dynamics tools and their application to simulation of internal turbulent flows[J]. Progress in Flight Physics-Volume 7, 2015,7:247-268.
[13]	Emelyanov, Vladislav, Anton Karpenko, Konstantin Volkov. Development and acceleration of unstructured mesh-based CFD solver[J]. Progress in Flight Physics-Volume 9, 2017,9:387-408.
[14]	Lai, Jianqi, Hua Li, Zhengyu Tian . CPU/GPU Heterogeneous Parallel CFD Solver and Optimizations [C]. In Proceedings of the 2018 International Conference on Service Robotics Technologies. ACM, 2018: 88-92.
[15]	LAI, Jianqi , et al. A Multi-GPU Parallel Algorithm in Hypersonic Flow Computations[J]. Mathematical Problems in Engineering, 2019.
[16]	.
[17]	.
[18]	Joseph L Steger, R.F Warming . Flux vector splitting of the inviscid gasdynamic equations with application to finite-difference methods[J]. Journal of Computational Physics, 40(2):263-293.
[19]	M.P. Martín, E.M. Taylor, M. Wu , et al. A bandwidth-optimized WENO scheme for the effective direct numerical simulation of compressible turbulence[J]. Journal of Computational Physics, 220(1):270-289.
[20]	李新亮 . OpenCFD-SC用户手册. .
[21]	Li X, Fu D, Ma Y . Direct numerical simulation of hypersonic boundary layer transition over a blunt cone with a small angle of attack[J]. Physics of Fluids, 2010,22(2):025105.
[22]	李新亮 . OpenCFD-EC理论手册. .

Rank	Machine	Architecture
1 2	Summit Sreera	异构加速器
3	神威太湖之光	异构众核
4	TH-2	异构加速器
5	Frontera	同构众核
6	Piz Daint	异构加速器
7	Trinity	同构众核
8	ABCI	异构加速器
9	SuperMUC-NG	同构众核
10	Lassen	异构加速器

Block大小	时间（ms)
3211	821.47
3221	449.34
3222	349.12
3242	349.37
3244	371.04

	GPU卡数	每步计算时间 (S/秒）
case 1	16	1.591
case 2	32	0.745
case 3	48	0.551
case 4	64	0.395