A Mixed Precision Multigrid Preconditioning Algorithm for Domestic DCU Accelerator and Its Application

doi:10.11871/jfdc.issn.2096-742X.2025.05.004

Abstract

Abstract:

[Objective] The multigrid method is an extremely effective approach for solving discretized systems of elliptic partial differential equations. The research and software development of its high-efficiency heterogeneous parallel algorithms have been key challenges in the field of scientific and engineering computing. [Methods] This paper proposes a heterogeneous parallel multigrid method tailored for domestic accelerators and applies it to solve the non-isothermal reservoir problem using the Constrained Pressure-Temperature Residual (CPTR) preconditioner. Based on this, two heterogeneous parallel mixed-precision acceleration algorithms are designed. [Conclusions] Numerical experiments show that both the heterogeneous parallel multigrid method and the CPTR preconditioner achieve significant acceleration, with a speedup of 36 and over 10, respectively. Additionally, the mixed-precision strategy improves the performance of the CPTR preconditioner while maintaining computational accuracy, achieving a performance improvement of 15%-32% compared to the double-precision version.

Key words: multigrid methods, multistage preconditioners, mixed precision, heterogeneous parallelism, domestic accelerators, reservoir simulation

ZHANG Linjie,XING Xin,ZHAO Li,FENG Chunsheng. A Mixed Precision Multigrid Preconditioning Algorithm for Domestic DCU Accelerator and Its Application[J]. Frontiers of Data and Computing, 2025, 7(5): 41-53, https://cstr.cn/32002.14.jfdc.CN10-1649/TP.2025.05.004.

Figures/Tables 16

Fig.1

Fig.2

Fig.3

Fig.4

Fig.5

Fig.6

Fig.7

Table 1

Table 2

Fig.8

Fig.9

Table 3

Table 4

Table 5

Table 6

Table 7

References 35

[1]	HACKBUSCH W. Multi-Grid Methods and Applications[M]. Berlin: Springer, 1985: 41-97.
[2]	BRANDT A, MCCORMICK S F, RUGE J W. Algebraic multigrid (AMG) for sparse matrix equations[C]// Sparsity and Its Applications. Cambridge: Cambridge University Press, 1985: 257-284.
[3]	RUGE J W, STÜBEN K. Efficient solution of finite difference and finite element equations[M]// Multigr-id Methods for Integral & Differential Equations. Oxford: Oxford University Press, 1985: 169-212.
[4]	CLEARY A J, FALGOUT R D, HENSON V E, et al. Coarse-grid selection for parallel algebraic multigrid[C]//Proceedings of the 5th International Symposium on Solving Irregularly Structured Problems in Parallel (IRREGULAR 98). Lecture Notes in Computer Science 1457. Berlin: Springer, 1998: 104-115.
[5]	STERCK H D, YANG U M, HEYS J J. Reducing complexity in parallel algebraic multigrid preconditioners[J]. SIAM Journal on Matrix Analysis and Applications, 2006, 27(4): 1019-1039.
[6]	WAN W L, CHAN T F, SMITH B. An energy-minimizing interpolation for robust multigrid methods[J]. SIAM Journal on Scientific Computing, 1999, 21(4):1632-1649.
[7]	BREZINA M, KETELSEN C, MANTEUFFEL T, et al. Relaxation-corrected bootstrap algebraic multigr-id (rBAMG)[J]. Numerical Linear Algebra with Applications, 2012, 19(2): 178-193.
[8]	YANG U M. On long-range interpolation operators for aggressive coarsening[J]. Numerical Linear Algebra with Applications, 2009, 17(2-3): 453-472.
[9]	VANĚK P, MANDEL J, BREZINA M. Algebraic multigrid by smoothed aggregation for second and fourth order elliptic problems[J]. Computing, 1996, 56(3):179-196.
[10]	NOTAY Y. An aggregation-based algebraic multigrid method[J]. Electronic Transactions on Numerical Analysis, 2010, 37: 123-146.
[11]	BALAY S, ABHYANKAR S, ADAMS M, et al. PETSc v4.0.0-Portable Toolkit for Scientific Computing[EB/OL]. (2023-12-31)[2025-02-28]. https://petsc.org/.
[12]	GRIES S, METSCH B, TEREKHOV K M, et al. System-AMG for fully coupled reservoir simulation with geomechanics[C]// Proceedings of the SPE Reservoir Simulation Symposium. SPE Symposium Proceedings. Houston: Society of Petroleum Engineers (SPE), 2019: SPE-193887-MS.
[13]	DEMIDOV D. AMGCL: An efficient, flexible, and extensible algebraic multigrid implementation[J]. L- obachevskii Journal of Mathematics, 2019, 40(5):535-546.
[14]	FALGOUT R, YANG U M, LI R P, et al. HYPRE: High Performance Preconditioners[EB/OL]. (2023-12-31)[2025-02-28]. https://github.com/hypre-space/hypre.
[15]	NAUMOV M, ARSAEV M, CASTONGUAY P, et al. AmgX: A library for GPU accelerated algebraic multigrid and preconditioned iterative methods[J]. SIAM Journal on Scientific Computing, 2015, 37(5): S602-S626.
[16]	XU J C, ZHANG C S, FENG C S, et al. FASP: Fast Auxiliary Space Preconditioning Framework[EB/OL]. (2023-12-31)[2025-02-28]. http://www.fasp.org.
[17]	XU X W, YUE X Q, MAO R Z, et al. JXPAMG: A Parallel Algebraic Multigrid Solver for Extreme-S-cale Numerical Simulations[J]. CCF Transactions on High Performance Computing, 2023, 5: 72-83.
[18]	CHEN Z X, HUAN G R, MA Y L. Computational Methods for Multiphase Flows in Porous Media[M]. Philadelphia: Society for Industrial and Applied Ma- thematics, 2006: 381-397.
[19]	ZHAO L, FENG C S, ZHANG C S, et al. Parallel multi-stage preconditioners with adaptive setup for the black oil model[J]. Computers & Geosciences, 2022, 168: 105230.
[20]	ZHAO L, LI S Z, ZHANG C S, et al. An improved multistage preconditioner on GPUs for compositional reservoir simulation[J]. CCF Transactions on High Performance Computing, 2023, 5: 144-159.
[21]	WALLIS J R. Incomplete Gaussian elimination as a preconditioning for generalized conjugate gradient acceleration[C]// SPE Reservoir Simulation Conferen-ce. SPE Symposium Proceedings. Houston: Society of Petroleum Engineers (SPE), 1983: SPE-12265.
[22]	ROY T, JONSTHOVEL T B, LEMON C, et al. A constrained pressure-temperature residual (CPTR) method for non-isothermal multiphase flow in porous media[J]. SIAM Journal on Scientific Computing, 2020, 42(4): B1014-B1040.
[23]	XIE G S, ZHANG W Z. Research of thread placement optimization strategy for CUDA programs[J]. Intelligent Computer and Applications, 2020, 10(2):341-345.
[24]	WILKINSON J H. Progress report on the automatic computing engine[R]. Teddington, UK: Mathematics Division, National Physical Laboratory, 1948.
[25]	ARIOLI M, DUFF I S. Using FGMRES to obtain backward stability in mixed precision[J]. Electronic Transactions on Numerical Analysis, 2009, 33: 31-44.
[26]	EMANS M, MEER A. Mixed-precision AMG as linear equation solver for definite systems[J]. Procedia Computer Science, 2010, 1(1): 175-183.
[27]	FLEGAR G, ANZT H, COJEAN T, et al. Adaptive precision block-Jacobi for high performance preconditioning in the Ginkgo linear algebra software[J]. ACM Transactions on Mathematical Software, 2021, 47(2): 1-28.
[28]	武鹏. 基于国产加速器的VASP移植与杂化密度泛函计算优化研究[D]. 郑州大学, 2023.
[29]	李荣振. 基于飞腾平台的GPU图形加速技术研究及应用[D]. 国防科学技术大学, 2013.
[30]	BO M Y, HE J L, YUE Y Y, et al. Transplantation and optimization of SpMV algorithm based on DCU accelerator[C]//Proceedings of SPIE 12331: International Conference on Mechanisms and Robotics (ICMAR 2022), Bellingham: SPIE, 2022: 1301-1308.
[31]	GUO H, YUE Y, BO M Y, et al. Research of thread granularity optimization strategy for the domestic DCU accelerator[C]// 2021 IEEE International Conference on Information Science, Parallel and Distributed Systems (ISPDS), Piscataway: IEEE, 2021: 110-115.
[32]	冯春生. 油藏数值模拟中面向异构体系的多水平法及高效解法器研究[D]. 湘潭大学, 2014.
[33]	FENG C S, LI S Z, LIU S H, et al. Application-oriented preconditioning of seepage mechanics[J]. Chinese Journal of Computational Physics, 2024, 41(1):98-109. doi: 10.19596/j.cnki.1001-246x.8791
[34]	赵梨. 两类油藏模型的多水平预条件方法研究[D]. 湘潭大学, 2023.
[35]	LI S Z, ZHANG C S. OpenCAEPoro: A simulator for multicomponent porous media flow[EB/OL].(2022-9-22)[2025-02-28]. https://faspdevteam.github.io/OpenCAEPoro.

${L}_{\theta }$	IT	RRN	总时间(s)	加速比
0	11	9.39E-07	90.154	—
1	11	9.39E-07	26.390	3.42
2	11	9.39E-07	8.653	10.42
3	11	9.39E-07	4.262	21.15
4	11	9.39E-07	3.123	28.87
5	11	9.39E-07	2.822	31.95
6	11	9.39E-07	2.761	32.65
7	11	9.39E-07	2.757	32.70
8	11	9.39E-07	2.757	32.70
9	11	9.39E-07	2.753	32.75
10	11	9.39E-07	2.756	32.71
11	11	9.39E-07	2.763	32.63
12	11	9.39E-07	2.762	32.64
13	11	9.39E-07	2.762	32.64
14	11	9.39E-07	2.765	32.61

${L}_{\theta }$	IT	RRN	总时间(s)	加速比
0	15	9.84E-07	76.102	—
1	15	9.84E-07	12.290	6.19
2	15	9.84E-07	3.541	21.49
3	15	9.84E-07	2.417	31.49
4	15	9.84E-07	2.341	32.51
5	15	9.84E-07	2.315	32.87
6	15	9.84E-07	2.315	32.87
7	15	9.84E-07	2.317	32.85
8	15	9.84E-07	2.319	32.82
9	15	9.84E-07	2.317	32.85

并行模式	TS	NS	IT	启动阶段		求解阶段		解法器
并行模式	TS	NS	IT	时间(s)	加速比	时间(s)	加速比	时间(s)	加速比
CPU1	55	209	4,666	533.85	1.94	5,296.18	29.12	5,918.08	10.56
CPU32	55	209	4,666	645.86	2.35	1,124.13	6.18	1,858.39	3.32
DU	55	209	4,666	274.65	—	181.85	—	560.57	—

Precision	TS	NS	IT	启动阶段		求解阶段		解法器
Precision	TS	NS	IT	时间(s)	加速比	时间(s)	加速比	时间(s)	加速比
Double	55	208	4,622	208.48	—	2,502.88	—	2,711.36	—
Mixed	55	208	4,622	178.42	1.17	2,089.62	1.20	2,268.04	1.20

Precision	TS	NS	IT	启动阶段		求解阶段		解法器
Precision	TS	NS	IT	时间(s)	加速比	时间(s)	加速比	时间(s)	加速比
Double	55	208	5,345	36.70	—	190.62	—	287.81	—
Mixed	55	208	5,345	32.37	1.13	155.85	1.22	236.91	1.21