A Job Duration Prediction Method for Computing Clusters Based on Ensemble Machine Learning

doi:10.11871/jfdc.issn.2096-742X.2023.02.013

Abstract

Abstract:

[Objective] This paper presents a new method to improve the accuracy of job duration prediction, increase the performance of job backfill scheduling, and improve the utilization of computing resources of computing clusters. [Context] Job scheduling plays an essential role in improving the utilization of computing resources in supercomputing clusters, and job duration prediction is the key judgment basis for job backfill scheduling strategy. [Methods] Based on the ensemble learning method, this paper predicts the job duration of the computing cluster, and integrates algorithms such as support vector regression, random forest, gradient lifting regression tree, and automatic machine learning. The job duration predicted by this method is used to conduct the job backfill scheduling experiment. [Results] The method is tested on three typical data sets: HPC2N, CEA Curie and KIT FH2. The root mean square error of the predicted value of operation duration is reduced by 60.30%, 51.91%, and 63.51% respectively compared with the user prediction. Compared with the linear regression method, the root mean square error of the predicted job duration is reduced by 44.37%, 31.98%, and 52.69%, respectively. [Conclusions] The experimental results show that this method can greatly improve the accuracy of job duration prediction. The average job waiting time is reduced by 9.07%, 8.80%, and 1.83% respectively compared with the user prediction, and the performance of job backfill scheduling is improved. The average bounded deceleration value is reduced by 7.72%, 0.96%, and 9.05% respectively compared with the user prediction, improving the utilization of computing resources of supercomputing clusters.

Key words: ensemble learning, supercomputing cluster, job duration prediction, backfill scheduling strategy

LI He,XIU Hanwen,LIU Yanjun,CAO Rongqiang,ZHOU Chunbao,WANG Yangang. A Job Duration Prediction Method for Computing Clusters Based on Ensemble Machine Learning[J]. Frontiers of Data and Computing, 2023, 5(2): 164-174, https://cstr.cn/32002.14.jfdc.CN10-1649/TP.2023.02.013.

Figures/Tables 10

Fig.1

Table 1

Table 2

Job duration prediction results of three supercom-puting clusters"

计算集群	方法	$R 2$	MAE	RMSE
CEA Curie	日志数据	1	0	0
CEA Curie	本论文	0.73	1201	1577
CEA Curie	用户预估	0.49	3365	3972
CEA Curie	线性回归	0.67	2973	2835
HPC2N	日志数据	1	0	0
HPC2N	本论文	0.74	1201	1893
HPC2N	用户预估	0.52	3867	3936
HPC2N	线性回归	0.69	2983	2783
KIT FH2	日志数据	1	0	0
KIT FH2	本论文	0.69	3201	1425
KIT FH2	用户预估	0.43	3492	3905
KIT FH2	线性回归	0.51	3037	3012

Table 2

Table 3

Fig.2

Fig.3

Fig.4

Table 4

Table 5

Fig.5

References 14

[1]	戴霖. 基于应用的高性能计算平台作业调度算法研究[J]. 电声技术, 2019,(5):52-53.
[2]	Wang, Xiebing, Huang, Kai, Knoll, Alois, Qian, Xue-hai. A Hybrid Framework for Fast and Accurate GPU Performance Estimation through Source-Level Analysis and Trace-Based Simulation[C]. In 2019 IEEE Internati-onal Symposium on High Performance Computer Arch-itecture (HPCA), 2009: 506-518.
[3]	Amaris Gonzalez, Marcos Dyab, Mohamed, Trystram, Denis, Camargo, Raphael, Goldman, Alfredo. A Com-parison of GPU Execution Time Prediction using Mach-ine Learning and Analytical Modeling[C]. In 15th IEEE International Symposium on Network Computing and Applications, 2016:326-333.
[4]	Lublin U, Feitelson D G. The workload on parallel sup-ercomputers: modeling the characteristics of rigid jobs[J]. Journal of Parallel and Distributed Computing, 2003, 63(11): 1105-1122. doi: 10.1016/S0743-7315(03)00108-4
[5]	Feitelson D G and Rudolph L. Job Scheduling Strategies for Parallel Processing[C]. In Berlin Heidelberg: Sprin-ger-Verlag, 2005: 176-193.
[6]	Guo, Jian & Nomura, Akihiro & Barton, Ryan & Haoyu, Zhang & Matsuoka, Satoshi. Erratum to: Machine Lear-ning Predictions for Underestimation of Job Runtime on HPC System[C]. In Supercomputing Frontiers, 2018. E1-E2.
[7]	Pinel, Frédéric & Yin, Jian-xiong & Hundt, Christian & Kieffer, Emmanuel & Varrette, Sébastien & Bouvry, Pascal & See, Simon. Evolving a Deep Neural Network Training Time Estimator[J]. In Communications in Com-puter and Information, 2020, 1173:13-24.
[8]	Rezaei, Mahdi & Salnikov, Alexey. Machine Learning Techniques to Perform Predictive Analytics of Task Qu-eues Guided by Slurm[J]. In Global Smart Industry Con-ference (GloSIC), 2018,1-6.
[9]	吴桂宝, 沈瑜, 张文帅, et al. Runtime Prediction of Jobs for Backfilling Optimization面向回填优化的作业时长预测[J]. 小型微型计算机系统, 2019, 040(001):6-12.
[10]	Wang, Qiqi & Li, Jing & Wang, Shuo & Wu, Guibao. A Novel Two-Step Job Runtime Estimation Method Based on Input Parameters in HPC System[J]. In Carbohydrate Polymers, 2021, 273:311-316.
[11]	许伦凡. 基于历史数据的作业时间预测[D]. 北京: 中国工程物理研究院, 2019.
[12]	Olson R S, Moore J H. TPOT: A Tree-Based Pipeline Optimization Tool for Automating Machine Learning[C]. Proceedings of the Workshop on Automatic Machine Learning, 2016, 64:66-74.
[13]	Swami A, Jain R. Scikit-learn: Machine Learning in Python[J]. Journal of Machine Learning Research, 2013, 12(10):2825-2830.
[14]	Feitelson D G, Tsafrir D, Krakov D. Experience with using the Parallel Workloads Archive[J]. Journal of Para-llel and Distributed Computing, 2014, 74:2967-2982.

日志名称	时间	作业数量
HPC2N	2002.07-2006.01	52,371
CEA Curie	2011.02-2012.10	773,138
KIT FH2	2016.06-2018.01	114,355

计算集群	原来	本方法	降低率
HPC2N	179.31	163.04	9.07%
KIT FH2	172.03	156.89	8.80%
CEA Curie	183.89	180.53	1.83%

计算集群	原来	本方法	降低率
HPC2N	1201.31	1108.59	7.72%
KIT FH2	1385.39	1372.06	0.96%
CEA Curie	9607.67	8737.82	9.05%