Application-Aware Method for Optimized Computing Power Scheduling

doi:10.11871/jfdc.issn.2096-742X.2022.05.001

Frontiers of Data and Computing ›› 2022, Vol. 4 ›› Issue (5): 3-10.

CSTR: 32002.14.jfdc.CN10-1649/TP.2022.05.001

doi: 10.11871/jfdc.issn.2096-742X.2022.05.001

• Special Issue: East-to-West Computing Requirement Transfer: A Century Project to Start the Era of Computility Economy • Previous Articles Next Articles

Application-Aware Method for Optimized Computing Power Scheduling

KOU Dazhi^1,^*(),WEI Jianwen²,TANG Xiaoyong³

1. Shanghai Supercomputer Center, Shanghai 201203, China
2. Center for High Performance Computing, Shanghai Jiao Tong University, Shanghai 200240, China
3. School of Computer and Communication Engineering, Changsha University of Science and Technology, Changsha, Hunan 410114, China

Received:2022-07-11 Online:2022-10-20 Published:2022-10-27
Contact: KOU Dazhi E-mail:dzkou@ssc.net.cn

Abstract

Abstract:

[Objective] Under the background of the project of “East-West Computing Requirement Transfer”, the super-computing resources distributed in different regions will be scheduled and managed. In order to avoid the problem of busy and unevenly distribution of computing resources, it is necessary to develop a multi-center task scheduling system by investigating the runtime characteristics of typical applications to achieve unified management of the national high-performance computing environment. [Methods] Firstly, the log data about application execution at several national supercomputing centers are collected and the database for the application log data is established. Secondly, by taking the user resources demand and the resource usage characteristics of typical applications into consideration, a machine learning framework is established to accurately depict the application execution features. Then migration of HPC applications across clusters using containers is implemented. Finally, a task scheduling system based on application-aware resource scheduling optimization is developed. [Results] This system provides powerful technical support for services and efficient operation of the national high-performance computing environment. [Conclusions] The application-aware method for computing power scheduling optimization is expected to effectively improve the reliability, availability, and maintainability of the “East-West Computing Requirement Transfer” project.

Key words: High Performance Computing system, historical database, application feature, computing power scheduling method

KOU Dazhi, WEI Jianwen, TANG Xiaoyong. Application-Aware Method for Optimized Computing Power Scheduling[J]. Frontiers of Data and Computing, 2022, 4(5): 3-10, https://cstr.cn/32002.14.jfdc.CN10-1649/TP.2022.05.001.

Figures/Tables 8

Fig.1

Fig.2

Fig.3

Fig.4

Fig.5

Fig.6

Fig.7

Fig.8

References 13

[1]	LEFF A, RAYFIELD J T, DIAS D M. Service-level agre-ements and commercial grids[J]. IEEE Internet Comput-ing, 2003, 7(4): 44-50.
[2]	王娟. Backfilling 算法介绍及其在集群中的应用分析[D]. 北京: 北京邮电大学, 2008.
[3]	Talby D, Feitelson D G. Supporting priorities and imp-roving utilization of the IBM SP scheduler using slack-based backfilling[C]// Proceedings 13th International Parallel Processing Symposium and 10th Symposium on Parallel and Distributed Processing, IPPS/SPDP 1999. IEEE, 1999: 513-517.
[4]	Srinivasan S, Kettimuthu R, Subramani V, et al. Selective reservation strategies for backfill job scheduling[C]// Workshop on Job Scheduling Strategies for Parallel Pro-cessing. Springer, Berlin, Heidelberg, 2002: 55-71.
[5]	Ward W A, Mahood C L, West J E. Scheduling jobs on parallel systems using a relaxed backfill strategy[C]// Workshop on Job Scheduling Strategies for Parallel Proc-essing, Springer, Berlin, Heidelberg, 2002: 88-102.
[6]	LeCun Y., Bengio Y., Hinton G. Deep learning[J]. Nature, 2015, 521(7553):436-444. doi: 10.1038/nature14539
[7]	Himanen L, Jäger MO, Morooka EV, Canova FF, Ran-awat YS, Gao DZ, Rinke P, Foster AS. DScribe: Lib-rary of descriptors for machine learning in materials science[J]. Computer Physics Communications, 2020, 247:106949. doi: 10.1016/j.cpc.2019.106949
[8]	Liu C, Li K, Li K. A game approach to multi-servers load balancing with load-dependent server availability consideration[J]. IEEE Transactions on Cloud Com-puting, 2021, 9 (1) : 1-13.
[9]	Liu C, Li K, Tang Z, et al. Bargaining game-based sched-uling for performance guarantees in cloud computing[J]. ACM Transactions on Modeling and Performance Evaluation of Computing Systems (TOMPECS), 2018, 3(1): 1-25.
[10]	Bellavista P, Cinque M, Corradi A, et al. GAMESH: A grid architecture for scalable monitoring and enhanced dependable job scheduling[J]. Future Generation Com-puter Systems, 2017, 71: 192-201.
[11]	Hu M, Veeravalli B. Requirement-aware scheduling of bag-of-tasks applications on grids with dynamic resilience[J]. IEEE Transactions on Computers, 2013, 62(10): 2108-2114. doi: 10.1109/TC.2012.164
[12]	Wyatt M, Herbein S, Ahn D, et al. Unstructured data analytics for next-generation HPC scheduler: capturing jobs’ needs from unstructured job scripts[R]. Lawrence Livermore National Laboratory(LLNL), Livermore, CA(United States), 2017.
[13]	Cunha R L F, Rodrigues E R, Tizzei L P, et al. Job place-ment advisor based on turnaround predictions for HPC hybird clouds[J]. Future Generation Computer Systems, 2017, 67:35-46. doi: 10.1016/j.future.2016.08.010

Application-Aware Method for Optimized Computing Power Scheduling

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 8

References 13

Related Articles 0

Recommended Articles

Metrics

Comments