面向人工智能和大数据的高效能计算

doi:10.11871/jfdc.issn.2096-742X.2020.01.003

数据与计算发展前沿 ›› 2020, Vol. 2 ›› Issue (1): 27-37.

doi: 10.11871/jfdc.issn.2096-742X.2020.01.003

所属专题： “高性能与高通量计算及应用”专刊

• 专刊:高性能与高通量计算及应用 • 上一篇下一篇

面向人工智能和大数据的高效能计算

李肯立^1,^2,^*(),阳王东^1,^*(),陈岑^1,²,陈建国¹,丁岩¹

1．湖南大学信息科学与工程学院,湖南长沙 410008
2．国家超级计算长沙中心,湖南长沙 410008

收稿日期:2019-10-28 出版日期:2020-02-20 发布日期:2020-03-28
通讯作者: 李肯立,阳王东
作者简介:李肯立,湖南大学信息科学与工程学院,教授,博士生导师,主要研究方向为高性能计算、并行计算、人工智能。
本文承担工作为：框架的整体结构设计、研究指导。
Li Kenli, Doctor, is a professor of School of Information Science and Engineering, Hunan University. His main research fields are high performance computing, parallel computing and artificial intelligence.
He undertakes the following tasks: the overall structure design and research guidance of the frame.|阳王东,湖南大学信息科学与工程学院,教授,博士生导师,主要研究方向为高性能计算、并行计算。
本文承担工作为：研究方向的凝练和论文的整合。
Yang Wangdong, Doctor, is a professor of School of Information Science and Engineering, Hunan University. His main research fields are high performance computing, parallel computing.
He undertakes the following tasks: the figure research direction out and the integration of papers.|陈岑,湖南大学信息科学与工程学院博士后,主要研究方向为大数据处理、并行计算与人工智能。
本文承担工作为：序言撰写和研究问题分析。
Chen Cen, post-doctoral researcher at the School of Information Science and Engineering, Hunan University, focuses on big data processing, parallel computing and artificial intelligence.
He undertakes the following tasks: preface writing and problem analysis.
E-mail: chencen@hnu.edu.cn|陈建国,湖南大学信息科学与工程学院,博士后,主要研究方向为大数据和人工智能。
本文承担工作为：框架的整体结构设计、研究指导。面向大数据和人工智能的高效能计算所面临的挑战分析。
Chen Jianguo, is a post-doctoral researcher at School of Information Science and Engineering, Hunan University. His major research areas include big data and artificial intelligence. He undertakes the following tasks: being the research director who is responsible for the design of the whole framework and analyzing the challenges of efficient computing for big data and artificial intelligence.
E-mail: jianguochen@hnu.edu.cn|丁岩,湖南大学信息科学与工程学院在读博士生,主要研究方向为边缘计算、数据挖掘。
本文承担工作为：深度神经网络模型剪枝与压缩方法调研。
Ding Yan, a PhD student at College of Information Science and Engineering, Hunan University. His research fields are edge computing and data mining.
He undertakes the following tasks: investigate on pruning and compression methods of deep neural network models.
E-mail: ding@hnu.edu.cn
基金资助:
国家重点研发计划(2018YFB1003401);国家杰出青年基金项目(61625202);国家自然科学基金项目(61572175);国家自然科学基金项目(61572175);国家自然科学基金项目(61751204);国家自然科学基金项目(61472124);国际交流合作项目(61860206011)

Efficient Computing for Artificial Intelligence and Big Data

Li kenli^1,^2,^*(),Yang Wangdong^1,^*(),Chen Cen^1,²,Chen Jianguo¹,Ding Yan¹

1．College of Information Science and Engineering, Hunan University, Changsha ,Hunan 410008, China
2．National Super-computer Center in Changsha, Changsha ,Hunan 410008, China

Received:2019-10-28 Online:2020-02-20 Published:2020-03-28
Contact: Li kenli,Yang Wangdong

摘要/Abstract

摘要：

[目的]本文主要分析人工智能和大数据应用随着迅速增大的数据规模,给计算机系统带来的主要挑战,并针对计算机系统的发展趋势给出了一些面向人工智能和大数据亟待解决的高效能计算的若干研究方向。[文献范围]本文广泛查阅国内外在超级计算和高性能计算平台进行大数据和人工智能计算的最新研究成果及解决的挑战性问题。[方法]大数据既为人工智能提供了日益丰富的训练数据集合,但也给计算机系统的算力提出了更高的要求。近年来我国超级计算机处于世界的前列,为大数据和人工智能的大规模应用提供了强有力的计算平台支撑。[结果]而目前以超级计算机为代表的高性能计算平台大多采用CPU+加速器构成的异构并行计算系统,其数量众多的计算核心能够为人工智能和大数据应用提供强大的计算能力。[局限性]由于体系结构复杂,在充分发挥计算能力和提高计算效率方面存在较大挑战。尤其针对有别于科学计算的人工智能和大数据领域,其并行计算效率的提升更为困难。[结论]因此需要从底层的资源管理、任务调度、以及基础算法设计、通信优化,到上层的模型并行化和并行编程等方面展开高效能计算的研究,全面提升人工智能和大数据应用在高性能计算平台上的计算能效。

关键词: 超级计算, 大数据, 高效能计算, 人工智能, 异构系统

Abstract:

[Objective] This paper mainly analyses the main challenges brought to computer system by the rapid increase of data scale of AI and big data application. In view of the development trend of computer system, some research directions of high-efficiency computing towards AI and big data are given. [Coverage] In this paper, the latest research results and challenges of big data and artificial intelligence computing on supercomputing and high performance computing platforms at home and abroad are extensively surveyed. [Methods] Big data not only provides an increasingly rich training data set for artificial intelligence, but also puts forward higher requirements for the computing power of computer systems. In recent years, China's supercomputer techniques are at the forefront of the world, which provides a powerful computing platform for large-scale applications of big data and artificial intelligence. [Results] At present, high-performance computing platforms represented by supercomputers mostly use heterogeneous parallel computing systems composed of CPUs and accelerators, where a large number of computing cores can provide powerful computing power for AI and big data applications. [Limitations]However, due to the complex architecture, there are major challenges in making full use of computing power and improving computing efficiency. The parallel computing efficiency is more difficult to improve, especially in the artificial intelligence and big data domains which are different from scientific computing. [Conclusions] Therefore, it is required to conduct research on high-performance computing from underlying resource management, task scheduling, basic algorithm design, and communication optimization to the upper level of model parallelization, so that the computational efficiency of artificial intelligence and big data applications on high-performance computing platforms can be improved.

Key words: artificial intelligence, big data, heterogeneous systems, high efficiency computing, supercomputing

李肯立,阳王东,陈岑,陈建国,丁岩. 面向人工智能和大数据的高效能计算[J]. 数据与计算发展前沿, 2020, 2(1): 27-37.

Li kenli,Yang Wangdong,Chen Cen,Chen Jianguo,Ding Yan. Efficient Computing for Artificial Intelligence and Big Data[J]. Frontiers of Data and Computing, 2020, 2(1): 27-37.

图/表 2

参考文献 59

[1]	John Walker S . Big data:A revolution that will transform how we live, work, and think[M]. 2014.
[2]	McAfee A, Brynjolfsson E, Davenport T H , et al. Big data: the management revolution[J]. Harvard business review, 2012,90(10):60-68.
[3]	Zhang Q, Yang L T, Chen Z , et al. A survey on deep learning for big data[J]. Information Fusion, 2018,42:146-157.
[4]	程学旗, 靳小龙, 王元卓, 郭嘉丰, 张铁赢, 李国杰 . 大数据系统和分析技术综述[J]. 软件学报, 2014,25(9):1889-1908.
[5]	O'Leary D E . Artificial intelligence and big data[J]. IEEE Intelligent Systems, 2013,28(2):96-99.
[6]	LeCun Y, Bengio Y, Hinton G . Deep learning[J]. nature, 2015,521(7553):436.
[7]	Deng J, Dong W, Socher R, et al. Imagenet: A large-scale hierarchical image database [C]//2009 IEEE conference on computer vision and pattern recognition. Ieee, 2009: 248-255.
[8]	Srivastava N, Salakhutdinov R R. Multimodal learning with deep boltzmann machines [C]//Advances in neural information processing systems. 2012: 2222-2230.
[9]	top500. . 2008.
[10]	Zhang Y, Sun J, Yuan G , et al. Perspectives of China’s HPC system development: a view from the 2009 China HPC TOP100 list[J]. Frontiers of Computer Science in China, 2010,4(4):437-444.
[11]	Zhang Y, Sun J, Yuan G , et al. A Brief Introduction to China HPC TOP100: from2002 to 2006[C]. In: Proc of Proceedings of the 2007 Asian technology informationprogram’s (ATIP’s) 3rd workshop on High performance computing in China: solutionapproaches to impediments for high performance computing. ACM, 2007, 32-36.
[12]	J.C. Chaves . Enabling High Productivity Computing through Virtualization[J]. Information Sciences, 2018,435:124-149.
[13]	李斌, 周清雷 , 等. 基于拟态计算的大数据高效能平台设计方法[J]. 计算机应用研究, 2019 ( 07):19-25.
[14]	祁琛 . 应用于神经网络的高效能计算单元的研究与实现[D]. 南京:东南大学, 2018.
[15]	D.H. Jones, A. Powell, C.-S. Bouganis, P.Y.K. Cheung. GPU Versus FPGA for High Productivity Computing [C]. IEEE International Conference on Field Programmable Logic and Applications (FPL). 2010 ( 06):112-119.
[16]	张小庆 . 高效能云计算虚拟机优化部署策略[J]. 计算机工程与应用, 2016 ( 04):28-36.
[17]	王永桂 . 流域大尺度水环境模型的高效能集群计算方法研究及其在三峡库区的应用[D]. 武汉: 武汉大学, 2015.
[18]	党林玉 . 可重构高效能计算系统中软硬件协同技术研究[D]. 解放军信息工程大学, 2014.
[19]	B. Betkaoui, D.B. Thomas, W. Luk. Comparing performance and energy efficiency of FPGAs and GPUs for high productivity computing [C]. IEEE International Conference on Field-Programmable Technology (FPT), 2010 ( 09):74-80.
[20]	阮利, 秦广军, 肖利民 , 等. 基于龙芯多核处理器的云计算节点机[J]. 通信学报, 2013(12):39-46.
[21]	刘勇鹏 . 大规模高效能计算的系统软件关键技术研究[D]. 国防科学技术大学, 2012.
[22]	J. Unpingco . User Friendly High Productivity Computational Workflows Using the VISION /HPC Prototype[C]. IEEE International Conference on High-performance Computing, 2018(03):93-105.
[23]	吴丹 . 高效能计算型存储器体系结构关键技术研究与实现[D]. 华中科技大学, 2012.
[24]	李波, 解建仓 , 等. 网格环境下的水利高性能计算平行系统及应用[J]. 华中科技大学学报, 2011 ( 06):73-82.
[25]	王之元 . 并行计算可扩展性分析与优化——能耗、可靠性与计算性能[D]. 国防科学技术大学, 2011.
[26]	Chu C, Kim S K, Lin Y, et al. Map-reduce for machine learning on multicore [C]//Proceedings of the 20 ^th Annual Conference on Neural Information Processing Systems, Amsterdam , 2007: 281-288.
[27]	Gao M, Pu J, Yang X, et al. TETRIS: Scalable and Efficient Neural Network Acceleration with 3D Memory [C] //Proceedings of the Twenty-Second International Conference on Architectural Support for Programming Languages and Operating Systems. ACM, 2017: 751-764.
[28]	Jin L, Wang Z, Gu R, et al. Training large scale deep neural networks on the intel xeon phi many- core coprocessor [C] //Proceedings of the International Parallel and Distributed Processing Symposium,Piscataway, 2014: 1622-1630.
[29]	Mei K, Dong P, Lei H , et al. A distributed approach for large-scale classifier training and image classification[J]. Neurocomputing, 2014,144:304-317.
[30]	杨柳, 景丽萍, 于剑 . 一种异构直推式迁移学习算法[J]. 软件学报, 2015,26(11):2762-2780.
[31]	王岳青, 窦勇, 吕启, 李宝峰, 李腾 . DLPF:基于异构体系结构的并行深度学习编程框架[J]. 计算机研究与发展, 2016,53(06):1202-1210.
[32]	洪文杰, 李肯立, 全哲, 阳王东, 李克勤, 郝子宇, 谢向辉 . 面向神威·太湖之光的PETSc 可扩展异构并行算法及其性能优化[J]. 计算机学报, 2017,40(9):1961-1973.
[33]	Chen C, Li K, Ouyang A , et al. GPU-Accelerated Parallel Hierarchical Extreme Learning Machine on Flink for Big Data[J]. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2017: 1-14.
[34]	Xing E P, Ho Q, Dai W , et al. Petuum: A new platform for distributed machine learning on big data[J]. IEEE Transactions on Big Data, 2015,1(2):49-67.
[35]	Z. Liu, M. Sun, T. Zhou, G. Huang, T. Darrell . Rethinking the Value of Network Pruning. 2018.
[36]	S. Han, H. Mao, W. J. Dally . A Deep Neural Network Compression Pipeline: Pruning, Quantization, Huffman Encoding. 2015.
[37]	Y. L. Cun, J. S. Denker, S. A. Solla . Optimal brain damage[J]. Advances in Neural Information Processing Systems, Vol. 2, No. 1, 1990, p. 1990.
[38]	B. Hassibi, D. G. Stork . Second Order Derivatives for Network Pruning: Optimal Brain Surgeon[J]. Advances in Neural Information Processing Systems, vol. 5, pp. 164-- 171, 1993.
[39]	S. Han, J. Pool, J. Tran, W. J. Dally . Learning both Weights and Connections for Efficient Neural Networks. 2015.
[40]	L. Hao, A. Kadav, I. Durdanovic, H. Samet, H. P. Graf . Pruning Filters for Efficient ConvNets. 2016.
[41]	P. Molchanov, S. Tyree, T. Karras, T. Aila, J. Kautz . Pruning Convolutional Neural Networks for Resource Efficient Transfer Learning. 2016.
[42]	J. H. Luo, J. Wu, W. Lin . ThiNet: A Filter Level Pruning Method for Deep Neural Network Compression. 2017.
[43]	R. Yu, A. Li, C. F. Chen, J. H. Lai, V. I. Morariu, X. Han, M. Gao, C. Y. Lin, L. S. Davis . NISP: Pruning Networks using Neuron Importance Score Propagation. 2017.
[44]	Y. He, X. Zhang, S. Jian . Channel Pruning for Accelerating Very Deep Neural Networks. 2017.
[45]	H. Song, X. Liu, H. Mao, P. Jing, A. Pedram, M. A. Horowitz, W. J. Dally . EIE: Efficient Inference Engine on Compressed Deep Neural Network. Acm Sigarch Computer Architecture News, vol. 44, no. 3, pp. 243-254, 2016.
[46]	X. Gao, Y. Zhao, L. Dudziak, R. Mullins, C. Z. Xu . Dynamic Channel Pruning: Feature Boosting and Suppression. 2018.
[47]	E. P. Yong-Deok Kim, Sungjoo Yoo, Taelim Choi, Lu Yang, and Dongjun Shin .COMPRESSION OF DEEP CONVOLUTIONAL NEURAL NETWORKS FOR FAST AND LOW POWER MOBILE APPLICATIONS. in ICLR, 2016.
[48]	E. Denton, W. Zaremba, J. Bruna, Y. Lecun, R. Fergus . Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation. 2014.
[49]	T. Cheng, X. Tong, Z. Yi, X. Wang, E. Weinan . Convolutional neural networks with low-rank regularization. Computer Science, 2016.
[50]	H. Y., Y. Z., J. Liu . End-to-End Learning of Energy-Constrained Deep Neural Networks. in IDLR, 2019.
[51]	T. J. Yang, Y. H. Chen, V. Sze . Designing Energy-Efficient Convolutional Neural Networks Using Energy-Aware Pruning. 2017.
[52]	K. Simonyan, A. Zisserman . Very Deep Convolutional Networks for Large-Scale Image Recognition[J]. Computer Science, 2014.
[53]	M. Denil, B. Shakibi, L. Dinh, M. A. Ranzato, N. D. Freitas . Predicting Parameters in Deep Learning.
[54]	M. Mathieu, M. Henaff, Y. Lecun . Fast Training of Convolutional Networks through FFTs. Eprint Arxiv, 2013.
[55]	Y. Gong, L. Liu, Y. Ming, L. Bourdev . Compressing Deep Convolutional Networks using Vector Quantization[J]. Computer Science, 2014.
[56]	W. Chen, J. T. Wilson, S. Tyree, K. Q. Weinberger, Y. Chen . Compressing Neural Networks with the Hashing Trick[J].Computer Science, pp. 2285-2294, 2015.
[57]	Y. Cheng, F. X. Yu, R. S. Feris, S. Kumar, A. Choudhary, S. F. Chang . Fast Neural Networks with Circulant Projections. 2015.
[58]	A. Novikov, D. Podoprikhin, A. Osokin, D. Vetrov . Tensorizing Neural Networks. 2015.
[59]	X. Zhang, J. Zou, M. Xiang, K. He, S. Jian . Efficient and Accurate Approximations of Nonlinear Convolutional Networks. 2014.

面向人工智能和大数据的高效能计算

Efficient Computing for Artificial Intelligence and Big Data

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

图/表 2

参考文献 59

相关文章 15

编辑推荐

Metrics

本文评价

[1]	周成祖, 吴文, 蔡晓强. 基于分类分级的数据安全防控策略研究[J]. 数据与计算发展前沿, 2023, 5(1): 128-135.
[2]	许淞源,刘峰. ESDRec：一种面向地球大数据平台的数据推荐模型[J]. 数据与计算发展前沿, 2023, 5(1): 55-64.
[3]	金天骄,栗蔚. 基于算力网络的大数据计算资源智能调度分配方法[J]. 数据与计算发展前沿, 2022, 4(6): 29-37.
[4]	刘嘉琪,杨斌艳. 我国人工智能与社会科学耦合发展的热点与趋势研究——基于CiteSpace的文献计量分析[J]. 数据与计算发展前沿, 2022, 4(6): 77-91.
[5]	季明辰,任勇毛,张运栋,周慧娟,周旭,周艳芳. 面向交通大数据的高速文件传输系统设计与实现[J]. 数据与计算发展前沿, 2022, 4(3): 141-151.
[6]	胡庆宝,郑伟,王佳荣,汪璐,颜田. 高能物理科学数据中心智能运维系统[J]. 数据与计算发展前沿, 2022, 4(1): 30-41.
[7]	陈文杰,胡正银,胡靖,庞弘燊,何雨娟. 多维数据驱动的粮食安全分析与智能决策系统研究与实践[J]. 数据与计算发展前沿, 2021, 3(6): 1-14.
[8]	鹿旭东,宋伟凤,郭伟,崔立真,林岳,姜涛. 大数据驱动的创新方法论与创新服务平台[J]. 数据与计算发展前沿, 2021, 3(5): 141-155.
[9]	张润滋,刘文懋. AISecOps智能安全运营技术体系框架[J]. 数据与计算发展前沿, 2021, 3(3): 32-47.
[10]	袁勇,欧阳丽炜,王晓,王飞跃. 基于区块链的智能组件：一种分布式人工智能研究新范式[J]. 数据与计算发展前沿, 2021, 3(1): 1-14.
[11]	张婕,郭印. 基于大数据语言实验平台的隐私安全研究[J]. 数据与计算发展前沿, 2020, 2(6): 90-102.
[12]	莫梓嘉,高志鹏,苗东. 边缘智能：人工智能向边缘分布式拓展的新触角[J]. 数据与计算发展前沿, 2020, 2(4): 16-27.
[13]	喻鹏,李文璟,丰雷,周凡钦,杨杨,邱雪松. 面向未来6G网络的智能管控架构与关键技术[J]. 数据与计算发展前沿, 2020, 2(3): 32-44.
[14]	陈梅丽,马英克,李茹姣,鲍一明. 基因组学数据分析方法现状和展望[J]. 数据与计算发展前沿, 2020, 2(2): 1-19.
[15]	王文生,郭雷风. 大数据技术农业应用[J]. 数据与计算发展前沿, 2020, 2(2): 101-110.