深度学习技术在学科融合研究中的应用

doi:10.11871/jfdc.issn.2096-742X.2020.05.010

数据与计算发展前沿 ›› 2020, Vol. 2 ›› Issue (5): 99-109.

doi: 10.11871/jfdc.issn.2096-742X.2020.05.010

深度学习技术在学科融合研究中的应用

刘晓东^1,^*(),倪浩然²()

1.中国科学院计算机网络信息中心,信息化战略发展与评估中心,北京 100190
2.华威大学,真实系统数学,英国考文垂,CV4 7HP

收稿日期:2020-07-02 出版日期:2020-10-20 发布日期:2020-10-30
通讯作者: 刘晓东
作者简介:Liu Xiaodong is an engineer of CNIC. His research interests are data science and artificial intelligence.
In this paper, he is mainly responsible for the data organization, experimental design and experimental result presentations.
E-mail: liuxiaodong@cnic.cn|Ni Haoran is a master student (leading to Ph.D.) at the Mathematics of Systems CDT at Warwick University. His research interests are distributed at Numerical Analysis, Natural Language Processing, Machine Learning Algorithms and Optimal Transports.
In this paper, he is mainly responsible for the experiments and the analysis of experimental results.
E-mail: Haoran.ni@warwick.ac.uk

Application of Deep Learning Technology in Discipline Integration Research

Liu Xiaodong^1,^*(),Ni Haoran²()

1. Center of Informatization Strategy and Evaluation, Computer Network Information Center, Chinese Academy of Sciences, Beijing 100190, China
2. Mathematics for Real-World Systems CDT, University of Warwick, Coventry CV4 7HP, United Kingdom

Received:2020-07-02 Online:2020-10-20 Published:2020-10-30
Contact: Liu Xiaodong

摘要/Abstract

摘要：

【目的】 我们使用深度学习模型对于文章进行多分类,研究论文发表机构的学科融合的科研现状。 【方法】 我们设计了“多类别分类”模型,并应用卷积神经网络对中国科学院产生的8个不同主题的研究论文摘要进行分类。 【结果】 结果表明,科学研究涉及的学科交叉融合变得日趋紧密。 【结论】 多学科的融合交叉促进了科研产出,该研究可进一步用于科研机构的战略规划部署和评价等问题。

关键词: 文本分类, 自然语言处理, 卷积神经网络, 分类算法

Abstract:

[Objective] We use deep learning models to multi-classify articles and analyze the disciplinary integration situation of the corresponding institutions. [Methods] In this paper, we design a one-versus-rest classification model and applied convolutional neural networks to categorize paper abstracts of 8 different main subjects produced by Chinese Academy of the Sciences. [Results] The results show that the cross-integration of disciplines involved in scientific research becomes a more frequent practice and the integration of academic fields are promoting the number of publications of scientific research papers. [Conclusions] This research can benefit the strategic planning and deployment for scientific research institutions.

Key words: text classification, natural language processing, convolutional neural network, classification algorithm

刘晓东,倪浩然. 深度学习技术在学科融合研究中的应用[J]. 数据与计算发展前沿, 2020, 2(5): 99-109.

Liu Xiaodong,Ni Haoran. Application of Deep Learning Technology in Discipline Integration Research[J]. Frontiers of Data and Computing, 2020, 2(5): 99-109.

图/表 3

参考文献 31

[1]	Bing L & Lei Z. A Survey of Opinion Mining and Sentiment Analysis[M] . Boston, MA: Springer US, 2012.
[2]	Aggarwal C & Zhai C, A Survey of Text Classification Algorithms[M] . Boston, MA: Springer US, 2012.
[3]	Zhang Y, Jin R & Zhou Z, Understanding bag-of-words model: a statistical framework[J] . International Journal of Machine Learning & Cybernetics, 2010,1(1-4):43-52.
[4]	Post M & Bergsma S. Explicit and implicit syntactic features for text classification[J] . In Proceedings of the 51st annual meeting of the association for computational linguistics, 2013: volume 2: Short papers, 866-872.
[5]	Mikolov T, Chen K & Corrado G, et al. fficient estimation of word representations in vector space [J/OL]. arXiv preprint arXiv, 2013,1301. 3781.
[6]	Bengio Y, Ducharme R & Vincent P, et al. A neural probabilistic language model[J]. Journal of machine learning research , 2003: 1137-1155.
[7]	Rong X. word2vec Parameter Learning Explained [J/OL]. Computer Science 2014,1411_2738.
[8]	Mikolov T, Sutskever I & Chen K, et al. istributed Representations of Words and Phrases and Their Compositionality [J/OL]. arXiv preprint arXiv, 2013,1310. 4546.
[9]	Pennington J, Socher R & Manning C. D., GloVe: Global Vectors for Word Representation [C]. Empirical methods in natural language processing, 20141532-1543. http:// www.aclweb.Org /anthology/D14-1162.
[10]	Kalchbrenner N., Grefenstette E & Blunsom P., A Convolutional Neural Network for Modelling Sentences[C]. Association for Computational Linguistics (ACL), 2014.
[11]	Lecun Y, Bottou L & Bengio Y, et al. Gradient-based learning applied to document recognition[C]. Proceedings of the IEEE (1998), 86(11):2278-2324. doi: 10.1109/5.726791.
[12]	Deerwester S, T. Dumais S & Landauer T, et al. Indexing by latent semantic analysis[J] . Journal of the American Society for Information Science, 1990,41:391-407. doi: 10.1002/(sici)1097-4571(199009)41:660;391::aid-asi162;3.0.co;2-9;391::aid-asi162;3.0.co;2-9. doi: 10.1002/(ISSN)1097-4571
[13]	Devlin J, Chang M, Lee K, et al. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding [J/OL]. arXiv preprint arXiv, 2018,1810_04805.
[14]	Ng A. Y.. Feature selection, L1 vs. L2 regularization, and rotational invariance[C] . Proceedings of the twenty-first international conference on Machine learning, 2004.
[15]	Socher R., Huang E & Pennington J. et al. Dynamic Pooling and Unfolding Recursive Autoencoders for Paraphrase Detection[J] . Advances in neural information processing systems, 2011: 24. pmid: 25152607
[16]	Socher R, Pennington J & Huang E, et al. PSemi-supervised recursive autoencoders for predicting sentiment distributions[C] . Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, 2011.
[17]	Socher R, Perelygin A & Wu J, et al. Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank[C] . Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, 2013.
[18]	Elman J. Finding structure in time[C] . Cognitive Science, 1990,14(2):179-211. https://onlinelibrary. wiley. com/doi/abs/10.1207/s15516709cog1402_1doi:10.1207/s15516709cog1402\_1 doi: 10.1207/s15516709cog1402_1
[19]	Liu P, Qiu X & Huang X, Recurrent Neural Network for Text Classification with Multi-Task Learning [J/OL]. arXiv preprint arXiv, 2016,1605_05101.
[20]	Ghosh S, Vinyals O, Strope B, et al. Contextual LSTM (CLSTM) models for Large scale NLP tasks [J/OL]. arXiv preprint arXiv, 2016,1602_06291.
[21]	Lai S, Xu L & Liu K, et al. Recurrent Convolutional Neural Networks for Text Classification[C] . Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence, 2015.
[22]	Zhang Y & Wallace B, A Sensitivity Analysis of (and Practitioners' Guide to) Convolutional Neural Networks for Sentence Classification [J/OL]. arXiv preprint arXiv, 2015,1510_03820.
[23]	Snoek J, Larochelle H & Adams R. P, Practical Bayesian Optimization of Machine Learning Algorithms[J/OL]. Advances in Neural Information Processing Systems 25, 2012.
[24]	Bergstra J, Bardenet R, Bengio Y, et al. Algorithms for Hyper-Parameter Optimization[C] . Advances in Neural Information Processing Systems, 2011.
[25]	Bergstra J & Cox D. Making a Science of Model Search: Hyperparameter Optimization in Hundreds of Dimensions for Vision Architectures[C]. Proceedings of the 30th International Conference on Machine Learning 2013: 115-123.
[26]	Gulcehre C, Moczulski M, Denil M, et al. Noisy Activation Functions[C] . International Conference on Machine Learning, 2016.
[27]	Glorot X & Bengio Y, Understanding the difficulty of training deep feedforward neural networks[C] . Proceedings of the 13th International Conference on Artificial Intelligence and Statistics, 2010,9, 249-256.
[28]	Nair V & Hinton G, E. Rectified Linear Units Improve Restricted Boltzmann Machines[C]. Proceedings of the 27th International Conference on Machine Learning, 2010.
[29]	Glorot X, Bordes A & Bengio Y, Deep Sparse Rectifier Neural Networks[C]. Proceedings of the 14th International Conference on Artificial Intelligence and Statistics, 2011: 15.
[30]	Goodfellow J, Warde-Farley D & Mirza M, et al. Maxout Networks[C]. Proceedings of the 30 th International Conference on Machine Learning, 2013(3)Vol.28:1319-1327. http://dblp.uni-trier.de/db/conf/icml/icml2013.html#GoodfellowWMCB13.
[31]	Collobert R, Weston J & Bottou L, et al. Natural Language Processing (Almost) from Scratch[J] . Journal of Machine Learning Research, 2011,12(1):2493-2537.

深度学习技术在学科融合研究中的应用

Application of Deep Learning Technology in Discipline Integration Research

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

图/表 3

参考文献 31

相关文章 8

编辑推荐

Metrics

本文评价

[1]	石雪梅,朱克亮,张祥民,张树涛,陈良锋. 基于生成对抗网络的有遮挡人脸修复方法[J]. 数据与计算发展前沿, 2022, 4(4): 123-131.
[2]	李东闻,钟震宇,申峻宇,王昊天,孙羽菲,张玉志. NKCorpus：利用海量网络数据构建大型高质量中文数据集[J]. 数据与计算发展前沿, 2022, 4(3): 30-45.
[3]	兰格,王瑾瑜,孙羽菲,张玉志. 基于知识图谱的图匹配文本分类[J]. 数据与计算发展前沿, 2022, 4(2): 39-49.
[4]	肖楠,周明珠,邢军,罗泽,李晓辉. 基于高分辨率网络和注意力机制的真伪卷烟包装鉴别[J]. 数据与计算发展前沿, 2021, 3(5): 118-129.
[5]	张猛,李健. 鸟类音频数据预处理方法[J]. 数据与计算发展前沿, 2021, 3(5): 130-140.
[6]	雷声,黎建辉,张丽丽. 基于无监督学习的可持续发展目标数据分类[J]. 数据与计算发展前沿, 2021, 3(4): 104-115.
[7]	陈子健,李俊,岳兆娟,赵泽方. 基于自编码器与属性信息的混合推荐模型[J]. 数据与计算发展前沿, 2021, 3(3): 148-155.
[8]	欧阳与点,谢鲲. 网络性能数据恢复算法[J]. 数据与计算发展前沿, 2020, 2(3): 55-65.