融合情感词典和自注意力的双通道情感分析模型

doi:10.11871/jfdc.issn.2096-742X.2023.04.009

数据与计算发展前沿 ›› 2023, Vol. 5 ›› Issue (4): 101-111.

CSTR: 32002.14.jfdc.CN10-1649/TP.2023.04.009

doi: 10.11871/jfdc.issn.2096-742X.2023.04.009

融合情感词典和自注意力的双通道情感分析模型

巨家骥(),黄勃^*(),张帅,郭茹燕

上海工程技术大学，电子电气工程学院，上海 201620

收稿日期:2022-02-28 出版日期:2023-08-20 发布日期:2023-08-23
通讯作者: *黄勃（E-mail: huangbosues@sues.edu.cn）
作者简介:巨家骥，上海工程技术大学电子电气工程学院，硕士研究生，主要研究方向为自然语言处理、情感分析、推荐系统。
本文中负责提供研究思路，对设计模型进行实验，论文撰写。
JU Jiaji is a postgraduate student at the School of Electrical and Electronic Engineering, Shanghai University of Engineering and Technology. His main research interests are natural language processing, sentiment analysis, and recommendation systems.
In this paper, he is responsible for proposing research ideas, conducting experiments on the designed model, and writing the paper.
E-mail: 17621930036@163.com|黄勃，上海工程技术大学电子电气学院，副教授，硕士生导师，主要研究方向为软件工程、人工智能、大数据、自然语言处理等。
本文中负责设计研究方案和框架，论文最终版本修订。
HUANG Bo is currently an associate professor in the School of Electronic and Electrical Engineering, Shanghai University of Engineering Science. His main research interests include software engineering, artificial intelligence, big data, and natural language processing.
In this paper, he is responsible for the design of the research protocol and framework and the revision of the final version of the paper.
E-mail: huangbosues@sues.edu.cn
基金资助:
国家重点研发计划(2020AAA0109300);上海市科委科技创新行动计划(22S31903700);上海市科委科技创新行动计划(21S31904200)

A Dual-Channel Sentiment Analysis Model Integrating Sentiment Lexcion and Self-Attention

JU Jiaji(),HUANG Bo^*(),ZHANG Shuai,GUO Ruyan

School of Electrical and Electronic Engineering, Shanghai University of Engineering and Technology, Shanghai 201620, China

Received:2022-02-28 Online:2023-08-20 Published:2023-08-23

摘要/Abstract

摘要：

【应用背景】针对自然语言处理中的情感分析任务，目前的深度学习方法还是通过大量的数据训练来逐步提升效果，并没有充分利用文本中的情感词信息。【方法】本文提出了一种集成了情感词典和注意力机制的双通道文本情感分析模型。基于自注意力机制的通道负责提取语义特征，基于情感注意力的通道负责提取情感特征，两个通道分别提取的特征融合后获得文本最终的向量表达。同时引入一种注意力软约束来平衡两个通道中的注意力。【结果】实验结果表明，双通道的结构能够分别关注文本的不同特征，语义特征和情感特征结合起来有效提升了模型的分类性能。由于集成了情感词典，模型还具有较好的可解释性。【结论】本文提出的情感分析模型与相关模型相比拥有较好的性能和可解释性。

关键词: 深度学习, 情感词典, 文本情感分析, 双通道, 注意力

Abstract:

[Application Background] For the task of sentiment analysis in natural language processing, current deep learning methods are based on big data training to gradually improve the effect, and do not fully exploit the information of sentiment words in the text. [Methods] This paper proposes a dual-channel text sentiment analysis model that integrates sentiment lexicon and attention mechanism. The channel based on self-attention is responsible for extracting semantic features, and the channel based on emotional attention is to extract emotional features, and the features extracted by the two channels are fused to obtain the final vector representation of the text. A soft constraint of attention is also introduced to balance the attention in both channels. [Results] The experimental results show that the dual-channel structure can focus on different features of the text separately, and the combination of semantic and sentiment features significantly improves the sentiment classification performance of the model. The model also has better interpretability due to the integration of the sentiment lexicon. [Conclusions] The sentiment analysis model proposed in this paper has better performance and interpretability compared with related models.

Key words: deep learning, lexicon, text sentiment analysis, dual channel, attention

巨家骥, 黄勃, 张帅, 郭茹燕. 融合情感词典和自注意力的双通道情感分析模型[J]. 数据与计算发展前沿, 2023, 5(4): 101-111.

JU Jiaji, HUANG Bo, ZHANG Shuai, GUO Ruyan. A Dual-Channel Sentiment Analysis Model Integrating Sentiment Lexcion and Self-Attention[J]. Frontiers of Data and Computing, 2023, 5(4): 101-111, https://cstr.cn/32002.14.jfdc.CN10-1649/TP.2023.04.009.

图/表 9

图1

图2

图3

表1

表2

表3

图4

图5

表4

参考文献 33

[1]	谢润忠, 李烨. 基于 BERT 和双通道注意力的文本情感分类模型[J]. 数据采集与处理, 2020, 35(4): 642-652.
[2]	王婷, 杨文忠. 文本情感分析方法研究综述[J]. 计算机工程与应用, 2021, 57(12): 11-24. doi: 10.3778/j.issn.1002-8331.2101-0022
[3]	Pang B, Lee L. Opinion mining and sentiment analysis[J]. Foundations and Trends in Information Retrieval, 2008, 2(1-2): 1-135. doi: 10.1561/1500000011
[4]	赵妍妍, 秦兵, 石秋慧, 等. 大规模情感词典的构建及其在情感分类中的应用[J]. 中文信息学报, 2017, 31(2): 187-193
[5]	Kiritchenko S, Zhu X, Cherry C, et al. Nrc-canada-2014: Detecting aspects and sentiment in customer reviews[C]. Proceedings of the 8th International Workshop on Se-mantic Evaluation (SemEval 2014), 2014: 437-442.
[6]	Perikos I, Hatzilygeroudis I. Recognizing emotions in text using ensemble of classifiers[J]. Engineering Applications of Artificial Intelligence, 2016, 51: 191-201. doi: 10.1016/j.engappai.2016.01.012
[7]	Kiritchenko S, Zhu X, Mohammad S M. Sentiment analysis of short informal texts[J]. Journal of Artificial Intelligence Research, 2014, 50: 723-762. doi: 10.1613/jair.4272
[8]	Liujie Z, Yanquan Z, Xiuyu D, et al. A Hierarchical multi-input and output Bi-GRU Model for Sentiment Analysis on Customer Reviews[J]. Iop Conference, 2018, 322: 062007.
[9]	Sun X, Li C, Ren F. Sentiment analysis for Chinese microblog based on deep neural networks with convol-utional extension features[J]. Neurocomputing, 2016, 210: 227-236. doi: 10.1016/j.neucom.2016.02.077
[10]	Chen S, Ding Y, Xie Z, et al. Chinese Weibo sentiment analysis based on character embedding with dualchannel convolutional neural network[C]. 2018 IEEE 3rd Interna-tional Conference on Cloud Computing and Big Data Analysis, IEEE, 2018: 107-111.
[11]	ZHAI P H, ZHANG D Y. Bidirectional-GRU based on attention mechanism for aspect-level sentiment ana-lysis[C]. Proceedings of the 2019 11th International Confer-ence on Machine Learning and Computing, 2019: 86-90.
[12]	范涛, 王昊, 吴鹏. 基于图卷积神经网络和依存句法分析的网民负面情感分析研究[J]. 数据分析与知识发现, 2021, 5(9): 97-106.
[13]	Zou Y, Gui T, Zhang Q, et al. A lexicon-based supervised attention model for neural sentiment analysis[C]. Pro-ceedings of the 27th International Conference on Compu-tational Linguistics, 2018: 868-877.
[14]	Yang X P, Zhang Z X, Wang L, et al. Automatic con-struction and optimization of sentiment lexicon based on Word2Vec[J]. Computer Science, 2017, 44(1): 42-47.
[15]	Cambria E, Poria S, Hazarika D, et al. SenticNet 5: Dis-covering conceptual primitives for sentiment analysis by means of context embeddings[C]. Proceedings of the AAAI Conference on Artificial Intelligence, 2018, 32(1): 1795-1802.
[16]	曾春艳, 严康, 王志锋, 等. 深度学习模型可解释性研究综述[J]. 计算机工程与应用, 2021, 57(8): 1-9. doi: 10.3778/j.issn.1002-8331.2012-0357
[17]	Mikolov T, Sutskever I, Chen K, et al. Distributed repre-sentations of words and phrases and their compositi-onality[C]. Proceedings of the 26th International Con-ference on Neural Information Processing Systems-Volume 2, 2013: 3111-3119.
[18]	Pennington J, Socher R, Manning C D. Glove: Global vectors for word representation[C]. Proceedings of the 2014 Conference on Empirical Methods in Natural Lan-guage Processing, 2014: 1532-1543.
[19]	方炯焜, 陈平华, 廖文雄. 结合GloVe和GRU的文本分类模型[J]. 计算机工程与应用, 2020, 56(20): 98-103. doi: 10.3778/j.issn.1002-8331.2001-0272
[20]	Ni R, Cao H. Sentiment Analysis based on GloVe and LSTM-GRU[C]. 2020 39th Chinese Control Conference (CCC), IEEE, 2020: 7492-7497.
[21]	石隽锋, 李济洪, 王瑞波. 一种改进的GloVe词向量表示学习方法[J]. 中文信息学报, 2021, 35(4): 16-22.
[22]	陈珍锐, 丁治明. 基于GloVe模型的词向量改进方法[J]. 计算机系统应用, 2019, 28(1): 194-199.
[23]	Bahdanau D, Cho K, Bengio Y. Neural machine tran-slation by jointly learning to align and translate[J]. arXiv preprint arXiv:1409.0473, 2014.
[24]	Liu L, Utiyama M, Finch A, et al. Neural Machine Translation with Supervised Attention[C]. Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, 2016: 3093-3102.
[25]	Luca M, Zervas G. Fake it till you make it: Reputation, competition, and Yelp review fraud[J]. Management Science, 2016, 62(12): 3412-3427. doi: 10.1287/mnsc.2015.2304
[26]	Wang S I, Manning C D. Baselines and bigrams: Simple, good sentiment and topic classification[C]. Proceedings of the 50th Annual Meeting of the Association for Com-putational Linguistics (Volume 2:Short Papers), 2012: 90-94.
[27]	Kim Y. Convolutional Neural Networks for Sentence Classification[C]. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, 2014: 1746-1751.
[28]	Kalchbrenner N, Grefenstette E, Blunsom P. A Convo-lutional Neural Network for Modelling Sentences[C]. Proceedings of the 52nd Annual Meeting of the Associ-ation for Computational Linguistics (Volume 1:Long Papers), 2014: 655-665.
[29]	Zhou P, Qi Z, Zheng S, et al. Text Classification Im-proved by Integrating Bidirectional LSTM with Two-dimensional Max Pooling[C]. Proceedings of COLING 2016, the 26th International Conference on Comp-utational Linguistics: Technical Papers, 2016: 3485-3495.
[30]	Zhang Y, Liu Q, Song L. Sentence-State LSTM for Text Representation[C]. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1:Long Papers), 2018: 317-327.
[31]	Hasani R, Lechner M, Amini A, et al. Closed-form continuous-depth models[J]. arXiv preprint arXiv: 2106.13898, 2021.
[32]	Wang J, Yu L C, Lai K R, et al. Dimensional sentiment analysis using a regional CNN-LSTM model[C]. Proc-eedings of the 54th Annual Meeting of the Association for Computational Linguistics (volume 2:Short papers), 2016: 225-230.
[33]	袁和金, 张旭, 牛为华, 等. 融合注意力机制的多通道卷积与双向GRU模型的文本情感分析研究[J]. 中文信息学报, 2019, 33(10): 109-118.

类别	数目	实例
Neg_dict	55,314	abortion -0.86, abortion_ban -0.86, abortive -0.78, abortus -0.96
Pos_dict	44,686	absence_of_risk 0.357, absenteeism 0.722, absently 0.308, absolute 0.208,

参数名称	参数值
Batch_size	64
Embedding_size	200
Gru_Size	256
Dropout	0.2
Learning_rate	0.001
$L_{2}$	0.001
$\lambda_{w}$	3
$\mu$	3

Model	Acc（%）
Model	IMDB	Yelp	MR
SVM	83.8	82.54	79.48
BoW	87.8	—	—
CNN	85.6	86.14	83.87
DCNN	86.8	—	—
Bi-LSTM	86.6	86.96	84.23
ATT-BiLSTM	89.3	89.81	87.78
S-LSTM	87.15	86.67	85.27
CFC	88.4	—	—
CNN+LSTM	88.9	88.56	86.43
ATT-MCNN-BGRUM	90.8	—	—
LDCA	92.52	91.83	89.54

Models	Acc (%)
Models	IMDB	Yelp
LDCA	92.52	91.83
-EEC	91.58	91.48
-SEC	87.54	86.23
-supervised attention	91.12	89.87

融合情感词典和自注意力的双通道情感分析模型

A Dual-Channel Sentiment Analysis Model Integrating Sentiment Lexcion and Self-Attention

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

图/表 9

参考文献 33

相关文章 15

编辑推荐

Metrics

本文评价

[1]	李俊飞, 徐黎明, 汪洋, 魏鑫. 基于深度学习技术的科技文献引文分类研究综述[J]. 数据与计算发展前沿, 2023, 5(4): 86-100.
[2]	张晓帆, 孙海春, 李欣. 融合多层注意力机制与BiLSTM的知识图谱补全算法研究[J]. 数据与计算发展前沿, 2023, 5(3): 123-137.
[3]	李妍,何洪波,王闰强. 微博热度预测研究综述[J]. 数据与计算发展前沿, 2023, 5(2): 119-135.
[4]	刘云帆,李琦,孙哲南,谭铁牛. 基于生成对抗网络的人脸年龄编辑方法综述[J]. 数据与计算发展前沿, 2023, 5(2): 2-23.
[5]	涂又友,郑奇靖,赵瑾. 基于深度学习方法研究分子/固体界面量子化质子耦合的电荷转移过程[J]. 数据与计算发展前沿, 2023, 5(2): 37-49.
[6]	许淞源,刘峰. ESDRec：一种面向地球大数据平台的数据推荐模型[J]. 数据与计算发展前沿, 2023, 5(1): 55-64.
[7]	赵忠斌,蔡满春,芦天亮. 融合多头注意力机制的网络恶意流量检测[J]. 数据与计算发展前沿, 2022, 4(5): 60-67.
[8]	刘琦玮,李俊,顾蓓蓓,赵泽方. TSAIE：图像增强文本的多模态情感分析模型[J]. 数据与计算发展前沿, 2022, 4(3): 131-140.
[9]	陈琼,杨咏,黄天林,冯媛. 小样本图像语义分割综述[J]. 数据与计算发展前沿, 2021, 3(6): 17-34.
[10]	蒲晓蓉,黄佳欣,刘军池,孙家瑜,罗纪翔,赵越,陈柯成,任亚洲. 面向临床需求的CT图像降噪综述[J]. 数据与计算发展前沿, 2021, 3(6): 35-49.
[11]	何涛,王桂芳,马廷灿. 基于词嵌入语义异常的跨学科研究内容发现方法[J]. 数据与计算发展前沿, 2021, 3(6): 50-59.
[12]	肖楠,周明珠,邢军,罗泽,李晓辉. 基于高分辨率网络和注意力机制的真伪卷烟包装鉴别[J]. 数据与计算发展前沿, 2021, 3(5): 118-129.
[13]	张怡宁,何洪波,王闰强. 热门数字音频预测技术综述[J]. 数据与计算发展前沿, 2021, 3(4): 81-92.
[14]	陈子健,李俊,岳兆娟,赵泽方. 基于自编码器与属性信息的混合推荐模型[J]. 数据与计算发展前沿, 2021, 3(3): 148-155.
[15]	肖建平,龙春,赵静,魏金侠,胡安磊,杜冠瑶. 基于深度学习的网络入侵检测研究综述[J]. 数据与计算发展前沿, 2021, 3(3): 59-74.