An Improved Sentiment Analysis Model Incorporating Textual Topic Features

doi:10.11871/jfdc.issn.2096-742X.2022.06.011

Abstract

Abstract:

[Objective] Massive user reviews are of great value to consumers and related enterprises. This paper addresses the problems of sparse data, unclear topics, and poor classification accuracy caused by the short length of review information. [Methods] This paper proposes a Bi-LSTM self-attention mechanism online review sentiment analysis model (TSC-BiLSTM) incorporating topic features. Compared with the traditional LSTM method, this method uses the Latent Dirichlet Allocation (LDA) topic model to obtain the topic word distribution of comments, stitches it with the comment word vector as input, mines the full-text feature information through Bi-LSTM, and combines with self-attention mechanism to dynamically assign weights. [Results] This model expands the feature space of the original short review text, reduces the sparsity of the data, clarifies the topic, and improves the accuracy of sentiment classification. [Conclusions] Experiments on review datasets of a hotel and a takeaway platform show that the proposed method achieves better performances compared with other related models. It provides a novel view of topic sentiment analysis methods.

Key words: LDA, topic words, Bi-LSTM, self-attention, sentiment analysis

ZHANG Shuai,HUANG Bo,JU Jiaji. An Improved Sentiment Analysis Model Incorporating Textual Topic Features[J]. Frontiers of Data and Computing, 2022, 4(6): 118-128, https://cstr.cn/32002.14.jfdc.CN10-1649/TP.2022.06.011.

Figures/Tables 18

Fig.1

Fig.2

Fig.3

Fig.4

Fig.5

Fig.6

Fig.7

Fig.8

Table 1

Fig. 9

Table 2

Table 3

Fig.10

Table 4

Table 5

Table 6

Fig.11

Table 7

References 22

[1]	Blei D M, Ng A Y, Jordan M I. Latent dirichlet alloca-tion[J]. the Journal of machine Learning research, 2003, 3: 993-1022.
[2]	Xue J, Chen J, Chen C, et al. Public discourse and sen-timent during the COVID 19 pandemic: Using Latent Dirichlet Allocation for topic modeling on Twitter[J]. PloS one, 2020, 15(9): e0239441. doi: 10.1371/journal.pone.0239441
[3]	Tago K, Jin Q. Influence analysis of emotional behaviors and user relationships based on Twitter data[J]. Tsinghua Science and Technology, 2018, 23(1): 104-113. doi: 10.26599/TST.2018.9010012
[4]	吴彦文, 黄凯, 王馨悦, 林娴. 一种融合主题模型的短文本情感分类方法[J]. 小型微型计算机系统, 2019, 40(10):2082-2086.
[5]	Shams M, Baraani-Dastjerdi A. Enriched LDA (ELDA): Combination of latent Dirichlet allocation with word co-occurrence analysis for aspect extraction[J]. Expert Systems with Applications, 2017, 80: 136-146. doi: 10.1016/j.eswa.2017.02.038
[6]	孟仕林, 赵蕴龙, 关东海, 翟象平. 融合情感与语义信息的情感分析方法[J]. 计算机应用, 2019, 39(07):1931-1935.
[7]	Tang D, Qin B, Feng X, et al. Effective LSTMs for targ-et-dependent sentiment classification[J]. arXiv preprint arXiv:1512.01100, 2015.
[8]	Bahdanau D, Cho K, Bengio Y. Neural machine tran-slation by jointly learning to align and translate[J]. arXiv preprint arXiv:1409.0473, 2014.
[9]	Li W, Liu P, Zhang Q, et al. An improved approach for text sentiment classification based on a deep neural net-work via a sentiment attention mechanism[J]. Future Internet, 2019, 11(4): 96. doi: 10.3390/fi11040096
[10]	Xie J, Chen B, Gu X, et al. Self-attention-based Bi-LSTM model for short text fine-grained sentiment classi-fication[J]. IEEE Access, 2019, 7: 180558-180570. doi: 10.1109/ACCESS.2019.2957510
[11]	Wang X, Chen G. Dependency-attention-based LSTM for target-dependent sentiment analysis[C]// Chinese National Conference on Social Media Processing. Springer, Sing-apore, 2017: 206-217.
[12]	胡朝举, 梁宁. 基于深层注意力的LSTM的特定主题情感分析[J]. 计算机应用研究, 2019, 36(04):1075-1079.
[13]	吴小华, 陈莉, 魏甜甜, 范婷婷. 基于Self-Attention和Bi-LSTM的中文短文本情感分析[J]. 中文信息学报, 2019, 33(06):100-107.
[14]	陶志勇, 李小兵, 刘影, 刘晓芳. 基于双向长短时记忆网络的改进注意力短文本分类方法[J]. 数据分析与知识发现, 2019, 3(12):21-29.
[15]	李磊, 吴旭辉, 刘继. 融合关键对象识别与深层自注意力的Bi-LSTM情感分析模型[J]. 小型微型计算机系统, 2021, 42(03):504-509.
[16]	Ghanbari-Adivi F, Mosleh M. Text emotion detection in social networks using a novel ensemble classifier based on Parzen Tree Estimator (TPE)[J]. Neural Computing and Applications, 2019, 31(12): 8971-8983. doi: 10.1007/s00521-019-04230-9
[17]	Park H, Park T, Lee Y S. Partially collapsed Gibbs sampling for latent Dirichlet allocation[J]. Expert Systems with Applications, 2019, 131: 208-218. doi: 10.1016/j.eswa.2019.04.028
[18]	Sherstinsky A. Fundamentals of recurrent neural network (RNN) and long short-term memory (LSTM) network[J]. Physica D: Nonlinear Phenomena, 2020, 404: 132306. doi: 10.1016/j.physd.2019.132306
[19]	Al-Sabahi K, Zuping Z, Nadher M. A hierarchical stru-ctured self-attentive model for extractive document sum-marization (HSSAS)[J]. IEEE Access, 2018, 6: 24205- 24212. doi: 10.1109/ACCESS.2018.2829199
[20]	Li Q, Li S, Zhang S, et al. A review of text corpus-based tourism big data mining[J]. Applied Sciences, 2019, 9(16): 3300. doi: 10.3390/app9163300
[21]	Li L, Wang P, Yan J, et al. Real-world data medical kno-wledge graph: construction and applications[J]. Artificial intelligence in medicine, 2020, 103: 101817. doi: 10.1016/j.artmed.2020.101817
[22]	Dong Y, Fu Y, Wang L, et al. A sentiment analysis me-thod of capsule network based on BiLSTM[J]. IEEE Ac-cess, 2020, 8: 37014-37020.

Topic 0	Topic1	Topic2
房间，0.037 服务，0.024	楼层，0.062 酒店，0.042	元，0.067 信用卡，0.028
好，0.020 遗憾，0.016 感觉，0.014 挺不错，0.014 很漂亮，0.013 态度，0.013 效果，0.010 喜欢，0.010	隔音，0.026 行李，0.023 宽敞，0.020 位于，0.018 电梯，0.018 房间，0.014 早餐，0.014 可惜，0.013	价格，0.026 结账，0.021 刷卡，0.020 前台，0.019 刷，0.019 银行，0.017 收到，0.016 卡，0.015

参数名称	参数值
词向量维度	300
LDAalpha LDAbeta hidden_units attention_size dropout bathch size	0.01 0.625 100 100 0.2 128

真实结果	预测结果
真实结果	正类	负类
正类	TP	FN
负类	FP	TN

模型方案	P/%	R/%	F1/%
Bi-LSTM	83.26	86.47	84.83
TC-LSTM	87.13	86.91	87.02
ATT-BiLSTM	85.65	87.31	86.98
AE-DATT-LSTM	88.08	89.23	88.65
C-SATT-BiLSTM	90.72	90.19	90.45
SAAE-BiLSTM	89.43	91.21	90.31
TSA-BiLSTM	91.73	90.74	91.23

模型方案	P/%	R/%	F1/%
Bi-LSTM	77.51	74.26	75.85
TC-LSTM	79.60	81.87	80.72
ATT-BiLSTM	82.77	83.32	83.04
AE-DATT-LSTM	85.53	85.12	85.32
C-SATT-BiLSTM	85.21	84.27	84.74
SAAE-BiLSTM	84.29	85.67	84.97
TSA -BiLSTM	87.31	85.83	86.56