Abstract:

[Objective] The use of natural language processing technology can provide technical support for the security of network public opinion. In order to solve the problem that the recurrent neural network in text sentiment analysis cannot obtain the feature information of deep and shallow layers, and the dynamic word vector deviates from the core semantics, a K-BERT-BiRESRU-ATT based on K-BERT and the residual recurrent unit is proposed. [Methods] First, the K-BERT model is used to obtain the semantic feature vector containing background knowledge; Then, the proposed Bidirectional Residual Simple Recurrent Unit (BiRESRU) is used to extract the sequence of the contextual features to obtain deep and shallow feature information; After that, the attention mechanism is used to enhance the keyword weight of the output of BiRESRU; Finally softmax is used to classify the results. [Results] On the ChnSentiCorp and Weibo datasets, the accuracy rates were 95.6% and 98.25%, respectively; the calculation time was reduced by nearly 5 minutes per iteration compared with other recurrent networks, and the computational efficiency was improved. [Conclusions] K-BERT-BiRESRU-ATT solves the problem of the dynamic word vector deviation from the core semantics, obtains the feature information of deep and shallow layers, accelerates the model calculation, and improves the classification accuracy. But it still has a large demand for computing ability.

Key words: simple recurrent unit, K-BERT, sentiment analysis, security of network public opinion