Classification Model of Agricultural Science and Technology Policies Based on Improved BERT-BiGRU-Attention

doi:10.11871/jfdc.issn.2096-742X.2024.06.006

Abstract

Abstract:

[Objective] The agricultural science and technology policy plays an important role in scientific and technological innovation and resource allocation in agricultural science and technology activities, which presents the directional guidance to the national and local agricultural science and technology activities. The automatic classification of agricultural science and technology policies can improve the efficiency of searching and matching agricultural policies by the stakeholders in “production, education, research and application”, which enables quick finding of the science and technology policy information related to their needs. [Methods] Combining with policy tools, policy objects, and policy objectives, this paper constructs a three-dimensional classification index system of agricultural science and technology policies, establishes a classification data set of agricultural science and technology policies in Shandong Province, constructs an improved Bert-BiGRU-Attention model, and conducts model performance evaluation and comparative experiments with the improved model on the data set of agricultural science and technology policies in Shandong province. [Results] The improved Bert-BiGRU-Attention model achieves higher classification accuracy on the data set, and the obtained F1 value is also better than other models compared. [Conclusions] The experimental results show that the F1 value of the model proposed in this paper on the agricultural science and technology policy dataset of Shandong Province is 0.9650, which shows that the model performs well in the policy classification task.

Key words: agriculture, science and technology policy, policy text, policy instrument, deep learning

WEI Yijin,FAN Jingchao. Classification Model of Agricultural Science and Technology Policies Based on Improved BERT-BiGRU-Attention[J]. Frontiers of Data and Computing, 2024, 6(6): 53-61, https://cstr.cn/32002.14.jfdc.CN10-1649/TP.2024.06.006.

Figures/Tables 9

Table 1

Fig.1

Fig.2

Fig.3

Table 2

Table 3

Table 4

Table 5

Table 6

References 27

[1]	罗光宇. 政策法制化是农村科技政策实现的有力保障[J]. 科技管理研究, 2009, 29(12): 70-72.
[2]	苏金树, 张博锋, 徐昕. 基于机器学习的文本分类技术研究进展[J]. 软件学报, 2006(9): 1848-1859.
[3]	贾澎涛, 孙炜. 基于深度学习的文本分类综述[J]. 计算机与现代化, 2021, 311(7): 29-37.
[4]	TSOUMAKAS G, KATAKIS I. Multi-label classification: An overview[J]. International Journal of Data Warehousing and Mining (IJDWM), 2007, 3(3): 1-13.
[5]	ZENG Q, ZHAO X, HU X, et al. Learning emotional word embeddings for sentiment analysis[J]. Journal of Intelligent & Fuzzy Systems, 2021, 40(5): 9515-9527.
[6]	YAO L, MAO C, LUO Y. Graph convolutional networks for text classification[C]// Proceedings of the AAAI conference on artificial intelligence, 2019, 33(1): 7370-7377.
[7]	KALCHBRENNER N, GREFENSTETTE E, BLU-NSOM P. A convolutional neural network for modelling sentences[J]. arXiv preprint arXiv:1404.2188, 2014.
[8]	LEE J Y, DERNONCOURT F. Sequential short-text classification with recurrent and convolutional neural networks[J]. arXiv preprint arXiv:1603.03827, 2016.
[9]	张文峰, 奚雪峰, 崔志明, 等. 多标签文本分类研究回顾与展望[J]. 计算机工程与应用, 2023, 59(18): 28-48.
[10]	XIAO L, HUANG X, CHEN B, et al. Label-specific document representation for multi-label text classification[C]//Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP), 2019: 466-475.
[11]	许英姿, 任俊玲. 基于改进的加权补集朴素贝叶斯物流新闻分类[J]. 计算机工程与设计, 2022, 43(1):179-185. DOI:10.16208/j.issn1000-7024.2022.01.024.
[12]	WAN C H, LEE L H, RAJKUMAR R, et al. A hybrid text classification approach with low dependency on parameter by integrating K-nearest neighbor and support vector machine[J]. Expert Systems with Applications, 2012, 39(15): 11880-11888.
[13]	殷亚博, 杨文忠, 杨慧婷, 等. 基于卷积神经网络和KNN的短文本分类算法研究[J]. 计算机工程, 2018, 44(7): 193-198. DOI:10.19678/j.issn.1000-3428.0047596.
[14]	陈雪松, 邹梦. 基于BERT-BiGRU-CNN模型的短文本分类研究[J]. 吉林大学学报(信息科学版), 2023, 41(6): 1048-1053.
[15]	康雁, 杨其越, 李浩, 等. 基于主题相似性聚类的自适应文本分类[J]. 计算机工程, 2020, 46(3): 93-98.DOI:10.19678/j.issn.1000-3428.0053717.
[16]	KIM Y. Convolutional neural networks for sentence classification[J]. arXiv preprint arXiv:1408.5882, 2014.
[17]	LAI S, XU L, LIU K, et al. Recurrent convolutional neural networks for text classification[C]// Proceedings of the AAAI conference on artificial intelligence. 2015, 29(1): 9513. DOI:10.1609/aaai.v29i1.9513.
[18]	LIU P, QIU X, HUANG X. Recurrent neural network for text classification with multi-task learning[J]. arXiv preprint arXiv:1605.05101, 2016.
[19]	HOCHREITER S, SCHMIDHUBER J. Long short-term memory[J]. Neural computation, 1997, 9(8): 1735-1780.
[20]	TANG D, QIN B, LIU T. Document modeling with gated recurrent neural network for sentiment classification[C]// Proceedings of the 2015 conference on empirical methods in natural language processing, 2015: 1422-1432.
[21]	YANG Z, YANG D, DYER C, et al. Hierarchical attention networks for document classification[C]// Proceedings of the 2016 conference of the North American chapter of the association for computational linguistics:human language technologies, 2016: 1480-1489.
[22]	DEVLIN J, CHANG M W, LEE K, et al. Bert: Pre-training of deep bidirectional transformers for language understanding[J]. arXiv preprint arXiv:1810. 04805, 2018.
[23]	姚妮, 高政源, 娄坤, 等. 基于BERT和BiGRU的在线评论文本情感分类研究[J]. 轻工学报, 2020, 35(5):80-86.
[24]	王紫音, 于青. 基于BERT-BiGRU模型的文本分类研究[J]. 天津理工大学学报, 2021, 37(4): 40-46.
[25]	刘燕. 基于BERT-BiGRU的中文专利文本自动分类[J]. 郑州大学学报(理学版), 2023, 55(2): 33-40.DOI:10.13705/j.issn.1671-6841.2022125.
[26]	李芸, 潘雅丽, 肖冬. 基于改进BERT-BiGRU模型的文本情感分类研究[J]. 电子技术应用, 2023, 49(2): 9-14. DOI:10.16157/j.issn.0258-7998.223108.
[27]	ROTHWELL R, ZEGVELD W. Reindusdalization and technology[M]. London: Logman Group Limited, 1985: 83-104.

一级指标	二级指标	三级指标
农业科技政策	政策工具	供给型政策工具
		环境型政策工具
		需求型政策工具
	政策对象	农业科技创新主体
		农业生产者和从业人
		农产品市场和消费者
		农村环境和生态系统
		农业经济体系
	政策目标	提高农产品质量和产量
		保护和改善生态环境
		促进农业生产方式的可持续发展
		促进农业地区社会经济繁荣
		保障国家粮食安全

项目	基本信息
操作系统	Linux-5.15.120+-x86_64-with-glibc2.35
CPU	Intel(R) Xeon(R) CPU @2.20 GHz
GPU	Tesla T4
开发语言	Python 3.10.12
开发平台	Pytorch2.1.0+cu118

序号	发文主体	发文量/份
1	山东省农业农村厅	102
2	青岛市农业农村局	64
3	山东省科技厅	35
4	山东省人民政府	27
5	济南市农业农村局	26

		预测值
		Positive	Negative
实际值	Positive	TP	FN
实际值	Negative	FP	TN

参数名称	参数值
Batch size	32
Hidden size	768
学习率	3e-5
Epoch	15
Dropout	0.5
Optimizer	Adam