A Framework for Intelligent Auditing of Agricultural Science Metadata Based on Multi-Model Collaboration and Dynamic Weighted Adjudication

doi:10.11871/jfdc.issn.2096-742X.2026.02.013

Abstract

Abstract:

[Objective] This study proposes a quality governance-oriented intelligent auditing framework to address inefficiency, inconsistent standards in agricultural science metadata review. [Methods] The framework adopts a two-stage hierarchical strategy: (1) Multiple heterogeneous Large Language Models (LLMs) perform parallel preliminary audits, evaluated through an “algorithmic triangulation” method across semantic, lexical, and structural dimensions; (2) A Dynamic Weighted System Disagreement (DWSD) algorithm quantifies inter-model conflicts and dynamically triggers high-performance adjudicator models for precise handling of challenging cases. [Results] On a real-world agricultural metadata dataset, the proposed framework improved F1-score, precision, and recall by 15.24%, 14.17%, and 15.00%, respectively, over the best baseline, significantly enhancing audit accuracy and coverage. [Limitations] The model was validated on a single Chinese agricultural dataset; its generalizability across different languages and domains requires further testing. Additionally, the effectiveness of the applied bias mitigation strategies at a larger scale needs ongoing assessment. [Conclusions] The intelligent framework proposed in this study can effectively enhance the intelligence, accuracy, and interpretability of agricultural metadata auditing, and provide a generalizable technical path for data quality governance tasks across multiple languages and domains.

Key words: agricultural metadata audit, multi-model collaboration, LLM-as-a-Judge, Dynamic Weighting, DWSD

REN Youqiang, ZHAO Hui, LI Wei, YUAN Huan, FAN Jingchao, ZHANG Jianhua, ZHOU Guomin. A Framework for Intelligent Auditing of Agricultural Science Metadata Based on Multi-Model Collaboration and Dynamic Weighted Adjudication[J]. Frontiers of Data and Computing, 2026, 8(2): 171-183, https://cstr.cn/32002.14.jfdc.CN10-1649/TP.2026.02.013.

Figures/Tables 13

Fig.1

Table 1

Fig.2

Fig.3

Fig.4

Fig.5

Fig.6

Fig.7

Fig.8

Fig.9

Fig.10

Fig.11

Table 2

References 19

[1]	樊景超, 满芮, 张翔鹤, 等. 国家农业观测数据共享元数据标准研究[J]. 农业大数据学报, 2020, 2(4): 14-19. doi: 10.19788/j.issn.2096-6369.200402
[2]	周国民, 樊景超. 农业科学观测数据汇聚管理平台设计与实现[J]. 农业大数据学报, 2019, 1(3). 38-45. doi: 10.19788/j.issn.2096-6369.190304
[3]	满芮, 王健. 我国农业元数据标准初探[J]. 中国科技资源导刊, 2016, 48(2): 7-12.
[4]	朱国良, 姚娜. 农业信息资源元数据研究进展[J]. 农业工程技术, 2023, 43(17): 16-17.
[5]	韦一金, 樊景超. 基于ChatGLM2-6B的农业政策问答系统[J]. 数据与计算发展前沿(中英文), 2024, 6(4): 116-127.
[6]	田永林, 王雨桐, 王兴霞, 等. 从RAG到SAGE: 现状与展望[J]. 自动化学报2025, 51(6): 1145-1169.
[7]	AMIRIZANIANI M, MARTIN E, ROOSTA T, et al. AuditLLM:A Tool for Auditing Large Language Models Using Multiprobe Approach[J/OL]. arXiv preprint arXiv: 2402.09334, 2024[2025-08-25]. https://doi.org/10.48550/arXiv.2402.09334.
[8]	AMIRIZANIANI M, YAO J, LAVERGNE A, et al. LLMAuditor:A Framework for Auditing Large Language Models Using Human-in-the-Loop[J/OL]. arXiv preprint arXiv: 2402.09346, 2024[2025-08-25]. https://doi.org/10.48550/arXiv.2402.09346.
[9]	HUANG J J, ZHU H R, XU C, et al. AuditWen:An Open-Source Large Language Model for Audit[J/OL]. arXiv preprint arXiv: 2410.10873, 2024[2025-08-25]. https://doi.org/10.48550/arXiv.2410.10873.
[10]	GRATTAFIORI A, DUBEY A, JAUHRI A, et al. The llama 3 herd of models[J/OL]. arXiv preprint arXiv: 2407.21783, 2024[2025-07-24]. https://doi.org/10.48550/arXiv.2407.21783.
[11]	HUI B, YANG J, CUI Z, et al. Qwen2.5-coder technical report[J/OL]. arXiv preprint arXiv: 2409.12186, 2024[2025-07-24]. https://doi.org/10.48550/arXiv.2409.12186.
[12]	YANG A, LI A, YANG B, et al. Qwen3 technical report[J/OL]. arXiv preprint arXiv: 2505.09388, 2025[2025-07-24]. https://doi.org/10.48550/arXiv.2505.09388.
[13]	GUO D, YANG D, ZHANG H, et al. DeepSeek-R1:Incentivizing reasoning capability in llms via reinforcement learning[J/OL]. arXiv preprint arXiv: 2501. 12948, 2025[2025-07-24]. https://doi.org/10.48550/arXiv.2501.12948.
[14]	RISTAD E S, YIANILOS P N. Learning string-edit distance[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2002, 20(5): 522-532. doi: 10.1109/34.682181
[15]	黄代雄, 杨嘉玮, 周伟. 基于Jaccard的智能变电站配置文件模型比对算法研究与应用[J]. 黑龙江科学, 2025, 16(6): 120-123.
[16]	李鲲程, 刘秋月, 范春梅. 数据清洗中文本相似度算法的比较与优化[J]. 通信管理与技术, 2021(5): 16-18.
[17]	LONGPRE S, HOU L, VU T, et al. The flan collection: designing data and methods for effective instruction tuning[C]// Proceedings of the 40th International Conference on Machine Learning, Honolulu. PMLR, 2023:22631-22648.
[18]	WHITE J, FU Q, HAYS S, et al. A prompt pattern catalog to enhance prompt engineering with chatgpt[J/OL]. arXiv preprint arXiv: 2302.11382, 2023[2025-07-24]. https://doi.org/10.48550/arXiv.2302.11382.
[19]	SCHULHOFF S, ILIE M, BALEPUR N, et al. The prompt report:A systematic survey of prompting techniques[J/OL]. arXiv preprint arXiv: 2406.06608, 2024[2025-07-24]. https://doi.org/10.48550/arXiv.2406.06608.

项目	具体配置
CPU	2 * Intel(R) Xeon(R) Platinum 8360Y @2.40GHz
GPU	4*A6000,192G显存
CUDA	12.4
LLM（temperature=0.1）	qwen2.5:32b-instruct
	qwen3:32b-q8_0
	llama3.1:8b-instruct-fp16
裁决LLM（temperature=0.1）	qwen3:32b-q8_0
裁决LLM（temperature=0.1）	DeepSeek-R1:32b-qwen-distill-q8_0
Ollama	0.6.6
裁决指令	{ ``` "final_instruction": "指令：检测到低分歧。请综合以下相似建议的优点，生成一个最完善的最终决定。在'justification'中，说明你主要采纳了哪个模型的建议，以及是否对其进行了优化。" },
system_message	"""你是一位顶级的、严谨的农业科学元数据质量控制专家。你的工作核心是确保每一条元数据的质量。你的核心原则是：1. 准确性 2. 完整性 3. 规范性 4. 简洁性。（中间略过）你的最终输出必须是一个JSON数组，每个对象对应一项任务的裁决结果。T1-T6，6个任务要处理完整，每个对象必须包含以下四个字段： - 'taskId': 任务ID (例如T1)。 - 'final_decision': 你做出的最终裁决内容。 - 'status': 你的决策类型，必须是 '已修正' 或 '采纳最优' 中的一个。 - 'justification': 一个解释你决策的对象，必须包含 'critique' (对原始建议的简要评判) 和 'rationale' (你最终决策的形成理由) 两个子字段。"""

模型/系统	精确率	召回率	F1分数
Llama 3.1 8B	64.17%	65.00%	66.20%
Qwen2.5 14B	66.25%	66.67%	66.46%
Qwen2.5 32B	68.75%	80.00%	73.96%
Qwen3-32B（裁决）	81.25%	93.33%	86.90%
DeepSeek-32B（裁决）	82.92%	95.00%	89.2%