基于摘要辅助认知增强的多模态科学数据检索方法

doi:10.11871/jfdc.issn.2096-742X.2026.01.017

数据与计算发展前沿 ›› 2026, Vol. 8 ›› Issue (1): 207-218.

CSTR: 32002.14.jfdc.CN10-1649/TP.2026.01.017

doi: 10.11871/jfdc.issn.2096-742X.2026.01.017

基于摘要辅助认知增强的多模态科学数据检索方法

杨斌¹(),吕梁^2,³,吕晓雯³,乔振^1,^*()

1.山东省科学技术情报研究院，山东济南 250101
2.山东大学，软件学院，山东济南 250101
3.山大地纬软件股份有限公司，山东济南 250200

收稿日期:2025-01-03 出版日期:2026-02-20 发布日期:2026-02-02
通讯作者: 乔振
作者简介:杨斌，山东省科学技术情报研究院副院长，硕士，主要从事科技情报管理和科技情报研究工作。
本文承担工作为：科技数据资料汇总及论文整体撰写思路规划。
YANG Bin, Vice President of Shandong Institute of Scientific and Technical Information, holds a Master’s degree and is mainly engaged in the management and research of scientific and technological information.
In this paper, he is mainly responsible for summarizing scientific and technological data and planning the overall structure and writing strategy of the manuscript.
E-mail: 13793166466@163.com|乔振，山东省科学技术情报研究院副研究员，硕士，主要从事资源管理、产业研究工作。
本文承担工作为：基于提取关键词的多模态增强检索模块及相关实验部分撰写。
QIAO Zhen, holding a master’s degree, is an associate researcher at Shandong Institute of Scientific and Technical Information. He is mainly engaged in resource management and industrial research work.
In this paper, he is mainly responsible for writing the multimodal enhanced retrieval module based on keyword extraction and related experimental sections.
E-mail: qz02412@163.com
基金资助:
山东省重点研发计划（软科学）项目“科学数据中心体系布局与建设研究”(2025RZA0808);山东省大数据局政务信息化项目“山东省科学数据管理系统”(鲁数审核[2023]36号)

Multi-Format Scientific Data Retrieval Method Based on Abstract Assisted Cognitive Enhancement

YANG Bin¹(),LYU Liang^2,³,LYU Xiaowen³,QIAO Zhen^1,^*()

1. Shandong Institute of Scientific and Technical Information, Jinan, Shandong 250101, China
2. School of Software, Shandong University, Jinan, Shandong 250101, China
3. Dareway Software Co., Ltd., Jinan, Shandong 250200, China

Received:2025-01-03 Online:2026-02-20 Published:2026-02-02
Contact: QIAO Zhen

摘要/Abstract

摘要：

【目的】由于当前科技项目数据资源存在术语专业性高、知识关联复杂、检索效率低等问题，传统检索模式难以满足多模态科学数据关联检索的需求，导致已有科技项目数据资源难以实现知识共享及融合应用。【方法】针对上述问题，为推进科技资源开放共享，提出一种基于摘要辅助认知增强的多模态科学数据检索方法。首先，构建基于大模型的多模态科学数据语义表示模型，将专业领域知识与结构引导机制相结合，生成包含关键知识的结构化摘要；随后，基于结构化摘要提供的关键字、描述信息等知识，构建定向思维链多路引导的多模态科学数据检索模型，通过认知提示知识增强结合多步动态推理，实现科学数据的多维关联分析，提升科学数据领域数据资源检索性能。【结果】最后，依托某省科学数据管理系统开展实验，结果表明，该方法对比当前已有Modular RAG框架，在测试集上的准确率和Rouge-n指标分别提升4.65%和3.18%。【结论】实验结果验证了所提方法在多模态、知识关联复杂的科学数据检索的适用性。

关键词: 多模态科学数据检索, 多视角语义关联, 结构化摘要, 认知增强, 思维链多路引导

Abstract:

[Objective] Due to the specialization of terminology, complex knowledge associations and low retrieval efficiency in the current technology project data resources, the traditional retrieval mode is difficult to meet the needs of multimodal scientific data association retrieval, resulting in the difficulty of knowledge sharing and integrated application of existing technology project data resources. [Methods] To address the above problems, a multimodal scientific data retrieval method based on summary-assisted cognitive enhancement is proposed to promote open sharing of scientific and technological resources. Firstly, a multimodal scientific data semantic representation model based on large models is constructed, which combines professional domain knowledge with structural guidance mechanisms to generate structured summaries containing key knowledge. Subsequently, based on the knowledge of keywords and description information provided by structured abstracts, a multimodal scientific data retrieval model is constructed with directed thought chain multiplex guidance. By combining cognitive cueing with multi-step dynamic reasoning, this model enables multi-dimensional correlation analysis of scientific data and improves retrieval performance. [Results] The results show that, compared with the existing Modular RAG framework, the proposed method improves accuracy and Rouge-n index on the test set by 4.65% and 3.18% respectively. [Conclusions] These results verify the applicability of the proposed method to scientific data retrieval involving multimodal data and complex knowledge associations.

Key words: multi-format scientific data retrieval, multi-perspective semantic association, structured summary, cognitive enhancement, multi-channel guidance of thinking chain

杨斌,吕梁,吕晓雯,乔振. 基于摘要辅助认知增强的多模态科学数据检索方法[J]. 数据与计算发展前沿, 2026, 8(1): 207-218.

YANG Bin,LYU Liang,LYU Xiaowen,QIAO Zhen. Multi-Format Scientific Data Retrieval Method Based on Abstract Assisted Cognitive Enhancement[J]. Frontiers of Data and Computing, 2026, 8(1): 207-218, https://cstr.cn/32002.14.jfdc.CN10-1649/TP.2026.01.017.

图/表 6

图1

图2

图3

图4

表1

表2

参考文献 17

[1]	罗鹏程, 王继民, 王世奇, 等. 基于深度学习的科学数据集检索方法研究[J]. 情报理论与实践, 2022, 45(7): 49-56. doi: 10.16353/j.cnki.1000-7490.2022.07.008
[2]	王广钰, 李英玉. 基于HADOOP的空间科学大数据的区域检索算法[J]. 电子设计工程, 2018, 26(5): 48-51.
[3]	刘春蔚, 邹自明, 佟继周. 基于LSI的日地空间领域科学数据语义检索模型[J]. 中国科学院大学学报, 2016, 33(5): 711-719. doi: 10.7523/j.issn.2095-6134.2016.05.020
[4]	李成赞, 沈志宏, 黎建辉. 面向科学数据的搜索引擎voovle[J]. 科研信息化技术与应用, 2011, 2(5): 36-43.
[5]	薛秀丽, 武洁雯, 庞明樊, 等. 基于能力的全球卫生教育文献计量学分析——以WEB OF SCIENCE核心合集为例[J]. 疾病监测, 2024, 39(4): 511-516.
[6]	张浩严, 吕文涛, 余润泽, 等. 大语言模型研究现状[J]. 无线电工程, 2024: 1-15.
[7]	LEWIS P, PEREZ E, PIKTUS A, et al. Retrieva-augmented generation for knowledge-intensive NLP tasks[J]. Advances in Neural Information Processing Systems, 2020, 33: 9459-9474.
[8]	THOMO A. Pubmed retrieval with RAG techniques[J]. Studies in Health Technology and Informatics, 2024, 316: 652-653. doi: 10.3233/SHTI240498 pmid: 39176826
[9]	田永林, 王兴霞, 王雨桐, 等. RAG-PHI: 检索增强生成驱动的平行人与平行智能[J]. 智能科学与技术学报, 2024, 6(1): 41-51. doi: 10.11959/j.issn.2096-6652.2024015
[10]	刘彦宏, 崔永瑞. 基于Wrd2Vec模型与RAG框架的医疗检索增强生成算法[J]. 人工智能与机器人研究, 2024, 13(3): 479-486.
[11]	TOUTANOVA K, TOUTANOVA J D, MING W C. Bert: Pre-training of deep bidirectional transformers for language understanding[C]. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies, 2019: 4171-4186.
[12]	ALEXEY D, LUCAS B, ALEXANDER K, et al. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale (ViT)[C]. ICLR 2021,11929.
[13]	YIWEI W, SHAOZU Y, RUOSONG Y, et al. Tackling modality heterogeneity with multi-view calibration network for multimodal sentiment detection[C]. ACL 2023: 5240-5252.
[14]	AMIRI K, FARAH M. Graph of concepts for semantic annotation of remotely sensed images based on direct neighbors in RAG[J]. Canadian Journal of Remote Sensing, 2018, 44(6): 551-574. doi: 10.1080/07038992.2019.1569507
[15]	MATSUOTO N, MORAN J, CHOI H, et al. KRAGEN: a knowledge graph-enhanced RAG framework for biomedical problem solving using large language models[J]. Bioinformatics, 2024, 40(6): btae353.
[16]	MANSUROVA A, NUGUMANOVA A. QA-RAG: exploring LLM reliance on external knowledge[J]. Big Data and Cognitive Computing, 2024, 8(9): 115. doi: 10.3390/bdcc8090115
[17]	SHAMANE S, RININDU W, ELLIOTT W, et al. Improving the domain adaptation of retrieval augmented generation (rag) models for open domain question answering[J]. Transactions of the Association for Computational Linguistics 2023, 11: 1-17. doi: 10.1162/tacl_a_00530

对比模型	大模型类		评价指标/%
对比模型	框架类型	优化方法	ACC	Rouge-n
基线模型	无RAG	—	53.72	60.85
基线模型	Naive RAG	—	64.81	70.59
	Advanced RAG	滑动窗口优化配置^[14]	72.18	78.57
		细粒度分段^[15]	72.74	80.34
		元数据合并^[16]	78.75	85.11
	Modular RAG	重写模块^[10]	78.76	85.75
		自适应反馈机制^[17]	82.42	87.82
本文模型	Modular RAG	多模态自适应反馈机制	85.79	90.05

方法	评估答案忠实度/%	评估答案相关性/%	评估上下文相关性/%
无id编码	38.26	32.78	26.12
无文本信息描述	30.12	24.18	35.36
无RAG关键词实体树	44.71	39.65	34.97
本文方法	49.31	55.92	38.55

基于摘要辅助认知增强的多模态科学数据检索方法

Multi-Format Scientific Data Retrieval Method Based on Abstract Assisted Cognitive Enhancement

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

图/表 6

参考文献 17

相关文章 0

编辑推荐

Metrics

本文评价