数据与计算发展前沿 ›› 2026, Vol. 8 ›› Issue (3): 29-39.

doi: 10.11871/jfdc.issn.2096-742X.2026.03.003

• 专刊：第二十一届全国科学计算与信息化会议征文 • 上一篇    下一篇

基于深度学习的Lustre文件系统异常行为智能检测方法

侯思琦1(),程垚松1,程耀东1,2,*(),李海波1,2,毕玉江1,姚秋玲1   

  1. 1 中国科学院高能物理研究所， 北京 100049
    2 中国科学院大学， 北京 100049
  • 收稿日期:2025-10-27 出版日期:2026-06-20 发布日期:2026-06-18
  • 通讯作者: 程耀东
  • 作者简介:侯思琦，中国科学院高能物理研究所，助理工程师，主要研究分布式存储系统运维及研发。
    本文主要承担工作为模型的设计和代码实现与验证。
    Hou Siqi is an assistant engineer at the Institute of High Energy Physics, Chinese Academy of Sciences. Her research focuses on the operation, maintenance, and development of storage systems.
    In this paper, she is mainly responsible for model design and code implementation and verification.
    E-mail: housq@ihep.ac.cn|程耀东，中国科学院高能物理研究所，研究员，博士生导师，研究方向为海量数据存储、云计算与大数据等。
    本文主要负责课题整体设计、论文统筹审稿及通讯联络工作。
    CHENG Yaodong is a researcher and doctoral supervisor at the Institute of High Energy Physics of the Chinese Academy of Sciences. His research interests include massive data storage, cloud computing, and big data.
    In this paper, he is mainly responsible for the overall design of the research project, the coordination and review of the manuscript, and communication duties.
    E-mail: chyd@ihep.ac.cn
  • 基金资助:
    中国科学院青年创新促进会(2023013);浪潮存储青蓝基金广域网分布式文件系统与基于SPDK高性能文件系统

A Deep Learning-Based Intelligent Anomaly Detection Method for the Lustre File System

HOU Siqi1(),CHENG Yaosong1,CHENG Yaodong1,2,*(),LI Haibo1,2,BI Yujiang1,YAO Qiuling1   

  1. 1 Institute of High Energy Physics, Chinese Academy of Science, Beijing 100049, China
    2 University of Chinese Academy of Sciences, Beijing 100049, China
  • Received:2025-10-27 Online:2026-06-20 Published:2026-06-18
  • Contact: CHENG Yaodong

RichHTML

0

PDF

34

摘要：

【背景】Lustre文件系统是科学计算中的重要基座之一，随着存储容量和用户业务量不断扩大，存储系统的负载持续升高。单一用户的异常读写请求往往容易导致存储系统的卡顿，影响其余用户的使用体验。传统的异常处理方式是由运维人员浏览日志，排查定位异常用户或异常读写请求，最后实施故障排除策略恢复系统访问速度。 【目的】为提升异常诊断效率，替代依赖运维人员人工筛查日志的低效模式，本研究将深度学习引入运维过程，以实现对异常读写行为的智能诊断。 【方法】本研究设计了面向Lustre文件系统的异常行为智能检测系统，涉及用户行为数据采集、时序数据处理、模型搭建及训练和部署验证等过程。该系统将用户读写信息转换为时间序列数据，通过搭建长短时记忆网络，使用无监督学习的方式训练模型。 【结论】本研究分别在Lustre的MDT和OST数据上完成了模型的训练和验证。实验结果表明，所提方法能够显著提升异常识别精确率，并有效降低异常误报率。本文提出的方法能够降低运维人员的错误定位时间成本，提升科学计算环境中文件存储系统的异常处理效率。

关键词: 智能运维, 存储系统, 时序数据处理

Abstract:

[Background] The Lustre file system is a crucial foundation for scientific computing. With the continuous expansion of storage capacity and user workload, the load on storage systems is constantly increasing. Abnormal read and write requests from a single user often lead to storage cluster lag, affecting the overall user experience. Traditional anomaly handling typically involves maintainers browsing logs, identifying and locating abnormal users or read/write requests, and finally implementing troubleshooting strategies to restore system access speed. [Purpose] To improve the efficiency of anomaly diagnosis and replace the inefficient mode of relying on manual screening of logs by maintainers, this study introduces deep learning into the operation and maintenance process to achieve intelligent diagnosis of abnormal read and write behavior. [Method] This study builds an intelligent abnormal behavior detection system for Lustre file systems, involving user behavior data collection, time-series data processing, model construction, training, and deployment verification. The system converts user read and write information into time series data, builds a long short-term memory network and trains the model using unsupervised learning. [Conclusions] This study successfully trained and validated the model on Lustre's MDT and OST data. Experimental results show that the proposed method can significantly improve the accuracy of anomaly detection and effectively reduce the false alarm rate. The proposed method can reduce the time cost for error localization for maintainers and improve the efficiency of anomaly handling in file systems within scientific computing environments.

Key words: AIOps, storage system, time-series data processing