数据驱动的威胁狩猎语言模型研究进展

doi:10.11871/jfdc.issn.2096-742X.2022.05.011

数据与计算发展前沿 ›› 2022, Vol. 4 ›› Issue (5): 98-107.

CSTR: 32002.14.jfdc.CN10-1649/TP.2022.05.011

doi: 10.11871/jfdc.issn.2096-742X.2022.05.011

数据驱动的威胁狩猎语言模型研究进展

张润滋^1,^*(),康彬²

1.绿盟科技集团股份有限公司,北京 100089
2.解放军96941部队,北京 100085

收稿日期:2021-08-27 出版日期:2022-10-20 发布日期:2022-10-27
通讯作者: 张润滋
作者简介:张润滋,绿盟科技集团股份有限公司,高级安全研究员,博士,清华大学博士后,主要研究方向为智能安全运营、威胁狩猎、可解释人工智能等。
本文中负责总体统稿,文章整体构思和设计,文献调研和论文撰写。
ZHANG Runzi, Ph.D., is a senior security researcher of Nsfo-cus Information Technology Co., Ltd and post-doctoral at Tsi-nghua University. His recent research interests include AISec-Ops, threat hunting, and explainable AI.
In this paper, he is responsible for the overall construction and draft, literature survey, and manuscript writing.
E-mail: runzi_zhang@163.com
基金资助:
中国博士后科学基金资助项目(2020M670181)

Research Progress in Data-Driven Threat Hunting Language Models

ZHANG Runzi^1,^*(),KANG Bin²

1. NSFOCUS Information Technology Co., Ltd., Beijing 100089, China
2. Unit 96941 of PLA, Beijing 100085, China

Received:2021-08-27 Online:2022-10-20 Published:2022-10-27
Contact: ZHANG Runzi

摘要/Abstract

摘要：

【目的】梳理数据驱动下,面向主动式威胁狩猎的语言模型研究进展,为高级威胁的检测、溯源提供技术前瞻。【方法】结合安全前沿学术与工业进展,从威胁狩猎的评估指标构建,多模多维多源数据的融合、依赖爆炸缓解及分析,和多模态威胁狩猎语言的建模等多个层次分别介绍总结相关研究。【结果】结合威胁狩猎的关键需求与相关技术趋势,从支持的数据类型、模式类型、建模方法、实时性等维度,全面总结了数据驱动威胁狩猎与威胁狩猎语言模型的研究现状与研究趋势。【结论】面对高对抗性APT等威胁检测取证场景,一方面需要构建多源异构的融合数据基础设施,并解决数据的依赖爆炸问题;另一方面,仍需要探索标准化、灵活的语言模型,来支持多模态、多源、多维数据的统一分析。

关键词: 威胁狩猎, 安全运营, 高级持续性威胁

Abstract:

[Objective] This paper summarizes the research progress of language models for proactive threat hunting driven by data, and provides technical foresight for the detection and source tracing of advanced threats. [Methods] This study combines the frontier academic and industrial progress of cyber security, introduces and summarizes related research from multiple levels such as the evaluation metric construction for threat hunting, the fusion of multi-modal, multi-dimensional, and multi-source data, the dependency explosion mitigation and analysis, and finally the modeling of multi-modal threat hunting language. [Results] Combining the key requirements of threat hunting and related technology trends, the supported data types, model types, modeling methods, timeliness, and other dimensions of the research status and future works of data-driven threat hunting and language models are comprehensively summarized. [Conclusions] In the face of threat detection and forensic scenarios for adversarial APT attacks, on one hand, it is necessary to build a multi-source heterogeneous fusion data infrastructure and solve the problem of data dependence explosion. On the other hand, it is still necessary to explore standardized and flexible language models to support the unified analysis of multi-modal, multi-source, and multi-dimensional data.

Key words: threat hunting, security operations, advanced persistent threats

张润滋,康彬. 数据驱动的威胁狩猎语言模型研究进展[J]. 数据与计算发展前沿, 2022, 4(5): 98-107.

ZHANG Runzi,KANG Bin. Research Progress in Data-Driven Threat Hunting Language Models[J]. Frontiers of Data and Computing, 2022, 4(5): 98-107, https://cstr.cn/32002.14.jfdc.CN10-1649/TP.2022.05.011.

图/表 8

表1

图1

图2

图3

图4

图5

表2

图6

参考文献 16

[1]	John Caimano, Kerry Matre. Your Metrics Suck! 5 Sec-Ops Metrics That Are Better Than MTTR[EB/OL]. (2020-06-23).[2021-06-12]. https://www.rsaconference.com/Library/presentation/USA/2021/your-metrics-suck-5-secops-metrics-that-are-better-than-mttr.
[2]	Gartner. Hype Cycle for Security Operations, 2020[EB/OL]. (2020-06-23).[2021-06-12]. https://www.gartner.com/en/documents/3986721/hype-cycle-for-security-operations-2020.
[3]	Greg Young. The Skeptic’s Guide to Using XDR to Get Zero Trust[EB/OL]. (2021-05-19).[2021-06-12]. https://www.rsaconference.com/library/Presentation/USA/2021/the-skeptics-guide-to-using-xdr-to-get-zero-trust.
[4]	Alsaheel A, Nan Y, Ma S, et al. {ATLAS}: A Sequence-based Learning Approach for Attack Investigation[C]. 30th {USENIX} Security Symposium ({USENIX} Sec-urity 21), 2021.
[5]	OASIS. Introduction to STIX[EB/OL]. (2021-05-20).[2021-06-12]. https://oasis-open.github.io/cti-doc-umen-tation/stix/intro.html.
[6]	The MITRE Corporation. Malware Attribute Enumeration and Characterization[EB/OL]. (2017-10-09).[2021-06-12]. https://oasis-open.github.io/cti-documentation/stix/intro.html.
[7]	The MITRE Corporation. ATT&CK[EB/OL]. (2017-5-17). [2021-06-12]https://attack.mitre.org/.
[8]	The Open Cybersecurity Alliance. ATT&opencybersecu-rityalliance/kestrel-lang[EB/OL]. (2017-5-19).[2021-06-12]. https://github.com/IBM/kestrel-lang.
[9]	Endgame. endgameinc/eql[EB/OL]. (2019-11-02).[2021-06-12]. https://github.com/endgameinc/eql.
[10]	Noel S, Harley E, Tam K H, et al. CyGraph: graph-based analytics and visualization for cybersecurity[M]. Handbook of Statistics: Elsevier, 2016: 117-167.
[11]	Shu X, Araujo F, Schales D L, et al. Threat Intelligence Computing[C]. Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security, 2018: 1883-1898.
[12]	Sigma. SigmaHQ/sigma[EB/OL].(2019-01-10).[2021-06-12]. https://github.com/SigmaHQ/sigma.
[13]	Flink. FlinkCEP - Complex event processing for Flink[EB/OL]. (2019-05-03).[2021-06-12]. https://ci.apache.org/projects/flink/flink-docs-stable/dev/libs/cep.html.
[14]	Gao P, Shao F, Liu X, et al. Enabling Efficient Cyber Threat Hunting With Cyber Threat Intelligence[J]. arXiv preprint arXiv:2010.13637, 2020.
[15]	Gao P, Xiao X, Li D, et al. {SAQL}: A stream-based query system for real-time abnormal system behavior detection[C]. 27th {USENIX} Security Symposium ({USENIX} Security 18), 2018: 639-656.
[16]	Gao P, Xiao X, Li Z, et al. {AIQL}: Enabling efficient attack investigation from system monitoring data[C]. 2018 {USENIX} Annual Technical Conference ({USE-NIX}{ATC} 18), 2018: 113-126.

监控维度	分项名称
分析活动 Analyst Activity	Events Per Analyst Hour (EPAH)
	Handling time per alert per stage per analyst
	Incidents by severity
	Severity updates
运营卫生 Hygiene	Unused rules
	Duplicate rules
	Top 10 and bottom 10
	# of tunes per technology
实现价值 Realized Value	% of features used
	% of visible traffic
	New use cases/alerts resulting from Threat Intel feeds
	Backlog of undeployed technology
过程偏离 Process Deviation	SOC procedure deviation
	Automation failures
	Training for consistency
	Certifications and continuous learning
分析负载分布 Analyst Work Distribution	Distribution of alerts per analyst
	Source of alerts
	Time spent according to 30/30/30 model
	Team structure

名称	数据类型				支持的模式类型					实时性			特性
名称	行为	环境	情报	知识	单点	集合	序列	静态图	时序图	模型	流式	批式	特性
Kestrel[8]	√	√	√		√	√		√		√		√	IBM Threat Hunting Language开源项目基于Jupyter notebook交互式实现Event Query Language
EQL[9]	√				√	√	√				√		无结构约束（Schema-independent）Elastic/Endgame终端分析开源实现CyGraph Query Language
CyQL[10]	√	√	√	√				√				√	基于MITRE CYGraph多源异构图架构
τ-calculus[11]	√	√	√		√	√	√	√	√			√	基于IBM Threat Intelligence Comput-ing时序图分析引擎
Sigma[12]	√				√	√						√	Generic Signature Formats for SIEM Systems应用层指纹识别格式语言
Flink CEP[13]	√	√	√		√	√	√				√	√	通用流式分析引擎灵的序列模式支持
TBQL [14]	√		√		√			√				√	Threat Behavior Query Language面向情报要素提取与匹配
SAQL [15]	√				√	√				√	√		Stream based Anornaly Query Language 异常检测专用语言,支持多种类型异常分析
AIQL[16]	√				√	√				√		√	Attack Investigation Query Language 与SAQL同体系架构

数据驱动的威胁狩猎语言模型研究进展

Research Progress in Data-Driven Threat Hunting Language Models

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

图/表 8

参考文献 16

相关文章 1

编辑推荐

Metrics

本文评价