Overview of Root Cause Localization Method in Microservice Architecture

doi:10.11871/jfdc.issn.2096-742X.2022.03.006

Abstract

Abstract:

[Objective] When the key performance indicators of the cloud native system are abnormal, operation and maintenance engineers are required to sort out the abnormal correlation behind the alarm storm and complex abnormal indicators in a timely manner, and perform accurate root cause localization and rapid recovery. [Methods] This paper introduces the way to build a fault propagation graph under a large-scale microservice architecture and a root cause localization technology based on graph reasoning. We investigate, compare and summarize the existing root cause localization methods based on the experience of operation and maintenance of the cloud platform and high-availability capacity building. [Results] The root cause localization method based on graph reasoning significantly improves the stability and reliability of the cloud system in large data centers. [Limitations] This method relies on a stable monitoring infrastructure and accurate anomaly detection capabilities for indicators. [Conclusions] With the deepening of digital transformation, the root cause localization technology of microservice indicators under the microservice architecture will play an increasingly important role in ensuring the stability of large-scale cloud platforms.

Key words: cloud native, microservices, AIOps, root cause localization

LI Siyi,MA Shiyu,CUI Liyue,ZHANG Shenglin,SUN Yongqian,ZHANG Yuzhi. Overview of Root Cause Localization Method in Microservice Architecture[J]. Frontiers of Data and Computing, 2022, 4(3): 78-89, https://cstr.cn/32002.14.jfdc.CN10-1649/TP.2022.03.006.

Figures/Tables 6

Fig.1

Fig.2

Fig.3

Fig.4

Fig.5

Table 1

References 25

[1]	蒋易民, 王恒, 胡泉, 于永恒, 蒋维杰, 杨祥合. 金融级IT架构[M]. 北京: 电子工业出版社, 2021:1-22.
[2]	Balalaie A, Heydarnoori A, Jamshidi P. Microservices architecture enables devops: Migration to a cloud-native architecture[J]. Ieee Software, 2016, 33(3): 42-52. doi: 10.1109/MS.2016.64
[3]	James Lewis, Martin Fowler. Microservices[EB/OL]. [2022-02-14]. https://martinfowler.com/articles/microser-vices.html.
[4]	中国信息通信研究院. 云计算发展白皮书(2018年)[EB/OL].[2022-02-14]. http://www.caict.ac.cn/kxyj/qwfb/bps/201808/P020180813540725575770.pdf.
[5]	Li W, Lemieux Y, Gao J, et al. Service mesh: Challenges, state of the art, and future research opportunities[C]// 2019 IEEE International Conference on Service-Oriented Sys-tem Engineering (SOSE), IEEE, 2019: 122-1225.
[6]	Brewer E A. Kubernetes and the path to cloud native[C]// Proceedings of the sixth ACM symposium on cloud com-puting, 2015: 167. https://kubernetes.io/docs/concepts/.
[7]	Keller A, Subramanian S. Best practices for deploying a CMDB in large-scale environments[C]// 2009 IFIP/IEEE International Symposium on Integrated Network Management. IEEE, 2009: 732-745.
[8]	Kalisch M, Bühlman P. Estimating high-dimensional directed acyclic graphs with the PC-algorithm[J]. Journal of Machine Learning Research, 2007, 8(3):613-636.
[9]	Tarjan R. Depth-first search and linear graph algorithms[J]. SIAM journal on computing, 1972, 1(2): 146-160. doi: 10.1137/0201010
[10]	Benesty J, Chen J, Huang Y, et al. Pearson correlation coefficient[M]//Noise reduction in speech processing. Springer, Berlin, Heidelberg, 2009: 1-4.
[11]	Xing W, Ghorbani A. Weighted pagerank algorithm[C]. InProceedings. Second Annual Conference on Communi-cation Networks and Services Research, IEEE, 2004:305-314.
[12]	Fouss F, Pirotte A, Renders J M, et al. Random-walk com-putation of similarities between nodes of a graph with application to collaborative recommendation[J]. IEEE Transactions on knowledge and data engineering, 2007, 19(3): 355-369. doi: 10.1109/TKDE.2007.46
[13]	Kim M, Sumbaly R, Shah S. Root cause detection in a service-oriented architecture[J]. ACM SIGMETRICS Performance Evaluation Review, 2013, 41(1): 93-104. doi: 10.1145/2494232.2465753
[14]	Wu L, Tordsson J, Elmroth E, et al. Microrca: Root cause localization of performance issues in microservices[C]// NOMS 2020-2020 IEEE/IFIP Network Operations and Management Symposium, IEEE, 2020: 1-9.
[15]	Jeh G, Widom J. Scaling personalized web search[C]// Proceedings of the 12th international conference on World Wide Web, 2003: 271-279.
[16]	Weng J, Wang J H, Yang J, et al. Root cause analysis of anomalies of multitier services in public clouds[J]. IEEE/ACM Transactions on Networking, 2018, 26(4): 1646-1659. doi: 10.1109/TNET.2018.2843805
[17]	Liu D, He C, Peng X, et al. MicroHECL: high-efficient root cause localization in large-scale microservice syste-ms[C]// 2021 IEEE/ACM 43rd International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP), IEEE, 2021: 338-347.
[18]	Wang H, Wu Z, Jiang H, et al. Groot: An Event-graph-based Approach for Root Cause Analysis in Industrial Settings[C]// 2021 36th IEEE/ACM International Confer-ence on Automated Software Engineering (ASE), IEEE, 2021: 419-429.
[19]	Wang P, Xu J, Ma M, et al. Cloudranger: Root cause identification for cloud native systems[C]// 2018 18th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID), IEEE, 2018: 492-502.
[20]	Ma M, Lin W, Pan D, et al. Ms-rank: Multi-metric and self-adaptive root cause diagnosis for microservice appl-ications[C]// 2019 IEEE International Conference on Web Services (ICWS), IEEE, 2019: 60-67.
[21]	Ma M, Lin W, Pan D, et al. ServiceRank: Root Cause Identification of Anomaly in Large-Scale Microservice Architecture[J]. IEEE Transactions on Dependable and Secure Computing, 2021: 1.
[22]	Meng Y, Zhang S, Sun Y, et al. Localizing failure root causes in a microservice through causality inference[C]// 2020 IEEE/ACM 28th International Symposium on Quality of Service (IWQoS), IEEE, 2020: 1-10.
[23]	Nie X, Zhao Y, Sui K, et al. Mining causality graph for automatic webbased service diagnosis[C]// 2016 IE-EE 35th International Performance Computing and Commu-nications Conference (IPCCC), IEEE, 2016: 1-8.
[24]	Lin J J, Chen P, Zheng Z. Microscope: Pinpoint perfor-mance issues with causal graphs in microservice envir-onments[C]// International Conference on ServiceOriented Computing, Springer, Cham, 2018: 3-20.
[25]	Chen P, Qi Y, Zheng P, et al. Causeinfer: Automatic and distributed performance diagnosis with hierarchical ca-usality graph in large distributed systems[C]// IEEE INF-OCOM 2014-IEEE Conference on Computer Communi-cations, IEEE, 2014:1887-1895.

论文	关系学习方法	根因分析方法
MonitorRank^[13]	系统信息	随机游走
MicroRCA^[14]	系统信息	PersonalizedPageRank^[15]
TON18^[16]	系统信息	随机游走
MicroHECL^[17]	系统信息	皮尔森相关系数
Groot^[18]	系统信息	PageRank
CloudRanger^[19]	PC算法	二阶随机游走
MSRank^[20]	PC算法	二阶随机游走
ServiceRank^[21]	PC算法	皮尔森相关系数
MicroCause^[22]	PCTS	TCORW
BRCA^[23]	相关系数分析	深度优先搜索
MicroScope^[24]	PC算法+系统信息	皮尔森相关系数
CauseInfer^[25]	PC算法+系统信息	深度优先搜索