A Visual Analysis System for SWF Log Event Stream Data

doi:10.11871/jfdc.issn.2096-742X.2021.04.001

Abstract

Abstract:

[Objective] HPC cluster log data is typically presented by traditional statistical charts, which has the disadvantage of no intuitive presentation and no interactivity. We design and implement SWFVis, an event stream visual analysis system for SWF log data to improve the presentation of the log data. [Methods] Based on the visual analysis technology of temporal event sequence data, combined with multi-dimensional visual analysis and associated visual analysis technology, SWFVis proposes a comprehensive visual analysis method for the sequential and multidimensional attributes of the jobs in the log event stream. [Results] Based on the real data generated by the public IPSC /860 and FORHLR II clusters, SWFVis intuitively displays the concurrent status and execution of the overall job in the logs, and analyzes the job processing patterns in the clusters. [Limitations] As for the visual rendering effect of event stream, there are still some problems in the three-dimensional scenes such as view blocking and disordered distribution. In the future, we consider achieving hierarchical clustering and other methods for further optimization. [Conclusions] SWFVis provides an interactive visual analysis method for SWF log data. It intuitively presents the working state of the cluster and user submission behavior, and supports interactive exploration of the processing pattern in the cluster operation, which provides support for the optimization of cluster scheduling.

Key words: HPC cluster, temporal event sequence data, event stream, visual analysis, SWF log

LI Yue,YANG Bo,LU Xuyi,SHAN Guihua. A Visual Analysis System for SWF Log Event Stream Data[J]. Frontiers of Data and Computing, 2021, 3(4): 3-17.

Figures/Tables 11

Fig. 1

Fig. 2

Fig. 3

Fig. 4

Fig. 5

Fig. 6

Fig. 7

Fig. 8

Fig. 9

Fig. 10

Fig. 11

References 22

[1]	Guo S N, Xu K, Zhao R W, et al. EventThread: visual summarization and stage analysis of event sequence data[J]. IEEE Transactions on Visualization and Computer Graphics, 2018,24(1):56-65. doi: 10.1109/TVCG.2017.2745320
[2]	Monroe M, Lan R J, Lee H, et al. Temporal event sequence simplification[J]. IEEE Transactions on Visualization and Computer Graphics, 2013,19(12):2227-2236. doi: 10.1109/TVCG.2013.200
[3]	LIU Z, WANG Y, DONTCHEVA M, et al. Patterns and sequences: Interactive exploration of clickstreams to un-derstand common visitor paths[J]. IEEE Transactions on Visualization and Computer Graphics, 2016,23(1):321-330. doi: 10.1109/TVCG.2016.2598797
[4]	CHEN Y, XU P, REN L. Sequence synopsis: Optimize visual summary of temporal event data[J]. IEEE transac-tions on visualization and computer graphics, 2017,24(1):45-55.
[5]	B H Park, Y Hui, S Boehm, R A Ashraf, C Layton, C En-gelmann. A Big Data Analytics Framework for HPC Log Data: Three Case Studies Using the Titan Supercomputer Log[C] //2018 IEEE International Conference on Cluster Computing (CLUSTER), 2018:571-579.
[6]	S Gupta, T Patel, C Engelmann, D Tiwari. Failures in large scale systems: Long-term measurement, analysis, and implications[C] //Proceedings of the International Conference for High Performance Computing, Networ-king, Storage and Analysis, ser. SC'17. New York, NY, USA: ACM, 2017: 44:1-44:12.
[7]	T Li, Y Jiang, C Zeng, B Xia, Z Liu, W Zhou, X Zhu, W Wang, L Zhang, J Wu, L Xue, D Bao. FLAP: An end-to-end event log analysis platform for system management[C] //Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ser. KDD '17. New York, NY, USA: ACM, 2017: 1547-1556.
[8]	B H Park, S Hukerikar, R Adamson, C Engelmann. Big data meets HPC log analytics: Scalable approach to under-standing systems at extreme scale[C] //IEEE Cluster 2017 at Workshop on Monitoring and Analysis for High Perfor-mance Computing Systems Plus Applications, Sept 2017: 758-765.
[9]	Dror G, Feitelson, Dan Tsafrir, David Krakov. Experience with using the Parallel Workloads Archive[J]. Journal of Parallel and Distributed Computing, 2014,74(10):2967-2982. doi: 10.1016/j.jpdc.2014.06.013
[10]	彭燕妮, 樊晓平, 赵颖, 周芳芳. 时间事件序列数据可视化综述[J]. 计算机辅助设计与图形学学报, 2019,31(10):1698-1710.
[11]	Huang D D, Tory M, Staub-French S, et al. Visualization techniques for schedule comparison[J]. Computer Graphics Forum, 2009,28(3):951-958. doi: 10.1111/cgf.2009.28.issue-3
[12]	Han Y, Rozga A, Dimitrova N, et al. Visual analysis of pro-ximal temporal relationships of social and communicative behaviors[J]. Computer Graphics Forum, 2015,34(3):51-60. doi: 10.1111/cgf.12617
[13]	Krstajic M, Bertini E, Keim D A. CloudLines: compact display of event episodes in multiple time-series[J]. IEEE Transactions on Visualization and Computer Graphics, 2011,17(12):2432-2439. doi: 10.1109/TVCG.2011.179
[14]	Wongsuphasawat K, Gomez J A G, Plaisant C, et al. LifeFlow: visualizing an overview of event sequences[C]. Proceedings of the SIGCHI Conference on Human Fac-tors in Computing Systems. New York: ACM Press, 2011: 1747-1756.
[15]	Wongsuphasawat K, Gotz D. Exploring flow, factors, and outcomes of temporal event sequences with the outflow visualization[J]. IEEE Transactions on Visualization and Computer Graphics, 2012,18(12):2659-2668. doi: 10.1109/TVCG.2012.225
[16]	Gotz D, Wongsuphasawat K. Interactive intervention analysis[C] //AMIA Annual Symposium Proceedings, 2012: 274-280.
[17]	Tanahashi Y, Hsueh C H, Ma K L. An efficient frame-work for generating storyline visualizations from strea-ming data[J]. IEEE Transactions on Visualization and Computer Graphics, 2015,21(6):730-742. doi: 10.1109/TVCG.2015.2392771
[18]	Ogawa M, Ma K L. Software evolution storylines[C] //Proceedings of the 5th International Symposium on Soft-ware Visualization. New York: ACM Press, 2010: 35-42.
[19]	Tanahashi Y, Ma K L. Design considerations for optimi-zing storyline visualizations[J]. IEEE Transactions on Visualization and Computer Graphics, 2012,18(12):2679-2688. doi: 10.1109/TVCG.2012.212
[20]	Liu S X, Wu Y C, Wei E X, et al. StoryFlow: tracking the evolution of stories[J]. IEEE Transactions on Visualiza-tion and Computer Graphics, 2013,19(12):2436-2445.
[21]	Perer A, Sun J M. MatrixFlow: temporal network visual analytics to track symptom evolution during disease progre-ssion[C] // AMIA Annual Symposium Proceedings, 2012: 716-725.
[22]	Zhao J, Liu Z C, Dontcheva M, et al. MatrixWave: visual comparison of event sequence data[C]. Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. New York: ACM Press, 2015: 259-268.