A Brief Review of Theory and Systematic Technologies for Big Data

doi:10.11871/jfdc.issn.2096.742X.2019.01.004

Abstract

Abstract:

[Objective] The article mainly gives a brief review for big data theory and systems, including the research background, the technical architecture and the key technologies following by estimating future research directions. [Method] On the basis of the brief introduction of the big data processing theory, this paper introduces the key technologies for big data systems by the three aspects: the data parallel processing methods, the Resource Description Framework (RDF) graph data query and matching, and the big data analysis technologies. [Results] The speed of data generation will be accelerated further more in near future, thus how to quickly process the data on the edge side would lead a research trend. [Conclusion] In the future, developing new technologies for big data theory and systems still warrant further attention, on which the researches on data processing by edge computing and fog computing in the scenario of Internet of Things era highlight based.

Key words: low complexity algorithms, data parallelism technologies, QoS mechanism, graph data processing, language model

Qiangsheng Hua,Zhigao Zheng,Zhenyu Hu,Zhiman Zhong,Changfu Lin,Feng Zhao,Hai Jin,Xuanhua Shi. A Brief Review of Theory and Systematic Technologies for Big Data[J]. Frontiers of Data and Computing, 2019, 1(1): 22-34.

Figures/Tables 7

Fig. 1

Fig. 2

Fig. 3

Fig. 4

Fig. 5.

Fig. 6

Fig. 7

References 17

[1]	. Xuhui Gong, Qiang-Sheng Hua, Lixiang Qian, Dongxiao Yu, Hai Jin . Communication-Efficient and Privacy-Preserving Data Aggregation without Trusted Authority . In proceedings of 2018 IEEE Conference on Computer Communications (INFOCOM 2018), Honolulu, United States, 2018, pp. 1250-1258.
[2]	. Fei Wang, Xuanhua Shi, Dongxiao Yu, Zhixiang Ke, Hai Jin, Song Wu . SDPA: An Optimizer for Program Analysis of Data-Parallel Applications . In proceedings of the 20th International Conference on High Performance Computing and Communications; IEEE 16th International Conference on Smart City; IEEE 4th International Conference on Data Science and Systems (HPCC/SmartCity/DSS 2018), Exeter, United Kingdom, 2018, pp. 14-21.
[3]	. Gaofeng Li, Pingpeng Yuan, Hai Jin . Svega: Answering Natural Language Question over Knowledge Base with Semantic Matching. In Proceedings of The 30th International Conference on Software Engineering and Knowledge Engineering (SEKE 2018), California, United States, 2018, pp. 615-616.
[4]	. Wang Tao , Pingpeng Yuan, Xiaofei Liao and Hai Jin. Parallel Processing SPARQL Theta Join on Large Scale RDF Graphs. In Proceedings of the 2018 IEEE Global Communications Conference (GLOBECOM). Abu Dhabi, United Arab Emirates, 2018, pp. 1-6.
[5]	. Changfu Lin, Jingjing Zhan, Hanhua Chen, Jie Tan, Hai Jin . Ares: A High Performance and Fault-tolerant Distributed Stream Processing System. In Proceedings of the 26 ^th IEEE International Conference on Network Protocols (ICNP 2018), Cambridge, United Kingdom, 2018, pp. 176-186.
[6]	. Feng Zhao, Zeliang Tian and Hai Jin . Entity-Based Language Model Smoothing Approach for Smart Search. IEEE Access, vol. 6, pp. 9991-10002, 2018.
[7]	. Yusheng Hua, Xuanhua Shi, Hai Jin, Wei Liu, Yan Jiang, Yong Chen, Ligang He . Software-defined QoS for I/O in exascale computing. CCF Transactions on High Performance Computing, Vol. 1, No. 1, pp 49-59.
[8]	. Hao Long, Pingpeng Yuan and Hai Jin . Distributed PathGraph: A Cluster Centric Framework for Distributed Processing Graph. In proceedings of the 10th IEEE Conference on Service-Oriented Computing and Applications (SOCA 2017), Kanazawa, Japan, 2017, pp. 34-41. 2017.
[9]	. Xuanhua Shi, Xuan Luo, Junling Liang, Peng Zhao, Sheng Di, Bingsheng He, Hai Jin . Frog: Asynchronous Graph Processing on GPU with Hybrid Coloring Model. IEEE Transactions on Knowledge and Data Engineering (TKDE), vol. 30, no. 1, pp. 29-42, 1 Jan. 2018.
[10]	. Xuanhua Shi, Zhigao Zheng, Yongluan Zhou, Hai Jin, Ligang He, Bo Liu, Qiang-Sheng Hua . Graph Processing on GPUs: A Survey. ACM Computing Surveys. Vol.50, No. 6, pp. 1-35, 2018.
[11]	. Pingpeng Yuan, Pu Liu, Buwen Wu, Hai Jin, Wenya Zhang, Ling Liu . TripleBit: a Fast and Compact System for Large Scale RDF Data. PVLDB, 6(7):517-528, 2013.
[12]	. Mazzeo Giuseppe, Carlo Zaniolo . Answering Controlled Natural Language Questions on RDF Knowledge Bases. In Proceedings of the 19th International Conference on Extending Database Technology (EDBT 2016), March 15-18, 2016, Bordeaux, France. pp. 608-611.
[13]	. Fangjia Xing, Liming Gui, Hanhua Chen, Changfu Lin, Hai Jin . Efficient Event Stream Dissemination in Online Social Networks Based on Community Detection. In Proceedings of the 2018 IEEE International Conference on Communications (ICC), Kansas City, United States, 2018, pp. 1-6.
[14]	. Sijie Wu, Hanhua Chen, Changfu Lin, Hai Jin . Shadow: Exploiting the Power of Choice for Efficient Shuffling in MapReduce. In Proceedings of the 23rd IEEE International Conference on Parallel and Distributed Systems (ICPADS 2017), Shenzhen, China, 2017, pp. 553-560.
[15]	. Fan Zhang, Hanhua Chen, Hai Jin . Piggyback Game: Efficient Event Stream Dissemination in Online Social Network Systems. IEEE Transactions on Parallel and Distributed Systems(TPDS), vol. 30, no. 3, pp. 692-709.
[16]	. Hanhua Chen, Fan Zhang, Hai Jin . Popularity-aware Differentiated Distributed Stream Processing on Skewed Streams, In Proceedings of the 25 ^th IEEE International Conference on Network Protocols (ICNP 2017), Toronto, Canada, 2017, pp. 1-10.
[17]	. Hanhua Chen, Liangyi Liao, Hai Jin, Jie Wu . The Dynamic Cuckoo Filter. In Proceedings of the 25 ^th IEEE International Conference on Network Protocols (ICNP 2017) , Toronto, Canada, 2017, pp. 1-10.