Artificial Intelligence: Developments and Advances in 2019

doi:10.11871/jfdc.issn.2096-742X.2019.02.001

Abstract

Abstract:

[Objective] Artificial Intelligence (AI) has become a hot technology topic in the society. This paper aims to provide updated information of AI developments and advances to government, research community, industry and public. [Methods] This paper mainly reviews AI news and scientific publications in 2019 to summarize the overall technology progresses of AI. [Results] The AI community has become more rational and pragmatic relatively in 2019. On one hand, AI technologies are getting mature with more and more successful applications in the era of Internet of Things, Big Data, Super-computing, and Deep Learning. On the other hand, there are more and more promising and new directions in AI researches such as reinforcement learning, automatic machine learning, brain-machine interface, brain-like intelligence, and interpretable artificial intelligence. [Limitations] This paper mainly introduces the current development status of AI with limited efforts in prediction of future AI directions. [Conclusions] Great progress has been achieved in the areas of neuromorphic chips, brain-machine interface, human-computer game, intelligent robot, natural language understanding, and computer vision in 2019. But it is still a challenging task towards general AI. Technology breakthroughs are needed to achieve a new generation of AI.

Key words: artificial intelligence, computer vision, neuromorphic chips, brain-machine integrated intelligence, intelligent robot

Sun Zhenan,Zhang Zhaoxiang,Wang Wei,Liu Fei,Tan Tieniu. Artificial Intelligence: Developments and Advances in 2019[J]. Frontiers of Data and Computing, 2019, 1(2): 1-16.

References 90

[1]	Gartner. 2019世界人工智能技术趋势分析报告[R]. 2019.8.
[2]	中国信息通信研究院, 中国人工智能产业发展联盟, 全球人工智能战略与政策观察（2019）[R]. 2019.8.
[3]	Intel’s Pohoiki Beach, a 64-Chip Neuromorphic System, Delivers Breakthrough Results in Research Tests [EB/OL]. [2019-7-15].https://newsroom.intel.com/news/intels-pohoiki-beach-64-chip-neuromorphic-system-delivers-breakthrough-results-research-tests/#gs.1pqqma.
[4]	Jing Pei, Lei Deng, Sen Song, Mingguo Zhao, Youhui Zhang, Shuang Wu, Guanrui Wang, Zhe Zou, Zhenzhi Wu, Wei He, Feng Chen, Ning Deng, Si Wu, Yu Wang, Yujie Wu, Zheyu Yang, Cheng Ma, Guoqi Li, Wentao Han, Huanglong Li, Huaqiang Wu, Rong Zhao, Yuan Xie, Luping Shi . Towards artificial general intelligence with hybrid Tianjic chip architecture[J]. Nature, 2019,572:106-111.
[5]	Elon Musk , Neurolink. An integrated brain-machine interface platform with thousands of channels[J]. bioRxiv 703801, 2019. Doi: https://doi.org/10.1101/703801.
[6]	Gopala K, Anumanchipalli, Josh Chartier, Edward F. Chang . Speech synjournal for neural decoding of spoken sentences[J]. Nature, 2019,568:493-498.
[7]	David A. Moses, Matthew K. Leonard, Joseph G. Makin, Edward F. Chang . Real-time decoding of question-and-answer speech dialogue using human cortical activity[J]. Nature Communications, 2019, 10, Article number: 3096.
[8]	Nathan Benaich and Ian Hogarth . State of AI 2019. https://www.stateof.ai/.
[9]	Noam Brown, Tuomas Sandholm . Superhuman AI for multiplayer poker[J]. Sciences, 2019,365(6456):885-890.
[10]	Guang-Zhong Yang, Robert J. Full, Neil Jacobstein, Peer Fischer, James Bellingham, Howie Choset, Henrik Christensen, Paolo Dario, Bradley J. Nelson, Russell Taylor. Ten robotics technologies of the year[J]. Science Robotics, 2019,4(26).
[11]	Atlas [EB/OL]. [2019].https://www.bostondynamics.com/atlas.
[12]	Shuguang Li, Richa Batra, David Brown, Hyun-Dong Chang, Nikhil Ranganathan, Chuck Hoberman, Daniela Rus, Hod Lipson . Particle robotics based on statistical mechanics of loosely coupled components[J]. Nature, 2019,567:361-365.
[13]	Zhenishbek Zhakypov, Kazuaki Mori, Koh Hosoda, Jamie Paik . Designing minimal and scalable insect-inspired multi-locomotion millirobots[J]. Nature, 2019,571:381-386.
[14]	Subramanian Sundaram, Petr Kellnhofer, Yunzhu Li, Jun-Yan Zhu, Antonio Torralba, Wojciech Matusik . Learning the signatures of the human grasp using a scalable tactile glove[J]. Nature, 2019,569:698-702.
[15]	Noah T. Jafferis E. Farrell Helbling, Michael Karpelson, Robert J. Wood . Untethered flight of an insect-sized flapping-wing microscale aerial vehicle[J]. Nature, 2019,570:491-495.
[16]	Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova . BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding[C]. https://arxiv.org/abs/1810.04805, 2018.
[17]	Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Ruslan Salakhutdinov, Quoc V. Le . XLNet: Generalized Autoregressive Pretraining for Language Understanding[C]. https://arxiv.org/abs/1906.08237, 2019.
[18]	Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, Veselin Stoyanov . RoBERTa: A Robustly Optimized BERT Pretraining Approach[C]. https://arxiv.org/abs/1907.11692, 2019.
[19]	Yu Sun, Shuohuan Wang, Yukun Li, Shikun Feng, Hao Tian, Hua Wu, Haifeng Wang . ERNIE 2.0: A Continual Pre-training Framework for Language Understanding[C]. https://arxiv.org/abs/1907.12412, 2019.
[20]	S. Xie, Z. Tu . Holistically-Nested Edge Detection[C]. International conference on computer vision (ICCV), 2015: 1395-1403.
[21]	J. Long, E. Shelhamer, T. Darrell . Fully Convolutional Networks for Semantic Segmentation[C]. International conference on computer vision and pattern recognition (CVPR), 2015: 3431-3440.
[22]	K. He, X. Zhang, S. Ren, J. Sun . Residual Learning for Image Recognition[C]. International conference on computer vision and pattern recognition (CVPR), 2016: 770-778.
[23]	R. Girshick, J. Donahue, T. Darrell, J. Malik . Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation[C]. International conference on computer vision and pattern recognition (CVPR), 2014: 580-587.
[24]	C. Lu, R. Krishna, M. Bernstein, Fei-Fei Li . Visual Relationship Detection with Language Priors[C]. European conference on computer vision (ECCV), 2016: 852-869.
[25]	K. Simonyan, A. Zisserman . Two-stream convolutional networks for action recognition in videos[C]. Neural information processing systems (NIPS), 2014: 568-576.
[26]	C. Si, W. Chen, W. Wang, L. Wang, T. Tan . An Attention Enhanced Graph Convolutional LSTM Network for Skeleton-Based Action Recognition[C]. International conference on computer vision and pattern recognition (CVPR), 2019: 1227-1236.
[27]	W. Wang, Y. Huang, L. Wang . Language-Driven Temporal Activity Localization: A Semantic Matching Reinforcement Learning Model[C]. International conference on computer vision and pattern recognition (CVPR), 2019: 334-343.
[28]	Hang Su, Varun Jampani, Deqing Sun, Subhransu Maji, Evangelos Kalogerakis, Ming-Hsuan Yang, Jan Kautz . SPLATNet: Sparse Lattice Networks for Point Cloud Processing[C]. International conference on computer vision and pattern recognition (CVPR), 2018: 2530-2539.
[29]	Xu K, Ba J, Kiros R, et al. Show, Attend and Tell: Neural Image Caption Generation with Visual Attention[C]. International conference on machine learning (ICML), 2015: 2048-2057.
[30]	J. Wang, W. Wang, Y. Huang, L. Wanng, T. Tan . M3: Multimodal Memory Modelling for Video Captioning[C]. International conference on computer vision and pattern recognition (CVPR), 2018: 7512-7520.
[31]	Y. Aystar et al. SoundNet: Learning Sound Represen-tations from Unlabeled Video[C]. Neural information processing systems (NIPS), 2016: 892-900.
[32]	M.A. Lee, Y. Zhu, K. Srinivasan, P. Shah, S. Savarese, Li Fei-Fei, A. Garg, J. Bohg . Making Sense of Vision and Touch: Self-Supervised Learning of Multimodal Representations for Contact-Rich Tasks[C]. International conference on robotics and automation (ICRA), 2019.
[33]	M. Savva . et al. Habitat: A Platform for Embodied AI Research[C]. International conference on computer vision (ICCV), 2019.
[34]	X. Wang . et al. Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigation[C]. International conference on computer vision and pattern recognition (CVPR), 2018: 6629-6638.
[35]	Goodfellow I J, Pougetabadie J, Mirza M, et al. Generative Adversarial Nets[C]. Neural information processing systems (NIPS), 2014: 2672-2680.
[36]	Salimans T, Goodfellow I J, Zaremba W, et al. Improved techniques for training GANs[C]. Neural information processing systems (NIPS), 2016: 2234-2242.
[37]	Isola P, Zhu J, Zhou T, et al. Image-to-Image Translation with Conditional Adversarial Networks[C]. International conference on computer vision and pattern recognition (CVPR), 2017: 5967-5976.
[38]	Ho J, Ermon S . Generative Adversarial Imitation Learning[C]. Neural information processing systems (NIPS), 2016: 4565-4573.
[39]	Arjovsky M, Chintala S, Bottou L, et al. Wasserstein Generative Adversarial Networks[C]. International conference on machine learning (ICML), 2017: 214-223.
[40]	Zhao J., Mathieu M., LeCun Y. Energy-based generative adversarial network[C]. https://arxiv.org/abs/1609.03126, 2016.
[41]	Zhu J, Park T, Isola P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks[C]. International conference on computer vision (ICCV), 2017: 2242-2251.
[42]	Heusel M, Ramsauer H, Unterthiner T, et al. GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium[C]. Neural information processing systems (NIPS), 2017: 6626-6637.
[43]	Huang H, Li Z, He R, et al. IntroVAE: Introspective Variational Autoencoders for Photographic Image Synconfproc[C]. Neural information processing systems (NIPS), 2018: 52-63.
[44]	Huang H, He R, Sun Z , et al. Wavelet Domain Generative Adversarial Network for Multi-scale Face Hallucination[J]. International Journal of Computer Vision (IJCV), 2019: 763-784.
[45]	Cao J, Hu Y, Zhang H, et al. Learning a High Fidelity Pose Invariant Model for High-resolution Face Frontalization[C]. Neural information processing systems (NIPS), 2018: 2867-2877.
[46]	Jie Cao, Yibo Hu, Hongwen Zhang, Ran He, Zhenan Sun. Towards High Fidelity Face Frontalization in the Wild[J], International Journal of Computer Vision (IJCV), 2019, Accepted.
[47]	Huang H., He R., Sun Z., Tan T . Wavelet Domain Generative Adversarial Network for Multi-scale Face Hallucination[J]. International Journal of Computer Vision (IJCV), 2019,127(6-7):763-784.
[48]	Liu Y., Li Q., & Sun Z . Attribute-Aware Face Aging with Wavelet-Based Generative Adversarial Networks[C]. IEEE Conference on computer vision and pattern recognition (CVPR), 2019: 11877-11886.
[49]	Srivastava N, Hinton G E, Krizhevsky A , et al. Dropout: a simple way to prevent neural networks from overfitting[J]. Journal of Machine Learning Research, 2014,15(1):1929-1958.
[50]	Zhao G, Wang J, Zhang Z, et al. Random Shifting for CNN: A solution to reduce information loss in down-sampling layers[C]. International joint conference on artificial intelligence (IJCAI), 2017: 3476-3482.
[51]	Sabour S, Frosst N, Hinton G E, et al. Dynamic Routing Between Capsules[C]. Neural information processing systems (NIPS), 2017: 3856-3866.
[52]	Zhu G. Zhang Z. Zhang X. Liu C. . Diverse neuron type selection for convolutional neural networks[C]. International Joint Conference on Artificial Intelligence (AAAI), 2017: 3560-3566.
[53]	Zhang X S. Gao S. B. Li R X. Du X. Y. Li C. Y. Li Y. J. A retinal mechanism inspired color constancy model[J]. IEEE Transactions on Image Processing (TIP), 2016,25(3):1219-1232.
[54]	Li Y. Li C . A color-opponency based biological model for color constancy. I -Perception, 2011,2(4):384-384.
[55]	Gao S. Li Y . A retinal mechanism based color constancy model[C]. Chinese Conference on Pattern Recognition, 2012: 422-429.
[56]	Li Y. Chen Y. Wang N. Zhang Z. Scale-aware trident networks for object detection[C]. International conference on computer vision (ICCV), 2019.
[57]	Jim M. David L. G . Multiclass object recognition with sparse, localized features[C]. International Conference on computer vision and pattern recognition (CVPR), 2006: 11-18.
[58]	Wallis G Rolls E. T . A model of invariant object recognition in the visual system[J]. Progress in Neurobiology, 1997,51, 167-194.
[59]	LeCun Y. Boser B. Denker J. S. Backpropagation applied to handwritten zip code recognition[J]. Neural computation, 1989,1(4):541-551.
[60]	Krizhevsky A. Sutskever I. Hinton G . Imagenet classification with deep convolutional neural networks[C]. Neural information processing systems (NIPS), 2012,141(5):1097-1105.
[61]	Simonyan K, Zisserman A . Very Deep Convolutional Networks for Large-Scale Image Recognition[C]. International conference on learning representations (ICLR), 2015.
[62]	Szegedy C. Liu W. Jia Y. Sermanet P. Reed S. Anguelov D. Erhan D. Vanhoucke V. Rabinovich A . Going deeper with convolutions[C]. International Conference on computer vision and pattern recognition (CVPR), 2015: 1-9.
[63]	Shi T. Liang M. Hu X . A reverse hierarchy model for predicting eye fixations[C]. International conference on computer vision and pattern recognition (CVPR), 2014: 2822-2829.
[64]	Hochstein S, Ahissar M . View from the Top: Hierarchies and Reverse Hierarchies in the Visual System[J]. Neuron, 2002,36(5):791-804.
[65]	Liang M. Hu X . Recurrent convolutional neural network for object recognition[C]. International Conference on computer vision and pattern recognition (CVPR), 2015: 3367-3375.
[66]	Yoo D. Park S. Lee J Y. Paek A S. So Kweon I . Attentionnet: Aggregating weak directions for accurate object detection[C]. International conference on computer vision and pattern recognition (CVPR), 2015: 2659-2667.
[67]	En Q. Duan L. Zhang Z. Bai X. Zhang Y. Human-Like Delicate Region Erasing Strategy for Weakly Supervised Detection[C]. National Conference on Artificial Intelligence (AAAI), 2019,33:3502-3509.
[68]	Fang Y. Ma Z. Zhang Z . Dynamic multi-task learning with convolutional neural network[C]. International joint conference on artificial intelligence (IJCAI), 2017: 19-25.
[69]	Hao W. Zhang Z. Guan H . Integrating both Visual and Audio Cues for Enhanced Video Caption[C]. National Conference on Artificial Intelligence (AAAI), 2018: 6894-6901.
[70]	Hao W. Zhang Z. Guan H . CMCGAN: A Uniform Framework for Cross-Modal Visual-Audio Mutual Generation[C]. National Conference on Artificial Intelligence (AAAI), 2018: 6886-6893.
[71]	Dong W. Zhang Z. Tan T . Attention-Aware Sampling via Deep Reinforcement Learning for Action Recognition [C]. National Conference on Artificial Intelligence (AAAI), 2019,33:8247-8254.
[72]	Yang H. M. Zhang X. Y. Yin F. Liu C. L. Robust Classification with Convolutional Prototype Learning [C]. International conference on computer vision and pattern recognition, 2018: 3474-3482.
[73]	Chen Y, Wang N, Zhang Z . DarkRank: Accelerating Deep Metric Learning via Cross Sample Similarities Transfer[C]. National Conference on Artificial Intelligence (AAAI), 2018: 2852-2859.
[74]	Rusu A. A. Rabinowitz N. C. Desjardins G. Soyer H. Kirkpatrick J. Kavukcuoglu K. Hadsell R. Progressive neural networks[C]. arXiv preprint. arXiv:1606.04671, 2016.
[75]	Fernando C. Banarse D. Blundell C. Zwols Y. Ha D. Rusu A. A. Wierstra D. Pathnet: Evolution channels gradient descent in super neural networks[C]. arXiv preprint. arXiv:1701.08734, 2017.
[76]	Kirkpatrick J, Pascanu R, Rabinowitz N C , et al. Overcoming catastrophic forgetting in neural networks[J]. Proceedings of the National Academy of Sciences of the United States of America, 2017,114(13):3521-3526.
[77]	Lake B M. Salakhutdinov R. Tenenbaum J B. Human-level concept learning through probabilistic program induction[J]. Science, 2015,350(6266):1332-1338.
[78]	Friston K J, Adams R A, Perrinet L U , et al. Perceptions as Hypotheses: Saccades as Experiments[J]. Frontiers in Psychology, 2012: 151.
[79]	Lu Y. Yin J. Chen Z. Gong H. Liu Y. Qian L. Li X. Liu R. Andolina IM. Wang W. Revealing Detail along the Visual Hierarchy: Neural Clustering Preserves Acuity from V1 to V4[J]. Neuron, 2018,98(2):417-428.
[80]	Groen I. Silson E. Baker C. Contributions of low- and high-level properties to neural processing of visual scenes in the human brain[J]. Philosophical Transactions of the Royal Society B, 2017: 1-11.
[81]	Mackey W E, Winawer J, Curtis C E , et al. Visual field map clusters in human frontoparietal cortex[J]. eLife, 2017.
[82]	Markov N. Misery P. Falchier A. Lamy C. Vezoli J. Quilodran R. Gariel MA. Giroud P. Ercsey-Ravasz M. Pilaz LJ. Huissoud C. Barone P. Dehay C. Toroczkai Z. Van Essen DC. Kennedy H. Knoblauch K. Weight consistency specifies regularities of macaque cortical networks. Cerebral Cortex, 2011,21(6):1254-1272.
[83]	Lotter W. Kreiman G. Cox D . A neural network trained to predict future video frames mimics critical properties of biological neuronal responses and perception[C]. arXiv preprint. arXiv:1805.10734, 2018.
[84]	O’Reilly R. C. Wyatte D. R. Rohrlich J . Deep Predictive Learning: A Comprehensive Model of Three Visual Streams[C]. arXiv preprint. arXiv:1709.04654, 2017.
[85]	Chong E, Familiar A, Shim W M , et al. Reconstructing representations of dynamic visual objects in early visual cortex[J]. Proceedings of the National Academy of Sciences of the United States of America, 2016,113(5):1453-1458.
[86]	Lawrence S. van Mourik T. Kok P. Koopmans P. Norris D. de Lange F . Laminar Organization of Working Memory Signals in Human Visual Cortex. Current biology, CB, 2018,28(21):3435-3440.
[87]	Petro L S, Paton A, Muckli L , et al. Contextual modulation of primary visual cortex by auditory signals[J]. Philosophical Transactions of the Royal Society B, 2017,372(1714).
[88]	Williams M. Baker C. Op de Beeck H. Mok Shim W. Dang S. Triantafyllou C Kanwisher N . Feedback of visual object information to foveal retinotopic cortex[J]. Nature Neuroscience, 2008,11(12):1439-1445.
[89]	Wyatte D. Curran T. O ˛a´ rReilly R . The limits of feedforward vision: recurrent processing promotes robust object recognition when objects are degraded[J]. Journal of Cognitive Neuroscience, 2012,24(11), 2248-2261.
[90]	Roelfsema P. de Lange F . Early Visual Cortex as a Multiscale Cognitive Blackboard [R]. Annual Review of Vision Science, 2016,2(1):131-151.