智能语音系统安全分析

doi:10.11871/jfdc.issn.2096-742X.2019.02.009

数据与计算发展前沿 ›› 2019, Vol. 1 ›› Issue (2): 98-109.

doi: 10.11871/jfdc.issn.2096-742X.2019.02.009

所属专题： “人工智能”专刊

智能语音系统安全分析

张笑宇^1,²,沈超^1,^2,^*(),陈宇飞^1,²,吴星辉^1,²,刘畅^1,²

1. 西安交通大学,智能网络与网络安全教育部重点实验室,陕西西安 710049
2. 西安交通大学,电子与信息工程学部,陕西西安 710049

收稿日期:2019-09-17 出版日期:2019-12-20 发布日期:2020-01-15
通讯作者: 沈超
作者简介:张笑宇,1999年生,西安交通大学电子与信息工程学部本科生,主要研究方向为智能语音系统安全。
本文主要承担工作为文献的整理、查阅及文章主要内容的撰写。
Zhang Xiaoyu, born in 1999, is currently an undergraduate student of Faculty of Electronic and Information Engineering, Xi’an Jiaotong University. His main research interests include security of intelligent acoustic systems.
His main contributions to this paper include collecting and summarizing references, and writing the main part of the paper.
E-mail: zxy0927@stu.xjtu.edu.cn|沈超,1985年生,西安交通大学电子与信息工程学部教授,博士生导师。主要研究方向为信息物理融合系统优化与安全,网络与系统安全,以及人工智能安全。
本文主要承担工作为文章的撰写与修改。
Shen Chao, born in 1985, PhD and PhD supervisor, is a professor of Faculty of Electronic and Information Engineering, Xi’an Jiaotong University. His main research interests include cyber-physical system optimization and security, network and system security, and artificial intelligence security. His main contributions to this paper include paper writing and revising.
Chao Shen is the corresponding author of this paper.|陈宇飞,1994年生,西安交通大学电子信息与工程学部博士研究生。主要研究方向为人工智能安全,人机交互分析。
本文主要承担工作为文章的修改。
Chen Yufei, born in 1994, is a PhD student of Faculty of Electronic and Information Engineering. His main research interests include security of intelligent systems and behavioral analysis.
His main contributions to this paper is paper revising.
E-mail:yfchen@sei.xjtu.edu.cn|吴星辉,1998年生,西安交通大学电子与信息工程学部本科生。主要研究方向为智能语音系统安全。
本文主要承担工作为文章的修改。
Wu Xinghui, born in 1998, is an undergraduate student of Faculty of Electronic and Information Engineering, Xi’an Jiaotong University. His main research interests include security of intelligent acoustic systems.
His main contributions to this paper include paper revising.
E-mail:wuxh994@stu.xjtu.edu.cn|刘畅,1995年生,西安交通大学电子信息与工程学部研究生。主要研究方向为人工智能安全,人机交互分析。
本文主要承担工作为文章的修改。
Liu Chang, born in 1995, is a PhD student Faculty of Electronic and Information Engineering. His main research interests include security of intelligent systems and behavioral analysis.
His main contributions to this paper include paper revising.
E-mail:kakeru@stu.xjtu.edu.cn
基金资助:
国家自然科学基金项目(61822309);国家自然科学基金项目(61773310);国家自然科学基金项目(U1736205)

An Analysis of Intelligent Acoustic System Security

Zhang Xiaoyu^1,²,Shen Chao^1,^2,^*(),Chen Yufei^1,²,Wu Xinghui^1,²,Liu Chang^1,²

1. Ministry of Education Key Laboratory for Intelligent Networks and Network Security, Xi’an Jiaotong University, Xi’an,Shanxi 710049, China;
2. Faculty of Electronic and Information Engineering, Xi’an Jiaotong University, Xi’an, Shaanxi 710049, China;

Received:2019-09-17 Online:2019-12-20 Published:2020-01-15
Contact: Shen Chao

摘要/Abstract

摘要：

【目的】智能语音系统的发展推动着人类生活的进步,人机之间实现便捷沟通的同时也暴露出了各种安全问题。本文对智能语音系统安全的研究现状进行全面的介绍,为中国相关研究工作开展提供参考。【文献范围】参考文献选自人工智能系统安全领域的国内外期刊和会议,总计50余篇。【方法】通过对智能语音系统安全相关研究进行介绍,从智能语音系统的各个环节入手,分析了该领域国内外研究现状以及未来的研究趋势。【结果】针对智能语音系统安全的研究已经取得了相当的成果,但整体处在初级阶段,仍有方向等待探索,仍有难题等待解决。【局限】语音系统安全近年在飞速发展,受限于获取资料的时效性,文中部分研究可能已有新的进展。【结论】关于语音系统安全的研究具备高度的实用价值,但现有的研究在研究角度、实用性等方面存在局限,仍存在大量的空白区域等待学者们进一步探索、研究。

关键词: 智能语音系统, 信息安全, 隐私保护

Abstract:

[Objective] The rapid development of intelligent acoustic systems has remarkably promoted the progress of human life. While the convenient communication is being build between human and machine, it has also exposed various security issues, which can greatly threaten users’ privacy, property security, and even personal safety. This paper makes a comprehensive introduction to the research status of intelligent acoustic system security and provides references for the related research work in China. [Coverage] The references are selected from domestic and international journals or conferences in the field of artificial intelligence system security, with a total of more than 50 articles. [Methods] We firstly establish the security model of the intelligent acoustic system and analyze the purposes and abilities of attackers. Then, we systematically summarize the existing acoustic system security studies into five topics, which are input components, pre-processing components, machine learning models, output components and actual implementation frameworks. Finally, we discuss future research trends. Particularly, we point out potential research directions of pre-processing and output components, which may need more attention. [Results] The researches about intelligent acoustic system security have achieved considerable results, but is still at the initial stage in all, which means there are more directions to be explored, and problems to be solved. [Limitations] Acoustic system security has developed rapidly in recent years. This article is limited by the timeliness of access to data, some included researches may have new progress. [Conclusions] The researches on speech system security have high practical value. However, the existing researches have limitations in research focus points and practicality. There are still a lot of blank areas awaiting for scholars to further explore and study.

Key words: intelligent acoustic system, information security, privacy protection

张笑宇,沈超,陈宇飞,吴星辉,刘畅. 智能语音系统安全分析[J]. 数据与计算发展前沿, 2019, 1(2): 98-109.

Zhang Xiaoyu,Shen Chao,Chen Yufei,Wu Xinghui,Liu Chang. An Analysis of Intelligent Acoustic System Security[J]. Frontiers of Data and Computing, 2019, 1(2): 98-109.

图/表 1

参考文献 56

[1]	詹新明, 黄南山, 杨灿 . 语音识别技术研究进展[J]. 现代计算机(专业版), 2008,24(9):43-45.
[2]	Sovie D, Mcmahon L, Murdoch R , et al. Reshape to relevance[R\OL]., Accenture, 2019.[2019-10-20]. https://www.accenture.com/_acnmedia/pdf-93/accenture-digital-consumer-2019-reshape-to-relevance.pdf.
[3]	Vanian. J, Pressman. A , How Amazon,Apple, Google, and Microsoft Created an Eavesdropping Explosion—Data Sheet [N\OL].3rd ed. Fortune,2019.8.8.[2019-10-20].https://fortune.com/2019/08/08/gogole-amazon-microsoft-listen-conversation-siri.
[4]	Andriotis. A, Stevens. L . 亚马逊考虑借助Alexa进入支付领域[N\OL].华尔街日报,2018-4-6.[2019-10-20].https://cn.wsj.com/articles/CN-TEC-20180406192723.
[5]	Papernot N, McDaniel P, Sinha A , et al. Towards the science of security and privacy in machine learning[J]. arXiv preprint arXiv:1611.03814, 2016.
[6]	Kasmi C, Esteves J L . IEMI Threats for Information Security: Remote Command Injection on Modern Smartphones[J]. IEEE Transactions on Electromagnetic Compatibility, 2015,57(6):1752-1755.
[7]	Roy N, Hassanieh H, Roy Choudhury R . Backdoor: Making microphones hear inaudible sounds[C]. Proceedings of the 15th Annual International Conference on Mobile Systems, Applications, and Services. ACM, 2017: 214.
[8]	Zhang G, Yan C, Ji X, et al. Dolphinattack: Inaudible voice commands[C]. Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security. ACM, 2017: 103-117
[9]	Song L, Mittal P . Inaudible voice commands[J]. arXiv preprint arXiv:1708.07238, 2017.
[10]	Roy N, Shen S, Hassanieh H, et al. Inaudible voice commands: The long-range attack and defense [C]// 15th {USENIX} Symposium on Networked Systems Design and Implementation( {NSDI} 18). 2018: 547-560.
[11]	Michalevsky Y, Nakibly G, Nakibly G . Gyrophone: recognizing speech from gyroscope signals [C]// Usenix Conference on Security Symposium. USENIX Association, 2014.
[12]	Schlegel R, Zhang K, Zhou X, et al. Soundcomber: A Stealthy and Context-Aware Sound Trojan for Smartphones [C]// NDSS. 2011,11:17-33.
[13]	Dalvi N, Domingos P, Sanghai S, et al. Adversarial classification[C]. Proceedings of the 10th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Seattle, Aug 22-25, 2004. New York: ACM, 2004: 99-108.
[14]	Lowd D, Meek C . Adversarial learning[C]. Proceedings of the 11th ACM SIGKDD International Conference on Knowledge Discovery in Data Mining, Chicago, Aug 21-24, 2005. New York: ACM, 2005: 641-647.
[15]	Huang L, Joseph A D, Nelson B, et al. Adversarial machine learning[C]. Proceedings of the 4th ACM Workshop on Security and Artificial Intelligence, Chicago, Oct 21, 2011. New York: ACM, 2011: 43-58.
[16]	Sharif M, Bhagavatula S, Bauer L, et al. Accessorize to a Crime: Real and Stealthy Attacks on State-of-the-Art Face Recognition [C]// the 2016 ACM SIGSAC Conference. ACM, 2016.
[17]	Su J, Vargas D V, Kouichi S . One pixel attack for fooling deep neural networks[J]. IEEE Transactions on Evolutionary Computation, 2017.
[18]	Szegedy C, Zaremba W, Sutskever I , et al. Intriguing properties of neural networks[J]. Computer Science, 2013.
[19]	Kurakin A, Goodfellow I, Bengio S . Adversarial examples in the physical world[J]. 2016.
[20]	Nguyen A, Yosinski J, Clune J . Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images[J]. 2014.
[21]	Papernot N, Mcdaniel P, Jha S , et al. The Limitations of Deep Learning in Adversarial Settings[J]. 2015.
[22]	Nikiforakis N, Balduzzi M, Desmet L , et al. Soundsquatting: Uncovering the Use of Homophones in Domain Squatting[M] // Information Security. Springer International Publishing, 2014.
[23]	Kumar D, Paccagnella R, Murley P, et al. Skill squatting attacks on amazon alexa[C]. 27th USENIX Security Symposium (USENIX Security 18). 2018: 33-47.
[24]	Zhang N, Mi X, Feng X, et al. Dangerous skills: Understanding and mitigating security risks of voice-controlled third-party functions on virtual personal assistant systems[C]. Dangerous Skills: Understanding and Mitigating Security Risks of Voice-Controlled Third-Party Functions on Virtual Personal Assistant Systems. IEEE, 2019.
[25]	Bispham M K, Agrafiotis I, Goldsmith M . Nonsense attacks on Google Assistant and missense attacks on Amazon Alexa[J]. 2018.
[26]	Zhang Y, Xu L, Mendoza A, et al. Life After Speech Recognition: Fuzzing Semantic Misinterpretation for Voice Assistant Applications[C]. NDSS 2019.
[27]	Abdullah H, Garcia W, Peeters C , et al. Practical Hidden Voice Attacks against Speech and Speaker Recognition Systems[J]. arXiv preprint arXiv:1904.05734, 2019.
[28]	Alzantot M, Balaji B, Srivastava M . Did you hear that? Adversarial Examples Against Automatic Speech Recognition[J]. 2018.
[29]	Neupane A, Saxena N, Hirshfield L M, et al. The Crux of Voice (In) Security: A Brain Study of Speaker Legitimacy Detection [C]// NDSS. 2019.
[30]	Yuan X, Chen Y, Zhao Y, et al. Commandersong: A systematic approach for practical adversarial voice recognition[C]. 27th USENIX Security Symposium (USENIX Security 18). 2018: 49-64.
[31]	Zhou M, Qin Z, Lin X , et al. Hidden Voice Commands: Attacks and Defenses on the VCS of Autonomous Driving Cars[J]. IEEE Wireless Communications, 2019.
[32]	Vaidya T, Zhang Y, Sherr M, et al. Cocaine noodles: exploiting the gap between human and machine speech recognition [C]// Usenix Conference on Offensive Technologies. USENIX Association, 2015.
[33]	Carlini N, Mishra P, Vaidya T, et al. Hidden voice commands [C]// 25th {USENIX} Security Symposium ({USENIX} Security 16). 2016: 513-530.
[34]	Carlini N, Wagner D . Audio adversarial examples: Targeted attacks on speech-to-text[C]. 2018 IEEE Security and Privacy Workshops (SPW). IEEE, 2018: 1-7.
[35]	Cisse M, Adi Y, Neverova N , et al. Houdini: Fooling Deep Structured Prediction Models[J]. 2017.
[36]	Szurley J, Kolter J Z . Perceptual Based Adversarial Audio Attacks[J]. arXiv preprint arXiv:1906.06355, 2019.
[37]	Schönherr L, Kohls K, Zeiler S , et al. Adversarial attacks against automatic speech recognition systems via psychoacoustic hiding[J]. arXiv preprint arXiv:1808.05665, 2018.
[38]	Kreuk F, Adi Y, Cisse M, et al. Fooling end-to-end speaker verification with adversarial examples [C]// 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2018: 1962-1966.
[39]	Serdyuk D, Audhkhasi K , Brakel, Philémon, et al. Invariant Representations for Noisy Speech Recognition[J]. 2016.
[40]	Sriram A, Jun H, Gaur Y , et al. Robust Speech Recognition Using Generative Adversarial Networks[J]. 2017.
[41]	Zeng Q, Su J, Fu C, et al. A multiversion programming inspired approach to detecting audio adversarial examples [C]// 2019 49th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN). IEEE, 2019: 39-51.
[42]	Yang Z, Li B, Chen P Y , et al. Characterizing Audio Adversarial Examples Using Temporal Dependency[J]. 2018.
[43]	Shokri R, Stronati M, Song C, et al. Membership inference attacks against machine learning models[C]. 2017 IEEE Symposium on Security and Privacy (SP). IEEE, 2017: 3-18.
[44]	Salem A, Zhang Y, Humbert M , et al. Ml-leaks: Model and data independent membership inference attacks and defenses on machine learning models[J]. arXiv preprint arXiv:1806.01246, 2018.
[45]	Miao Y Zhao B Z H, Xue M, , et al. The Audio Auditor: Participant-Level Membership Inference in Voice-Based IoT[J]. arXiv preprint arXiv:1905.07082, 2019.
[46]	Dwork C, Roth A . The algorithmic foundations of differential privacy[J]. Foundations and Trends® in Theoretical Computer Science, 2014,9(3-4):211-407.
[47]	Chaudhuri K, Monteleoni C, Sarwate A D . Differentially private empirical risk minimization[J]. Journal of Machine Learning Research, 2011,12(Mar):1069-1109.
[48]	Jadhav S, Rawate A M . A New Audio Steganography with Enhanced Security based on Location Selection Scheme[J]. International Journal of Performability Engineering, 2016,12(5).
[49]	Kong Y, Zhang J . Adversarial Audio: A New Information Hiding Method and Backdoor for DNN-based Speech Recognition Models[J]. arXiv preprint arXiv:1904.03829, 2019.
[50]	Wang B, Yao Y, Shan S, et al. Neural cleanse: Identifying and mitigating backdoor attacks in neural networks[C]. Neural Cleanse: Identifying and Mitigating Backdoor Attacks in Neural Networks. IEEE, 2019.
[51]	Elsayed G F, Goodfellow I, Sohl-Dickstein J . Adversarial reprogramming of neural networks[J]. arXiv preprint arXiv:1806.11146, 2018.
[52]	Mukhopadhyay D, Shirvanian M, Saxena N. All Your Voices are Belong to Us: Stealing Voices to Fool Humans and Machines[M]// Computer Security - ESORICS 2015. Springer International Publishing, 2015.
[53]	Lei X, Tu G H, Liu A X , et al. The Insecurity of Home Digital Voice Assistants - Amazon Alexa as a Case Study[J]. 2017.
[54]	Lai C I, Abad A, Richmond K, et al. Attentive filtering networks for audio replay attack detection [C]// ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2019: 6316-6320.
[55]	Diao W, Liu X, Zhou Z, et al. Your voice assistant is mine: How to abuse speakers to steal information and control your phone[C]. Proceedings of the 4th ACM Workshop on Security and Privacy in Smartphones & Mobile Devices. ACM, 2014: 63-74.
[56]	Xiao Q, Li K, Zhang D, et al. Security risks in deep learning implementations[C]. 2018 IEEE Security and Privacy Workshops (SPW). IEEE, 2018: 123-128.

智能语音系统安全分析

An Analysis of Intelligent Acoustic System Security

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

图/表 1

参考文献 56

相关文章 5

编辑推荐

Metrics

本文评价

[1]	陈磊,刘文懋. 合规视角下的数据安全技术前沿与应用[J]. 数据与计算发展前沿, 2021, 3(3): 19-31.
[2]	赵睿斌,张道德,杨绍亮,江伟玉. 一种IP头部标识符的隐私保护方法[J]. 数据与计算发展前沿, 2021, 3(3): 48-58.
[3]	吴一铭,王伟,延志伟,汪洋. 基于多解析器的域名隐私保护机制[J]. 数据与计算发展前沿, 2021, 3(3): 75-85.
[4]	陈远平,周小军,何鹏. 电子政务国产化应用探索及实践[J]. 数据与计算发展前沿, 2021, 3(2): 86-92.
[5]	刘思瀚,徐石成,何光宇. 基于区块链技术的电动汽车电池溯源系统构建[J]. 数据与计算发展前沿, 2020, 2(5): 76-83.