Review of Research on Secure Inference in Machine Learning

doi:10.11871/jfdc.issn.2096-742X.2024.05.001

Abstract

Abstract:

[Objective] This paper analyzes existing research on secure machine learning inference and proposes future research directions. [Methods] Using the security assumptions of different schemes as a basis for classification, this study conducts analysis and comparison of secure inference techniques that utilize various technological combinations for application in different machine learning contexts. [Results] While current schemes facilitate secure machine learning inference, they exhibit limitations in computational efficiency, security, scalability, and practical applicability. [Limitations] Due to limited data availability, experiments and comparisons of the analyzed schemes under the same benchmark were not conducted. [Conclusions] Designing secure machine learning inference schemes based on application scenarios, ensuring security while improving usability and reducing costs, will be a sustained development direction in this field.

Key words: privacy-preserving machine learning, machine learning, data privacy, secure multi-party computation

LONG Chun, LI Lisha, LI Jing, YANG Fan, WEI Jinxia, Fu Yuhao. Review of Research on Secure Inference in Machine Learning[J]. Frontiers of Data and Computing, 2024, 6(5): 1-12, https://cstr.cn/32002.14.jfdc.CN10-1649/TP.2024.05.001.

Figures/Tables 4

Fig.1

Table 1

Table 2

Fig.2

References 69

[1]	YAO A C. Protocols for secure computations[C]// 23rd annual symposium on foundations of computer science (sfcs 1982). IEEE, 1982: 160-164.
[2]	SONG L, LIN G, WANG J, et al. Sok: Training machine learning models over multiple sources with privacy preservation[J]. arXiv preprint arXiv: 2012.03 386, 2020.
[3]	郭娟娟, 王琼霄, 许新, 等. 安全多方计算及其在机器学习中的应用[J]. 计算机研究与发展, 2021, 58(10): 2163-2186.
[4]	HAO Y, QIN B, SUN Y. Privacy-preserving decision-tree evaluation with low complexity for communication[J]. Sensors, 2023, 23(5): 2624.
[5]	JI K, ZHANG B, LU T, et al. UC Secure private branching program and decision tree evaluation[J]. IEEE Transactions on Dependable and Secure Computing, 2022, 20(4): 2836-2848.
[6]	CHEN X, CHEN X, DONG Y, et al. Roger: A round optimized gpu-friendly secure inference framework[C]// ICC 2024-IEEE International Conference on Communications. IEEE, 2024: 61-66.
[7]	FAN T, CHEN X, DONG Y, et al. Comet: Communication-efficient batch secure three-party neural network inference with client-aiding[C]// ICC 2024-IEEE International Conference on Communications. IEEE, 2024: 752-757.
[8]	FENG Q, HE D, LIU Z, et al. SecureNLP: A system for multi-party privacy-preserving natural language processing[J]. IEEE Transactions on Information Forensics and Security, 2020, 15: 3709-3721.
[9]	HUANG Z, LU W, HONG C, et al. Cheetah: Lean and fast secure {Two-Party} deep neural network inference[C]// 31st USENIX Security Symposium (USENIX Security 22). 2022: 809-826.
[10]	DONG Y, CHEN X, JING W, et al. Meteor: improved secure 3-party neural network inference with reducing online communication costs[C]// Proceedings of the ACM Web Conference 2023. 2023: 2087-2098.
[11]	DONG Y, CHEN X, SONG X, et al. FLEXBNN: fast private binary neural network inference with flexible bit-width[J]. IEEE Transactions on Information Forensics and Security, 2023, 18: 2382-2397.
[12]	LU Y, ZHANG B, REN K. Maliciously secure mpc from semi-honest 2 pc in the server-aided model[J]. IEEE Transactions on Dependable and Secure Computing, 2024 (4): 3109-3125.
[13]	LI Y, XU W. PrivPy: General and scalable privacy-preserving data mining[C]// Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2019: 1299-1307.
[14]	SONG L, WANG J, WANG Z, et al. Pmpl: A robust multi-party learning framework with a privileged party[C]// Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security. 2022: 2689-2703.
[15]	MA J, ZHENG Y, FENG J, et al. {SecretFlow-SPU}: A performant and {user-friendly} framework for {privacy-preserving} machine learning[C]// 2023 USENI-X Annual Technical Conference (USENIX ATC 23). 2023: 17-33.
[16]	谭作文, 张连福. 机器学习隐私保护研究综述[J]. 软件学报, 2020, 31(7): 2127-2156.
[17]	HAZAY C, VENKITASUBURAMANIAM M, WE- ISS M. The price of active security in cryptographic protocols[C]// Annual International Conference on the Theory and Applications of Cryptographic Techniqu-es. Cham: Springer International Publishing, 2020: 1 84-215.
[18]	RABIN M O. How to exchange secrets with oblivious transfer[J]. IACR Cryptol. ePrint Arch, 2005 ( 2005): 187.
[19]	曲亚东, 侯紫峰, 韦卫. 基于不经意传输的合同签订协议[J]. 计算机研究与发展, 2003, (4): 615-619.
[20]	陈晓洪. 基于安全多方计算的电子投票系统应用研究[D]. 南京理工大学, 2010.
[21]	查俊. 安全多方计算在密钥协商中的应用研究[D]. 解放军信息工程大学, 2012.
[22]	李宗育, 桂小林, 顾迎捷, 等. 同态加密技术及其在云计算隐私保护中的应用[J]. 软件学报, 2018, 29(7):1830-1851.
[23]	ACAR A, AKSU H, ULUAGAC A S, et al. A survey on homomorphic encryption schemes: Theory and implementation[J]. ACM Computing Surveys (Csur), 2018, 51(4): 1-35.
[24]	RIVEST R L, SHAMIR A, ADLEMAN L. A method for obtaining digital signatures and public-key cryptosystems[J]. Communications of the ACM, 1978, 21(2): 120-126.
[25]	ElGAMAL T. A public key cryptosystem and a signature scheme based on discrete logarithms[J]. IEEE transactions on information theory, 1985, 31(4): 469-472.
[26]	PAILLIER P. Public-key cryptosystems based on composite degree residuosity classes[C]// International c- onference on the theory and applications of cryptographic techniques. Berlin, Heidelberg: Springer Berlin Heidelberg, 1999: 223-238.
[27]	BONEH D, GOH E J, NISSIM K. Evaluating 2-DNF formulas on ciphertexts[C]// Theory of Cryptography: Second Theory of Cryptography Conference, TCC 2005, Cambridge, MA, USA, February 10-12, 2005. Proceedings 2. Springer Berlin Heidelberg, 2005: 325-341.
[28]	GENTRY C. Fully homomorphic encryption using ideal lattices[C]// Proceedings of the forty-first annual ACM symposium on Theory of computing. 2009: 169-178.
[29]	SMART N P, VERCAUTEREN F. Fully homomorphic encryption with relatively small key and ciphertext sizes[C]// International Workshop on Public Key Cryptography. Berlin, Heidelberg: Springer Berlin Heidelberg, 2010: 420-443.
[30]	VAN Dijk M, GENTRY C, HALEVI S, et al. Fully homomorphic encryption over the integers[C]// Adva-nces in Cryptology-EUROCRYPT 2010: 29th Annual International Conference on the Theory and Applications of Cryptographic Techniques, French Riviera, May 30-June 3, 2010. Proceedings 29. Springer Berlin Heidelberg, 2010: 24-43.
[31]	CORON J S, MANDAL A, NACCACHE D, et al. Fully homomorphic encryption over the integers with shorter public keys[C]// Annual Cryptology Confere-nce. Berlin, Heidelberg: Springer Berlin Heidelberg, 2011: 487-504.
[32]	SHAMIR A. How to share a secret[J]. Communications of the ACM, 1979, 22(11): 612-613.
[33]	BEAVER D. Efficient multiparty protocols using circuit randomization[C]// Advances in Cryptology-CR-YPTO’91: Proceedings 11. Springer Berlin Heidelb-erg, 1992: 420-432.
[34]	BLAKLEY G R. Safeguarding cryptographic keys[C]// Managing requirements knowledge, international workshop on. IEEE Computer Society, 1979: 313-313.
[35]	GALTON F. Regression towards mediocrity in hereditary stature[J]. The Journal of the Anthropological Institute of Great Britain and Ireland, 1886, 15: 246-263.
[36]	CORTES C, VAPNIK V. Support-vector networks[J]. Machine Learning, 1995, 20(3): 273-297.
[37]	QUINLAN J R. C4.5: Programs for machine learning[M]. Morgan Kaufmann, 1993.
[38]	LECUN Y, BOTTOU L, BENGIO Y, et al. Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998, 86(11): 2278-2324.
[39]	NIKOLAENKO V, WEINSBERG U, IOANNIDIS S, et al. Privacy-preserving ridge regression on hund-reds of millions of records[C]// 2013 IEEE symposium on security and privacy. IEEE, 2013: 334-348.
[40]	GASCÓN A, SCHOPPMANN P, BALLE B, et al. Secure linear regression on vertically partitioned datasets[J]. IACR Cryptol. ePrint Arch. 2016 (2016): 892.
[41]	RAHULAMATHAVAN Y, PHAN R C W, VELURU S, et al. Privacy-preserving multi-class support vector machine for outsourcing the data classification in cloud[J]. IEEE Transactions on Dependable and Secure Computing, 2013, 11(5): 467-479.
[42]	RAHULAMATHAVAN Y, VELURU S, PHAN R C W, et al. Privacy-preserving clinical decision support system using gaussian kernel-based classification[J]. IEEE journal of biomedical and health informatics, 2013, 18(1): 56-66.
[43]	LIU X, LU R, MA J, et al. Privacy-preserving patient-centric clinical decision support system on naive Ba- yesian classification[J]. IEEE journal of biomedical and health informatics, 2015, 20(2): 655-668.
[44]	BOST R, POPA R A, TU S, et al. Machine learning classification over encrypted data[C]// Network and Distributed System Security Symposium. 2014. DOI:10.14722/ndss.2015.23241.
[45]	WU D J, FENG T, NAEHRIG M, et al. Privately evaluating decision trees and random forests[J]. Proceedings on Privacy Enhancing Technologies, 2016, (4): 335-355.
[46]	BACKES M, BERRANG P, BIEG M, et al. Identifying personal DNA methylation profiles by genotype inference[C]// 2017 IEEE symposium on security and privacy (SP). IEEE, 2017: 957-976.
[47]	DE COCK M, DOWSLEY R, HORST C, et al. Efficient and private scoring of decision trees, support vector machines and logistic regression models based on pre-computation[J]. IEEE Transactions on Dependable and Secure Computing, 2017, 16(2): 217-230.
[48]	KISS Á, NADERPOUR M, LIU J, et al. SoK: Modular and efficient private decision tree evaluation[J]. Proceedings on Privacy Enhancing Technologies, 2019, (2): 187-208.
[49]	TUENO A, KERSCHBAUM F, KATZENBEISSER S. Private evaluation of decision trees using sublinear cost[J]. Proceedings on Privacy Enhancing Technologies, 2019(1): 266-286.
[50]	MA J P K, TAI R K H, ZHAO Y, et al. Let’s stride blindfolded in a forest: sublinear multi-client decision trees evaluation[C]. Proceedings 2021 Network and Distributed System Security Symposium, 2021. DOI:10.14722/ndss.2021.23166.
[51]	XIE P, BILENKO M, FINLEY T, et al. Crypto-nets: Neural networks over encrypted data[J]. arXiv preprint arXiv:1412.6181, 2014.
[52]	GILAD-BACHRACH R, DOWLIN N, LAINE K, et al. Cryptonets: Applying neural networks to encrypted data with high throughput and accuracy[C]// International conference on machine learning. PMLR, 20 16: 201-210.
[53]	LIU J, JUUTI M, LU Y, et al. Oblivious neural network predictions via minionn transformations[C]// Proceedings of the 2017 ACM SIGSAC conference on computer and communications security. 2017: 619-631.
[54]	RIAZI M S, WEINERT C, TKACHENKO O, et al. Chameleon: A hybrid secure computation framework for machine learning applications[C]// Proceedings of the 2018 on Asia conference on computer and communications security. 2018: 707-721.
[55]	JUVEKAR C, VAIKUNTANATHAN V, CHANDRA-KASAN A. {GAZELLE}:A low latency framework for secure neural network inference[C]// 27th USENIX security symposium (USENIX security 18). 2018: 1651-1669.
[56]	MISHRA P, LEHMKUHL R, SRINIVASAN A, et al. Delphi: A Cryptographic Inference Service for Neural Networks[C]// USENIX Security Symposium, 2020: 2505-2522.
[57]	RATHEE D, RATHEE M, KUMAR N, et al. Cryptflow2: Practical 2-party secure inference[C]//Proceedings of the 2020 ACM SIGSAC Conference on Computer and Communications Security. 2020: 325-342.
[58]	BAI J, SONG X, ZHANG X, et al. Mostree: malicious secure private decision tree evaluation with sublinear communication[C]// Proceedings of the 39th Annual Computer Security Applications Conference. 2023: 799-813.
[59]	HAZAY C, ISHAI Y, MARCEDONE A, et al. LevioSA: Lightweight secure arithmetic computation[C]// Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security. 2019: 327-344.
[60]	LEHMKUHL R, MISHRA P, SRINIVASAN A, et al. Muse: Secure inference resilient to malicious clients[C]// 30th USENIX Security Symposium (USENIX Security 21). 2021: 2201-2218.
[61]	CHANDRAN N, GUPTA D, OBBATTU S L B, et al. {SIMC}:{ML} inference secure against malicious clients at {Semi-Honest} cost[C]// 31st USENIX Security Symposium (USENIX Security 22). 2022: 1361-1378.
[62]	DONG C, WENG J, LIU J N, et al. Fusion: Efficient and secure inference resilient to malicious servers[C]. Proceedings 2023 Network and Distributed System Security Symposium, 2023. DOI:10.14722/ndss.2023.23199.
[63]	贾轩, 白玉真, 马智华. 隐私计算应用场景综述[J]. 信息通信技术与政策, 2022, (5): 45-52.
[64]	胡浩. 隐私计算产业的发展及金融行业应用[J]. 银行家, 2023, (3): 108-110.
[65]	郑灏. 隐私计算在金融行业数据融合场景中的应用探析[J]. 中国金融电脑, 2022, (6): 90-91.
[66]	马龙, 陈奕博. 基于技术的治理: 隐私计算技术赋能政府数据开放的价值与路径研究[J]. 中国行政管理, 2023, 39(9): 105-113.
[67]	凡航, 徐葳, 范晓昱, 等. 隐私计算在新型电力系统中的应用分析与展望[J]. 电力系统自动化, 2023, 47(19): 187-199.
[68]	肖霞. 基于隐私计算的药物—药物相互作用预测方法研究[D]. 湖南大学, 2023.
[69]	辛均益, 陈如梵, 王林, 等. 生物医学大数据中的隐私计算[J]. 医学信息学杂志, 2022, 43(10): 2-7.

方案	机器学习方法	密码学技术	运算时间	通信开销	是否支持扩展为恶意安全假设
[39]	岭回归	GC+HE	很大	很大	√
[40]	线性回归	GC+HE	很大	—	√
[41]	支持向量机	HE	较大	—
[42]	支持向量机	HE	较大	—
[43]	朴素贝叶斯分类	HE	较小	—
[44]	超平面决策、朴素贝叶斯、决策树	HE	较大	较大
[45]	决策树、随机森林	HE+OT	较小	较小	√
[46]	随机森林	HE	较大	较大
[47]	决策树	SS+OT	较小	较大
[48]	决策树	HE/GC/OT	—	—
[49]	决策树	SS+OT+GC	较小	较大
[50]	决策树	SS+OT+GC	较小	较大	√
[4]	决策树	HE	较大	较小
[5]	决策树	OT+SS	较小	较小

方案	神经网络规模	数据集规模	密码学技术	通信开销	运算时间
crypto-nets^[51]	小	—	Leveled-FHE	—	—
CryptoNets^[52]	小	MNIST	Leveled-FHE	很大	很大
MiniONN^[53]	小	MNIST	HE+GC+SS	较大	较大
Chameleon^[54]	小	MNIST、CIFAR-10	GC+SS	较大	较大
Gazelle^[55]	小	MNIST、CIFAR-10	HE+GC+SS	较小	较小
Delphi^[56]	ResNet32	CIFAR-10、CIFAR-100	HE+GC+SS+OT	较小	较小
CrypTFlow2^[57]	SqNet、RN50、DNet121	ImageNet	SS+HE+OT	较小	较小
Cheetah^[9]	SqNet、RN50、DNet121	ImageNet	SS+HE+OT	较小	较小