数据与计算发展前沿 ›› 2023, Vol. 5 ›› Issue (6): 42-57.

CSTR: 32002.14.jfdc.CN10-1649/TP.2023.06.005

doi: 10.11871/jfdc.issn.2096-742X.2023.06.005

• • 上一篇    下一篇

基于增量学习的深度人脸伪造检测

赵泽军1,2(),范振峰1,2,丁博1,2,夏时洪1,2,*()   

  1. 1.中国科学院计算技术研究所，北京 100190
    2.中国科学院大学，北京 100049
  • 出版日期:2023-12-20 发布日期:2023-12-25
  • 通讯作者: 夏时洪（xsh@ict.ac.cn）
  • 作者简介:赵泽军，中国科学院计算技术研究所，硕士研究生，中国计算机学会会员，主要研究方向为连续学习、人脸深度伪造检测。
    本文中负责实验验证和论文主体撰写。
    ZHAO Zejun is currently a graduate student in School of Institute of Computing Technology, Chinese Academy of Sciences, and he is now pursuing his master's degree. CCF member. His research interests include continuous Learning, face forgery detection.
    His contributions to this paper are experimental verification and manuscript writing.
    E-mail: zhaozejun21s@ict.ac.cn|夏时洪，中国科学院计算技术研究所，研究员，博士生导师，中国科学院大学计算机科学与技术学院岗位教授，主要研究方向为计算机图形学、虚拟现实、人工智能。
    本文中负责写作指导以及论文最终审定。
    XIA Shihong is currently a professor with the Institute of Computing Technology, Chinese Academy of Sciences. His research interests include computer graphics, virtual reality, artificial intelligence.
    His contributions to this paper are writing instruction and manuscript reviewing.
    E-mail: xsh@ict.ac.cn
  • 基金资助:
    国家自然科学基金项目(62106250);博士后面上基金项目(2021M703272)

Deepfake Detection Based on Incremental Learning

ZHAO Zejun1,2(),FAN Zhenfeng1,2,DING Bo1,2,XIA Shihong1,2,*()   

  1. 1. Institute of Computing Technology, Beijing 100190, China
    2. Chinese Academy of Sciences University, Beijing 100049, China
  • Online:2023-12-20 Published:2023-12-25

RichHTML

23

PDF

6553

摘要：

【目的】 随着计算机视觉、计算机图形学以及深度学习技术的发展，深度人脸伪造（DeepFake）技术取得了以假乱真的效果，若被非法利用，将给个人、社会和国家带来严重的安全隐患。已有的人脸伪造检测方法大多通过一次训练来推断或预测伪造人脸存在的某种特定“指纹”进行真伪检测，当面对新的伪造类型时，这些方法使用全部数据重新训练网络以保持其检测能力，否则检测效果将急剧下降。然而，重新训练网络需要相对大的代价，并且阻碍了模型实时学习新知识的能力。鉴于此，本文提出一种检测伪造人脸的增量学习方法。【方法】 引入动态可扩展的增量学习框架，以保证模型在吸收新知识的同时能保留对旧知识的记忆；使用多分类指导二分类的方式来提高模型的分类能力，最终实现对人脸图像的精确分类。【结果】 在两个公开数据集上进行实验。在实验定义的FF++扩充集和ForgeryNet扩充集上，本文方法能同时保持在新旧任务上的人脸伪造检测性能；在实验定义的ForgeryNet扩充集上，现有的人脸伪造检测方法达到了近98.33%的平均ACC（accuracy），本文方法达到了96.16%的平均ACC，但前者使用了超出后者接近3倍的存储和计算资源；将实验定义的ForgeryNet扩充集的后5个任务视为新任务，每个任务下每个类别仅包含100张训练数据，现有方法在该5个任务上达到了近88.72%的平均ACC，本文方法达到了93.83%的平均ACC。【局限】 为了保持正负样本的平衡，训练时需要成对的训练样本，这引入了不必要的训练数据，增加了训练负担。【结论】 本文提出的人脸伪造检测方法通过增量学习的方式提高了模型对检测不断出现的伪造样本的有效性。实验结果表明，本文方法能以较低的计算代价，达到与现有方法相当的检测能力；在训练数据匮乏的情况下，达到比现有方法更优的伪造检测能力。

关键词: 深度人脸伪造检测, 深度伪造, 增量学习, 连续学习, 灾难性遗忘

Abstract:

[Objective] With the development of computer vision, computer graphics and deep learning technologies, deep face forgery (DeepFake) technology has achieved realistic effects. If used illegally, it will bring serious security risks to individuals, society and the country. Most existing face forgery detection methods infer or predict a specific “fingerprint” of forged faces through one-time training for authenticity detection. When facing new types of forgery, these methods use all data to retrain the network to maintain their detection ability, otherwise their detection effect will drop sharply. However, retraining the network requires a relatively high cost and hinders the model's ability to learn new knowledge in real-time. In view of this, this paper proposes an incremental learning method for detecting forged faces. [Method] The method introduces a dynamic and scalable incremental learning framework to ensure that the model can retain memory of old knowledge while absorbing new knowledge, uses multi-classification to guide binary classification to improve the model's classification ability, and ultimately achieves accurate classification of face images. [Result] Experiments are conducted on two public datasets. On the FF++ expansion set and the ForgeryNet expansion set defined in the experiment, our method can simultaneously maintain the performance of face forgery detection on both old and new tasks. On the ForgeryNet expansion set, existing face forgery detection methods achieves an average accuracy of nearly 98.33%, while our method achieves an average accuracy of 96.16%, while the former uses three times more storage and computing resources than the latter. The last five tasks of the expansion set of ForgeryNet in the experiment are considered as new tasks, with each class containing only 100 training data. Existing methods achieves an average accuracy of near 88.72% on these five tasks, while the method proposed in this paper achieves an average accuracy of 93.83%. [Limitations] To maintain a balance of positive and negative samples, pairs of training samples are required for training, which introduces unnecessary training data and increases the training burden. [Conclusion] The face forgery detection method proposed in this paper improves the effectiveness of the model in detecting constantly emerging forgery samples through incremental learning. Experimental results show that this method can achieve detection capability comparable to existing methods at a lower computational cost; and achieve better forgery detection capability than existing methods in the case of limited training data.

Key words: deepfake detection, deep fakes, incremental learning, continuous learning, catastrophic forgetting