数据与计算发展前沿 ›› 2022, Vol. 4 ›› Issue (4): 123-131.

CSTR: 32002.14.jfdc.CN10-1649/TP.2022.04.012

doi: 10.11871/jfdc.issn.2096-742X.2022.04.012

• 技术与应用 • 上一篇    下一篇

基于生成对抗网络的有遮挡人脸修复方法

石雪梅1(),朱克亮1,张祥民1,张树涛2,3,陈良锋2,*()   

  1. 1.国网安徽省电力有限公司建设分公司,安徽 合肥 230022
    2.中国科学院合肥物质科学研究院,安徽 合肥 230026
    3.中国科学技术大学,安徽 合肥 230031
  • 收稿日期:2021-07-23 出版日期:2022-08-20 发布日期:2022-08-10
  • 通讯作者: 陈良锋
  • 作者简介:石雪梅,国网安徽省电力有限公司建设分公司,硕士,研究方向为计算机技术。
    本文中负责撰稿。
    SHI Xuemei, master's degree, works in State Grid Anhui Electric Power Co., Ltd, Construction Branch. Her current research direction is computer technology.
    In this paper, she is responsible for the paper writing.
    E-mail: 524104473@qq.com|陈良锋,中国科学院合肥物质科学研究院,博士,工程师,研究方向为机器视觉。
    在本文中负责核心算法论证。
    CHEN Liangfeng, Ph.D., is an engineer of Hefei Institutes of Physical Science, Chinese Academy of Sciences. His cur-rent research interests include machine visi-on.
    In this paper, he is responsible for core algorithm demonstration.
    E-mail: quinear@hfcas.ac.cn
  • 基金资助:
    国家电网公司科技项目(B3120A190005)

Occluded Face Inpainting Method Based on Generative Adversarial Networks

SHI Xuemei1(),ZHU Keliang1,ZHANG Xiangmin1,ZHANG Shutao2,3,CHEN Liangfeng2,*()   

  1. 1. State Grid Anhui Electric Power Co, Ltd. Construction Company, Hefei, Anhui 230022, China
    2. Hefei Institutes of Physical Science, Chinese Academy of Sciences, Hefei, Anhui 230026, China
    3. University of Science and Technology of China, Hefei, Anhui 230031, China
  • Received:2021-07-23 Online:2022-08-20 Published:2022-08-10
  • Contact: CHEN Liangfeng

摘要:

【目的】实现有遮挡人脸图像修复,以提升人脸识别系统中有遮挡条件下人脸识别的准确率。【应用背景】在无感人脸识别场景中,人脸图像常因帽子、口罩等遮挡物影响,现有人脸识别算法出现准确率下降甚至无法识别现象。【方法】本文综合考虑图像内容相似性、真实性和生成的图像连贯性,提出一种由单一生成器和双判别器组成的生成对抗网络,将生成器生成的人脸被遮挡部分输入局部判别器,并将其与去除遮挡部分的原始人脸图像融合后输入全局判别器,提升生成器的稳定性及精确性。【结果】采用LFW人脸数据集实验分析可知,本文方法更好地实现了有遮挡的人脸图像修复,在图像结构相似度(SSIM)和峰值信噪比(PSNR)上,较经典DCGAN提高了67%和40%。【结论】针对实际应用场景,以图像内容相似性为生成对抗网络输入条件,提高了网络的收敛效率和稳定性,可进一步优化网络结构、降低层间损失以提升生成图像的质量。

关键词: 人脸修复, 生成对抗网络, 图像融合, 卷积神经网络, 损失函数

Abstract:

[Objective] Face recognition, a basic technology in artificial intelligence applications, would be invalid due to the occlusion phenomenon. To improve the face recognition accuracy in the presence of partial occlusions, we propose a method to generate occluded regions of a face image automatically. [Context] The accuracy is the essential factor in face recognition applications. But its performance drastically decreases, and even to zero, when face images are occluded by objects such as helmets, glasses, and masks. [Methods] Taking image attributes into accounts, such as similarity, validity, and consistency, we present an improved generative adversarial network which is composed of one generator and two discriminators. To improve the stability and accuracy of the generator, the occluded part of the face generated by the generator is input to the local discriminator, and together with the original face image without the occluded part serve as the input to the globe discriminator. [Results] The experiments conducted on the LFW dataset show that the Structural Similarity (SSIM) of our method is 67% higher than DCGAN, and the Peak Signal to Noise Ratio (PSNR) is 40% higher. [Conclusions] The results show that prior conditions such as the similarity between pair images can improve the stability and convergence rate of a generator. In future work, a better network architecture that reduces interlayer loss would be proposed to improve the quality of generated images.

Key words: face inpainting, generative adversarial network, image fusion, convolutional neural network, loss function