Domain Independent Cycle-GAN for Cross Modal Medical Image Generation

doi:10.11871/jfdc.issn.2096-742X.2024.02.008

Frontiers of Data and Computing ›› 2024, Vol. 6 ›› Issue (2): 80-88.

CSTR: 32002.14.jfdc.CN10-1649/TP.2024.02.008

doi: 10.11871/jfdc.issn.2096-742X.2024.02.008

• Special Issue: Advance of Intelligent Healthcare • Previous Articles Next Articles

Domain Independent Cycle-GAN for Cross Modal Medical Image Generation

LI Haopeng¹(),ZHOU Wanting^1,^*(),CHEN Yu²,ZHANG Man¹

1. Department of Artificial Intelligence, Beijing University of Posts and Telecommunications, Beijing 100876, China
2. Beijing Tiantan Hospital, Capital Medical University, Beijing 100070, China

Received:2023-10-29 Online:2024-04-20 Published:2024-04-26

Abstract

Abstract:

[Objective] To solve the problem of misaligned and inaccurate image structure generated by training with unpaired data in cross-modal medical image generation tasks, [Methods] this article proposes a cross-modal medical image generation model based on domain independent cyclic generation adversarial network, which constrains the structural consistency of images before and after modal transformation by aligning the intermediate features during cyclic generation. [Results] The experimental results on the brain CT-MRI dataset show that the method proposed in this article can improve the consistency of image structure before and after cross-modal transformation, thereby improving the quality of cross-modal medical image generation. [Limitations] This study has conducted a large number of experiments on the multimodal brain dataset. Further verification of its universality is needed in other datasets. [Conclusions] The method proposed in this article outperforms the current best performing cross-modal medical image generation model in various metrics for measuring the quality of generated images.

Key words: generative adversarial networks, medical image synthesis, self-attention mechanism

LI Haopeng, ZHOU Wanting, CHEN Yu, ZHANG Man. Domain Independent Cycle-GAN for Cross Modal Medical Image Generation[J]. Frontiers of Data and Computing, 2024, 6(2): 80-88, https://cstr.cn/32002.14.jfdc.CN10-1649/TP.2024.02.008.

Figures/Tables 5

Fig.1

Table 1

Table 2

Fig.2

Fig.3

References 15

[1]	WANI N, RAZA K. Multiple Kernel Learning approach for Medical Image Analysis[M]// Soft Computing Based Medical Image Analysis, 2018: 31-47.
[2]	LEI Y, HARMS J, WANG T, et al. MRI-based synthetic CT generation using semantic random forest with iterative refinement[J]. Physics in Medicine & Biology, 2019, 64(8): 085001.
[3]	GOODFELLOW I, POUGET-ABADIE J, MIRZA M, et al. Generative Adversarial Nets[C]. Advances in neural information processing systems, 2014:2672-2680.
[4]	MIRZA M, OSINDERO S. Conditional Generative Adversarial Nets[J]. arXiv preprint arXiv:1411. 1784, 2014.
[5]	ISOLA P, ZHU J Y, ZHOU T, et al. Image-to-Image Translation with Conditional Adversarial Networks[C]// Proceedings of the IEEE conference on computer vision and pattern recognition, 2017: 1125-1134.
[6]	ZHU J Y, PARK T, ISOLA P, et al. Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks[C]// Proceedings of the IEEE international conference on computer vision, 2017: 2223-2232.
[7]	HE K, ZHANG X, REN S, et al. Deep Residual Learning for Image Recognition[C]// Proceedings of the IEEE conference on computer vision and pattern recognition, 2016: 770-778.
[8]	VASWANI A, SHAZEER N, PARMAR N, et al. Attention is All you Need[C]. Advances in neural information processing systems, 2017:5998-6008.
[9]	DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale[J]. arXiv preprint arXiv:2010.11929, 2020.
[10]	HEINRICH M P, JENKINSON M, BHUSHAN M, et al. MIND: Modality independent neighbourhood descriptor for multi-modal deformable registration[J]. Medical image analysis, 2012, 16(7): 1423-1435. doi: 10.1016/j.media.2012.05.008 pmid: 22722056
[11]	YANG H, SUN J, CARASS A, et al. Unsupervised MR-to-CT Synthesis Using Structure-Constrained CycleGAN[J]. IEEE transactions on medical imaging, 2020, 39(12): 4249-4261. doi: 10.1109/TMI.42
[12]	DALMAZ O, YURT M, CUKUR T. ResViT: Residual vision transformers for multi-modal medical image synthesis.[J]. IEEE Transactions on Medical Imaging, 2022, 41(10): 2598-2614. doi: 10.1109/TMI.2022.3167808
[13]	WANG J, WU Q M J, POURPANAH F. DC-cycleGAN: Bidirectional CT-to-MR Synthesis from Unpaired Data[J]. Computerized Medical Imaging and Graphics, 2023, 108(10): 102249. doi: 10.1016/j.compmedimag.2023.102249
[14]	SKANDARANI Y, JODOIN P M, LALANDE A. Gans for medical image synthesis: An empirical study[J]. Journal of Imaging, 2023, 9(3): 69. doi: 10.3390/jimaging9030069
[15]	SINGH N K, RAZA K. Medical image generation using generative adversarial networks: A review[J]. Health informatics: A computational perspective in healthcare, 2021, 932(1): 77-96.

Model	PSNR	SSIM	MAE
CycleGAN^[6]	14.9900	0.6330	0.0709
CycleGAN+ViT^[9]	15.1623	0.6332	0.0700
CycleGAN+DIIFA	16.0079	0.6512	0.0599
CycleGAN+ViT+DIIFA(Ours)	16.2761	0.6587	0.0563

Model	PSNR	SSIM	MAE
CycleGAN^[6]	16.1156	0.5482	0.1217
CycleGAN+ViT^[9]	17.0127	0.5518	0.1179
CycleGAN+DIIFA	17.4820	0.5772	0.1037
CycleGAN+ViT+DIIFA(Ours)	17.5553	0.5810	0.1035

Domain Independent Cycle-GAN for Cross Modal Medical Image Generation

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 5

References 15

Related Articles 0

Recommended Articles

Metrics

Comments