数据与计算发展前沿 ›› 2024, Vol. 6 ›› Issue (2): 80-88.

CSTR: 32002.14.jfdc.CN10-1649/TP.2024.02.008

doi: 10.11871/jfdc.issn.2096-742X.2024.02.008

• 专刊:智慧医疗前沿与进展（上） • 上一篇    下一篇

基于域无关循环生成对抗网络的跨模态医学影像生成

李浩鹏1(),周琬婷1,*(),陈玉2,张曼1   

  1. 1.北京邮电大学，人工智能学院，北京 100876
    2.首都医科大学附属北京天坛医院，北京 100070
  • 收稿日期:2023-10-29 出版日期:2024-04-20 发布日期:2024-04-26
  • 通讯作者: *周琬婷（E-mail: wanting.zhou@bupt.edu.cn）
  • 作者简介:李浩鹏，北京邮电大学人工智能学院，博士研究生，主要研究方向为计算机视觉、图像生成等。
    本文中负责模型的实现、实验的进行与文章的撰写。
    LI Haopeng is a doctoral student at the Department of Artificial Intelligence, Beijing University of Posts and Telecommunications, Beijing, China. His main research interests include computer vision and image generation.
    In this paper, he is responsible for implementing the model, conducting the experiments, and writing the paper.
    E-mail: haopeng_li@bupt.edu.cn|周琬婷，北京邮电大学，特聘副研究员，博士生导师，主要研究方向为计算机视觉、模式识别，在国际知名期刊和会议（TMM、TCSVT、TMI等）合作发表论文二十余篇，主持或参与科研项目14项。获得博士后创新人才支持计划（2019）、剑桥大学终身校友等。
    本文中负责框架确定与撰写指导。
    ZHOU Wanting is an associate professor with the Department of Artificial Intelligence, Beijing University of Posts and Telecommunications, Beijing, China. She has published over 20 papers in internationally renowned journals and conferences (TMM, TCSVT, TMI, etc.). She received the National Postdoctoral Program for Innovative Talents from China Postdoctoral Science Foundation in 2019 and the Cambridge University Life Member. Her research interests include computer vision and pattern recognition.
    In this paper, she is responsible for determining the framework and providing writing guidance.
    E-mail: wanting.zhou@bupt.edu.cn
  • 基金资助:
    国家自然科学基金面上项目“面向脑动静脉畸形辅助诊疗的多模态医学影像分析”(62376037);国家自然科学基金青年项目“移动场景眼部多模态生物特征识别”(62006227);辽宁省感知与理解人工智能重点实验室开放课题基金“复杂医学影像分析与生成研究”(20230006)

Domain Independent Cycle-GAN for Cross Modal Medical Image Generation

LI Haopeng1(),ZHOU Wanting1,*(),CHEN Yu2,ZHANG Man1   

  1. 1. Department of Artificial Intelligence, Beijing University of Posts and Telecommunications, Beijing 100876, China
    2. Beijing Tiantan Hospital, Capital Medical University, Beijing 100070, China
  • Received:2023-10-29 Online:2024-04-20 Published:2024-04-26

RichHTML

1

PDF

34

摘要：

【目的】为解决跨模态医学影像生成任务中因利用未配对数据训练而导致生成图像结构不对齐、精确度低的问题。【方法】本文提出了一种基于域无关循环生成对抗网络的跨模态医学影像生成模型，通过对齐循环生成时的中间特征，约束模态转换前后图像的结构一致性。【结果】在脑部CT-MRI数据集上的实验结果表明，本文所提出的方法能够提升模型在跨模态转换前后图像结构的一致性，从而提高跨模态医学影像的生成质量。【局限】本文目前在脑部多模态数据集上进行了大量实验，还需要在其他数据集中进一步验证其通用性。【结论】本文提出的方法在各类衡量生成图像质量的指标上均优于目前性能最佳的跨模态医学影像生成模型。

关键词: 生成式对抗网络, 医学影像生成, 自注意力机制

Abstract:

[Objective] To solve the problem of misaligned and inaccurate image structure generated by training with unpaired data in cross-modal medical image generation tasks, [Methods] this article proposes a cross-modal medical image generation model based on domain independent cyclic generation adversarial network, which constrains the structural consistency of images before and after modal transformation by aligning the intermediate features during cyclic generation. [Results] The experimental results on the brain CT-MRI dataset show that the method proposed in this article can improve the consistency of image structure before and after cross-modal transformation, thereby improving the quality of cross-modal medical image generation. [Limitations] This study has conducted a large number of experiments on the multimodal brain dataset. Further verification of its universality is needed in other datasets. [Conclusions] The method proposed in this article outperforms the current best performing cross-modal medical image generation model in various metrics for measuring the quality of generated images.

Key words: generative adversarial networks, medical image synthesis, self-attention mechanism