基于因果约束的Transformer医学图像分割方法

doi:10.11871/jfdc.issn.2096-742X.2024.02.009

数据与计算发展前沿 ›› 2024, Vol. 6 ›› Issue (2): 89-100.

CSTR: 32002.14.jfdc.CN10-1649/TP.2024.02.009

doi: 10.11871/jfdc.issn.2096-742X.2024.02.009

• 专刊:智慧医疗前沿与进展（上） • 上一篇下一篇

基于因果约束的Transformer医学图像分割方法

郭冠辰^1,²(),李军¹,蔡程飞^1,^2,³,焦一平¹,徐军^1,^*()

1.南京信息工程大学，人工智能学院智慧医疗研究院，江苏南京 210044
2.南京信息工程大学，自动化学院，江苏南京 210044
3.泰州学院，信息工程学院，江苏泰州 225300

收稿日期:2023-10-30 出版日期:2024-04-20 发布日期:2024-04-26
通讯作者: *徐军（E-mail: jxu@nuist.edu.cn）
作者简介:郭冠辰，南京信息工程大学自动化学院，硕士研究生，主要研究方向为医学图像计算。
本文承担工作为文献的收集整理以及整体内容的撰写。
GUO Guanchen is a master’s student in the School of Automation, Nanjing University of Information Science and Technology. His main research directions include medical image calculation.
In this paper, he is responsible for collective literature reviews and thesis writing.
E-mail: guoguanchen456@163.com|徐军，南京信息工程大学人工智能学院副院长，智慧医疗研究院执行院长，博士研究生导师，二级教授。主要研究方向为医学图像计算、计算病理、数字病理、疾病辅助诊疗和预后。
本文承担的工作为整体规划和论文指导。
XU Jun is the Vice Dean of the School of Artificial Intelligence and the Executive Director of the Institute of Smart Healthcare at Nanjing University of Information Science and Technology. He is a doctoral supervisor and holds the title of Professor at the second level. His primary research areas include medical image computing, computational pathology, digital pathology, disease-assisted diagnosis, and prognosis.
In this paper, he is responsible for the overall planning and paper guidance.
E-mail: jxu@nuist.edu.cn
基金资助:
国家自然科学基金(62171230);国家自然科学基金(62101365);国家自然科学基金(92159301);国家自然科学基金(91959207);国家自然科学基金(62301263);国家自然科学基金(62301265);国家自然科学基金(62302228);国家自然科学基金(82302291);国家自然科学基金(82302352)

Causal Restraint Transformer for Medical Image Segmentation

GUO Guanchen^1,²(),LI Jun¹,CAI Chengfei^1,^2,³,JIAO Yiping¹,XU Jun^1,^*()

1. Institute for AI in Medicine, School of Artificial Intelligence, Nanjing University of Information Science and Technology, Nanjing, Jiangsu 210044, China
2. School of Automation, Nanjing University of Information Science and Technology, Nanjing, Jiangsu 210044, China
3. College of Information Engineering, Taizhou University, Taizhou, Jiangsu 225300, China

Received:2023-10-30 Online:2024-04-20 Published:2024-04-26

摘要/Abstract

摘要：

【目的】数据分布对深度学习模型的性能影响较大。模型学习了与分割目标无关的特征后，这些无关特征通常不适用于新的数据集，从而导致模型泛化能力不足。【方法】为缓解这一问题，本文提出基于因果约束的Transformer医学图像分割方法。以MCRformer为网络主体，利用形态约束流模块提取形态约束先验信息，网状Transformer进一步提取局部信息和网络各层次信息，并加入因果约束模块降低目标区域相关特征和无关特征之间的相关性，通过形态先验和因果先验信息为模型选出具有代表性的特征，最终提高分割性能。【结果】在公开数据集Synapse上，Dice相关系数和Hausdorff距离的均值分别达到了80.01%和19.39 mm，在公开数据集ACDC上，Dice相关系数均值达到了90.95%，优于其他对比方法。【结论】实验证明，本文提出的方法可以有效提升CT和MRI中多器官的分割性能，并验证因果约束模块在不同模型上的有效性。

关键词: 医学图像分割, 形态约束, Transformer, 因果约束

Abstract:

[Purpose] The data distribution has a significant impact on the performance of deep learning models. However, deep models may learn features irrelevant to the segmentation target, which is usually inapplicable for new datasets, resulting in insufficient generalization ability. [Methods] To alleviate this problem, this paper proposes a Transformer-based medical image segmentation method with causal restraint. Taking MCRformer as the main body of the network, the Morphological Constraint Stream module is used to extract morphological constraint prior information. The meshed Transformer further extracts local and network-level information. The method introduces a Causal Restraint module to alleviate the correlation between features related to regions of interest (ROI) and irrelative features. Representative features are selected for the model through morphological and causal prior information, ultimately improving segmentation performance. [Results] On the public Synapse dataset, the Dice Similarity Coefficient and Hausdorff Distance achieved mean values of 80.01% and 19.39mm, respectively. On the public ACDC dataset, the mean DSC reached 90.95%, outperforming other comparative methods. [Conclusions] Experiment results demonstrate that the proposed method effectively enhances multi-organ segmentation performance on CT and MRI images and validates the feasibility of the causal restraint module across different models.

Key words: medical image segmentation, morphological constraint, Transformer, causal restraint

郭冠辰, 李军, 蔡程飞, 焦一平, 徐军. 基于因果约束的Transformer医学图像分割方法[J]. 数据与计算发展前沿, 2024, 6(2): 89-100.

GUO Guanchen, LI Jun, CAI Chengfei, JIAO Yiping, XU Jun. Causal Restraint Transformer for Medical Image Segmentation[J]. Frontiers of Data and Computing, 2024, 6(2): 89-100, https://cstr.cn/32002.14.jfdc.CN10-1649/TP.2024.02.009.

图/表 12

图1

图2

图3

表1

表2

表3

表4

表5

图4

图5

图6

图7

参考文献 27

[1]	田娟秀, 刘国才, 谷珊珊, 等. 医学图像分析深度学习方法研究与挑战[J]. 自动化学报, 2018, 44(3): 401-424.
[2]	POLAŃCZYK A, STRZELECKI M, WOŹNIAK T, et al. 3D blood vessels reconstruction based on segmented CT data for further simulations of hemodynamic in human artery branches[J]. Foundations of Computing and Decision Sciences, 2017, 42(4): 359-371. doi: 10.1515/fcds-2017-0018
[3]	曹玉红, 徐海, 刘荪傲, 等. 基于深度学习的医学影像分割研究综述[J]. 计算机应用, 2021, 41(8): 2273-2287. doi: 10.11772/j.issn.1001-9081.2020101638
[4]	施俊, 汪琳琳, 王珊珊, 等. 深度学习在医学影像中的应用综述[J]. 中国图象图形学报, 2020, 25(10): 1953-1981.
[5]	王琮智, 许梓璧, 马祥园, 等. 基于数据扩增和迁移学习的Mask R-CNN脑CT图像自动分割研究[J]. 中国生物医学工程学报, 2021, 40(4): 410-418.
[6]	LONG J, SHELHAMER E, DARRELL T. Fully convolutional networks for semantic segmentation[C]// Proceedings of the IEEE conference on computer vision and pattern recognition, Boston, USA: IEEE, 2015: 3431-3440.
[7]	RONNEBERGER O, FISCHER P, BROX T. U-net: Convolutional networks for biomedical image segmentation[C]//Medical Image Computing and Computer-Assisted Intervention-MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, Springer International Publishing, 2015: 234-241.
[8]	XIAO X, LIAN S, LUO Z, et al. Weighted res-unet for high-quality retina vessel segmentation[C]// 2018 9th international conference on information technology in medicine and education (ITME), Zhejiang, China: IEEE, 2018: 327-331.
[9]	OKTAY O, SCHLEMPER J, FOLGOC L L, et al. Attention u-net: Learning where to look for the pancreas[J]. arXiv preprint arXiv:1804.03999, 2018.
[10]	ZHOU Z, SIDDIQUEE M M R, TAJBAKHSH N, et al. Unet++: Redesigning skip connections to exploit multiscale features in image segmentation[J]. IEEE Transactions on Medical Imaging, 2019, 39(6): 1856-1867. doi: 10.1109/TMI.42
[11]	JADERBERG M, SIMONYAN K, ZISSERMAN A. Spatial transformer networks[J]. Advances in Neural Information Processing Systems, 2015, 28: 2017-2025.
[12]	SINHA A, DOLZ J. Multi-scale self-guided attention for medical image segmentation[J]. IEEE Journal of Biomedical and Health Informatics, 2020, 25(1): 121-130. doi: 10.1109/JBHI.6221020
[13]	VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]// 31st Conference on Neural Information Processing Systems(NIPS 2017), California, USA: Curran Associates, 2017: 5998-6008.
[14]	CHEN J, LU Y, YU Q, et al. Transunet: Transformers make strong encoders for medical image segmentation[J]. arXiv preprint arXiv:2102.04306, 2021.
[15]	CAO H, WANG Y, CHEN J, et al. Swin-unet: Unet-like pure transformer for medical image segmentation[C]// European conference on computer vision. Tel Aviv, Israel: Cham: Springer Nature Switzerland, 2022: 205-218.
[16]	LIU Z, LIN Y, CAO Y, et al. Swin transformer: Hierarchical vision transformer using shifted windows[C]// Proceedings of the IEEE/CVF international conference on computer vision, Montreal, Canada: IEEE, 2021: 10012-10022.
[17]	LIANG J, CAO J, SUN G, et al. Swinir: Image restoration using swin transformer[C]// Proceedings of the IEEE/CVF international conference on computer vision, Montreal, Canada: IEEE, 2021: 1833-1844.
[18]	LIN A, CHEN B, XU J, et al. Ds-transunet: Dual swin transformer u-net for medical image segmentation[J]. IEEE Transactions on Instrumentation and Measurement, 2022, 71: 1-15.
[19]	LI J, CHEN N, ZHOU H, et al. MCRformer: Morphological Constraint Reticular Transformer for 3D Medical Image Segmentation[J]. Expert Systems with Applications, 2023: 120877.
[20]	曾春艳, 严康, 王志锋, 等. 深度学习模型可解释性研究综述[J]. 计算机工程与应用, 2021, 57(8): 1-9. doi: 10.3778/j.issn.1002-8331.2012-0357
[21]	LOPEZ-PAZ D, NISHIHARA R, CHINTALA S, et al. Discovering causal signals in images[C]// Proceedings of the IEEE conference on computer vision and pattern recognition, Honolulu, USA: IEEE, 2017: 6979-6987.
[22]	LAKE B M, ULLMAN T D, TENENBAUM J B, et al. Building machines that learn and think like people[J]. Behavioral and Brain Sciences, 2017, 40: e253. doi: 10.1017/S0140525X16001837
[23]	KUANG K, XIONG R, CUI P, et al. Stable prediction with model misspecification and agnostic distribution shift[C]// Proceedings of the AAAI Conference on Artificial Intelligence, New York, USA: AAAI, 2020, 34(04): 4485-4492.
[24]	ZHANG X, CUI P, XU R, et al. Deep stable learning for out-of-distribution generalization[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Montreal, Canada: IEEE, 2021: 5372-5382.
[25]	RAHIMI A, RECHT B. Random features for large-scale kernel machines[J]. Advances in Neural Information Processing Systems, 2007, 20: 1177-1184.
[26]	TAKIKAWA T, ACUNA D, JAMPANI V, et al. Gated-scnn: Gated shape cnns for semantic segmentation[C]// Proceedings of the IEEE/CVF international conference on computer vision, Seoul, Korea: IEEE, 2019: 5229-5238.
[27]	JHA A, KUMAR A, PANDE S, et al. MT-UNET: a novel U-Net based multi-task architecture for visual scene understanding[C]// 2020 IEEE International Conference on Image Processing (ICIP), Abu Dhabi, United Arab Emirates: IEEE, 2020: 2191-2195.

Methods	Year	Mean DSC	Mean HD	DSC
Methods	Year	Mean DSC	Mean HD	Aorta	Gallbladder	Kidney(L)	Kidney(R)	Liver	Pancreas	Spleen	Stomach
U-Net^[7]	2015	76.85	39.70	89.07	69.72	77.77	68.60	93.43	53.98	86.67	75.58
Att-Unet^[9]	2018	77.77	36.02	89.55	68.88	77.98	71.11	93.57	58.04	87.30	75.75
*Trans-Unet^[14]	2021	76.65	31.84	86.92	59.67	81.42	74.92	94.17	55.62	84.38	76.06
*Swin-Unet^[15]	2021	76.63	24.95	85.28	65.64	80.22	76.29	93.80	55.17	86.89	69.73
MT-Unet^[27]	2022	78.59	26.59	87.92	64.99	81.47	77.29	93.06	59.46	87.75	76.81
MCRformer^[19]	2023	80.46	20.79	85.78	67.55	85.83	80.29	94.34	59.34	90.85	79.73
CRformer	2023	80.01	19.39	86.55	67.89	84.76	79.81	94.07	58.59	90.09	78.28

Methods	Year	Mean DSC	DSC
Methods	Year	Mean DSC	RV	Myo	LV
R50-Unet^[14]	2021	87.55	87.10	80.63	94.92
R50-AttUnet^[14]	2021	86.75	87.58	79.20	93.47
*Trans-Unet^[14]	2021	90.37	88.12	87.80	95.20
*Swin-Unet^[15]	2021	90.24	88.35	87.43	94.94
MT-Unet^[27]	2022	90.43	86.64	89.04	95.62
MCRformer^[19]	2023	90.75	88.93	88.20	95.11
CRformer	2023	90.95	88.85	88.50	95.50

方法	Swin- Unet^[15]	Swin- Unet+CR	MCR-former^[19]	MCR-former+CR
Mean DSC	78.68	78.97	80.46	80.01
Aorta	84.90	85.98	85.78	86.55
Gallbladder	67.72	64.68	67.55	67.89
Kidney(L)	81.21	83.41	85.83	84.76
Kidney(R)	77.89	79.99	80.29	79.81
Liver	93.65	93.71	94.34	94.07
Pancreas	60.52	58.33	59.34	58.59
Spleen	89.29	89.78	90.85	90.09
Stomach	74.30	75.91	79.73	78.28

方法	Swin- Unet^[15]	Swin- Unet+CR	MCR- former^[19]	MCR-former+CR
Mean HD	22.96	21.68	20.79	19.39
Aorta	13.64	6.38	10.33	10.30
Gallbladder	27.32	30.92	28.18	22.94
Kidney(L)	31.55	30.55	28.51	28.69
Kidney(R)	37.67	43.49	34.22	29.34
Liver	21.65	19.12	18.47	15.71
Pancreas	11.94	12.64	12.94	12.75
Spleen	22.90	14.39	17.72	19.62
Stomach	17.04	15.92	15.93	15.75

方法	Swin- Unet^[15]	Swin- Unet+CR	MCR-former^[19]	MCR- former+CR
Mean DSC	90.24	90.26	90.75	90.95
RV	88.35	88.56	88.93	88.85
Myo	87.43	87.36	88.20	88.50
LV	94.94	94.87	95.11	95.50

基于因果约束的Transformer医学图像分割方法

Causal Restraint Transformer for Medical Image Segmentation

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

图/表 12

参考文献 27

相关文章 2

编辑推荐

Metrics

本文评价

[1]	陈栋, 李明, 陈淑文. 结合Transformer和多层特征聚合的高光谱图像分类算法[J]. 数据与计算发展前沿, 2023, 5(3): 138-151.
[2]	刘琦玮,李俊,顾蓓蓓,赵泽方. TSAIE：图像增强文本的多模态情感分析模型[J]. 数据与计算发展前沿, 2022, 4(3): 131-140.