基于深度学习的口腔全景片牙齿自动分割方法

doi:10.11871/jfdc.issn.2096-742X.2024.03.017

摘要/Abstract

摘要：

【目的】从口腔全景片中准确分割每颗独立的牙齿对于口腔医学数字化中的疾病诊断和治疗至关重要，然而由于口腔全景片中存在重叠的解剖结构、模糊的边界和伪影，导致单个牙齿的精确分割成为一项具有挑战性的任务。【方法】为了解决这个问题，本文提出一种基于深度学习的方法，用于从口腔全景片中准确、自动地分割每颗独立的牙齿。所提出的方法结合了多个深度神经网络，并利用牙齿形态图和多尺度形态引导注意力机制（MMAM）来精确分割每个牙齿。【结果】在真实临床场景中收集的测试数据集上评估了这一方法的分割性能，并与目前先进的方法进行了比较。结果表明，本文提出的方法在单个牙齿上的分割结果为平均Dice系数94.65%，平均Jaccard系数90.29%，平均召回率94.06%，平均精确度95.62%，与其他方法相比获得了更准确的分割性能。【结论】基于深度学习的口腔全景片自动分割方法可以很好地应用于口腔医学数字化自动病理诊断之中。

关键词: 人工智能, 深度学习, 口腔全景片, 深度神经网络, 牙齿分割

Abstract:

[Objective] Accurate segmentation of individual teeth from dental panoramic radiographs is essential for providing important assistance to the diagnosis and treatment planning of various diseases in digital dentistry. However, accurate segmentation of individual teeth is a challenging task due to the overlapping anatomical structures, blurry boundaries, and artifacts exhibiting in panoramic radiographs. [Methods] To solve these problems, we propose a deep learning-based method for accurate and fully automatic segmentation of individual teeth from the panoramic radiograph. The proposed method combines multiple deep neural networks and utilizes a teeth morphology map together with a multi-scale morphology-guided attention mechanism (MMAM) to precisely segment each tooth. [Results] We evaluate the segmentation performance of our proposed method and compare it with the state-of-the-art methods on testing datasets collected from the real-world clinical scenarios. The segmentation results indicate that our proposed method achieves more accurate segmentation performance (mean Dice: 94.65%, mean Jaccard: 90.29%, mean Recall: 94.06%, and mean Precision: 95.62%). [Conclusions] The proposed method might be applied in the first step of automatic pathology diagnosis from dental panoramic radiographs.

Key words: artificial intelligence, deep learning, dental panoramic radiographs, deep neural network, teeth segmentation

寇大治. 基于深度学习的口腔全景片牙齿自动分割方法[J]. 数据与计算发展前沿, 2024, 6(3): 162-172.

KOU Dazhi. Automatic Teeth Segmentation on Dental Panoramic Radiographs with Deep Learning[J]. Frontiers of Data and Computing, 2024, 6(3): 162-172, https://cstr.cn/32002.14.jfdc.CN10-1649/TP.2024.03.017.

图/表 9

图1

图2

表1

图3

表2

表3

图4

表4

图5

参考文献 39

[1]	KRIZHEVSKY A, SUTSKEVER I, HINTON G. ImageNet classification with deep convolutional neural networks[J]. Advances in Neural Information Processing Systems, 2012, 25(2) : 84-90.
[2]	SHELHAMER E, LONG J, DARRELL T. Fully convolutional networks for semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(4): 640-651. doi: 10.1109/TPAMI.2016.2572683 pmid: 27244717
[3]	RONNEBERGER O, FISCHER P, BROX T. U-net: Convolutional networks for biomedical image segmentation.[C]. In Proceedings of 18th International Conference on Medical Image Computing and Computer-Assisted Intervention, 2015: 234-241.
[4]	ZHANG Z, LIU Q, WANG Y. Road Extraction by Deep Residual U-Net[J]. IEEE Geoscience and Remote Sensing Letters, 2018, 15(5): 749-753.
[5]	CHEN L C, ZHU Y, PAPANDREOU G, et al. Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation[C]. European Conference on Computer Vision, 2018: 11211.
[6]	REDMON J, DIVVALA S, GIRSHICK R, et al. You Only Look Once: Unified, Real-Time Object Detection[C]. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016: 779-788.
[7]	REN S, HE K, GIRSHICK R, et al. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137-1149. doi: 10.1109/TPAMI.2016.2577031 pmid: 27295650
[8]	LIN T, DOLLÁR P, GIRSHICK R, et al. Feature Pyramid Networks for Object Detection.[C]. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017: 936-944.
[9]	HE K, GKIOXARI G, DOLLÁR P, et al. Mask R-CNN.[C]. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017: 2980-2988.
[10]	CHU X, ZHENG A, ZHANG X, et al. Detection in Crowded Scenes: One Proposal, Multiple Predictions.[C]. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020: 12211-12220.
[11]	SHEN D, WU G, SUK H I. Deep Learning in Medical Image Analysis[J]. Annual Review of Biomedical Engineering, 2017, 19(1): 221-248.
[12]	HWANG J J, JUNG Y H, CHO B H, et al. An overview of deep learning in the field of dentistry[J]. Imaging science in dentistry. 2019, 49(1): 1-7.
[13]	MINNEMA J, VAN EIJNATTEN M, HENDRIKSEN A A, et al. Segmentation of dental cone-beam CT scans affected by metal artifacts using a mixed-scale dense convolutional neural network[J]. Medical Physics, 2019, 46(11): 5027-5035. doi: 10.1002/mp.13793 pmid: 31463937
[14]	WANG H, MINNEMA J, BATENBURG K J, et al. Multiclass CBCT ImageSegmentation for Orthodontics with Deep Learning[J]. Journal of Dental Research, 2021, 100(9): 943-949.
[15]	WANG Y, ZHAO L, SONG Z, et al. Organ at Risk Segmentation in Head and Neck CT Images by Using a Two-Stage Segmentation Framework Based on 3D U-Net: in IEEE Access, 10.1109/ACCESS.2019.2944958[P]. 2019, 7: 144591-144602.
[16]	QIU B, GUO J, KRAEIMA J, et al. Automatic segmentation of the mandible from computed tomography scans for 3D virtual surgical planning using the convolutional neural network[J]. Physics in Medicine and Biology, 2019, 64(17): 175020.
[17]	CUI Z, LI C, WNAG W. ToothNet: Automatic Tooth Instance Segmentation and Identification From Cone Beam CT Images[C]. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019: 6361-6370.
[18]	CUI Z, FANG Y, MEI L, et al. A fully automatic AI system for tooth and alveolar bone segmentation from cone-beam CT images[J]. Nature Communications, 2022, 13(1): 2096. doi: 10.1038/s41467-022-29637-2 pmid: 35440592
[19]	ZHU H, CAO Z, LIAN L, et al. CariesNet: a deep 1eaming approach for segmentation of mu1ti stage caries 1esion from ora1 panoramic x ray image[J]. Neural Computing and Applications, 2022: 1-9.
[20]	YANG H, JO E, KIM H J, et al. Deep Learning for Automated Detection of Cyst and Tumors of the Jaw in Panoramic Radiographs[J]. Journal of Clinical Medicine, 2020, 9(6): 1839.
[21]	SUKEGAWA S, YOSHII K, HARA T, et al. Multi-task deep learning model for classification of dental implant brand and treatment stage using dental panoramic radiograph images[J]. Biomolecules, 2021, 11(6): 815.
[22]	KROIS J, EKERT T, MEINHOLD L, et al. Deep learning for the radiographic detection of periodontal bone loss[J]. Scientific Reports, 2019, 9(1): 1-6.
[23]	KIM J, LEE H S, SONG I S, et al. DeNTNet: Deep Neural Transfer Network for the detection of periodontal bone loss using panoramic dental radiographs[J]. Scientific Reports, 2019, 9(1): 17615. doi: 10.1038/s41598-019-53758-2 pmid: 31772195
[24]	CHANG H J, LEE S J, YONG T H, et al. Deep Learning Hybrid Method to Automatically Diagnose Periodontal Bone Loss and Stage Periodontitis[J]. Scientific Reports, 2020, 10(1): 7531.
[25]	JIANG L, CHEN D, CAO Z, et al. A two-stage deep learning architecture for radiographic staging of periodontal bone loss[J]. BMC Oral Health, 2022, 22(1): 106. doi: 10.1186/s12903-022-02119-z pmid: 35365122
[26]	TUZOFF D V, TUZOVA L N, BORNSTEINM M, et al. Tooth detection and numbering in panoramic radiographs using convolutional neural networks[J]. Dentomaxillofacial Radiology, 2019, 48(4): 20180051.
[27]	VINAYAHALINGAM S, XI T, BERGÉ S, et al. Automated detection of third molars and mandibular nerve by deep learning[J]. Scientific Reports, 2019, 9(1): 9007. doi: 10.1038/s41598-019-45487-3 pmid: 31227772
[28]	KOCH TL, PERSLEV M, IGEL C, et al. Accurate segmentation of dental panoramic radiographs with U-Nets[C]. 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019), 2019: 15-19.
[29]	CHEN Q, ZHAO Y, LIU Y, et al. MSLPNet: multi-scale location perception network for dental panoramic X-ray image segmentation[J]. Neural Computing and Applications, 2021, 33(16): 10277-10291.
[30]	JADER G, FONTINELI J, RUIZ M, et al. Deep instance segmentation of teeth in panoramic X-ray images[C]. 31st Conference on Graphics, Patterns and Images (SIBGRAPI), 2018: 400-407
[31]	LEE J H, HAN S S, KIM Y H, et al. Application of a fully deep convolutional neural network to the automation of tooth segmentation on panoramic radiographs[J]. Oral Surgery, Oral Medicine, Oral Pathology and Oral Radiology, 2020, 129(6): 635-642.
[32]	LEITE A F, GERVEN A V, WILLEMS H, et al. Artificial intelligence-driven novel tool for tooth detection and segmentation on panoramic radiographs[J]. Clinical Oral Investigations, 2020, 25(4): 2257-2267.
[33]	CHA J Y, YOON H I, YEO I S, et al. Panoptic segmentation on panoramic radiographs: Deep learning-based segmentation of various structures including maxillary sinus and mandibular canal[J]. Journal of Clinical Medicine, 2021, 10(12): 2577
[34]	VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]. NIPS'17: Proceedings of the 31st International Conference on Neural Information Processing Systems, 2017: 6000-6010.
[35]	OKTAY O, SCHLEMPER J, FOLGOC L L, et al. Attention U-Net: Learning Where to Look for the Pancreas[C]. 1st Conference on Medical Imaging with Deep Learning(MIDL 2018), 2018.
[36]	HU J, SHEN L, ALBANIE G, et al. Squeeze-and-excitation networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(8): 2011-2023. doi: 10.1109/TPAMI.2019.2913372 pmid: 31034408
[37]	ROY G, NAVAB N, WACHINGER C. Recalibrating fully convolutional networks with spatial and channel “Squeeze and Excitation” blocks[J]. IEEE Transactions on Medical Imaging, 2019, 38(2): 540-549.
[38]	CHOLLET F, Keras[OL]. https://github.com/fchollet/keras.
[39]	CHEN J, LU Y, YU Q, et al. TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation[J/OL]. arXiv:2102.043062021.https://doi.org/10.48550/arXiv.2102.04306.

牙齿类型	Dice(%)	Jaccard(%)	Recall(%)	Precision(%)
中切牙	94.46±3.74	89.72±6.29	93.29±5.74	96.01±4.50
侧切牙	95.00±3.17	90.63±5.36	94.16±4.65	96.10±4.21
尖牙	95.48±3.22	91.51±5.22	94.93±4.08	96.23±4.40
第一前磨牙	94.23±6.61	89.56±7.72	93.72±7.74	95.07±7.24
第二前磨牙	94.99±6.72	90.93±7.67	94.48±7.51	95.71±7.03
第一磨牙	94.40±6.56	89.85±7.57	93.84±7.42	95.18±6.96
第二磨牙	95.32±2.28	91.15±4.05	95.19±3.79	95.65±3.62
第三磨牙	93.34±10.26	88.56±11.27	92.43±10.92	94.78±11.32
平均	94.65±5.70	90.29±7.07	94.06±6.71	95.62±6.39

方法	Dice(%)	Jaccard(%)	Recall(%)	Precision(%)
Mask R-CNN	91.07±10.01	84.56±10.62	88.08±10.62	94.76±10.91
TDN+UNet	92.36±9.39	86.77±11.36	90.66±11.19	94.83±9.06
TDN+Residual U-Net	93.41±8.37	88.40±9.93	94.28±8.40	92.97±9.80
TDN+Attention U-Net	93.78±8.51	89.07±10.00	94.44±9.11	93.55±9.16
TDN+DeepLab V3+	93.80±8.04	89.01±9.23	94.45±8.35	93.52±9.02
TDN+TransUNet	94.09±8.24	89.55±9.50	93.58±8.66	94.93±9.01
Ours(w/o MMAM)	94.48±5.79	89.92±7.39	93.29±7.20	96.00±5.98
Ours(w/o MD)	94.57±6.49	90.17±7.84	94.28±7.31	95.16±7.22
Ours	94.65±5.70	90.29±7.07	94.06±6.71	95.62±6.39