Automatic Teeth Segmentation on Dental Panoramic Radiographs with Deep Learning

doi:10.11871/jfdc.issn.2096-742X.2024.03.017

Abstract

Abstract:

[Objective] Accurate segmentation of individual teeth from dental panoramic radiographs is essential for providing important assistance to the diagnosis and treatment planning of various diseases in digital dentistry. However, accurate segmentation of individual teeth is a challenging task due to the overlapping anatomical structures, blurry boundaries, and artifacts exhibiting in panoramic radiographs. [Methods] To solve these problems, we propose a deep learning-based method for accurate and fully automatic segmentation of individual teeth from the panoramic radiograph. The proposed method combines multiple deep neural networks and utilizes a teeth morphology map together with a multi-scale morphology-guided attention mechanism (MMAM) to precisely segment each tooth. [Results] We evaluate the segmentation performance of our proposed method and compare it with the state-of-the-art methods on testing datasets collected from the real-world clinical scenarios. The segmentation results indicate that our proposed method achieves more accurate segmentation performance (mean Dice: 94.65%, mean Jaccard: 90.29%, mean Recall: 94.06%, and mean Precision: 95.62%). [Conclusions] The proposed method might be applied in the first step of automatic pathology diagnosis from dental panoramic radiographs.

Key words: artificial intelligence, deep learning, dental panoramic radiographs, deep neural network, teeth segmentation

KOU Dazhi. Automatic Teeth Segmentation on Dental Panoramic Radiographs with Deep Learning[J]. Frontiers of Data and Computing, 2024, 6(3): 162-172, https://cstr.cn/32002.14.jfdc.CN10-1649/TP.2024.03.017.

Figures/Tables 9

Fig.1

Fig.2

Table 1

Fig.3

Table 2

Table 3

Fig.4

Table 4

Fig. 5

References 39

[1]	KRIZHEVSKY A, SUTSKEVER I, HINTON G. ImageNet classification with deep convolutional neural networks[J]. Advances in Neural Information Processing Systems, 2012, 25(2) : 84-90.
[2]	SHELHAMER E, LONG J, DARRELL T. Fully convolutional networks for semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(4): 640-651. doi: 10.1109/TPAMI.2016.2572683 pmid: 27244717
[3]	RONNEBERGER O, FISCHER P, BROX T. U-net: Convolutional networks for biomedical image segmentation.[C]. In Proceedings of 18th International Conference on Medical Image Computing and Computer-Assisted Intervention, 2015: 234-241.
[4]	ZHANG Z, LIU Q, WANG Y. Road Extraction by Deep Residual U-Net[J]. IEEE Geoscience and Remote Sensing Letters, 2018, 15(5): 749-753.
[5]	CHEN L C, ZHU Y, PAPANDREOU G, et al. Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation[C]. European Conference on Computer Vision, 2018: 11211.
[6]	REDMON J, DIVVALA S, GIRSHICK R, et al. You Only Look Once: Unified, Real-Time Object Detection[C]. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016: 779-788.
[7]	REN S, HE K, GIRSHICK R, et al. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137-1149. doi: 10.1109/TPAMI.2016.2577031 pmid: 27295650
[8]	LIN T, DOLLÁR P, GIRSHICK R, et al. Feature Pyramid Networks for Object Detection.[C]. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017: 936-944.
[9]	HE K, GKIOXARI G, DOLLÁR P, et al. Mask R-CNN.[C]. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017: 2980-2988.
[10]	CHU X, ZHENG A, ZHANG X, et al. Detection in Crowded Scenes: One Proposal, Multiple Predictions.[C]. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020: 12211-12220.
[11]	SHEN D, WU G, SUK H I. Deep Learning in Medical Image Analysis[J]. Annual Review of Biomedical Engineering, 2017, 19(1): 221-248.
[12]	HWANG J J, JUNG Y H, CHO B H, et al. An overview of deep learning in the field of dentistry[J]. Imaging science in dentistry. 2019, 49(1): 1-7.
[13]	MINNEMA J, VAN EIJNATTEN M, HENDRIKSEN A A, et al. Segmentation of dental cone-beam CT scans affected by metal artifacts using a mixed-scale dense convolutional neural network[J]. Medical Physics, 2019, 46(11): 5027-5035. doi: 10.1002/mp.13793 pmid: 31463937
[14]	WANG H, MINNEMA J, BATENBURG K J, et al. Multiclass CBCT ImageSegmentation for Orthodontics with Deep Learning[J]. Journal of Dental Research, 2021, 100(9): 943-949.
[15]	WANG Y, ZHAO L, SONG Z, et al. Organ at Risk Segmentation in Head and Neck CT Images by Using a Two-Stage Segmentation Framework Based on 3D U-Net: in IEEE Access, 10.1109/ACCESS.2019.2944958[P]. 2019, 7: 144591-144602.
[16]	QIU B, GUO J, KRAEIMA J, et al. Automatic segmentation of the mandible from computed tomography scans for 3D virtual surgical planning using the convolutional neural network[J]. Physics in Medicine and Biology, 2019, 64(17): 175020.
[17]	CUI Z, LI C, WNAG W. ToothNet: Automatic Tooth Instance Segmentation and Identification From Cone Beam CT Images[C]. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019: 6361-6370.
[18]	CUI Z, FANG Y, MEI L, et al. A fully automatic AI system for tooth and alveolar bone segmentation from cone-beam CT images[J]. Nature Communications, 2022, 13(1): 2096. doi: 10.1038/s41467-022-29637-2 pmid: 35440592
[19]	ZHU H, CAO Z, LIAN L, et al. CariesNet: a deep 1eaming approach for segmentation of mu1ti stage caries 1esion from ora1 panoramic x ray image[J]. Neural Computing and Applications, 2022: 1-9.
[20]	YANG H, JO E, KIM H J, et al. Deep Learning for Automated Detection of Cyst and Tumors of the Jaw in Panoramic Radiographs[J]. Journal of Clinical Medicine, 2020, 9(6): 1839.
[21]	SUKEGAWA S, YOSHII K, HARA T, et al. Multi-task deep learning model for classification of dental implant brand and treatment stage using dental panoramic radiograph images[J]. Biomolecules, 2021, 11(6): 815.
[22]	KROIS J, EKERT T, MEINHOLD L, et al. Deep learning for the radiographic detection of periodontal bone loss[J]. Scientific Reports, 2019, 9(1): 1-6.
[23]	KIM J, LEE H S, SONG I S, et al. DeNTNet: Deep Neural Transfer Network for the detection of periodontal bone loss using panoramic dental radiographs[J]. Scientific Reports, 2019, 9(1): 17615. doi: 10.1038/s41598-019-53758-2 pmid: 31772195
[24]	CHANG H J, LEE S J, YONG T H, et al. Deep Learning Hybrid Method to Automatically Diagnose Periodontal Bone Loss and Stage Periodontitis[J]. Scientific Reports, 2020, 10(1): 7531.
[25]	JIANG L, CHEN D, CAO Z, et al. A two-stage deep learning architecture for radiographic staging of periodontal bone loss[J]. BMC Oral Health, 2022, 22(1): 106. doi: 10.1186/s12903-022-02119-z pmid: 35365122
[26]	TUZOFF D V, TUZOVA L N, BORNSTEINM M, et al. Tooth detection and numbering in panoramic radiographs using convolutional neural networks[J]. Dentomaxillofacial Radiology, 2019, 48(4): 20180051.
[27]	VINAYAHALINGAM S, XI T, BERGÉ S, et al. Automated detection of third molars and mandibular nerve by deep learning[J]. Scientific Reports, 2019, 9(1): 9007. doi: 10.1038/s41598-019-45487-3 pmid: 31227772
[28]	KOCH TL, PERSLEV M, IGEL C, et al. Accurate segmentation of dental panoramic radiographs with U-Nets[C]. 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019), 2019: 15-19.
[29]	CHEN Q, ZHAO Y, LIU Y, et al. MSLPNet: multi-scale location perception network for dental panoramic X-ray image segmentation[J]. Neural Computing and Applications, 2021, 33(16): 10277-10291.
[30]	JADER G, FONTINELI J, RUIZ M, et al. Deep instance segmentation of teeth in panoramic X-ray images[C]. 31st Conference on Graphics, Patterns and Images (SIBGRAPI), 2018: 400-407
[31]	LEE J H, HAN S S, KIM Y H, et al. Application of a fully deep convolutional neural network to the automation of tooth segmentation on panoramic radiographs[J]. Oral Surgery, Oral Medicine, Oral Pathology and Oral Radiology, 2020, 129(6): 635-642.
[32]	LEITE A F, GERVEN A V, WILLEMS H, et al. Artificial intelligence-driven novel tool for tooth detection and segmentation on panoramic radiographs[J]. Clinical Oral Investigations, 2020, 25(4): 2257-2267.
[33]	CHA J Y, YOON H I, YEO I S, et al. Panoptic segmentation on panoramic radiographs: Deep learning-based segmentation of various structures including maxillary sinus and mandibular canal[J]. Journal of Clinical Medicine, 2021, 10(12): 2577
[34]	VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]. NIPS'17: Proceedings of the 31st International Conference on Neural Information Processing Systems, 2017: 6000-6010.
[35]	OKTAY O, SCHLEMPER J, FOLGOC L L, et al. Attention U-Net: Learning Where to Look for the Pancreas[C]. 1st Conference on Medical Imaging with Deep Learning(MIDL 2018), 2018.
[36]	HU J, SHEN L, ALBANIE G, et al. Squeeze-and-excitation networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(8): 2011-2023. doi: 10.1109/TPAMI.2019.2913372 pmid: 31034408
[37]	ROY G, NAVAB N, WACHINGER C. Recalibrating fully convolutional networks with spatial and channel “Squeeze and Excitation” blocks[J]. IEEE Transactions on Medical Imaging, 2019, 38(2): 540-549.
[38]	CHOLLET F, Keras[OL]. https://github.com/fchollet/keras.
[39]	CHEN J, LU Y, YU Q, et al. TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation[J/OL]. arXiv:2102.043062021.https://doi.org/10.48550/arXiv.2102.04306.

牙齿类型	Dice(%)	Jaccard(%)	Recall(%)	Precision(%)
中切牙	94.46±3.74	89.72±6.29	93.29±5.74	96.01±4.50
侧切牙	95.00±3.17	90.63±5.36	94.16±4.65	96.10±4.21
尖牙	95.48±3.22	91.51±5.22	94.93±4.08	96.23±4.40
第一前磨牙	94.23±6.61	89.56±7.72	93.72±7.74	95.07±7.24
第二前磨牙	94.99±6.72	90.93±7.67	94.48±7.51	95.71±7.03
第一磨牙	94.40±6.56	89.85±7.57	93.84±7.42	95.18±6.96
第二磨牙	95.32±2.28	91.15±4.05	95.19±3.79	95.65±3.62
第三磨牙	93.34±10.26	88.56±11.27	92.43±10.92	94.78±11.32
平均	94.65±5.70	90.29±7.07	94.06±6.71	95.62±6.39

方法	Dice(%)	Jaccard(%)	Recall(%)	Precision(%)
Mask R-CNN	91.07±10.01	84.56±10.62	88.08±10.62	94.76±10.91
TDN+UNet	92.36±9.39	86.77±11.36	90.66±11.19	94.83±9.06
TDN+Residual U-Net	93.41±8.37	88.40±9.93	94.28±8.40	92.97±9.80
TDN+Attention U-Net	93.78±8.51	89.07±10.00	94.44±9.11	93.55±9.16
TDN+DeepLab V3+	93.80±8.04	89.01±9.23	94.45±8.35	93.52±9.02
TDN+TransUNet	94.09±8.24	89.55±9.50	93.58±8.66	94.93±9.01
Ours(w/o MMAM)	94.48±5.79	89.92±7.39	93.29±7.20	96.00±5.98
Ours(w/o MD)	94.57±6.49	90.17±7.84	94.28±7.31	95.16±7.22
Ours	94.65±5.70	90.29±7.07	94.06±6.71	95.62±6.39