数据与计算发展前沿 ›› 2021, Vol. 3 ›› Issue (6): 17-34.
doi: 10.11871/jfdc.issn.2096-742X.2021.06.002
收稿日期:
2021-11-11
出版日期:
2021-12-20
发布日期:
2022-01-26
通讯作者:
陈琼
作者简介:
陈琼, 华南理工大学计算机科学与工程学院,博士,硕士生导师,中国计算机学会会员、中国人工智能学会离散智能计算专委会委员,主要研究方向为机器学习、图像分类和分割、深度强化学习。基金资助:
CHEN Qiong(),YANG Yong(),HUANG Tianlin(),FENG Yuan
Received:
2021-11-11
Online:
2021-12-20
Published:
2022-01-26
Contact:
CHEN Qiong
摘要:
【目的】对小样本语义分割方法进行系统而全面的介绍,为后续小样本分割算法设计工作提供参考。【方法】当前的小样本分割方法借助基于度量的元学习方法来完成少样本情况下的语义分割任务。根据度量工具是否可学习,将小样本分割算法分为基于参数结构和基于原型结构的小样本分割算法,简述了两类算法的优缺点。【结果】对该领域的一些经典工作和近年来的工作做了具体的分析,并给出了小样本分割算法的主要应用场景。【结论】在此基础上,分析了小样本分割存在的关键问题和挑战,对小样本分割未来的发展方向和趋势进行了讨论。
陈琼,杨咏,黄天林,冯媛. 小样本图像语义分割综述[J]. 数据与计算发展前沿, 2021, 3(6): 17-34.
CHEN Qiong,YANG Yong,HUANG Tianlin,FENG Yuan. A Survey on Few-Shot Image Semantic Segmentation[J]. Frontiers of Data and Computing, 2021, 3(6): 17-34.
表1
小样本分割算法在PASCAL-5i的分割性能"
模型 | 方法 | 骨干网络 | Mean-IoU (1-shot) | FB-IoU (1-shot) | Mean-IoU (5-shot) | FB-IoU (5-shot) |
---|---|---|---|---|---|---|
PANet[ SG-One[ FWB[ PPNet[ SimPropNet[ SAGNN[ MLC[ ASGNet[ | 原型 原型 原型 原型 原型 原型 原型 原型 | VGG-16 VGG-16 ResNet-101 ResNet-50 ResNet-50 ResNet-50 ResNet-50 ResNet-50 | 48.1 46.3 56.2 51.5 57.2 62.1 62.1 59.3 | 68.5 63.1 - - 73.0 73.2 69.2 | 55.7 47.1 59.9 62.0 60.0 62.8 66.1 63.9 | 70.7 65.9 - - 72.9 73.3 74.2 |
CANet[ PFENet[ BriNet[ PMMS[ PGNet[ CRNet[ SCL_PFENet[ SCL_CANet[ CMN[ CWT[ | 参数 参数 参数 参数 参数 参数 参数 参数 参数 参数 | ResNet-50 ResNet-50 ResNet-50 ResNet-50 ResNet-50 ResNet-50 ResNet-50 ResNet-50 ResNet-50 ResNet-50 | 55.4 60.8 57.1 55.2 56.0 55.7 61.8 57.5 62.8 56.4 | 66.2 73.3 69.9 66.8 71.9 70.3 72.3 | 57.1 61.9 56.8 58.5 58.8 62.9 59.2 63.7 63.7 | 69.6 73.9 70.5 71.5 72.8 70.7 72.8 |
表2
小样本分割算法在COCO-20i的分割性能"
模型 | 方法 | 骨干网络 | Mean-IoU (1-shot) | FB-IoU (1-shot) | Mean-IoU (5-shot) | FB-IoU (5-shot) |
---|---|---|---|---|---|---|
PANet[ FWB[ PPNet[ SAGNN[ MLC[ ASGNet[ | 原型 原型 原型 原型 原型 原型 | VGG-16 ResNet-101 ResNet-50 ResNet-101 ResNet-50 ResNet-50 | 20.9 21.2 29.0 37.2 33.9 34.6 | 59.2 60.9 60.4 | 29.7 23.7 38.5 42.9 40.6 42.5 | 63.5 63.4 67.0 |
CANet[ PFENet[ BriNet[ PMMS[ SCL_PFENet[ CMN[ CWT[ | 参数 参数 参数 参数 参数 参数 参数 | ResNet-50 ResNet-101 ResNet-50 ResNet-50 ResNet-50 ResNet-50 ResNet-50 | 32.4 34.4 29.6 37.0 39.3 32.9 | 58.6 61.7 | 37.4 34.3 39.9 43.1 41.3 | 61.9 63.3 |
[1] | Dong G, Yan Y, Shen C, et al. Real-time high-perfo-rmance semantic image segmentation of urban street scenes[J]. IEEE Transactions on Intelligent Transpor-tation Systems, 2020,22(6):3258-3274. |
[2] | Badrinarayanan V, Kendall A, Cipolla R. Segnet: A deep convolutional encoder-decoder architecture for image segmentation[J]. IEEE transactions on pattern analysis and machine intelligence, 2017,39(12):2481-2495. |
[3] | Shaban A, Bansal S, Liu Z, et al. One-Shot Learning for Semantic Segmentation[C]//British Machine Vision Conference, 2017: 167. 1- 167.13. |
[4] | Vinyals O, Blundell C, Lillicrap T, et al. Matching net-works for one shot learning[J]. Advances in neural infor-mation processing systems, 2016,29:3630-3638. |
[5] | Huisman M, van Rijn J N, Plaat A. A survey of deep meta-learning[J]. Artificial Intelligence Review, 2021: 1-59. |
[6] | Boudiaf M, Kervadec H, Masud Z I, et al. Few-Shot Seg-mentation Without Meta-Learning: A Good Transductive Inference Is All You Need? [C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021: 13979-13988. |
[7] | Hospedales T, Antoniou A, Micaelli P, et al. Meta-learning in neural networks: A survey[J]. arXiv preprint arXiv: 2004. 05439, 2020. |
[8] | Vanschoren J. Meta-learning: A survey[J]. arXiv preprint arXiv: 1810. 03548, 2018. |
[9] | Chen W Y, Liu Y C, Kira Z, et al. A closer look at few-shot classification[J]. arXiv preprint arXiv: 1904. 04232, 2019. |
[10] | Xu W, Wang H, Tu Z. Attentional Constellation Nets for Few-Shot Learning[C/OL]. International Conference on Learning Representations, 2021 -06-23[2021-12-11].https://openreview.net/pdf?id=vujTf_I8Kmc. |
[11] | Min J, Kang D, Cho M. Hypercorrelation squeeze for few-shot segmentation[J]. arXiv preprint arXiv: 2104. 01538, 2021. |
[12] | Wu Z, Shi X, Lin G, et al. Learning meta-class memory for few-shot semantic segmentation [C]//Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021: 517-526. |
[13] | Zhu C, Chen F, Ahmed U, et al. Semantic relation reaso-ning for shot-stable few-shot object detection [C]//Proce-edings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021: 8782-8791. |
[14] | Hu H, Bai S, Li A, et al. Dense Relation Distillation with Context-aware Aggregation for Few-Shot Object Detec-tion [C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021: 10185-10194. |
[15] | Sun B, Li B, Cai S, et al. FSCE: Few-shot object detec-tion via contrastive proposal encoding [C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021: 7352-7362. |
[16] | Lai X, Tian Z, Jiang L, et al. Semi-Supervised Semantic Segmentation With Directional Context-Aware Consistency [C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021: 1205-1214. |
[17] | Zhong Y, Yuan B, Wu H, et al. Pixel Contrastive-Con-sistent Semi-Supervised Semantic Segmentation [C]//Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021: 7273-7282. |
[18] | Chang Y T, Wang Q, Hung W C, et al. Weakly-supervised semantic segmentation via sub-category exploration [C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020: 8991-9000. |
[19] | Zhang M, Zhou Y, Zhao J, et al. A survey of semi-and weakly supervised semantic segmentation of images[J]. Artificial Intelligence Review, 2020,53(6):4259-4288. |
[20] | Wang C, Farazi M, Barnes N. Recursive Training for Zero-Shot Semantic Segmentation [C]//2021 International Joint Conference on Neural Networks (IJCNN), IEEE, 2021: 1-8. |
[21] | Bucher M, Vu T H, Cord M, et al. Zero-shot semantic segmentation[J]. Advances in Neural Information Proces-sing Systems, 2019,32:468-479. |
[22] | Rakelly K, Shelhamer E, Darrell T, et al. Conditional networks for few-shot semantic segmentation[J/OL]. 2018 -04-4[2021-12-11]. https://openreview.net/pdf?id=SkMjFKJwG. |
[23] | Dong N, Xing E P. Few-shot semantic segmentation with prototype learning [C]//BMVC, 2018,3(4):79. |
[24] | Tian P, Wu Z, Qi L, et al. Differentiable meta-learning model for few-shot semantic segmentation [C]//Pro-ceedings of the AAAI Conference on Artificial Intel-ligence. 2020,34(07):12087-12094. |
[25] | Yang Y, Meng F, Li H, et al. A new local transformation module for few-shot segmentation [C]//International Con-ference on Multimedia Modeling. Springer, Cham, 2020: 76-87. |
[26] | Yang B, Liu C, Li B, et al. Prototype mixture models for few-shot semantic segmentation [C]//European Confe-rence on Computer Vision. Springer, Cham, 2020: 763-778. |
[27] | Bhunia A K, Bhunia A K, Ghose S, et al. A deep one-shot network for query-based logo retrieval[J]. Pattern Recognition, 2019,96:106965. |
[28] | Nguyen K, Todorovic S. Feature weighting and boosting for few-shot segmentation [C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. 2019: 622-631. |
[29] | Snell J, Swersky K, Zemel R S. Prototypical networks for few-shot learning[J]. arXiv preprint arXiv: 1703. 05175, 2017. |
[30] | Li G, Jampani V, Sevilla-Lara L, et al. Adaptive Prototype Learning and Allocation for Few-Shot Segmentation [C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021: 8334-8343. |
[31] | Liu Y, Zhang X, Zhang S, et al. Part-aware prototype network for few-shot semantic segmentation [C]//Eur-opean Conference on Computer Vision. Springer, Cham, 2020: 142-158. |
[32] | Zhang C, Lin G, Liu F, et al. Canet: Class-agnostic se-gmentation networks with iterative refinement and atten-tive few-shot learning [C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019: 5217-5226. |
[33] | Tian Z, Zhao H, Shu M, et al. Prior guided feature enri-chment network for few-shot segmentation[J]. IEEE Tran-sactions on Pattern Analysis & Machine Intelligence, 2020 ( 01):1-1. |
[34] | Yang X, Wang B, Chen K, et al. Brinet: Towards brid-ging the intra-class and inter-class gaps in one-shot seg-mentation[J]. arXiv preprint arXiv: 2008. 06226, 2020. |
[35] | He K, Zhang X, Ren S, et al. Deep residual learning for image recognition [C]//Proceedings of the IEEE conference on computer vision and pattern recognition, 2016: 770-778. |
[36] | Liu W, Zhang C, Lin G, et al. Crnet: Cross-reference networks for few-shot segmentation [C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020: 4165-4173. |
[37] | Chen L C, Papandreou G, Kokkinos I, et al. Semantic image segmentation with deep convolutional nets and fully connected crfs[J]. arXiv preprint arXiv: 1412. 7062, 2014. |
[38] | Chen L C, Papandreou G, Kokkinos I, et al. Semantic image segmentation with deep convolutional nets and fully connected crfs[J]. arXiv preprint arXiv: 1412. 7062, 2014. |
[39] | Chen L C, Zhu Y, Papandreou G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation [C]//Proceedings of the European confe-rence on computer vision (ECCV), 2018: 801-818. |
[40] | Zhao H, Shi J, Qi X, et al. Pyramid scene parsing network [C]//Proceedings of the IEEE conference on computer vision and pattern recognition, 2017: 2881-2890. |
[41] | Zhang B, Xiao J, Qin T. Self-Guided and Cross-Guided Learning for Few-Shot Segmentation [C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021: 8312-8321. |
[42] | Xie G S, Liu J, Xiong H, et al. Scale-Aware Graph Neural Network for Few-Shot Semantic Segmentation [C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021: 5475-5484. |
[43] | Tao A, Sapra K, Catanzaro B. Hierarchical multi-scale attention for semantic segmentation[J]. arXiv preprint arXiv: 2005. 10821, 2020. |
[44] | Huang Z, Wang X, Huang L, et al. Ccnet: Criss-cross attention for semantic segmentation [C]//Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019: 603-612. |
[45] | Zhang H, Zhang H, Wang C, et al. Co-occurrent features in semantic segmentation [C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog-nition, 2019: 548-557. |
[46] | Yuan Y, Chen X, Wang J. Object-contextual repre-sentations for semantic segmentation[C]//Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part VI 16. Springer International Publishing, 2020: 173-190. |
[47] | Li X, Zhong Z, Wu J, et al. Expectation-maximization attention networks for semantic segmentation [C]//Procee-dings of the IEEE/CVF International Conference on Com-puter Vision, 2019: 9167-9176. |
[48] | Zhang C, Lin G, Liu F, et al. Pyramid graph networks with connection attentions for region-based one-shot semantic segmentation [C]//Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019: 9587-9595. |
[49] | Wang X, Girshick R, Gupta A, et al. Non-local neural networks [C]//Proceedings of the IEEE conference on computer vision and pattern recognition, 2018: 7794-7803. |
[50] | Xie G S, Xiong H, Liu J, et al. Few-shot semantic segm-entation with cyclic memory network [C]//Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021: 7293-7302. |
[51] | Lu Z, He S, Zhu X, et al. Simpler is Better: Few-shot Semantic Segmentation with Classifier Weight Transfor-mer [C]//Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021: 8741-8750. |
[52] | Siam M, Oreshkin B N, Jagersand M. Amp: Adaptive masked proxies for few-shot segmentation [C]//Procee-dings of the IEEE/CVF International Conference on Computer Vision, 2019: 5249-5258. |
[53] | Zhang X, Wei Y, Yang Y, et al. Sg-one: Similarity gui-dance network for one-shot semantic segmentation[J]. IEEE transactions on cybernetics, 2020,50(9):3855-3865. |
[54] | Wang K, Liew J H, Zou Y, et al. Panet: Few-shot image semantic segmentation with prototype alignment [C]//Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019: 9197-9206. |
[55] | Zhang X, Wei Y, Li Z, et al. Rich Embedding Features for One-Shot Semantic Segmentation[J]. IEEE Transactions on Neural Networks and Learning Systems, 2021: 1-10. |
[56] | Gairola S, Hemani M, Chopra A, et al. Simpropnet: Improved similarity propagation for few-shot image segmentation[J]. arXiv preprint arXiv: 2004. 15014, 2020. |
[57] | Kim A. Fast slic[EB/OL].https://github.com/Algy/fast-slic. |
[58] | Yang L, Zhuo W, Qi L, et al. Mining Latent Classes for Few-shot Segmentation[J]. arXiv preprint arXiv: 2103. 15402, 2021. |
[59] | Ouyang C, Biffi C, Chen C, et al. Self-supervision with superpixels: Training few-shot medical image segmen-tation without annotation [C]//European Conference on Computer Vision. Springer, Cham, 2020: 762-780. |
[60] | Feyjie A R, Azad R, Pedersoli M, et al. Semi-supervised few-shot learning for medical image segmentation[J]. arXiv preprint arXiv: 2003. 08462, 2020. |
[61] | Guo Y, Wang H, Hu Q, et al. Deep Learning for 3D Point Clouds: A Survey[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020,99:1-1. |
[62] | 王文曦, 李乐林. 深度学习在点云分类中的研究综述[J/OL]. 计算机工程与应用:1-17[ 2021- 10- 25].http://kns.cnki.net/kcms/detail/11.2127.TP.20211021.1009.004.html. |
[63] | Landrieu L, Simonovsky M. Large-scale point cloud semantic segmentation with superpoint graphs [C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2018: 4558-4567. |
[64] | Hu Q, Yang B, Xie L, et al. Randla-net: Efficient semantic segmentation of large-scale point clouds [C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020: 11108-11117. |
[65] | Qi C R, Su H, Mo K, et al. Pointnet: Deep learning on point sets for 3d classification and segmentation [C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2017: 652-660. |
[66] | 李梦怡, 朱定局. 基于全卷积网络的图像语义分割方法综述[J]. 计算机系统应用, 2021,30(09):41-52. |
[67] | 刘念. 基于小样本深度学习的遥感地物分类与分割方法研究[D]. 重庆大学, 2019. |
[68] | Pan X, Shi J, Luo P, et al. Spatial as deep: Spatial cnn for traffic scene understanding [C]//Thirty-Second AAAI Conference on Artificial Intelligence. 2018: 7276— 7283. |
[69] | Wang Z, Ren W, Qiu Q. Lanenet: Real-time lane detec-tion networks for autonomous driving[J]. arXiv preprint arXiv: 1807. 01726, 2018. |
[70] | Hou Y, Ma Z, Liu C, et al. Learning lightweight lane detection cnns by self attention distillation [C]//Procee-dings of the IEEE/CVF international conference on computer vision, 2019: 1013-1021. |
[71] | Everingham M, Van Gool L, Williams C K I, et al. The pascal visual object classes (voc) challenge[J]. Interna-tional journal of computer vision, 2010,88(2):303-338. |
[72] | Hariharan B, Arbeláez P, Bourdev L, et al. Semantic contours from inverse detectors [C]//2011 International Conference on Computer Vision. IEEE, 2011: 991-998. |
[73] | Lin T Y, Maire M, Belongie S, et al. Microsoft coco: Common objects in context [C]//European conference on computer vision. Springer, Cham, 2014: 740-755. |
[74] | Li X, Wei T, Chen Y P, et al. Fss-1000: A 1000-class dataset for few-shot segmentation [C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020: 2869-2878. |
[1] | 许淞源,刘峰. ESDRec:一种面向地球大数据平台的数据推荐模型[J]. 数据与计算发展前沿, 2023, 5(1): 55-64. |
[2] | 蒲晓蓉,黄佳欣,刘军池,孙家瑜,罗纪翔,赵越,陈柯成,任亚洲. 面向临床需求的CT图像降噪综述[J]. 数据与计算发展前沿, 2021, 3(6): 35-49. |
[3] | 何涛,王桂芳,马廷灿. 基于词嵌入语义异常的跨学科研究内容发现方法[J]. 数据与计算发展前沿, 2021, 3(6): 50-59. |
[4] | 张怡宁,何洪波,王闰强. 热门数字音频预测技术综述[J]. 数据与计算发展前沿, 2021, 3(4): 81-92. |
[5] | 陈子健,李俊,岳兆娟,赵泽方. 基于自编码器与属性信息的混合推荐模型[J]. 数据与计算发展前沿, 2021, 3(3): 148-155. |
[6] | 肖建平,龙春,赵静,魏金侠,胡安磊,杜冠瑶. 基于深度学习的网络入侵检测研究综述[J]. 数据与计算发展前沿, 2021, 3(3): 59-74. |
[7] | 李序,连一峰,张海霞,黄克振. 网络安全知识图谱关键技术[J]. 数据与计算发展前沿, 2021, 3(3): 9-18. |
[8] | 赵伟昱,张宏海,仲波. 基于深度学习的遥感影像地块分割方法[J]. 数据与计算发展前沿, 2021, 3(2): 133-141. |
[9] | 沈飙,陈扬,杨琛,刘博文. 海洋科学中尺度涡的计算机视觉检测和分析方法[J]. 数据与计算发展前沿, 2020, 2(6): 30-41. |
[10] | 任荟颖,王婧,王彦棡. 基于AutoML的湍流建模[J]. 数据与计算发展前沿, 2020, 2(4): 121-131. |
[11] | 张圣林,林潇霏,孙永谦,张玉志,裴丹. 基于深度学习的无监督KPI异常检测[J]. 数据与计算发展前沿, 2020, 2(3): 87-100. |
[12] | 陈雷,袁媛. 基于深度迁移学习的农业病害图像识别[J]. 数据与计算发展前沿, 2020, 2(2): 111-119. |
[13] | 孙哲南,张兆翔,王威,刘菲,谭铁牛. 2019 年人工智能新态势与新进展[J]. 数据与计算发展前沿, 2019, 1(2): 1-16. |
[14] | 刘成林. 文档图像识别技术回顾与展望[J]. 数据与计算发展前沿, 2019, 1(2): 17-25. |
[15] | 俞益洲, 马杰超, 石德君, 周振. 深度学习在医学影像分析中的应用综述[J]. 数据与计算发展前沿, 2019, 1(2): 37-52. |
阅读次数 | ||||||
全文 |
|
|||||
摘要 |
|
|||||