Deficient Shoeprint Matching via Fusion of Multi-Scale Deep Features

doi:10.11871/jfdc.issn.2096-742X.2026.04.005

Abstract

Abstract:

[Background] Shoeprints, common physical evidence at crime scenes, play a key role in criminal investigations. The “shoeprint+surveillance” technology narrows down suspect identification by combining on-site shoeprints with video surveillance. [Objective] To address limitations of existing shoeprint matching algorithms (limited task specificity, poor robustness, low accuracy, high computational complexity), this paper proposes a deficient shoeprint matching algorithm based on improved EfficientNetV2-S with multi-scale deep feature fusion. [Methods] We constructed the DP-database with 7,455 shoeprint samples (1,342 categories, 70% deficient). Using lightweight EfficientNetV2-S as backbone, we optimized Fused-MBConv via ECA attention and integrated Stage2-Stage6 deep features through multi-level fusion for multi-scale shoeprint information extraction. [Results] On CSS-200, it achieved 79% top-1 accuracy, 89.5% top-10 accuracy, and 100% top-200 recall. On CS-Database, top-1 detection rate reached 100% for high-quality and bloodstained shoeprints, 100% top-5 for dust ones, outperforming mainstream algorithms. [Conclusions] The proposed method improves accuracy and robustness while maintaining efficiency, effectively recognizes low-quality shoeprints (deficient, blurred), and meets public security's rapid investigation requirements.

Key words: matching algorithm, feature fusion, partial shoe print, EfficientNetV2-S

SONG Jiuchang, TANG Yunqi, GUO Wei, ZHANG Jianan. Deficient Shoeprint Matching via Fusion of Multi-Scale Deep Features[J]. Frontiers of Data and Computing, 2026, 8(4): 72-86, https://cstr.cn/32002.14.jfdc.CN10-1649/TP.2026.04.005.

Figures/Tables 21

Fig.1

Fig.2

Table 1

Table 2

Fig.3

Fig.4

Table 3

Fig.5

Fig.6

Fig.7

Fig.8

Fig.9

Table 4

Fig.10

Fig.11

Table 5

Table 6

Table 7

Fig.12

Fig.13

Fig.14

References 44

[1]	金益锋, 白艳平, 刘寰. 全国16个省份足迹自动识别系统应用情况分析[J]. 刑事技术, 2017, 42(6): 504-507. doi: 10.16467/j.1008-3650.2017.06.016
[2]	张嘉楠, 孟昕, 唐云祁. 浅谈视频侦查与现场足迹联系方法[J]. 法制博览, 2016(14): 172-173.
[3]	张涛, 朱振东, 王慧, 等. 基于鞋印图像的性别预测方法[J]. 计算机工程, 2022, 48(11): 306-313. doi: 10.19678/j.issn.1000-3428.0062940
[4]	袁小萍. 对快侦快破现行案件的几点思考[J]. 公安大学学报(社会科学版), 1998(4): 34-37.
[5]	刘伟, 姜乐怡, 朱婷鸽, 等. 鞋印图像匹配研究现状与发展趋势[J]. 西安邮电大学学报, 2021, 26(2): 70-76.
[6]	BELSER C, INEICHEN M, PFEFFERLI P. Evaluation of the ISAS system after two years of practical experience in forensic police work[J]. Forensic Science International, 1996, 82(1): 53-58. doi: 10.1016/0379-0738(96)01966-4
[7]	MIKKONEN S, ASTIKAINEN T. Databased classification system for shoe sole patterns-identification of partial footwear impression found at a scene of crime[J]. Journal of Forensic Sciences, 1994, 39(5): 1227-1236. doi: 10.1520/JFS13708J
[8]	ALEXANDRE G. Computerized classification of the shoeprints of burglar soles[J]. Forensic Science International, 1996, 82(1): 59-65. doi: 10.1016/0379-0738(96)01967-6
[9]	ASHLEY W. What shoe was that? The use of computerized image database to assist in identification[J]. Forensic Science International, 1996, 82(1): 7-20. doi: 10.1016/0379-0738(96)01962-7
[10]	ALGARNI G, AMIANE M. A novel technique for automatic shoeprint image retrieval[J]. Forensic Science International, 2008, 181(1): 10-14. doi: 10.1016/j.forsciint.2008.07.004
[11]	TEAGUE M R. Image analysis via the general theory of moments[J]. Journal of the Optical Society of America, 1980, 70(8): 920-930. doi: 10.1364/JOSA.70.000920
[12]	HUYNH C, DE CHAZAL P, MCERLEAN D, et al. Automatic classification of shoeprints for use in forensic science based on the Fourier transform[C]// International Conference on Image Processing, IEEE, 2003: 569-572.
[13]	DE CHAZAL P, FLYNN J, REILLY R B. Automated processing of shoeprint images based on the Fourier transform for use in forensic science[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2005, 27(3): 341-350. doi: 10.1109/TPAMI.2005.48 pmid: 15747790
[14]	CERVELLI F, DARDI F, CARRATO S. A translational and rotational invariant descriptor for automatic shoe marks[C]// European Signal Processing Conference. Aalborg, Denmark: IEEE, 2010: 1665-1669.
[15]	OJALA T, PIETIKAINEN M, HARWOOD D. Performance evaluation of texture measures with classification based on Kullback discrimination of distributions[C]// Pattern Recognition, 1994. Vol. 1 - Conference A: Computer Vision & Image Processing. Proceedings of the 12th IAPR International Conference on IEEE, 1994: 582-583.
[16]	LOWE D G. Distinctive image feature from scale-invariant key points[J]. International Journal of Computer Vision, 2004, 60(2): 91-110. doi: 10.1023/B:VISI.0000029664.99615.94
[17]	DALAL N, TRIGGS B. Histograms of oriented gradients for human detection[C]// IEEE Computer Society Conference on Computer Vision & Pattern Recognition, IEEE, 2005: 886-893.
[18]	WANG X, SUN H, YU Q, et al. Automatic shoeprint retrieval algorithm for real crime scenes[C]// Asian Conference on Computer Vision. Lecture Notes in Computer Science, Springer, 2014: 399-413.
[19]	彭飞. 基于局部语义块和流行排序的鞋印图像匹配[D]. 大连: 大连海事大学, 2019.
[20]	周思越. 基于局部语义滤波器组的低质量鞋印图像匹配算法[D]. 大连: 大连海事大学, 2020.
[21]	焦扬, 杨传颖, 石宝. 基于SVM相关反馈的鞋印图像匹配算法[J]. 计算机科学, 2020, 47(S2): 244-247.
[22]	CUI J, ZHAO X, LI D. An automatic shoeprint retrieval method using neural codes for commercial shoeprint scanners[C]// Chinese Conference on Computer Vision. Lecture Notes in Computer Science, Springer, 2017: 158-169.
[23]	KONG B, SUPANCIC J, RAMANAN D, et al. Cross-domain image matching with deep feature maps[J]. International Journal of Computer Vision, 2018, 127(3): 1750-1768.
[24]	CUI J, ZHAO X, LIU N, et al. Robust shoeprint retrieval method based on local-to-global feature matching for real crime scenes[J]. Journal of Forensic Sciences, 2018, 64(1): 422-430. doi: 10.1111/jfo.2019.64.issue-2
[25]	MA Z, DING Y, WEN S, et al. Shoeprint image retrieval with multi-part weighted CNN[J]. IEEE Access, 2019, 7: 59728-59736. doi: 10.1109/Access.6287639
[26]	史文韬, 唐云祁. 基于微调VGG-16的现场鞋印匹配算法[J]. 中国人民公安大学学报(自然科学版), 2020, 26(3): 22-29.
[27]	史文韬, 唐云祁. 基于选择性卷积特征描述子融合的鞋印匹配算法[J]. 科学技术与工程, 2021, 21(16): 6772-6779.
[28]	韩雨彤. 基于深度学习网络的鞋印匹配算法[D]. 北京: 中国人民公安大学, 2022.
[29]	赵梦影. 基于深度特征的跨模态鞋印图像匹配[D]. 北京: 北京邮电大学, 2020.
[30]	韩雨彤, 郭威, 唐云祁, 等. 基于混合域注意力机制的鞋印匹配算法[J]. 中国人民公安大学学报(自然科学版), 2022, 28(2): 22-30.
[31]	辛一冉, 唐云祁, 蔡能斌. 融合全局与局部特征的鞋印特征提取网络设计[J]. 激光与光电子学进展, 2023, 60(6): 162-171.
[32]	WEN Z, CURRAN J M, WEVERS G. Shoeprint image retrieval and crime scene shoeprint image linking by using convolutional neural network and normalized cross correlation[J]. Science & Justice, 2023, 63(4): 439-450.
[33]	TAN M, LE Q V. EfficientNetV2: Smaller models and faster training[C]// Proceedings of the 38th International Conference on Machine Learning (ICML), PMLR, 2021: 10096-10106.
[34]	HOWARD A, SANDLER M, CHU G, et al. Searching for MobileNetV3[C]// Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), IEEE, 2019: 1314-1324.
[35]	WANG Q, WU B, ZHU P, et al. ECA-Net: Efficient channel attention for deep convolutional neural networks[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, 2020: 11534-11542.
[36]	ZHOU Z, SIDDIQUEE M M R, TAJBAKHSH N, et al. UNet++: Redesigning skip connections to exploit multiscale features in image segmentation[J]. IEEE Transactions on Medical Imaging, 2020, 39(6): 1856-1867. doi: 10.1109/TMI.2019.2959609 pmid: 31841402
[37]	ZHANG P, WANG D, LU H, et al. Deep hierarchical feature concatenation for visual tracking[J]. IEEE Signal Processing Letters, 2017, 24(8): 1228-1232. doi: 10.1109/LSP.2017.2717083
[38]	HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, 2016: 770-778.
[39]	史文韬. 基于卷积神经网络的鞋印匹配算法研究[D]. 北京: 中国人民公安大学, 2021: 13-17.
[40]	RICHETELLI N, LEE M C, LASKY C A, et al. Classification of footwear outsole patterns using Fourier transform and local interest points[J]. Forensic Science International, 2017, 275: 102-109. doi: S0379-0738(17)30084-1 pmid: 28343023
[41]	段硕古. 细粒度鞋印图像检索算法研究[D]. 大连: 大连海事大学, 2022.
[42]	KORTYLEWSKI A, ALBRECHT T, VETTER T. Unsupervised footwear impression analysis and retrieval from crime scene data[C]// Asian Conference on Computer Vision. Lecture Notes in Computer Science, Springer, 2015: 644-658.
[43]	WANG X, ZHANG C, WU Y, et al. A manifold ranking based method using hybrid features for crime scene shoeprint retrieval[J]. Multimedia Tools and Applications, 2017, 76(9): 21629-21649. doi: 10.1007/s11042-016-4029-3
[44]	LUOSTARINEN T, LEHMUSSOLA A. Measuring the accuracy of automatic shoeprint recognition methods[J]. Journal of Forensic Sciences, 2014, 59(6): 1627-1634. doi: 10.1111/1556-4029.12474 pmid: 24846765

模型	参数量/h	推理时间/h	训练时间/h	跨域测试mAP/%
ResNet-50	23.5	11.2	5	54.2
EfficientNetV2-S	86.1	12.3	12	66.4
VisionTransformer-Base	144.2	14.9	16	52.3
SwinTransformer-Small	74.8	13.2	33	44.7

层级	模块	尺寸	通道	模块数
1	Conv3×3	300×300	24	1
2	FusedMBConv	150×150	48	2
3	FusedMBConv	75×75	64	4
4	FusedMBConv	38×38	128	4
5	MBConv	19×19	160	6
6	MBConv	19×19	256	9
7	MBConv	10×10	518	15
8	FC	10×10	1,280	1

符号	符号说明
Q	查询图像的集合
q	单个查询图像
\|Q\|	查询集合的大小（查询图像总数）
k	排名位置或指定的排名阈值
top(q)	查询q的匹配在匹配结果中的排名
M	查询q的相关（正确匹配）样本总数
P(k)	前k个结果中正确匹配的数量/k
rel(k)	指示函数，正确匹配为1，否则为0
Σ	求和符号
AP(q)	查询q的Average Precision值

Stage	accuracy/%
Stage	Top1	Top5	Top1%	Top5%	Top10%
Original	62.5	71	91	97	99
P2346	70	80	94	97.5	99.5
P2356	67	77	96	99.5	100
P2345	66	76.5	96	98	99.5
P2456	64.5	72	95.5	99	100
P3456	67.5	77.5	95	98.5	100
P23456	77.5	81	96	99.5	100

Methods	accuracy/%
Methods	Top1	Top10	Top1%	Top10%
微调VGG-16[26]	52	75.5	83	87
VGG-19＋SCDA[27]	62.5	85.5	92.5	94.5
Resnet34＋混合域[30]	56	79	89.5	94.5
Efficientnet-B3＋PCA[31]	69	75	95.5	99.5
本文方法	79	89.5	98	100