JPEG File Fragment Recognition Based on Transformer and Depth-Wise Convolution

doi:10.11871/jfdc.issn.2096-742X.2025.06.001

Abstract

Abstract:

[Objective] This paper aims to design a highly effective JPEG file fragment recognition method. [Methods] This work constructs a deep network for file fragment recognition based on the Transformer and depthwise convolution. The network uses original bytes as input. First, the embedding layer is utilized to reduce data dimensions and learn the relationships between bytes. The output of the embedding layer is respectively fed into a global feature extraction branch network composed of Transformer encoders and a local feature extraction branch network consisting of inception modules. Finally, the extracted global features and local features are concatenated and fed into the decision module for classification. [Results] On the FFT-75 dataset, for identifying 75 different types of file fragments, an overall classification accuracy of 70.7% (512 bytes) and 83.3% (4096 bytes) and a JPEG file fragment classification accuracy of 91.8% (512 bytes) and 94.2% (4096 bytes) can be achieved. [Conclusions] Both in terms of the overall classification accuracy and the JPEG file fragment classification accuracy, moderate improvement can be achieved when compared with reference methods, which verifies the effectiveness of the proposed fusion strategy of global features and local features.

Key words: electronic data forensics, file fragment type identification, JPEG file, Transformer, CNN

ZHU Nan,HUANG Zhiyuan. JPEG File Fragment Recognition Based on Transformer and Depth-Wise Convolution[J]. Frontiers of Data and Computing, 2025, 7(6): 1-12, https://cstr.cn/32002.14.jfdc.CN10-1649/TP.2025.06.001.

Figures/Tables 13

Fig.1

Fig.2

Fig.3

Fig.4

Fig.5

Fig.6

Table 1

Table 2

Table 3

Table 4

Table 5

Table 6

Table 7

References 29

[1]	刘浩阳, 杨秀雷. 公安机关电子数据司法鉴定的现状与挑战[J]. 中国司法鉴定, 2022, 1: 54-59.
[2]	赵宪伟, 刘政, 徐衍. 海量电子数据审查的实现路径[J]. 中国司法鉴定, 2022, 1: 30-37.
[3]	SESTER J, HAYES D, SCANLON M, et al. A comparative study of support vector machine and neural networks for file type identification using n-gram an-alysis[J]. Forensic Science International: Digital Investigation, 2021, 36: 301121.
[4]	KARRESAND M, SHAHMEHRI N. Oscar - File type identification of binary data in disk clusters and RAM pages[C]// Proceedings of the 21st IFIP TC-11 International Information Security Conference, Boston, MA, US, 2006: 413-424.
[5]	KARRESAND M, SHAHMEHRI N. File type identification of data fragments by their binary structure[C]// Proceedings of the IEEE Information Assurance Wo-rkshop, Egham, Surrey, UK, 2006: 140-147.
[6]	AXELSSON S, BAJWA K A, SRIKANTH M V. File fragment analysis using normalized compression distance[C]// Proceedings of the 9th IFIP WG 11.9 International Conference on Digital Forensics, Orlando, FL, USA, 2013:171-182.
[7]	BEEBE N L, MADDOX L A, LIU L, et al. Sceadan: Using concatenated n-gram vectors for improved file and data type classification[J]. IEEE Transactions on Information Forensics and Security, 2013, 8(9): 1519-1530.
[8]	WANG F, QUACH T T, WHEELER J, et al. Sparse coding for n-gram feature extraction and training for file fragment classification[J]. IEEE Transactions on Information Forensics and Security, 2018, 13(10): 25 53-2562.
[9]	MASOUMI M, KESHAVARZ A, FOTOHI R. File fragment recognition based on content and statistical features[J]. Multimedia Tools and Applications, 2021, 80: 18859-18874.
[10]	TEIMOURI M, SEYEDGHORBAN Z, AMIRJANI F. Fragments-Expert: A graphical user interface MATLAB toolbox for classification of file fragments[J]. Concurrency and Computation: Practice and Experience, 2021, 33(9): e6154.
[11]	XU T, XU M, REN Y, et al. A file fragment classification method based on grayscale image[J]. Journal of Computer, 2014, 9(8): 1863-1870.
[12]	江翀. 文件碎片类型的高效识别技术研究与实现[D]. 杭州电子科技大学, 2017.
[13]	ALSUBHI M A, MOUSSA A N, ALFAQIRI A S, et al. GenSpec: A file fragment classification approach[C]// Proceedings of the 23rd International Conference on Computing and Information Technology (ICCIT-1441), Tabuk, Saudi Arabia, 2020: 1-5.
[14]	BHATT M, MISHRA A, KABIR M W U, et al. Hierarchy-based file fragment classification[J]. Machine Learning & Knowledge Extraction, 2021, 2: 216-232.
[15]	WANG Y, SU Z, SONG D. File fragment type identification with convolutional neural networks[C]// Proceedings of International Conference on Machine Le-arning Technologies, Jinan, China, 2018: 41-47.
[16]	CHEN Q, LIAO Q, JIANG Z L, et al. File fragment classification using grayscale image conversion and deep learning in digital forensics[C]// Proceedings of IEEE Security and Privacy Workshops, San Francisco, CA, USA, 2018: 140-147.
[17]	MITTAL G, KORUS P, MEMON N. FiFTy: large-scale file fragment type identification using convolutional neural networks[J]. IEEE Transactions on Information Forensics and Security, 2021, 16: 28-41.
[18]	SAAIM K M, FELEMBAN M, ALSALEH S, ALMULHEM A. Light-weight file fragments classification using depthwise separable convolutions[C]// Me-ng W, Fischer-Hübner S, Jensen C D (eds) ICT Systems Security and Privacy Protection. SEC 2022. IFIP Advances in Information and Communication Te-chnology, Springer, Cham, 2022, 648: 196-211.
[19]	ZHU N, LIU Y, WANG K, et al. File fragment type identification based on CNN and LSTM[C]// Proceedings of the 7th International Conference on Digital Signal Processing, Chengdu, China, 2023: 16-22.
[20]	WANG Y, WU K, LIU W, et al. Image representation and deep inception-attention for file-type and malware classification[C]// Proceedings of the 2023 IE-EE International Symposium on Circuits and Systems, Monterey, CA, USA, 2023: 1-5.
[21]	LIU W, WANG Y, WU K, et al. A byte sequence is worth an image: CNN for file fragment classification using bit shift and n-Gram embeddings[C]// Proceedings of the 2023 IEEE 5th International Conference on Artificial Intelligence Circuits and Systems, Hangzhou, China, 2023: 1-5.
[22]	VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]// Proceedings of the 31st Conference on Neural Information Processing Systems, Long Beach, CA, USA, 2017: 1-11.
[23]	DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An image is worth 16x16 words: Transformers for image recognition at scale[C]// Proceedings of the 9th International Conference on Learning Representati-ons, Virtual Event, Austria, 2021: 1-21.
[24]	SZEGEDY C, LIU W, JIA Y, et al. Going deeper with convolutions[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, 2015: 1-9.
[25]	MITTAL G, KORUS P, MEMON N. File Fragment Type (FFT)-75 Dataset[EB/OL]. 2022. http://dx.doi.org/10.21227/kfxw-8084.
[26]	ZHANG T, XU W, LUO B, et al. Depth-wise convolutions in vision Transformers for efficient training on small datasets[J]. Neurocomputing, 2025, 617: 128998.
[27]	ZHU J, LIU D, CHEN H, et al. DTSFormer: Decoupled temporal-spatial diffusion transformer for enhanced long-term time series forecasting[J] Knowledge Based Systems, 2025, 309: 112828.
[28]	DAI Y, GIESEKE F, OEHMCKE S, et al. Attentional feature fusion[C]// Proceedings of the IEEE/CVF Wi-nter Conference on Applications of Computer Vision, Waikoloa, HI, USA, 2021: 3560-3569.
[29]	WANG Y, WEI W. Local and global feature attention fusion network for face recognition[J]. Pattern Recognition, 2025, 161: 111227.

种类	类型数	文件扩展名
位图	6	JPEG, TIFF, HEIC, BMP, GIF, PNG
RAW	11	ARW, 3FR, CR2, GPR, NEF, NRW, RAF, ORF, PEF, RW2, DNG
矢量图形	3	AI, EPS, PSD
视频	7	MOV, MP4, 3GP, AVI, MKV, OGV, WEBM
压缩文件	13	APK, JAR, MSI, DMG, 7Z, BZ2, DEB, GZ, PKG, RAR, RPM, XZ, ZIP
可执行文件	4	EXE, MACH-O, ELF, DLL
办公文档	7	DOC, PPT, KEY, XLS, PPTX, XLSX, DOCX
出版物	4	DJVU, EPUB, MOBI, PDF
可读性文本文件	9	MD, RTF, TXT, TEX, JSON, HTML, XML, LOG, CSV
音频	7	AIFF, FLAC, WMA, M4A, MP3, OGG, WAV
其他	4	PCAP, TTF, DWG, SQLITE

场景	类型数	文件扩展名
1	75	FFT-75所有文件类型
2	11	位图、RAW、矢量图形、视频、压缩文件、可执行文件、办公文档、出版物、可读性文本文件、音频各为一类，其它为一类
3	25	位图（6种），RAW（11种）, 视频（7种），其它为一类
4	5	JPEG，RAW，视频，5个除JPEG外的位图各为一类，其它为一类
5	2	JPEG，其它为一类
6	2	JPEG，RAW（11种）、3GP、MOV、MKV、TIFF、HEIC为一类

方法	整体分类准确率		JPEG文件碎片分类准确率
方法	512 字节	4,096 字节	512 字节	4,096 字节
NN-GF	45.3	46.7	53.2	48.6
NN-CO	64.3	75.2	77.5	92.5
FiFty	65.5	77.4	83.5	86.3
DSC	65.9	78.5	85.2	88.4
CNN-LSTM	66.5	78.6	87.0	90.2
DIA	68.3	81.0	87.5	90.6
BSnGE	71.0	82.1	89.2	93.4
T-DWC	70.7	83.3	91.8	94.2

方法	整体分类准确率		JPEG文件碎片分类准确率
方法	512 字节	4,096 字节	512 字节	4,096 字节
FiFty	90.2	94.1	98.6	99.1
DSC	87.1	94.2	98.2	99.2
CNN-LSTM	90.4	94.5	98.8	99.2
DIA	92.8	96.1	98.1	98.7
BSnGE	93.6	96.1	97.4	98.6
T-DWC	90.2	95.1	98.8	99.3

方法	整体分类准确率		JPEG文件碎片分类准确率
方法	512 字节	4,096 字节	512 字节	4,096 字节
FiFty	99.0	99.1	99.1	99.2
DSC	99.0	99.2	99.2	99.3
CNN-LSTM	99.0	99.2	99.3	99.3
DIA	99.1	99.2	99.3	99.4
BSnGE	99.2	99.3	99.3	99.4
T-DWC	99.1	99.2	99.3	99.4