چالش های موقعیت یابی متن فارسی در تصاویر طبیعی و اهمیت مجموعه دادگان جدید برای ارزیابی مدل های یادگیری عمیق
الموضوعات : electrical and computer engineering
زبیر رئیسی
1
,
رسول دامنی
2
,
اسماعیل سارانی
3
,
ولی محمد نظرزهی حاد
4
1 - گروه مهندسی برق، دانشکده مهندسی دریا، دانشگاه دریانوردی و علوم دریایی چابهار، چابهار، ایران
2 - گروه مهندسی برق، دانشکده مهندسی دریا، دانشگاه دریانوردی و علوم دریایی چابهار، چابهار، ایران
3 - گروه مهندسی برق، دانشکده مهندسی دریا، دانشگاه دریانوردی و علوم دریایی چابهار، چابهار، ایران
4 - گروه مهندسی برق، دانشکده مهندسی دریا، دانشگاه دریانوردی و علوم دریایی چابهار، چابهار، ایران
الکلمات المفتاحية: مجموعه داده متن فارسی, موقعیتیابی متن در تصاویر, مدلهای یادگیری عمیق , مجموعه داده FATD.,
ملخص المقالة :
به دلیل پیچیدگیهای ساختاری خط فارسی و کمبود مجموعهدادههای (دادگانهای) استاندارد و معتبر، موقعیتیابی متن فارسی و جداسازی کلمات در تصاویر ثبتشده با دوربینهای معمولی، همچنان به عنوان یک چالش کلیدی در حوزه پردازش تصویر مطرح است. در این مقاله، ابتدا یک مجموعه دادگان جامع برای موقعیت یابی متن فارسی با نام FATD معرفی شده است.این مجموعه شامل بیش از 2000 تصویر متنوع است که متنهایی با فونتها، اندازهها و زاویههای مختلف، در شرایط محیطی متفاوت و با سطوح پیچیدگی بالا را در بر میگیرد. سپس، در مجموع شش مدل یادگیری عمیق شامل دو مدل مبتنی بر شبکه عصبی کانولوشنی (8YOLOv و CRAFT)، دو مدل ترانسفورمری (RRDETR و RRBDETR) و همچنین دو مدل زبان-بینایی (-VL5/2Qwen و 2Florence-)، تحت شرایط یکسان بر روی مجموعه دادگان معرفی شده، ارزیابی و مقایسه می شوند. نتایج ارزیابی نشان میدهد که ترانسفورمرها به قیمت هزینه محاسباتی بالاتر، عملکرد بهتر و دقیقتری را ارائه میدهند و بر بر اساس معیار ارزیابی H-mean دقتی تا 65 درصد را کسب میکنند. در مقابل، شبکه های عصبی کانولوشنی (CNN) با سرعت پردازش مناسب، دقت رقابتی ارائه میکنند. همچنین علیرغم آموزش محدود مدل های جامع زبان-بینایی روی دادههای متنی فارسی، بر اساس معیار ارزیابی H-meanاین مدلها در موقعیت یابی عملکرد قابل قبولی را به نمایش میگذارند.
[1] Y. Zhu, C. Yao, and X. Bai, "Scene text detection and recognition: Recent advances and future trends," Front. Comp Sci., vol. 10, no. 1, pp. 19-36, 2016.
[2] H. Lin, P. Yang, and F. Zhang, "Review of scene text detection and recognition," Arch. Comput. Methods Eng, pp. 1-22, 2019.
[3] Z. Raisi, M. A. Naiel, P. Fieguth, S. Wardell, and J. Zelek, Text Detection and Recognition in the Wild: A Review, arXiv: arXiv:2006.04305, 2020.
[4] Z. Raisi and J. Zelek, "Text detection and recognition for robot localization," J. Electr. Comput. Eng. Innov. JECEI, vol. 12, no. 1, pp. 163174, 2024.
[5] X. Han, J. Gao, C. Yang, Y. Yuan, and Q. Wang, "Focus entirety and perceive environment for arbitrary-shaped text detection," IEEE Trans. Multimed., 2024.
[6] J. Xu et al., "FSANet: Feature shuffle and adaptive channel attention network for arbitrary shape scene text detection," Neurocomputing, Article ID: 129443, 2025.
[7] K. Wang, B. Babenko, and S. Belongie, "End-to-end scene text recognition," in Proc. Int. Conf. on Comp. Vision, pp. 1457-1464, Barcelona, Spain, 6-13 Nov. 2011.
[8] A. Bissacco, M. Cummins, Y. Netzer, and H. Neven, " PhotoOCR: Reading text in uncontrolled conditions," in Proc. IEEE Int. Conf. on Comp. Vision, pp. 785-792, Sydney, Australia, 1-8 Dec. 2013.
[9] X. Zhou et al., "EAST: An efficient and accurate scene text detector," in Proc. IEEE Conf. on Comp. Vision and Pattern Recognition, pp. 5551-5560, Honolulu, HI, USA, 21-26 Jul.2017.
[10] M. Liao, B. Shi, and X. Bai, "Textboxes++: A single-shot oriented scene text detector," IEEE Trans Image Process, vol. 27, no. 8, pp. 3676-3690, Apr. 2018.
[11] Y. Baek, B. Lee, D. Han, S. Yun, and H. Lee, "Character region awareness for text detection," in Proc. IEEE Conf. on Comp. Vision and Pattern Recognition, pp. 9365-9374, Long Beach, CA, USA, 16-17 Jun. 2019.
[12] Z. Raisi, M. A. Naiel, G. Younes, S. Wardell, and J. S. Zelek, "Transformer-Based text detection in the wild," in Proc. IEEE/CVF Conf. on Computer Vision and Pattern Recognition Workshops, pp. 3162-3171, Nashville, TN, USA,19-25 Jun. 2021.
[13] Z. Raisi, G. Younes, and J. Zelek, "Arbitrary shape text detection using transformers,” in Proc. 26th Int. Conf. on Pattern Recognition, pp. 3238-3245, Montreal, Canada,2022.
[14] Z. Raisi and J. Zelek, "Visual place recognition from end-to-end semantic scene text features," Front. Robot. AI, vol. 11, Article ID: 1424883, Sept. 2024.
[15] ز. حیدران داروقه امنیه، س. م. رستگار فاطمی، م. رستگارپور و گ. آقایی قزوینی، "افزایش دقت شبکه¬های عصبی کانولوشنی مبتنی بر مدل چهار-جریان با فیلترهای پردازش تصویر و نگاشت خطی¬ساز فضای عدم تشابه"، نشریه روش¬های هوشمند در صنعت برق، سال 16، شماره 61، صص. 28-1، بهار 1404. [16] م. روحی، ج. مظلوم، م. ع. پورمینا و ب. قلمکاری، "طبقه¬بندی سکته مغزی بر اساس روش یادگیری عمیق در سیستم تصویربرداری ریزموجی از مغز"، نشریه روش¬های هوشمند در صنعت برق، سال 15، شماره 57، صص. 132-121، بهار 1403. [17] ف. علی¬مرادی، ف. رحمانی، ل. ربیعی، م. خوانساریو م. مازوچی، "ساخت مجموعه داده تصاویر برای تشخیص و بازشناسی متن در تصاویر، "فصلنامه اطلاعات و ارتباطات ایران، سال 14، شماره 53، صص. 95-78، پاییز-زمستان 1401. [18] S. Kheirinejad, N. Riaihi, and R. Azmi, "Persian text-based traffic sign detection with convolutional neural network: A new dataset," in Proc. 10th Int. Conf. on Computer and Knowledge Engineering, pp. 060-064, Mashhad, Iran, 29-30 Oct. 2020.
[19] A. Fateh, M. Rezvani, A. Tajary, and M. Fateh, "Persian printed text line detection based on font size," Multimed. Tools Appl., vol. 82, no. 2, pp. 2393-2418, Jan. 2023.
[20] M. Rahmati, M. Fateh, M. Rezvani, A. Tajary, and V. Abolghasemi, "Printed Persian OCR system using deep learning," IET Image Process., vol. 14, no. 15, pp. 3920-3931, Dec. 2020.
[21] B. Epshtein, E. Ofek, and Y. Wexler, "Detecting text in natural scenes with stroke width transform," in Proc. IEEE Conf. on Comp. Vision and Pattern Recognition, pp. 2963-2970, San Francisco, CA, USA, 13-18 Jun. 2010.
[22] H. Chen, et al., "Robust text detection in natural images with edge-enhanced maximally stable extremal regions," in Proc. IEEE Int. Conf. on Image Processing, pp. 2609-2612, Barcelona, Spain,6-13 Nov. 2011.
[23] S. Ren, K. He, R. Girshick, and J. Sun, “Faster R-CNN: Towards real-time object detection with region proposal networks," in Proc. Adv. in Neural Info. Process. Sys., pp. 91-99, Montreal, Canada, 7-12 Dec.2015.
[24] M. Yaseen, What is YOLOv8: An In-Depth Exploration of the Internal Features of the Next-Generation Object Detector, arXiv preprint arXiv:2408.15857, 2024.
[25] W. Liu, et al., "SSD: Single shot multibox detector," in Eur. Conf. on Comp. Vision, Springer, pp. 21-37, 2016.
[26] J. Long, E. Shelhamer, and T. Darrell, "Fully convolutional networks for semantic segmentation," in Proc. IEEE Conf. on Comp. Vision and Pattern Recognition, 2015, pp. 3431–3440.
[27] K. He, G. Gkioxari, P. Dollár, and R. Girshick, "Mask R-CNN," in Proc. IEEE Int. Conf. on Comp. Vision, pp. 2961-2969, 2017.
[28] N. Carion, et al., End-to-End Object Detection with Transformers, arXiv Preprint arXiv200512872, 2020.
[29] X. Zhu, W. Su, L. Lu, B. Li, X. Wang, and J. Dai, Deformable DETR: Deformable Transformers for End-to-End Object Detection, arXiv Preprint arXiv201004159, 2020.
[30] Z. Tian, W. Huang, T. He, P. He, and Y. Qiao, "Detecting text in natural image with connectionist text proposal network," in Proc. Eur. Conf. on Comp. Vision, Springer, pp. 56-72, 2016.
[31] S. Long, et al., "Textsnake: A flexible representation for detecting text of arbitrary shapes," in Proc. Eur. Conference. on Computer Vision, pp. 20-36, 2018.
[32] D. Deng, H. Liu, X. Li, and D. Cai, "Pixellink: Detecting scene text via instance segmentation," in Proc. AAAI Conf. on Artif. Intell., 2018.
[33] O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation," in Proc. Int. Conf. on Medical Image Computing and Computer-Assisted Intervention, Springer, pp. 234-241, 2015.
[34] L. Yuan, et al., Florence: A New Foundation Model for Computer Vision, arXiv preprint arXiv:2111.11432, 2021.
[35] B. Xiao, et al., Florence-2: Advancing a Unified Representation for a Variety of Vision Tasks, arXiv preprint arXiv:2311.06242, 2023.
[36] S. Bai, et al., Qwen2.5-VL Technical Report, arXiv preprint arXiv:2502.13923, 2025.
[37] J. Bai, et al., Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond, arXiv preprint arXiv:2308.12966, 2023.
[38] A. Hurst, et al., GPT-4o System Card, arXiv preprint arXiv:2410.21276, 2024.
[39] K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition," in Proc. Int. Conf. on Learning Representations, 2015.
[40] O. Ronneberger, P. Fischer, and T. Brox, "U-Net: Convolutional networks for biomedical image segmentation," in Proc. Medical Image Computing and Computer-Assisted Intervention, pp. 234-241, 2015.
[41] J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, You Only Look Once: Unified, Real-Time Object Detection, arXiv PreprintarXiv1506.02640, 2016.
[42] D. Karatzas et al., "ICDAR 2013 robust reading competition," in Proc. Int. Conf. on Document Anal. and Recognition, pp. 1484-1493, 2013.
[43] D. Karatzas et al., "ICDAR 2015 competition on robust reading," in Proc. Int. Conf. on Document Anal. and Recognition, pp. 1156-1160, 2015.
[44] A. Veit, T. Matera, L. Neumann, J. Matas, and S. Belongie, Coco-Text: Dataset and Benchmark for Text Detection and Recognition in Natural Images, arXiv preprint. arXiv160107140, 2016.
[45] A. Singh, et al., "TextOCR: Towards large-scale end-to-end reasoning for arbitrary-shaped scene text," in Proc. IEEE/CVF Conf. on Computer Vision and Pattern Recognition, pp. 8802-8812, 2021.
[46] C. K. Ch’ng and C. S. Chan, "Total-text: A comprehensive dataset for scene text detection and recognition," in Proc. IAPR Int. Conf. on Document Anal. and Recognition, pp. 935-942, Kyoto, Japan, 9-11 Nov. 2017.
[47] L. Yuliang, J. Lianwen, Z. Shuaitao, and Z. Sheng, Detecting Curve Text in the Wild: New Dataset and New Solution, arXiv preprint arXiv:1712.02170, 2017.
[48] S. Long, S. Qin, Y. Fujii, A. Bissacco, and M. Raptis, "Hierarchical text spotter for joint text spotting and layout analysis," in Pro. of the IEEE/CVF Winter Conf. on Applications of Computer Vision, pp. 903-913, 2024.
[49] T. Kil, S. Kim, S. Seo, Y. Kim, and D. Kim, "Towards unified scene text spotting based on sequence generation," in Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition, pp. 15223–15232, 2023.
[50] A. Gupta, A. Vedaldi, and A. Zisserman, "Synthetic Data for Text Localisation in Natural Images," in Proc. IEEE Conf. on Comp. Vision and Pattern Recognition, 2016.
[51] M. Jaderberg, K. Simonyan, A. Vedaldi, and A. Zisserman, Synthetic Data and Artificial Neural Networks for Natural Scene Text Recognition, arXiv PreprINT. ArXiv14062227, 2014.
[52] M. Iwamura, et al., "ICDAR2017 robust reading challenge on omnidirectional video,” in Proc. IAPR Int. Conf. on Document Anal. and Recognition, 2017, pp. 1448–1453.
[53] C. -K. Chng, et al., ICDAR2019 Robust Reading Challenge on Arbitrary-Shaped Text (RRC-ArT), arXiv preprint arXiv1909.07145, 2019.
[55] W. Wu, et al., "ICDAR 2023 competition on video text reading for dense and small text," in Proc. Int. Conf. on Document Analysis and Recognition, pp. 405-419, 2023.
[56] Z. Wan, J. Zhang, L. Zhang, J. Luo, and C. Yao, "On vocabulary reliance in scene text recognition," in Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition, pp. 11425-11434, 2020.
[57] R. Zhang et al., "ICDAR 2019 robust reading challenge on reading Chinese text on signboard," in Proc. International Conf. on Document Analysis and Recognition, pp. 1577–1581, 2019.
[58] M. Tounsi, I. Moalla, A. M. Alimi, and F. Lebouregois, "Arabic characters' recognition in natural scenes using sparse coding for feature representations," in Proc. 13th Int. Conference on Document Analysis and Recognition, pp. 1036-1040, 2015.
[59] M. Tounsi, I. Moalla, and A. M. Alimi, "ARASTI: A database for Arabic scene text recognition," in Proc. 1st Int. Workshop on Arabic Script Analysis and Recognition, pp. 140-144, 2017. Asian Conf. on Pattern Recognition, pp. 747-752, 2017.
[61] S. Sabour, N. Frosst, and G. E. Hinton, "Dynamic routing between capsules," in Proc. Advances in Neural Information Processing [60] M. Jain, M. Mathew, and C. Jawahar, "Unconstrained OCR for Urdu using deep CNN-RNN hybrid networks," in Proc. 4th IAPRSystems, pp. 3856-3866, 2017.
[62] A. Rahman, A. Ghosh, and C. Arora, "UTRNet: High-Resolution Urdu Text Recognition in Printed Documents," in Proc. Int. Conf. on Document Analysis and Recognition, pp. 305-324, 2023.
[63] E. Shabaninia, F. Eslami, A. Afkari-Fahandari, and H. Nezamabadi-pour, "SUT: a new multi-purpose synthetic dataset for Farsi document image analysis," in Proc. 13th Int. Conf. on Computer and Knowledge Engineering, pp. 253-258, 2023.
[64] F. Asadi-Zeydabadi, E. Shabaninia, H. Nezamabadi-Pour, and M. Shojaee, "Farsi optical character recognition using a transformer-based model," in Proc. 13th Int. Conf. on Computer and Knowledge Engineering, pp. 293-299, 2023.
[65] M. Mosannafat, F. Taherinezhad, H. Khotanlou, and E. Alighardash, "Farsi text detection and localization in videos and images," in Proc. 9th Iranian Joint Congress on Fuzzy and Intelligent Systems, 6 pp., Bam, Iran, 2-4 Mar. 2022.
[66] A. Salmasi and E. Kabir,"Farsi text in scene: A new dataset," in Proc. 13th Int. Conf. on Computer and Knowledge Engineering, pp. 510-514, Mashhad, Iran, Nov. 2023.
[67] A. Dutta and A. Zisserman, "The VIA annotation software for images, audio and video," in Proc. of the 27th ACM Int. Conf. on Multimedia, New York, NY, USA. doi: 10.1145/3343031.3350535.
[68] T.-Y. Lin et al., "Microsoft COCO: Common objects in context" in Proc. Euro. Conf. on Comp.Vision, pp. 740-755, Milan, Itay, 29 Sept.-4 Oct. 2014.
[69] A. Ramesh, P. Dhariwal, A. Nichol, C. Chu, and M. Chen, Hierarchical Text-Conditional Image Generation with Clip Latents, arXiv Preprint, arXiv220406125, 2022.
[70] J. Achiam et al., GPT-4 Technical Report, arXiv preprint arXiv230308774, 2023.
[71] D. Guo,et al., DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning, arXiv Preprint arXiv250112948, 2025.
[72] G. Team, et al., Gemini: A Family of Highly Capable Multimodal Models, arXiv Preprint, arXiv231211805, 2023.
[73] A. Kirillov et al., "Segment anything," in Proc. of the IEEE/CVF Int. Conf. on Computer Vision, pp. 4015-4026, 2023.
[74] A. Kortylewski, Q. Liu, H. Wang, Z. Zhang, and A. Yuille, "Combining compositional models and deep networks for robust object classification under occlusion," in Proc. IEEE Winter Conf. on Applications of Computer Vision, pp. 1333-1341, 2020.
[75] Z. Raisi and J. Zelek, "Occluded text detection and recognition in the wild," in Proc. 19th Conf. on Robots and Vision, pp. 140-150, Toronto, Canada, May 2022.