Arabic sign language recognition using independent sensor-based machine learning and image-based deep learning approaches
Sign language (SL) facilitates communication between the hearing community and those who are deaf or hard of hearing. Yet communication challenges persist in healthcare and education. SL recognition (SLR) systems use machine learning and deep learning on sensor-based and image-based inputs to interpret sign gestures. This work applies artificial intelligence techniques to Arabic Sign Language recognition (ArSLR). Arabic SL is used by Arabic-speaking individuals who are deaf or hard of hearing. Accurate ArSLR tools are crucial, especially for those who do not understand Arabic SL. While SLR has advanced for several languages, ArSLR remains relatively underdeveloped. This paper presents two methodologies for recognizing isolated Arabic SL signs: one using sensor data with classical machine learning, the other using image data with deep learning. The sensor-based approach uses wearable gloves with inertial and flex sensors to capture finger and hand motion. After preprocessing, several algorithms—support vector machine, random forest, k-nearest neighbors, and logistic regression—were evaluated. The image-based approach uses a convolutional neural network (CNN) built on a pre-trained MobileNetV2 base, trained with data augmentation to mitigate signer variability and environmental noise. Results show the sensor-based system achieves high accuracy, particularly for complex finger articulations, while the CNN excels in recognizing visually distinct signs. Both methods show potential for scalable Arabic SL recognition. The two systems were evaluated on independently collected datasets with different participant groups and sign sets; therefore, direct accuracy comparisons between modalities are not intended. This work compares two Arabic SL approaches and discusses their suitability for different application scenarios.
Aftab, M., Khan, A., Rahman, A., Ali, F., Youldash, M., Ahmed, M. S., Almutairi, F., Alawad, A., Safhi, M., Al-Soqair, A.-S., Alshammary, A., Alqahtani, Y., Farooqui, M., & Bakry, A. (2025). Harnessing deep learning to Arabic sign language recognition. International Journal of Intelligent Engineering and Systems, 18(8), 743–759. https://doi.org/10.22266/ijies2025.0930.45
Alani, A. A., & Cosma, G. (2021). ArSL-CNN: A convolutional neural network for Arabic sign language gesture recognition. Indonesian Journal of Electrical Engineering and Computer Science, 22(2), 1096–1107. https://doi.org/10.11591/IJEECS.V22.I2.PP1096-1107
Alasmari, N., & Asiri, S. (2025). ASLDetect: Arabic sign language detection using ResNet and U-Net like component. Scientific Reports, 15(1). https://doi.org/10.1038/s41598-025-01588-w
Algethami, N., Farhud, R., Alghamdi, M., Almutairi, H., Sorani, M., & Aleisa, N. (2025). Continuous Arabic sign language recognition models. Sensors, 25(9), 2916. https://doi.org/10.3390/s25092916
Ashfaq, U., Wang, Q., Merabet, B., & Zhang, J. (2026). SignViT: An enhanced vision transformer framework for attention-based sign language hand gesture recognition. Biomedical Signal Processing and Control, 112, 108602. https://doi.org/10.1016/J.BSPC.2025.108602
ASLAD-190K. (n.d.). Arabic sign language alphabet dataset. Kaggle. Accessed September 28, 2026. https://www.kaggle.com/datasets/salmaselmoghazy/aslad-190k-arabic-sign-language-alphabet-dataset
Balaha, M. M., El-Kady, S., Balaha, H. M., Salama, M., Emad, E., Hassan, M., & Saafan, M. M. (2023). A vision-based deep learning approach for independent-users Arabic sign language interpretation. Multimedia Tools and Applications, 82(5), 6807–6826. https://doi.org/10.1007/s11042-022-13423-9
Boulesnane, A., Bellil, L., & Ghiri, M. G. (2024). A hybrid CNN-random forest model with landmark angles for real-time Arabic sign language recognition. Neural Computing and Applications. https://doi.org/10.1007/s00521-024-10729-7
Breiman, L. (2001). Random forests. Machine Learning, 45(1), 5–32. https://doi.org/10.1023/A:1010933404324
Caraka, R. E., Supardi, K., Kurniawan, R., Kim, Y., Gio, P. U., Yuniarto, B., Mubarok, F. Z., & Pardamean, B. (2025). Empowering deaf communication: A novel LSTM model for recognizing Indonesian sign language. Universal Access in the Information Society, 24(1), 771–783. https://doi.org/10.1007/S10209-024-01095-1
Cortes, C., & Vapnik, V. (1995). Support-vector networks. Machine Learning, 20(3), 273–297. https://doi.org/10.1007/BF00994018
"Dawwie" in sign language. (n.d.). UNICEF Egypt. Accessed July 4, 2025. https://www.unicef.org/egypt/stories/dawwie-sign-language
Deafness and hearing loss. (n.d.). World Health Organization. Accessed July 4, 2025. https://www.who.int/health-topics/hearing-loss?utm_source=chatgpt.com#tab=tab_1
Delpreto, J., Hughes, J., D'Aria, M., De Fazio, M., & Rus, D. (2022). A wearable smart glove and its application of pose and gesture detection to sign language classification. IEEE Robotics and Automation Letters, 7(4), 10589–10596. https://doi.org/10.1109/LRA.2022.3191232
Filipowska, A., Filipowski, W., Raif, P., Pieniążek, M., Bodak, J., Ferst, P., Pilarski, K., Sieciński, S., Doniec, R. J., Mieszczanin, J., Skwarek, E., Bryzik, K., Henkel, M., & Grzegorzek, M. (2024). Machine learning-based gesture recognition glove: Design and implementation. Sensors, 24(18), 6157. https://doi.org/10.3390/s24186157
GitHub-NadeenElh. (n.d.). GP-Codes [Computer software]. GitHub. Accessed July 4, 2025. https://github.com/NadeenElh/GP-Codes/tree/main
Guan, M., Wang, Y., Ma, G., Liu, J., & Sun, M. (2025). MSKA: Multi-stream keypoint attention network for sign language recognition and translation. Pattern Recognition, 165, 111602. https://doi.org/10.1016/J.PATCOG.2025.111602
Haile, L. M., Kamenov, K., Briant, P. S., Orji, A. U., Steinmetz, J. D., Abdoli, A., Abdollahi, M., Abu-Gharbieh, E., Afshin, A., Ahmed, H., Rashid, T. A., Akalu, Y., Alahdab, F., Alanezi, F. M., Alanzi, T. M., Al Hamad, H., Ali, L., Alipour, V., Al-Raddadi, R. M., … Chadha, S. (2021). Hearing loss prevalence and years lived with disability, 1990–2019: Findings from the Global Burden of Disease Study 2019. The Lancet, 397(10278), 996–1009. https://doi.org/10.1016/S0140-6736(21)00516-X
Halabi, M., & Harkouss, Y. (2025). Real-time Arabic sign language recognition system using sensory glove and machine learning. Neural Computing and Applications, 37(9), 6977–6993. https://doi.org/10.1007/s00521-025-11010-1
Huang, J., & Ling, C. X. (2005). Using AUC and accuracy in evaluating learning algorithms. IEEE Transactions on Knowledge and Data Engineering, 17(3), 299–310. https://doi.org/10.1109/TKDE.2005.50
Hugar, G., & Kagalkar, R. M. (2025). Hybrid dual-stream deep learning approach for real-time Kannada sign language recognition in assistive healthcare. Journal of Information Systems Engineering and Business Intelligence, 11(3), 393–406. https://doi.org/10.20473/jisebi.11.3.393-406
Hugar, G., Kagalkar, R. M., & Das, A. (2025). Comparative study of hybrid deep learning models for Kannada sign language recognition. International Journal of Computational Intelligence Systems, 18(1), 191. https://doi.org/10.1007/s44196-025-00922-4
Kharoua, R. El., & Jiang, X. (2024). Deep learning recognition for Arabic alphabet sign language RGB dataset. Journal of Computer and Communications, 12(3), 32–51. https://doi.org/10.4236/JCC.2024.123003
Kim, H. J., & Baek, S. W. (2023). Application of wearable gloves for assisted learning of sign language using artificial neural networks. Processes, 11(4), 1065. https://doi.org/10.3390/pr11041065
Krstinić, D., Braović, M., Šerić, L., & Božić-Štulić, D. (2020). Multi-label classifier performance evaluation with confusion matrix. Computer Science & Information Technology, 1–14. https://doi.org/10.5121/csit.2020.100801
Lahiani, H., & Frikha, M. (2024). Exploring CNN-based transfer learning approaches for Arabic alphabets sign language recognition using the ArSL2018 dataset. International Journal of Intelligent Engineering Informatics, 12(2), 236–260. https://doi.org/10.1504/ijiei.2024.138858
Li, W., & Liu, Z. (2011). A method of SVM with normalization in intrusion detection. Procedia Environmental Sciences, 11(Part A), 256–262. https://doi.org/10.1016/J.PROENV.2011.12.040
Liang, Y., Jettanasen, C., & Chiradeja, P. (2024). Progression learning convolution neural model-based sign language recognition using wearable glove devices. Computation, 12(4), 72. https://doi.org/10.3390/computation12040072
Liu, L., Wang, L., Ma, Z., & Abdulhussain, S. H. (2024). Improved lightweight YOLOv5 based on ShuffleNet and its application on traffic signs detection. PLoS ONE, 19(9). https://doi.org/10.1371/journal.pone.0310269
Lu, C., Amino, S., & Jing, L. (2023). Data glove with bending sensor and inertial sensor based on weighted DTW fusion for sign language recognition. Electronics, 12(3), 613. https://doi.org/10.3390/electronics12030613
Maharjan, S., Shrestha, S., & Fernando, S. (2024). Sign language detection and translation using smart glove. In Lecture Notes in Networks and Systems (pp. 535–554). Springer Nature Singapore, Singapore. https://doi.org/10.1007/978-981-97-3591-4_41
NAD-National Association of the Deaf. (n.d.). Position paper on the CRPD. Accessed January 10, 2026. https://nad.org/knowledge-hub/position-statements/position-paper-on-the-crpd/
Nick, T. G., & Campbell, K. M. (2007). Logistic regression. Methods in Molecular Biology, 404, 273–301. https://doi.org/10.1007/978-1-59745-530-5_14
Piskozub, J., & Strumillo, P. (2025). A sensory glove with a limited number of sensors for recognition of the finger alphabet of Polish Sign Language. IEEE Access, 13, 28408–28418. https://doi.org/10.1109/ACCESS.2025.3540578
Pranav, & Katarya, R. (2025). Effi-CNN: Real-time vision-based system for interpretation of sign language using CNN and transfer learning. Multimedia Tools and Applications, 84(6), 3137–3159. https://doi.org/10.1007/S11042-024-20585-1
Saleh, A. A., & Joshi, H. (2025). Sensor-equipped gloves for recognizing Arabic sign language. International Journal of Applied Mathematics, 38(6s), 1148–1169. https://doi.org/10.12732/IJAM.V38I6S.458
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., & Chen, L. C. (2018). MobileNetV2: Inverted residuals and linear bottlenecks. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (pp. 4510–4520). IEEE. https://doi.org/10.1109/CVPR.2018.00474
Santana-Hernández, D. J., & Jeremy, J. (2023). Support for EHC matters, now more than ever. Community Ear & Hearing Health, 20, 24. Accessed July 4, 2025. https://www.lshtm.ac.uk/sites/default/files/2023-12/CEHH%20Issue%2024%20Ear%20and%20hearing%20care%20needs%20everyone%E2%80%99s%20support%21.pdf
Steinbach, M., & Tan, P. N. (2009). kNN: k-nearest neighbors. In The top ten algorithms in data mining (pp. 151–161). Chapman & Hall/CRC, Boca Raton, FL. https://doi.org/10.1201/9781420089653-15
Tao, T., Zhao, Y., Liu, T., & Zhu, J. (2024). Sign language recognition: A comprehensive review of traditional and deep learning approaches, datasets, and challenges. IEEE Access, 12, 75034–75060. https://doi.org/10.1109/ACCESS.2024.3398806
Tawfik, N., Emara, H. M., El-Shafai, W., Soliman, N. F., Algarni, A. D., & Abd El-Samie, F. E. (2024). Enhancing early detection of lung cancer through advanced image processing techniques and deep learning architectures for CT scans. Computers, Materials and Continua, 81(1), 271–307. https://doi.org/10.32604/cmc.2024.052404
UN marks its first International Day of Sign Languages. (n.d.). United Nations Enable. Accessed January 10, 2026. https://www.un.org/development/desa/disabilities/news/dspd/international-day-sign-languages.html?utm_source
Venkatesh, S., Kshirsagar, P. R., Thiagarajan, R., Tak, T. K., & Sivaneasan, B. (2026). Optimization model for sign language recognition using hybrid convolution networks. Signal Processing: Image Communication, 141, 117444. https://doi.org/10.1016/J.IMAGE.2025.117444