Data Mesh-Driven Enhancements in Sign Language Recognition Using LRCN and 3DCNN Models
DOI:
https://doi.org/10.21467/proceedings.7.6.1Keywords:
3DCNN, LRCN, Machine LearningAbstract
This study aims to narrow the communication gap between sign language users and non-users by developing a sign language recognition application. Employing machine learning techniques, particularly the LRCN and 3DCNN models, the study modifies and enhances its baselines to meet its objectives. Techniques such as the sliding window method, dropout layers, and L2 regularization are employed to expand the dataset and reduce overfitting, resulting in significant improvements in model accuracy. Utilizing the WLASL dataset, renowned for its extensive vocabulary and diverse signers, the study focuses on 10 selected classes for implementation. Comparative analysis based on the F1-score is conducted between LRCN and 3DCNN models, with both achieving results above 80% in each class and 95% and 94% overall, respectively. Resource usage monitoring indicates that LRCN requires more memory due to higher epochs needed for accuracy.
References
[1] G. S. Kashyap, K. Malik, S. Wazir, and R. Khan, “Using Machine Learning to Quantify the Multimedia Risk Due to Fuzzing,” Multimed. Tools Appl., vol. 81, no. 25, pp. 36685–36698, Oct. 2022, doi: 10.1007/s11042-021-11558-9.
[2] S. Wazir, G. S. Kashyap, and P. Saxena, “MLOps: A Review,” Aug. 2023, Accessed: Sep. 16, 2023. [Online]. Available: https://arxiv.org/abs/2308.10908v1
[3] G. S. Kashyap et al., “Revolutionizing Agriculture: A Comprehensive Review of Artificial Intelligence Techniques in Farming,” Feb. 2024, doi: 10.21203/RS.3.RS-3984385/V1.
[4] S. Naz and G. S. Kashyap, “Enhancing the predictive capability of a mathematical model for pseudomonas aeruginosa through artificial neural networks,” Int. J. Inf. Technol. 2024, pp. 1–10, Feb. 2024, doi: 10.1007/S41870-023-01721-W.
[5] G. S. Kashyap et al., “Detection of a facemask in real-time using deep learning methods: Prevention of Covid 19,” Jan. 2024, Accessed: Feb. 04, 2024. [Online]. Available: https://arxiv.org/abs/2401.15675v1
[6] S. Wazir, G. S. Kashyap, K. Malik, and A. E. I. Brownlee, “Predicting the Infection Level of COVID-19 Virus Using Normal Distribution-Based Approximation Model and PSO,” Springer, Cham, 2023, pp. 75–91. doi: 10.1007/978-3-031-33183-1_5.
[7] J. Carletta et al., “The AMI Meeting Corpus: A pre-announcement,” in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Springer, Berlin, Heidelberg, 2006, pp. 28–39. doi: 10.1007/11677482_3.
[8] F. Abu-Amara, A. Bensefia, H. Mohammad, and H. Tamimi, “Robot and virtual reality-based intervention in autism: a comprehensive review,” Int. J. Inf. Technol., vol. 13, no. 5, pp. 1879–1891, Oct. 2021, doi: 10.1007/s41870-021-00740-9.
[9] T. Baltrusaitis, A. Zadeh, Y. C. Lim, and L. P. Morency, “OpenFace 2.0: Facial behavior analysis toolkit,” in Proceedings - 13th IEEE International Conference on Automatic Face and Gesture Recognition, FG 2018, Institute of Electrical and Electronics Engineers Inc., Jun. 2018, pp. 59–66. doi: 10.1109/FG.2018.00019.
[10] A. Anzulewicz, K. Sobota, and J. T. Delafield-Butt, “Toward the Autism Motor Signature: Gesture patterns during smart tablet gameplay identify children with autism,” Sci. Rep., vol. 6, Aug. 2016, doi: 10.1038/srep31107.
[11] A. Guarino, D. Malandrino, R. Zaccagnino, C. Capo, and N. Lettieri, “Touchscreen gestures as images. A transfer learning approach for soft biometric traits recognition,” Expert Syst. Appl., vol. 219, p. 119614, Jun. 2023, doi: 10.1016/j.eswa.2023.119614.
[12] M. A. Rilvan, J. Chao, and M. S. Hossain, “Capacitive Swipe Gesture Based Smartphone User Authentication and Identification,” in Proceedings - 2020 IEEE International Conference on Cognitive and Computational Aspects of Situation Management, CogSIMA 2020, Institute of Electrical and Electronics Engineers Inc., Aug. 2020, pp. 59–66. doi: 10.1109/CogSIMA49017.2020.9215998.
[13] A. R. Khan, T. Saba, M. Z. Khan, S. M. Fati, and M. U. G. Khan, “Classification of human’s activities from gesture recognition in live videos using deep learning,” Concurr. Comput. Pract. Exp., vol. 34, no. 10, p. e6825, May 2022, doi: 10.1002/cpe.6825.
[14] W. Guo, X. Liu, C. Lu, and L. Jing, “PIFall: A Pressure Insole-Based Fall Detection System for the Elderly Using ResNet3D,” Electron. 2024, Vol. 13, Page 1066, vol. 13, no. 6, p. 1066, Mar. 2024, doi: 10.3390/ELECTRONICS13061066.
[15] D. Lee, A. Franchi, P. R. Giordano, H. Il Son, and H. H. Bülthoff, “Haptic teleoperation of multiple unmanned aerial vehicles over the internet,” in Proceedings - IEEE International Conference on Robotics and Automation, 2011, pp. 1341–1347. doi: 10.1109/ICRA.2011.5979993.
[16] T. L. Nguyen, S. K. Ro, and J. K. Park, “Study of ball screw system preload monitoring during operation based on the motor current and screw-nut vibration,” Mech. Syst. Signal Process., vol. 131, pp. 18–32, Sep. 2019, doi: 10.1016/j.ymssp.2019.05.036.
[17] S. D. Bansod and A. V. Nandedkar, “Crowd anomaly detection and localization using histogram of magnitude and momentum,” Vis. Comput., vol. 36, no. 3, pp. 609–620, Mar. 2020, doi: 10.1007/S00371-019-01647-0/FIGURES/9.
[18] D. Nishiyama, S. Arita, D. Fukui, M. Yamanaka, and H. Yamada, “Accurate fall risk classification in elderly using one gait cycle data and machine learning,” Clin. Biomech., vol. 115, p. 106262, May 2024, doi: 10.1016/j.clinbiomech.2024.106262.
[19] S. LaConte, S. Strother, V. Cherkassky, J. Anderson, and X. Hu, “Support vector machines for temporal classification of block design fMRI data,” Neuroimage, vol. 26, no. 2, pp. 317–329, Jun. 2005, doi: 10.1016/j.neuroimage.2005.01.048.
[20] R. L. Hughes, “A continuum theory for the flow of pedestrians,” Transp. Res. Part B Methodol., vol. 36, no. 6, pp. 507–535, Jul. 2002, doi: 10.1016/S0191-2615(01)00015-7.
[21] G. S. Kashyap, A. Siddiqui, R. Siddiqui, K. Malik, S. Wazir, and A. E. I. Brownlee, “Prediction of Suicidal Risk Using Machine Learning Models,” Dec. 25, 2021. Accessed: Feb. 04, 2024. [Online]. Available: https://papers.ssrn.com/abstract=4709789
[22] H. Habib, G. S. Kashyap, N. Tabassum, and T. Nafis, “Stock Price Prediction Using Artificial Intelligence Based on LSTM– Deep Learning Model,” in Artificial Intelligence & Blockchain in Cyber Physical Systems: Technologies & Applications, CRC Press, 2023, pp. 93–99. doi: 10.1201/9781003190301-6.
[23] N. Marwah, V. K. Singh, G. S. Kashyap, and S. Wazir, “An analysis of the robustness of UAV agriculture field coverage using multi-agent reinforcement learning,” Int. J. Inf. Technol., vol. 15, no. 4, pp. 2317–2327, May 2023, doi: 10.1007/s41870-023-01264-0.
[24] P. Kaur, G. S. Kashyap, A. Kumar, M. T. Nafis, S. Kumar, and V. Shokeen, “From Text to Transformation: A Comprehensive Review of Large Language Models’ Versatility,” Feb. 2024, Accessed: Mar. 21, 2024. [Online]. Available: https://arxiv.org/abs/2402.16142v1
[25] F. Alharbi and G. S. Kashyap, “Empowering Network Security through Advanced Analysis of Malware Samples: Leveraging System Metrics and Network Log Data for Informed Decision-Making,” Int. J. Networked Distrib. Comput., pp. 1–15, Jun. 2024, doi: 10.1007/s44227-024-00032-1.
[26] M. Kanojia, P. Kamani, G. S. Kashyap, S. Naz, S. Wazir, and A. Chauhan, “Alternative Agriculture Land-Use Transformation Pathways by Partial-Equilibrium Agricultural Sector Model: A Mathematical Approach,” Aug. 2023, Accessed: Sep. 16, 2023. [Online]. Available: https://arxiv.org/abs/2308.11632v1
Downloads
Published
Issue
Section
License
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
