Real-Time Vision-Based Hand Gesture Recognition System for Multi-Functional Contactless Control of Digital Applications

Authors

  • Sanjivani Adsul Department of Artificial Intelligence & Data Science, Vishwakarma Institute of Technology, Pune, India Author
  • Sachi Khobragade Department of Artificial Intelligence & Data Science, Vishwakarma Institute of Technology, Pune, India Author
  • Nakul Kohad Department of Artificial Intelligence & Data Science, Vishwakarma Institute of Technology, Pune, India Author
  • Parth Khot Department of Artificial Intelligence & Data Science, Vishwakarma Institute of Technology, Pune, India Author
  • Shashank Kumbhar Department of Artificial Intelligence & Data Science, Vishwakarma Institute of Technology, Pune, India Author

DOI:

https://doi.org/10.21467/proceedings.7.6.16

Keywords:

Gesture Control, Computer Vision, Hand Gesture Recognition

Abstract

Hand gesture recognition emerged as a significant advancement in enabling intuitive and contactless human-computer interaction. This study presented the design and implementation of a real-time, multi-functional hand gesture control system capable of managing digital tasks such as volume adjustment, media playback, scrolling, tab switching, slide navigation, and AI-powered mathematical problem solving. The primary objective was to develop a seamless interaction system that operated without the need for physical input devices, thereby enhancing accessibility and user convenience across various use cases. The system utilized computer vision and machine learning techniques, incorporating the MediaPipe framework for 3D hand landmark detection and OpenCV for real-time video processing. Spatial features such as distances between landmarks, angles at finger joints, and motion vectors were extracted and used to classify gestures through trained machine learning models. The solution was implemented with hardware independence in mind, requiring only a standard webcam, and featured a user-friendly graphical interface built with PyQt5. An AI calculator module was also integrated using backend processing for gesture-based mathematical interpretation. The system was tested under diverse conditions to evaluate accuracy, responsiveness, and robustness. Experimental results showed a gesture recognition accuracy of approximately 95%, with low latency and consistent real-time performance. Functional demonstrations confirmed the effectiveness of the system in controlling multiple applications, highlighting its adaptability to different environments and use scenarios. In conclusion, the developed system offered a reliable, accessible, and scalable alternative to conventional input methods. It proved suitable for smart environments, assistive technologies, and hands-free digital control. Future developments may explore extended gesture sets, real-time customization, and integration with edge devices or multimodal systems for enhanced interaction capabilities.

References

[1] S. Makant, “Small Deep Learning Models for Hand Gesture Recognition,” in *Proc. IEEE Int. Conf. on Consumer Electronics (ICCE)*, 2020.

[2] A. Roy, R. Sharma, and P. Sen, “A Survey on Hand Gesture Recognition Using Machine Learning Techniques,” *IEEE Trans. Ind. Electron.*, vol. 68, no. 4, pp. 3203–3211, 2021.

[3] P. Gupta and K. Singh, “Vision-Based Gesture Recognition Using Transfer Learning,” in *Proc. IEEE Int. Conf. on Computer Vision Workshops (ICCVW)*, 2022.

[4] S. Zhao, L. Zhang, and X. Wu, “Dynamic Hand Gesture Recognition for Human-Machine Interfaces,” in *Proc. IEEE Int. Conf. on Humanoid Robotics (ICHR)*, 2021.

[5] J. Brown and M. Kumar, “3D Convolutional Neural Networks for Sign Language Recognition,” in *Proc. IEEE/CVF Conf. on Computer Vision and Pattern Recognition Workshops (CVPRW)*, 2020.

[6] A. Chawla and V. Gupta, “Gesture Recognition in Smart Homes: Challenges and Opportunities,” in *Proc. IEEE Int. Congress on Internet of Things (ICIOT)*, 2019.

[7] B. Patel, “Multimodal Hand Gesture Recognition Using Sensor Fusion,” *IEEE Sensors J.*, vol. 21, no. 12, pp. 13478–13485, 2021.

[8] R. Li and J. Chen, “Self-Supervised Learning for Hand Gesture Recognition,” in *Proc. IEEE Conf. on Computer Vision and Pattern Recognition Workshops (CVPRW)*, 2022.

[9] S. Kim and D. Park, “Gesture Recognition Using Transformers,” *IEEE Trans. Autom. Control*, vol. 68, no. 9, pp. 5124–5131, 2023.

[10] Y. Lee, J. H. Lee, and M. Y. Kim, “Real-Time Gesture Detection in Augmented Reality Applications,” in *Proc. IEEE Virtual Reality Conf. (VR)*, 2022.

[11] P. Chowdhury and K. Tiwari, “Gesture Recognition Using Embedded Neural Networks for IoT Devices,” in *Proc. IEEE Int. Conf. on Information and Communication Systems (ICICS)*, 2021.

[12] V. Rajan and N. Dutta, “Wearable Gesture Recognition Systems: Advances and Challenges,” *IEEE Trans. Biomed. Eng.*, vol. 70, no. 5, pp. 1234–1243, 2023.

[13] J. Zhang and R. Liu, “Privacy-Preserving Hand Gesture Recognition Using Federated Learning,” in *Proc. IEEE Int. Conf. on Pattern Recognition Workshops (ICPRW)*, 2022.

Downloads

Published

2025-11-21

Issue

Vol. 7 No. 6 (2025): Proceedings of the 3rd International Conference on Artificial Intelligence, Machine Learning and Cybersecurity

Section

Articles

License

Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

How to Cite

[1]
S. Adsul, S. Khobragade, N. Kohad, P. Khot, and S. Kumbhar, “Real-Time Vision-Based Hand Gesture Recognition System for Multi-Functional Contactless Control of Digital Applications”, AIJR Proc., vol. 7, no. 6, pp. 124–134, Nov. 2025, doi: 10.21467/proceedings.7.6.16.