VISION BASED GESTURE CONTROL FOR LOW COST ASSISTIVE AUTOMATION IN HEALTHCARE SETTINGS
L.D.R.P. Liyanage1*, L.D.N.G. Liyanage2 and S. S. Morapitiya3
1,3Institute of Technology, University of Moratuwa, Homagama, Sri Lanka, 2Department of Computer Science, University of Sri Jayawardenepura, Sri Lanka
Session: Technical Session C
Abstract
This project introduces a budget-friendly system enabling individuals with mobility challenges to control medical devices such as wheelchairs and lights using simple hand gestures. This system which is specially designed for healthcare settings in Sri Lanka uses an ESP32-CAM for gestures recognition and an ESP8266 for device operation. As an affordable alternative to expensive assistive tools like Microsoft Kinect, which can cost over Rs. 60,000, this system demonstrates practicality with 96–98% accuracy in gesture recognition and a response time of less than one second, all while consuming only 1.8 A of power. In tests conducted across 5 homes with 20 participants (5 individuals with mobility challenges and 15 without), the system recognized gestures with 96–98% accuracy. Data processing is performed locally on the device, preserving user privacy, and addressing concerns regarding the use of cameras in healthcare settings. At roughly Rs 5000, the system is affordable for many families and can be expanded to support additional devices. Although challenges such as low lighting and simultaneous control of multiple devices remain, promising results have been demonstrated in boosting independence in settings where advanced technology is not accessible.
Keywords: ESP32-CAM, ESP8266, hand gesture control, healthcare automation, patient privacy
DOI: 10.64752/HTWA9709