Robotická paže ovládaná lidskou rukou pomocí zpracování obrazu
Loading...
Date
2024-06-11
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
This research presents an interdisciplinary methodology for the design, implementation and
development of the robotic gesture control system integrating with computer vision,
mechanical principles, and electronics circuitry. The interdisciplinary approach of the project
involves the collaboration of mechanical, computer and electrical electronics domains.
The circuit design architecture has generic components such as ESP32 WROOM32 Module
and MG996R Servo Motor. This system architecture has been developed to obtain the precise
angular control of servo motors with the help of PWM signals generated by the ESP32 module.
The interfacing and configuration step of the ESP32 WROOM Module involves firmware
development using Arduino IDE and leading communication with peripheral devices via
UART protocol. Serial communication has been established between C++ running on ESP32
and Python script on Laptop.
Computer Vision Algorithm allows us to detect and track the objects desired objects in footage.
The Libraries used for gesture recognition by using MediaPipe & OpenCV Libraries. Over the
pre-processed footage with the help of libraries, I have implemented a custom post-processing
landmark detection technique for angular calculation via vector approach.
The mechanical structure design and fabrication involves Modelling and 3D printing of the
parts. After 3D printing involves the assembly of parts and calibrating with servo angular
motion for operating precisely.
The demonstrated methodology connects engineering principles with robotic control strategy
to obtain a robust control algorithm using computer vision to operate finger movement
precisely.
This project can be the promising for the applications such as prosthetics Hands and HMI
applications on unmanned fields.
This research presents an interdisciplinary methodology for the design, implementation and development of the robotic gesture control system integrating with computer vision, mechanical principles, and electronics circuitry. The interdisciplinary approach of the project involves the collaboration of mechanical, computer and electrical electronics domains. The circuit design architecture has generic components such as ESP32 WROOM32 Module and MG996R Servo Motor. This system architecture has been developed to obtain the precise angular control of servo motors with the help of PWM signals generated by the ESP32 module. The interfacing and configuration step of the ESP32 WROOM Module involves firmware development using Arduino IDE and leading communication with peripheral devices via UART protocol. Serial communication has been established between C++ running on ESP32 and Python script on Laptop. Computer Vision Algorithm allows us to detect and track the objects desired objects in footage. The Libraries used for gesture recognition by using MediaPipe & OpenCV Libraries. Over the pre-processed footage with the help of libraries, I have implemented a custom post-processing landmark detection technique for angular calculation via vector approach. The mechanical structure design and fabrication involves Modelling and 3D printing of the parts. After 3D printing involves the assembly of parts and calibrating with servo angular motion for operating precisely. The demonstrated methodology connects engineering principles with robotic control strategy to obtain a robust control algorithm using computer vision to operate finger movement precisely. This project can be the promising for the applications such as prosthetics Hands and HMI applications on unmanned fields.
This research presents an interdisciplinary methodology for the design, implementation and development of the robotic gesture control system integrating with computer vision, mechanical principles, and electronics circuitry. The interdisciplinary approach of the project involves the collaboration of mechanical, computer and electrical electronics domains. The circuit design architecture has generic components such as ESP32 WROOM32 Module and MG996R Servo Motor. This system architecture has been developed to obtain the precise angular control of servo motors with the help of PWM signals generated by the ESP32 module. The interfacing and configuration step of the ESP32 WROOM Module involves firmware development using Arduino IDE and leading communication with peripheral devices via UART protocol. Serial communication has been established between C++ running on ESP32 and Python script on Laptop. Computer Vision Algorithm allows us to detect and track the objects desired objects in footage. The Libraries used for gesture recognition by using MediaPipe & OpenCV Libraries. Over the pre-processed footage with the help of libraries, I have implemented a custom post-processing landmark detection technique for angular calculation via vector approach. The mechanical structure design and fabrication involves Modelling and 3D printing of the parts. After 3D printing involves the assembly of parts and calibrating with servo angular motion for operating precisely. The demonstrated methodology connects engineering principles with robotic control strategy to obtain a robust control algorithm using computer vision to operate finger movement precisely. This project can be the promising for the applications such as prosthetics Hands and HMI applications on unmanned fields.
Description
Subject(s)
Computer Vision, ESP32 Wroom, 3D Printing, Hand Gesture Recognition, Bionic Hand
Motion and Motion Control, HMI