It tracks your hand in real time and mimics your movements with two servo motors. A Raspberry Pi streams the video to your PC where MediaPipe calculates your hand position and the coordinates are sent right back to the Rasperry PI where it moves the two servos.
The Raspberry Pi captures the video and controls the two servo motors. A separate PC runs MediaPiPe hand detection.
rpicam-vid captures video on the Pi.
→Frames are sent over HTTP to the server on your PC.
→HandLandmarker detects the hand position and calculates the coordinates
→The Pi receives the coordinates and drives two servos via GPIO.
The physical components that make up the tracking system.
Captures video via the camera module and controls the servo motors through GPIO. The Pi handles all the hardware interactions.
Two servo motors are mounted on a cardboard bracket so nothings tips over when the motors move.
The camera is connected to the Rasperry pi and captures the video
Running MediaPipe hand detection on a separate PC instead of the Raspberry Pi is a important choice in my case.
MediaPipe hand detection is really intensive and generates much heat so running it on the Rasperry PI was not an option in my Case so I had to move MediaPiPe to my much more powerful PC to handle it.
My PC has a more powerful CPU, resulting in faster MediaPiPe perfomance. This keeps the frame rate high and steady
The project is split into three repositories.
The Repo where the Raspberry Pi captures the video and controls the two servo motors.
github.com/javinwittig/hand_tracker →The server where the frame are sent to, in my case my PC. In this Repo is all the MediaPiPe code.
github.com/javinwittig/hantracking_server →This Repo is the project Website, which is hosted on GitHub Pages
github.com/javinwittig/hand_tracker_website →