Developer Robin van Soelen has released gesture.live, a free browser-based instrument that you play with hand gestures through your webcam.
This is not a plugin, and there is nothing to install. It’s pretty cool, though. You open the website, allow camera access, keep both hands inside the webcam frame, and start controlling the instrument with your movements.
I tested gesture.live in Chrome, and it worked great here. There is a learning curve, though. You need to spend some time with the gesture animations on the homepage and learn what each hand position does before it starts feeling natural.
Once it clicks, it is a very cool experience. Seeing the lines and interface elements react to your hands on screen definitely makes you feel like some sort of musical android for a moment.
I wasn’t able to make anything particularly good-sounding, but it was definitely fun.
The basic setup uses the left hand for chords and the right hand for melody, bass, drums, or effects. The site works best in landscape because both hands need to share one wide 16:9 frame, and the control rail runs across the screen.
Robin says gesture.live was inspired by Indecisive Eric’s gesture synth, which used both hands to play chords through a finger-count chord language. gesture.live builds on that idea by compressing the chord logic to one hand, freeing the other hand for additional performance controls.
Under the hood, gesture.live is a Next.js application using Google’s MediaPipe for gesture recognition and Tone.js for sound generation. The recent v1.2 update added a new Sculpt mode for shaping the chord sound, changed the way drum and bass patterns are selected, and simplified the layout so the performer is less hidden behind the interface.
There is no full manual yet because the website is still changing, but the homepage shows the main gestures and interactions through short animations. Robin says the next updates will focus on making the site easier to understand, easier to share, and more useful for creative ideas he has not thought of yet.
gesture.live is free to use in your browser.
More info: gesture.live (FREE)
Deal of the day 🔥: Get the VATRA mix finisher plugin by Flame Sound!More:
