HandVu: Vision-based Hand Gesture Recognition and User Interface

http://www.movesinstitute.org/~kolsch/HandVu/HandVu.html

HandVu: Vision-based Hand Gesture Recognition and User Interface

With almost any color camera and sufficient processing power, this software collection implements a hand gesture interface. HandVu detects the hand in a standard posture, then tracks it and recognizes key postures - all in real-time and without the need for camera or user calibration. The output is accessible through a client-server infrastructure in a custom format and as OSC packets. Read more about HandVu or even download the software below

News

• Jan 17, 2011: An Apple Mac OS X package (installer) is now available. I have not had much a chance to debug it yet, nor to include documentation, but it runs fine on several MacBooks that I had for testing.
• Oct 15, 2010: Finally, a new version is available: Beta4 runs on Windows with OpenCV 2.1.0, and you can use Haar detectors for your own favorite hand postures. As usual, this is in the download section.
• Jan 8, 2006: A binary installer for Beta3 for Windows is available in the download section.
• Jan 3, 2006: HandVu Beta3 is ready for testing. It includes: A new capture method for Windows and libdc1394 support for Linux. So far there's source code for all platforms - the Windows installer is in the works.
• Jan 12, 2005: HandVu was featured on the Point Grey Newsletter!
• Jan 11, 2005: Beta2 is out! There might be some silly bugs since a lot of things have changed, so please let us know and we'll probably be able to fix it quickly. Among the features: a ./configure script and an ARToolKit demo!
• Nov 24, 2004: A Windows installer is available now, for running HandVu without compiling it.
• Nov 24, 2004: There is a Yahoo HandVu Group now to discuss installation and development problems publicly.
• Nov 1, 2004: version beta1 is available for download, open-source and for Linux and Windows!
• Sept 10, 2004: ISWC and ISMAR demo accepted: we will present HandVu at the demo session on Monday, Nov. 1st, 5:30pm and on Thursday, Nov. 4th, at 3pm.
• April 22, 2004: released alpha3 for Windows (self-extracting zip, MSI installer)!
• Jan 13, 2004: a new version, alpha2, is available for Windows.
• Nov 19, 2003: alpha1 is the first publicly released HandVu version, available as of now!

Overview

This software collection implements a vision-based hand gesture interface. Called HandVu, it detects the hand in a standard posture, then tracks it and recognizes key postures - all in real-time and without the need for camera or user calibration. The output is accessible through library calls, a client-server infrastructure in a custom format and as OpenSound Control packets, allowing control of music applications that use this format.
The software package consists of the main HandVu library and several applications that demonstrate video capture with OpenCV's highgui, DirectShow, and soon the ARtoolkit. If your application includes video processing, you would just make a few library calls and hand gesture recognition results will be available to you. If your application does not use camera input already, you can run one of the sample programs on the side and interface to your application with a networking interface. Functionality to capture screenshots and snapshots of the tracked object is availabe, allowing data collection for research or other purposes.

Screenshots

Videos are all in WMV format for now: The first one shows mouse-based interaction with a head-worn camera. The second one focuses on the hand tracking aspect in both indoor and outdoor scenes. The third video demonstrates interaction with virtual objects and two-handed interaction with help of the ARToolKit.

still images

interacting with virtual objects and ARToolKitmouse and button interface with hand gesturesrubber band with-two handed interactiongesturing to our wearable computer

Requirements

• Windows (tested on XP and Vista) or Mac (tested on OSX 10.6.4),
• a 1.5 GHz or faster CPU (performance increases superlinearly with a faster CPU),
• OpenCV is included with the download, no separate installation required,
• a camera; either firewire, USB, or other frame grabber, that works with the OpenCV demos.

Download

Here's the installer for Windows: HandVu_beta4.exe 
And this is the installer (package) for MacOSX: HandVu_Beta4.pkg.zip. Download the ZIP archive. It usually extracts automatically. If not, double-click the ZIP archive and extract the package that it contains. Double-click the package (PKG, or installer; it might be in your Downloads folder, or wherever the ZIP archive ended up) and follow the instructions. Once installed, you should be able to just go to the Applications folder and double-click on HandVu. Feedback appreciated!

Installation

Windows installer

On Windows XP, if you don't have Microsoft Visual Studio installed, then you first need to run the MSVS re-distributable installer: vcredist_x86.exe. These files come with Windows Vista by default. Then, execute the HandVu installer that you downloaded from above and follow the instructions. To run HandVu, double-click on either the HandVu_OpenCV icon or the HandVu icon on the desktop. For troubleshooting, see the Windows Installation documentation.

Apple Mac OS X Installer

THE INSTALLER IS NOT READY YET! If you know how to turn a command-line tool into a bundle (specifics for Info.plist file?), let me know! Once it's ready.... Download the DMG file from above. Double-click it. Drag the HandVu icon to the application folder. Run HandVu.

Documentation

Quick Start Guide

1. Connect a camera to your computer.
2. Run the desktop shortcut to HandVu_OpenCV or to HandVu. The live video will appear in the application window.
3. Perform the closed posture at about the location shown below. After recognition of your hand, tracking will start and a number of key postures will be recognized.

   
   

Tijmen Kuipers kindly wrote an Installation Guide in Dutch

For more details, please have a look at the Application Documentation, which is included in the download package as well. Also, please make use of the Yahoo HandVu Group discuss installation or development problems.

Publications

Please see here.