Monday, 16 May 2011

Real time hand detection and gesture recognition

This was a computer vision research project. Hand detection and gesture recognition can be used in many helpful applications, and I've implemented some efficient techniques and algorithms to detect hand gestures, and to be able to control the PC and play games using the detected gestures.

One of the techniques I've used depends on the skin color features in the YCrCb color space. This color space is much preferable than RGB and HSV, as the skin color can be much efficiently differentiated in the YCrCb.

For a more efficient detection, I implemented a background subtraction algorithm to differentiate between skin like objects and real skin colors. Initially, a frame is captured with only the background in the scene, after that, for every captured frame, each pixel in the new frame is compared to its corresponding one in the initial frame, if they pass a certain threshold according to specific algorithm computations, then this pixel is considered from the human body and it will be drawn in a new frame with its original color. If this pixel is below the threshold, then those two pixels are considered the same and they are considered as background so the corresponding pixel will take a zero color in the third frame. After repeating this for all frames' pixels, now we will have a new frame with only a human appearing in it, and all the background took a color of zero.

After that, I apply the skin color detection algorithm on the resultant frame as I'm having now a clear human body, this algorithm will result in two objects only (assuming the user will use one hand) that are the user's hand and his face. Here the face will have two clear holes in the eyes' places, so it will be easily detected as a face and it will be discarded from the scene as shown in the next figure.

Now we are having the detected hand, I applied on this hand object an efficient gesture recognition algorithm, that draws a convex hull over the hand object, and counts the number of defects in this hull, if no defects found, then it is a closed hand, if five defects found, then there are five fingers waving, and so on.

Here is a video showing how it works

This technique proved that it is better than a lot of others I also implemented, that depend on HSV color space and chamfer matching.


  1. great work!!!!!!!
    i m also trying to implement something like these with visual studio2010 and opencv2.1 but bit confuse from where to start. i m new to all these so kindly help

  2. You should start first by learning image processing and analysis techniques (edge detection, segmentation, noise removal, contrast enhancement, ... etc) you can Google it and you'll find resources for complete courses.

  3. @Ahmed Orabi, Hi there! We are currently developing a program that allows a user to manipulate a PowerPoint Presentation (move next slide, previous slide) and perhaps even add zoom functionality using fingers and mouse manipulation. Do you think you could teach us how we can accomplish this? Many thanks in advance! It would be a great help for us graduating students.

  4. hello friend
    i m working on same project using opencv.
    can u please help me out with code

  5. could u mail me the code to
    i need it very urgent... pls help me

  6. أحلى أبو حميد.. هلأ صاروا الشباب يركدو وراك وعلى: بليز هيلب.. بليز هيلب
    صديقك في الكلية: براء

  7. ممتاز يا عزيزي ولكن لماذا لا تكتب باللغة الام لكي ترتقي بالمحتوي العربي للنت. ان الاجانب يمكلون من يكتب لهم بلغتهم

  8. Hi
    am 4th year student in engineering faculty, CS Dept,
    my graduation project contain hand detection but i don't know from where to start..
    if you can mail me with that demo code..will help me a lot
    please mail me at :

  9. hlo bro,
    Can you sent me this project code that really help me for my project to open cv @