I'm writing here to share some thoughts on my projects, during my computer engineering study.
Monday, 16 May 2011
Real time hand detection and gesture recognition
This was a computer vision research project. Hand detection and gesture recognition can be used in many helpful applications, and I've implemented some efficient techniques and algorithms to detect hand gestures, and to be able to control the PC and play games using the detected gestures.
One of the techniques I've used depends on the skin color features in the YCrCb color space. This color space is much preferable than RGB and HSV, as the skin color can be much efficiently differentiated in the YCrCb.
For a more efficient detection, I implemented a background subtraction algorithm to differentiate between skin like objects and real skin colors. Initially, a frame is captured with only the background in the scene, after that, for every captured frame, each pixel in the new frame is compared to its corresponding one in the initial frame, if they pass a certain threshold according to specific algorithm computations, then this pixel is considered from the human body and it will be drawn in a new frame with its original color. If this pixel is below the threshold, then those two pixels are considered the same and they are considered as background so the corresponding pixel will take a zero color in the third frame. After repeating this for all frames' pixels, now we will have a new frame with only a human appearing in it, and all the background took a color of zero.
After that, I apply the skin color detection algorithm on the resultant frame as I'm having now a clear human body, this algorithm will result in two objects only (assuming the user will use one hand) that are the user's hand and his face. Here the face will have two clear holes in the eyes' places, so it will be easily detected as a face and it will be discarded from the scene as shown in the next figure.
Now we are having the detected hand, I applied on this hand object an efficient gesture recognition algorithm, that draws a convex hull over the hand object, and counts the number of defects in this hull, if no defects found, then it is a closed hand, if five defects found, then there are five fingers waving, and so on.
Here is a video showing how it works
This technique proved that it is better than a lot of others I also implemented, that depend on HSV color space and chamfer matching.