Real time hand detection and gesture recognition

This was a computer vision research project. Hand detection and gesture recognition can be used in many helpful applications, and I've implemented some efficient techniques and algorithms to detect hand gestures, and to be able to control the PC and play games using the detected gestures.

One of the techniques I've used depends on the skin color features in the YCrCb color space. This color space is much preferable than RGB and HSV, as the skin color can be much efficiently differentiated in the YCrCb.

For a more efficient detection, I implemented a background subtraction algorithm to differentiate between skin like objects and real skin colors. Initially, a frame is captured with only the background in the scene, after that, for every captured frame, each pixel in the new frame is compared to its corresponding one in the initial frame, if they pass a certain threshold according to specific algorithm computations, then this pixel is considered from the human body and it will be drawn in a new frame with its original color. If this pixel is below the threshold, then those two pixels are considered the same and they are considered as background so the corresponding pixel will take a zero color in the third frame. After repeating this for all frames' pixels, now we will have a new frame with only a human appearing in it, and all the background took a color of zero.

After that, I apply the skin color detection algorithm on the resultant frame as I'm having now a clear human body, this algorithm will result in two objects only (assuming the user will use one hand) that are the user's hand and his face. Here the face will have two clear holes in the eyes' places, so it will be easily detected as a face and it will be discarded from the scene as shown in the next figure.

Now we are having the detected hand, I applied on this hand object an efficient gesture recognition algorithm, that draws a convex hull over the hand object, and counts the number of defects in this hull, if no defects found, then it is a closed hand, if five defects found, then there are five fingers waving, and so on.

Here is a video showing how it works

This technique proved that it is better than a lot of others I also implemented, that depend on HSV color space and chamfer matching.

I-Deliver

I would like to talk about my first robot :)

Actually it is not a real robot :) it can be considered a delivery car controlled from the PC. This car can deliver papers or any lightweight materials from one place to another automatically without human help or interference. 

First you give it a map for the building or the flat you live in, this map has checkpoints for the rooms, kitchens and any other stop point you would like to have. The map can be generated using a simple interface to tell the car about places it can move through, and places it cannot.

After installing your map, you can order the car to deliver anything from its current place to a destination point, you make your orders using a keyboard matrix installed on the car where you can select the room number and simply press GO, after that, the car will move according to your map, going left and right, bypassing walls, making sound alerts if someone stopped in front of it, until it reaches its destination point at the requested room.

We have built this car from A to Z in about 10 days of hard work, Its cost is not huge, It is simple, but actually it costed us much money :) as we burned and destroyed hundreds of ICs :). Note that this was our first time to deal with a complex hardware and to implement electronic circuits by hand in a real project.

We used:

• Two stepper motors (5V 1A each) to control the two front wheels, and two interfacing drivers one for each motor (M106)
• A keyboard matrix, to input the commands and room numbers from the user
• An infrared sensor
• A parallel cable, to interface the devices with the PC (for sure it is not a good option but remember it is not an embedded project)
• And of course there were some transistors, resistors, voltage regulators, capacitors and different sorts of ICs.

Don't laugh at its shape, I know it is like a duck :).

This is before the surgery finishes:

Its final shape:

Namaa Management System

Back again, to write about my voluntary application I've successfully delivered.

This is a desktop based application, to manage an Egyptian charity organization called Namaa. It is to manage the data of orphans, poor, volunteers, offers, donations, loans and projects. It was implemented using C#.net and MS SQL Server. It contains very simple UI, has a lot of functionalities and very easy to use.

Here are some screen shots of it