A Face Detection and Recognition System for Color Images using Neural Networks with Boosting and Deep Learning

Abstract

A face detection and recognition system is a biometric identification mechanism which compared to other methods such as finger print identification, speech, signature, hand written and iris recognition, is shown to be more important both theoretically and practically. In principle, the biometric identification methods use a wide range of techniques such as machine learning, computer vision, image processing, pattern recognition and neural networks. The methods have various applications such as in photo and film processing, control access networks, etc. In recent years, the automatic recognition of a human face has become an important problem in pattern recognition. The main reasons are structural similarity of human faces and great impact of illumination conditions, facial expression and face orientation. Face recognition is considered one of the most challenging problems in pattern recognition. A face recognition system consists of two main components, face detection and recognition. In this dissertation a face detection and recognition system using color images with multiple faces is designed, implemented, and evaluated. In color images, the information of skin color is used in order to distinguish between the skin pixels and non-skin pixels, dividing the image into several components. Neural networks and deep learning methods has been used in order to detect skin pixels in the image. A skin database has been built that contains skin pixels from different human skin colors. Information from different color spaces has been used and applied to neural networks. In order to improve system performance, bootstrapping and parallel neural networks with voting have been used. Deep learning has been used as another method for skin detection and compared to other methods. Experiments have shown that in the case of skin detection, deep learning and neural networks methods produce better results in terms of precision and recall compared to the other methods in this field. The step after skin detection is to decide which of these components belong to human face. A template based method has been modified in order to detect the faces. The template is rotated and rescaled to match the component and then the correlation between the template and component is calculated, to determine if the component belongs to a human face. The designed algorithm also succeeds if there are more than one face in the component. A rule based method has been designed in order to detect the eyes and lips in the detected components. After detecting the location of eyes and lips in the component, the face can be detected. After face detection, the faces which were detected in the previous step are to be recognized. Appearance based methods used in this work are one of the most important methods in face recognition due to the robustness of the algorithms to head rotation in the images, noise, low quality images, and other challenges. Different appearance based methods have been designed, implemented and tested. Canonical correlation analysis has been used in order to increase the recognition rate.