Face recognition has become one of the most important biometrics authentication technologies in the past few years. Two main reasons for extensive attention on face recognition technology are: 1) Aptness in various applications including in content-based video processing system, law enforcement system and in security systems. A strong need for a robust automatic system is obvious due the widespread use of photo-ID for personal identification and security 2) although there are reliable methods of biometrics identifications existed such as fingerprint scans and iris scans, face recognition is proven effective for its user-friendliness. The system does not require its users to do anything; it is contactless. On top, as one of the core components, the maturity of the digital camera technology with competitive price is also a contributing factor to the strong emergence of face recognition technology.

Most of the face recognition techniques have evolved in order to overcome two main challenges: illumination and pose variation. Either of these problems can cause serious performance degradation in a face recognition system. Illumination can change the appearance of an object drastically, and in the most of the cases these differences induced by illumination are larger than differences between individuals, what makes difficult the recognition task. The same statement is valid for pose variation. Usually, the training data used by face recognition systems are frontal view face images of individuals. Frontal view images contain more specific information of a face than profile or other pose angle images. The problem appears when the system has to recognize a rotated face using this frontal view training data. Furthermore, the appearance of a face can also change drastically if the illumination conditions vary. Therefore, pose and illumination (among other challenges) are the main causes for the degradation of 2D face recognition algorithms.

Some of the new face recognition strategies tend to overcome both challenges from a 3D perspective. The 3D data points corresponding to the surface of the face may be acquired using different alternatives: a multi camera system (stereoscopy), structured light, range cameras or 3D laser and scanner devices. The main advantage of using 3D data is that depth information does not depend on pose and illumination and therefore the representation of the object do not change with these parameters, making the whole system more robust. However, the main drawback of the majority of 3D face recognition approaches is that they need all the elements of the system to be well calibrated and synchronized to acquire accurate 3D data (texture and depth maps). Moreover, most of them also require the cooperation or collaboration of the subject making them not useful for uncontrolled or semi-controlled scenarios where the only input of the algorithms will be a 2D intensity image acquired from a single camera.

This article discusses two main algorithm families commonly used to recognize faces: two-dimensional based and three-dimensional based recognition. Both of these two algorithms recognize faces images in different ways; two-dimensional algorithm is based on information theory concepts, seeks a computational model that best describes a face by extracting the most relevant information contained in that face while three-dimensional facial geometry represents the internal anatomical structure of the face rather than its external appearance influenced by environmental factors. As will be shown in this article, both algorithms have advantages and disadvantages. FingerTec® continued research and development work has led to a more accuracy and robust face recognition technology, the FingerTec® Face Recognition solution.

Over the past decades, FingerTec® has concentrated on developing face recognition methods within the framework of biometrics security systems and is now applying face recognition technology to other markets. FingerTec® face recognition technology can be implemented as a functionally independent application, or seamlessly integrated into new or existing biometrics security solutions by system integrators and solution providers.

FingerTec® face recognition presented a novel and highly descriptive 2D-3D mixed local feature and demonstrated its performance on a challenging interclass recognition problem. By combining the 2D and 3D local features, it provides high speed and high accuracy for facial detection and facial features extraction and achieved a significant improvement in performance. Moreover, the combined performance deterioration is significantly lower than that of the individual features.

Principal component analysis (PCA) is one of the widely used 2D face recognition algorithm. It is based on information theory concepts, seeks a computational model that best describes a face by extracting the most relevant information contained in that face. The Eigenfaces approach is a PCA method, in which a small set of characteristic pictures are used to describe the variation between face images. The goal is to find the eigenvectors (eigenfaces) of the covariance matrix of the distribution, spanned by training a set of face images. Later, every face image is represented by a linear combination of these eigenvectors. Recognition is performed by projecting a new image onto the subspace spanned by the eigenfaces and then classifying the face by comparing its position in the face space with the positions of known individuals.

The PCA-eigenfaces system capture the image and change it to light and dark areas. Both the initial facial image and the facial image in question are also captured in a two-deimensional form. Then, the two images are compared according to the points of the two eigenface image. It picks out certain features and calculates the distances between them. The points are the facial features such as eyes, nose, mouth, bone curves, and other distinct features. The eigenface algorithm firstly forms overall average image. This is the image just adding all images and dividing by number of images in training set. And the eigenvectors of covariance matrix that is formed by combining all deviations of training set’s images from average iamge is formed in order to applu eigenfaces algorithm. After finding overall average image, the order is to find eigenvectors of the covariance matrix. Visualization of eigenvectors is carried out simply applying a quantization that is if the found eigenvectors have components that are greater than 255 and smaller than 0 round them to 255, and 0 respectively.

Three-dimensional face recognition (3D face recognition) is a modality of facial recognition methods in which the three-dimensional geometry of the human face is used. 3D face recognition has the potential to achieve better accuracy than its 2D counterpart by measuring geometry of rigid features on the face. This avoids such pitfalls of 2D face recognition algorithms as change in lighting, different facial expressions, make-up and head orientation. Another approach is to use the 3D model to improve accuracy of traditional image based recognition by transforming the head into a known view. Additionally, most range scanners acquire both a 3D mesh and the corresponding texture. This allows combining the output of pure 3D matchers with the more traditional 2D face recognition algorithms, thus yielding better performance (as shown in FRVT 2006). The main technological limitation of 3D face recognition methods is the acquisition of 3D images, which usually requires a range camera. This is also a reason why 3D face recognition methods have emerged significantly later (in the late 1980s) than 2D methods. Recently commercial solutions have implemented depth perception by projecting a grid onto the face and integrating video capture of it into a high resolution 3D model. This allows for good recognition accuracy with low cost off-the-shelf components.

3D Face recognition system consists of four modules: Device Module, Data Processing Module, Feature Extraction Module and Matching Engine Module. The Device Module acquires initial 3D facial data by a 3D surface scanner VGA camera and transfers it to the processor. After receiving raw data (the distorted pattern on the target object), the Data Processing Module performs image filtering (noise reduction) and then instantly reconstructs the 3D surface, smoothing and Interpolating data to avoid holes and optimize the mesh. The Feature Extraction Module receives the optimized 3D surface for further feature vector (biometric template) extraction. During biometric template extraction, a proprietary twostage algorithm is used. At the first stage, the surface “semantic analysis” is performed, resulting in the location of key crania-facial landmarks (points) on the facial surface and the fitting of the surface to a generic topological map of the face. At the second stage, when the location of specific surface patches (eye-sockets, super ciliary’s arches, forehead zone, nasolabial zone, chin zone, etc.) is known, information about local surface curvature characteristics is extracted. This local curvature information is used further to build a single geometric descriptor and packs this data into a biometric template. The output of the module is a biometric template uniquely characterizing the person, which is used in the next matching stage. The Matching Engine working in identification mode compares the extracted biometric template with all of the stored templates in the database and produces a similarity score for each of the stored templates. The template with the best similarity score is the top match.

Both 2D and 3D face recognition techniques are used in FingerTec® face recognition algorithm technology. This leads to a new paradigm using the mixed 2D-3D face recognition systems where 3D data is used in the training but either 2D or 3D information can be used in the recognition depending on the scenario. Following this concept, where only part of the information (partial concept) is used in the recognition, a novel method is presented in this work. This has been called Partial Principal Component Analysis (P2CA) since it fuses the partial concept with the fundamentals of the well known PCA algorithms. Both strategies have been proven to be very robust in pose variation scenarios showing that the 3D training process retains all the spatial information of the face while the 2D picture effectively recovers the face information from the available data. Simulation results have shown recognition rates above 91% when using face images with a view range of 45º around the human face in the training stage and 2D face pictures taken from different angles (from -45º to +45º) in the recognition stage.

FingerTec® has been gone through many tests based on different image capturing resolution, lighting environment, poses angles where 100000 faces images are obtained to test the performance of the FingerTec® face recognition algorithm in the past 2 years. The summary of the results as below:

Face processing is one of the most active research fields as demonstrated by more than 1000 publications that have appeared in different conferences and journals in the last few years. Additionally, it is also a mature topic with more than 30 years. Recently, a new trend of 3D face recognition approaches showed an increase in the recognition rate if 3D data is available. Nevertheless, cost of the set-up, acquisition time and cooperation of the subjects are still some of the requirements for obtaining accurate 3D data that may not be available during the recognition stage. Thus, we have presented a mixed 2D-3D face recognition philosophy, the system is trained with 3D data but it can use either 2D or 3D data in the test stage. We have presented the extension of the 2D statistical PCA method to a 2D-3D face recognition scheme (Partial Principal Component Analysis). However, this philosophy may be extended also to other face recognition statistical approaches like LDA or ICA with have shown a higher robustness in the presence of illumination variations. Additionally, we have presented an automatic approach for the creation of aligned virtual view images using nine different views. These aligned virtual view images are used as training data for the 2D-3D mixed technique. The virtual view image is created by using a cylindrical approximation for the real object surface. The alignment is done by global and local transformations of the whole image and face features, respectively. Results show an improvement in the recognition rate when using the local alignment procedure proposed. FingerTec® will continuously run field test of the complete system to get statistics for continued improvement of the 2D-3D mixed face recognition performance.