Deep Detection of Face Morphing Attacks

Identification tasks with identity cards, passports and visa are often performed automatically by biometric face recognition systems. Criminals can trick these systems, such that two people can use the same passport for authentication. This attack (morphing attack) is performed by fusing two face images to a synthetic face image that contains characteristics of both people. Using this image on a passport, both people are authenticated by a biometric face recognition system.

To generate a face morph, we align two face images, such that each facial feature (e.g. left corner of the right eye) is at the same position in both images. Since the haircut and the ears are difficult or even impossible to align and usually ignored by commercial facial recognition systems, we blend only the face region and stitch it into one of the aligned input images. The transitions are calculated individually for different frequency bands and an optimal stitch line is estimated.

In order to decide whether a face image is authentic or crated by a morphing algorithm, we trained deep convolutional neural networks on morphed and original face images. Since we focus on semantic image content, like highlights in the eyes or the shape and appearance of facial features to determine the authenticity of a face image, we applied several preprocessing steps on our training data to remove sensor or camera specific information. In addition to a decision about the authenticity of an image, we are also interested in a reason for decision. For this purpose, we analyze the regions in the face image that were relevant for the networks decision using the Layer-wise Relevance Propagation (LRP) Toolbox.
 

In addition to our face morphing detector that provides only a decision and a confidence score, we developed a system to precisely highlight traces of forgery in morphed face images. In order to develop such a system, we started with an in-depth analysis of our DNN-based face morphing detectors and discovered that a naïve training of these detectors has some drawbacks. The naively trained DNNs tend to focus mostly on the eyes and pay little attention to traces of forgery in other regions of the face. In order to build a tool that can detect and localize traces of forgery in all parts of the face, this can be a problem and a detector that is sensitive to face morphing typical artifacts in all regions of a face image is required. We modified our DNN-based detectors by changing the DNN’s structure and the training loss to help the DNN to develop artifact-aware neurons for all regions in der output of the DNN’s feature extractor. These neurons can be exploited by Focused Layer-wise Relevance Propagation (FLRP) to detect traces of forgery in all parts of the image. FLRP is an extension of LRP that we developed to put focus more on detecting class specific regions than on the overall decision of the DNN. FLRP, our modification of the DNN and the new training loss provide in combination an excellent basis for a framework that precisely highlights all traces of forgery in morphed face images. The high accuracy of the morphing artifact localization has been demonstrated in extensive experiments with locally induced traces of forgery. This makes it a perfect tool to highlight suspicious regions in morphed face images to make the detector’s decision more transparent to end-users.

One important step in the process of creating a morphed face image is the blending of two aligned images. During this process, high frequency details, like wrinkles, scars or pore structures are smoothed or get lost and the resulting image appears dull. A detector might use such characteristics to detect morphed faces, but a professional attacker can manually enhance such details to stay undetected. We developed a method based on Neural Style Transfer that counters this image quality degeneration caused by blending. Thus, we are able to generate morphed face images with characteristics similar to the genuine input images. Our detectors were not able to detect such improved images in many cases at first. After adding the improved morphs to our pool of training data, our detectors can reliable detect them too. In addition, our detectors become even better in detecting morphs that have been improved by post-processing methods, which are not reflected by the training data. Improving the face morphs for our training data thus helped to improve the robustness of our detectors against unknown post-processing methods and to develop detectors that are more general.

Face images on the chip of electronic Machine Readable Travel Documents are strongly compressed using JPEG or JP2000 compression methods. We analyzed the effect of these lossy compression methods on the accuracy of different detection methods for morphed face images. We found that our DNN-based detections methods are already very robust against image compression. Our DNN-based methods are still able to detect them, if artifacts, which would be used by a human to classify a face image as a morph, are not recognizable for a human inspector. Further investigations in the intermediate steps of our detectors show that they learned features that are already invariant against compression.