Face and liveness detection with criminal identification using machine learning and image processing techniques for security system

ABSTRACT


INTRODUCTION
Nowadays, biometrics is one of the most widely used authentication technologies.Face recognition technology is one of them, and it is widely used due to its simplicity and accuracy.Face recognition technology is now being used in a wide range of facial spoof attacks, including those on smartphones, tablets, and laptop computers.Face recognition technology allows us to recognize other people.This facial recognition application works by photographing a person's face with a camera and then running the image through a specific algorithm to determine whether or not the face is recognized from a database [1].Nonetheless, the facial recognition strategy has a flaw known as spoofing attacks.Facial recognition systems can't tell the difference between real Int J Artif Intell ISSN: 2252-8938  faces and spoofing attacks like masks, videos, or photos.As a result, these flaws allow someone to deceive the machine.Furthermore, obtaining someone's face is far easier than obtaining other biometrics such as fingerprints.Using social media or a profile photo, you can easily obtain someone's face [2].
Face spoofing attacks can be static or dynamic [3].Dynamic 2D demonstration spoofing attacks use video replays or a large number of photos in a sequence, whereas static attacks use photos or masks.Static 3D demonstration attacks may employ 3D sculptures, prints, or even masks, whereas animated versions employ complex robots to mimic facial expressions, complete with cosmetics.
Another technique for identifying real people is liveness detection, and Eye-blink detection is a highly accurate liveness detection evaluation.Natural blinking is an easy way to determine whether a face is alive or dead.A blink closes one's eyes for about 250-300 milliseconds.In [4] a typical person blinks 5-10 times per minute.Eye blink detection can be used to analyze face landmarks and calculate the surface area of the eyes.However, because modern technology makes it easy to attack video replays with devices like smartphones or tablets, relying on blinking eye detection is no longer sufficient [5].
By analyzing individual facial motions, the movement detection method attempts to recognize vital signs.Humans may be distinguished from inanimate items like images by this movement.Changes in face expression, blinking eyes, and lip motions are a few of the most often used motion detection methods [6].Motion-based evaluation methods are typically adequate for preventing inactive representation strikes such as photo-spoofing, but they fail to prevent dynamic rendering attacks such as videos [7].
3D cameras or photoplethysmography [8] are the most reliable anti-spoofing methods.Because we can distinguish between a face and a flat object, pixel depth advice may provide high precision against demonstration attacks [9], [10].Cameras, on the other hand, continue to be one of the most reliable anti-spoofing methods available.Furthermore, even though customers have access to cameras, few have them on their computers, and it is not suitable for use on mobile devices such as smartphones [11].
This system is intended to assist any investigation department in identifying criminals.Images of criminals are stored in our database alongside their details in this system, and these images are then segmented into four slices-foreheads, eyes, nose, and lips.These images are then saved in a different database record to help with identification.The slices that appear on the screen will be chosen by eyewitnesses, and we will use them to retrieve the image of the face from the database.As a result, if the criminal's record is found in the database, this system provides a very friendly environment in which both the operator and the eyewitness can easily identify the criminal [12].
Deep learning and convolutional neural networks (CNN) are two other anti-spoofing technologies.The system could train CNN to distinguish between genuine and spoofed images.However, there is one problem.The convolutional network sees and understands no consistent set of features [13].The entire model was built on the hope that the system would detect what our eyes couldn't see.As a result, i believe it is critical to combine detection methods for signs of life, such as blinking or lip movements, with CNN analysis methods.To limit the scope of our face liveness detection, we will use blink detection and lip movement detection because these two signs are the most common and simple to detect.
As a result, this study looks into an advanced face liveness detection method and CNN for telling the difference between fake and real faces.It is straightforward, and, more importantly, it is more resistant to environmental changes and various attack methods.The significant contributions of the work are listed: i) the proposed procedure is completely accurate because it uses CNN and deep transfer learning to learn signs that reflect both real and fake face characteristics; ii) the proposed method is simple to implement and does not necessitate the purchase of any additional hardware; and iii) the proposed anti-spoofing scheme is strong and detects spoofing in real time.In complex real-world indoor and outdoor scenarios, it can deal with various spoofing attacks (print, replay, and mask).
In this research, we will assess how well deep CNN produced face pictures and convolutional layers' adaptive learning of convolutional features from real-world face photos can identify the liveness of a face.In addition, a layer for balancing the merging of convolutional features from both deep CNN produced face pictures and convolutional features from real-world face images is presented.The suggested technique outperforms state-of-the-art algorithms on face liveness detection, according to extensive trials on cutting-edge face anti-spoofing databases including CASIA, OULU, and Replay-Attack, with both intra-database and cross-database situations [14]- [17].

LITERATURE REVIEW
Yuan et al. [1] propose a system for dealing with this fingerprint animosity detection, as well as a workable anti-dismissal tool (FLD).Furthermore, the profound neural network (DCNN) based FLD methods were significantly different from most shallowness due to their quick operation, few parameters, and end-to-end self-learning.Methods for creating detailed features.Meanwhile, DCNN is confronted with two opposing challenges.On the one hand, multi-faceted perception (MLPs) continues to rise and is finally becoming  ISSN: 2252-8938 stable.To increase the number of MLPs, the results will be reduced further.However, extensive research indicates that obtaining high performance detection requires a certain minimum number of MLP.For the first time, we used FLD to resolve the conflict known as the deep residual network in this paper (DRN).Then, to eliminate interference from incorrect portions of given photos, an extraction algorithm (ROI) is proposed.Then, adaptive DRNs are exploring ways to avoid the parameters learned falling into local optimization by automatically adjusting the learning rate if such monitoring parameters (checking correctness) are stable.
The study in [2] a "desktop anti-spoofing application" is proposed in this paper.To determine whether a face is living, this programme counts the number of eyes blinking.Face detection and identification are the application's primary stages, as well as determining the user's liveness status.It has been demonstrated that liveness detection can prevent video playback attacks and the use of printed photographs to compromise security.The webcam captures the user's image at regular intervals.The image is checked for liveness after it has passed the authentication process.In the event of a security breach, countermeasures are put in place.This includes photographing an adversary and logging off or exiting the system.
Killioglu et a.l [3] concentrated on liveness detection for spoofing facial recognition systems with artificial face motion.A modest piece of hardware was used by the authors to create a pupil direction observation system for face recognition systems.The first step is to extract the eye area from a real-time camera using a specifically trained eye region recognition classifier and the haar-cascade classifier.Feature points were extracted and traced using the Kanade-Lucas-Tomasi (KLT) algorithm to minimize a person head movements and obtain a stable eye region.The real-time camera frame is chopped and rotated to stabilize the eye region.Then, using a new, enhanced method, the pupils are retrieved from the eye region.
Li et al. [4] face recognition is a popular biometric technology due to its ease of use; however, It is susceptible to spoofing attacks from fake faces, such as those in a real user's picture or video.An essential technique for confirming that the input face belongs to a real person is face liveness detection.Traditional liveness detection methods, such as texture analysis and motion detection, remain extremely difficult.The objective of this study is to provide a framework that is effective at dealing with face liveness detection and identification as well as a multifunctional feature descriptor.This framework employs a multiscale directional transform to define new feature descriptors (shearlet transform).Then, to detect the liveness of a face and identify the person, stacked auto-encoders and a softmax classifier are combined.The authors tested this approach using the CASIA face anti-spoofing database, and the results show that when tested using the database's evaluation protocols, our approach outperforms state-of-the-art techniques, indicating that it is possible to significantly improve the security of face recognition biometric systems.
Peng and Chan [5] a typical adversarial approach in facial recognition is spoofing, in which the attacker presents the user's photos or videos in front of the camera while pretending to be them.In order to maintain the security of the system, face liveness detection is employed to separate photographs taken from a live face from those taken from a faked face.In this study, the authors provide a face liveness detection approach based on the high frequency descriptor to counter spoofing assaults.By revealing more hair and skin features and creating a shine on the flat surface, more lighting may both boost and drop the energy of high frequency components of a genuine face.
Cai and Quan, [6] facial anti-spoofing is a small part of face recognition systems, which are necessary for access control and financial payment systems.By merging convolutional neural network with brightness equalisation, an unique approach is proposed that overcomes the issues of unstable face alignment, complicated lighting, and complex face anti-spoofing detection network structure.A multi-task convolutional neural network (MTCNN) built on a cascade of three CNNs, P-net, R-net, and O-net, is employed first to obtain precise face positioning, and the identified face bounding box is cropped by a predetermined multiple.
Mohamed et al. [7] face recognition is one of the most frequently utilised biometric techniques.Numerous applications make use of face recognition.The authentication of mobile devices is one of these areas.Mobile security is becoming necessary as more people use mobile devices every year.Face recognition, on the other hand, is susceptible to deceptive face spoofing.Using facial images that have been derived from pictures or movies, this trick is used to fool face recognition software.Other cheats dress in official uniforms to make it appear as though they are authorised personnel to the recognition camera.For scope of identifying face spoofing, liveness detection is a crucial study area.
Hadiprakoso et al. [8] biometrics based on facial recognition are now widely used.A face identification system should be able to recognise not only people's faces, but also attempts at spoofing using printed faces or digital presentations.Examining liveness of the face, such as eye blinking and lip movement, is an effective spoofing prevention strategy.Nonetheless, when dealing with video-based replay attacks, this approach is rendered ineffective.As a result, this paper proposes a method for detecting the liveness of a face using a CNN classifier.The blinking eye module, which assesses eye opening and lip movement, and the ConvNet classifier module make up the anti-spoofing approach.Data from a number of freely accessible sources may be used to train our CNN classification system.Kumar et al. [9] biometrics is now one of the most widely used security applications.Face recognition is the most widely used method due to it is uniqueness.To grant access to confidential resources, authors must ensure that only genuine live face images are used.Hackers, on the other hand, undermine this system by impersonating genuine users through photo attacks, video replay attacks, and 3-D attacks.To deter these impostors, many deterrent mechanisms have been devised.As a result, a neural network-based detection method for spoofed faces is proposed.The architecture of the convolutional network is intended to prevent spoofed faces from gaining access in the name of legitimate users.
Singh et al. [10] face anti-spoofing is critical for keeping face recognition systems secure.Deep learning approaches have previously treated face anti-spoofing as a binary classification problem.Many of them struggle to understand appropriate spoofing cues and make incorrect generalizations.The authors argue in this paper that auxiliary supervision is critical for guiding learning toward discriminative and generalizable cues.A CNN-RNN model is trained with pixel-by-pixel supervision to estimate face depth and rPPG signals with sequence-by-sequence supervision.To distinguish between live and spoof faces, the estimated depth and rPPG are combined.

PROPOSED WORK
This study proposes developing an anti-spoofing model with three major modules: face anti-spoofing detection, liveness detection, and criminal identification using CNN classifier.The operation scheme of this model is quite simple.The face anti-spoofing module will process the input and detect photos, posters, masks, or Smartphones.When a face is detected, the input is sent to the CNN classifier module, which determines whether the face is real or fake.The following input will be processed for the liveness detection module, which detects eye blinks and lip movements.If the input is processed by both modules, it is designated as a real face.
Finally, concentrate on the third approach, which is a criminal identification module that will detect face recognition input during face anti-spoofing detection.If a real face is found in the face anti-spoofing detection module, this face provides input to the criminal identification module, which determines whether the face is normal or criminal [18]- [20].The methodology we propose is made up of several general steps.The steps to develop the CNN classifier modules are: data collection, data pre-processing, model training, model evaluation, and testing.
The life sign (liveness) detection module on the face has two sub-modules: blink detection and lips motion detection.The lip-movement-net module [21] is used in this module to detect lip motion.A simple recurrent neural network (RNN)-based detector algorithm determines whether someone is speaking by analyzing their lip movements for 1 second of video using the Python programming language as part of the module.The detector module can be run in real-time on a video file or camera output.This module detects lip movement by first creating a filter to determine the upper and lower lip locations and then calculating the lips separation distance [22].Figure 1 shows proposed system architecture details flow.To determine whether or not the eyes are blinking, a module developed in previous research [23] will be used.To determine whether or not the eyes are blinking, we use an eye area filter.The presence of the eye area in a person's face photo input can be detected by filters.The next step is to detect eye openness after capturing the eye area.This step employs the classification of eye openness.This classification produces a probability of opening the eye to the input image, which is then analyzed based on the value difference between the maximum and minimum eye openings.If the difference is significant, the eyes are blinking, which means that at least one transition between the eyes is open and closed.We prepared a dataset of faces with closed eyes and a dataset of faces with open eyes to create an eye classification module [23].

Face liveness detection
Face recognition is a biometric system that compares data from people in a database of known faces to features extracted from someone's face.Researchers have developed a variety of methods for recognizing a person's face, overcoming challenges such as different facial expressions, different angles, and poor lighting.In the last decade, it has spread rapidly.Applications for it include facial recognition in attendance systems, purchases, and mobile device authentication [1].In forensics and security access, it is also utilised [2].Face faking is one of the problems that developers have while putting in place a face recognition system.Attackers who attempt to trick facial recognition software are said to engage in face spoofing.The most well-known face spoofing techniques are printed photos, films, and 3D masks, as illustrated in Figure 2.These techniques let an attacker contact a target without their permission and get around a face recognition system.

Figure 2. Face spoofing types
One of several techniques used to prevent face spoofing attacks is face liveness detection.Sensing face liveness is a relatively new technology, as fingerprints and passwords are the most commonly used security methods.However, many businesses require face spoofing detection to prevent unauthorised access to their systems.The use of a projected picture (or video) or 3D mask of a real person in front of a security camera allows cheaters to get unauthorised access.Access to the faces of live individuals is one of the security system's functions.In order to stop face spoofing assaults, it will be essential to detect face liveness.

Face recognition system
Using real users' photographs, videos, and 3D face masks, certain unlawful intruders are now able to execute spoof face assaults against systems.Face recognition can only identify the identity of a face; therefore, it cannot shield users from attacks from non-living faces like spoof faces, which are a serious security risk to the system.Face recognition simply detects the identity of the face; hence it cannot protect against attacks from non-living faces like spoof faces.Face anti-spoofing detection technology is mostly used to distinguish between actual and fake faces in order to stop spoof attacks.It is crucial to create a face anti-spoofing strategy with high detection accuracy and great generalisation capacity to support the face recognition and authentication system against malicious assaults since face anti-spoofing is a crucial security defensive mechanism in face recognition systems [24].Face recognition system illustrate in Figure 3.

Criminal identification
There has been an unusual increase in the crime rate, as well as an increase in the number of criminals, raising serious concerns about security.The primary issues confronting police personnel are crime prevention and criminal identification, Police are primarily concerned with protecting property and lives, yet there aren't many officers available to fight crime.An automated facial recognition system for a criminal database was suggested in this paper using a well-known CNN classifier.This technology will enable real-time face detection and identification of criminal which shown in Figure .4.Accurately identifying the face is still challenging.To locate faces and other objects in a picture, researchers frequently employ the Viola-Jones framework.With open communities like OpenCV, face detection classifiers are shared [25].With a full solution for image-based face identification and recognition with improved accuracy, a higher response rate, and as a first step towards video surveillance, this system aims to assess face detection and recognition methods.

RESULTS AND DISCUSSION OF PROPOSED WORK
The main contribution of the proposed framework is: i) The objective of this study is to evaluate face liveness, including eye blinking and lip movement, as a defence against spoofing.Nevertheless, this tactic is worthless when facing attacks based on replay videos.This provides a method that combines facial liveness detection with CNN classifier.ii) We'll provide you a thorough performance study of the recommended system's handling of the anti-spoofing problem in both intra-and cross-database scenarios in this part.Also covered are the placement of an adaptive convolutional-feature fusion layer in a CNN network and how that affects the efficiency of face liveness detection in general.iii) It is necessary to build and create a system for criminal identification in the third module of the CNN module employing a facial recognition method to crime prevention.iv) To enhance the efficiency and precision of suggested algorithms.v) To use a deep CNN network with an adaptive convolutional-feature fusion layer to combine convolutional features learned from real-world face photographs with deep CNN-based auto-encoder produced (DNG) face images in a weighted manner.

CONCLUSION
Provide In this research work Face identification and recognition is the process of comparing data from a camera to a database of known faces and finding the match.This general face recognition method has flaws.What if someone impersonates someone else or is a criminal?A liveness check overcomes this by distinguishing between a real face and a photograph.The reliability of the face recognition application is increased by the detection of liveness through eye-blink and lip movement.The suggested strategy is a multi-platform programme that will increase the security of a business, governmental, or banking system.This is an inexpensive, automated solution that runs without user input.Application testing is done on real data in challenging environments to show how reliable and effective the suggested work is.Using the ORL, OULU, and CASIA datasets, the performance evaluation of the increased functionality utilising CNN as a classifier provided results that were adequate.

Int
Face and liveness detection with criminal identification using machine learning … (Pratibha Shinde) 725

Figure 1 .
Figure 1.Proposed system architecture diagram

Figure 4 .
Figure 4. Structure of criminal identification using face recognition system