Facing the Future: Navigating the Complexities of Facial Recognition

Take a moment to imagine the consequences of people having their cell phone PIN written across their forehead.  The security implications would be severe; however, the credential used to unlock a cell phone is already on display to the public regularly.  This is not something known or possessed, but rather a part of each one of us – a face.  As facial recognition technology advances, it is vitally important that the benefits, challenges, and implications are widely understood by both those implementing these solutions and those directly affected by them.  This article will focus specifically on factors that ought to be considered when looking into facial recognition solutions, and touch on technology and deployment variables, legal factors, and ethical considerations. What was once science fiction is now a reality, and the many forms of facial recognition technology are advancing quickly and becoming widely available to consumers. 

In its most simplistic form, facial recognition technology utilizes video and image processing to map data points across an unknown face in an attempt to match it against either a database of known faces or similar faces in pre-recorded video.  The number of system or personnel actions that may occur after a facial match is innumerable; however, this technology is widely used either operationally or for investigative purposes.  This may include as a credential to provide access to specific information (on a cell phone) or a specific area (electronic access control), as well as to detect and locate people of interest across a surveillance system in real time.  As facial recognition technologically expands, it is crucial for industry personnel, integrators, and end-users to understand the balance required when considering this technology for their organizational needs.

Improving security, investigations, & operational efficiency

Technological innovation will continue to advance both human efficiency as well as positively augment operations and security.  Commercial organizations using facial recognition may choose to enroll their employees into an internal database of faces, which would allow these employees to use their face as an access control credential.  Using a face eliminates the issues associated with conventional credentials like FOBs, cards, or PIN numbers such as loss or human error in entering a code, respectively.  Facial recognition as a credential also simultaneously provides video verification of that employee which is recorded alongside the access control logs.  There are a variety of challenges that an organization utilizing facial recognition in this way may encounter that will be reviewed in the following section, though it is vitally important that the image database be kept up to date and that a redundant credential is made available.

For security operators and law enforcement, facial recognition can quickly identify persons of interest involved in previously observed criminal activity or VIPs who may need to be actively monitored while visiting a site.  One of the major benefits of facial recognition is that, unlike other forensic or AI based analytics that classify attributes such as clothing, it may be used to classify and locate specific individuals very quickly within video spanning multiple days or even weeks.  The blending of operational efficiency and security is most easily observed when facial recognition is incorporated as a tool during a post-incident search of recorded video.  If not for facial recognition technology, security personnel would otherwise have to potentially spend hours scrubbing video to acquire the information needed for an investigation.  Provided a system has sufficient camera coverage and meets the requirements for the facial recognition software, the time scrubbing video may be cut from hours to minutes.  This is just one of many instances where this technology may be used to positively augment the human factor in security response and investigation.  Key takeaways are that in both examples the technology must be adequately supported by the infrastructure and used as a tool to improve efficiency.  Facial recognition software should not be considered a replacement for operator judgment, but rather as a way to positively augment their ability to swiftly respond to specific system triggers.

Challenges & considerations of deployment

There are a variety of factors that play into effective facial recognition, and while these will vary between solutions, there are major considerations that must be taken into account when employing this technology.  Conditions under which facial recognition is effective, the issue of privacy, and the legal variables depending on where the system is deployed should be consistently assessed during the research and procurement phases of a project.  System integrators and end users should diligently research and discuss the implications of these types of technologies with their legal counsel.

Ensuring that facial recognition software operates effectively relies on a variety of key system components and design factors.  To recognize a face, that face must be adequately observed, and observation is directly affected by a variety of controlled, independent, and dependent variables.  Given that facial recognition software is normally tied to video surveillance systems, it’s important to break down the multiple factors at play.

Controlled variables, or variables held constant, may include camera resolution and lens configuration, or more importantly pixels-on-target at the capture location, image processing and quality, camera mounting height and angle of incidence, and artificial lighting.  The better quality the image of a subject’s face, the greater the chance of an accurate classification when paired with a high-quality database of faces.  These controls are exponentially more important if the images that will be added to the facial recognition database are taken from video captured by the video surveillance system.

Another set of factors to consider are independent variables; specifically, what images are applied to the database the solution is using to match against.  These may be updated or changed by the organization but are not directly affected by outside variables during a classification.  When utilizing facial recognition for access control, employee ID photos are a great way to provide the system a well-lit, high-quality control image to work from.  In the cases of criminal activity where a quality image may be inaccessible, stills from the surveillance system may be used.  Of course, the lower quality these images, the more difficulty the system will have in accurately matching subjects to the database.  This may also result in an increased number of false positives.

The most difficult part of effective facial recognition is limiting dependent variables, or those elements of a scene or subject that cannot be controlled.  These include changes in appearance like aging or facial hair, uncontrolled lighting and contrast on the face caused either by changes in natural light throughout the day, inclement weather, or shadows caused by the addition of a ball cap.  Subjects must also be captured in specific areas or within a specific distance to a camera, and if that device is not dedicated to capturing faces, the subject may not be classified unless they look directly at the camera.  Luckily, tactics exist to limit these dependent variables.  For example, intercom devices that feature a surveillance camera may be used at facility entrances because they are generally mounted within parameters that provide a clear, front-facing shot of a subject’s face without becoming compromised by the aforementioned variables.

It's also becoming increasingly important for organizations to consider the privacy of the individuals being recorded by their systems.  In Europe, the General Data Protection Regulation (GDPR) provides strict guidelines for the use of personal data, of which facial recognition is a part.  For facial recognition technology, users must clearly state their intent for employing it, provide individuals recorded by the system complete control over their biometric data, and employ strict data protection measures to ensure database security.  Users outside of Europe should be prudent to provide the same level of care when it comes to the retention of individual facial data, including captured data as well as still images provided with active consent, as different state and local regulations may affect the degree to which this technology may be implemented legally.

Technology & implications: a balancing act 

The future of facial recognition in surveillance should be considered a balancing act.  The technology represents a powerful tool with both significant benefits and complex implications.  It is vitally important for industry personnel, system integrators, and end-users alike to understand its use cases, limitations, and possible future ramifications.  While it may offer apparent simplicity in solving complex organizational and security related problems, users of this technology should work diligently to find harmony between leveraging facial recognition, respecting privacy rights, and providing responsible stewardship of personal data.  

There are numerous providers of this technology, and it is therefore imperative for organizations to thoroughly evaluate and vet possible solutions.  Only through careful consideration and conscientious action can organizations benefit from facial recognition while also safeguarding the privacy of others and limiting their legal liabilities.  Current and prospective users should be constantly reassessing and taking a proactive approach when implementing facial recognition technology, and work to ensure legal compliance while upholding ethical integrity at every step.