Facial Recognition Bias: When Algorithms Mistake Innocent People

Blurs.co.uk
11 Min Read
Biometrics illustration, originally published via Pixabay; dedicated to the public domain under CC0 1.0. Source: Wikimedia Commons.
Summarize with AI

Facial recognition is often presented as a neutral technology: a camera sees a face, software compares it with a database and a match appears. But the system’s errors are not distributed evenly in every circumstance, and the consequences of a false match can be severe. Research has found demographic differences in the performance of some algorithms, while poor images, unsuitable thresholds and flawed procedures can turn an uncertain computer result into a real-world accusation.

Conceptual illustration representing biometric identification and facial recognition
Biometrics illustration, originally published via Pixabay; dedicated to the public domain under CC0 1.0. Source: Wikimedia Commons.

Two ways a system can be wrong

A false positive occurs when a system says two different people are likely to be the same person. In a one-to-many search, it may nominate an innocent person as the closest candidate even though the actual person is not in the database. A false negative occurs when the system fails to recognise two images of the same person.

These errors have different consequences. A false negative may mean that a wanted person is not identified or that a genuine user cannot unlock a device. A false positive can direct suspicion towards an innocent person, trigger a police stop or lead investigators to focus on the wrong individual. The seriousness depends on what happens after the algorithm produces its result.

There is no universal accuracy percentage that captures these risks. Results depend on the algorithm, the image quality, the database, the threshold and the task being performed. A system can perform well in controlled tests and still behave differently with low-resolution CCTV, unusual camera angles or images captured in poor light.

Why demographic differences matter

Studies by the US National Institute of Standards and Technology have found that demographic effects vary between algorithms and applications. Some systems have shown higher false-positive rates for particular demographic groups, while image quality can affect false-negative rates. NIST has identified factors such as under-exposure of darker-skinned faces, over-exposure of lighter-skinned faces and camera angles that are not suitable for people of different heights.

These findings do not mean every facial recognition product has the same bias or that every demographic group is affected in the same way. Performance differences vary across software versions, vendors, datasets and conditions. That is precisely why broad claims that a technology is either “biased” or “unbiased” are inadequate. Independent testing needs to examine the particular system that will actually be used.

Training data also matters. If a system has been developed using images that do not adequately represent the population it will encounter, its learned patterns may work less reliably for some groups. Differences in image capture can compound the problem. A camera positioned too high, poor lighting or a low-quality source image may affect matching even when the underlying algorithm is strong.

The UK policing concern

The issue is not limited to the United States. In 2025, bias was reported in the algorithm used for facial recognition searches within the UK Police National Database after testing by the National Physical Laboratory. In an update published on 18 August 2026, the Information Commissioner’s Office said the finding increased the likelihood of incorrect matches for people in some demographic groups.

The ICO said the Home Office and National Police Chiefs’ Council had put mitigations in place, including staff training, oversight reporting and equality impact assessments, alongside plans to replace the algorithm. The regulator said it was monitoring cases of detriment from incorrect matches and continuing to engage with the relevant bodies.

This is an important distinction between live facial recognition in public spaces and retrospective facial recognition searches. The Police National Database issue concerned an algorithm used for searches against stored images; it should not automatically be described as proof that every live camera deployment has the same performance problem. Different systems and uses require separate testing.

When a match becomes a wrongful accusation

Facial recognition is increasingly used as an investigative lead: an image from CCTV is compared with stored photographs and a candidate is returned. The candidate may then be investigated by officers. The danger arises when the result is treated as stronger evidence than it really is, or when investigators allow it to shape their interpretation of other evidence.

A match is not a fingerprint identification or a DNA result. It is a probabilistic comparison. A human reviewer should examine the original images, assess their quality, consider differences in angle and age, and look for independent evidence. A suspect should not be arrested or charged solely because a software system produced a candidate without proper corroboration and lawful grounds.

Research published in the journal Psychological Trauma has examined the contribution of facial recognition technology to wrongful arrests and psychological trauma. The issue is not merely technical. Being wrongly identified can involve detention, public embarrassment, loss of work, anxiety and lasting distrust of institutions. A later correction does not necessarily undo the harm.

Human review is necessary, but not a cure-all

Many systems include a human operator who reviews a possible match before action is taken. This is important, but human review does not automatically remove bias. An operator may over-trust a computer-generated result, especially when the software presents a candidate with a high confidence score or when the reviewer is under pressure to act quickly.

Good practice requires training that explains what the score means and what it does not mean. Reviewers should be able to reject a match, should not be encouraged to treat the highest-ranked candidate as necessarily correct, and should record the reasons for their decision. Where the image is poor or the match uncertain, the correct response may be to stop the process rather than force a conclusion.

Organisations should also test the complete operational process, not only the algorithm. This includes camera placement, image capture, watchlist composition, operator training, alert thresholds, escalation procedures and the way decisions are recorded. A technically sound model can still produce harmful outcomes if the surrounding process is weak.

What meaningful independent testing looks like

Testing should use representative datasets and report false-positive and false-negative rates separately. It should break down performance by relevant demographic characteristics and examine intersections where appropriate, while protecting the privacy of test participants. Tests should include realistic conditions rather than relying solely on ideal photographs.

Results should be published in a form that allows independent scrutiny. A vendor’s own marketing claims are not a substitute for independent evaluation. Systems should be retested after software updates, changes to cameras or databases, and changes in the way the technology is used. Thresholds should be set with the consequences of errors in mind, not simply to maximise the number of alerts.

Where the technology is used by police, oversight should include records of deployments, candidate matches, human decisions, false alerts, complaints and outcomes. If particular groups experience more errors or stops, authorities need to investigate and correct the cause. Public confidence cannot be built by withholding inconvenient performance data.

Beyond accuracy: the question of whether to use it

Even a system with excellent measured performance raises questions about consent, necessity and proportionality. A person walking through a shopping centre or train station may not have a realistic opportunity to avoid a camera. The face is a persistent identifier, and biometric data can be difficult to change once compromised.

Organisations should explain why facial recognition is needed, whether a less intrusive method could achieve the same purpose, what database is being searched, how long information is kept and how people can challenge a decision. For police deployments, clear legal authority and independent oversight are essential. For private companies, data-protection duties and transparency remain important even when the stated purpose is loss prevention or security.

Facial recognition can be useful, but its results must be treated as fallible evidence rather than objective fact. The most important safeguard is a culture in which an algorithm is allowed to be wrong, its errors are measured openly and people are not made to bear the consequences of an unverified computer guess.

Sources and further reading

Share This Article
Leave a Comment

Leave a Reply