How to Read a Face Recognition Evaluation

A vendor quotes an accuracy figure for its face matching, or points to a place in a public evaluation table. Before that number can inform a purchase, you need to know what it measures, at which threshold, on which images, and what it leaves out. This article is for a security reviewer or a procurement lead and answers one question: what do false match rate, false non-match rate and a threshold mean, how does the US Face Recognition Technology Evaluation report them, and what should a buyer ask about the dataset behind a number?

Two errors and one dial

A one-to-one face verification system compares two images and produces a similarity score. Someone has to decide a threshold: scores at or above it count as "same person", scores below it count as "different people". Two kinds of error follow from that single decision.

The public evaluation page of the US National Institute of Standards and Technology defines both. The false non-match rate, FNMR, is the proportion of mated comparisons, meaning two images of the same person, that fall below a threshold set to achieve the false match rate specified. The false match rate, FMR, is the proportion of impostor comparisons, meaning images of different people, at or above that threshold.

Read that definition twice, because it contains a central idea of the whole subject: the two rates are linked through the threshold. You do not get an FNMR on its own. You get an FNMR at a stated FMR, because the evaluation first fixes the threshold that produces the stated FMR and then counts how many genuine pairs fall below it. A bare accuracy percentage with no FMR attached cannot be interpreted. An FNMR stated together with the FMR at which it was measured can be.

The ICO, the data protection regulator of the United Kingdom, explains the same trade-off in guidance for organisations: a threshold is the point at which a system considers the similarity to be statistically significant, a comparison resulting in acceptance means that the similarity has met the threshold, and the lower the threshold, the higher the chance that an accepted comparison does not actually relate to the same person. The regulator adds that decisions are statistically informed estimates and that some degree of error cannot be removed entirely.

What a regulated baseline looks like

For one specific use, the US guideline on identity proofing, Special Publication 800-63A-4, sets numbers. Where one-to-one comparison is used to verify an applicant against a claimed identity, the provider must meet a false match rate of 1:10,000 or better and a false non-match rate of 1:100 or better. The same text requires that performance for applicants of different demographic types be no more than 25 % worse than the performance for the overall population, measured with one fixed threshold, and that results be made publicly available. It gives an example: if the false non-match rate overall is 0.006, the rate for a demographic group cannot exceed 0.0075.

These figures are a floor set for a particular federal purpose. They show how a requirement can be written so that it is testable: two error rates, a stated ratio for each, and a rule for demographic spread.

How the evaluation is run

The Face Recognition Technology Evaluation, FRTE, has been running on an ongoing basis since February 2017. Developers submit algorithms behind a published programming interface at their own pace, and the algorithms are evaluated on a first-come, first-served basis; submissions come no more often than every four calendar months. The page states that participation carries no charge and is open to a global audience of developers. As of the participation statistics updated on 6 October 2026, the page lists 1483 algorithms from 455 unique developers across the years 2017 to 2026.

The goals section names a wide range of applications: visa images, deduplication of passports, recognition across photojournalism images and identification of child exploitation victims. The page says reports will cover accuracy, speed, storage and memory consumption, and resilience, and will report how performance depends on the properties of the images and the subjects.

Two features of the main table matter to a reader. First, the table shows FNMR for several different datasets, with algorithms ordered initially by their result on one dataset at one fixed FMR; a reader who sorts by another column sees a different order. Second, only one algorithm per developer appears, chosen by a mean of within-developer FNMR ranking taken over six columns. A single row is therefore a summary of a larger set of results, and which dataset is shown first is a presentation choice.

Datasets are not interchangeable

The page makes clear that results depend on the images. Some examples from its own text:

A change to the programming interface in February 2022 let algorithms produce templates from several faces in one image, and the page says this occurs in approximately 3% of border images and 7% of kiosk images.
Twins have their own table: the false match rate on twin probe images is the proportion of twin comparisons at or above a threshold set to achieve an FMR of 0.0001 on mugshot images.
A mugshot control is included to show how the algorithms perform when mated comparisons fall below a threshold set to the same FMR.

The ICO guidance makes the same point from the buyer's side: it asks organisations to be clear about the known or expected false acceptance and rejection rates of the system they intend to use, and to ask whether there are groups for whom the results are likely to be less accurate.

Demographic differentials

The evaluation publishes separate summaries of demographic variation. For false negatives, the page notes that they are strongly dependent on image quality, so poor photography can induce a demographic effect. It gives two examples: inadequate lighting or under-exposure of dark-skinned individuals, or over-exposure of fair-skinned subjects, and failure to adjust a camera for very tall or short individuals. For false positives, the page explains that variation occurs even with good image quality and arises when an algorithm produces similarity score distributions that are displaced for one demographic compared with another, which can occur due to under-representation of a demographic in the training data.

The summary tables report several measures per algorithm, including the most favourable and the least favourable value across groups, the maximum rate over the geometric mean rate, where the ideal result is 1 and indicates parity, and a Gini coefficient that quantifies how errors are concentrated in certain demographics. For the false match summary the threshold is fixed for each algorithm to give an FMR of 0.0003 overall.

Speed and size are separate columns

Accuracy tables are paired with a resources table that records the duration of template generation, finalization and search calls, measured on an unloaded server-class machine, and the size of the recognition models and libraries. The page also announces that starting on 30 November 2026 template generation must take under 2,000 milliseconds per image on a single core of a named server processor. For an integration team, these columns are as relevant as accuracy because they decide cost and latency.

Questions to put to any vendor number

Which FMR was the threshold set to, and what is the FNMR at that FMR?
Which dataset, and how do its images compare with your users' images: capture device, lighting, age of the portrait, document photos versus selfies?
Is the number from an independent evaluation, a vendor test, or both, and which algorithm version does it describe?
How does the result vary across age groups, sexes and regions of birth, and at what threshold?
What are the template generation time and model size on your hardware?
How will you measure the same two rates on your own sample before and after launch?

How NeuroVision fits

The face match step of the NeuroVision verification session compares the selfie with the portrait on the document in a one-to-one comparison, and includes foreign-face detection and an age check against the date of birth.

To plan a measurement on your own sample, contact the team.

Sources

All sources were fetched on 27 October 2026.

Face Recognition Technology Evaluation (FRTE) 1:1 Verification, US National Institute of Standards and Technology.
Digital Identity Guidelines, Special Publication 800-63A-4, Identity Proofing and Enrollment, US National Institute of Standards and Technology.
Biometric recognition, Information Commissioner's Office, United Kingdom.
How do we process biometric data lawfully?, Information Commissioner's Office, United Kingdom.