Buyer Guide: Evaluating Liveness Detection Vendors by Evidence, Not by Claims
A working method for buying a liveness component: what a claimed test level rests on, a phased pilot protocol, an acceptance matrix, and an evidence and change-control schedule.
Every liveness vendor can say that the product detects spoofs, deepfakes and masks. Very few sentences in a sales deck can be checked by the person who has to sign the contract. This guide is for that person, and it is a working document: a way to read a claimed test level, a phased pilot protocol, an acceptance matrix to fill in before the pilot starts, and a schedule that keeps the evidence current after the contract is signed. It does not name or rank any vendor.
The background is in three existing articles, and this guide does not repeat them. How the test levels and error rates work is in the article on what presentation attack detection testing measures. Where each kind of attack and defence sits is in the article on presentation attacks, deepfakes and injection attacks. The checklist for the whole provider, including security and privacy questions, is in the buyer guide on security and privacy.
What you are actually buying
Regulators do not define liveness as one technique. The European Banking Authority's guidelines on remote customer onboarding say that liveness detection verifications may include procedures where a specific action from the customer is required to show presence in the communication session, or may be based on the analysis of the received data without a specific action by the customer. In everyday language the first is called active and the second passive, and the guidelines allow either form. A liveness component bundles several things, and a vendor may have evidence for some and not for others.
- A capture routine that guides the user, controls lighting and framing, and decides when to stop.
- A presentation attack detector that judges whether a live person is in front of the camera.
- An injection defence that judges whether the video came from the camera at all.
- A comparison step that matches the face to the document portrait or an earlier enrolment.
- A decision policy with thresholds, retries and a route to a human or to an alternative.
The acceptance matrix below has one row group for each of these, so ask the vendor which it sells, which it buys from someone else, and which parts any test evidence covers.
Reading a claimed level
"Tested at a level" is the sentence that does the most work in a sales deck and says the least until five facts are attached. The ladder of evidence puts a statement at the bottom, the vendor's own test above it, an independent laboratory report above that, your own pilot above that, and monitoring in production at the top. The UK trust framework lets biometric testing be external, by an independent laboratory recognised under the international laboratory arrangement, or internal, following a recognised methodology that meets international standards. The EBA guidelines say institutions should monitor the ongoing adequacy and reliability of the solution by one or more of quality assurance testing, automated critical alerts, regular automated quality reports, sample testing and manual reviews.
The five facts behind a level
- The standard and its edition. Part 3 of the 30107 series covers the testing and reporting of presentation attack detection, and the 2023 edition replaced the 2017 edition, which is withdrawn. Ask which edition the laboratory worked to, because a report to a withdrawn edition describes a different method.
- The scope the standard has. It establishes principles and methods for assessing the performance of presentation attack detection mechanisms, the reporting of test results and a classification of known attack types. It leaves out the standardisation of specific mechanisms and the overall system-level security or vulnerability assessment, and it considers only attacks that take place at the capture device during presentation. A level under this standard is therefore a statement about presentation attacks at the capture device, not an approval of a product and not a statement about injection.
- The laboratory and its protocol. The laboratory whose public explanation is used in the testing article says that a conforming result is not by itself the same as a formal approval of the vendor's product. Ask for the protocol document, the attack instrument list, the number of subjects and the pass criteria as the laboratory applied them.
- The thing tested. Which version of the product does the report name? Which device configuration was tested? Which mechanism was under test: the attack detector alone, the capture with quality checks, or the full system?
- The error rates and the limit on genuine users. Does the report state both the attack error rate and the genuine-user error rate, and the limit applied to genuine users?
The UK framework sets out what a biometric testing report must let a reader decide, and it works as a checklist for any report you receive. It must show whether the number of false matches and false non-matches suits the provider's security and usability needs, whether the attack presentation classification error rate and the bona fide presentation classification error rate suit its needs, and whether performance is consistent across demographics. It must state the test environment and make clear how that environment relates to the service, and the framework gives the example that test reports about a building entry system are not suitable evidence for a remote onboarding system on a mobile device.
Injection needs its own evidence
A presentation attack test does not answer the injection question. The European technical specification CEN/TS 18099:2024, titled biometric data injection attack detection, states that presentation attack testing is out of scope, and that no presentation attack testing is performed by the laboratory to be compliant with it. Its stated purpose includes guidance on creating a test plan for the evaluation of an injection attack detection system. The standards body's page gives the specification as valid from 16 December 2024. When ENISA wrote its report in March 2024, it said that unlike presentation attack detection no globally accepted standards existed for injection attack detection, and that a European standard was not expected before the end of 2024. So the buyer's question is simple: has the product been evaluated for injection attacks, against which methods, and by whom? ENISA's own conclusion is that no silver bullet exists and that the optimal solution is a multi-layered incorporation of the various control types, so a vendor that offers a single layer and calls it complete has not read this text. Independent research points the same way: a study presented at a 2022 security symposium evaluated facial liveness verification services offered as cloud interfaces and found that most of the interfaces it examined did not use anti-deepfake detection, and that where such defences existed their effectiveness was concerning. That study is dated and the market has moved, so it is a reminder to test the claim against current methods and not a verdict on any product.
A worked pilot protocol
The EBA guidelines give a model for a pilot. They say an institution adopting a remote onboarding solution should carry out a pre-implementation assessment, and list at least: an assessment of adequacy regarding the completeness and accuracy of the data and documents to be collected; an assessment of the impact on business-wide risks; mitigating measures; tests to assess fraud risks including impersonation fraud risks and other ICT and security risks; and end-to-end testing of the functioning of the solution targeting the customers, products and services in scope. Paragraph 14 of the guidelines is the source. The protocol below turns that list into phases with outputs. It is the buyer's suggestion and not a requirement from any of the texts.
| Phase | Purpose | What runs | Output | Who signs off |
|---|---|---|---|---|
| Zero: preconditions | Fix the rules before any data exists | Threat list, metric definitions, device list, data protection basis, frozen version and thresholds | A written pilot charter | Product owner and security lead |
| One: shadow mode | Measure on real traffic without affecting customers | The vendor's decision is recorded; the current control still decides | Completion, abort, retry and handover rates by device class; disagreement list against the current control | Fraud lead |
| Two: staged attacks | Measure detection against attacks you chose | Your security team runs the attack set against each device class, with the vendor informed | An attack results table, one row per attack and device class | Security lead |
| Three: enforced canary | Test the real flow with a small share of traffic | The vendor's decision enforced, with human review as the fallback | Review queue load, false rejection complaints, later-found errors | Product owner |
| Four: decision | Grade against the acceptance matrix | All outputs read against the pre-agreed pass conditions | A recommendation: adopt, adopt with conditions, or reject | Steering group |
Phase zero: the charter
Write the pilot charter before the vendor's sandbox is opened, because numbers chosen after the results are rationalisations. The charter names the attack families, the metrics with their definitions, the device classes, the sample sizes, the thresholds in use and who may change them, and the version of the product under test. Take the attack families from the ENISA report on remote identity proofing, which describes presentation attacks by the instrument used and lists biometric artefacts such as masks, a photograph or digital image of the real user, and the replay of a fake or recorded video. Add the injection methods the same report describes: virtual cameras, device emulators and function hooking. Agree the sample size as a number of attempts per attack and device class, and write down that a result of no failures in a small sample bounds the rate only loosely, so the vendor reports the confidence bound with the observed rate and not the observed rate alone.
Phase one: shadow mode
In shadow mode the vendor's component sees the live capture and returns a decision, and the decision is stored but not acted on. Compare it with what your current control decided, session by session. Every disagreement goes on a list that a person reviews, and the reviewer records which side was right. Report the rates by device class, lighting condition and user group, because an average over all sessions hides the device that fails. The Information Commissioner's Office states that if you use a biometric recognition system you are processing special category biometric data, so the data protection basis for the shadow phase must exist before it starts. It also says that decisions of such a system are statistically informed estimates, and that you should not interpret the outcome of a comparison as a matter of fact.
Phase two: staged attacks
Run the attack set from the charter against every device class, with the vendor informed and your security team designing the cases. For each attack record the device, the lighting, the instrument, how many attempts were made, how many passed, and whether a human reviewer would have caught the rest. Cover the injection methods in a separate block, because they need a different test setup and a different person to run them. Keep a log of the attack material and delete it on a stated date, since staged attacks create biometric data of the testers as well.
Phase three: enforced canary
Turn enforcement on for a small share of traffic, chosen at random and not by segment, and keep the human review route open. Check that the process does not continue when technical shortcomings or unexpected connection interruptions occur, as the EBA guidelines expect. Track the decisions that were later found wrong, the complaints from genuine users and the share of cases that went to a person. A canary that never produces a false rejection complaint has probably not reached enough users, or the complaints are not being collected.
The acceptance matrix
Fill in the third column before phase one. A row that has no pass condition at the start cannot fail at the end.
| Row | Evidence | Pass condition (set in the charter) | If it fails |
|---|---|---|---|
| Presentation attacks, each family separately | Laboratory report for the production version; phase two results | Every family at or below its agreed attack acceptance rate, on each device class in scope | Reject, or accept only with a compensating control named in writing |
| Injection attacks | Evaluation against the injection test plan, or phase two injection block | Every staged method detected, or covered by another layer you operate | Reject for any flow where injection is a stated threat |
| Genuine users, each device class | Phase one and phase three rates | Completion and genuine rejection rates inside the agreed band for every class | Limit the flow to classes that pass, with an alternative for the rest |
| Fairness across user groups | Report showing consistency across demographics; phase one rates by group | Differences between groups disclosed and inside the agreed band | Require a vendor remediation plan before go-live |
| Alternatives and access | Documented non-biometric route; accessibility test | A person who cannot pass has a route to an outcome | Block go-live until the route exists |
| Data terms | Contract and data protection impact assessment | Retention, training use, location and access match the assessment | Renegotiate; no go-live on a mismatch |
| Version and evidence match | Report, contract, production build | The version in production is the version the evidence names | Re-test or pin the older version |
Three rows rest on texts you can cite. On fairness, the UK framework requires a provider to be able to show in the testing report whether performance is consistent across demographics. On alternatives, for identity profiles up to and including medium the framework requires a provider that offers a biometric check and a non-biometric check to offer the non-biometric alternative at the same time. The Reserve Bank of India's direction states that the liveness check shall not result in exclusion of persons with special needs. On the version, a laboratory letter names a version and a device configuration, so a model change in production moves the vendor outside the evidence.
Grading
| Grade | What it looks like |
|---|---|
| Strong | A report you can read that matches the version, device and context you will use; injection covered by a stated plan; a pilot you can reproduce; clear data terms |
| Conditional | A report for a different version or context, or injection answered with a promise, and a named compensating control for each gap |
| Weak | Statements without documents, a test by the vendor only, or no answer on user groups and alternatives |
A weak answer is not always disqualifying. It tells you which risks you must cover with other controls, such as a human review step or a lower transaction limit.
An evidence and change-control schedule
The evidence that justified the purchase has a shelf life. The EBA guidelines name the triggers for ad hoc reviews after go-live: changes in risk exposure, deficiencies detected in monitoring, audit or supervisory activity, a perceived increase in fraud attempts, and changes to the legal or regulatory framework. The schedule below adds the vendor-side triggers, and each row says what the vendor delivers and what the buyer does.
| Trigger | The vendor delivers | The buyer does |
|---|---|---|
| A model, threshold or capture change | Written notice before it reaches production, the new version identifier, and a statement of what the change affects | Re-runs the phase two attack set and the phase one rates on a sample before accepting the change |
| A new application or SDK version | Release notes and the list of device classes affected | Checks the device matrix and the injection controls again |
| A new device class or market | Evidence for the new class, or a statement that none exists | Adds the class to the charter and runs phases one and two for it |
| A new attack technique reported by a regulator, researcher or your own fraud team | Its assessment of exposure and a date for a fix | Adds the technique to the attack set |
| A laboratory letter that ages out or a new edition of the standard | A new report to the current edition, or a dated plan | Marks the evidence as stale in the register until the report arrives |
| A detected forgery campaign or near miss | Prompt notice, with the indicators | Updates monitoring and the attack set; the Reserve Bank of India's direction expects institutions to upgrade technology based on detected, attempted and near-miss cases of forged identity |
| A change of subcontractor or data location | Prior notice and the new terms | Repeats the data protection assessment |
| A fixed review date | An annual statement of versions, devices and open defects | Reads it against the matrix and signs the register |
Keep the register in one place with four columns: the evidence item, the version it names, the date it expires or was last checked, and the person who owns the check. The same terms belong in the contract, as the buyer's suggestions: version pinning with notice, a duty to supply new evidence on the triggers above, a right to run controlled attack tests against your own tenant, incident duties, data terms that fix retention, training use and location, and a duty to name any subcontractor that supplies part of the liveness component. On data, the UK framework requires a provider that captures biometric data to explain to users what will happen to it, including whether it is used for a face match or checked against multiple records, how long it will be retained, where it is stored and whether it will be used as training data for artificial neural networks. The article on retention and erasure of identity data shows how to align the answers with statutory retention.
A worked example, with invented vendors
This example is illustrative and describes no real supplier. Vendor A offers a laboratory letter for an earlier version, a passive design, and no injection evaluation. Vendor B offers no letter, an active design with a changing challenge, a written injection test plan, and agrees to the full pilot. On the matrix, Vendor A grades conditional on attacks, because the letter names a version that is not in production, and weak on injection, because the answer is a promise. Vendor B grades weak on the laboratory row, because it has no report at all; staged attacks in phase two test your attack set, but they do not replace a laboratory report and they say nothing about genuine users. The policy in this example is the matrix as written: a laboratory report for the production version is mandatory on the attack row, so neither vendor can pass that row today. Vendor A can close it by re-testing the production version or by pinning the evaluated one; Vendor B can close it only by obtaining a report. Both then go through every mandatory row in the same way: staged-attack results from phase two on the attack row, genuine-user measurements from phases one and three on the genuine-user rows, and the demographic, accessibility, contract and version rows on their own evidence. "Strong" is reserved for a vendor that meets every mandatory row; a buyer who decides to accept buyer-generated evidence instead of a report should write that change into the matrix before the pilot, not after it. The point of the protocol is that the result comes from the same attack set, the same device classes and the same pass conditions for both.
Where this sits in a verification flow
A flow built on NeuroVision KYC places the identity steps in one session: document, selfie, liveness and face match, with the scenario assembled in an editor. The liveness step is active: the person moves the face closer through two ovals. Deepfake and face-swap detection works inside the selfie and liveness steps. iBeta ready. This is the Company's own statement and is not a laboratory result.
To see how these steps look in your own scenario, contact the team.
Sources
All sources were fetched on 27 October 2026.
- Remote ID Proofing Good Practices, European Union Agency for Cybersecurity (ENISA), March 2024.
- Guidelines on the use of Remote Customer Onboarding Solutions, European Banking Authority.
- UK digital verification services trust framework (1.0), Office for Digital Identities and Attributes, GOV.UK.
- Biometric presentation attack detection, Part 3: Testing and reporting, standard page, International Organization for Standardization.
- CEN/TS 18099:2024, Biometric data injection attack detection, scope and general information, standards store of the Estonian standardisation body (evs.ee).
- Seeing is Living? Rethinking the Security of Facial Liveness Verification in the Deepfake Era, USENIX Security Symposium.
- Presentation attack detection test methodology and confirmation letters, explanation page of the testing laboratory.
- Biometric recognition, Information Commissioner's Office.
- Master Direction - Know Your Customer (KYC) Direction, 2016, Reserve Bank of India.