As biometric technologies become increasingly common in applications ranging from border security and law enforcement to digital identity verification, ensuring they perform fairly across the populations they serve has become a critical priority.
Facial recognition systems are often evaluated using large datasets, but the size of a dataset alone doesn’t tell us whether a technology will perform equitably across a diverse population. Meaningful evaluation requires representative testing, defined demographic comparisons and expected outcomes that allow differences in performance to be identified and understood.
This is particularly important in countries with diverse populations. Aotearoa New Zealand is a superdiverse country that reflects global demographics, with a population representing more than 200 distinct birthplaces and over 150 spoken languages. Its population includes Māori, Pacific cultures and diverse immigrant communities. That diversity matters when evaluating technologies that need to work equitably across all communities.
Beyond a single measure of accuracy
A biometric system can demonstrate strong overall performance while still performing differently across demographic groups.
This is why representative biometric testing needs to look beyond an aggregate accuracy figure. A robust testing framework should assess defined groups within a representative population against appropriate expected outcomes. Comparing results across these groups can provide greater insight into where performance is consistent and where differences may exist.
This approach helps organizations develop a more complete understanding of how a biometric technology is likely to perform in the population it is intended to serve.
It can also support more informed decisions when governments, organizations and other stakeholders are evaluating biometric technologies for use in their communities.
The importance of independent evaluation
Independent testing adds another important dimension.
Technology providers may conduct extensive testing as part of developing and improving their systems, but independent evaluation can provide an additional level of transparency and comparability. A consistent testing framework allows different technologies to be assessed against the same types of representative populations and expected outcomes.
This can help organizations better understand the strengths and limitations of the technologies they are considering, while providing greater transparency for the communities affected by their use.
For biometric technologies to earn public trust, people need confidence that systems have been properly evaluated and that potential differences in performance have not been overlooked.
Supporting representative testing in Aotearoa New Zealand
Digital Identity New Zealand (DINZ), part of Tech New Zealand, is helping advance this work through its Biometrics Special Interest Group and the Kiwi Faces initiative.
Kiwi Faces is helping establish a framework for independent, representative testing of facial recognition technologies in the context of Aotearoa New Zealand’s population. The initiative recognizes that a testing approach designed around a generic or non-representative population may not provide a complete picture of how a technology will perform across Aotearoa New Zealand’s diverse communities.
A locally relevant testing framework can help identify differences in performance, support more informed technology decisions and contribute to greater confidence in the responsible use of biometrics.
Attain Insight is pleased to be a member of DINZ and to be participating in their Biometrics Special Interest Group, supporting the Kiwi Faces initiative. We believe collaboration between industry, government and researchers is essential to developing practical approaches to biometric evaluation that reflect the populations these technologies are intended to serve.
As biometric adoption continues to grow, representative and rigorous testing will be an increasingly important part of responsible deployment. The goal is to understand how well it works, for whom, and under what conditions.
That is an important foundation for building biometric technologies that can be trusted by the communities they serve.
