Sum of Group Error Differences: A Critical Examination of Bias Evaluation in Biometric Verification and a Dual-Metric Measure
arxiv(2024)
摘要
Biometric Verification (BV) systems often exhibit accuracy disparities across
different demographic groups, leading to biases in BV applications. Assessing
and quantifying these biases is essential for ensuring the fairness of BV
systems. However, existing bias evaluation metrics in BV have limitations, such
as focusing exclusively on match or non-match error rates, overlooking bias on
demographic groups with performance levels falling between the best and worst
performance levels, and neglecting the magnitude of the bias present.
This paper presents an in-depth analysis of the limitations of current bias
evaluation metrics in BV and, through experimental analysis, demonstrates their
contextual suitability, merits, and limitations. Additionally, it introduces a
novel general-purpose bias evaluation measure for BV, the “Sum of Group Error
Differences (SEDG)”. Our experimental results on controlled synthetic datasets
demonstrate the effectiveness of demographic bias quantification when using
existing metrics and our own proposed measure. We discuss the applicability of
the bias evaluation metrics in a set of simulated demographic bias scenarios
and provide scenario-based metric recommendations. Our code is publicly
available under .
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要