BTPBench is a Python library and command-line application for evaluating face-recognition systems and biometric template protection (BTP) algorithms. It provides reproducible workflows for identification, verification, irreversibility, key selection, diversity, and unlinkability.
The repository contains protocol definitions, but not the biometric datasets. You must obtain each dataset under its own terms and point the system configuration at your local copy.
BTPBench requires Python 3.12 and Git LFS. Install and initialize Git LFS before cloning so that the bundled model files are checked out correctly:
git lfs install
git clone https://gitlab.idiap.ch/primeaid/btpbench.git
cd btpbenchThe recommended installer is uv:
uv syncFor development tools, install the dev extra:
uv sync --extra devYou can also install the project with another PEP 517-compatible installer.
The IResNet and MediaPipe parameters are stored with Git LFS. EdgeFace and FaceNet fetch their upstream model parameters automatically on first use and therefore require network access at that point.
Everything needed to run these models is shipped with BTPBench or its installed dependencies, or fetched automatically on first use, so no extra setup actions are required. The following table covers the upstream implementations and pretrained parameters used by BTPBench. Implementation and checkpoint terms are listed separately where they differ.
| Model | Configuration key | Parameter source | Upstream license and weight terms |
|---|---|---|---|
| iResNet50 | iresnet50 |
Bundled pytorch-insightface checkpoint | Implementation: MIT; weights: non-commercial research use only |
| iResNet100 | iresnet100 |
Bundled pytorch-insightface checkpoint | Implementation: MIT; weights: non-commercial research use only |
| EdgeFaceBase | edgeface |
Downloaded EdgeFace-Base checkpoint | Model and weights: CC BY-NC-SA 4.0 |
| EdgeFaceXS | edgefacexs |
Downloaded EdgeFace-XS-GAMMA-06 checkpoint | Model and weights: CC BY-NC-SA 4.0 |
| FaceNet | facenet |
Downloaded facenet-pytorch VGGFace2 checkpoint, ported from David Sandberg's 20180402-114759 model | Implementations: MIT (PyTorch port) and MIT (TensorFlow source) |
| MediaPipe Face Detector (BlazeFace short-range) | detector: mediapipe |
Bundled MediaPipe BlazeFace short-range detector.tflite model |
Upstream implementation and BTPBench asset record: Apache-2.0 |
Follow the user-specific quickstart for one complete, step-by-step experiment. It measures baseline protected verification, selects inversion-resistant user keys, repeats security and recognition measurements, then compares random and selected keys for unlinkability and diversity. Every score, metric table, and plot is generated locally by the documented commands.
For a custom dataset or experiment, start from
config/system_config.yaml
and
config/experiment_config.yaml,
then use the detailed guides below. Every command has local help, for example:
uv run btpbench verification pipeline --helpProtocol CSVs map local images or videos to subject and template IDs and define the samples used by each evaluation. The protocol authoring guide documents the shared CSV schema, identification directory layout, image and video path rules, verification and unlinkability sample lists, irreversibility files, and a validation workflow for adding a new database.
| Experiment | Purpose | Guide |
|---|---|---|
| Identification | 1:N template matching and DIR analysis | Identification |
| Verification | Pairwise biometric comparison and DET analysis | Verification |
| Irreversibility | Resistance of protected templates to inversion attacks | Irreversibility |
| Key selection | User-specific and system-specific BTP key evaluation | Key selection |
| Diversity | Mutually non-matching protected-template sets | Diversity |
| Unlinkability | Linkability analysis across differently protected templates | Unlinkability |
| Standalone plots | Distribution, histogram, and PolyProtect visualizations | Plots |
See system configuration and experiment configuration for the complete YAML reference. The Python API guide covers direct library use.
Run the test suite and repository checks with:
uv run pytest
uv run pre-commit run --all-files
uv run reuse lintTo install the Git hooks locally, run uv run pre-commit install once.
Project-authored code and documentation are licensed under the
Research Only and Non Commercial License.
See
THIRDPARTY.md
for dependency licenses and the LICENSES/
directory for license texts.
Bundled and downloaded third-party components retain their upstream terms. Review the model-specific terms above before redistributing the package or using the weights.