PII Detection Benchmark

Measuring what matters: not just whether PII is found, but whether your detector understands context.

1,113 Samples
4 Systems
4 Categories
69.1% Best CSS Score

Overall Results

F1 scores across all four benchmark categories. Higher is better.

System Overall F1 Standard Ambiguous CSS Adversarial

Context Sensitivity Score

Can your detector classify the same string differently based on context?

Document A

"Jordan presented the quarterly results to the board."

PERSON
vs
Document B

"The shipment arrived in Jordan yesterday."

LOCATION

Same string, different entity types. A system that classifies both correctly scores +1. Both wrong = 0.

Per-Entity F1

How each system performs on individual entity types.

System Strengths

Radar chart showing each system's strengths across categories.

Latency

Per-document server-side processing time. AmbientMeta times reflect actual detection processing, not network round-trip.

System p50 (ms) p95 (ms) p99 (ms) Throughput

Run It Yourself

Clone, install, run. Reproducible results in under 5 minutes.

git clone https://github.com/ambientmeta/ambientmeta-pii-detection-benchmark
cd ambientmeta-pii-detection-benchmark
pip install -e .
python -m spacy download en_core_web_lg
python run_benchmark.py

The benchmark runs without an AmbientMeta API key. Set AMBIENTMETA_API_KEY to include AmbientMeta in the comparison.

Get a Free API Key View on GitHub