Face Matching: Comparing a Selfie to an ID Without Keeping Either

Face matching answers one narrow question at onboarding: is the person holding the phone the same person shown on the identity document? It is a 1:1 comparison, not a 1:N search against a database. That distinction matters for both accuracy and privacy, and it is frequently misunderstood by teams who conflate face matching with facial recognition surveillance.

What a 1:1 comparison actually does

The pipeline takes two inputs: the portrait extracted from the ID (often the JPEG in the document's chip or a cropped photo page) and a live capture of the subject's face. Each image is converted into a numerical embedding — typically a vector of 128 to 512 floating-point values — by a neural network. The system computes a distance between the two vectors. A small distance suggests the same person; a large distance suggests a mismatch.

The raw pixels are not compared directly. This is why lighting, pose, and camera quality degrade gracefully rather than breaking outright: the embedding tolerates variation the model was trained to ignore. It also means face matching should always be paired with liveness detection. Matching alone cannot tell a live face from a printed photo held up to the ID; that is a separate control addressing presentation and injection attacks.

Reading the error rates honestly

Two metrics govern tuning. The False Match Rate (FMR) is how often two different people are accepted as the same. The False Non-Match Rate (FNMR) is how often the same person is rejected. They trade off against each other via the decision threshold: tighten it to cut fraud and you reject more legitimate users; loosen it and impostors slip through.

• A common operating point for onboarding is an FMR near 1 in 10,000, with FNMR in the low single-digit percentages depending on image quality.
• NIST's ongoing FRVT 1:1 evaluations publish vendor-by-vendor numbers and document demographic differentials across age, sex, and skin tone.
• Those differentials are real. A single global threshold can produce uneven FNMR across groups, so measure rejection rates per segment rather than assuming a headline figure applies uniformly.

Set thresholds from your own population and risk appetite, and route borderline scores to human review or a step-up rather than auto-rejecting at the margin.

Matching without retaining biometrics

Biometric data is a special category under the GDPR and carries statutory obligations under laws such as Illinois BIPA. The engineering goal is to extract the verification decision while holding the underlying data for as little time as possible.

• Compute, then discard. The match score is the durable artifact. Once the comparison is logged, the selfie and extracted portrait can be deleted rather than archived by default.
• Template protection. If an embedding must persist, apply cancelable-biometric transforms or keep it encrypted with keys separated from the record, so a leaked vector cannot be reused elsewhere.
• Ephemeral processing. Run the match in memory and avoid writing raw frames to persistent storage. Keep an auditable record of the decision, model version, and threshold — not the face itself.
• Explicit consent and retention windows. Capture consent before collection and define a deletion schedule measured in days, not indefinitely.

Retaining a score with a deletion timestamp preserves your audit trail while shrinking the blast radius of any incident.

Running it inside a chat flow

Chat-based verification changes the capture surface without changing the math. In Telegram or WhatsApp, the document image and the live selfie arrive as messages, the embeddings are computed server-side, and only the pass or fail — plus the metadata a reviewer needs — is written back to the case. The media itself never has to linger in the conversation history once processing completes.

Configure the threshold, the review queue for borderline scores, and the retention window to match your jurisdiction and risk profile. This is general information, not legal advice; confirm biometric handling requirements with counsel for each market you operate in before going live.