Skip to content
ML Academy

project-050 · LLM and GenAI · finance

PII redaction pipeline evaluation before documents cross an LLM trust boundary

A financial-services team wants to send customer documents to an LLM summariser hosted outside its trust boundary, but only after personal data is redacted. Which redaction pipeline - regular expressions with validation, an open-source analyzer such as Presidio, or a combination - leaks the fewest high-risk identifiers while keeping enough text for the summary to remain useful?

Licence
Apache 2.0 (dataset card); all PII is synthetic
Reference run
none

At a glance

  • Data

    gretelai/synthetic_pii_finance_multilingual, English documents (synthetic financial documents with PII spans) (public) · English test file 2,891 documents, mean about 1,410 characters; card lists 29 PII types (card record counts are inconsistent between metadata and prose)

    Details on data

  • Baseline

    Regular expressions with validation for structured identifiers: e-mail, phone formats, IBAN with checksum, payment card numbers with the Luhn check, routing and account…

    Details on baseline

  • Validation

    Develop patterns, recognizers and thresholds on the English train sample; evaluate once on the English test file.

    Details on validation

  • Metric

    Span-level recall and precision per PII type (exact and overlap), document-level leak rate for high-risk types, over-redaction rate of non-PII tokens, latency per…

    Details on metric

Success criterion

Pre-set rule: a pipeline is acceptable only if, on the held-out documents, recall for every high-risk PII type in the team's policy table is at least 0.98 at the span level and the document-level leak rate (any unredacted high-risk span) is at most 2%, with over-redaction of non-PII tokens reported. If no pipeline meets this, the finding is that documents must not cross the boundary yet, which is a legitimate outcome.

Context

Day 53 covers privacy leakage and access boundaries; Day 55 asks for a defensible production design. Redaction is a detection problem with asymmetric costs: a missed account number is worse than an over-redacted date. The Gretel dataset gives synthetic financial documents with labelled PII character spans, so span-level recall can be measured without exposing anyone's data. Using an external LLM to find PII would send the PII across the boundary it is meant to protect, so model-based detection is only considered with a model running inside the boundary.

Data

Dataset
gretelai/synthetic_pii_finance_multilingual, English documents (synthetic financial documents with PII spans) (public)
Licence
Apache 2.0 (dataset card); all PII is synthetic
Size
English test file 2,891 documents, mean about 1,410 characters; card lists 29 PII types (card record counts are inconsistent between metadata and prose)
Verification
download verified

Access

Hugging Face parquet (no login, not gated). Use the English files: develop on a sample of the English train file and evaluate on the English test file (2,891 documents, download-verified). Hash the files. Optional transfer set: 30 learner-written synthetic airline passenger messages with synthetic PII (passport-style numbers, booking references), labelled synthetic.

Grain

One document with a JSON list of PII spans (start, end, label); evaluation at span level and document level.

ColumnMeaning
generated_textSynthetic document text
pii_spansJSON list of {start, end, label} character spans
document_type / domainDocument format (e.g. loan agreement, statement) and domain (finance)
languageLanguage of the document; this project uses English only
policy_riskDerived: high / medium / not-PII per label from the team's policy table

EDA and data-quality rules

  • Build a policy table mapping each dataset label to high risk, medium risk or not treated as PII (the dataset labels 'company' and 'date' as PII; your policy decides), and justify each mapping.

  • Validate spans: offsets inside the text, no overlapping spans with different labels without a rule, and a spot check of 30 spans by eye.

  • Count spans per label in train sample and test; types with fewer than 30 test spans get intervals, not point estimates.

  • Check for labelled identifiers that are syntactically invalid (e.g. card numbers failing a checksum), because checksum-based detectors will miss them by design.

Baseline

Regular expressions with validation for structured identifiers: e-mail, phone formats, IBAN with checksum, payment card numbers with the Luhn check, routing and account number patterns, dates. Each pattern is unit-tested. This is the floor most teams start from.

Features

Span matching rules fixed in advance: exact match, and overlap match (any character overlap with the same policy class) as the primary leak metric, because a partially redacted account number may still leak. Redaction replaces spans with typed placeholders ([ACCOUNT_1]) so downstream summaries stay readable and consistent within a document.

Candidate methods

  • Presidio analyzer with default recognizers plus custom recognizers for policy types

    Combines patterns, checksums, context words and an NLP model for names and addresses; needs a spaCy model download.

  • Union of tuned regex and Presidio with an allow-list

    Maximises recall for high-risk types; measure the over-redaction cost.

  • Local model-based detector (optional)

    May catch free-form identifiers patterns miss; only acceptable if the model runs inside the trust boundary, and must be logged and evaluated the same way.

Validation design

Develop patterns, recognizers and thresholds on the English train sample; evaluate once on the English test file. Bootstrap over documents for recall and leak-rate intervals. Report the optional airline transfer set separately as an out-of-domain check.

Leakage checks

Do not write patterns by reading test documents; freeze the pipeline before scoring test. Logs of the pipeline must not store unredacted text.

Bias and representativeness

Synthetic text is cleaner and more regular than real documents, so results are optimistic; names in synthetic data may under-represent some naming conventions, which affects name recall. Report per-type recall and test the transfer set.

Robustness checks

Perturbations: identifiers split across line breaks, spaces inside card numbers, OCR-style character swaps, identifiers in tables. Measure recall under each.

Technical metrics

Span-level recall and precision per PII type (exact and overlap), document-level leak rate for high-risk types, over-redaction rate of non-PII tokens, latency per document.

Business interpretation

Documents that can cross the boundary safely, residual leak exposure per 10,000 documents (with interval), and the usefulness of redacted text for the summariser (spot-checked).

Error slices

  • PII type, especially names, addresses and free-form identifiers vs checksum-validated numbers.

  • Document type.

  • Perturbation type (line breaks, spacing, OCR noise).

Implementation steps

  • Download and hash the English files; build the policy table; validate spans.

  • Implement and unit-test regex+validation detectors; implement the span scorer with exact and overlap rules.

  • Configure Presidio (record its version and the spaCy model version) with custom recognizers; tune on the train sample.

  • Evaluate all pipelines once on the test file; compute per-type recall, leak rate and over-redaction with intervals; run perturbation tests.

  • Write the boundary decision and the residual-risk statement.

Reproduction

Learner-authored; for example uv run python -m redact.prepare --lang English, uv run python -m redact.run --pipeline regex and --pipeline presidio. CPU-only; Presidio needs its package and a spaCy model download, both pinned and recorded. No external API is called with document text.

Limitations

  • Synthetic documents make detection look easier than on real customer files.

  • The policy table is a team decision; a different mapping changes every headline number.

  • Span recall does not cover indirect identification (combinations of harmless fields), which needs a separate privacy review.

  • Span labels were produced by an LLM generator plus an NER labeller and only spot-checked by the provider (dataset card), so a few labels are missing or wrong; recall against them is approximate, and your own 30-span check should be reported.

Production and monitoring

Run redaction inside the trust boundary, fail closed (if the redactor errors, nothing is sent), log only span counts and types, and sample redacted outputs for audit. Monitor per-type detection counts for sudden drops (a format change can silently disable a pattern). Keep a re-identification map only if the business needs it, under access control and retention limits.

Interview questions

  • Why measure leak rate at the document level as well as recall at the span level?

  • Would you use a hosted LLM to detect PII before sending text to a hosted LLM? Explain the boundary problem.

  • How do you decide whether dates or company names count as PII for this use case?

Postmortem and alternatives

If names and addresses drive the leak rate, discuss whether the use case needs full documents at all; sending only extracted, non-personal fields to the summariser may remove most of the risk with less detection work.

Rubric3 criteria

CriterionMeetsExceeds
Policy and metric designPolicy table with justified mappings; exact and overlap span metrics; document-level leak rate.Shows how the policy mapping changes conclusions.
Pipeline comparisonRegex baseline and at least one candidate evaluated once on frozen test data with intervals.Perturbation and transfer results explain where each pipeline fails.
Boundary decisionClear go/no-go with residual risk.Proposes a data-minimisation alternative and its effect on risk.

References5