Engineering, production, governance, cost, communication · insight-1282
Test for proxies of a sensitive attribute instead of relying on having dropped it
- Evidence type
- practitioner heuristic
- Difficulty
- medium
- Roles
- Data scientist, ML engineer, Applied AI
- Industries
- insurance, banking, public-sector
Summary: situation, action, reason, boundary
Situation
Compliance asks for gender and age to be removed from a pricing model's features, and the model is then described as neutral (hypothetical).
Action
Keep the attribute in an evaluation-only table, try to predict it from the model's features as a proxy test, and compare outcomes and error rates by group. If the proxy test is strong, review the contributing features and the decision, not just the column list.
Reason
Many weakly informative features together can reveal a sensitive attribute, so a model trained without it can learn an equivalent encoding.
Boundary
Pricing, underwriting, hiring and eligibility models trained on wide behavioural or geographic features.
Symptom or risk
Postcode, occupation and product history can jointly encode the removed attribute, so the model reproduces the same disparity while the documentation says the attribute was not used.
Exceptions
Where law forbids even evaluation-time use of the attribute, use approved surrogates under legal guidance.
Trade-offs
Holding sensitive attributes adds a protection burden; restrict access and purpose.
Pitfalls
Reporting that an attribute was excluded as if it were evidence about group outcomes.
Lessons and concepts
Claims and their sources
The Barocas, Hardt and Narayanan text argues that fairness through unawareness can be ineffective because redundant encodings let a model recover a removed sensitive attribute from other features.