Skip to content
ML Academy

SQL, data modeling, data quality, EDA, visualization · insight-0042

Use NOT EXISTS instead of NOT IN when the subquery column can contain NULL

Evidence type
documented fact
Difficulty
medium
Roles
Data analyst, Data scientist
Industries
cross-industry

Summary: situation, action, reason, boundary

  1. Situation

    A query finds customers who never booked with WHERE customer_id NOT IN (SELECT customer_id FROM bookings), and returns zero rows.

  2. Action

    Write anti-joins as NOT EXISTS with a correlated condition, or LEFT JOIN and filter on the right key IS NULL. Add a NOT NULL constraint or filter in the subquery when you keep NOT IN.

  3. Reason

    Under three-valued logic, "x differs from every value" cannot be true if one value is unknown.

  4. Boundary

    Churn lists, "never purchased" segments, orphan checks and exclusion lists.

Symptom or risk

One NULL in the subquery makes NOT IN evaluate to NULL for every non-matching row, so the result is empty or wrong without an error.

Exceptions

Subqueries on columns that are declared NOT NULL behave as expected with NOT IN.

Trade-offs

NOT EXISTS is more verbose; query planners generally handle it well.

Pitfalls

Testing on clean development data with no NULLs, so the bug appears only in production.

Lessons and concepts

Claims and their sources