Engineering, production, governance, cost, communication · insight-0261

Enforce backward-compatible schema changes and upgrade consumers first

Situation
An event producer wants to add a field and drop an old one in the stream your features read from.
Action
Register schemas with a compatibility rule (backward by default), make new fields optional with defaults, remove only optional or defaulted fields, and roll out consumer changes before producers write the new schema.
Reason
Under backward compatibility, consumers using the new schema can read data written with older ones, which requires consumers to upgrade first.
Boundary
Event streams, Avro or Protobuf contracts, feature tables shared across teams.

Symptom or risk

Incompatible changes break deserialisation or silently fill fields with nulls in feature pipelines.

Full or transitive compatibility is safer when consumers replay old history.

Strict rules slow schema clean-up; old fields linger.

A field that keeps its name and type but changes meaning passes every compatibility check.

Confluent Schema Registry's default compatibility is BACKWARD, which allows deleting fields and adding optional fields and requires upgrading consumers before producers.
Schema Evolution and Compatibility for Schema Registry (Confluent documentation)