Hi Kafka Community, I would like to start a discussion on KIP-1253: Add TopologyValidator Utility for Kafka Streams Topology Compatibility
When a topology evolves, especially with unnamed internal stores and topics, adding a new source near the beginning can shift auto-generated identifiers (for example, KSTREAM-AGGREGATE-STATE-STORE-0000000001 to ...0000000002). This silently changes internal changelog and repartition topic names. In practice, simply replaying the source topics is often insufficient to reconstruct the previous state, and the new state store may never converge to what the old one contained. To help catch these cases earlier, the KIP introduces an offline validator in streams-test-utils. It compares two topology.describe().toString() snapshots, parses them into a simplified topology context (internal changelog, repartition, subscription/response topics, plus source and sink topics), and applies pluggable rules to detect changes. The validator returns structured TopologyChange entries; tests and CI pipelines can then decide how to react (for example, fail on internal topic changes and only log warnings for others). The tool is fully opt-in and is meant as a lightweight guardrail, not a complete compatibility checker. You can find the full details here: - KIP wiki: https://cwiki.apache.org/confluence/display/KAFKA/KIP-1253%3A+Add+TopologyValidator+Utility+for+Kafka+Streams+Topology+Compatibility - JIRA: https://issues.apache.org/jira/browse/KAFKA-19935 I would appreciate feedback on the overall approach, the choice to place this in streams-test-utils, and the initial set of rules and change types. Best regards, Sanghyeok An
