Rhys, thanks for your enthusiasm! In your example, us-west.us-east.us-central.us-west.topic is an invalid "remote topic" name because us-west appears twice. MM2 will not replicate us-east.us-central.us-west.topic into us-west a second time, because the source topic already has us-west in the prefix. This is what I mean by "cycle detection" -- cyclical replication does not result in infinite recursion.
It's important to note that MM2 does NOT disallow these sort of cycles, it just knows how to deal with them properly. Also notice this is done at the topic level, not per record. The records don't need any special header or anything for this cycle detection mechanism to work. Thanks! Ryanne On Mon, Oct 15, 2018 at 3:40 PM McCaig, Rhys <rhys_mcc...@comcast.com> wrote: > Hi Ryanne, > > This KIP is fantastic. It provides a great vision for how MirrorMaker > should evolve in the Kafka project. > > I have a question on cycle detection - In a scenario where I have 3 > clusters replicating between each other, it seems it may be easy to > misconfigure the connectors if auto topic creation is turned on so that > records become replicated to increasingly longer topic names (until the > topic name limit is reached). Consider clusters us-west, us-central, > us-east: > > us-west: topic > us-central: us-west.topic > us-east: us-central.us-west.topic > us-west: us-east.us-central.us-west.topic > us-central: us-west.us-east.us-central.us-west.topic > > I’m not sure whether this scenario would actually justify implementing > additional measures to avoid such a configuration, rather than ensuring > that the documentation is clear on how to avoid such scenarios - would be > good to hear what others think on this. > > Excited to see the discussion on this one. > > Rhys > > > On Oct 15, 2018, at 9:16 AM, Ryanne Dolan <ryannedo...@gmail.com> wrote: > > > > Hey y'all! > > > > Please take a look at KIP-382: > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-382%3A+MirrorMaker+2.0 > > > > Thanks for your feedback and support. > > > > Ryanne > >
