> Oh - got it, it checks the entire prefix, which seems obvious to me in retrospect :)
Rhys, I've changed the wording to make this more clear, thanks for calling it out. Ryanne On Tue, Oct 16, 2018 at 4:16 PM McCaig, Rhys <rhys_mcc...@comcast.com> wrote: > > > In your example, us-west.us-east.us-central.us-west.topic is an invalid > > "remote topic" name because us-west appears twice. MM2 will not replicate > > us-east.us-central.us-west.topic into us-west a second time, because the > > source topic already has us-west in the prefix. This is what I mean by > > "cycle detection" -- cyclical replication does not result in infinite > > recursion. > > Oh - got it, it checks the entire prefix, which seems obvious to me in > retrospect :) > > Rhys > > > > On Oct 15, 2018, at 3:18 PM, Ryanne Dolan <ryannedo...@gmail.com> wrote: > > > > Rhys, thanks for your enthusiasm! > > > > In your example, us-west.us-east.us-central.us-west.topic is an invalid > > "remote topic" name because us-west appears twice. MM2 will not replicate > > us-east.us-central.us-west.topic into us-west a second time, because the > > source topic already has us-west in the prefix. This is what I mean by > > "cycle detection" -- cyclical replication does not result in infinite > > recursion. > > > > It's important to note that MM2 does NOT disallow these sort of cycles, > it > > just knows how to deal with them properly. > > > > Also notice this is done at the topic level, not per record. The records > > don't need any special header or anything for this cycle detection > > mechanism to work. > > > > Thanks! > > Ryanne > > > > On Mon, Oct 15, 2018 at 3:40 PM McCaig, Rhys <rhys_mcc...@comcast.com> > > wrote: > > > >> Hi Ryanne, > >> > >> This KIP is fantastic. It provides a great vision for how MirrorMaker > >> should evolve in the Kafka project. > >> > >> I have a question on cycle detection - In a scenario where I have 3 > >> clusters replicating between each other, it seems it may be easy to > >> misconfigure the connectors if auto topic creation is turned on so that > >> records become replicated to increasingly longer topic names (until the > >> topic name limit is reached). Consider clusters us-west, us-central, > >> us-east: > >> > >> us-west: topic > >> us-central: us-west.topic > >> us-east: us-central.us-west.topic > >> us-west: us-east.us-central.us-west.topic > >> us-central: us-west.us-east.us-central.us-west.topic > >> > >> I’m not sure whether this scenario would actually justify implementing > >> additional measures to avoid such a configuration, rather than ensuring > >> that the documentation is clear on how to avoid such scenarios - would > be > >> good to hear what others think on this. > >> > >> Excited to see the discussion on this one. > >> > >> Rhys > >> > >>> On Oct 15, 2018, at 9:16 AM, Ryanne Dolan <ryannedo...@gmail.com> > wrote: > >>> > >>> Hey y'all! > >>> > >>> Please take a look at KIP-382: > >>> > >>> > >> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-382%3A+MirrorMaker+2.0 > >>> > >>> Thanks for your feedback and support. > >>> > >>> Ryanne > >> > >> > >
