Marcin, I just created this jira per your suggestion: https://issues.apache.org/jira/browse/KAFKA-9726
I've assigned it to myself but I'm happy to give it up. Ryanne On Mon, Mar 16, 2020 at 2:37 AM Marcin Wieloch <marcin.wiel...@gmail.com> wrote: > Hello Ryanne, > > Thank you for these clarifications. I am glad that the use case of > “prefixless” replication is in-line with your vision for further > development of MirrorMaker2; I forgot indeed that already KIP-382 mentioned > that LegacyReplicationPolicy. > > I have not found however any JIRA issue addressing specifically > LegacyReplicationPolicy, and the main ticket for KIP-382 (KAFKA-7500) is > already resolved. > > So what would be the next step you recommend I take? Should I file my > feature request (rephrased to adjust for your comments)? Or is the need for > the legacy policy already tracked somehow and we don’t have to open any new > tickets for it? > > Marcin > > On 2020/03/11 14:20:44, Ryanne Dolan <ryannedo...@gmail.com> wrote: > > Hello Marcin, actually KIP-382 includes LegacyReplicationPolicy (not yet > > released) which is similar to your PrefixlessReplicationPolicy and the > > MigrationReplicationPolicy mentioned. I don't think these are hacky, > though > > maybe the documentation could better address what to do when a method > > doesn't necessarily apply cleanly. > > > > Frankly, I anticipated needing to make changes to MM2 itself to > accommodate > > these sort of use-cases -- the fact that you got this working as-is is > > encouraging :) > > > > Re the proposal, I think we could ship one or more of these replication > > policies with MM2 to support prefixless replication. There may be > slightly > > different implementations depending on the use-case, i.e. "migration" > from > > an old cluster vs active/passive w/o renaming topics vs mimicking MM1 > with > > LegacyReplicationPolicy. Maybe they are all identical and we just need > one? > > > > I like that use-cases are driving these requests rather than us try to > > enumerate all possible ReplicationPolicy implementations. Given a > > particular use-case and a successful application of a custom > > ReplicationPolicy, I for one would support including it in Apache Kafka. > > > > Ryanne > > > > On Wed, Mar 11, 2020, 5:26 AM Marcin Wieloch <marcin.wiel...@gmail.com> > > wrote: > > > > > Dear All, > > > > > > I would like to raise a feature request for ‘prefixless’ topic > > > mirroring with MirrorMaker2 (see detailed description below). > > > > > > As after quite a few tries I am not 100% sure, could you please > > > confirm that by now > > > the ‘prefixless’ mirroring is not supported and that my feature > > > request is valid? I would then file it in JIRA. > > > > > > Best regards, > > > > > > Marcin > > > > > > > > > Feature Request > > > > > > We would like to use MirrorMaker2 as a DRP solution for a simple setup > > > with two sites working in active-passive configuration > > > where all topics are always mirrored in one direction only and the > > > downstream topics keep their original names, i.e., without being > > > prefixed by MirrorMaker2. > > > > > > After a few rounds of 'trial and error' we came up with a customized > > > replication policy which, although hacky, seems to fulfill our need; > > > it is presented in stackoverflow here: > > > https://stackoverflow.com/a/60619233 > > > and source code is available here: > > > > > > > https://github.com/mawileo/mm2-prefixless-policy/blob/feature/prefixless/src/main/java/ch/mawileo/kafka/mm2/PrefixlessReplicationPolicy.java > > > > > > This solution is 'hacky' in that it violates specification for the > > > topicSource and upstreamTopic methods as expressed in javadoc of the > > > ReplicationPolicy interface. > > > > > > That makes us suspect that MirrorMaker2 has been never meant to work > > > in such a configuration and that our solution may stop > > > working with future releases of Kafka. > > > > > > We would like then to ask for making the possibility of 'prefixless' > > > topic mirroring in a simplified two-cluster setup a standard > > > feature of Kafka Mirror Maker 2. > > > > > >
