[jira] [Comment Edited] (KAFKA-15102) Mirror Maker 2 - KIP690 backward compatibility

2023-08-02 Thread Omnia Ibrahim (Jira)


[ 
https://issues.apache.org/jira/browse/KAFKA-15102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17750371#comment-17750371
 ] 

Omnia Ibrahim edited comment on KAFKA-15102 at 8/2/23 3:06 PM:
---

The KIP got accepted and the PR is ready


was (Author: omnia_h_ibrahim):
The KIP is accepted now and the PR is ready

> Mirror Maker 2 - KIP690 backward compatibility
> --
>
> Key: KAFKA-15102
> URL: https://issues.apache.org/jira/browse/KAFKA-15102
> Project: Kafka
>  Issue Type: Bug
>  Components: mirrormaker
>Affects Versions: 3.1.0
>Reporter: David Dufour
>Assignee: Omnia Ibrahim
>Priority: Major
> Fix For: 3.3, 3.2.4, 3.1.3, 3.6.0, 3.4.2, 3.5.2
>
>
> According to KIP690, "When users upgrade an existing MM2 cluster they don’t 
> need to change any of their current configuration as this proposal maintains 
> the default behaviour for MM2."
> Now, the separator is subject to customization.
> As a consequence, when an MM2 upgrade is performed, if the separator was 
> customized with replication.policy.separator, the name of this internal topic 
> changes. It then generates issues like:
> Caused by: java.util.concurrent.ExecutionException: 
> org.apache.kafka.common.errors.InvalidTopicException: Topic 
> 'mm2-offset-syncs_bkts28_internal' collides with existing topics: 
> mm2-offset-syncs.bkts28.internal
> It has been observed that the replication can then be broken sometimes 
> several days after the upgrade (reason not identified). By deleting the old 
> topic name, it recovers.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Comment Edited] (KAFKA-15102) Mirror Maker 2 - KIP690 backward compatibility

2023-08-02 Thread Omnia Ibrahim (Jira)


[ 
https://issues.apache.org/jira/browse/KAFKA-15102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17750371#comment-17750371
 ] 

Omnia Ibrahim edited comment on KAFKA-15102 at 8/2/23 3:06 PM:
---

The KIP is accepted now and the PR is ready


was (Author: omnia_h_ibrahim):
The KIP is accepted and the PR is ready

> Mirror Maker 2 - KIP690 backward compatibility
> --
>
> Key: KAFKA-15102
> URL: https://issues.apache.org/jira/browse/KAFKA-15102
> Project: Kafka
>  Issue Type: Bug
>  Components: mirrormaker
>Affects Versions: 3.1.0
>Reporter: David Dufour
>Assignee: Omnia Ibrahim
>Priority: Major
> Fix For: 3.3, 3.2.4, 3.1.3, 3.6.0, 3.4.2, 3.5.2
>
>
> According to KIP690, "When users upgrade an existing MM2 cluster they don’t 
> need to change any of their current configuration as this proposal maintains 
> the default behaviour for MM2."
> Now, the separator is subject to customization.
> As a consequence, when an MM2 upgrade is performed, if the separator was 
> customized with replication.policy.separator, the name of this internal topic 
> changes. It then generates issues like:
> Caused by: java.util.concurrent.ExecutionException: 
> org.apache.kafka.common.errors.InvalidTopicException: Topic 
> 'mm2-offset-syncs_bkts28_internal' collides with existing topics: 
> mm2-offset-syncs.bkts28.internal
> It has been observed that the replication can then be broken sometimes 
> several days after the upgrade (reason not identified). By deleting the old 
> topic name, it recovers.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Comment Edited] (KAFKA-15102) Mirror Maker 2 - KIP690 backward compatibility

2023-07-04 Thread Omnia Ibrahim (Jira)


[ 
https://issues.apache.org/jira/browse/KAFKA-15102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17739924#comment-17739924
 ] 

Omnia Ibrahim edited comment on KAFKA-15102 at 7/4/23 1:29 PM:
---

[~ChrisEgerton] I updated the compatibility section in the KIP with the 
impacted versions and linked to this JIRA. I can open a small KIP to have 
`{{{}replication.policy.internal.topic.separator.enabled` if you don't have 
time to do it. {}}}


was (Author: omnia_h_ibrahim):
[~ChrisEgerton] I updated the compatibility section in the KIP with the 
impacted versions and linked to this JIRA. I can open a small KIP to have 
`{{{}replication.policy.internal.topic.separator.enabled` if you don't have 
time to do it. {}}}

> Mirror Maker 2 - KIP690 backward compatibility
> --
>
> Key: KAFKA-15102
> URL: https://issues.apache.org/jira/browse/KAFKA-15102
> Project: Kafka
>  Issue Type: Bug
>  Components: mirrormaker
>Affects Versions: 3.1.0
>Reporter: David Dufour
>Priority: Major
>
> According to KIP690, "When users upgrade an existing MM2 cluster they don’t 
> need to change any of their current configuration as this proposal maintains 
> the default behaviour for MM2."
> Now, the separator is subject to customization.
> As a consequence, when an MM2 upgrade is performed, if the separator was 
> customized with replication.policy.separator, the name of this internal topic 
> changes. It then generates issues like:
> Caused by: java.util.concurrent.ExecutionException: 
> org.apache.kafka.common.errors.InvalidTopicException: Topic 
> 'mm2-offset-syncs_bkts28_internal' collides with existing topics: 
> mm2-offset-syncs.bkts28.internal
> It has been observed that the replication can then be broken sometimes 
> several days after the upgrade (reason not identified). By deleting the old 
> topic name, it recovers.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Comment Edited] (KAFKA-15102) Mirror Maker 2 - KIP690 backward compatibility

2023-06-29 Thread Omnia Ibrahim (Jira)


[ 
https://issues.apache.org/jira/browse/KAFKA-15102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17738744#comment-17738744
 ] 

Omnia Ibrahim edited comment on KAFKA-15102 at 6/29/23 7:30 PM:


Thanks, [~ddufour1a] for raising this. The backward compatibility mentioned in 
the KIP accounted only for using the default separator configuration and didn't 
address the usage custom separator (my mistake here). [~ChrisEgerton] I think 
having `{{{}replication.policy.internal.topic.separator.enabled` is a good 
option.{}}}

{{{}We can also update the backward compatibility section in KIP-690 to ask 
users to provide a custom implementation that the override 
`ReplicationPolicy.{}}}offsetSyncsTopic`and 
`ReplicationPolicy.checkpointsTopic` methods to use old topics if they still 
want to use the old internal topics or delete them so they are aware of this 
issue.


was (Author: omnia_h_ibrahim):
Thanks, [~ddufour1a] for raising this. The backward compatibility mentioned in 
the KIP accounted only for using the default separator configuration and didn't 
address the usage custom separator (my mistake here). [~ChrisEgerton] I think 
having `{{{}replication.policy.internal.topic.separator.enabled` is a good 
option.{}}}

{{{}We can also update the backward compatibility section in KIP-690 to ask 
users to override the `ReplicationPolicy.{}}}offsetSyncsTopic`and 
`ReplicationPolicy.checkpointsTopic` methods to use old topics if they still 
want to use the old internal topics or delete them so they are aware of this 
issue.

> Mirror Maker 2 - KIP690 backward compatibility
> --
>
> Key: KAFKA-15102
> URL: https://issues.apache.org/jira/browse/KAFKA-15102
> Project: Kafka
>  Issue Type: Bug
>  Components: mirrormaker
>Affects Versions: 3.1.0
>Reporter: David Dufour
>Priority: Major
>
> According to KIP690, "When users upgrade an existing MM2 cluster they don’t 
> need to change any of their current configuration as this proposal maintains 
> the default behaviour for MM2."
> Now, the separator is subject to customization.
> As a consequence, when an MM2 upgrade is performed, if the separator was 
> customized with replication.policy.separator, the name of this internal topic 
> changes. It then generates issues like:
> Caused by: java.util.concurrent.ExecutionException: 
> org.apache.kafka.common.errors.InvalidTopicException: Topic 
> 'mm2-offset-syncs_bkts28_internal' collides with existing topics: 
> mm2-offset-syncs.bkts28.internal
> It has been observed that the replication can then be broken sometimes 
> several days after the upgrade (reason not identified). By deleting the old 
> topic name, it recovers.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Comment Edited] (KAFKA-15102) Mirror Maker 2 - KIP690 backward compatibility

2023-06-29 Thread Omnia Ibrahim (Jira)


[ 
https://issues.apache.org/jira/browse/KAFKA-15102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17738744#comment-17738744
 ] 

Omnia Ibrahim edited comment on KAFKA-15102 at 6/29/23 7:01 PM:


Thanks, [~ddufour1a] for raising this. The backward compatibility mentioned in 
the KIP accounted only for using the default separator configuration and didn't 
address the usage custom separator (my mistake here). [~ChrisEgerton] I think 
having `{{{}replication.policy.internal.topic.separator.enabled` is a good 
option.{}}}

{{{}We can also update the backward compatibility section in KIP-690 to ask 
users to override the `ReplicationPolicy.{}}}offsetSyncsTopic`and 
`ReplicationPolicy.checkpointsTopic` methods to use old topics if they still 
want to use the old internal topics or delete them so they are aware of this 
issue.


was (Author: omnia_h_ibrahim):
Thanks, [~ddufour1a] for raising this. The backward compatibility mentioned in 
the KIP accounted only for using the default separator configuration and didn't 
address the usage custom separator (my mistake here). [~ChrisEgerton] I think 
having `{{{}replication.policy.internal.topic.separator.enabled` is a good 
option.{}}}

{{{}We can also update the backward compatibility section in KIP-690 to ask 
customers to override the `ReplicationPolicy.{}}}offsetSyncsTopic`and 
`ReplicationPolicy.checkpointsTopic` methods to use old topics if they still 
want to use the old internal topics or delete them so they are aware of this 
issue.

> Mirror Maker 2 - KIP690 backward compatibility
> --
>
> Key: KAFKA-15102
> URL: https://issues.apache.org/jira/browse/KAFKA-15102
> Project: Kafka
>  Issue Type: Bug
>  Components: mirrormaker
>Affects Versions: 3.1.0
>Reporter: David Dufour
>Priority: Major
>
> According to KIP690, "When users upgrade an existing MM2 cluster they don’t 
> need to change any of their current configuration as this proposal maintains 
> the default behaviour for MM2."
> Now, the separator is subject to customization.
> As a consequence, when an MM2 upgrade is performed, if the separator was 
> customized with replication.policy.separator, the name of this internal topic 
> changes. It then generates issues like:
> Caused by: java.util.concurrent.ExecutionException: 
> org.apache.kafka.common.errors.InvalidTopicException: Topic 
> 'mm2-offset-syncs_bkts28_internal' collides with existing topics: 
> mm2-offset-syncs.bkts28.internal
> It has been observed that the replication can then be broken sometimes 
> several days after the upgrade (reason not identified). By deleting the old 
> topic name, it recovers.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Comment Edited] (KAFKA-15102) Mirror Maker 2 - KIP690 backward compatibility

2023-06-29 Thread Omnia Ibrahim (Jira)


[ 
https://issues.apache.org/jira/browse/KAFKA-15102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17738744#comment-17738744
 ] 

Omnia Ibrahim edited comment on KAFKA-15102 at 6/29/23 7:00 PM:


Thanks, [~ddufour1a] for raising this. The backward compatibility mentioned in 
the KIP accounted only for using the default separator configuration and didn't 
address the usage custom separator (my mistake here). [~ChrisEgerton] I think 
having `{{{}replication.policy.internal.topic.separator.enabled` is a good 
option. {}}}

{{{}We can also update the backward compatibility section in KIP-690 to ask 
customers to override the `ReplicationPolicy.{}}}offsetSyncsTopic`and 
`ReplicationPolicy.checkpointsTopic` methods to use old topics if they still 
want to use the old internal topics or delete them so they are aware of this 
issue.


was (Author: omnia_h_ibrahim):
Thanks, [~ddufour1a] for raising this. The backward compatibility mentioned in 
the KIP accounted only for using the default separator configuration and didn't 
address the usage custom separator (my mistake here). [~ChrisEgerton] I think 
having `{{{}replication.policy.internal.topic.separator.enabled` is a good 
option. {}}}{{{}We can also update the backward compatibility section in 
KIP-690 to ask customers to override the 
`ReplicationPolicy.{}}}offsetSyncsTopic`and 
`ReplicationPolicy.checkpointsTopic` methods to use old topics if they still 
want to use the old internal topics or delete them so they are aware of this 
issue.

> Mirror Maker 2 - KIP690 backward compatibility
> --
>
> Key: KAFKA-15102
> URL: https://issues.apache.org/jira/browse/KAFKA-15102
> Project: Kafka
>  Issue Type: Bug
>  Components: mirrormaker
>Affects Versions: 3.1.0
>Reporter: David Dufour
>Priority: Major
>
> According to KIP690, "When users upgrade an existing MM2 cluster they don’t 
> need to change any of their current configuration as this proposal maintains 
> the default behaviour for MM2."
> Now, the separator is subject to customization.
> As a consequence, when an MM2 upgrade is performed, if the separator was 
> customized with replication.policy.separator, the name of this internal topic 
> changes. It then generates issues like:
> Caused by: java.util.concurrent.ExecutionException: 
> org.apache.kafka.common.errors.InvalidTopicException: Topic 
> 'mm2-offset-syncs_bkts28_internal' collides with existing topics: 
> mm2-offset-syncs.bkts28.internal
> It has been observed that the replication can then be broken sometimes 
> several days after the upgrade (reason not identified). By deleting the old 
> topic name, it recovers.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Comment Edited] (KAFKA-15102) Mirror Maker 2 - KIP690 backward compatibility

2023-06-29 Thread Omnia Ibrahim (Jira)


[ 
https://issues.apache.org/jira/browse/KAFKA-15102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17738744#comment-17738744
 ] 

Omnia Ibrahim edited comment on KAFKA-15102 at 6/29/23 7:00 PM:


Thanks, [~ddufour1a] for raising this. The backward compatibility mentioned in 
the KIP accounted only for using the default separator configuration and didn't 
address the usage custom separator (my mistake here). [~ChrisEgerton] I think 
having `{{{}replication.policy.internal.topic.separator.enabled` is a good 
option.{}}}

{{{}We can also update the backward compatibility section in KIP-690 to ask 
customers to override the `ReplicationPolicy.{}}}offsetSyncsTopic`and 
`ReplicationPolicy.checkpointsTopic` methods to use old topics if they still 
want to use the old internal topics or delete them so they are aware of this 
issue.


was (Author: omnia_h_ibrahim):
Thanks, [~ddufour1a] for raising this. The backward compatibility mentioned in 
the KIP accounted only for using the default separator configuration and didn't 
address the usage custom separator (my mistake here). [~ChrisEgerton] I think 
having `{{{}replication.policy.internal.topic.separator.enabled` is a good 
option. {}}}

{{{}We can also update the backward compatibility section in KIP-690 to ask 
customers to override the `ReplicationPolicy.{}}}offsetSyncsTopic`and 
`ReplicationPolicy.checkpointsTopic` methods to use old topics if they still 
want to use the old internal topics or delete them so they are aware of this 
issue.

> Mirror Maker 2 - KIP690 backward compatibility
> --
>
> Key: KAFKA-15102
> URL: https://issues.apache.org/jira/browse/KAFKA-15102
> Project: Kafka
>  Issue Type: Bug
>  Components: mirrormaker
>Affects Versions: 3.1.0
>Reporter: David Dufour
>Priority: Major
>
> According to KIP690, "When users upgrade an existing MM2 cluster they don’t 
> need to change any of their current configuration as this proposal maintains 
> the default behaviour for MM2."
> Now, the separator is subject to customization.
> As a consequence, when an MM2 upgrade is performed, if the separator was 
> customized with replication.policy.separator, the name of this internal topic 
> changes. It then generates issues like:
> Caused by: java.util.concurrent.ExecutionException: 
> org.apache.kafka.common.errors.InvalidTopicException: Topic 
> 'mm2-offset-syncs_bkts28_internal' collides with existing topics: 
> mm2-offset-syncs.bkts28.internal
> It has been observed that the replication can then be broken sometimes 
> several days after the upgrade (reason not identified). By deleting the old 
> topic name, it recovers.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Comment Edited] (KAFKA-15102) Mirror Maker 2 - KIP690 backward compatibility

2023-06-29 Thread Omnia Ibrahim (Jira)


[ 
https://issues.apache.org/jira/browse/KAFKA-15102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17738744#comment-17738744
 ] 

Omnia Ibrahim edited comment on KAFKA-15102 at 6/29/23 7:00 PM:


Thanks, [~ddufour1a] for raising this. The backward compatibility mentioned in 
the KIP accounted only for using the default separator configuration and didn't 
address the usage custom separator (my mistake here). [~ChrisEgerton] I think 
having `{{{}replication.policy.internal.topic.separator.enabled` is a good 
option. {}}}



{{{}We can also update the backward compatibility section in KIP-690 to ask 
customers to override the `ReplicationPolicy.{}}}offsetSyncsTopic`and 
`ReplicationPolicy.checkpointsTopic` methods to use old topics if they still 
want to use the old internal topics or delete them so they are aware of this 
issue.


was (Author: omnia_h_ibrahim):
Thanks, [~ddufour1a] for raising this. The backward compatibility mentioned in 
the KIP accounted only for using the default separator configuration and didn't 
address the usage custom separator (my mistake here). [~ChrisEgerton] I think 
having `{{{}replication.policy.internal.topic.separator.enabled` is a good 
option. {}}}

{{{}We can also update the backward compatibility section in KIP-690 to ask 
customers to override the `ReplicationPolicy.{}}}offsetSyncsTopic`and 
`ReplicationPolicy.checkpointsTopic` methods to use old topics if they still 
want to use the old internal topics or delete them so they are aware of this 
issue.

> Mirror Maker 2 - KIP690 backward compatibility
> --
>
> Key: KAFKA-15102
> URL: https://issues.apache.org/jira/browse/KAFKA-15102
> Project: Kafka
>  Issue Type: Bug
>  Components: mirrormaker
>Affects Versions: 3.1.0
>Reporter: David Dufour
>Priority: Major
>
> According to KIP690, "When users upgrade an existing MM2 cluster they don’t 
> need to change any of their current configuration as this proposal maintains 
> the default behaviour for MM2."
> Now, the separator is subject to customization.
> As a consequence, when an MM2 upgrade is performed, if the separator was 
> customized with replication.policy.separator, the name of this internal topic 
> changes. It then generates issues like:
> Caused by: java.util.concurrent.ExecutionException: 
> org.apache.kafka.common.errors.InvalidTopicException: Topic 
> 'mm2-offset-syncs_bkts28_internal' collides with existing topics: 
> mm2-offset-syncs.bkts28.internal
> It has been observed that the replication can then be broken sometimes 
> several days after the upgrade (reason not identified). By deleting the old 
> topic name, it recovers.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Comment Edited] (KAFKA-15102) Mirror Maker 2 - KIP690 backward compatibility

2023-06-29 Thread Omnia Ibrahim (Jira)


[ 
https://issues.apache.org/jira/browse/KAFKA-15102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17738744#comment-17738744
 ] 

Omnia Ibrahim edited comment on KAFKA-15102 at 6/29/23 7:00 PM:


Thanks, [~ddufour1a] for raising this. The backward compatibility mentioned in 
the KIP accounted only for using the default separator configuration and didn't 
address the usage custom separator (my mistake here). [~ChrisEgerton] I think 
having `{{{}replication.policy.internal.topic.separator.enabled` is a good 
option. {}}}

{{{}We can also update the backward compatibility section in KIP-690 to ask 
customers to override the `ReplicationPolicy.{}}}offsetSyncsTopic`and 
`ReplicationPolicy.checkpointsTopic` methods to use old topics if they still 
want to use the old internal topics or delete them so they are aware of this 
issue.


was (Author: omnia_h_ibrahim):
Thanks, [~ddufour1a] for raising this. The backward compatibility mentioned in 
the KIP accounted only for using the default separator configuration and didn't 
address the usage custom separator (my mistake here). [~ChrisEgerton] I think 
having `{{{}replication.policy.internal.topic.separator.enabled` is a good 
option. {}}}



{{{}We can also update the backward compatibility section in KIP-690 to ask 
customers to override the `ReplicationPolicy.{}}}offsetSyncsTopic`and 
`ReplicationPolicy.checkpointsTopic` methods to use old topics if they still 
want to use the old internal topics or delete them so they are aware of this 
issue.

> Mirror Maker 2 - KIP690 backward compatibility
> --
>
> Key: KAFKA-15102
> URL: https://issues.apache.org/jira/browse/KAFKA-15102
> Project: Kafka
>  Issue Type: Bug
>  Components: mirrormaker
>Affects Versions: 3.1.0
>Reporter: David Dufour
>Priority: Major
>
> According to KIP690, "When users upgrade an existing MM2 cluster they don’t 
> need to change any of their current configuration as this proposal maintains 
> the default behaviour for MM2."
> Now, the separator is subject to customization.
> As a consequence, when an MM2 upgrade is performed, if the separator was 
> customized with replication.policy.separator, the name of this internal topic 
> changes. It then generates issues like:
> Caused by: java.util.concurrent.ExecutionException: 
> org.apache.kafka.common.errors.InvalidTopicException: Topic 
> 'mm2-offset-syncs_bkts28_internal' collides with existing topics: 
> mm2-offset-syncs.bkts28.internal
> It has been observed that the replication can then be broken sometimes 
> several days after the upgrade (reason not identified). By deleting the old 
> topic name, it recovers.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Comment Edited] (KAFKA-15102) Mirror Maker 2 - KIP690 backward compatibility

2023-06-29 Thread Chris Egerton (Jira)


[ 
https://issues.apache.org/jira/browse/KAFKA-15102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17738733#comment-17738733
 ] 

Chris Egerton edited comment on KAFKA-15102 at 6/29/23 6:43 PM:


[~ddufour1a] Thanks for raising this. I believe if we wanted to preserve 
backward compatibility perfectly, we would have had to ignore the custom 
separator when creating topics affected by KIP-690, and possibly introduced 
separate opt-in configuration logic to disable that behavior (i.e., resume 
taking custom separators into account, even for topics that were not affected 
prior to KIP-690).

However, since 3.1.0 came out a year and a half ago, things are less 
cut-and-dry: there may be more negative fallout for users who are accustomed to 
the current behavior than what's currently being experienced by those who are 
used to the previous behavior.

One possibility could be to revert to older behavior for the remainder of our 
3.x.y releases with a single configuration property such as 
{{replication.policy.internal.topic.separator.enabled}} that defaults to 
{{false}} for now and, come 4.0, defaults to {{{}true{}}}.

[~omnia_h_ibrahim] [~mimaison] do either of you have thoughts on how to 
approach this accidental break in compatibility? It's probably worth bringing 
this up in a discussion thread on the user/dev mailing lists, but I'd rather 
get some feedback on a potential fix before bringing this before a larger 
audience.


was (Author: chrisegerton):
[~ddufour1a] Thanks for raising this. I believe if we wanted to preserve 
backward compatibility perfectly, we would have had to ignore the custom 
separator when creating topics affected by KIP-690, and possibly introduced 
separate opt-in configuration logic to disable that behavior (i.e., resume 
taking custom separators into account, even for topics that were not affected 
prior to KIP-690).

However, since 3.1.0 came out a year and a half ago, things are less 
cut-and-dry: there may be more negative fallout for users who are accustomed to 
the current behavior than what's currently being experienced by those who are 
used to the previous behavior.

One possibility could be to revert to older behavior for the remainder of our 
3.x.y releases with a single configuration property such as 
{{replication.policy.internal.topic.separator.enabled}} that defaults to 
{{false}} for now and, come 4.0, defaults to {{{}true{}}}.

[~omnia_h_ibrahim] [~mimaison] do either of you have thoughts on how to 
approach this accidental break in compatibility?

> Mirror Maker 2 - KIP690 backward compatibility
> --
>
> Key: KAFKA-15102
> URL: https://issues.apache.org/jira/browse/KAFKA-15102
> Project: Kafka
>  Issue Type: Bug
>  Components: mirrormaker
>Affects Versions: 3.1.0
>Reporter: David Dufour
>Priority: Major
>
> According to KIP690, "When users upgrade an existing MM2 cluster they don’t 
> need to change any of their current configuration as this proposal maintains 
> the default behaviour for MM2."
> Now, the separator is subject to customization.
> As a consequence, when an MM2 upgrade is performed, if the separator was 
> customized with replication.policy.separator, the name of this internal topic 
> changes. It then generates issues like:
> Caused by: java.util.concurrent.ExecutionException: 
> org.apache.kafka.common.errors.InvalidTopicException: Topic 
> 'mm2-offset-syncs_bkts28_internal' collides with existing topics: 
> mm2-offset-syncs.bkts28.internal
> It has been observed that the replication can then be broken sometimes 
> several days after the upgrade (reason not identified). By deleting the old 
> topic name, it recovers.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)