Hello everyone, I would like to start a discussion on KIP-709, which addresses some missing features in MM2 dedicated mode. https://cwiki.apache.org/confluence/display/KAFKA/KIP-710%3A+Full+support+for+distributed+mode+in+dedicated+MirrorMaker+2.0+clusters
Currently, the dedicated mode of MM2 does not fully support running in a cluster. The core issue is that the Connect REST Server is not included in the dedicated mode, which makes follower->leader communication impossible. In some cases, this results in the cluster not being able to react to dynamic configuration changes (e.g. dynamic topic filter changes). Another smaller detail is that MM2 dedicated mode eagerly resolves config provider references in the Connector configurations, which is undesirable and a breaking change compared to vanilla Connect. This can cause an issue for example when there is an environment variable reference, which contains some host-specific information, like a file path. The leader resolves the reference eagerly, and the resolved value is propagated to other MM2 nodes instead of the reference being resolved locally, separately on each node. The KIP addresses these by adding the Connect REST Server to the MM2 dedicated mode for each replication flow, and postponing the config provider reference resolution. Please discuss, I know this is a major change, but also an important feature for MM2 users. Daniel
