Iftach, I think I've found a bug. I tested this fix in my infra and worked fine. Putting a PR upstream for review: https://github.com/apache/kafka/pull/9313.
On Wed, Sep 16, 2020 at 1:26 PM Iftach Ben-Yosef <iben-yo...@outbrain.com> wrote: > Hey Samuel, > > I am facing a similar issue with the partitions.assignment.strategy > consumer setting on MM2. > > Let me know if you have any success with setting this (I am also interested > with the auto.offset.reset config as well but its a lower priority for my > testing. I assume the solution will be the same for both configs) > > Good luck and feel free to ping me if you want to try and discuss this. > > Iftach > > > On Wed, Sep 16, 2020, 19:59 Samuel Cantero <scante...@gmail.com> wrote: > > > So the problem is that this is hardcoded for 2.5.1 and 2.6: > > > > > https://github.com/apache/kafka/blob/62abe01bee0396515253373e906253bd0e081cb9/connect/mirror/src/main/java/org/apache/kafka/connect/mirror/MirrorConnectorConfig.java#L235 > > . > > > > Already fixed in https://github.com/apache/kafka/pull/8921. > > > > I will try to understand why it is failing for other properties now. > > > > On Wed, Sep 16, 2020 at 12:20 PM Ning Zhang <ning2008w...@gmail.com> > > wrote: > > > > > just a possibility that `source-cluster-alias.` does not align with the > > > <source_cluster_alias> used in `clusters:` > > > > > > On 2020/09/16 14:24:39, Samuel Cantero <scante...@gmail.com> wrote: > > > > Definitely not working for me. I can't change any consumer attribute > > for > > > > source-alias or cluster-alias. It is working for the producer though. > > The > > > > config is rendered fine so I need to dig deeper on my side as this is > > > > working for you. > > > > > > > > Thanks, > > > > > > > > On Tue, Sep 15, 2020 at 5:29 PM Ning Zhang <ning2008w...@gmail.com> > > > wrote: > > > > > > > > > Hello Samuel, > > > > > > > > > > I tried other consumer config and it worked. Basically, it should > > > follow > > > > > the pattern `source-cluster-alias.consumer.<consumer_config_name>`. > > In > > > your > > > > > case, I think it should be > > > > > `source-cluster-alias.consumer.auto.offset.reset`. But feel free to > > try > > > > > other consumer config and see if they work > > > > > > > > > > On 2020/09/15 20:53:12, Samuel Cantero <scante...@gmail.com> > wrote: > > > > > > I tried this but even the override does not work for me as I can > > > still > > > > > see > > > > > > "earliest" on the ConsumerConfig properties. Did you manage > > actually > > > to > > > > > > make this work or are you just recommending this config? > > > > > > > > > > > > Best, > > > > > > > > > > > > On Mon, Sep 14, 2020 at 7:26 PM Ning Zhang < > ning2008w...@gmail.com > > > > > > > > wrote: > > > > > > > > > > > > > Hello Samuel, > > > > > > > > > > > > > > I guess you are talking about the case where you start to > mirror > > a > > > new > > > > > > > topic? If yes, I think the config should be > > > > > > > `source-cluster-alias.consumer.auto.offset.reset`, instead of > > > `target`, > > > > > > > because `target` is where the producer of MM2 replicates the > > > messages > > > > > > > towards, right? > > > > > > > > > > > > > > If MM2 mirrors a topic that has been previously mirrored and > the > > > > > consumer > > > > > > > offsets have preserved, `auto.offset.reset`` should not matter, > > > > > because MM2 > > > > > > > always start mirroring from the last known offsets > > > > > > > > > > > > > > On 2020/09/10 23:03:16, Samuel Cantero <scante...@gmail.com> > > > wrote: > > > > > > > > Hello, > > > > > > > > > > > > > > > > I'd like to know whether starting mirroring from latest is > > doable > > > > > with > > > > > > > > mirrormaker2. I tried setting the > > > > > > > > target-cluster-alias.consumer.auto.offset.reset to latest but > > it > > > > > doesn't > > > > > > > > work (for an active/passive setup). Any help would be > > > appreciated. > > > > > > > > > > > > > > > > Best, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > The above terms reflect a potential business arrangement, are provided > solely as a basis for further discussion, and are not intended to be and > do > not constitute a legally binding obligation. No legally binding > obligations > will be created, implied, or inferred until an agreement in final form is > executed in writing by all parties involved. > > > This email and any > attachments hereto may be confidential or privileged. If you received > this > communication by mistake, please don't forward it to anyone else, please > erase all copies and attachments, and please let me know that it has gone > to the wrong person. Thanks. >
