Question about MM2's source-cluster.checkpoints.internal topic

Hi, I am using MM2 version 3.1.0 for a uni-directional mirroring of source-cluster -> target-cluster. I am confused with the values found in the source-cluster.checkpoints.internal topic at the target cluster when the source's earliest offset is not 0. In my test setup, I prepared a topic

Re: When using MM2, how should the consumer be switched from source to target?

fka/commit/9063db0f8c4a53f5d9764612af898981d499a7b7 regards, aki El mié, 13 ene 2021 a las 3:17, Ning Zhang () escribió: > > Hello Aki, > > Can you elaborate on "when there are some messages left in the original > cluster" ? > > On 2021/01/12 13:01:40, Aki

Re: When using MM2, how should the consumer be switched from source to target?

line tool > to do it. I've done the latter for Cloudera's srm-control tool, as > documented here: > https://docs.cloudera.com/csp/2.0.1/srm-using/topics/srm-migrating-consumer-groups.html > > Ryanne > > On Thu, Dec 17, 2020, 5:23 PM Aki Yoshida wrote: > > > I'm answerin

Re: --override option for bin/connect-distributed.sh

afka+Improvement+Proposals > > On Mon, Dec 21, 2020 at 8:57 PM Aki Yoshida wrote: > > > Hi Kafka team, > > I think the --override option of Kafka is very practical in starting > > Kafka for various situations without changing the properties file. I > > missed t

--override option for bin/connect-distributed.sh

Hi Kafka team, I think the --override option of Kafka is very practical in starting Kafka for various situations without changing the properties file. I missed this feature in Kafka-Connect and I wanted to have it, so I created a patch in this commit in my forked repo.

MirrorConnectorsIntegrationTest's testOneWayReplicationWithAutoOffsetSync

Hi, I am new to MirrorMaker and I have been trying to understand its behavior, in particular about its consumer offset synchronization. I saw KIP-545 https://cwiki.apache.org/confluence/display/KAFKA/KIP-545%3A+support+automated+consumer+offset+sync+across+clusters+in+MM+2.0, which talks about

Re: When using MM2, how should the consumer be switched from source to target?

, there is no way to use the console client but use java API RemoteClusterUtils.translateOffsets() to get the offset and re-consuming from the new topic. Is this correct? El jue, 17 dic 2020 a las 21:18, Aki Yoshida () escribió: > > Hi, > I have a question regarding how to migrate a cons

When using MM2, how should the consumer be switched from source to target?

Hi, I have a question regarding how to migrate a consumer when the subscribing topic has been migrated to the target Kafka broker. Suppose a consumer is consuming from topic1 at the source Kafka broker. When MM2 mirrors this topic using options source.cluster.alias="",

Using FileConfigProvider with restricted path?

Hi, I am wondering if there is a possibility to restrict the path used for the FileConfigProvider's value in the configuration e.g., ${file:/xxxpath/xxxfile.properties:xxxname}. I am looking for a way to restrict this path. Thanks regards, aki

Re: swagger/openapi spec for Kafka-Connect REST API?

EST API? > > Hi, > > I'm also searching for this. If you find it, please let me know. > Thank you. > -- > Miguel Silvestre > > > On Mon, May 18, 2020 at 8:59 AM Aki Yoshida wrote: > > > I am looking for the openapi spec for Kafka-Connect API but I couldn't > >

swagger/openapi spec for Kafka-Connect REST API?

I am looking for the openapi spec for Kafka-Connect API but I couldn't find one. If it is available somewhere, could someone tell me where I can find it? regards, aki

Looking for some info on kafka homepage

Hi, I am looking for the following information on Kafka's project home page but am not able to find them. The home page seems to have been updated recently (looks really nice ;-) and maybe I just don't know where to navigate through. The information that I would like to find are: - List of Kafka

question about property advertised.listeners

I have a question regarding how to use advertised.listeners to expose a different port to the outside of the container where the broker is running. When using advertised.listeners to register a different port as the actual broker port, do we need a port forwarding from this configured port to the

ZK path name mentioned in the online Replication Design document seems to be incorrect

I am trying to understand the replication procedure and saw this document https://cwiki.apache.org/confluence/display/KAFKA/kafka+Detailed+Replication+Design+V3 describing: LeaderAndISR path: stores leader and ISR of a partition /brokers/topics/[topic]/[partition_id]/leaderAndISR -->