[jira] [Updated] (KAFKA-16798) Mirrormaker2 dedicated mode - sync.group.offsets.interval not working
[
https://issues.apache.org/jira/browse/KAFKA-16798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thanos Athanasopoulos updated KAFKA-16798:
--
Description:
Single instance MirrorMaker2 in dedicated mode, active passive replication
logic.
sync.group.offsets.interval.seconds=2 configuration is enabled and active
{noformat}
[root@x-x ~]# docker logs cc-mm 2>&1 -f | grep -i
"auto.commit.interval\|checkpoint.interval\|consumer.commit.interval\|sync.topics.interval\|sync.group.offsets.interval\|offset-syncs.interval.seconds
"
auto.commit.interval.ms = 5000
auto.commit.interval.ms = 5000
auto.commit.interval.ms = 5000
auto.commit.interval.ms = 5000
auto.commit.interval.ms = 5000
auto.commit.interval.ms = 5000
sync.group.offsets.interval.seconds = 2
sync.group.offsets.interval.seconds = 2
auto.commit.interval.ms = 5000
sync.group.offsets.interval.seconds = 2
auto.commit.interval.ms = 5000
auto.commit.interval.ms = 5000
auto.commit.interval.ms = 5000
auto.commit.interval.ms = 5000
auto.commit.interval.ms = 5000
auto.commit.interval.ms = 5000
auto.commit.interval.ms = 5000
auto.commit.interval.ms = 5000
sync.group.offsets.interval.seconds = 2
auto.commit.interval.ms = 5000
auto.commit.interval.ms = 5000
sync.group.offsets.interval.seconds = 2
sync.group.offsets.interval.seconds = 2
auto.commit.interval.ms = 5000
{noformat}
but is not working, the commit of offsets happens *always every 60 seconds* as
you can see in the logs
{noformat}
[2024-05-20 09:32:44,847] INFO [MirrorSourceConnector|task-0|offsets]
WorkerSourceTask{id=MirrorSourceConnector-0} Committing offsets for 20
acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
[2024-05-20 09:32:44,852] INFO [MirrorSourceConnector|task-1|offsets]
WorkerSourceTask{id=MirrorSourceConnector-1} Committing offsets for 12
acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
[2024-05-20 09:32:44,875] INFO [MirrorHeartbeatConnector|task-0|offsets]
WorkerSourceTask{id=MirrorHeartbeatConnector-0} Committing offsets for 12
acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
[2024-05-20 09:32:44,881] INFO [MirrorCheckpointConnector|task-0|offsets]
WorkerSourceTask{id=MirrorCheckpointConnector-0} Committing offsets for 1
acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
[2024-05-20 09:32:44,890] INFO [MirrorHeartbeatConnector|task-0|offsets]
WorkerSourceTask{id=MirrorHeartbeatConnector-0} Committing offsets for 12
acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
[2024-05-20 09:33:44,850] INFO [MirrorSourceConnector|task-0|offsets]
WorkerSourceTask{id=MirrorSourceConnector-0} Committing offsets for 21
acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
[2024-05-20 09:33:44,854] INFO [MirrorSourceConnector|task-1|offsets]
WorkerSourceTask{id=MirrorSourceConnector-1} Committing offsets for 12
acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
[2024-05-20 09:33:44,878] INFO [MirrorHeartbeatConnector|task-0|offsets]
WorkerSourceTask{id=MirrorHeartbeatConnector-0} Committing offsets for 12
acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
[2024-05-20 09:33:44,883] INFO [MirrorCheckpointConnector|task-0|offsets]
WorkerSourceTask{id=MirrorCheckpointConnector-0} Committing offsets for 2
acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
[2024-05-20 09:33:44,895] INFO [MirrorHeartbeatConnector|task-0|offsets]
WorkerSourceTask{id=MirrorHeartbeatConnector-0} Committing offsets for 12
acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
[2024-05-20 09:34:44,852] INFO [MirrorSourceConnector|task-0|offsets]
WorkerSourceTask{id=MirrorSourceConnector-0} Committing offsets for 20
acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
[2024-05-20 09:34:44,857] INFO [MirrorSourceConnector|task-1|offsets]
WorkerSourceTask{id=MirrorSourceConnector-1} Committing offsets for 12
acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
[2024-05-20 09:34:44,880] INFO [MirrorHeartbeatConnector|task-0|offsets]
WorkerSourceTask{id=MirrorHeartbeatConnector-0} Committing offsets for 12
acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
[2024-05-20 09:34:44,886] INFO [MirrorCheckpointConnector|task-0|offsets]
WorkerSourceTask{id=MirrorCheckpointConnector-0} Committing offsets for 1
acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
[2024-05-20 09:34:44,897] INFO [MirrorHeartbeatConnector|task-0|offsets]
WorkerSource
[jira] [Updated] (KAFKA-16798) Mirrormaker2 dedicated mode - sync.group.offsets.interval not working
[
https://issues.apache.org/jira/browse/KAFKA-16798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thanos Athanasopoulos updated KAFKA-16798:
--
Description:
Single instance MirrorMaker2 in dedicated mode, active passive replication
logic.
sync.group.offsets.interval.seconds=2 configuration is enabled and active
{noformat}
[root@x-x ~]# docker logs cc-mm 2>&1 -f | grep -i
"auto.commit.interval\|checkpoint.interval\|consumer.commit.interval\|sync.topics.interval\|sync.group.offsets.interval\|offset-syncs.interval.seconds
"
auto.commit.interval.ms = 5000
auto.commit.interval.ms = 5000
auto.commit.interval.ms = 5000
auto.commit.interval.ms = 5000
auto.commit.interval.ms = 5000
auto.commit.interval.ms = 5000
sync.group.offsets.interval.seconds = 2
sync.group.offsets.interval.seconds = 2
auto.commit.interval.ms = 5000
sync.group.offsets.interval.seconds = 2
auto.commit.interval.ms = 5000
auto.commit.interval.ms = 5000
auto.commit.interval.ms = 5000
auto.commit.interval.ms = 5000
auto.commit.interval.ms = 5000
auto.commit.interval.ms = 5000
auto.commit.interval.ms = 5000
auto.commit.interval.ms = 5000
sync.group.offsets.interval.seconds = 2
auto.commit.interval.ms = 5000
auto.commit.interval.ms = 5000
sync.group.offsets.interval.seconds = 2
sync.group.offsets.interval.seconds = 2
auto.commit.interval.ms = 5000
{noformat}
but is not working, the commit of offsets happens *always every 60 seconds* as
you can see in the logs
{noformat}
[2024-05-20 09:32:44,847] INFO [MirrorSourceConnector|task-0|offsets]
WorkerSourceTask{id=MirrorSourceConnector-0} Committing offsets for 20
acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
[2024-05-20 09:32:44,852] INFO [MirrorSourceConnector|task-1|offsets]
WorkerSourceTask{id=MirrorSourceConnector-1} Committing offsets for 12
acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
[2024-05-20 09:32:44,875] INFO [MirrorHeartbeatConnector|task-0|offsets]
WorkerSourceTask{id=MirrorHeartbeatConnector-0} Committing offsets for 12
acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
[2024-05-20 09:32:44,881] INFO [MirrorCheckpointConnector|task-0|offsets]
WorkerSourceTask{id=MirrorCheckpointConnector-0} Committing offsets for 1
acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
[2024-05-20 09:32:44,890] INFO [MirrorHeartbeatConnector|task-0|offsets]
WorkerSourceTask{id=MirrorHeartbeatConnector-0} Committing offsets for 12
acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
[2024-05-20 09:33:44,850] INFO [MirrorSourceConnector|task-0|offsets]
WorkerSourceTask{id=MirrorSourceConnector-0} Committing offsets for 21
acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
[2024-05-20 09:33:44,854] INFO [MirrorSourceConnector|task-1|offsets]
WorkerSourceTask{id=MirrorSourceConnector-1} Committing offsets for 12
acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
[2024-05-20 09:33:44,878] INFO [MirrorHeartbeatConnector|task-0|offsets]
WorkerSourceTask{id=MirrorHeartbeatConnector-0} Committing offsets for 12
acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
[2024-05-20 09:33:44,883] INFO [MirrorCheckpointConnector|task-0|offsets]
WorkerSourceTask{id=MirrorCheckpointConnector-0} Committing offsets for 2
acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
[2024-05-20 09:33:44,895] INFO [MirrorHeartbeatConnector|task-0|offsets]
WorkerSourceTask{id=MirrorHeartbeatConnector-0} Committing offsets for 12
acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
[2024-05-20 09:34:44,852] INFO [MirrorSourceConnector|task-0|offsets]
WorkerSourceTask{id=MirrorSourceConnector-0} Committing offsets for 20
acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
[2024-05-20 09:34:44,857] INFO [MirrorSourceConnector|task-1|offsets]
WorkerSourceTask{id=MirrorSourceConnector-1} Committing offsets for 12
acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
[2024-05-20 09:34:44,880] INFO [MirrorHeartbeatConnector|task-0|offsets]
WorkerSourceTask{id=MirrorHeartbeatConnector-0} Committing offsets for 12
acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
[2024-05-20 09:34:44,886] INFO [MirrorCheckpointConnector|task-0|offsets]
WorkerSourceTask{id=MirrorCheckpointConnector-0} Committing offsets for 1
acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
[2024-05-20 09:34:44,897] INFO [MirrorHeartbeatConnector|task-0|offsets]
WorkerSource
[jira] [Updated] (KAFKA-16798) Mirrormaker2 dedicated mode - sync.group.offsets.interval not working
[
https://issues.apache.org/jira/browse/KAFKA-16798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thanos Athanasopoulos updated KAFKA-16798:
--
Description:
Single instance MirrorMaker2 in dedicated mode, active passive replication
logic.
sync.group.offsets.interval.seconds=2 configuration is enabled and active
{noformat}
[root@x-x ~]# docker logs cc-mm 2>&1 -f | grep -i
"auto.commit.interval\|checkpoint.interval\|consumer.commit.interval\|sync.topics.interval\|sync.group.offsets.interval\|offset-syncs.interval.seconds
"
auto.commit.interval.ms = 5000
auto.commit.interval.ms = 5000
auto.commit.interval.ms = 5000
auto.commit.interval.ms = 5000
auto.commit.interval.ms = 5000
auto.commit.interval.ms = 5000
sync.group.offsets.interval.seconds = 2
sync.group.offsets.interval.seconds = 2
auto.commit.interval.ms = 5000
sync.group.offsets.interval.seconds = 2
auto.commit.interval.ms = 5000
auto.commit.interval.ms = 5000
auto.commit.interval.ms = 5000
auto.commit.interval.ms = 5000
auto.commit.interval.ms = 5000
auto.commit.interval.ms = 5000
auto.commit.interval.ms = 5000
auto.commit.interval.ms = 5000
sync.group.offsets.interval.seconds = 2
auto.commit.interval.ms = 5000
auto.commit.interval.ms = 5000
sync.group.offsets.interval.seconds = 2
sync.group.offsets.interval.seconds = 2
auto.commit.interval.ms = 5000
{noformat}
but is not working, the commit of offsets happens *always every 60 seconds* as
you can see in the logs
{noformat}
[2024-05-20 09:32:44,847] INFO [MirrorSourceConnector|task-0|offsets]
WorkerSourceTask{id=MirrorSourceConnector-0} Committing offsets for 20
acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
[2024-05-20 09:32:44,852] INFO [MirrorSourceConnector|task-1|offsets]
WorkerSourceTask{id=MirrorSourceConnector-1} Committing offsets for 12
acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
[2024-05-20 09:32:44,875] INFO [MirrorHeartbeatConnector|task-0|offsets]
WorkerSourceTask{id=MirrorHeartbeatConnector-0} Committing offsets for 12
acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
[2024-05-20 09:32:44,881] INFO [MirrorCheckpointConnector|task-0|offsets]
WorkerSourceTask{id=MirrorCheckpointConnector-0} Committing offsets for 1
acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
[2024-05-20 09:32:44,890] INFO [MirrorHeartbeatConnector|task-0|offsets]
WorkerSourceTask{id=MirrorHeartbeatConnector-0} Committing offsets for 12
acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
[2024-05-20 09:33:44,850] INFO [MirrorSourceConnector|task-0|offsets]
WorkerSourceTask{id=MirrorSourceConnector-0} Committing offsets for 21
acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
[2024-05-20 09:33:44,854] INFO [MirrorSourceConnector|task-1|offsets]
WorkerSourceTask{id=MirrorSourceConnector-1} Committing offsets for 12
acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
[2024-05-20 09:33:44,878] INFO [MirrorHeartbeatConnector|task-0|offsets]
WorkerSourceTask{id=MirrorHeartbeatConnector-0} Committing offsets for 12
acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
[2024-05-20 09:33:44,883] INFO [MirrorCheckpointConnector|task-0|offsets]
WorkerSourceTask{id=MirrorCheckpointConnector-0} Committing offsets for 2
acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
[2024-05-20 09:33:44,895] INFO [MirrorHeartbeatConnector|task-0|offsets]
WorkerSourceTask{id=MirrorHeartbeatConnector-0} Committing offsets for 12
acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
[2024-05-20 09:34:44,852] INFO [MirrorSourceConnector|task-0|offsets]
WorkerSourceTask{id=MirrorSourceConnector-0} Committing offsets for 20
acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
[2024-05-20 09:34:44,857] INFO [MirrorSourceConnector|task-1|offsets]
WorkerSourceTask{id=MirrorSourceConnector-1} Committing offsets for 12
acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
[2024-05-20 09:34:44,880] INFO [MirrorHeartbeatConnector|task-0|offsets]
WorkerSourceTask{id=MirrorHeartbeatConnector-0} Committing offsets for 12
acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
[2024-05-20 09:34:44,886] INFO [MirrorCheckpointConnector|task-0|offsets]
WorkerSourceTask{id=MirrorCheckpointConnector-0} Committing offsets for 1
acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
[2024-05-20 09:34:44,897] INFO [MirrorHeartbeatConnector|task-0|offsets]
WorkerSource
[jira] [Updated] (KAFKA-16798) Mirrormaker2 dedicated mode - sync.group.offsets.interval not working
[
https://issues.apache.org/jira/browse/KAFKA-16798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thanos Athanasopoulos updated KAFKA-16798:
--
Description:
Single instance MirrorMaker2 in dedicated mode, active passive replication
logic.
sync.group.offsets.interval.seconds=2 configuration is enabled and active
{noformat}
[root@x-x ~]# docker logs cc-mm 2>&1 -f | grep -i
"auto.commit.interval\|checkpoint.interval\|consumer.commit.interval\|sync.topics.interval\|sync.group.offsets.interval\|offset-syncs.interval.seconds
"
auto.commit.interval.ms = 5000
auto.commit.interval.ms = 5000
auto.commit.interval.ms = 5000
auto.commit.interval.ms = 5000
auto.commit.interval.ms = 5000
auto.commit.interval.ms = 5000
sync.group.offsets.interval.seconds = 2
sync.group.offsets.interval.seconds = 2
auto.commit.interval.ms = 5000
sync.group.offsets.interval.seconds = 2
auto.commit.interval.ms = 5000
auto.commit.interval.ms = 5000
auto.commit.interval.ms = 5000
auto.commit.interval.ms = 5000
auto.commit.interval.ms = 5000
auto.commit.interval.ms = 5000
auto.commit.interval.ms = 5000
auto.commit.interval.ms = 5000
sync.group.offsets.interval.seconds = 2
auto.commit.interval.ms = 5000
auto.commit.interval.ms = 5000
sync.group.offsets.interval.seconds = 2
sync.group.offsets.interval.seconds = 2
auto.commit.interval.ms = 5000
{noformat}
but is not working, the commit of offsets happens *always every 60 seconds* as
you can see in the logs
{noformat}
[2024-05-20 09:32:44,847] INFO [MirrorSourceConnector|task-0|offsets]
WorkerSourceTask{id=MirrorSourceConnector-0} Committing offsets for 20
acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
[2024-05-20 09:32:44,852] INFO [MirrorSourceConnector|task-1|offsets]
WorkerSourceTask{id=MirrorSourceConnector-1} Committing offsets for 12
acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
[2024-05-20 09:32:44,875] INFO [MirrorHeartbeatConnector|task-0|offsets]
WorkerSourceTask{id=MirrorHeartbeatConnector-0} Committing offsets for 12
acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
[2024-05-20 09:32:44,881] INFO [MirrorCheckpointConnector|task-0|offsets]
WorkerSourceTask{id=MirrorCheckpointConnector-0} Committing offsets for 1
acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
[2024-05-20 09:32:44,890] INFO [MirrorHeartbeatConnector|task-0|offsets]
WorkerSourceTask{id=MirrorHeartbeatConnector-0} Committing offsets for 12
acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
[2024-05-20 09:33:44,850] INFO [MirrorSourceConnector|task-0|offsets]
WorkerSourceTask{id=MirrorSourceConnector-0} Committing offsets for 21
acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
[2024-05-20 09:33:44,854] INFO [MirrorSourceConnector|task-1|offsets]
WorkerSourceTask{id=MirrorSourceConnector-1} Committing offsets for 12
acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
[2024-05-20 09:33:44,878] INFO [MirrorHeartbeatConnector|task-0|offsets]
WorkerSourceTask{id=MirrorHeartbeatConnector-0} Committing offsets for 12
acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
[2024-05-20 09:33:44,883] INFO [MirrorCheckpointConnector|task-0|offsets]
WorkerSourceTask{id=MirrorCheckpointConnector-0} Committing offsets for 2
acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
[2024-05-20 09:33:44,895] INFO [MirrorHeartbeatConnector|task-0|offsets]
WorkerSourceTask{id=MirrorHeartbeatConnector-0} Committing offsets for 12
acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
[2024-05-20 09:34:44,852] INFO [MirrorSourceConnector|task-0|offsets]
WorkerSourceTask{id=MirrorSourceConnector-0} Committing offsets for 20
acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
[2024-05-20 09:34:44,857] INFO [MirrorSourceConnector|task-1|offsets]
WorkerSourceTask{id=MirrorSourceConnector-1} Committing offsets for 12
acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
[2024-05-20 09:34:44,880] INFO [MirrorHeartbeatConnector|task-0|offsets]
WorkerSourceTask{id=MirrorHeartbeatConnector-0} Committing offsets for 12
acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
[2024-05-20 09:34:44,886] INFO [MirrorCheckpointConnector|task-0|offsets]
WorkerSourceTask{id=MirrorCheckpointConnector-0} Committing offsets for 1
acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
[2024-05-20 09:34:44,897] INFO [MirrorHeartbeatConnector|task-0|offsets]
WorkerSource
[jira] [Updated] (KAFKA-16798) Mirrormaker2 dedicated mode - sync.group.offsets.interval not working
[
https://issues.apache.org/jira/browse/KAFKA-16798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thanos Athanasopoulos updated KAFKA-16798:
--
Description:
Single instance MirrorMaker2 in dedicated mode, active passive replication
logic.
sync.group.offsets.interval.seconds=2 configuration is enabled and active
{noformat}
[root@x-x ~]# docker logs cc-mm 2>&1 -f | grep -i
"auto.commit.interval\|checkpoint.interval\|consumer.commit.interval\|sync.topics.interval\|sync.group.offsets.interval\|offset-syncs.interval.seconds
"
auto.commit.interval.ms = 5000
auto.commit.interval.ms = 5000
auto.commit.interval.ms = 5000
auto.commit.interval.ms = 5000
auto.commit.interval.ms = 5000
auto.commit.interval.ms = 5000
sync.group.offsets.interval.seconds = 2
sync.group.offsets.interval.seconds = 2
auto.commit.interval.ms = 5000
sync.group.offsets.interval.seconds = 2
auto.commit.interval.ms = 5000
auto.commit.interval.ms = 5000
auto.commit.interval.ms = 5000
auto.commit.interval.ms = 5000
auto.commit.interval.ms = 5000
auto.commit.interval.ms = 5000
auto.commit.interval.ms = 5000
auto.commit.interval.ms = 5000
sync.group.offsets.interval.seconds = 2
auto.commit.interval.ms = 5000
auto.commit.interval.ms = 5000
sync.group.offsets.interval.seconds = 2
sync.group.offsets.interval.seconds = 2
auto.commit.interval.ms = 5000
{noformat}
but is not working, the commit of offsets happens always in 60 seconds as you
can see in the logs
{noformat}
[2024-05-20 09:32:44,847] INFO [MirrorSourceConnector|task-0|offsets]
WorkerSourceTask{id=MirrorSourceConnector-0} Committing offsets for 20
acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
[2024-05-20 09:32:44,852] INFO [MirrorSourceConnector|task-1|offsets]
WorkerSourceTask{id=MirrorSourceConnector-1} Committing offsets for 12
acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
[2024-05-20 09:32:44,875] INFO [MirrorHeartbeatConnector|task-0|offsets]
WorkerSourceTask{id=MirrorHeartbeatConnector-0} Committing offsets for 12
acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
[2024-05-20 09:32:44,881] INFO [MirrorCheckpointConnector|task-0|offsets]
WorkerSourceTask{id=MirrorCheckpointConnector-0} Committing offsets for 1
acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
[2024-05-20 09:32:44,890] INFO [MirrorHeartbeatConnector|task-0|offsets]
WorkerSourceTask{id=MirrorHeartbeatConnector-0} Committing offsets for 12
acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
[2024-05-20 09:33:44,850] INFO [MirrorSourceConnector|task-0|offsets]
WorkerSourceTask{id=MirrorSourceConnector-0} Committing offsets for 21
acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
[2024-05-20 09:33:44,854] INFO [MirrorSourceConnector|task-1|offsets]
WorkerSourceTask{id=MirrorSourceConnector-1} Committing offsets for 12
acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
[2024-05-20 09:33:44,878] INFO [MirrorHeartbeatConnector|task-0|offsets]
WorkerSourceTask{id=MirrorHeartbeatConnector-0} Committing offsets for 12
acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
[2024-05-20 09:33:44,883] INFO [MirrorCheckpointConnector|task-0|offsets]
WorkerSourceTask{id=MirrorCheckpointConnector-0} Committing offsets for 2
acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
[2024-05-20 09:33:44,895] INFO [MirrorHeartbeatConnector|task-0|offsets]
WorkerSourceTask{id=MirrorHeartbeatConnector-0} Committing offsets for 12
acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
[2024-05-20 09:34:44,852] INFO [MirrorSourceConnector|task-0|offsets]
WorkerSourceTask{id=MirrorSourceConnector-0} Committing offsets for 20
acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
[2024-05-20 09:34:44,857] INFO [MirrorSourceConnector|task-1|offsets]
WorkerSourceTask{id=MirrorSourceConnector-1} Committing offsets for 12
acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
[2024-05-20 09:34:44,880] INFO [MirrorHeartbeatConnector|task-0|offsets]
WorkerSourceTask{id=MirrorHeartbeatConnector-0} Committing offsets for 12
acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
[2024-05-20 09:34:44,886] INFO [MirrorCheckpointConnector|task-0|offsets]
WorkerSourceTask{id=MirrorCheckpointConnector-0} Committing offsets for 1
acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
[2024-05-20 09:34:44,897] INFO [MirrorHeartbeatConnector|task-0|offsets]
WorkerSourceTask{
[jira] [Updated] (KAFKA-16798) Mirrormaker2 dedicated mode - sync.group.offsets.interval not working
[
https://issues.apache.org/jira/browse/KAFKA-16798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thanos Athanasopoulos updated KAFKA-16798:
--
Summary: Mirrormaker2 dedicated mode - sync.group.offsets.interval not
working (was: Mirrormaker2 consumer groups sync.group.offsets.interval not
working)
> Mirrormaker2 dedicated mode - sync.group.offsets.interval not working
> -
>
> Key: KAFKA-16798
> URL: https://issues.apache.org/jira/browse/KAFKA-16798
> Project: Kafka
> Issue Type: Bug
> Components: mirrormaker
>Affects Versions: 3.7.0
>Reporter: Thanos Athanasopoulos
>Priority: Major
>
> Single instance MirrorMaker2 in dedicated mode, active passive replication
> logic.
> sync.group.offsets.interval.seconds=2 configuration is enabled and active
> {noformat}
> [root@x-x ~]# docker logs cc-mm 2>&1 -f | grep -i
> "auto.commit.interval\|checkpoint.interval\|consumer.commit.interval\|sync.topics.interval\|sync.group.offsets.interval\|offset-syncs.interval.seconds
> "
> auto.commit.interval.ms = 5000
> auto.commit.interval.ms = 5000
> auto.commit.interval.ms = 5000
> auto.commit.interval.ms = 5000
> auto.commit.interval.ms = 5000
> auto.commit.interval.ms = 5000
> sync.group.offsets.interval.seconds = 2
> sync.group.offsets.interval.seconds = 2
> auto.commit.interval.ms = 5000
> sync.group.offsets.interval.seconds = 2
> auto.commit.interval.ms = 5000
> auto.commit.interval.ms = 5000
> auto.commit.interval.ms = 5000
> auto.commit.interval.ms = 5000
> auto.commit.interval.ms = 5000
> auto.commit.interval.ms = 5000
> auto.commit.interval.ms = 5000
> auto.commit.interval.ms = 5000
> sync.group.offsets.interval.seconds = 2
> auto.commit.interval.ms = 5000
> auto.commit.interval.ms = 5000
> sync.group.offsets.interval.seconds = 2
> sync.group.offsets.interval.seconds = 2
> auto.commit.interval.ms = 5000
> {noformat}
> but is not working, the commit of offsets happens always in 60 seconds as you
> can see in the logs
> {noformat}
> [2024-05-20 09:32:44,847] INFO [MirrorSourceConnector|task-0|offsets]
> WorkerSourceTask{id=MirrorSourceConnector-0} Committing offsets for 20
> acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
> [2024-05-20 09:32:44,852] INFO [MirrorSourceConnector|task-1|offsets]
> WorkerSourceTask{id=MirrorSourceConnector-1} Committing offsets for 12
> acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
> [2024-05-20 09:32:44,875] INFO [MirrorHeartbeatConnector|task-0|offsets]
> WorkerSourceTask{id=MirrorHeartbeatConnector-0} Committing offsets for 12
> acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
> [2024-05-20 09:32:44,881] INFO [MirrorCheckpointConnector|task-0|offsets]
> WorkerSourceTask{id=MirrorCheckpointConnector-0} Committing offsets for 1
> acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
> [2024-05-20 09:32:44,890] INFO [MirrorHeartbeatConnector|task-0|offsets]
> WorkerSourceTask{id=MirrorHeartbeatConnector-0} Committing offsets for 12
> acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
> [2024-05-20 09:33:44,850] INFO [MirrorSourceConnector|task-0|offsets]
> WorkerSourceTask{id=MirrorSourceConnector-0} Committing offsets for 21
> acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
> [2024-05-20 09:33:44,854] INFO [MirrorSourceConnector|task-1|offsets]
> WorkerSourceTask{id=MirrorSourceConnector-1} Committing offsets for 12
> acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
> [2024-05-20 09:33:44,878] INFO [MirrorHeartbeatConnector|task-0|offsets]
> WorkerSourceTask{id=MirrorHeartbeatConnector-0} Committing offsets for 12
> acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
> [2024-05-20 09:33:44,883] INFO [MirrorCheckpointConnector|task-0|offsets]
> WorkerSourceTask{id=MirrorCheckpointConnector-0} Committing offsets for 2
> acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
> [2024-05-20 09:33:44,895] INFO [MirrorHeartbeatConnector|task-0|offsets]
> WorkerSourceTask{id=MirrorHeartbeatConnector-0} Committing offsets for 12
> acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
> [2024-05-20 09:34:44,852] INFO [MirrorSourceConnector|task-0|offsets]
> WorkerSourceTask{id=MirrorSourceConnector-0} Committing offsets for 20
> acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask:233)
> [2024-05-20 09:34:44,857] INFO [Mir
