[
https://issues.apache.org/jira/browse/KAFKA-4942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15939453#comment-15939453
]
Stephane Maarek edited comment on KAFKA-4942 at 3/23/17 11:57 PM:
------------------------------------------------------------------
[~ewencp] nothing weird from the log, except that EVERY offset commit gives me
that behaviour. You know what's even weirder? Offsets still are committed. It's
because the line you linked me doesn't interrupt the offset commit, it just
throws a warning.
Now the reason I would see that line being triggered right away is when
now >= commitTimeoutMs
now >= commitStarted +
workerConfig.getLong(WorkerConfig.OFFSET_COMMIT_TIMEOUT_MS_CONFIG)
I believe the setting workerConfig.getLong should return something 60000 ms, so
it hints towards commitStarted being corrupted.
And I think I just found your issue.....
commitTimeoutMs happens before the try block.
so no matter what happens in the commitOffsets(now, false); , it won't impact
the comparison:
if (committing && now >= commitTimeoutMs)
And that's the bug :) One offset that times out on your thread and all the
subsequent tries will show that warning
was (Author: stephane.maa...@gmail.com):
[~ewencp] nothing weird from the log, except that EVERY offset commit gives me
that behaviour. You know what's even weirder? Offsets still are committed. It's
because the line you linked me doesn't interrupt the offset commit, it just
throws a warning.
Now the reason I would see that line being triggered right away is when
now >= commitTimeoutMs
now >= commitStarted +
workerConfig.getLong(WorkerConfig.OFFSET_COMMIT_TIMEOUT_MS_CONFIG)
I believe the setting workerConfig.getLong should return something 60000 ms, so
it hints towards commitStarted being corrupted.
And I think I just found your issue.....
commitTimeoutMs happens before the try block.
so no matter what happens in the commitOffsets(now, false); , it won't impact
the comparison:
if (committing && now >= commitTimeoutMs)
And that's the bug :)
> Kafka Connect: Offset committing times out before expected
> ----------------------------------------------------------
>
> Key: KAFKA-4942
> URL: https://issues.apache.org/jira/browse/KAFKA-4942
> Project: Kafka
> Issue Type: Bug
> Components: KafkaConnect
> Affects Versions: 0.10.2.0
> Reporter: Stephane Maarek
>
> On Kafka 0.10.2.0
> I run a connector that deals with a lot of data, in a kafka connect cluster
> When the offsets are getting committed, I get the following:
> {code}
> [2017-03-23 03:56:25,134] INFO WorkerSinkTask{id=MyConnector-1} Committing
> offsets (org.apache.kafka.connect.runtime.WorkerSinkTask)
> [2017-03-23 03:56:25,135] WARN Commit of WorkerSinkTask{id=MyConnector-1}
> offsets timed out (org.apache.kafka.connect.runtime.WorkerSinkTask)
> {code}
> If you look at the timestamps, they're 1 ms apart. My settings are the
> following:
> {code}
> offset.flush.interval.ms = 120000
> offset.flush.timeout.ms = 60000
> offset.storage.topic = _connect_offsets
> {code}
> It seems the offset flush timeout setting is completely ignored for the look
> of the logs. I would expect the timeout message to happen 60 seconds after
> the commit offset INFO message, not 1 millisecond later.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
