Daniel Urban created KAFKA-14902:
------------------------------------
Summary: KafkaBasedLog infinite retries can lead to
StackOverflowError
Key: KAFKA-14902
URL: https://issues.apache.org/jira/browse/KAFKA-14902
Project: Kafka
Issue Type: Bug
Components: KafkaConnect
Reporter: Daniel Urban
Assignee: Daniel Urban
KafkaBasedLog subclasses use an infinite retry on producer sends, using a
callback. Sometimes, when specific errors are encountered, the callback is
invoked in the send call, on the calling thread. If this happens enough times,
a stack overflow happens.
Example stacktrace from 2.5 (but the newest code can also encounter the same):
{code:java}
2023-01-14 12:48:23,487 ERROR org.apache.kafka.connect.runtime.WorkerTask:
WorkerSourceTask{id=MirrorSourceConnector-1} Task threw an uncaught and
unrecoverable exception java.lang.StackOverflowError: null at
org.apache.kafka.common.metrics.stats.SampledStat.record(SampledStat.java:50)
at org.apache.kafka.common.metrics.stats.Rate.record(Rate.java:60) at
org.apache.kafka.common.metrics.stats.Meter.record(Meter.java:80) at
org.apache.kafka.common.metrics.Sensor.record(Sensor.java:188) at
org.apache.kafka.common.metrics.Sensor.record(Sensor.java:178) at
org.apache.kafka.clients.producer.internals.BufferPool.recordWaitTime(BufferPool.java:202)
at
org.apache.kafka.clients.producer.internals.BufferPool.allocate(BufferPool.java:147)
at
org.apache.kafka.clients.producer.internals.RecordAccumulator.append(RecordAccumulator.java:221)
at
org.apache.kafka.clients.producer.KafkaProducer.doSend(KafkaProducer.java:941)
at org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:862)
at org.apache.kafka.connect.util.KafkaBasedLog.send(KafkaBasedLog.java:238) at
org.apache.kafka.connect.storage.KafkaStatusBackingStore$4.onCompletion(KafkaStatusBackingStore.java:298)
... at
org.apache.kafka.clients.producer.KafkaProducer.doSend(KafkaProducer.java:959)
at org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:862)
at org.apache.kafka.connect.util.KafkaBasedLog.send(KafkaBasedLog.java:238) at
org.apache.kafka.connect.storage.KafkaStatusBackingStore$4.onCompletion(KafkaStatusBackingStore.java:298){code}
Note the repeated KafkaProducer.send -> KafkaProducer.doSend ->
KafkaStatusBackingStore$4.onCompletion calls, causing the issue.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
