Robert Metzger created FLINK-2049:
-------------------------------------
Summary: KafkaSink sporadically fails to send message
Key: FLINK-2049
URL: https://issues.apache.org/jira/browse/FLINK-2049
Project: Flink
Issue Type: Bug
Components: Kafka Connector, Streaming
Affects Versions: 0.9
Reporter: Robert Metzger
This test https://travis-ci.org/StephanEwen/incubator-flink/jobs/63147661
failed with:
{code}
10:38:22,415 ERROR org.apache.flink.streaming.runtime.tasks.StreamTask
- StreamOperator failed due to: java.lang.RuntimeException:
java.lang.RuntimeException: kafka.common.FailedToSendMessageException: Failed
to send messages after 10 tries.
at
org.apache.flink.streaming.api.operators.StreamOperator.callUserFunctionAndLogException(StreamOperator.java:142)
at
org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:34)
at
org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:139)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:562)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException:
kafka.common.FailedToSendMessageException: Failed to send messages after 10
tries.
at
org.apache.flink.streaming.api.operators.StreamOperator.callUserFunctionAndLogException(StreamOperator.java:142)
at
org.apache.flink.streaming.api.operators.ChainableStreamOperator.collect(ChainableStreamOperator.java:54)
at
org.apache.flink.streaming.api.collector.CollectorWrapper.collect(CollectorWrapper.java:39)
at
org.apache.flink.streaming.connectors.kafka.KafkaITCase$3.run(KafkaITCase.java:326)
at
org.apache.flink.streaming.api.operators.StreamSource.callUserFunction(StreamSource.java:40)
at
org.apache.flink.streaming.api.operators.StreamOperator.callUserFunctionAndLogException(StreamOperator.java:137)
... 4 more
Caused by: kafka.common.FailedToSendMessageException: Failed to send messages
after 10 tries.
at
kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:90)
at kafka.producer.Producer.send(Producer.scala:77)
at kafka.javaapi.producer.Producer.send(Producer.scala:33)
at
org.apache.flink.streaming.connectors.kafka.api.KafkaSink.invoke(KafkaSink.java:183)
at
org.apache.flink.streaming.api.operators.StreamSink.callUserFunction(StreamSink.java:39)
at
org.apache.flink.streaming.api.operators.StreamOperator.callUserFunctionAndLogException(StreamOperator.java:137)
... 9 more
{code}
I've extracted the relevant logs:
https://gist.github.com/rmetzger/ddbb0fead5efdd58a539.
The error comes from Kafka's producer code. We are not doing much in our Kafka
Sink, so I really think this is not really a flink issue.
When the issue occurs again, I'll write to the Kafka list so seek for help.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
