Gunnar Morling created KAFKA-7336:
-------------------------------------
Summary: Kafka Connect source task when producing record with
invalid topic name
Key: KAFKA-7336
URL: https://issues.apache.org/jira/browse/KAFKA-7336
Project: Kafka
Issue Type: Bug
Components: KafkaConnect
Affects Versions: 2.0.0
Reporter: Gunnar Morling
If a Kafka Connect source task returns a {{SourceRecord}} with an invalid topic
name (e.g. "dbserver1.inventory.test@data"), that source task hangs (presumably
indefinitely?) and doesn't continue it's polling loop. The log is flooded with
this message:
{code}
connect_1 | 2018-08-24 08:47:29,014 WARN || [Producer
clientId=producer-4] Error while fetching metadata with correlation id 833 :
{dbserver1.inventory.test@data=INVALID_TOPIC_EXCEPTION}
[org.apache.kafka.clients.NetworkClient]
{code}
The producer thread is stuck in the loop here:
{code}
KafkaProducer<K,V>.waitOnMetadata(String, Integer, long) line: 938
KafkaProducer<K,V>.doSend(ProducerRecord<K,V>, Callback) line: 823
KafkaProducer<K,V>.send(ProducerRecord<K,V>, Callback) line: 803
WorkerSourceTask.sendRecords() line: 318
WorkerSourceTask.execute() line: 228
WorkerSourceTask(WorkerTask).doRun() line: 175
WorkerSourceTask(WorkerTask).run() line: 219
Executors$RunnableAdapter<T>.call() line: 511
FutureTask<V>.run() line: 266
ThreadPoolExecutor.runWorker(ThreadPoolExecutor$Worker) line: 1149
ThreadPoolExecutor$Worker.run() line: 624
Thread.run() line: 748
{code}
This causes the task to remain in RUNNING state, but no further invocations of
{{poll()}} are done.
Of course we'll work around this and make sure to not produce records with
invalid topic names, but I think the source task should transition to FAILED
state in this case.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
