Hello, today I my kafka producer stop working, here is the story:
Spark: v1.6.2
Kafka broker: v0.9.0.1
Kafka client: tried 0.9.x and 0.10.x

In a spark batch job, I created a kafka producer and tried to send some
messages to kafka broker, it works without any problem for > one month
until today :(

My spark job just hangs, and after wait some time, the broker logs follwong
exception:

ERROR Processor got uncaught exception. (kafka.network.Processor)
java.lang.ArrayIndexOutOfBoundsException: 18
at org.apache.kafka.common.protocol.ApiKeys.forId(ApiKeys.java:68)
at
org.apache.kafka.common.requests.AbstractRequest.getRequest(AbstractRequest.java:39)
at kafka.network.RequestChannel$Request.<init>(RequestChannel.scala:79)
at kafka.network.Processor$$anonfun$run$11.apply(SocketServer.scala:426)
at kafka.network.Processor$$anonfun$run$11.apply(SocketServer.scala:421)
at scala.collection.Iterator$class.foreach(Iterator.scala:742)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1194)
at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
at kafka.network.Processor.run(SocketServer.scala:421)
at java.lang.Thread.run(Thread.java:745)

I tried switching clients, using another topic, remove all consumer groups
(in zk), with no luck.

Someone tells me maybe it's related to this bug
https://issues.apache.org/jira/browse/KAFKA-3547, is it true?

And if I upgrade my broker to v0.10.0.0, will it provide backward
compatibility with my old clients (v0.9.0.1) clients? What about kafka
streams used in spark?

Thanks in advance!

Reply via email to