Hi,
Can you check Kafka topic replication ? And leader information?
Regards,
Surendra M
-- Surendra Manchikanti
On Thu, Mar 17, 2016 at 7:28 PM, Ascot Moss wrote:
> Hi,
>
> I have a SparkStream (with Kafka) job, after running several days, it
> failed with following errors:
> ERROR DirectKa
Sounds like you're using one of the KafkaUtils.createDirectStream
overloads that needs to do some broker communication in order to even
construct the stream, because you aren't providing topicpartitions?
Just wrap your construction attempt in a try / catch and retry in
whatever way makes sense for
Hi,
I have a SparkStream (with Kafka) job, after running several days, it
failed with following errors:
ERROR DirectKafkaInputDStream:
ArrayBuffer(java.nio.channels.ClosedChannelException)
Any idea what would be wrong? will it be SparkStreaming buffer overflow
issue?
Regards
*** from the
Cody et. al,
I am seeing a similar error. I've increased the number of retries. Once
I've got a job up and running I'm seeing it retry correctly. However, I am
having trouble getting the job started - number of retries does not seem to
help with startup behavior.
Thoughts?
Regards,
Bryan Jeff
That's a networking error when the driver is attempting to contact
leaders to get the latest available offsets.
If it's a transient error, you can look at increasing the value of
spark.streaming.kafka.maxRetries, see
http://spark.apache.org/docs/latest/configuration.html
If it's not a transient