Re: ERROR ArrayBuffer(java.nio.channels.ClosedChannelException

Hi, Can you check Kafka topic replication ? And leader information? Regards, Surendra M -- Surendra Manchikanti On Thu, Mar 17, 2016 at 7:28 PM, Ascot Moss wrote: > Hi, > > I have a SparkStream (with Kafka) job, after running several days, it > failed with following

Re: ERROR ArrayBuffer(java.nio.channels.ClosedChannelException

Sounds like you're using one of the KafkaUtils.createDirectStream overloads that needs to do some broker communication in order to even construct the stream, because you aren't providing topicpartitions? Just wrap your construction attempt in a try / catch and retry in whatever way makes sense for

ERROR ArrayBuffer(java.nio.channels.ClosedChannelException

Hi, I have a SparkStream (with Kafka) job, after running several days, it failed with following errors: ERROR DirectKafkaInputDStream: ArrayBuffer(java.nio.channels.ClosedChannelException) Any idea what would be wrong? will it be SparkStreaming buffer overflow issue? Regards *** from

Re: ERROR ArrayBuffer(java.nio.channels.ClosedChannelException

Cody et. al, I am seeing a similar error. I've increased the number of retries. Once I've got a job up and running I'm seeing it retry correctly. However, I am having trouble getting the job started - number of retries does not seem to help with startup behavior. Thoughts? Regards, Bryan

Re: ERROR ArrayBuffer(java.nio.channels.ClosedChannelException

That's a networking error when the driver is attempting to contact leaders to get the latest available offsets. If it's a transient error, you can look at increasing the value of spark.streaming.kafka.maxRetries, see http://spark.apache.org/docs/latest/configuration.html If it's not a transient