Hi Gerard,
Excellent, indeed your inputs helped. Thank you for the quick reply.
I modified the code based on inputs.
Now the application starts and it reads from the topic. Now we stream like
50,000 messages on the Kafka topic.
After a while we terminate the application using YARN kill and
Hi Arpan,
The error suggests that the streaming context has been started with
streamingContext.start() and after that statement, some other
dstream operations have been attempted.
A suggested pattern to manage the offsets is the following:
var offsetRanges: Array[OffsetRanger] = _
//create
Hi all,
In our cluster we have Kafka 0.10.1 and Spark 2.1.0. We are trying to store
the offsets in Kafka in order to achieve restartability of the streaming
application. ( Using checkpoints, I already implemented, we will require to
change code in production hence checkpoint won't work)
Checking
