RE: Handle empty kafka in Spark Streaming

2016-06-15 Thread David Newberger
Newberger -Original Message- From: Yogesh Vyas [mailto:informy...@gmail.com] Sent: Wednesday, June 15, 2016 8:30 AM To: David Newberger Subject: Re: Handle empty kafka in Spark Streaming I am looking for something which checks the JavaPairReceiverInputDStreambefore further going for any

RE: Handle empty kafka in Spark Streaming

2016-06-15 Thread David Newberger
If you're asking how to handle no messages in a batch window then I would add an isEmpty check like: dStream.foreachRDD(rdd => { if (!rdd.isEmpty()) ... } Or something like that. David Newberger -Original Message- From: Yogesh Vyas [mailto:informy...@gmail.com] Sent: Wednesday,