I tried below code but still carrying out the action even though there is no new data.
JavaPairInputDStream<LongWritable, Text> input = ssc.fileStream(iFolder, LongWritable.class,Text.class, TextInputFormat.class); if(input != null){ //do some action if it is not empty } On 21 October 2015 at 18:00, diplomatic Guru <diplomaticg...@gmail.com> wrote: > > Hello All, > > I have a Spark Streaming job that should do some action only if the RDD > is not empty. This can be done easily with the spark batch RDD as I could > .take(1) and check whether it is empty or not. But this cannot been done > in Spark Streaming DStrem > > > JavaPairInputDStream<LongWritable, Text> input = ssc.fileStream(iFolder, > LongWritable.class,Text.class, TextInputFormat.class); > > if(inputLines!=null){ > //do some action if it is not empty > } > > Any ideas please? > > > >