Re: Spark Streaming stateful operation to HBase

Hi, Check the number of records inside the DStream at a batch before you do the save. Gist the code with mapWithState and save? Jacek On 9 Jun 2016 7:58 a.m., "soumick dasgupta" wrote: Hi, I am using mapwithstate to keep the state and then ouput the result to

Spark Streaming stateful operation to HBase

Hi, I am using mapwithstate to keep the state and then ouput the result to HBase. The problem I am facing is when there are no files arriving, the RDD is still emitting the previous state result due to the checkpoint. Is there a way I can restrict not to write that result to HBase, i.e., when the