Re: Number of batches in the Streaming Statics visualization screen

Yes, the data is stored in driver memory. Mehdi Ben Haj Abbes 于2016年1月29日星期五 18:13写道： > Thanks Terry for the quick answer. > > I did not tried it. Lets say I will increase the value to 2, what > side effect should I expect. In fact in the explanation of the property

Re: Number of batches in the Streaming Statics visualization screen

Thanks Terry for the quick answer. I did not tried it. Lets say I will increase the value to 2, what side effect should I expect. In fact in the explanation of the property "How many finished batches the Spark UI and status APIs remember before garbage collecting." So the data is stored in

Re: Number of batches in the Streaming Statics visualization screen

Hi Mehdi, Do you try a larger value of "spark.streaming.ui.retainedBatches"(default is 1000)? Regards, - Terry On Fri, Jan 29, 2016 at 5:45 PM, Mehdi Ben Haj Abbes wrote: > Hi folks, > > I have a streaming job running for more than 24 hours. It seems that there > is a

Number of batches in the Streaming Statics visualization screen

Hi folks, I have a streaming job running for more than 24 hours. It seems that there is a limit on the number of the batches displayed in the Streaming Statics visualization screen. For example if I would launch a job Friday I will not be able to have the statistics for what happened during