Spark Streaming, Batch interval, Windows length and Sliding Interval settings

2016-05-04 Thread Mich Talebzadeh
Hi, Just wanted opinions on this. In Spark streaming the parameter val ssc = new StreamingContext(sparkConf, Seconds(n)) defines the batch or sample interval for the incoming streams In addition there is windows Length // window length - The duration of the window below that must be multiple

Re: Spark Streaming, Batch interval, Windows length and Sliding Interval settings

2016-05-05 Thread Mich Talebzadeh
Any ideas/experience on this? Dr Mich Talebzadeh LinkedIn * https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw * http://talebzadehmich.wordpress.com On 4 May 2016 at 21:45, Mic

RE: Spark Streaming, Batch interval, Windows length and Sliding Interval settings

2016-05-05 Thread Ryan Harris
] Sent: Thursday, May 05, 2016 4:26 AM To: user Subject: Re: Spark Streaming, Batch interval, Windows length and Sliding Interval settings Any ideas/experience on this? Dr Mich Talebzadeh LinkedIn https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdO