Re: help with streaming batch interval question needed

2018-05-25 Thread Peter Liu
Hi Jacek, This is exact what i'm looking for. Thanks!! Also thanks for the link. I just noticed that I can unfold the link of trigger and see the examples in java and scala languages - what a general help for a new comer :-)

Re: help with streaming batch interval question needed

2018-05-25 Thread Jacek Laskowski
Hi Peter, > Basically I need to find a way to set the batch-interval in (b), similar as in (a) below. That's trigger method on DataStreamWriter. http://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.sql.streaming.DataStreamWriter import

re: help with streaming batch interval question needed

2018-05-24 Thread Peter Liu
Hi there, from my apache spark streaming website (see links below), - the batch-interval is set when a spark StreamingContext is constructed (see example (a) quoted below) - the StreamingContext is available in older and new Spark version (v1.6, v2.2 to v2.3.0) (see