Re: Structured Streaming: multiple sinks

2017-08-25 Thread Tathagata Das
> 3. triggerExecution is effectively end-to-end processing time for the >>> micro-batch, note all other durations sum closely to triggerExecution, >>> there >>> is a little slippage based on book-keeping activities in StreamExecution. >>> >> &

Re: Structured Streaming: multiple sinks

2017-08-25 Thread Chris Bowden
e the DataFrame. > > >> 3. triggerExecution is effectively end-to-end processing time for the >> micro-batch, note all other durations sum closely to triggerExecution, >> there >> is a little slippage based on book-keeping activities in StreamExecution. >> >

Re: Structured Streaming: multiple sinks

2017-08-25 Thread Tathagata Das
re > is a little slippage based on book-keeping activities in StreamExecution. > Yes. Precisely. > > > > -- > View this message in context: http://apache-spark-user-list. > 1001560.n3.nabble.com/Structured-Streaming-multiple- > sinks-tp29056p29105.html > Se

Re: Structured Streaming: multiple sinks

2017-08-24 Thread cbowden
processing time for the micro-batch, note all other durations sum closely to triggerExecution, there is a little slippage based on book-keeping activities in StreamExecution. -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Structured-Streaming-multiple-sinks

Structured Streaming: multiple sinks

2017-08-11 Thread aravias
ionMs" : { "addBatch" : 2263426, "getBatch" : 12, "getOffset" : 273, "queryPlanning" : 13, "triggerExecution" : 2264288, "walCommit" : 552 }, regards aravias -- View this message in context: http://apache-spark-user-list.