subject:"couple naive questions on Spark Structured Streaming"

Re: couple naive questions on Spark Structured Streaming

2017-05-22 Thread kant kodali

HI Burak, My response is inline. Thanks a lot! On Mon, May 22, 2017 at 9:26 AM, Burak Yavuz wrote: > Hi Kant, > >> >> >> 1. Can we use Spark Structured Streaming for stateless transformations >> just like we would do with DStreams or Spark Structured Streaming is only >>

Re: couple naive questions on Spark Structured Streaming

2017-05-22 Thread Burak Yavuz

Hi Kant, > > > 1. Can we use Spark Structured Streaming for stateless transformations > just like we would do with DStreams or Spark Structured Streaming is only > meant for stateful computations? > Of course you can do stateless transformations. Any map, filter, select, type of transformation

couple naive questions on Spark Structured Streaming

2017-05-20 Thread kant kodali

Hi, 1. Can we use Spark Structured Streaming for stateless transformations just like we would do with DStreams or Spark Structured Streaming is only meant for stateful computations? 2. When we use groupBy and Window operations for event time processing and specify a watermark does this mean the