Re: Guaranteed processing orders of each batch in Spark Streaming

I guess the order is guaranteed unless you set the spark.streaming.concurrentJobs to a higher number than 1. Thanks Best Regards On Mon, Oct 19, 2015 at 12:28 PM, Renjie Liu wrote: > Hi, all: > I've read source code and it seems that there is no guarantee that the >

Guaranteed processing orders of each batch in Spark Streaming

Hi, all: I've read source code and it seems that there is no guarantee that the order of processing of each RDD is guaranteed since jobs are just submitted to a thread pool. I believe that this is quite important in streaming since updates should be ordered.