Re: Spark Streaming: Is it possible to schedule multiple active batches?
spark.streaming.concurrentJobs may help. Experimental according to TD from an older thread here http://stackoverflow.com/questions/23528006/how-jobs-are-assigned-to-executors-in-spark-streaming On Sat, Feb 20, 2016 at 11:24 AM, Jorge Rodriguez wrote: > > Is it possible to have the scheduler sche
Spark Streaming: Is it possible to schedule multiple active batches?
Is it possible to have the scheduler schedule the next batch even if the previous batch has not completed yet? I'd like to schedule up to 3 batches at the same time if this is possible.