Q: In Spark Streaming if your DStream transformation and output action take
longer than the batch duration will the system process the next batch in
another thread? Or will it just wait until the first batch’s RDD is
processed? In other words does it build up a queue of buffered RDDs
awaiting
: Spark Streaming Threading Model
Q: In Spark Streaming if your DStream transformation and output action take
longer than the batch duration will the system process the next batch in
another thread? Or will it just wait until the first batch’s RDD is processed?
In other words does it build up
:
Batches will wait for the previous batch to finish. The monitoring
console will show you the backlog of waiting batches.
From: Asim Jalis asimja...@gmail.com
Date: Friday, December 19, 2014 at 1:16 PM
To: user user@spark.apache.org
Subject: Spark Streaming Threading Model
Q: In Spark
