run for a while, it is easy to see
when some classes' instance count only increases forever, which in turn
makes it easier to find the memory leak.
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Long-running-Spark-Streaming-Job-increasing-executing-time-per
Thats quite odd. Yes, with checkpoint the lineage does not increase. Can
you tell which stage is the processing of each batch is causing the
increase in the processing time?
Also, what is the batch interval, and checkpoint interval?
TD
On Thu, Jun 19, 2014 at 8:45 AM, Skogberg, Fredrik
Hi TD,
Thats quite odd. Yes, with checkpoint the lineage does not increase. Can you
tell which stage is the processing of each batch is causing the increase in
the processing time?
I haven’t been able to determine exactly what stage that is causing the
increase in processing time. Any
