The latency to start a Spark Job is nowhere close to 2-4 seconds under typical conditions. You appear to be creating a new Spark Application everytime instead of running multiple Jobs in one Application.
On Fri, Jul 6, 2018 at 3:12 AM Tien Dat <tphan....@gmail.com> wrote: > Dear Timothy, > > It works like a charm now. > > BTW (don't judge me if I am to greedy :-)), the latency to start a Spark > job > is around 2-4 seconds, unless I am not aware of some awesome optimization > on > Spark. Do you know if Spark community is working on reducing this latency? > > Best > > > > -- > Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/ > > --------------------------------------------------------------------- > To unsubscribe e-mail: user-unsubscr...@spark.apache.org > >
