It seems to me they must communicate for joins, sorts, grouping, and so forth, where the original data partitioning needs to change. You could repeat your experiment for different code snippets. I'll bet it depends on what you do.
On Thu, Sep 22, 2016 at 8:54 AM, gusiri <dreame...@gmail.com> wrote: > Hi, > > When I increase the network latency among spark nodes, > > I see compute time (=executor computing time in Spark Web UI) also > increases. > > In the graph attached, left = latency 1ms vs right = latency 500ms. > > Is there any communication between worker and driver/master even 'during' > executor computing? or any idea on this result? > > > <http://apache-spark-user-list.1001560.n3.nabble.com/ > file/n27779/Screen_Shot_2016-09-21_at_5.png> > > > > > > Thank you very much in advance. > > //gusiri > > > > > -- > View this message in context: http://apache-spark-user-list. > 1001560.n3.nabble.com/Is-executor-computing-time- > affected-by-network-latency-tp27779.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe e-mail: user-unsubscr...@spark.apache.org > >
