On a related note, I'd also minimize any kind of executor movement. I.e., once an executor is spawned and data cached in the executor, I want that executor to live all the way till the job is finished, or the machine fails in a fatal manner.
What would be the best way to ensure that this is the case? On Thu, Jun 5, 2014 at 3:13 PM, Sung Hwan Chung <coded...@cs.stanford.edu> wrote: > I noticed that sometimes tasks would switch from PROCESS_LOCAL (I'd assume > that this means fully cached) to NODE_LOCAL or even RACK_LOCAL. > > When these happen things get extremely slow. > > Does this mean that the executor got terminated and restarted? > > Is there a way to prevent this from happening (barring the machine > actually going down, I'd rather stick with the same process)? >
