Re: question about barrier execution mode in Spark 2.4.0

2018-12-19 Thread Xiangrui Meng
On Mon, Nov 12, 2018 at 7:33 AM Joe wrote: > Hello, > I was reading Spark 2.4.0 release docs and I'd like to find out more > about barrier execution mode. > In particular I'd like to know what happens when number of partitions > exceeds number of nodes (which I think is allowed, Spark tuning doc

question about barrier execution mode in Spark 2.4.0

2018-11-12 Thread Joe
Hello, I was reading Spark 2.4.0 release docs and I'd like to find out more about barrier execution mode. In particular I'd like to know what happens when number of partitions exceeds number of nodes (which I think is allowed, Spark tuning doc mentions that)? Does Spark guarantee that all