@Eran, are Server 1 and Server 2 both part of the same cluster / do they have similar positions in the network topology w.r.t the Spark executors? If Server 1 had fast network access to the executors but Server 2 was across a WAN then I'd expect the job to run slower from Server 2 duet to the extra network latency / reduced bandwidth. This is assuming that you're running the driver in non-cluster deploy mode (so the driver process runs on the machine which submitted the job).
On Mon, Dec 21, 2015 at 1:30 PM, Igor Berman <igor.ber...@gmail.com> wrote: > look for differences: packages versions, cpu/network/memory diff etc etc > > > On 21 December 2015 at 14:53, Eran Witkon <eranwit...@gmail.com> wrote: > >> Hi, >> I know it is a wide question but can you think of reasons why a pyspark >> job which runs on from server 1 using user 1 will run faster then the same >> job when running on server 2 with user 1 >> Eran >> > >
