jobs much slower in cluster mode vs local

Hello, In general, I am usually able to run spark submit jobs in local mode, in a 32-cores node with plenty of memory ram. The performance is significantly faster in local mode than when using a cluster of spark workers. How can this be explained and what measures can one take in order to

Re: jobs much slower in cluster mode vs local

Hi, you can try to use spark job server and submit jobs to it. The thing is that the most expensive part is context creation. J. 2016-01-15 15:28 GMT+01:00 : > Hello, > > In general, I am usually able to run spark submit jobs in local mode, in a > 32-cores node

RE: jobs much slower in cluster mode vs local

January 2016 14:29 To: user@spark.apache.org Subject: jobs much slower in cluster mode vs local Hello, In general, I am usually able to run spark submit jobs in local mode, in a 32-cores node with plenty of memory ram. The performance is significantly faster in local mode than when using

RE: jobs much slower in cluster mode vs local

Thank you, this looks useful indeed for what I have in mind. Saif From: Jiří Syrový [mailto:syrovy.j...@gmail.com] Sent: Friday, January 15, 2016 12:06 PM To: Ellafi, Saif A. Cc: user@spark.apache.org Subject: Re: jobs much slower in cluster mode vs local Hi, you can try to use spark job