Thank you, this looks useful indeed for what I have in mind.
Saif
From: Jiří Syrový [mailto:syrovy.j...@gmail.com]
Sent: Friday, January 15, 2016 12:06 PM
To: Ellafi, Saif A.
Cc: user@spark.apache.org
Subject: Re: jobs much slower in cluster mode vs local
Hi,
you can try to use spark job
Hi,
you can try to use spark job server and submit jobs to it. The thing is
that the most expensive part is context creation.
J.
2016-01-15 15:28 GMT+01:00 :
> Hello,
>
> In general, I am usually able to run spark submit jobs in local mode, in a
> 32-cores node with plenty of memory ram. The pe
]
Sent: 15 January 2016 14:29
To: user@spark.apache.org
Subject: jobs much slower in cluster mode vs local
Hello,
In general, I am usually able to run spark submit jobs in local mode, in a
32-cores node with plenty of memory ram. The performance is significantly
faster in local mode than wh
Hello,
In general, I am usually able to run spark submit jobs in local mode, in a
32-cores node with plenty of memory ram. The performance is significantly
faster in local mode than when using a cluster of spark workers.
How can this be explained and what measures can one take in order to impro
