Re: Spark job taking 10s to allocate executors and memory before submitting job

Sounds like such a small job , if you running in on a cluster have you consider simply running it locally (master = local) ? On Wed, Sep 27, 2017 at 7:06 AM, navneet sharma wrote: > Hi, > > I am running spark job taking total 18s, in that 8 seconds for actual >

Spark job taking 10s to allocate executors and memory before submitting job

Hi, I am running spark job taking total 18s, in that 8 seconds for actual processing logic(business logic) and 10s for allocating executors and memory. How to reduce initial time. Any ideas how to reduce time before spark job goes to submit state. thanks, Navneet Sharma