Since spark can run multiple tasks in one executor, so I am curious to know how does spark manage memory across these tasks. Say if one executor takes 1GB memory, then if this executor can run 10 tasks simultaneously, then each task can consume 100MB on average. Do I understand it correctly ? It doesn't make sense to me that spark run multiple tasks in one executor.
- How does spark manage the memory of executor with multiple... canan chen
- RE: How does spark manage the memory of executor with... Evo Eftimov
- Re: How does spark manage the memory of executor ... canan chen
- RE: How does spark manage the memory of execu... Evo Eftimov
- Re: How does spark manage the memory of e... canan chen
- Re: How does spark manage the memory... Arush Kharbanda
- Re: How does spark manage the memory of executor with... Evo Eftimov