Hello
Dynamic allocation feature allows you to add executors and scale
computation power. This is great, however, I feel like we also need a way
to dynamically scale storage. Currently, if the disk is not able to hold
the spilled/shuffle data, the job is aborted causing frustration and loss
of tim
If people agree that is desired, I am willing to submit a SIP for this and
find time to work on it.
Thanks,
Aniket
On Sun, Nov 6, 2016 at 1:06 PM Aniket Bhatnagar
wrote:
> Hello
>
> Dynamic allocation feature allows you to add executors and scale
> computation power. This is great, however, I f