Re: Spark with External Shuffle Service - using saved shuffle files in the event of executor failure

2021-05-12 Thread Attila Zsolt Piros
Hello, I have answered it on the Stack Overflow. Best Regards, Attila On Wed, May 12, 2021 at 4:57 PM Chris Thomas wrote: > Hi, > > I am pretty confident I have observed Spark configured with the Shuffle > Service continuing to fetch shuffle files on a node in the event of > executor

Spark with External Shuffle Service - using saved shuffle files in the event of executor failure

2021-05-12 Thread Chris Thomas
Hi, I am pretty confident I have observed Spark configured with the Shuffle Service continuing to fetch shuffle files on a node in the event of executor failure, rather than recompute the shuffle files as happens without the Shuffle Service. Can anyone confirm this? (I have a SO questionÂ