Re: distinct query getting stuck at ShuffleBlockFetcherIterator

Hi Yash, Thank you for the response. Sorry it was not at distinct but it was at a join stage. It was a self join. There were no errors and the jobs were stuck at the step for a around 7 hrs, the last message that came through was . *ShuffleBlockFetcherIterator: Started 4 remote fetches* Thanks,

Re: distinct query getting stuck at ShuffleBlockFetcherIterator

Hi Ramesh, Could you share some logs please? pastebin ? dag view ? Did you check for GC pauses if any. On Thu, 6 Apr 2017 at 21:55 Ramesh Krishnan wrote: > I have a use case of distinct on a dataframe. When i run the application > is getting stuck at LINE

distinct query getting stuck at ShuffleBlockFetcherIterator

I have a use case of distinct on a dataframe. When i run the application is getting stuck at LINE *ShuffleBlockFetcherIterator: Started 4 remote fetches *forever. Can someone help . Thanks Ramesh