Hi Yash, Thank you for the response. Sorry it was not at distinct but it was at a join stage. It was a self join. There were no errors and the jobs were stuck at the step for a around 7 hrs, the last message that came through was .
*ShuffleBlockFetcherIterator: Started 4 remote fetches* Thanks, Ramesh On Fri, Apr 7, 2017 at 10:05 AM, Yash Sharma <yash...@gmail.com> wrote: > Hi Ramesh, > Could you share some logs please? pastebin ? dag view ? > Did you check for GC pauses if any. > > On Thu, 6 Apr 2017 at 21:55 Ramesh Krishnan <ramesh.154...@gmail.com> > wrote: > >> I have a use case of distinct on a dataframe. When i run the application >> is getting stuck at LINE *ShuffleBlockFetcherIterator: Started 4 remote >> fetches *forever. >> >> Can someone help . >> >> >> Thanks >> Ramesh >> >
