Re: a spark job hangs 20 hours and it shows 2 tasks not finished in stage page but all tasks shows finished or failed in task page

You can click on the description of stage. It redirects to the tasks page. Please share that. On Thu, May 7, 2020, 7:44 AM zhangliyun wrote: > Hi all: > i want to ask a question, it seems that my spark job hangs as 20+ hours. > in the spark history log, it shows 8999 completed task while 2

a spark job hangs 20 hours and it shows 2 tasks not finished in stage page but all tasks shows finished or failed in task page

Hi all: i want to ask a question, it seems that my spark job hangs as 20+ hours. in the spark history log, it shows 8999 completed task while 2 not finished. but when i go to the tasks page. i did not find any running tasks. All tasks are failed or Successful. I guess it seems that all

Screen Shot 2020-05-07 at 6.35.43 AM

Hi all: i want to ask a question, it seems that my spark job hangs as 20+ hours. in the spark history log, it shows 8999 completed task while 2 not finished. but when i go to the tasks page. i did not find any running tasks. All tasks are failed or Successful. I guess it seems that all

Re: RDD order guarantees

Thanks a lot for the reply Steve! If you don't see a way to fix this in Spark itself, then I will try to improve the docs. Antonin On 06/05/2020 17:19, Steve Loughran wrote: > > > On Tue, 7 Apr 2020 at 15:26, Antonin Delpeuch > wrote: > > Hi, > >

Re: RDD order guarantees

On Tue, 7 Apr 2020 at 15:26, Antonin Delpeuch wrote: > Hi, > > Sorry to dig out this thread but this bug is still present. > > The fix proposed in this thread (creating a new FileSystem implementation > which sorts listed files) was rejected, with the suggestion that it is the >