Yes, there are a few bugs in the UI in the event of a node failure. The duplicated stages in both the active and completed tables should be fixed by this PR: https://github.com/apache/spark/pull/1262 The fact that the progress bar on the stages page displays an overflow (e.g. 5/4) is still an open issue, but a related PR fixed the tasks page side of it: https://github.com/apache/spark/pull/1236 (merged)
Keep reporting any additional anomalies you observe (or better yet, file a JIRA here <https://issues.apache.org/jira/browse/SPARK>)! 2014-07-10 7:09 GMT-07:00 Daniel Siegmann <daniel.siegm...@velos.io>: > One thing to keep in mind is that the progress bar doesn't take into > account tasks which are rerun. If you see 4/4 but the stage is still > active, click the stage name and look at the task list. That will show you > if any are actually running. When rerun tasks complete, it can result in > the number of successful tasks being greater than the number of total > tasks; e.g. the progress bar might display 5/4. > > Another bug is that a stage might complete and be moved to the completed > list, but if tasks are then rerun it will appear in both the completed and > active stages list. If it completes again, you will see that stage *twice* > in the completed stages list. > > Of course, you should only be seeing this behavior if things are going > wrong; a node failing, for example. > > > On Thu, Jul 10, 2014 at 4:21 AM, Haopu Wang <hw...@qilinsoft.com> wrote: > >> I'm running an App for hours in a standalone cluster. From the data >> injector and "Streaming" tab of web ui, it's running well. >> >> However, I see quite a lot of Active stages in web ui even some of them >> have all of their tasks completed. >> >> I attach a screenshot for your reference. >> >> Do you ever see this kind of behavior? >> >> > > > -- > Daniel Siegmann, Software Developer > Velos > Accelerating Machine Learning > > 440 NINTH AVENUE, 11TH FLOOR, NEW YORK, NY 10001 > E: daniel.siegm...@velos.io W: www.velos.io >
