[
https://issues.apache.org/jira/browse/SPARK-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Or updated SPARK-2147:
-----------------------------
Description:
When an application exits cleanly, the Master will remove all executors from
the application's ApplicationInfo, causing the historic "Completed
Applications" page to report that there were no executors associated with that
application.
On the contrary, if the application exits uncleanly, then the Master will
remove the application FIRST, and will not actually remove the executors from
the ApplicationInfo page. This causes the executors to show up correctly in the
"Completed Applications" page.
The correct behavior would probably be to gather a history of all executors (so
we'd retain executors that we had at one point but were removed during the
job), and not remove lost executors.
was:
When an application exists cleanly, the Master will remove all executors from
the application's ApplicationInfo, causing the historic "Completed
Applications" page to report that there were no executors associated with that
application.
On the contrary, if the application exits uncleanly, then the Master will
remove the application FIRST, and will not actually remove the executors from
the ApplicationInfo page. This causes the executors to show up correctly in the
"Completed Applications" page.
The correct behavior would probably be to gather a history of all executors (so
we'd retain executors that we had at one point but were removed during the
job), and not remove lost executors.
> Master UI forgets about Executors when application exits cleanly
> ----------------------------------------------------------------
>
> Key: SPARK-2147
> URL: https://issues.apache.org/jira/browse/SPARK-2147
> Project: Spark
> Issue Type: Bug
> Components: Web UI
> Affects Versions: 1.0.0
> Reporter: Aaron Davidson
> Assignee: Andrew Or
>
> When an application exits cleanly, the Master will remove all executors from
> the application's ApplicationInfo, causing the historic "Completed
> Applications" page to report that there were no executors associated with
> that application.
> On the contrary, if the application exits uncleanly, then the Master will
> remove the application FIRST, and will not actually remove the executors from
> the ApplicationInfo page. This causes the executors to show up correctly in
> the "Completed Applications" page.
> The correct behavior would probably be to gather a history of all executors
> (so we'd retain executors that we had at one point but were removed during
> the job), and not remove lost executors.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
