[
https://issues.apache.org/jira/browse/SPARK-36582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sungpeo Kook updated SPARK-36582:
---------------------------------
Description:
Current historypage show a attemptId column in case of hasMultipleAttempts is
true.
if hasMultipleAttempts is false, remove the attemptId column.
But, applications in yarn could be failed even not logged to spark application
history.
(application_1628518360417_0029 is normal case. Don't have to consider it)
*application_1628518360417_0028*'s attemptId is 2, but size of attempt in spark
history is 1.
(First try failed to starting spark driver in cluster mode, and second one was
succeeded.)
For this case, the attemptId column is needed.
{code:java}
[
{
"id": "application_1628518360417_0029",
"name": "Spark Pi",
"attempts": [
{
"attemptId": "1",
"startTime": "2021-08-25T05:20:15.521GMT",
"endTime": "2021-08-25T05:20:30.398GMT",
"lastUpdated": "2021-08-25T05:20:30.475GMT",
"duration": 14877,
"sparkUser": "elixir-kook",
"completed": true,
"appSparkVersion": "2.4.5",
"endTimeEpoch": 1629868830398,
"lastUpdatedEpoch": 1629868830475,
"startTimeEpoch": 1629868815521
}
]
},
{
"id": "application_1628518360417_0028",
"name": "Spark Pi",
"attempts": [
{
"attemptId": "2",
"startTime": "2021-08-25T05:19:22.850GMT",
"endTime": "2021-08-25T05:19:44.662GMT",
"lastUpdated": "2021-08-25T05:19:44.726GMT",
"duration": 21812,
"sparkUser": "elixir-kook",
"completed": true,
"appSparkVersion": "2.4.5",
"endTimeEpoch": 1629868784662,
"lastUpdatedEpoch": 1629868784726,
"startTimeEpoch": 1629868762850
}
]
},
....
]
{code}
!Screen Shot 2021-08-25 at 2.35.30 PM.png!
attemptId should be exist.
was:
Current historypage show a attemptId column in case of hasMultipleAttempts is
true.
if hasMultipleAttempts is false, remove the attemptId column.
But, applications in yarn could be failed even not logged to spark application
history.
(application_1628518360417_0029 is normal case. Don't have to consider it)
*application_1628518360417_0028*'s attemptId is 2, but size of attempt in spark
history is 1.
(First try failed to starting spark driver in cluster mode, and second one was
succeeded.)
For this case, the attemptId column is needed.
{code:java}
[
{
"id": "application_1628518360417_0029",
"name": "Spark Pi",
"attempts": [
{
"attemptId": "1",
"startTime": "2021-08-25T05:20:15.521GMT",
"endTime": "2021-08-25T05:20:30.398GMT",
"lastUpdated": "2021-08-25T05:20:30.475GMT",
"duration": 14877,
"sparkUser": "elixir-kook",
"completed": true,
"appSparkVersion": "2.4.5",
"endTimeEpoch": 1629868830398,
"lastUpdatedEpoch": 1629868830475,
"startTimeEpoch": 1629868815521
}
]
},
{
"id": "application_1628518360417_0028",
"name": "Spark Pi",
"attempts": [
{
"attemptId": "2",
"startTime": "2021-08-25T05:19:22.850GMT",
"endTime": "2021-08-25T05:19:44.662GMT",
"lastUpdated": "2021-08-25T05:19:44.726GMT",
"duration": 21812,
"sparkUser": "elixir-kook",
"completed": true,
"appSparkVersion": "2.4.5",
"endTimeEpoch": 1629868784662,
"lastUpdatedEpoch": 1629868784726,
"startTimeEpoch": 1629868762850
}
]
},
....
]
{code}
attemptId should be exist.
> Spark HistoryPage may show 'NotFound' in some multiple attempts cases
> ---------------------------------------------------------------------
>
> Key: SPARK-36582
> URL: https://issues.apache.org/jira/browse/SPARK-36582
> Project: Spark
> Issue Type: Bug
> Components: Web UI
> Affects Versions: 2.4.8, 3.1.2
> Reporter: Sungpeo Kook
> Priority: Major
> Attachments: Screen Shot 2021-08-25 at 2.35.30 PM.png
>
>
> Current historypage show a attemptId column in case of hasMultipleAttempts is
> true.
> if hasMultipleAttempts is false, remove the attemptId column.
>
> But, applications in yarn could be failed even not logged to spark
> application history.
>
> (application_1628518360417_0029 is normal case. Don't have to consider it)
> *application_1628518360417_0028*'s attemptId is 2, but size of attempt in
> spark history is 1.
> (First try failed to starting spark driver in cluster mode, and second one
> was succeeded.)
> For this case, the attemptId column is needed.
>
> {code:java}
> [
> {
> "id": "application_1628518360417_0029",
> "name": "Spark Pi",
> "attempts": [
> {
> "attemptId": "1",
> "startTime": "2021-08-25T05:20:15.521GMT",
> "endTime": "2021-08-25T05:20:30.398GMT",
> "lastUpdated": "2021-08-25T05:20:30.475GMT",
> "duration": 14877,
> "sparkUser": "elixir-kook",
> "completed": true,
> "appSparkVersion": "2.4.5",
> "endTimeEpoch": 1629868830398,
> "lastUpdatedEpoch": 1629868830475,
> "startTimeEpoch": 1629868815521
> }
> ]
> },
> {
> "id": "application_1628518360417_0028",
> "name": "Spark Pi",
> "attempts": [
> {
> "attemptId": "2",
> "startTime": "2021-08-25T05:19:22.850GMT",
> "endTime": "2021-08-25T05:19:44.662GMT",
> "lastUpdated": "2021-08-25T05:19:44.726GMT",
> "duration": 21812,
> "sparkUser": "elixir-kook",
> "completed": true,
> "appSparkVersion": "2.4.5",
> "endTimeEpoch": 1629868784662,
> "lastUpdatedEpoch": 1629868784726,
> "startTimeEpoch": 1629868762850
> }
> ]
> },
> ....
> ]
> {code}
> !Screen Shot 2021-08-25 at 2.35.30 PM.png!
>
> attemptId should be exist.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
