Github user jerryshao commented on the pull request:
https://github.com/apache/spark/pull/10821#issuecomment-196154044
Hi @vanzin , thanks a lot for your response. I just checked the branch-1.6,
looks like the behavior (attempt id) is actually changed, and this change is
introduced in #9182.
Originally `attemptId` is gotten from `spark.yarn.app.attemptId` which is
set in `ApplicationMaster`.
And in `ApplicationMaster`, the way to get `attemptId` is
`appAttemptId.getAttemptId().toString()`, so here the attemptId is "1", "2".
But this behavior is changed in master branch. Here we use the full
`attemptId` rather than attempt counter
[here](https://github.com/apache/spark/blob/master/yarn/src/main/scala/org/apache/spark/scheduler/cluster/YarnClusterSchedulerBackend.scala#L34).
So this affects not only the file name of event log, also the url of
history server's each application.
Here if we accept the new way of `attempId`, then this url link should be
updated to the new one. Oppositely if we treat this new way of `attemptId` as a
regression, then there's no issue here, all we should change is to loop back
the original `attemptId`.
What's your opinion? @vanzin .
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
