Github user sryza commented on the pull request:
https://github.com/apache/spark/pull/2746#issuecomment-60197171
So yeah it internally decrements the pending number to 8. The app can and
is expected to infer YARN has decremented the counter. Maybe TMI, but for
getting a grasp on it, it might be helpful to understand the race conditions
this approach exposes - i.e. there are situations where YARN can overallocate.
For example imagine you requested 10 and then you decide you want 11. YARN just
got 2 for you and decremented its counter to 8. You might tell YARN you want
11 before finding out about the 2 YARN is giving to you, which means you would
overwrite the 8 with 11. In the brief period before you can go back to YARN
and tell it you only want 9 now, it could conceivably give you 11 containers,
for a total of 13, which is more than you ever asked for. The app is expected
to handle these situations and release allocated containers that it doesn't
need.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
