Github user sryza commented on the pull request:
https://github.com/apache/spark/pull/4818#issuecomment-76514683
My opinion is that we should make the default true, as the vanilla YARN
default of `FIFOScheduler` will run into this issue (though most vendor
distributions have a better default). There are no versions of YARN that will
return containers smaller than were requested, except in this weird situation
where the scheduler doesn't support CPU scheduling. I actually think it might
be better to avoid a config at all and always just avoid matching on CPU. It's
really hard to imagine any situation where it would actually benefit someone to
set the config to false. The only one I can think of is debugging incorrect
behavior in YARN, and, if we care about that, it would be better to just log
something.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
