[
https://issues.apache.org/jira/browse/FLINK-2213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14642702#comment-14642702
]
Robert Metzger commented on FLINK-2213:
---------------------------------------
It does not affect the amount of CPU resources you can use with your container
(at least not with the {{DefaultContainerExecutor}} of YARN, which is the one
I've seen everywhere in the wild (including EMR and bdutil)). With the
{{LinuxContainerExecutor}}, the number of cores will indeed be limited to 1 [1].
I once tried configuring the LCE on google compute but I gave up at some point.
I don't know how often it is used in the wild.
Fixing this should be a one-line change.
I'm not sure how the different YARN schedulers are taking the vcores into
account when scheduling containers. Maybe it would be good to add a
configuration parameter allowing users to hard-core the vcores to a fixed value
(say "1").
[1] http://riccomini.name/posts/hadoop/2013-06-14-yarn-with-cgroups/
> Configure number of vcores
> --------------------------
>
> Key: FLINK-2213
> URL: https://issues.apache.org/jira/browse/FLINK-2213
> Project: Flink
> Issue Type: Improvement
> Components: YARN Client
> Affects Versions: master
> Reporter: Ufuk Celebi
>
> Currently, the number of vcores per YARN container is set to 1.
> It is desirable to allow configuring this value. As a simple heuristic it
> makes sense to at least set it to the number of slots per container.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
