[
https://issues.apache.org/jira/browse/YARN-1024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13729839#comment-13729839
]
Sandy Ryza commented on YARN-1024:
----------------------------------
If I am used to running my single-threaded task on a fast core (let's say rated
at 250 YVCs), and then I migrate it to another cluster with slower cores (let's
say rated at 150 YVCs), and still request 250 YVCs, my task will run no faster
than if I had requested it with 150 YVCs. I won't get predictable performance,
and, from a scheduling perspective, I'd be better off requesting 150 YVCs on
the slower cluster.
In a single pcore-to-vcore world, if I know that my task is CPU-bound and uses
X threads, I know that each vcore I ask for up to X vcores will predictably
improve its performance, whatever cluster I am running on. In a world where
different cores have different YVCs, I don't get a clear concept of when I
should increase my YVCs requested, and the advantage of doing so depends mostly
on the cluster I am running on.
A virtual core definition based on processing power masks the fact that two 1.5
GHz cores mean something very different than three 1.0 GHz cores. And makes it
very hard to reason about how many virtual cores to request.
> Define a virtual core unambigiously
> -----------------------------------
>
> Key: YARN-1024
> URL: https://issues.apache.org/jira/browse/YARN-1024
> Project: Hadoop YARN
> Issue Type: Improvement
> Reporter: Arun C Murthy
> Assignee: Arun C Murthy
>
> We need to clearly define the meaning of a virtual core unambiguously so that
> it's easy to migrate applications between clusters.
> For e.g. here is Amazon EC2 definition of ECU:
> http://aws.amazon.com/ec2/faqs/#What_is_an_EC2_Compute_Unit_and_why_did_you_introduce_it
> Essentially we need to clearly define a YARN Virtual Core (YVC).
> Equivalently, we can use ECU itself: *One EC2 Compute Unit provides the
> equivalent CPU capacity of a 1.0-1.2 GHz 2007 Opteron or 2007 Xeon processor.*
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
