[
https://issues.apache.org/jira/browse/YARN-6245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15954180#comment-15954180
]
Wangda Tan commented on YARN-6245:
----------------------------------
bq. I see the difference between LightResource and FinalResource and the need
for both of them. Is the latter an immutable version of the former?
Yes, you can think the FinalResource is an immutable version and backed by
LightResource.
bq. I am not sure I fully understand. Mind posting some pseudo code for how
this operation should look with the API you have in mind?
Sure, in my example: {{ d = (res_a * float_x + res_b) / res_c}}. If all
resources are immutable, computation becomes:
{code}
# Please note that immutable_{op} means do the operation and return
# an immutable object.
temp_1 = immutable_multiply(res_a, float_x);
temp_2 = immutable_add(temp_1 + res_b);
d = divide(temp_2, res_c);
{code}
It creates 2 temp instances during the computation.
And if we can have {{LightResource}}, the computation becomes:
{code}
light_1 = light_multiply(res_a, float_x);
light_add_to(light_1, res_b);
d = divide(light_1, res_c);
{code}
Which only creates 1 temp instance.
> Add FinalResource object to reduce overhead of Resource class instancing
> ------------------------------------------------------------------------
>
> Key: YARN-6245
> URL: https://issues.apache.org/jira/browse/YARN-6245
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Wangda Tan
> Attachments: observable-resource.patch,
> YARN-6245.preliminary-staled.1.patch
>
>
> There're lots of Resource object creation in YARN Scheduler, since Resource
> object is backed by protobuf, creation of such objects is expensive and
> becomes bottleneck.
> To address the problem, we can introduce a FinalResource (Is it better to
> call it ImmutableResource?) object, which is not backed by PBImpl. We can use
> this object in frequent invoke paths in the scheduler.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
