[
https://issues.apache.org/jira/browse/YARN-7244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16207983#comment-16207983
]
Sunil G commented on YARN-7244:
-------------------------------
Thanks [~kshukla] and [~jlowe]
Usage of {{AuxiliaryLocalPathHandler}} looks very clean here.
Minor comments,
# Do you think is it better to have a setter and update
{{AuxiliaryLocalPathHandler}} to {{AuxServices}} rather than changing
{{AuxServices}} ctor.
# {{AuxiliaryLocalPathHandler}} could be in
{{org.apache.hadoop.yarn.server.api}}? any reasons to move to api?
# All apis in {{AuxiliaryLocalPathHandlerImpl}} could have Override annotation.
# Does {{ContainerManagerImpl}} need to have a {{getAuxiliaryLocalPathHandler}}
?
> ShuffleHandler is not aware of disks that are added
> ---------------------------------------------------
>
> Key: YARN-7244
> URL: https://issues.apache.org/jira/browse/YARN-7244
> Project: Hadoop YARN
> Issue Type: Bug
> Reporter: Kuhu Shukla
> Assignee: Kuhu Shukla
> Attachments: YARN-7244.001.patch, YARN-7244.002.patch,
> YARN-7244.003.patch, YARN-7244.004.patch, YARN-7244.005.patch,
> YARN-7244.006.patch, YARN-7244.007.patch, YARN-7244.008.patch,
> YARN-7244.009.patch, YARN-7244.010.patch
>
>
> The ShuffleHandler permanently remembers the list of "good" disks on NM
> startup. If disks later are added to the node then map tasks will start using
> them but the ShuffleHandler will not be aware of them. The end result is that
> the data cannot be shuffled from the node leading to fetch failures and
> re-runs of the map tasks.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
