[
https://issues.apache.org/jira/browse/YARN-7244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kuhu Shukla updated YARN-7244:
------------------------------
Attachment: YARN-7244-branch-2.8.001.patch
Attaching 2.8 version of the patch which needed some extra changes. The
important one is in LocalDirsHandlerService which was missing
getLocalPathForRead() method from trunk which went in as part of YARN-3998. I
have added just that method rather than change the visibility of
getPathToRead().
> ShuffleHandler is not aware of disks that are added
> ---------------------------------------------------
>
> Key: YARN-7244
> URL: https://issues.apache.org/jira/browse/YARN-7244
> Project: Hadoop YARN
> Issue Type: Bug
> Reporter: Kuhu Shukla
> Assignee: Kuhu Shukla
> Fix For: 2.9.0, 3.0.0
>
> Attachments: YARN-7244-branch-2.8.001.patch, YARN-7244.001.patch,
> YARN-7244.002.patch, YARN-7244.003.patch, YARN-7244.004.patch,
> YARN-7244.005.patch, YARN-7244.006.patch, YARN-7244.007.patch,
> YARN-7244.008.patch, YARN-7244.009.patch, YARN-7244.010.patch,
> YARN-7244.011.patch, YARN-7244.012.patch, YARN-7244.013.patch
>
>
> The ShuffleHandler permanently remembers the list of "good" disks on NM
> startup. If disks later are added to the node then map tasks will start using
> them but the ShuffleHandler will not be aware of them. The end result is that
> the data cannot be shuffled from the node leading to fetch failures and
> re-runs of the map tasks.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
