[ https://issues.apache.org/jira/browse/MAPREDUCE-7294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17199190#comment-17199190 ]
Mingliang Liu commented on MAPREDUCE-7294: ------------------------------------------ [~wzzdreamer] I saw a new patch file. We can either need to upload the patch file and mark the JIRA as "Patch Available" or create a new PR (base branch is {{branch-2.10}}) and link it here (manually if it does not show up automatically). I recommend the latter follow community new preference. Thanks, > Only application master should upload resource to Yarn Shared Cache > ------------------------------------------------------------------- > > Key: MAPREDUCE-7294 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-7294 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 > Affects Versions: 2.10.0, 3.3.0, 3.2.1, 3.1.4 > Reporter: zhenzhao wang > Assignee: zhenzhao wang > Priority: Major > Fix For: 3.2.2, 3.4.0, 3.1.5, 3.3.1 > > Attachments: MAPREDUCE-72942-2.10.001.patch > > > The design of yarn shared cache manager is only to allow application master > should upload the jar/files/resource. However, there was a bug in the code > since 2.9.0. Every node manager that take the job task will try to upload the > jar/resources. Let's say one job have 5000 tasks. Then there will be up to > 5000 NMs try to upload the jar. This is like DDOS and create a snowball > effect. It will end up with inavailability of yarn shared cache manager. It > wil cause time out in localization and lead to job failure. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org