[
https://issues.apache.org/jira/browse/MAPREDUCE-7294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
zhenzhao wang moved YARN-10398 to MAPREDUCE-7294:
-------------------------------------------------
Component/s: (was: yarn)
yarn
Key: MAPREDUCE-7294 (was: YARN-10398)
Affects Version/s: (was: 3.1.5)
(was: 3.3.1)
(was: 3.4.0)
(was: 3.1.4)
(was: 3.2.2)
(was: 3.1.3)
(was: 2.9.3)
(was: 3.2.1)
(was: 3.3.0)
(was: 3.1.2)
(was: 3.0.4)
(was: 3.0.3)
(was: 2.9.2)
(was: 3.1.1)
(was: 3.2.0)
(was: 3.0.2)
(was: 3.0.1)
(was: 2.9.1)
(was: 3.1.0)
(was: 3.0.0)
(was: 2.9.0)
3.3.1
3.1.5
3.4.0
3.2.2
2.9.3
2.9.0
3.0.0
3.1.0
3.0.1
2.9.1
3.0.2
3.2.0
3.0.3
3.1.1
2.9.2
3.1.2
3.3.0
3.2.1
3.1.3
3.1.4
Project: Hadoop Map/Reduce (was: Hadoop YARN)
> Every NM will try to upload Jar/Archives/Files/Resources to Yarn Shared Cache
> Manager Like DDOS
> -----------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-7294
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7294
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: yarn
> Affects Versions: 3.1.4, 3.1.3, 3.2.1, 3.3.0, 3.1.2, 2.9.2, 3.1.1, 3.0.3,
> 3.2.0, 3.0.2, 2.9.1, 3.0.1, 3.1.0, 3.0.0, 2.9.0, 2.9.3, 3.2.2, 3.4.0, 3.1.5,
> 3.3.1
> Reporter: zhenzhao wang
> Assignee: zhenzhao wang
> Priority: Major
>
> The design of yarn shared cache manager is only to allow application master
> should upload the jar/files/resource. However, there was a bug in the code
> since 2.9.0. Every node manager that take the job task will try to upload the
> jar/resources. Let's say one job have 5000 tasks. Then there will be up to
> 5000 NMs try to upload the jar. This is like DDOS and create a snowball
> effect. It will end up with inavailability of yarn shared cache manager. It
> wil cause time out in localization and lead to job failure.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
