[jira] [Commented] (MAPREDUCE-5887) Move split creation from submission client to MRAppMaster
[ https://issues.apache.org/jira/browse/MAPREDUCE-5887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13997431#comment-13997431 ] Steve Loughran commented on MAPREDUCE-5887: --- This is good, especially when the client is something like a laptop trying to submit to in-cloud deployments. One test to try there is what happens when the blocksize is reported as very, very small (you can configure this in swiftfs). in the client this will cause the submitting process to OOM and fail. Presumably the same outcome in the AM is the simplest to implement -we just need to make sure that YARN recognises this as a failure and only tries a couple of times > Move split creation from submission client to MRAppMaster > - > > Key: MAPREDUCE-5887 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5887 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: applicationmaster, client >Affects Versions: 2.4.0 >Reporter: Gera Shegalov >Assignee: Gera Shegalov > Attachments: MAPREDUCE-5887.v01.patch > > > This JIRA is filed to improve scalability of job submission, specifically > when there is a significant latency between the submission client and the > cluster nodes RM and NN, e.g. in a multi-datacenter environment. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (MAPREDUCE-5887) Move split creation from submission client to MRAppMaster
[ https://issues.apache.org/jira/browse/MAPREDUCE-5887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13995215#comment-13995215 ] Gera Shegalov commented on MAPREDUCE-5887: -- Thanks for pointing to an earlier JIRA, [~sandyr]! > Move split creation from submission client to MRAppMaster > - > > Key: MAPREDUCE-5887 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5887 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: applicationmaster, client >Affects Versions: 2.4.0 >Reporter: Gera Shegalov >Assignee: Gera Shegalov > Attachments: MAPREDUCE-5887.v01.patch > > > This JIRA is filed to improve scalability of job submission, specifically > when there is a significant latency between the submission client and the > cluster nodes RM and NN, e.g. in a multi-datacenter environment. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (MAPREDUCE-5887) Move split creation from submission client to MRAppMaster
[ https://issues.apache.org/jira/browse/MAPREDUCE-5887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13994812#comment-13994812 ] Sandy Ryza commented on MAPREDUCE-5887: --- Mind attaching your patch on MAPREDUCE-207? > Move split creation from submission client to MRAppMaster > - > > Key: MAPREDUCE-5887 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5887 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: applicationmaster, client >Affects Versions: 2.4.0 >Reporter: Gera Shegalov >Assignee: Gera Shegalov > Attachments: MAPREDUCE-5887.v01.patch > > > This JIRA is filed to improve scalability of job submission, specifically > when there is a significant latency between the submission client and the > cluster nodes RM and NN, e.g. in a multi-datacenter environment. -- This message was sent by Atlassian JIRA (v6.2#6252)
