[jira] [Commented] (MAPREDUCE-6265) make INITIAL_POOL_SIZE in ContainerLauncherImpl configurable to better control the thread pool size to launch/kill containers.

2015-03-13 Thread zhihai xu (JIRA) 

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14360873#comment-14360873
 ] 

zhihai xu commented on MAPREDUCE-6265:
--

[~ozawa], yes, that is a very good suggestion, I uploaded a new patch 
MAPREDUCE-6265.001.patch which addressed your comment. Please review it.
many thanks for the review.

 make INITIAL_POOL_SIZE in ContainerLauncherImpl configurable to better 
 control the thread pool size to launch/kill containers.
 --

 Key: MAPREDUCE-6265
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6265
 Project: Hadoop Map/Reduce
  Issue Type: Improvement
  Components: mrv2
Reporter: zhihai xu
Assignee: zhihai xu
 Attachments: MAPREDUCE-6265.000.patch, MAPREDUCE-6265.001.patch


 make INITIAL_POOL_SIZE in ContainerLauncherImpl configurable to better 
 control the thread pool size to launch/kill containers
 Currently INITIAL_POOL_SIZE in ContainerLauncherImpl is hard-coded at 
 {code}
   protected static final int INITIAL_POOL_SIZE = 10;
 {code}
 We should make it configurable because the thread pool size will be decided 
 by INITIAL_POOL_SIZE, limitOnPoolSize and number of node used by the AM.
 Since we already made limitOnPoolSize configurable, it make senses to also 
 make INITIAL_POOL_SIZE configurable to better manage the thread pool size.
 We saw some issue due to the small thread pool size when some node is down. 
 The recovery from a shutdown node take very long time due to all the 
 ContainerLauncher threads are blocked by IPC client connection to the 
 shutdown node.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MAPREDUCE-6265) make INITIAL_POOL_SIZE in ContainerLauncherImpl configurable to better control the thread pool size to launch/kill containers.

2015-03-13 Thread Hadoop QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14360963#comment-14360963
 ] 

Hadoop QA commented on MAPREDUCE-6265:
--

{color:green}+1 overall{color}.  Here are the results of testing the latest 
attachment 
  
http://issues.apache.org/jira/secure/attachment/12704478/MAPREDUCE-6265.001.patch
  against trunk revision f446669.

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 1 new 
or modified test files.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  There were no new javadoc warning messages.

{color:green}+1 eclipse:eclipse{color}.  The patch built with 
eclipse:eclipse.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 2.0.3) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 core tests{color}.  The patch passed unit tests in 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core.

Test results: 
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5288//testReport/
Console output: 
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5288//console

This message is automatically generated.

 make INITIAL_POOL_SIZE in ContainerLauncherImpl configurable to better 
 control the thread pool size to launch/kill containers.
 --

 Key: MAPREDUCE-6265
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6265
 Project: Hadoop Map/Reduce
  Issue Type: Improvement
  Components: mrv2
Reporter: zhihai xu
Assignee: zhihai xu
 Attachments: MAPREDUCE-6265.000.patch, MAPREDUCE-6265.001.patch


 make INITIAL_POOL_SIZE in ContainerLauncherImpl configurable to better 
 control the thread pool size to launch/kill containers
 Currently INITIAL_POOL_SIZE in ContainerLauncherImpl is hard-coded at 
 {code}
   protected static final int INITIAL_POOL_SIZE = 10;
 {code}
 We should make it configurable because the thread pool size will be decided 
 by INITIAL_POOL_SIZE, limitOnPoolSize and number of node used by the AM.
 Since we already made limitOnPoolSize configurable, it make senses to also 
 make INITIAL_POOL_SIZE configurable to better manage the thread pool size.
 We saw some issue due to the small thread pool size when some node is down. 
 The recovery from a shutdown node take very long time due to all the 
 ContainerLauncher threads are blocked by IPC client connection to the 
 shutdown node.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MAPREDUCE-6265) make INITIAL_POOL_SIZE in ContainerLauncherImpl configurable to better control the thread pool size to launch/kill containers.

2015-03-13 Thread Tsuyoshi Ozawa (JIRA) 

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14361602#comment-14361602
 ] 

Tsuyoshi Ozawa commented on MAPREDUCE-6265:
---

I meant containerlauncher.thread-initial-size should be 
containerlauncher.*threadpool*-initial-size;

 make INITIAL_POOL_SIZE in ContainerLauncherImpl configurable to better 
 control the thread pool size to launch/kill containers.
 --

 Key: MAPREDUCE-6265
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6265
 Project: Hadoop Map/Reduce
  Issue Type: Improvement
  Components: mrv2
Reporter: zhihai xu
Assignee: zhihai xu
 Attachments: MAPREDUCE-6265.000.patch, MAPREDUCE-6265.001.patch


 make INITIAL_POOL_SIZE in ContainerLauncherImpl configurable to better 
 control the thread pool size to launch/kill containers
 Currently INITIAL_POOL_SIZE in ContainerLauncherImpl is hard-coded at 
 {code}
   protected static final int INITIAL_POOL_SIZE = 10;
 {code}
 We should make it configurable because the thread pool size will be decided 
 by INITIAL_POOL_SIZE, limitOnPoolSize and number of node used by the AM.
 Since we already made limitOnPoolSize configurable, it make senses to also 
 make INITIAL_POOL_SIZE configurable to better manage the thread pool size.
 We saw some issue due to the small thread pool size when some node is down. 
 The recovery from a shutdown node take very long time due to all the 
 ContainerLauncher threads are blocked by IPC client connection to the 
 shutdown node.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MAPREDUCE-6265) make INITIAL_POOL_SIZE in ContainerLauncherImpl configurable to better control the thread pool size to launch/kill containers.

2015-03-13 Thread Tsuyoshi Ozawa (JIRA) 

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14361601#comment-14361601
 ] 

Tsuyoshi Ozawa commented on MAPREDUCE-6265:
---

[~zxu] should we also update the following line(name of property name itself)?

{code}
+  public static final String MR_AM_CONTAINERLAUNCHER_THREADPOOL_INITIAL_SIZE =
+  MR_AM_PREFIX+containerlauncher.thread-initial-size;
{code}

 make INITIAL_POOL_SIZE in ContainerLauncherImpl configurable to better 
 control the thread pool size to launch/kill containers.
 --

 Key: MAPREDUCE-6265
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6265
 Project: Hadoop Map/Reduce
  Issue Type: Improvement
  Components: mrv2
Reporter: zhihai xu
Assignee: zhihai xu
 Attachments: MAPREDUCE-6265.000.patch, MAPREDUCE-6265.001.patch


 make INITIAL_POOL_SIZE in ContainerLauncherImpl configurable to better 
 control the thread pool size to launch/kill containers
 Currently INITIAL_POOL_SIZE in ContainerLauncherImpl is hard-coded at 
 {code}
   protected static final int INITIAL_POOL_SIZE = 10;
 {code}
 We should make it configurable because the thread pool size will be decided 
 by INITIAL_POOL_SIZE, limitOnPoolSize and number of node used by the AM.
 Since we already made limitOnPoolSize configurable, it make senses to also 
 make INITIAL_POOL_SIZE configurable to better manage the thread pool size.
 We saw some issue due to the small thread pool size when some node is down. 
 The recovery from a shutdown node take very long time due to all the 
 ContainerLauncher threads are blocked by IPC client connection to the 
 shutdown node.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MAPREDUCE-6265) make INITIAL_POOL_SIZE in ContainerLauncherImpl configurable to better control the thread pool size to launch/kill containers.

2015-03-13 Thread zhihai xu (JIRA) 

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14361609#comment-14361609
 ] 

zhihai xu commented on MAPREDUCE-6265:
--

I uploaded a new patch MAPREDUCE-6265.002.patch to fix this issue.

 make INITIAL_POOL_SIZE in ContainerLauncherImpl configurable to better 
 control the thread pool size to launch/kill containers.
 --

 Key: MAPREDUCE-6265
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6265
 Project: Hadoop Map/Reduce
  Issue Type: Improvement
  Components: mrv2
Reporter: zhihai xu
Assignee: zhihai xu
 Attachments: MAPREDUCE-6265.000.patch, MAPREDUCE-6265.001.patch, 
 MAPREDUCE-6265.002.patch


 make INITIAL_POOL_SIZE in ContainerLauncherImpl configurable to better 
 control the thread pool size to launch/kill containers
 Currently INITIAL_POOL_SIZE in ContainerLauncherImpl is hard-coded at 
 {code}
   protected static final int INITIAL_POOL_SIZE = 10;
 {code}
 We should make it configurable because the thread pool size will be decided 
 by INITIAL_POOL_SIZE, limitOnPoolSize and number of node used by the AM.
 Since we already made limitOnPoolSize configurable, it make senses to also 
 make INITIAL_POOL_SIZE configurable to better manage the thread pool size.
 We saw some issue due to the small thread pool size when some node is down. 
 The recovery from a shutdown node take very long time due to all the 
 ContainerLauncher threads are blocked by IPC client connection to the 
 shutdown node.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MAPREDUCE-6265) make INITIAL_POOL_SIZE in ContainerLauncherImpl configurable to better control the thread pool size to launch/kill containers.

2015-03-13 Thread zhihai xu (JIRA) 

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14361608#comment-14361608
 ] 

zhihai xu commented on MAPREDUCE-6265:
--

[~ozawa], good catch, I forget to update the string value. thanks for your 
thorough review.

 make INITIAL_POOL_SIZE in ContainerLauncherImpl configurable to better 
 control the thread pool size to launch/kill containers.
 --

 Key: MAPREDUCE-6265
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6265
 Project: Hadoop Map/Reduce
  Issue Type: Improvement
  Components: mrv2
Reporter: zhihai xu
Assignee: zhihai xu
 Attachments: MAPREDUCE-6265.000.patch, MAPREDUCE-6265.001.patch, 
 MAPREDUCE-6265.002.patch


 make INITIAL_POOL_SIZE in ContainerLauncherImpl configurable to better 
 control the thread pool size to launch/kill containers
 Currently INITIAL_POOL_SIZE in ContainerLauncherImpl is hard-coded at 
 {code}
   protected static final int INITIAL_POOL_SIZE = 10;
 {code}
 We should make it configurable because the thread pool size will be decided 
 by INITIAL_POOL_SIZE, limitOnPoolSize and number of node used by the AM.
 Since we already made limitOnPoolSize configurable, it make senses to also 
 make INITIAL_POOL_SIZE configurable to better manage the thread pool size.
 We saw some issue due to the small thread pool size when some node is down. 
 The recovery from a shutdown node take very long time due to all the 
 ContainerLauncher threads are blocked by IPC client connection to the 
 shutdown node.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MAPREDUCE-6265) make INITIAL_POOL_SIZE in ContainerLauncherImpl configurable to better control the thread pool size to launch/kill containers.

2015-03-13 Thread Hadoop QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14361615#comment-14361615
 ] 

Hadoop QA commented on MAPREDUCE-6265:
--

{color:green}+1 overall{color}.  Here are the results of testing the latest 
attachment 
  
http://issues.apache.org/jira/secure/attachment/12704570/MAPREDUCE-6265.002.patch
  against trunk revision 32741cf.

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 1 new 
or modified test files.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  There were no new javadoc warning messages.

{color:green}+1 eclipse:eclipse{color}.  The patch built with 
eclipse:eclipse.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 2.0.3) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 core tests{color}.  The patch passed unit tests in 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core.

Test results: 
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5289//testReport/
Console output: 
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5289//console

This message is automatically generated.

 make INITIAL_POOL_SIZE in ContainerLauncherImpl configurable to better 
 control the thread pool size to launch/kill containers.
 --

 Key: MAPREDUCE-6265
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6265
 Project: Hadoop Map/Reduce
  Issue Type: Improvement
  Components: mrv2
Reporter: zhihai xu
Assignee: zhihai xu
 Attachments: MAPREDUCE-6265.000.patch, MAPREDUCE-6265.001.patch, 
 MAPREDUCE-6265.002.patch


 make INITIAL_POOL_SIZE in ContainerLauncherImpl configurable to better 
 control the thread pool size to launch/kill containers
 Currently INITIAL_POOL_SIZE in ContainerLauncherImpl is hard-coded at 
 {code}
   protected static final int INITIAL_POOL_SIZE = 10;
 {code}
 We should make it configurable because the thread pool size will be decided 
 by INITIAL_POOL_SIZE, limitOnPoolSize and number of node used by the AM.
 Since we already made limitOnPoolSize configurable, it make senses to also 
 make INITIAL_POOL_SIZE configurable to better manage the thread pool size.
 We saw some issue due to the small thread pool size when some node is down. 
 The recovery from a shutdown node take very long time due to all the 
 ContainerLauncher threads are blocked by IPC client connection to the 
 shutdown node.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MAPREDUCE-6265) make INITIAL_POOL_SIZE in ContainerLauncherImpl configurable to better control the thread pool size to launch/kill containers.

2015-03-13 Thread Tsuyoshi Ozawa (JIRA) 

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14360641#comment-14360641
 ] 

Tsuyoshi Ozawa commented on MAPREDUCE-6265:
---

[~zxu] looks good to me overall. Minor nits: I think the name of the property 
should be yarn.app.mapreduce.am.containerlauncher.*threadpool*-initial-size and 
MR_AM_CONTAINERLAUNCHER_*THREADPOOL*_INITIAL_SIZE for accuracy since it's a bit 
difficult to understand what thread initial size means. What do you think?

 make INITIAL_POOL_SIZE in ContainerLauncherImpl configurable to better 
 control the thread pool size to launch/kill containers.
 --

 Key: MAPREDUCE-6265
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6265
 Project: Hadoop Map/Reduce
  Issue Type: Improvement
  Components: mrv2
Reporter: zhihai xu
Assignee: zhihai xu
 Attachments: MAPREDUCE-6265.000.patch


 make INITIAL_POOL_SIZE in ContainerLauncherImpl configurable to better 
 control the thread pool size to launch/kill containers
 Currently INITIAL_POOL_SIZE in ContainerLauncherImpl is hard-coded at 
 {code}
   protected static final int INITIAL_POOL_SIZE = 10;
 {code}
 We should make it configurable because the thread pool size will be decided 
 by INITIAL_POOL_SIZE, limitOnPoolSize and number of node used by the AM.
 Since we already made limitOnPoolSize configurable, it make senses to also 
 make INITIAL_POOL_SIZE configurable to better manage the thread pool size.
 We saw some issue due to the small thread pool size when some node is down. 
 The recovery from a shutdown node take very long time due to all the 
 ContainerLauncher threads are blocked by IPC client connection to the 
 shutdown node.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MAPREDUCE-6265) make INITIAL_POOL_SIZE in ContainerLauncherImpl configurable to better control the thread pool size to launch/kill containers.

2015-03-12 Thread zhihai xu (JIRA) 

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14359867#comment-14359867
 ] 

zhihai xu commented on MAPREDUCE-6265:
--

I uploaded a patch MAPREDUCE-6265.000.patch for review.
The initial thread pool size is a important parameter for ContainerLauncherImpl.

 make INITIAL_POOL_SIZE in ContainerLauncherImpl configurable to better 
 control the thread pool size to launch/kill containers.
 --

 Key: MAPREDUCE-6265
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6265
 Project: Hadoop Map/Reduce
  Issue Type: Improvement
  Components: mrv2
Reporter: zhihai xu
Assignee: zhihai xu
 Attachments: MAPREDUCE-6265.000.patch


 make INITIAL_POOL_SIZE in ContainerLauncherImpl configurable to better 
 control the thread pool size to launch/kill containers
 Currently INITIAL_POOL_SIZE in ContainerLauncherImpl is hard-coded at 
 {code}
   protected static final int INITIAL_POOL_SIZE = 10;
 {code}
 We should make it configurable because the thread pool size will be decided 
 by INITIAL_POOL_SIZE, limitOnPoolSize and number of node used by the AM.
 Since we already made limitOnPoolSize configurable, it make senses to also 
 make INITIAL_POOL_SIZE configurable to better manage the thread pool size.
 We saw some issue due to the small thread pool size when some node is down. 
 The recovery from a shutdown node take very long time due to all the 
 ContainerLauncher threads are blocked by IPC client connection to the 
 shutdown node.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MAPREDUCE-6265) make INITIAL_POOL_SIZE in ContainerLauncherImpl configurable to better control the thread pool size to launch/kill containers.

2015-03-12 Thread Hadoop QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14359899#comment-14359899
 ] 

Hadoop QA commented on MAPREDUCE-6265:
--

{color:green}+1 overall{color}.  Here are the results of testing the latest 
attachment 
  
http://issues.apache.org/jira/secure/attachment/12704343/MAPREDUCE-6265.000.patch
  against trunk revision 8212877.

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 1 new 
or modified test files.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  There were no new javadoc warning messages.

{color:green}+1 eclipse:eclipse{color}.  The patch built with 
eclipse:eclipse.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 2.0.3) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 core tests{color}.  The patch passed unit tests in 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core.

Test results: 
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5285//testReport/
Console output: 
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5285//console

This message is automatically generated.

 make INITIAL_POOL_SIZE in ContainerLauncherImpl configurable to better 
 control the thread pool size to launch/kill containers.
 --

 Key: MAPREDUCE-6265
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6265
 Project: Hadoop Map/Reduce
  Issue Type: Improvement
  Components: mrv2
Reporter: zhihai xu
Assignee: zhihai xu
 Attachments: MAPREDUCE-6265.000.patch


 make INITIAL_POOL_SIZE in ContainerLauncherImpl configurable to better 
 control the thread pool size to launch/kill containers
 Currently INITIAL_POOL_SIZE in ContainerLauncherImpl is hard-coded at 
 {code}
   protected static final int INITIAL_POOL_SIZE = 10;
 {code}
 We should make it configurable because the thread pool size will be decided 
 by INITIAL_POOL_SIZE, limitOnPoolSize and number of node used by the AM.
 Since we already made limitOnPoolSize configurable, it make senses to also 
 make INITIAL_POOL_SIZE configurable to better manage the thread pool size.
 We saw some issue due to the small thread pool size when some node is down. 
 The recovery from a shutdown node take very long time due to all the 
 ContainerLauncher threads are blocked by IPC client connection to the 
 shutdown node.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MAPREDUCE-6265) make INITIAL_POOL_SIZE in ContainerLauncherImpl configurable to better control the thread pool size to launch/kill containers.

2015-03-12 Thread zhihai xu (JIRA) 

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14359929#comment-14359929
 ] 

zhihai xu commented on MAPREDUCE-6265:
--

Hi [~ozawa], The patch passed all the tests. Do you have time to review it? 
thanks

 make INITIAL_POOL_SIZE in ContainerLauncherImpl configurable to better 
 control the thread pool size to launch/kill containers.
 --

 Key: MAPREDUCE-6265
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6265
 Project: Hadoop Map/Reduce
  Issue Type: Improvement
  Components: mrv2
Reporter: zhihai xu
Assignee: zhihai xu
 Attachments: MAPREDUCE-6265.000.patch


 make INITIAL_POOL_SIZE in ContainerLauncherImpl configurable to better 
 control the thread pool size to launch/kill containers
 Currently INITIAL_POOL_SIZE in ContainerLauncherImpl is hard-coded at 
 {code}
   protected static final int INITIAL_POOL_SIZE = 10;
 {code}
 We should make it configurable because the thread pool size will be decided 
 by INITIAL_POOL_SIZE, limitOnPoolSize and number of node used by the AM.
 Since we already made limitOnPoolSize configurable, it make senses to also 
 make INITIAL_POOL_SIZE configurable to better manage the thread pool size.
 We saw some issue due to the small thread pool size when some node is down. 
 The recovery from a shutdown node take very long time due to all the 
 ContainerLauncher threads are blocked by IPC client connection to the 
 shutdown node.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MAPREDUCE-6265) make INITIAL_POOL_SIZE in ContainerLauncherImpl configurable to better control the thread pool size to launch/kill containers.

2015-03-12 Thread Tsuyoshi Ozawa (JIRA) 

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14359952#comment-14359952
 ] 

Tsuyoshi Ozawa commented on MAPREDUCE-6265:
---

Sure, I'll check it.

 make INITIAL_POOL_SIZE in ContainerLauncherImpl configurable to better 
 control the thread pool size to launch/kill containers.
 --

 Key: MAPREDUCE-6265
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6265
 Project: Hadoop Map/Reduce
  Issue Type: Improvement
  Components: mrv2
Reporter: zhihai xu
Assignee: zhihai xu
 Attachments: MAPREDUCE-6265.000.patch


 make INITIAL_POOL_SIZE in ContainerLauncherImpl configurable to better 
 control the thread pool size to launch/kill containers
 Currently INITIAL_POOL_SIZE in ContainerLauncherImpl is hard-coded at 
 {code}
   protected static final int INITIAL_POOL_SIZE = 10;
 {code}
 We should make it configurable because the thread pool size will be decided 
 by INITIAL_POOL_SIZE, limitOnPoolSize and number of node used by the AM.
 Since we already made limitOnPoolSize configurable, it make senses to also 
 make INITIAL_POOL_SIZE configurable to better manage the thread pool size.
 We saw some issue due to the small thread pool size when some node is down. 
 The recovery from a shutdown node take very long time due to all the 
 ContainerLauncher threads are blocked by IPC client connection to the 
 shutdown node.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)