[ https://issues.apache.org/jira/browse/HADOOP-9870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13738972#comment-13738972 ]
Kai Zheng commented on HADOOP-9870: ----------------------------------- Similar issue was also found for HDFS related services, ref. HDFS-5087. It looks like we have two approaches to set JAVA heap max, by configuring JAVA_HEAP_MAX or adding -Xmx option directly in related *_OPTS. We might avoid using the both and the conflict. Also looked at other components like YARN. To be consistent, we could fix this by I would suggest: 1. Add variable HADOOP_CLIENT_HEAPSIZE, and user can set it either in hadoop-env.sh or hadoop-config.sh; 2. In appropriate script, check HADOOP_CLIENT_HEAPSIZE and set the value to JAVA_HEAP_MAX if any. > Mixed configurations for JVM -Xmx in hadoop command > --------------------------------------------------- > > Key: HADOOP-9870 > URL: https://issues.apache.org/jira/browse/HADOOP-9870 > Project: Hadoop Common > Issue Type: Bug > Reporter: Wei Yan > > When we use hadoop command to launch a class, there are two places setting > the -Xmx configuration. > *1*. The first place is located in file > {{hadoop-common-project/hadoop-common/src/main/bin/hadoop}}. > {code} > exec "$JAVA" $JAVA_HEAP_MAX $HADOOP_OPTS $CLASS "$@" > {code} > Here $JAVA_HEAP_MAX is configured in hadoop-config.sh > ({{hadoop-common-project/hadoop-common/src/main/bin/hadoop-config.sh}}). The > default value is "-Xmx1000m". > *2*. The second place is set with $HADOOP_OPTS in file > {{hadoop-common-project/hadoop-common/src/main/bin/hadoop}}. > {code} > HADOOP_OPTS="$HADOOP_OPTS $HADOOP_CLIENT_OPTS" > {code} > Here $HADOOP_CLIENT_OPTS is set in hadoop-env.sh > ({{hadoop-common-project/hadoop-common/src/main/conf/hadoop-env.sh}}) > {code} > export HADOOP_CLIENT_OPTS="-Xmx512m $HADOOP_CLIENT_OPTS" > {code} > Currently the final default java command looks like: > {code}java -Xmx1000m .... -Xmx512m CLASS_NAME ARGUMENTS"{code} > And if users also specify the -Xmx in the $HADOOP_CLIENT_OPTS, there will be > three -Xmx configurations. > The hadoop setup tutorial only discusses hadoop-env.sh, and it looks that > users should not make any change in hadoop-config.sh. > We should let hadoop smart to choose the right one before launching the java > command, instead of leaving for jvm to make the decision. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira