[
https://issues.apache.org/jira/browse/HADOOP-9870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13738993#comment-13738993
]
Wei Yan commented on HADOOP-9870:
---------------------------------
[~drankye] I agree with your approach.
Just one concern: $HADOOP_CLIENT_OPTS tries to wrap all configurations from
user-side and pass to hadoop command. If we introduce another variable
$HADOOP_CLIENT_HEAPSIZE, it may complex the original mechanism.
Another possible approach may be:
1. In {{hadoop-env.sh}}, before {{export HADOOP_CLIENT_OPTS="-Xmx512m
$HADOOP_CLIENT_OPTS"}}, we can check whether $HADOOP_CLIENT_OPTS contains -Xmx.
If has, ignore the default -Xmx512; otherwise, take the -Xmx512m.
2. In {{hadoop}}, before {{exec "$JAVA" $JAVA_HEAP_MAX $HADOOP_OPTS $CLASS
"$@"}}, we can check whether $HADOOP_OPTS containers -Xmx. If has, don't
include $JAVA_HEAP_MAX in the command; otherwise, take the $JAVA_HEAP_MAX.
Let's wait for some comments from other guys.
> Mixed configurations for JVM -Xmx in hadoop command
> ---------------------------------------------------
>
> Key: HADOOP-9870
> URL: https://issues.apache.org/jira/browse/HADOOP-9870
> Project: Hadoop Common
> Issue Type: Bug
> Reporter: Wei Yan
>
> When we use hadoop command to launch a class, there are two places setting
> the -Xmx configuration.
> *1*. The first place is located in file
> {{hadoop-common-project/hadoop-common/src/main/bin/hadoop}}.
> {code}
> exec "$JAVA" $JAVA_HEAP_MAX $HADOOP_OPTS $CLASS "$@"
> {code}
> Here $JAVA_HEAP_MAX is configured in hadoop-config.sh
> ({{hadoop-common-project/hadoop-common/src/main/bin/hadoop-config.sh}}). The
> default value is "-Xmx1000m".
> *2*. The second place is set with $HADOOP_OPTS in file
> {{hadoop-common-project/hadoop-common/src/main/bin/hadoop}}.
> {code}
> HADOOP_OPTS="$HADOOP_OPTS $HADOOP_CLIENT_OPTS"
> {code}
> Here $HADOOP_CLIENT_OPTS is set in hadoop-env.sh
> ({{hadoop-common-project/hadoop-common/src/main/conf/hadoop-env.sh}})
> {code}
> export HADOOP_CLIENT_OPTS="-Xmx512m $HADOOP_CLIENT_OPTS"
> {code}
> Currently the final default java command looks like:
> {code}java -Xmx1000m .... -Xmx512m CLASS_NAME ARGUMENTS"{code}
> And if users also specify the -Xmx in the $HADOOP_CLIENT_OPTS, there will be
> three -Xmx configurations.
> The hadoop setup tutorial only discusses hadoop-env.sh, and it looks that
> users should not make any change in hadoop-config.sh.
> We should let hadoop smart to choose the right one before launching the java
> command, instead of leaving for jvm to make the decision.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
