[ https://issues.apache.org/jira/browse/AMBARI-24302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dmitry Lysnichenko resolved AMBARI-24302. ----------------------------------------- Resolution: Fixed > -Dhdp.version shows blank value in process output for Datanodes > --------------------------------------------------------------- > > Key: AMBARI-24302 > URL: https://issues.apache.org/jira/browse/AMBARI-24302 > Project: Ambari > Issue Type: Bug > Components: ambari-server > Affects Versions: 2.6.2 > Reporter: Dmitry Lysnichenko > Assignee: Dmitry Lysnichenko > Priority: Blocker > Labels: pull-request-available > Fix For: 2.7.1 > > Time Spent: 2h > Remaining Estimate: 0h > > # When we check the output of {{ps -ef | grep SecureDataNodeStarter}} it > shows multiple instances of {{-Dhdp.version}} being blank/empty and some > having the right value as shown below: > {quote}hdfs 40829 40798 1 14:11 ? 00:00:18 jsvc.exec > -Dproc_datanode -outfile /hdplogs/hadoop/hdfs/jsvc.out -errfile > /hdplogs/hadoop/hdfs/jsvc.err -pidfile > /var/run/hadoop/hdfs/hadoop_secure_dn.pid -nodetach -user hdfs -cp > /usr/hdp/current/hadoop-client/conf:/usr/hdp/2.6.3.0-235/hadoop/lib/*:/usr/hdp/2.6.3.0-235/hadoop/.//*:/usr/hdp/2.6.3.0-235/hadoop-hdfs/./:/usr/hdp/2.6.3.0-235/hadoop-hdfs/lib/*:/usr/hdp/2.6.3.0-235/hadoop-hdfs/.//*:/usr/hdp/2.6.3.0-235/hadoop-yarn/lib/*:/usr/hdp/2.6.3.0-235/hadoop-yarn/.//*:/usr/hdp/2.6.3.0-235/hadoop-mapreduce/lib/*:/usr/hdp/2.6.3.0-235/hadoop-mapreduce/.//* > -Xmx1024m {color:#14892c}*-Dhdp.version=2.6.3.0-235*{color} > -Djava.net.preferIPv4Stack=true {color:#d04437}-Dhdp.version= > {color}-Djava.net.preferIPv4Stack=true {color:#d04437}-Dhdp.version= > {color}-Djava.net.preferIPv4Stack=true -Dhadoop.log.dir=/hdplogs/hadoop/ > -Dhadoop.log.file=hadoop.log -Dhadoop.home.dir=/usr/hdp/2.6.3.0-235/hadoop > -Dhadoop.id.str= -Dhadoop.root.logger=INFO,console > -Djava.library.path=:/usr/hdp/2.6.3.0-235/hadoop/lib/native/Linux-amd64-64:/usr/hdp/2.6.3.0-235/hadoop/lib/native > -Dhadoop.policy.file=hadoop-policy.xml -Djava.net.preferIPv4Stack=true > {color:#14892c}*-Dhdp.version=2.6.3.0-235*{color} > -Dhadoop.log.dir=/hdplogs/hadoop/ > -Dhadoop.log.file=hadoop-hdfs-datanode-xxxxx.log > -Dhadoop.home.dir=/usr/hdp/2.6.3.0-235/hadoop -Dhadoop.id.str= > -Dhadoop.root.logger=INFO,RFA > -Djava.library.path=:/usr/hdp/2.6.3.0-235/hadoop/lib/native/Linux-amd64-64:/usr/hdp/2.6.3.0-235/hadoop/lib/native:/usr/hdp/2.6.3.0-235/hadoop/lib/native/Linux-amd64-64:/usr/hdp/2.6.3.0-235/hadoop/lib/native > -Dhadoop.policy.file=hadoop-policy.xml -Djava.net.preferIPv4Stack=true > -Dhadoop.log.dir=/hdplogs/hadoop/hdfs -Dhadoop.id.str=hdfs -jvm server > -server -XX:ParallelGCThreads=4 -XX:+UseConcMarkSweepGC > -XX:CMSInitiatingOccupancyFraction=70 -XX:+UseCMSInitiatingOccupancyOnly > -XX:ErrorFile=/hdplogs/hadoop/hdfs/hs_err_pid%p.log -XX:NewSize=200m > -XX:MaxNewSize=200m -Xloggc:/hdplogs/hadoop/hdfs/gc.log-201807051411 > -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps > -Xms1024m -Xmx1024m -Dhadoop.security.logger=INFO,DRFAS > -Dhdfs.audit.logger=INFO,DRFAAUDIT -server -XX:ParallelGCThreads=4 > -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=70 > -XX:+UseCMSInitiatingOccupancyOnly > -XX:ErrorFile=/hdplogs/hadoop/hdfs/hs_err_pid%p.log -XX:NewSize=200m > -XX:MaxNewSize=200m -Xloggc:/hdplogs/hadoop/hdfs/gc.log-201807051411 > -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps > -Xms1024m -Xmx1024m -Dhadoop.security.logger=INFO,DRFAS > -Dhdfs.audit.logger=INFO,DRFAAUDIT -server -XX:ParallelGCThreads=4 > -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=70 > -XX:+UseCMSInitiatingOccupancyOnly > -XX:ErrorFile=/hdplogs/hadoop/hdfs/hs_err_pid%p.log -XX:NewSize=200m > -XX:MaxNewSize=200m -Xloggc:/hdplogs/hadoop/hdfs/gc.log-201807051411 > -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps > -Xms1024m -Xmx1024m -Dhadoop.security.logger=INFO,DRFAS > -Dhdfs.audit.logger=INFO,DRFAAUDIT -Dhadoop.security.logger=INFO,RFAS > org.apache.hadoop.hdfs.server.datanode.SecureDataNodeStarter{quote} > A few params have been repeated multiple times as seen above, example > {{-Dhdp.version}} and {{-Djava.net.preferIPv4Stack}} > It will be great to fix this so it appears consistent. > As a temporary workaround, I was able to bypass this issue by updating > hadoop-env template from Ambari > HDFS > Configs > Advanced > Advanced > hadoop-env > Old Value: {{export HADOOP_OPTS="-Dhdp.version=$HDP_VERSION $HADOOP_OPTS"}} > Updated Value: {{export HADOOP_OPTS="-Dhdp.version=`hdp-select --version` > $HADOOP_OPTS"}} > P.S. I also checked this on a HDP-2.6.2 cluster and this problem was not seen > there. -- This message was sent by Atlassian JIRA (v7.6.3#76005)