[ https://issues.apache.org/jira/browse/HADOOP-10996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14106305#comment-14106305 ]
Allen Wittenauer edited comment on HADOOP-10996 at 8/22/14 1:33 AM: -------------------------------------------------------------------- (Source: https://issues.apache.org/jira/secure/EditComment!default.jspa?id=12665400&commentId=14106278) Given this: {code} $ export HADOOP_COMMON_HOME=$(pwd)/$(ls -d hadoop-common-project/hadoop-common/target/hadoop-common-*/) $ export HADOOP_HDFS_HOME=$(pwd)/$(ls -d hadoop-hdfs-project/hadoop-hdfs/target/hadoop-hdfs-*/) $ export PATH=$HADOOP_COMMON_HOME/bin:$HADOOP_HDFS_HOME/bin:$PATH $ hdfs ERROR: Unable to exec (path)target/hadoop-hdfs-3.0.0-SNAPSHOT/bin/../libexec/hadoop-functions.sh. {code} How do we make hdfs work properly? First, what is happening? The code tries to find where to look for hdfs-config.sh is located. It does this by looking for ../libexec, where it finds it. It now makes the (false) assumption that this must be the one, true libexec dir. So it now tries to fire up hadoop-config.sh and hadoop-functions.sh which fail. There are a couple of different ways to solve this: * Look to see if HADOOP_COMMON_HOME is defined and look for hadoop-config.sh/hadoop-functions.sh is there as well. * Throw caution to the wind and see if this stuff is in our current path. * Recalculate HADOOP_LIBEXEC_DIR in hadoop-config.sh might work too, since clearly hdfs found it. * Do the full gamut of checks for HADOOP_HDFS_HOME, etc, for hdfs-config.sh + some of the stuff above. One sticking point is what happens if hadoop-layout.sh redefines the directory structure? The code is sort of in a catch-22. was (Author: aw): (Source: https://issues.apache.org/jira/secure/EditComment!default.jspa?id=12665400&commentId=14106278) Given this: {code} $ export HADOOP_COMMON_HOME=$(pwd)/$(ls -d hadoop-common-project/hadoop-common/target/hadoop-common-*/) $ export HADOOP_HDFS_HOME=$(pwd)/$(ls -d hadoop-hdfs-project/hadoop-hdfs/target/hadoop-hdfs-*/) $ export PATH=$HADOOP_COMMON_HOME/bin:$HADOOP_HDFS_HOME/bin:$PATH $ hdfs ERROR: Unable to exec (path)target/hadoop-hdfs-3.0.0-SNAPSHOT/bin/../libexec/hadoop-functions.sh. {code} How do we make hdfs work properly? First, what is happening? The code tries to find where to look for hdfs-config.sh is located. It does this by looking for ../libexec, where it finds it. It now makes the (false) assumption that this must be the one, true libexec dir. So it now tries to fire up hadoop-config.sh and hadoop-functions.sh which fail. There are a couple of different ways to solve this: * Look to see if HADOOP_COMMON_HOME is defined and look for hadoop-config.sh/hadoop-functions.sh is there as well. * Throw caution to the wind and see if this stuff is in our current path. * Do the full gamut of checks for HADOOP_HDFS_HOME, etc, for hdfs-config.sh + the stuff above. One sticking point is what happens if hadoop-layout.sh redefines the directory structure? The code is sort of in a catch-22. > run hdfs, yarn, mapred, etc from target tree > -------------------------------------------- > > Key: HADOOP-10996 > URL: https://issues.apache.org/jira/browse/HADOOP-10996 > Project: Hadoop Common > Issue Type: Improvement > Components: scripts > Affects Versions: 3.0.0 > Reporter: Allen Wittenauer > > There is a developer use case for running the shell scripts from the build > tree. What would it take to make it work? -- This message was sent by Atlassian JIRA (v6.2#6252)