[
https://issues.apache.org/jira/browse/HADOOP-9450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13625666#comment-13625666
]
Chris Nauroth commented on HADOOP-9450:
---------------------------------------
Based on the mention of line 133 in the description, I think the version of the
hadoop shell script at the following link corresponds to the version that
you're running. (Let me know if I have it wrong. I want to make sure I'm
looking at the same version of the code that you see.)
https://github.com/apache/hadoop-common/blob/release-1.1.0-rc4/bin/hadoop
CLASSPATH construction begins at line 131, and the full sequence of steps
starting from an empty CLASSPATH is:
# Append $HADOOP_CONF_DIR.
# If $HADOOP_USER_CLASSPATH_FIRST is defined, then append $HADOOP_CLASSPATH.
# Append developer build paths.
# Append distro paths.
# If $HADOOP_USER_CLASSPATH_FIRST is not defined, then append $HADOOP_CLASSPATH.
Both steps 2 and 5 append $HADOOP_CLASSPATH, but step 2 only runs if
$HADOOP_USER_CLASSPATH_FIRST is defined, and step 5 only runs if
$HADOOP_USER_CLASSPATH_FIRST is not defined. This means there are 2 possible
outcomes:
# If $HADOOP_USER_CLASSPATH_FIRST is defined, then
CLASSPATH=$HADOOP_CONF_DIR:$HADOOP_CLASSPATH:<developer build paths>:<distro
paths>
# If $HADOOP_USER_CLASSPATH_FIRST is not defined, then
CLASSPATH=$HADOOP_CONF_DIR:<developer build paths>:<distro
paths>:$HADOOP_CLASSPATH
With the first one, classes will load from your $HADOOP_CLASSPATH first. This
works because of the sequence of the steps described above, so I expect that
patching the code to do prepend instead of append won't be necessary. (One
additional note: $HADOOP_CONF_DIR must always be first to prevent accidentally
loading test configs instead of your own configs.)
This has been working fine for me. For example, here is one of my configs:
{code}
export HADOOP_USER_CLASSPATH_FIRST=true
HADOOP_REPO=/Users/chris/git/hadoop-common
export HADOOP_CLASSPATH=$HADOOP_REPO/build/classes
{code}
Are you seeing the script behave differently? If so, it might help us
troubleshoot further if you could change the script to run {{echo
$HADOOP_USER_CLASSPATH_FIRST; echo $HADOOP_CLASSPATH}} at the start of the
script and also run {{echo $CLASSPATH}} after each of the steps that appends to
it.
> HADOOP_USER_CLASSPATH_FIRST is not honored; CLASSPATH is PREpended instead of
> APpended
> --------------------------------------------------------------------------------------
>
> Key: HADOOP-9450
> URL: https://issues.apache.org/jira/browse/HADOOP-9450
> Project: Hadoop Common
> Issue Type: Bug
> Reporter: Mitch Wyle
>
> On line 133 of the hadoop shell wrapper, CLASSPATH is set as:
> CLASSPATH=${CLASSPATH}:${HADOOP_CLASSPATH}
> Notice that the built-up CLASSPATH, along with all the libs and unwanted JARS
> are pre-pended BEFORE the user's HADOOP_CLASSPATH. Therefore there is no way
> to put your own JARs in front of those that the hadoop wrapper script sets.
> We propose a patch that reverses this order. Failing that, we would like to
> add a command line option to override this behavior and enable a user's JARs
> to be found before the wrong ones in the Hadoop library paths.
> We always welcome your opinions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
