The first element of the classpath is the right one already.. but I STILL get the hive-site.xml is not found in classpath. Only hive gives me issues. hdfs, mapred, hbase are all running fine.
HADOOP_CLASSPATH=:*/shared/hive/conf:* /shared/hive/lib/antlr-runtime-3.0.1.jar:/shared/hive/lib/commons-cli-1.2.jar:/shared/hive/lib/commons-codec-1.3.jar:/shared/hive/lib/commons-collections-3.2.1.jar:/shared/hive/lib/commons-dbcp-1.4.jar:/shared/hive/lib/commons-lang-2.4.jar:/shared/hive/lib/commons-logging-1.0.4.jar:/shared/hive/lib/commons-logging-api-1.0.4.jar:/shared/hive/lib/commons-pool-1.5.4.jar:/shared/hive/lib/datanucleus-connectionpool-2.0.3.jar:/shared/hive/lib/datanucleus-core-2.0.3.jar:/shared/hive/lib/datanucleus-enhancer-2.0.3.jar:/shared/hive/lib/datanucleus-rdbms-2.0.3.jar:/shared/hive/lib/derby-10.4.2.0.jar:/shared/hive/lib/guava-r09.jar:/shared/hive/lib/hbase-0.92.0.jar:/shared/hive/lib/hbase-0.92.0-tests.jar:/shared/hive/lib/hive-builtins-0.9.0.jar:/shared/hive/lib/hive-cli-0.9.0.jar:/shared/hive/lib/hive-common-0.9.0.jar:/shared/hive/lib/hive-contrib-0.9.0.jar:/shared/hive/lib/hive_contrib.jar:/shared/hive/lib/hive-exec-0.9.0.jar:/shared/hive/lib/hive-hbase-handler-0.9.0.jar:/shared/hive/lib/hive-hwi-0.9.0.jar:/shared/hive/lib/hive-jdbc-0.9.0.jar:/shared/hive/lib/hive-metastore-0.9.0.jar:/shared/hive/lib/hive-pdk-0.9.0.jar:/shared/hive/lib/hive-serde-0.9.0.jar:/shared/hive/lib/hive-service-0.9.0.jar:/shared/hive/lib/hive-shims-0.9.0.jar:/shared/hive/lib/jackson-core-asl-1.8.8.jar:/shared/hive/lib/jackson-jaxrs-1.8.8.jar:/shared/hive/lib/jackson-mapper-asl-1.8.8.jar:/shared/hive/lib/jackson-xc-1.8.8.jar:/shared/hive/lib/JavaEWAH-0.3.2.jar:/shared/hive/lib/jdo2-api-2.3-ec.jar:/shared/hive/lib/jline-0.9.94. steve@mithril:/shared/hive/bin$ *ls -lrta /shared/hive/conf/hive-site.xml* -rw-r--r-- 1 steve steve 4415 2012-11-25 23:02 /shared/hive/conf/hive-site.xml 2012/11/30 Lauren Yang <[email protected]> > You can see if the classpath is being passed correctly to hadoop by > putting in an echo statement around line 150 of the hive cli script where > it passes the CLASSPATH variable to HADOOP_CLASSPATH.**** > > # pass classpath to hadoop**** > > export HADOOP_CLASSPATH="${HADOOP_CLASSPATH}:${CLASSPATH}"**** > > ** ** > > You could also echo the classpath in the hadoop script (in your > HADOOP_HOME\bin directory) to see if the classpath is being passed > correctly to the time when the cli jar is invoked.**** > > ** ** > > As far as the logs location, if this is not set in your hive-site.xml, you > can set it by passing in HIVE_OPTS when you invoke the command line.**** > > ** ** > > Like so:**** > > EXPORT HIVE_OPTS= -hiveconf hive.log.dir=$ HIVE_HOME\logs"**** > > Then run “hive”**** > > ** ** > > Or:**** > > Run “hive --hiveconf hive.log.dir=$ HIVE_HOME\logs”**** > > ** ** > > Thanks,**** > > Lauren**** > > ** ** > > ** ** > > *From:* Stephen Boesch [mailto:[email protected]] > *Sent:* Friday, November 30, 2012 12:16 AM > *To:* [email protected] > *Subject:* Re: hive-site.xml not found on classpath**** > > ** ** > > running 0.9.0 (you can see it from the classpath shown below);**** > > ** ** > > steve@mithril:/shared/cdh4$ echo $HIVE_CONF_DIR**** > > /shared/hive/conf**** > > steve@mithril:/shared/cdh4$ ls -l $HIVE_CONF_DIR**** > > total 152**** > > -rw-r--r-- 1 steve steve 46053 2011-12-13 00:36 hive-default.xml.template* > *** > > -rw-r--r-- 1 steve steve 1615 2012-11-13 23:37 hive-env.bullshit.sh**** > > -rw-r--r-- 1 steve steve 1671 2012-11-28 01:43 hive-env.sh**** > > -rw-r--r-- 1 steve steve 1593 2011-12-13 00:36 hive-env.sh.template**** > > -rw-r--r-- 1 steve steve 1637 2011-12-13 00:36 > hive-exec-log4j.properties.template**** > > -rw-r--r-- 1 root root 2056 2012-11-28 01:38 hive-log4j.properties**** > > -rw-r--r-- 1 steve steve 2056 2012-03-25 12:49 > hive-log4j.properties.template**** > > -rw-r--r-- 1 steve steve 4415 2012-11-25 23:02 hive-site.xml**** > > steve@mithril:/shared/cdh4$ echo $HIVE_HOME**** > > /shared/hive**** > > steve@mithril:/shared/cdh4$ echo $(which hive)**** > > /shared/hive/bin/hive**** > > ** ** > > also you can see the hive/conf is the first entry**** > > ** ** > > After adding the debug statement: **** > > ** ** > > classpath=*/shared/hive/conf:* > /shared/hive/lib/antlr-runtime-3.0.1.jar:/shared/hive/lib/commons-cli-1.2.jar:/shared/hive/lib/commons-codec-1.3.jar:/shared/hive/lib/commons-collections-3.2.1.jar:/shared/hive/lib/commons-dbcp-1.4.jar:/shared/hive/lib/commons-lang-2.4.jar:/shared/hive/lib/commons-logging-1.0.4.jar:/shared/hive/lib/commons-logging-api-1.0.4.jar:/shared/hive/lib/commons-pool-1.5.4.jar:/shared/hive/lib/datanucleus-connectionpool-2.0.3.jar:/shared/hive/lib/datanucleus-core-2.0.3.jar:/shared/hive/lib/datanucleus-enhancer-2.0.3.jar:/shared/hive/lib/datanucleus-rdbms-2.0.3.jar:/shared/hive/lib/derby-10.4.2.0.jar:/shared/hive/lib/guava-r09.jar:/shared/hive/lib/hbase-0.92.0.jar:/shared/hive/lib/hbase-0.92.0-tests.jar:/shared/hive/lib/hive-builtins-0.9.0.jar:/shared/hive/lib/hive-cli-0.9.0.jar:/shared/hive/lib/hive-common-0.9.0.jar:/shared/hive/lib/hive-contrib-0.9.0.jar:/shared/hive/lib/hive_contrib.jar:/shared/hive/lib/hive-exec-0.9.0.jar:/shared/hive/lib/hive-hbase-handler-0.9.0.jar:/shared/hive/lib/hive-hwi-0.9.0.jar:/shared/hive/lib/hive-jdbc-0.9.0.jar:/shared/hive/lib/hive-metastore-0.9.0.jar:/shared/hive/lib/hive-pdk-0.9.0.jar:/shared/hive/lib/hive-serde-0.9.0.jar:/shared/hive/lib/hive-service-0.9.0.jar:/shared/hive/lib/hive-shims-0.9.0.jar:/shared/hive/lib/jackson-core-asl-1.8.8.jar:/shared/hive/lib/jackson-jaxrs-1.8.8.jar:/shared/hive/lib/jackson-mapper-asl-1.8.8.jar:/shared/hive/lib/jackson-xc-1.8.8.jar:/shared/hive/lib/JavaEWAH-0.3.2.jar:/shared/hive/lib/jdo2-api-2.3-ec.jar:/shared/hive/lib/jline-0.9.94.jar:/shared/hive/lib/json-20090211.jar:/shared/hive/lib/libfb303-0.7.0.jar:/shared/hive/lib/libfb303.jar:/shared/hive/lib/libthrift-0.7.0.jar:/shared/hive/lib/libthrift.jar:/shared/hive/lib/log4j-1.2.16.jar:/shared/hive/lib/mysql-connector-java-5.1.18-bin.jar:/shared/hive/lib/slf4j-api-1.6.1.jar:/shared/hive/lib/slf4j-log4j12-1.6.1.jar:/shared/hive/lib/stringtemplate-3.1-b1.jar:/shared/hive/lib/zookeeper-3.4.3.jar: > **** > > ** ** > > ** ** > > But even so:**** > > - the log dir is still wrong (writing to /tmp/${user}/hive.log instead > of $HIVE_HOME/logs)**** > - the following message in the log file**** > > 2012-11-30 00:12:31,775 WARN conf.HiveConf (HiveConf.java:<clinit>(70)) > - *hive-site.xml not found on CLASSPATH***** > > * ***** > > ** ** > > ** ** > > ** ** > > ** ** > > 2012/11/30 Bing Li <[email protected]>**** > > which version of hive do you use? > > Could you try to add the following debug line in bin/hive before hive real > executes, and see the result? > > *echo "CLASSPATH=$CLASSPATH"* > > if [ "$TORUN" = "" ]; then > echo "Service $SERVICE not found" > echo "Available Services: $SERVICE_LIST" > exit 7 > else > $TORUN "$@" > fi > > The version I used is 0.9.0**** > > > > **** > > 2012/11/30 Stephen Boesch <[email protected]>**** > > Yes i do mean the log is in the wrong location, since it was set to a > persistent path in the $HIVE_CONF_DIR/lhive-log4j.properties.**** > > ** ** > > None of the files in that directory appear to be picked up properly: > neither the hive-site.xml nor log4j.properties.**** > > ** ** > > I have put echo statements into the 'hive" and the hive-config.sh shell > scripts and the echo statements prove that HIVE_CONF_DIR is set properly: > /shared/hive/conf **** > > ** ** > > But even so the following problems occur:**** > > - the message "hive-site.xml is not found in the classpath"**** > - none of the hive-site.xml values are taking properly**** > - the log4j.properties in that same directory is not taking effect.**** > > ** ** > > ** ** > > 2012/11/29 Bing Li <[email protected]>**** > > Hi, Stephen > what did you mean the "wrong place under /tmp" in > "I am seeing the following message in the logs (which are in the wrong > place under /tmp..)" ? > > Did you mean that you set a different log dir but it didn't work? > > the log dir should be set in conf/hive-log4j.properties, > conf/hive-exec-log4j.properties > and you can try to reset HIVE_CONF_DIR in conf/hive-env.sh with ‘export" > command. > > - Bing**** > > ** ** > > 2012/11/30 Stephen Boesch <[email protected]>**** > > thought i mentioned in the posts those were already set and verified.. but > yes in any case that's first thing looked at. > > steve@mithril:~$ echo $HIVE_CONF_DIR > /shared/hive/conf > steve@mithril:~$ echo $HIVE_HOME > /shared/hive**** > > ** ** > > 2012/11/29 [email protected] <[email protected]>**** > > ** ** > > Have you tried setting HIVE_HOME and HIVE_CONF_DIR?**** > > ** ** > > On Thu, Nov 29, 2012 at 2:46 PM, Stephen Boesch <[email protected]> wrote: > **** > > Yes. **** > > ** ** > > 2012/11/29 Shreepadma Venugopalan <[email protected]>**** > > Are you seeing this message when your bring up the standalone hive cli by > running 'hive'?**** > > ** ** > > On Thu, Nov 29, 2012 at 12:56 AM, Stephen Boesch <[email protected]> > wrote:**** > > i am running under user steve. the latest log (where this shows up ) is > /tmp/steve/hive.log**** > > ** ** > > 2012/11/29 Viral Bajaria <[email protected]>**** > > You are seeing this error when you run the hive cli or in the tasktracker > logs when you run a query ?**** > > ** ** > > On Thu, Nov 29, 2012 at 12:42 AM, Stephen Boesch <[email protected]> > wrote:**** > > ** ** > > I am seeing the following message in the logs (which are in the wrong > place under /tmp..)**** > > ** ** > > hive-site.xml not found on classpath**** > > ** ** > > My hive-site.xml is under the standard location $HIVE_HOME/conf so this > should not happen.**** > > ** ** > > Now some posts have talked that the HADOOP_CLASSPATH was mangled. Mine is > not..**** > > ** ** > > So what is the underlying issue here?**** > > ** ** > > Thanks**** > > ** ** > > stephenb**** > > ** ** > > ** ** > > ** ** > > ** ** > > > > **** > > ** ** > > -- > Swarnim**** > > ** ** > > ** ** > > ** ** > > ** ** > > ** ** >
