You can see if the classpath is being passed correctly to hadoop by putting in
an echo statement around line 150 of the hive cli script where it passes the
CLASSPATH variable to HADOOP_CLASSPATH.
# pass classpath to hadoop
export HADOOP_CLASSPATH="${HADOOP_CLASSPATH}:${CLASSPATH}"
You could also echo the classpath in the hadoop script (in your HADOOP_HOME\bin
directory) to see if the classpath is being passed correctly to the time when
the cli jar is invoked.
As far as the logs location, if this is not set in your hive-site.xml, you can
set it by passing in HIVE_OPTS when you invoke the command line.
Like so:
EXPORT HIVE_OPTS= -hiveconf hive.log.dir=$ HIVE_HOME\logs"
Then run "hive"
Or:
Run "hive --hiveconf hive.log.dir=$ HIVE_HOME\logs"
Thanks,
Lauren
From: Stephen Boesch [mailto:[email protected]]
Sent: Friday, November 30, 2012 12:16 AM
To: [email protected]
Subject: Re: hive-site.xml not found on classpath
running 0.9.0 (you can see it from the classpath shown below);
steve@mithril:/shared/cdh4$ echo $HIVE_CONF_DIR
/shared/hive/conf
steve@mithril:/shared/cdh4$ ls -l $HIVE_CONF_DIR
total 152
-rw-r--r-- 1 steve steve 46053 2011-12-13 00:36 hive-default.xml.template
-rw-r--r-- 1 steve steve 1615 2012-11-13 23:37
hive-env.bullshit.sh<http://hive-env.bullshit.sh>
-rw-r--r-- 1 steve steve 1671 2012-11-28 01:43 hive-env.sh
-rw-r--r-- 1 steve steve 1593 2011-12-13 00:36 hive-env.sh.template
-rw-r--r-- 1 steve steve 1637 2011-12-13 00:36
hive-exec-log4j.properties.template
-rw-r--r-- 1 root root 2056 2012-11-28 01:38 hive-log4j.properties
-rw-r--r-- 1 steve steve 2056 2012-03-25 12:49 hive-log4j.properties.template
-rw-r--r-- 1 steve steve 4415 2012-11-25 23:02 hive-site.xml
steve@mithril:/shared/cdh4$ echo $HIVE_HOME
/shared/hive
steve@mithril:/shared/cdh4$ echo $(which hive)
/shared/hive/bin/hive
also you can see the hive/conf is the first entry
After adding the debug statement:
classpath=/shared/hive/conf:/shared/hive/lib/antlr-runtime-3.0.1.jar:/shared/hive/lib/commons-cli-1.2.jar:/shared/hive/lib/commons-codec-1.3.jar:/shared/hive/lib/commons-collections-3.2.1.jar:/shared/hive/lib/commons-dbcp-1.4.jar:/shared/hive/lib/commons-lang-2.4.jar:/shared/hive/lib/commons-logging-1.0.4.jar:/shared/hive/lib/commons-logging-api-1.0.4.jar:/shared/hive/lib/commons-pool-1.5.4.jar:/shared/hive/lib/datanucleus-connectionpool-2.0.3.jar:/shared/hive/lib/datanucleus-core-2.0.3.jar:/shared/hive/lib/datanucleus-enhancer-2.0.3.jar:/shared/hive/lib/datanucleus-rdbms-2.0.3.jar:/shared/hive/lib/derby-10.4.2.0.jar:/shared/hive/lib/guava-r09.jar:/shared/hive/lib/hbase-0.92.0.jar:/shared/hive/lib/hbase-0.92.0-tests.jar:/shared/hive/lib/hive-builtins-0.9.0.jar:/shared/hive/lib/hive-cli-0.9.0.jar:/shared/hive/lib/hive-common-0.9.0.jar:/shared/hive/lib/hive-contrib-0.9.0.jar:/shared/hive/lib/hive_contrib.jar:/shared/hive/lib/hive-exec-0.9.0.jar:/shared/hive/lib/hive-hbase-handler-0.9.0.jar:/shared/hive/lib/hive-hwi-0.9.0.jar:/shared/hive/lib/hive-jdbc-0.9.0.jar:/shared/hive/lib/hive-metastore-0.9.0.jar:/shared/hive/lib/hive-pdk-0.9.0.jar:/shared/hive/lib/hive-serde-0.9.0.jar:/shared/hive/lib/hive-service-0.9.0.jar:/shared/hive/lib/hive-shims-0.9.0.jar:/shared/hive/lib/jackson-core-asl-1.8.8.jar:/shared/hive/lib/jackson-jaxrs-1.8.8.jar:/shared/hive/lib/jackson-mapper-asl-1.8.8.jar:/shared/hive/lib/jackson-xc-1.8.8.jar:/shared/hive/lib/JavaEWAH-0.3.2.jar:/shared/hive/lib/jdo2-api-2.3-ec.jar:/shared/hive/lib/jline-0.9.94.jar:/shared/hive/lib/json-20090211.jar:/shared/hive/lib/libfb303-0.7.0.jar:/shared/hive/lib/libfb303.jar:/shared/hive/lib/libthrift-0.7.0.jar:/shared/hive/lib/libthrift.jar:/shared/hive/lib/log4j-1.2.16.jar:/shared/hive/lib/mysql-connector-java-5.1.18-bin.jar:/shared/hive/lib/slf4j-api-1.6.1.jar:/shared/hive/lib/slf4j-log4j12-1.6.1.jar:/shared/hive/lib/stringtemplate-3.1-b1.jar:/shared/hive/lib/zookeeper-3.4.3.jar:
But even so:
* the log dir is still wrong (writing to /tmp/${user}/hive.log instead of
$HIVE_HOME/logs)
* the following message in the log file
2012-11-30 00:12:31,775 WARN conf.HiveConf (HiveConf.java:<clinit>(70)) -
hive-site.xml not found on CLASSPATH
2012/11/30 Bing Li <[email protected]<mailto:[email protected]>>
which version of hive do you use?
Could you try to add the following debug line in bin/hive before hive real
executes, and see the result?
echo "CLASSPATH=$CLASSPATH"
if [ "$TORUN" = "" ]; then
echo "Service $SERVICE not found"
echo "Available Services: $SERVICE_LIST"
exit 7
else
$TORUN "$@"
fi
The version I used is 0.9.0
2012/11/30 Stephen Boesch <[email protected]<mailto:[email protected]>>
Yes i do mean the log is in the wrong location, since it was set to a
persistent path in the $HIVE_CONF_DIR/lhive-log4j.properties.
None of the files in that directory appear to be picked up properly: neither
the hive-site.xml nor log4j.properties.
I have put echo statements into the 'hive" and the hive-config.sh shell
scripts and the echo statements prove that HIVE_CONF_DIR is set properly:
/shared/hive/conf
But even so the following problems occur:
* the message "hive-site.xml is not found in the classpath"
* none of the hive-site.xml values are taking properly
* the log4j.properties in that same directory is not taking effect.
2012/11/29 Bing Li <[email protected]<mailto:[email protected]>>
Hi, Stephen
what did you mean the "wrong place under /tmp" in
"I am seeing the following message in the logs (which are in the wrong place
under /tmp..)" ?
Did you mean that you set a different log dir but it didn't work?
the log dir should be set in conf/hive-log4j.properties,
conf/hive-exec-log4j.properties
and you can try to reset HIVE_CONF_DIR in conf/hive-env.sh with 'export"
command.
- Bing
2012/11/30 Stephen Boesch <[email protected]<mailto:[email protected]>>
thought i mentioned in the posts those were already set and verified.. but yes
in any case that's first thing looked at.
steve@mithril:~$ echo $HIVE_CONF_DIR
/shared/hive/conf
steve@mithril:~$ echo $HIVE_HOME
/shared/hive
2012/11/29 [email protected]<mailto:[email protected]>
<[email protected]<mailto:[email protected]>>
Have you tried setting HIVE_HOME and HIVE_CONF_DIR?
On Thu, Nov 29, 2012 at 2:46 PM, Stephen Boesch
<[email protected]<mailto:[email protected]>> wrote:
Yes.
2012/11/29 Shreepadma Venugopalan
<[email protected]<mailto:[email protected]>>
Are you seeing this message when your bring up the standalone hive cli by
running 'hive'?
On Thu, Nov 29, 2012 at 12:56 AM, Stephen Boesch
<[email protected]<mailto:[email protected]>> wrote:
i am running under user steve. the latest log (where this shows up ) is
/tmp/steve/hive.log
2012/11/29 Viral Bajaria
<[email protected]<mailto:[email protected]>>
You are seeing this error when you run the hive cli or in the tasktracker logs
when you run a query ?
On Thu, Nov 29, 2012 at 12:42 AM, Stephen Boesch
<[email protected]<mailto:[email protected]>> wrote:
I am seeing the following message in the logs (which are in the wrong place
under /tmp..)
hive-site.xml not found on classpath
My hive-site.xml is under the standard location $HIVE_HOME/conf so this should
not happen.
Now some posts have talked that the HADOOP_CLASSPATH was mangled. Mine is not..
So what is the underlying issue here?
Thanks
stephenb
--
Swarnim
