Re: Any existing UDTF to flatten map
I ended up getting an error (Hive 0.7.1), but I would have thought
something like the following would work:
SELECT
user_id,
obj_key,
obj[obj_key] AS obj_item
FROM (
SELECT
"user1" user_id,
MAP("k1", "v1", "k2", "v2") obj
FROM calendar
LIMIT 1
) tmp
LATERAL VIEW EXPLODE(MAP_KEYS(obj)) obj_tbl AS obj_key;
There error that I get for this is: FAILED: Error in semantic analysis:
Line 3:4 Non-constant expression for map indexes not supported obj_key
Matt
On Fri, Nov 30, 2012 at 1:18 PM, xu ji wrote:
> Hi all,
> I have a table like this:
> hive> desc mytable;
> ts bigint
> content map
> hive> select * from mytable;
> 1354299050{"F1":"id-1"}
> 1354299040{"F1":"id-2","F2":"id-3"}
> 1354299030{"F1":"id-3","F2":"id-1","F3":"id-4"}
>
> Does anyone know how to generate a table like this:
>
> hive> desc resulttable;
> id string
> ts bigint
> hive> select * from resulttable;
> id-11354299050
> id-21354299040
> id-31354299040
> id-31354299030
> id-11354299030
> id-41354299030
>
>
> I mean I can certainly write a UDTF to generate a table like this, but I
> wonder is there an existing UDTF to do this job?
>
> Thanks,
> Xu
>
> --
> Ji Xu
> Mobile: (+86) 138-115-10101
> E-mail: jixuxj...@gmail.com
> Homepage: http://about.me/jixu
>
> Room 825, School of Software,
> Main Building, Tsinghua University,
> Beijing, 100084, P. R. China.
>
>
Re: Exception with datanucleus while running hive tests in eclipse
I ran into the same problem on the same MAC OS version. This seems to be a JVM command line issue. it exceeds its limits and it's platform independent. I know IntelliJ Idea handles this case. On Wed, May 16, 2012 at 5:40 PM, kulkarni.swar...@gmail.com < kulkarni.swar...@gmail.com> wrote: > I installed datanucleus eclipse plugin as I realized that it is needed to > run some of the hive tests in eclipse. While trying to run the enhancer > tool, I keep getting this exception: > > "Exception occurred executing command line. Cannot run program > "/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home/bin/java" > ...Argument list is too long" > > Has anyone else encountered this too? My machine is running OS X 10.7. > > Thanks, > > Swarnim >
RE: hive-site.xml not found on classpath
You can see if the classpath is being passed correctly to hadoop by putting in
an echo statement around line 150 of the hive cli script where it passes the
CLASSPATH variable to HADOOP_CLASSPATH.
# pass classpath to hadoop
export HADOOP_CLASSPATH="${HADOOP_CLASSPATH}:${CLASSPATH}"
You could also echo the classpath in the hadoop script (in your HADOOP_HOME\bin
directory) to see if the classpath is being passed correctly to the time when
the cli jar is invoked.
As far as the logs location, if this is not set in your hive-site.xml, you can
set it by passing in HIVE_OPTS when you invoke the command line.
Like so:
EXPORT HIVE_OPTS= -hiveconf hive.log.dir=$ HIVE_HOME\logs"
Then run "hive"
Or:
Run "hive --hiveconf hive.log.dir=$ HIVE_HOME\logs"
Thanks,
Lauren
From: Stephen Boesch [mailto:java...@gmail.com]
Sent: Friday, November 30, 2012 12:16 AM
To: user@hive.apache.org
Subject: Re: hive-site.xml not found on classpath
running 0.9.0 (you can see it from the classpath shown below);
steve@mithril:/shared/cdh4$ echo $HIVE_CONF_DIR
/shared/hive/conf
steve@mithril:/shared/cdh4$ ls -l $HIVE_CONF_DIR
total 152
-rw-r--r-- 1 steve steve 46053 2011-12-13 00:36 hive-default.xml.template
-rw-r--r-- 1 steve steve 1615 2012-11-13 23:37
hive-env.bullshit.sh
-rw-r--r-- 1 steve steve 1671 2012-11-28 01:43 hive-env.sh
-rw-r--r-- 1 steve steve 1593 2011-12-13 00:36 hive-env.sh.template
-rw-r--r-- 1 steve steve 1637 2011-12-13 00:36
hive-exec-log4j.properties.template
-rw-r--r-- 1 root root 2056 2012-11-28 01:38 hive-log4j.properties
-rw-r--r-- 1 steve steve 2056 2012-03-25 12:49 hive-log4j.properties.template
-rw-r--r-- 1 steve steve 4415 2012-11-25 23:02 hive-site.xml
steve@mithril:/shared/cdh4$ echo $HIVE_HOME
/shared/hive
steve@mithril:/shared/cdh4$ echo $(which hive)
/shared/hive/bin/hive
also you can see the hive/conf is the first entry
After adding the debug statement:
classpath=/shared/hive/conf:/shared/hive/lib/antlr-runtime-3.0.1.jar:/shared/hive/lib/commons-cli-1.2.jar:/shared/hive/lib/commons-codec-1.3.jar:/shared/hive/lib/commons-collections-3.2.1.jar:/shared/hive/lib/commons-dbcp-1.4.jar:/shared/hive/lib/commons-lang-2.4.jar:/shared/hive/lib/commons-logging-1.0.4.jar:/shared/hive/lib/commons-logging-api-1.0.4.jar:/shared/hive/lib/commons-pool-1.5.4.jar:/shared/hive/lib/datanucleus-connectionpool-2.0.3.jar:/shared/hive/lib/datanucleus-core-2.0.3.jar:/shared/hive/lib/datanucleus-enhancer-2.0.3.jar:/shared/hive/lib/datanucleus-rdbms-2.0.3.jar:/shared/hive/lib/derby-10.4.2.0.jar:/shared/hive/lib/guava-r09.jar:/shared/hive/lib/hbase-0.92.0.jar:/shared/hive/lib/hbase-0.92.0-tests.jar:/shared/hive/lib/hive-builtins-0.9.0.jar:/shared/hive/lib/hive-cli-0.9.0.jar:/shared/hive/lib/hive-common-0.9.0.jar:/shared/hive/lib/hive-contrib-0.9.0.jar:/shared/hive/lib/hive_contrib.jar:/shared/hive/lib/hive-exec-0.9.0.jar:/shared/hive/lib/hive-hbase-handler-0.9.0.jar:/shared/hive/lib/hive-hwi-0.9.0.jar:/shared/hive/lib/hive-jdbc-0.9.0.jar:/shared/hive/lib/hive-metastore-0.9.0.jar:/shared/hive/lib/hive-pdk-0.9.0.jar:/shared/hive/lib/hive-serde-0.9.0.jar:/shared/hive/lib/hive-service-0.9.0.jar:/shared/hive/lib/hive-shims-0.9.0.jar:/shared/hive/lib/jackson-core-asl-1.8.8.jar:/shared/hive/lib/jackson-jaxrs-1.8.8.jar:/shared/hive/lib/jackson-mapper-asl-1.8.8.jar:/shared/hive/lib/jackson-xc-1.8.8.jar:/shared/hive/lib/JavaEWAH-0.3.2.jar:/shared/hive/lib/jdo2-api-2.3-ec.jar:/shared/hive/lib/jline-0.9.94.jar:/shared/hive/lib/json-20090211.jar:/shared/hive/lib/libfb303-0.7.0.jar:/shared/hive/lib/libfb303.jar:/shared/hive/lib/libthrift-0.7.0.jar:/shared/hive/lib/libthrift.jar:/shared/hive/lib/log4j-1.2.16.jar:/shared/hive/lib/mysql-connector-java-5.1.18-bin.jar:/shared/hive/lib/slf4j-api-1.6.1.jar:/shared/hive/lib/slf4j-log4j12-1.6.1.jar:/shared/hive/lib/stringtemplate-3.1-b1.jar:/shared/hive/lib/zookeeper-3.4.3.jar:
But even so:
* the log dir is still wrong (writing to /tmp/${user}/hive.log instead of
$HIVE_HOME/logs)
* the following message in the log file
2012-11-30 00:12:31,775 WARN conf.HiveConf (HiveConf.java:(70)) -
hive-site.xml not found on CLASSPATH
2012/11/30 Bing Li mailto:sarah.lib...@gmail.com>>
which version of hive do you use?
Could you try to add the following debug line in bin/hive before hive real
executes, and see the result?
echo "CLASSPATH=$CLASSPATH"
if [ "$TORUN" = "" ]; then
echo "Service $SERVICE not found"
echo "Available Services: $SERVICE_LIST"
exit 7
else
$TORUN "$@"
fi
The version I used is 0.9.0
2012/11/30 Stephen Boesch mailto:java...@gmail.com>>
Yes i do mean the log is in the wrong location, since it was set to a
persistent path in the $HIVE_CONF_DIR/lhive-log4j.properties.
None of the files in that directory appear to be picked up properly: neither
the hive-site.xml nor log4j.properties.
I have put echo statements into the 'hive" and the hive-config.sh shell
scripts and the echo statements prove that HIVE_CONF_DIR
Any existing UDTF to flatten map
Hi all,
I have a table like this:
hive> desc mytable;
ts bigint
content map
hive> select * from mytable;
1354299050{"F1":"id-1"}
1354299040{"F1":"id-2","F2":"id-3"}
1354299030{"F1":"id-3","F2":"id-1","F3":"id-4"}
Does anyone know how to generate a table like this:
hive> desc resulttable;
id string
ts bigint
hive> select * from resulttable;
id-11354299050
id-21354299040
id-31354299040
id-31354299030
id-11354299030
id-41354299030
I mean I can certainly write a UDTF to generate a table like this, but I
wonder is there an existing UDTF to do this job?
Thanks,
Xu
--
Ji Xu
Mobile: (+86) 138-115-10101
E-mail: jixuxj...@gmail.com
Homepage: http://about.me/jixu
Room 825, School of Software,
Main Building, Tsinghua University,
Beijing, 100084, P. R. China.
Re: Skew join failure
Hi David, It seems like Hive is unable to find the skewed keys on HDFS. Did you set *hive.skewjoin.key property? If so, to what value?* Mark On Fri, Nov 30, 2012 at 2:10 AM, David Morel wrote: > Hi, > > I am trying to solve the "last reducer hangs because of GC because of > truckloads of data" issue that I have on some queries, by using SET > hive.optimize.skewjoin=true; Unfortunately, every time I try this, I > encounter an error of the form: > ... > 2012-11-30 10:42:39,181 Stage-10 map = 100%, reduce = 100%, Cumulative > CPU 406984.1 sec > MapReduce Total cumulative CPU time: 4 days 17 hours 3 minutes 4 seconds > 100 msec > Ended Job = job_201211281801_0463 > java.io.FileNotFoundException: File hdfs://nameservice1/tmp/hive-** > dmorel/hive_2012-11-30_09-23-**00_375_8178040921995939301/-** > mr-10014/hive_skew_join_**bigkeys_0 does not exist. > at org.apache.hadoop.hdfs.**DistributedFileSystem.**listStatus(** > DistributedFileSystem.java:**365) > at org.apache.hadoop.hive.ql.**plan.**ConditionalResolverSkewJoin. > **getTasks(**ConditionalResolverSkewJoin.**java:96) > at org.apache.hadoop.hive.ql.**exec.ConditionalTask.execute(** > ConditionalTask.java:81) > at org.apache.hadoop.hive.ql.**exec.Task.executeTask(Task.** > java:133) > at org.apache.hadoop.hive.ql.**exec.TaskRunner.runSequential(** > TaskRunner.java:57) > at org.apache.hadoop.hive.ql.**Driver.launchTask(Driver.java:** > 1332) > at org.apache.hadoop.hive.ql.**Driver.execute(Driver.java:**1123) > at org.apache.hadoop.hive.ql.**Driver.run(Driver.java:931) > ... > > Googling didn't give me any indication on how to debug/solve this, so I'd > be glad if I could get any indication where to start looking. > > I'm using CMF4.0 currently, so Hive 0.8.1. > > Thanks a lot! > > David Morel >
Re: Storing data in TSV with changing headers
Hi Marc, While what Dean said is true for different schemas in general, there is a way to do it all in the same table if the schema changes to the TSV file are just additions of new tab-separated columns at the very end of each row and no existing columns are being deleted. Let's say your TSV file looked like a_val1b_val1c_val1 a_val2b_val2c_val2 Your table definition could look like: CREATE TABLE t1(a STRING, b STRING, c STRING) ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t'; Now say you insert more data into this table (externally through HDFS commands) which looks like a_val3b_val3c_val3d_val3 a_val4b_val4c_val4d_val4 Now, if you haven't changed the table definition just yet, it still says 3 columns, so a select * from t1 would still give you 3 columns. In order words, the newly added 4th column doesn't get read and doesn't have any adverse effect on the table if the table definition is still old. a_val1b_val1c_val1 a_val2b_val2c_val2 a_val3b_val3c_val3 a_val4b_val4c_val4 Now let's go ahead and change the table definition. You could use one of the ALTER TABLE commands for this. I personally like to create external tables, drop the tables and recreate them on top of the same data. Your new table create statement may look something like: CREATE TABLE t1(a STRING, b STRING, c STRING,d STRING) ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t'; A select * from t1 in this case would reveal: a_val1b_val1c_val1NULL a_val2b_val2c_val2NULL a_val3b_val3c_val3d_val3 a_val4b_val4c_val4d_val4 New columns show up correctly. Values from old data for which those columns didn't exist show up as NULLs. So, to summarize, a single table may work, if the following 3 conditions are met. 1. New columns get added to the very end of the existing columns 2. No existing columns are deleted 3. You are ok with NULL showing up as value for old records that don't have a given new column. I tested the above with CSV files, TSV should be the same as well. Mark On Fri, Nov 30, 2012 at 5:40 AM, Dean Wampler < dean.wamp...@thinkbiganalytics.com> wrote: > You'll have to define separate tables for the different schemas. You can > "unify" them in a query with the union feature. You should also remove the > header lines in the files, if you still have them, because Hive does not > ignore them, but treats them as "data". > > dean > > > On Fri, Nov 30, 2012 at 2:59 AM, Marc Canaleta wrote: > >> Hi all! >> >> We want to use hive to analyze our logs. Our logs will be TSV files, one >> per hour, and as it is possible that we add/remove more columns in the >> future, we will include headers (column names) in each file. >> >> So it is possible that two TSV files for different days/hours have >> different headers. >> >> Is it possible to do this with Hive? >> >> Thanks! >> > > > > -- > *Dean Wampler, Ph.D.* > thinkbiganalytics.com > +1-312-339-1330 > > >
Re:
I just configured hive.metastore.warehouse.dir with the new path and it works. Fixed .. 2012/11/28 Nitin Pawar > can you try providing a location 'path_to_file' and create the table again > > > On Wed, Nov 28, 2012 at 2:09 PM, imen Megdiche wrote: > >> Hello, >> >> I got this error when trying to create a test table with hive >> FAILED: Error in metadata: MetaException (message: Got exception: >> java.io.FileNotFoundException File file :/ user / hive / warehouse / test >> does >> not exist.) >> >> >> I changed the default directories warhouse hive.metastore.warehouse.dir the >> file hive-defaut.xml and I execute the commands for hadoop HDFS tocreate >> the new warehouse directory.but I have >> still this error >> >> thank you in advance >> > > > > -- > Nitin Pawar > >
Map Output in a file ?
Hello, It is possible to write map and merge outputs in external files in order to see them. Otherwhise how can i do to see the intermediate results. Thank you
Re: Storing data in TSV with changing headers
You'll have to define separate tables for the different schemas. You can "unify" them in a query with the union feature. You should also remove the header lines in the files, if you still have them, because Hive does not ignore them, but treats them as "data". dean On Fri, Nov 30, 2012 at 2:59 AM, Marc Canaleta wrote: > Hi all! > > We want to use hive to analyze our logs. Our logs will be TSV files, one > per hour, and as it is possible that we add/remove more columns in the > future, we will include headers (column names) in each file. > > So it is possible that two TSV files for different days/hours have > different headers. > > Is it possible to do this with Hive? > > Thanks! > -- *Dean Wampler, Ph.D.* thinkbiganalytics.com +1-312-339-1330
Skew join failure
Hi, I am trying to solve the "last reducer hangs because of GC because of truckloads of data" issue that I have on some queries, by using SET hive.optimize.skewjoin=true; Unfortunately, every time I try this, I encounter an error of the form: ... 2012-11-30 10:42:39,181 Stage-10 map = 100%, reduce = 100%, Cumulative CPU 406984.1 sec MapReduce Total cumulative CPU time: 4 days 17 hours 3 minutes 4 seconds 100 msec Ended Job = job_201211281801_0463 java.io.FileNotFoundException: File hdfs://nameservice1/tmp/hive-dmorel/hive_2012-11-30_09-23-00_375_8178040921995939301/-mr-10014/hive_skew_join_bigkeys_0 does not exist. at org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:365) at org.apache.hadoop.hive.ql.plan.ConditionalResolverSkewJoin.getTasks(ConditionalResolverSkewJoin.java:96) at org.apache.hadoop.hive.ql.exec.ConditionalTask.execute(ConditionalTask.java:81) at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:133) at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57) at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1332) at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1123) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:931) ... Googling didn't give me any indication on how to debug/solve this, so I'd be glad if I could get any indication where to start looking. I'm using CMF4.0 currently, so Hive 0.8.1. Thanks a lot! David Morel
Re: hive-site.xml not found on classpath
running 0.9.0 (you can see it from the classpath shown below);
steve@mithril:/shared/cdh4$ echo $HIVE_CONF_DIR
/shared/hive/conf
steve@mithril:/shared/cdh4$ ls -l $HIVE_CONF_DIR
total 152
-rw-r--r-- 1 steve steve 46053 2011-12-13 00:36 hive-default.xml.template
-rw-r--r-- 1 steve steve 1615 2012-11-13 23:37 hive-env.bullshit.sh
-rw-r--r-- 1 steve steve 1671 2012-11-28 01:43 hive-env.sh
-rw-r--r-- 1 steve steve 1593 2011-12-13 00:36 hive-env.sh.template
-rw-r--r-- 1 steve steve 1637 2011-12-13 00:36
hive-exec-log4j.properties.template
-rw-r--r-- 1 root root 2056 2012-11-28 01:38 hive-log4j.properties
-rw-r--r-- 1 steve steve 2056 2012-03-25 12:49
hive-log4j.properties.template
-rw-r--r-- 1 steve steve 4415 2012-11-25 23:02 hive-site.xml
steve@mithril:/shared/cdh4$ echo $HIVE_HOME
/shared/hive
steve@mithril:/shared/cdh4$ echo $(which hive)
/shared/hive/bin/hive
also you can see the hive/conf is the first entry
After adding the debug statement:
classpath=*/shared/hive/conf:*
/shared/hive/lib/antlr-runtime-3.0.1.jar:/shared/hive/lib/commons-cli-1.2.jar:/shared/hive/lib/commons-codec-1.3.jar:/shared/hive/lib/commons-collections-3.2.1.jar:/shared/hive/lib/commons-dbcp-1.4.jar:/shared/hive/lib/commons-lang-2.4.jar:/shared/hive/lib/commons-logging-1.0.4.jar:/shared/hive/lib/commons-logging-api-1.0.4.jar:/shared/hive/lib/commons-pool-1.5.4.jar:/shared/hive/lib/datanucleus-connectionpool-2.0.3.jar:/shared/hive/lib/datanucleus-core-2.0.3.jar:/shared/hive/lib/datanucleus-enhancer-2.0.3.jar:/shared/hive/lib/datanucleus-rdbms-2.0.3.jar:/shared/hive/lib/derby-10.4.2.0.jar:/shared/hive/lib/guava-r09.jar:/shared/hive/lib/hbase-0.92.0.jar:/shared/hive/lib/hbase-0.92.0-tests.jar:/shared/hive/lib/hive-builtins-0.9.0.jar:/shared/hive/lib/hive-cli-0.9.0.jar:/shared/hive/lib/hive-common-0.9.0.jar:/shared/hive/lib/hive-contrib-0.9.0.jar:/shared/hive/lib/hive_contrib.jar:/shared/hive/lib/hive-exec-0.9.0.jar:/shared/hive/lib/hive-hbase-handler-0.9.0.jar:/shared/hive/lib/hive-hwi-0.9.0.jar:/shared/hive/lib/hive-jdbc-0.9.0.jar:/shared/hive/lib/hive-metastore-0.9.0.jar:/shared/hive/lib/hive-pdk-0.9.0.jar:/shared/hive/lib/hive-serde-0.9.0.jar:/shared/hive/lib/hive-service-0.9.0.jar:/shared/hive/lib/hive-shims-0.9.0.jar:/shared/hive/lib/jackson-core-asl-1.8.8.jar:/shared/hive/lib/jackson-jaxrs-1.8.8.jar:/shared/hive/lib/jackson-mapper-asl-1.8.8.jar:/shared/hive/lib/jackson-xc-1.8.8.jar:/shared/hive/lib/JavaEWAH-0.3.2.jar:/shared/hive/lib/jdo2-api-2.3-ec.jar:/shared/hive/lib/jline-0.9.94.jar:/shared/hive/lib/json-20090211.jar:/shared/hive/lib/libfb303-0.7.0.jar:/shared/hive/lib/libfb303.jar:/shared/hive/lib/libthrift-0.7.0.jar:/shared/hive/lib/libthrift.jar:/shared/hive/lib/log4j-1.2.16.jar:/shared/hive/lib/mysql-connector-java-5.1.18-bin.jar:/shared/hive/lib/slf4j-api-1.6.1.jar:/shared/hive/lib/slf4j-log4j12-1.6.1.jar:/shared/hive/lib/stringtemplate-3.1-b1.jar:/shared/hive/lib/zookeeper-3.4.3.jar:
But even so:
- the log dir is still wrong (writing to /tmp/${user}/hive.log instead
of $HIVE_HOME/logs)
- the following message in the log file
2012-11-30 00:12:31,775 WARN conf.HiveConf
(HiveConf.java:(70)) - *hive-site.xml
not found on CLASSPATH*
* *
*
*
2012/11/30 Bing Li
> which version of hive do you use?
>
> Could you try to add the following debug line in bin/hive before hive real
> executes, and see the result?
>
> *echo "CLASSPATH=$CLASSPATH"*
>
> if [ "$TORUN" = "" ]; then
>echo "Service $SERVICE not found"
>echo "Available Services: $SERVICE_LIST"
>exit 7
> else
>$TORUN "$@"
> fi
>
> The version I used is 0.9.0
>
>
>
> 2012/11/30 Stephen Boesch
>
>> Yes i do mean the log is in the wrong location, since it was set to a
>> persistent path in the $HIVE_CONF_DIR/lhive-log4j.properties.
>>
>> None of the files in that directory appear to be picked up properly:
>> neither the hive-site.xml nor log4j.properties.
>>
>> I have put echo statements into the 'hive" and the hive-config.sh shell
>> scripts and the echo statements prove that HIVE_CONF_DIR is set properly:
>> /shared/hive/conf
>>
>> But even so the following problems occur:
>>
>>- the message "hive-site.xml is not found in the classpath"
>>- none of the hive-site.xml values are taking properly
>>- the log4j.properties in that same directory is not taking effect.
>>
>>
>>
>>
>> 2012/11/29 Bing Li
>>
>>> Hi, Stephen
>>> what did you mean the "wrong place under /tmp" in
>>> "I am seeing the following message in the logs (which are in the wrong
>>> place under /tmp..)" ?
>>>
>>> Did you mean that you set a different log dir but it didn't work?
>>>
>>> the log dir should be set in conf/hive-log4j.properties,
>>> conf/hive-exec-log4j.properties
>>> and you can try to reset HIVE_CONF_DIR in conf/hive-env.sh with ‘export"
>>> command.
>>>
>>> - Bing
>>>
>>>
>>> 2012/11/30 Stephen Boesch
>>>
thought i mentioned in the posts those were already set and verified..
but yes in any case that's first thing looked at.
Re: hive-site.xml not found on classpath
which version of hive do you use? Could you try to add the following debug line in bin/hive before hive real executes, and see the result? *echo "CLASSPATH=$CLASSPATH"* if [ "$TORUN" = "" ]; then echo "Service $SERVICE not found" echo "Available Services: $SERVICE_LIST" exit 7 else $TORUN "$@" fi The version I used is 0.9.0 2012/11/30 Stephen Boesch > Yes i do mean the log is in the wrong location, since it was set to a > persistent path in the $HIVE_CONF_DIR/lhive-log4j.properties. > > None of the files in that directory appear to be picked up properly: > neither the hive-site.xml nor log4j.properties. > > I have put echo statements into the 'hive" and the hive-config.sh shell > scripts and the echo statements prove that HIVE_CONF_DIR is set properly: > /shared/hive/conf > > But even so the following problems occur: > >- the message "hive-site.xml is not found in the classpath" >- none of the hive-site.xml values are taking properly >- the log4j.properties in that same directory is not taking effect. > > > > > 2012/11/29 Bing Li > >> Hi, Stephen >> what did you mean the "wrong place under /tmp" in >> "I am seeing the following message in the logs (which are in the wrong >> place under /tmp..)" ? >> >> Did you mean that you set a different log dir but it didn't work? >> >> the log dir should be set in conf/hive-log4j.properties, >> conf/hive-exec-log4j.properties >> and you can try to reset HIVE_CONF_DIR in conf/hive-env.sh with ‘export" >> command. >> >> - Bing >> >> >> 2012/11/30 Stephen Boesch >> >>> thought i mentioned in the posts those were already set and verified.. >>> but yes in any case that's first thing looked at. >>> >>> steve@mithril:~$ echo $HIVE_CONF_DIR >>> /shared/hive/conf >>> steve@mithril:~$ echo $HIVE_HOME >>> /shared/hive >>> >>> >>> 2012/11/29 kulkarni.swar...@gmail.com >>> >>> Have you tried setting HIVE_HOME and HIVE_CONF_DIR? On Thu, Nov 29, 2012 at 2:46 PM, Stephen Boesch wrote: > Yes. > > > 2012/11/29 Shreepadma Venugopalan > >> Are you seeing this message when your bring up the standalone hive >> cli by running 'hive'? >> >> >> On Thu, Nov 29, 2012 at 12:56 AM, Stephen Boesch >> wrote: >> >>> i am running under user steve. the latest log (where this shows up >>> ) is /tmp/steve/hive.log >>> >>> >>> 2012/11/29 Viral Bajaria >>> You are seeing this error when you run the hive cli or in the tasktracker logs when you run a query ? On Thu, Nov 29, 2012 at 12:42 AM, Stephen Boesch >>> > wrote: > > I am seeing the following message in the logs (which are in the > wrong place under /tmp..) > > hive-site.xml not found on classpath > > My hive-site.xml is under the standard location $HIVE_HOME/conf > so this should not happen. > > Now some posts have talked that the HADOOP_CLASSPATH was mangled. > Mine is not.. > > So what is the underlying issue here? > > Thanks > > stephenb > >>> >> > -- Swarnim >>> >>> >> >
