Re: hive-site.xml not found on classpath

2012-11-29 Thread Stephen Boesch
Yes i do mean the log is in the wrong location, since it was set to a persistent path in the $HIVE_CONF_DIR/lhive-log4j.properties. None of the files in that directory appear to be picked up properly: neither the hive-site.xml nor log4j.properties. I have put echo statements into the 'hive" and t

Re: hive-site.xml not found on classpath

2012-11-29 Thread Bing Li
Hi, Stephen what did you mean the "wrong place under /tmp" in "I am seeing the following message in the logs (which are in the wrong place under /tmp..)" ? Did you mean that you set a different log dir but it didn't work? the log dir should be set in conf/hive-log4j.properties, conf/hive-exec-log

Re: hive-site.xml not found on classpath

2012-11-29 Thread Stephen Boesch
thought i mentioned in the posts those were already set and verified.. but yes in any case that's first thing looked at. steve@mithril:~$ echo $HIVE_CONF_DIR /shared/hive/conf steve@mithril:~$ echo $HIVE_HOME /shared/hive 2012/11/29 kulkarni.swar...@gmail.com > Have you tried setting HIVE_HOME

Re: hive-site.xml not found on classpath

2012-11-29 Thread Stephen Boesch
Yes. 2012/11/29 Shreepadma Venugopalan > Are you seeing this message when your bring up the standalone hive cli by > running 'hive'? > > > On Thu, Nov 29, 2012 at 12:56 AM, Stephen Boesch wrote: > >> i am running under user steve. the latest log (where this shows up ) is >> /tmp/steve/hive.lo

Re: hive-site.xml not found on classpath

2012-11-29 Thread Shreepadma Venugopalan
Are you seeing this message when your bring up the standalone hive cli by running 'hive'? On Thu, Nov 29, 2012 at 12:56 AM, Stephen Boesch wrote: > i am running under user steve. the latest log (where this shows up ) is > /tmp/steve/hive.log > > > 2012/11/29 Viral Bajaria > >> You are seeing

Re: Hive Loading Zip CSV Files

2012-11-29 Thread Mark Grover
Adding user@hive.apache.org Ben, That's great to hear. It would be awesome if you'd like to contribute this back to Hive so others in the community could use it too. Let us know what you think! Mark On Wed, Nov 28, 2012 at 8:05 PM, ben wrote: > Mark, > > Just wanted to let you know that I got

Re: Starting with Hive - writing custom SerDe

2012-11-29 Thread Dean Wampler
Similarly, Pig is pretty nice for this type of data cleansing and it's a little more flexible However, for completeness, I'll mention an alternative to writing a UDF; use the Hive streaming feature, where you stream the data through a program you write in any language you want and format the data

RE: Starting with Hive - writing custom SerDe

2012-11-29 Thread Connell, Chuck
I meant PLAIN tab-separated text. From: Connell, Chuck [mailto:chuck.conn...@nuance.com] Sent: Thursday, November 29, 2012 9:51 AM To: user@hive.apache.org Subject: RE: Starting with Hive - writing custom SerDe You might save yourself a lot of work by pre-processing the data, before putting it

RE: Starting with Hive - writing custom SerDe

2012-11-29 Thread Connell, Chuck
You might save yourself a lot of work by pre-processing the data, before putting it into Hive. A Python script should be able to find all the fields, and change the data to plan tab-separated text. This will load directly into Hive, and removes the need to write a custom SerDe. Chuck Connell Nu

Starting with Hive - writing custom SerDe

2012-11-29 Thread Fernando Andrés Doglio Turissini
Hello everyone, I'm starting to play around with Hive, and I have to load a traffic data log file into a table. My problem is that the lines of the file don't really have a nice separator for each field (on the same line, there are serveral blank or hyphens or single blank spaces used as separators

Re: is Hive support TTL(time to live) now ?

2012-11-29 Thread Edward Capriolo
their is a table level property retention it does not do anything but you can write a program to sweep through the metastore and take action based on that or other custom properties. On Thursday, November 29, 2012, Nitin Pawar wrote: > hive will not delete anything unless you ask for it > if your

Re: Compile hive on hadoop 1.0.4

2012-11-29 Thread Barak Yaish
Still: BUILD FAILED /workspace/development/org/hive/trunk/build.xml:256: The following error occurred while executing this line: /workspace/development/org/hive/trunk/ant/build.xml:38: /workspace/development/org/hive/trunk/build/ivy/lib/hadoop0.20.shim not found. Anything else I missed? On Thu,

Re: is Hive support TTL(time to live) now ?

2012-11-29 Thread Nitin Pawar
hive will not delete anything unless you ask for it if your tables are on hdfs tmp folder, hadoop will wipe them off if you dont use them for the tmp time period. If you want to delete data from tables or complete tables, you will need to schedule crons or look at few workflow engines I may be wr

RE: is Hive support TTL(time to live) now ?

2012-11-29 Thread Hezhiqiang (Ransom)
Hi Nitin TTL for data alive in hive. Can hive automatically delete data when out of date? Regards Ransom From: Nitin Pawar [mailto:nitinpawar...@gmail.com] Sent: Thursday, November 29, 2012 7:53 PM To: user@hive.apache.org Cc: Zhouxunmiao; Zhangjian (Jack); Zhaojun (Terry) Sub

RE: Compile hive on hadoop 1.0.4

2012-11-29 Thread Hezhiqiang (Ransom)
Hi Barak You can used –Dhadoop.version=1.0.4 For example: ant -Dhadoop.version=2.0.1 very-clean package tar test -Dinclude.postgres=true -Doverwrite=true -Dtest.silent=false -logfile ant.log Regards Ransom From: Barak Yaish [mailto:barak.ya...@gmail.com] Sent: Thursday, November 29, 2012 6:53 A

Re: is Hive support TTL(time to live) now ?

2012-11-29 Thread Nitin Pawar
TTL for what ? for hive queries or hive over thrift or anything else you have in mind? for query execution they can run by default till the time map-reduce runs. On Thu, Nov 29, 2012 at 5:09 PM, Hezhiqiang (Ransom) < ransom.hezhiqi...@huawei.com> wrote: > Hi all > Is Hive support TTL(t

is Hive support TTL(time to live) now ?

2012-11-29 Thread Hezhiqiang (Ransom)
Hi all Is Hive support TTL(time to live) now? I search in JIRA but can't found it , is hive support it? Regards Ransom

Re: hive-site.xml not found on classpath

2012-11-29 Thread Stephen Boesch
i am running under user steve. the latest log (where this shows up ) is /tmp/steve/hive.log 2012/11/29 Viral Bajaria > You are seeing this error when you run the hive cli or in the tasktracker > logs when you run a query ? > > On Thu, Nov 29, 2012 at 12:42 AM, Stephen Boesch wrote: > >> >> I

Re: hive-site.xml not found on classpath

2012-11-29 Thread Viral Bajaria
You are seeing this error when you run the hive cli or in the tasktracker logs when you run a query ? On Thu, Nov 29, 2012 at 12:42 AM, Stephen Boesch wrote: > > I am seeing the following message in the logs (which are in the wrong > place under /tmp..) > > hive-site.xml not found on classpath >

hive-site.xml not found on classpath

2012-11-29 Thread Stephen Boesch
I am seeing the following message in the logs (which are in the wrong place under /tmp..) hive-site.xml not found on classpath My hive-site.xml is under the standard location $HIVE_HOME/conf so this should not happen. Now some posts have talked that the HADOOP_CLASSPATH was mangled. Mine is no