why alter table tbl add partition(...) location ... need write permission to the target location?

2009-05-20 Thread Min Zhou
Hey guys, I tried to add an external partition for my web_log table, but failed like below, hive> ALTER TABLE web_log ADD PARTITION (logdate='20090519') LOCATION '/user/zhoumin/web_log/20090519/'; FAILED: Error in metadata: MetaException(message:Got exception: org.apache.hadoop.security.AccessCon

Re: why alter table tbl add partition(...) location ... need write permission to the target location?

2009-05-20 Thread Min Zhou
Sorry, my bad! the directory I loaded isn't exists yet. On Wed, May 20, 2009 at 4:10 PM, Min Zhou wrote: > Hey guys, > > I tried to add an external partition for my web_log table, but failed like > below, > > hive> ALTER TABLE web_log ADD PARTITION (logdate='20090519') LOCATION > '/user/zhoumin/

Re: hive hwi

2009-05-20 Thread Manhee Jo
Thank you Edward, It went well at last by copying the *.jar files to hive/lib. But it's really strange. I have all the directories in my path environment. In addition, I've tried many times to set it using my .tcshrc files, hadoop-env.sh etc. Excuse me for a naive question, but which is the best

Re: hive hwi

2009-05-20 Thread Edward Capriolo
I have a RedHat/CentOS system. I find the best way is to add scripts to /etc/profile.d /etc/profile.d/hive.sh --- PATH=$PATH:/opt/hive/bin ANT_LIB=/opt/ant/lib export PATH export ANT_LIB --- I did something like that here. http://wiki.apache.org/hadoop/HiveDerbyServerMode On Wed, May 20, 2009 a

Can Hive recognize commented out line in data files while loading?

2009-05-20 Thread Manhee Jo
Is it possible for hive to recognize commented rows in a file when it loads a csv file? For example, say contents of test.csv is, #123 #Red, Brown, Black, Blue 3, AB, 5, 3 2, AA, 1, 4 ... In hive, how to ignore first two lines while loading? Thanks, Manhee

Re: Can Hive recognize commented out line in data files while loading?

2009-05-20 Thread Zheng Shao
The Hive internal Serdes do not allow this format yet. We will need to change Hive code to make that happen. Specifically, it's the LazySimpleSerDe class. Zheng On Wed, May 20, 2009 at 11:05 PM, Manhee Jo wrote: > Is it possible for hive to recognize commented rows in a file when it loads > a c

Re: Can Hive recognize commented out line in data files while loading?

2009-05-20 Thread Raghu Murthy
I guess the question was about loading the data. In the load command, we currently just copy over the data without parsing it (via the CopyTask). Even though we could choose to neglect commented rows at query time (via SerDes), its probably more efficient to do it once while loading. In addition,