Re: Can hive 0.8.1 work with hadoop 0.23.0?

Hi Xiaofeng, Which mode are you running Hadoop in, e.g. local, pseudo-distributed, or distributed? Thanks. Carl 2012/2/1 张晓峰 zhangxiaofe...@q.com.cn Hi, ** ** I installed hadoop 0.23.0 which can work. The version of my hive is 0.8.1. The query like ‘select * from tablename’

RE: Can hive 0.8.1 work with hadoop 0.23.0?

Hi Xiaofeng, Backup “hive_exec.jar” in all hadop directory, then delete “hive_exec.jar”. Try it. Because “select * just use hdfs . And “select col1” will use MapReduce. Best regards Ransom. From: Carl Steinbach [mailto:c...@cloudera.com] Sent: Tuesday, February 21, 2012 4:45 PM To:

SerDe and InputFormat

I make changes to the Configuration in my SerDe expecting those to be passed to the InputFormat (and OutputFormat). Yet the InputFormat seems to get an unchanged JobConf? Is this a known limitation? I find it very confusing since the Configuration is the main way to communicate with the MapReduce

Re: 2 questions about SerDe

Have a look at the code for the LazySerDes. When you deserialize in the SerDe, you don't actually have to deserialize all the columns. Deserialized could return an object that is not actually deserialized and you can write an ObjectInspector that deserializes a field from that structure but only

Re: help with compression and index

Hi Hamilton     When you are doing indexing(generate index files) is compression enabled? If so you are running into this known issue https://issues.apache.org/jira/browse/HIVE-2331 Which is fixed in hive 0.8 . An upgrade should get it rolling for you and is recommended. Regards Bejoy.K.S

Custom SerDe -- tracking down stack trace

I have a custom SerDe that's initializing properly and works on one data set. I built it to adapt to a couple of different data formats, though, and it's choking on a different data set (different partitions in the same table). A null pointer exception is being thrown on deserialize, that's

Re: help with compression and index

Hi Robert, As per https://issues.apache.org/jira/browse/HIVE-1644, Hive 0.8 introduces automatic accessing of indexes. That might come in handy too! Mark Mark Grover, Business Intelligence Analyst OANDA Corporation www: oanda.com www: fxtrade.com e: mgro...@oanda.com Best Trading Platform

Re: Custom SerDe -- tracking down stack trace

One more data point: I can read data from this partition as long as I don't reference the partition explicitly… E.g., I my partition column is ArrivalDate, and I have several different partitions: 2012-02-01…, and a partition with my test data with ArrivalDate=test. This works: 'select *