Re: Regex and serde with hive

2011-12-22 Thread Vijay
If the format is simply delimited like this, you don't need to use the RegexSerde. Hive's default format with the right "FIELDS TERMINATED BY" setting will work great. -Vijay On Thu, Dec 22, 2011 at 8:49 PM, Raghunath, Ranjith wrote: > I have been struggling with this for a while so I would appr

Re: Regex and serde with hive

2011-12-22 Thread Loren Siebert
The input regexp does not look right to me. You are expecting a space between groups, but your example contains no spaces. And where do you handle the first/last quotes? Wouldn’t it look more like this: "input.regex" = “\"([^\"~]*)[\"~]*([^\"~]*)[\"~]*([^\"~]*)\"" Rather than trying to tackle it

Regex and serde with hive

2011-12-22 Thread Raghunath, Ranjith
I have been struggling with this for a while so I would appreciate any advice that you any of you may have. I have a file of the format "Xyz"~"qsd"~"1234" I created the following table definition to get the data loaded CREATE TABLE dummy (f1 string, f2string, f3 string) ROW FORMA

Re:Re: a serious issue of hiveserver

2011-12-22 Thread 王锋
The Exception “java.lang.StackOverflowError” should be depended on the Xss value . 在 2011-12-22 16:15:41,"alo alt" 写道: >Hi, > >depends on the memory you configured for hive (xms, xmx). > >- Alex > >2011/12/14 王锋 : >> when hiveserver is executing sql, it throws an exception: >> >> 2011-12-14

Re: millions of columns in Hive

2011-12-22 Thread ameet chaubal
thanks Kirk, Actually, it saves the columns in a row format in a table "COLUMNS" So all the columns are stored as records in this table. i.e. for a million row table, there will be a million rows which MySQL should have no problem with, right? I did see a post regarding the 4000 bytes,but that w

Re: millions of columns in Hive

2011-12-22 Thread Kirk True
IIRC, the names and types of the columns are kept in a *_PARAMS table in the metastore. At one point (depending on your DB) this was only ~4000 characters and thus posed a limitation on the number of columns one could use. I believe there was work to turn some of the *_PARAMS table columns int

Re: LEFT OUTER JOIN and partition

2011-12-22 Thread Mark Grover
Hi Michael, Thank you for the detailed explanation. I wouldn't have expected SELECT count(A.id) FROM a LEFT OUTER JOIN b ON (A.id=B.id AND A.dt=X AND B.dt=X) WHERE B.id IS NULL; to give |A(dt=*)-B(dt=X)| Can you try the following?: set hive.optimize.ppd=true; SELECT count(A.id) FROM a LEFT OUTE

Re: Schemas/Databases in Hive

2011-12-22 Thread Raghunath, Ranjith
Thanks Bejoy. This was helpful. Thanks, Ranjith From: bejoy...@yahoo.com [mailto:bejoy...@yahoo.com] Sent: Thursday, December 22, 2011 11:23 AM To: user@hive.apache.org Subject: Re: Schemas/Databases in Hive Also multiple databases have proved helpful for me in organizing tables into correspon

Re: Schemas/Databases in Hive

2011-12-22 Thread bejoy_ks
Also multiple databases have proved helpful for me in organizing tables into corresponding databases when you have quite a large number of tables to manage. Also I believe it'd be helpful in providing access restrictions. Regards Bejoy K S -Original Message- From: bejoy...@yahoo.com Da

Re: Schemas/Databases in Hive

2011-12-22 Thread bejoy_ks
Ranjith Hive do support multiple data bases if you are on some of the latest versions of hive try Create database testdb; Use testdb; It should give you what you are looking for. Regards Bejoy K S -Original Message- From: "Raghunath, Ranjith" Date: Thu, 22 Dec 2011 17:02:09 To:

Re: Hive installation problem: java.lang.NoSuchFieldError: type

2011-12-22 Thread Jian Fang
Please also upgrade the thrift library. The thrift version Hive uses is lower than the one in Cassandra 1.0.X and will cause problem if I try to run hive --service hiveserver Thanks, John On Thu, Nov 10, 2011 at 2:07 PM, Ashutosh Chauhan wrote: > We are planning to upgrade antlr version which

Schemas/Databases in Hive

2011-12-22 Thread Raghunath, Ranjith
What is the intent of having tables in different databases or schemas in Hive? Thanks Thank you, Ranjith

RE: LEFT OUTER JOIN and partition

2011-12-22 Thread mdefoinplatel.ext
I have been looking at this problem in more details... So let say you have to tables A and B containing distinct Ids and partitioned by date. Let say for each partition, B is a subset of A (all the rows in B are present in A) If you want to compute the size of set difference A - B for a given

Re: Hive installation problem: java.lang.NoSuchFieldError: type

2011-12-22 Thread Jian Fang
Thanks Nicolas. You are right. I looked at my hadoop classpath and found it included an antlr-3.2.jar jar in the Cassandra library (My hadoop cluster is integrated with a Cassandra cluster). Thanks, John 2011/11/10 Nicolas Lalevée > > Le 4 nov. 2011 à 17:19, Jian Fang a écrit : > > > Hi, > > >

LEFT OUTER JOIN and partition

2011-12-22 Thread mdefoinplatel.ext
Hi folks, Ok, I give you the context first : I have seen in the mailing list people advising the use of a ... LEFT OUTER JOIN ON ... followed by a WHERE close to write a NOT IN type of query. So to make it clear, it should be possible to rewrite the following query: SELECT a.id FROM a WHERE a.id

Re: a serious issue of hiveserver

2011-12-22 Thread alo alt
Hi, depends on the memory you configured for hive (xms, xmx). - Alex 2011/12/14 王锋 : > when hiveserver is executing sql, it throws an exception: > > 2011-12-14 00:40:43,762 Stage-1 map = 90%,  reduce = 13% > 2011-12-14 00:40:45,771 Stage-1 map = 92%,  reduce = 13% > 2011-12-14 00:40:46,777 Stage

Does hive REAL enable TestHadoop20SAuthBridge in hive-0.8.0 ? -- [HIVE-2257] patch doesn't work for me

2011-12-22 Thread Bing Li
Hi, All When I ran hive UT, I found that TestHadoop20SAuthBridge wasn't compiled, so TestHadoop20SAuthBridge won't be run by "ant test" command. In src/shims/build.xml, I found the following lines: Then, I commented off lines in blue, and it could generate the class file of