RE: [Hive] Slow Loading Data Process with Parquet over 30k Partitions

2015-04-13 Thread Xu, Cheng A
Hi Tianqi, Can you attach hive.log as more detailed information? +Sergio Yours, Ferdinand Xu From: Tianqi Tong [mailto:tt...@brightedge.com] Sent: Friday, April 10, 2015 1:34 AM To: user@hive.apache.org Subject: [Hive] Slow Loading Data Process with Parquet over 30k Partitions Hello Hive, I'm a

same query works with TEXTFILE and fails with ORC

2015-04-13 Thread Verhaeghe Philippe
I'm getting an error in Hive when executing a query on a table in ORC format. After several trials, I succeeded to run the same query on the same table in TEXTFILE format. I 've been able to reproduce the error with the simple sql script below. I create the same table in TEXFILE and in ORC and I

Re: same query works with TEXTFILE and fails with ORC

2015-04-13 Thread Gopal Vijayaraghavan
I¹m getting an error in Hive when executing a query on a table in ORC format. This is not an ORC bug, this looks like a vectorization issue. Can you try comparing both query plans (³explain query²) for the Execution mode: vectorized markers? TextFile queries are not vectorized today, since you

RE: same query works with TEXTFILE and fails with ORC

2015-04-13 Thread Verhaeghe Philippe
Bug created in JIRA as HIVE-10316 -Message d'origine- De : Gopal Vijayaraghavan [mailto:go...@hortonworks.com] De la part de Gopal Vijayaraghavan Envoyé : Monday, April 13, 2015 11:46 PM À : user@hive.apache.org Objet : Re: same query works with TEXTFILE and fails with ORC I¹m getting