Re: Null Pointer Exception with tez

2014-10-06 Thread Suma Shivaprasad
UNION ALL is supported on Tez 0.4 onwards. If any of the tables are empty, that could be an issue- refer HIVE-8227 Thanks Suma On Mon, Oct 6, 2014 at 12:51 PM, João Alves wrote: > Hey Anusha, > > You sure about that? > > The following works for me: > > set hive.execution.engine=tez; > select A

Re: OOM with Hive on Tez

2014-08-26 Thread Suma Shivaprasad
INFO [Thread-13]: exec.Task (TezTask.java:execute(171)) - org.apache.hadoop.hive.ql.exec.MapOperator$Counter: 2014-08-26 14:06:41,212 INFO [Thread-13]: exec.Task (TezTask.java:execute(173)) -DESERIALIZE_ERRORS: 0 Thanks Suma On Tue, Aug 26, 2014 at 7:47 PM, Suma Shivaprasad

OOM with Hive on Tez

2014-08-26 Thread Suma Shivaprasad
Trying to run a query on Tez with the following configurations *set hive.tez.container.size=5120* *set mapreduce.map.child.java.opts=-Xmx5120M* *set hive.tez.java.opts=-Xmx4096M* *set hive.auto.convert.join.noconditionaltask.size=805306000* *set tez.am.resource.memory.mb=5120* *set tez.am.java.op

Hive Tez Counters

2014-08-19 Thread Suma Shivaprasad
Hi, Needed info on where I can get detailed job counters for Hive on Tez. Am running this on a HDP cluster with Hive 0.13 and see only the following job counters through Hive Tez in Yarn logs . Cannot see any ReduceOperator counters and the CPU_MILLISECONDS in some cases in -ve. Is this accurate?

Re: Null pointer error with UNION ALL on partitioned tables

2014-08-15 Thread Suma Shivaprasad
Hi Chris, According to this - https://issues.apache.org/jira/browse/HIVE-6098, UNION ALL is still not supported on Tez , and so are SMB joins Thanks Suma On Sat, Aug 16, 2014 at 4:09 AM, Dragga, Christopher < chris.dra...@netapp.com> wrote: > Hi- > > > > I’m using Hive 0.13.1 on Tez and run

Re: Exception in Hive with SMB join and Parquet

2014-07-30 Thread Suma Shivaprasad
Wed, Jul 30, 2014 at 6:43 PM, Suma Shivaprasad < > sumasai.shivapra...@gmail.com> wrote: > >> Am using 0.13.0 version of hive with parquet table having 34 columns with >> the following props while creating the table >> >> >> *CLUSTERED BY (udid) SORTED BY

Exception in Hive with SMB join and Parquet

2014-07-30 Thread Suma Shivaprasad
Am using 0.13.0 version of hive with parquet table having 34 columns with the following props while creating the table *CLUSTERED BY (udid) SORTED BY (udid ASC) INTO 256 BUCKETS STORED as PARQUET TBLPROPERTIES ("parquet.compression"="SNAPPY"); * The query I am running is *set hive.optimize.bu

Re: Reg:Column Statistics with Parquet

2014-07-25 Thread Suma Shivaprasad
Hi , I tried the same with compute statistics for columns a, b,c as above and still seeing the same results in explain plan. How do I confirm if its generating all the column stats for a given column. If this is confirmed, we can debug why Hive is still not using it? Thanks Suma On Thu, Jul 24

Column Statistics with Parquet

2014-07-24 Thread Suma Shivaprasad
I am trying to enable Column statistics usage with Parquet tables. This is the query I am executing. However on explain, I see that even though *Basic stats: COMPLETE *is seen *Column stats *is seen as* NONE.* Can someone please explain what else I need to debug/fix this. set hive.compute.query.us

Fwd: Column Stats with parquet

2014-07-24 Thread Suma Shivaprasad
I am trying to enable Column statistics usage with Parquet tables. This is the query I am executing. However on explain, I see that even though *Basic stats: COMPLETE *is seen *Column stats *is seen as* NONE.* Can someone please explain what else I need to debug/fix this. set hive.compute.query.us

Column Stats with parquet

2014-07-24 Thread Suma Shivaprasad
I am trying to enable Column statistics usage with Parquet tables. This is the query I am executing. However on explain, I see that even though *Basic stats: COMPLETE *is seen *Column stats *is seen as* NONE.* Can someone please explain what else I need to debug/fix this. set hive.compute.query.us

Column Statistics with Parquet

2014-07-23 Thread Suma Shivaprasad
I am trying to enable Column statistics usage with Parquet tables. This is the query I am executing. However on explain, I see that even though *Basic stats: COMPLETE *is seen *Column stats *is seen as* NONE.* Can someone please explain what else I need to debug/fix this. set hive.compute.query.us

Column Statistics with Parquet

2014-07-23 Thread Suma Shivaprasad
I am trying to enable Column statistics usage with Parquet tables. This is the query I am executing. However on explain, I see that even though *Basic stats: COMPLETE *is seen *Column stats *is seen as* NONE.* Can someone please explain what else I need to debug/fix this. set hive.compute.query.us

ORC with bloom filters

2014-07-17 Thread Suma Shivaprasad
Hi, I do not see any examples of extending ORC with bloom filters on Hive documentation. Can someone please explain what exactly needs to be done to enable this. Thanks