Re: nested join issue

2015-06-12 Thread Gautam
Found that turning off hive.optimize.remove.identity.project ( ref: HIVE-8435 ) fixes the issue. This gives us a workaround, but dunno the performance degradation this impacts yet. Thanks! -Gautam. On Fri, Jun 12, 2015 at 6:02 PM, Gautam wrote:

Re: delta file compact take no effect

2015-06-12 Thread Eugene Koifman
Delta files that are no longer needed are deleted asynchronously. For example, you may have some query using delta_002_002. A minor compaction, for example, can run concurrently and create delta_001_003 but it will leave delta_001_001, delta_002_002, delta_003_

Equal predicate on timestamp column

2015-06-12 Thread Jie Zhang
Hi, I have a table partitioned on every hour, the partitioning column ds is timestamp type. However, I could not locate one partition with the equal predicate on ds, only the range predicates works. Here are the ddl and queries: create table test (c1 int, c2 string) partitioned by (ds timestamp)

Re: nested join issue

2015-06-12 Thread Gautam
Done. https://issues.apache.org/jira/browse/HIVE-10996 On Fri, Jun 12, 2015 at 1:47 PM, Gopal Vijayaraghavan wrote: > Hi > > > Thanks for investigating.. Trying to locate the patch that fixes this > >between 1.1 and 2.0.0-SNAPSHOT. Any leads on what Jira this fix was part > >of? Or what part of

Hive transaction feature in Hive 1.0

2015-06-12 Thread Jim Green
Hi Team, Sharing the article which explains the Hive transaction features in Hive 1.0: Hive transaction feature in Hive 1.0 -- Thanks, www.openkb.info (Open KnowledgeBase for Hadoop/Database/OS/Network/Tool)

Re: nested join issue

2015-06-12 Thread Gopal Vijayaraghavan
Hi > Thanks for investigating.. Trying to locate the patch that fixes this >between 1.1 and 2.0.0-SNAPSHOT. Any leads on what Jira this fix was part >of? Or what part of the code the patch is likely to be on? git bisect is the only way usually to identify these things. But before you hunt into

Re: HBase and Hive integration

2015-06-12 Thread Buntu Dev
Thanks Nick for the write up. It was quite helpful for a newbie like me. Is there any Hive config to provide the zookeeper quorum for the HBase cluster since I got Hive and HBase on separate clusters? Thanks! On Tue, Jun 9, 2015 at 12:03 AM, Nick Dimiduk wrote: > Hi there. > > I go through a c

Writing uniontype to ORC file outside of hive

2015-06-12 Thread Dave Maughan
Hi, I'm trying to write ORC files outside of hive and I'm currently looking at uniontype. I've identified multiple ways to do this. I could use a Writable or Java StandardStructObjectInspector and pass in a StandardUnion. However, the OrcInputFormat uses OrcStruct as the value type and I'd like t

hive on spark spark.executor.instances has no effect

2015-06-12 Thread ????
I based on "http://blog.cloudera.com/blog/2015/02/download-the-hive-on-spark-beta/"; and "http://blog.cloudera.com/blog/2015/02/download-the-hive-on-spark-beta/ " Set parameters "spark.executor.instances = 12" (I have four nodes), when I execute hive sql, spark executors are always 3, 1 driver a