Re: Review Request 33251: HIVE-10302 Cache small tables in memory [Spark Branch]

2015-04-22 Thread Jimmy Xiang
On April 21, 2015, 2:17 p.m., Xuefu Zhang wrote: ql/src/java/org/apache/hadoop/hive/ql/exec/spark/HashTableLoader.java, line 120 https://reviews.apache.org/r/33251/diff/2/?file=937278#file937278line120 1. For clarity, it might be good to put this in a separate private method.

[jira] [Created] (HIVE-10440) Architecture for ResultSet Compression via external plugin

2015-04-22 Thread Rohit Dholakia (JIRA)
Rohit Dholakia created HIVE-10440: - Summary: Architecture for ResultSet Compression via external plugin Key: HIVE-10440 URL: https://issues.apache.org/jira/browse/HIVE-10440 Project: Hive

Re: Review Request 33251: HIVE-10302 Cache small tables in memory [Spark Branch]

2015-04-22 Thread Jimmy Xiang
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33251/ --- (Updated April 22, 2015, 4:36 p.m.) Review request for hive, Chao Sun, Szehon

[jira] [Created] (HIVE-10439) Architecture for ResultSet Compression via external plugin

2015-04-22 Thread Rohit Dholakia (JIRA)
Rohit Dholakia created HIVE-10439: - Summary: Architecture for ResultSet Compression via external plugin Key: HIVE-10439 URL: https://issues.apache.org/jira/browse/HIVE-10439 Project: Hive

Re: Review Request 33367: Aggregate stats cache for RDBMS based metastore codepath

2015-04-22 Thread Mostafa Mokhtar
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33367/#review81170 ---

[jira] [Created] (HIVE-10438) Architecture for ResultSet Compression via external plugin

2015-04-22 Thread Rohit Dholakia (JIRA)
Rohit Dholakia created HIVE-10438: - Summary: Architecture for ResultSet Compression via external plugin Key: HIVE-10438 URL: https://issues.apache.org/jira/browse/HIVE-10438 Project: Hive

[jira] [Created] (HIVE-10437) NullPointerException on queries where map/reduce is not involved on tables with partitions

2015-04-22 Thread Demeter Sztanko (JIRA)
Demeter Sztanko created HIVE-10437: -- Summary: NullPointerException on queries where map/reduce is not involved on tables with partitions Key: HIVE-10437 URL: https://issues.apache.org/jira/browse/HIVE-10437

Build failed in Jenkins: HIVE-TRUNK-JAVA8 #62

2015-04-22 Thread hiveqa
See http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/HIVE-TRUNK-JAVA8/62/changes Changes: [hashutosh] HIVE-10396 : decimal_precision2.q test is failing on trunk (Jason Dere via Ashutosh Chauhan) [prasanthj] HIVE-10303: HIVE-9471 broke forward compatibility of ORC files (Prasanth

Re: Preparation for Hive-1.2 release

2015-04-22 Thread 김영우
Hi, I would like to add a request for support Spark 1.3, HIVE-9726. Currently Apache Bigtop includes Spark 1.3 and we ran into a compatibility issue for Hive 1.1+. The issue have been resolved on 'spark-branch' but could not find a 'fix version' for it. I hope that the fix merge into release

Re: Review Request 33367: Aggregate stats cache for RDBMS based metastore codepath

2015-04-22 Thread Thejas Nair
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33367/#review81122 ---

Re: Review Request 33367: Aggregate stats cache for RDBMS based metastore codepath

2015-04-22 Thread Thejas Nair
On April 22, 2015, 3:47 a.m., Thejas Nair wrote: metastore/src/java/org/apache/hadoop/hive/metastore/AggregateStatsCache.java, line 489 https://reviews.apache.org/r/33367/diff/1/?file=935572#file935572line489 this is not being used anywhere, can be removed. never mind, i see its

Re: Review Request 33367: Aggregate stats cache for RDBMS based metastore codepath

2015-04-22 Thread Mostafa Mokhtar
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33367/#review81127 --- common/src/java/org/apache/hive/common/util/BloomFilter.java

Re: Preparation for Hive-1.2 release

2015-04-22 Thread Sushanth Sowmyan
@Lefty : Yup, I've been following the other thread and https://issues.apache.org/jira/browse/INFRA-9488 - I think that'll stabilize before we fork. I'll definitely wait for it, but also wanted to do a reminder/call-for-patches before it. :) @Youngwoo: Thanks, I'll add that to the tracker. I do

svn commits

2015-04-22 Thread Ashutosh Chauhan
are they already turned off because of git transition? I tried a commit on svn trunk today morning and got access denied exception. Any one else getting this exception? If it indeed is because of git transition, do any one know when commits on git repo will be enabled?

Re: svn commits

2015-04-22 Thread Xuefu Zhang
Yes. I believe so. You can follow the progress in INFRA-9488 On Wed, Apr 22, 2015 at 10:23 AM, Ashutosh Chauhan hashut...@apache.org wrote: are they already turned off because of git transition? I tried a commit on svn trunk today morning and got access denied exception. Any one else getting

Re: svn commits

2015-04-22 Thread Szehon Ho
I spent about few days trying to merge spark branch, including this change, and was waiting the 24 hours but now the svn repo is locked. Is the svn repo read-only for good now? Szehon On Wed, Apr 22, 2015 at 11:06 AM, Jimmy Xiang jxi...@cloudera.com wrote: The svn tree is locked due to

[jira] [Created] (HIVE-10444) HIVE-10223 breaks hadoop-1 build

2015-04-22 Thread Prasanth Jayachandran (JIRA)
Prasanth Jayachandran created HIVE-10444: Summary: HIVE-10223 breaks hadoop-1 build Key: HIVE-10444 URL: https://issues.apache.org/jira/browse/HIVE-10444 Project: Hive Issue Type:

Re: Review Request 33004: HIVE-10275 GenericUDF getTimestampValue should return Timestamp instead of Date

2015-04-22 Thread Alexander Pivovarov
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33004/#review81192 --- Ship it! Ship It! - Alexander Pivovarov On April 9, 2015, 4:32

Re: Preparation for Hive-1.2 release

2015-04-22 Thread Szehon Ho
Yes, I don't think it is fair to fork it this week, the change in question HIVE-10347 was reviewed and waiting 24 hours, but then svn repo is suddenly locked, and now the merge process has to be done again in git? I would vote to push it back one week to give everyone a chance to figure this out,

[jira] [Created] (HIVE-10441) Fix confusing log statement in SessionState about hive.execution.engine setting

2015-04-22 Thread Jason Dere (JIRA)
Jason Dere created HIVE-10441: - Summary: Fix confusing log statement in SessionState about hive.execution.engine setting Key: HIVE-10441 URL: https://issues.apache.org/jira/browse/HIVE-10441 Project:

[jira] [Created] (HIVE-10442) HIVE-10098 broke hadoop-1 build

2015-04-22 Thread Prasanth Jayachandran (JIRA)
Prasanth Jayachandran created HIVE-10442: Summary: HIVE-10098 broke hadoop-1 build Key: HIVE-10442 URL: https://issues.apache.org/jira/browse/HIVE-10442 Project: Hive Issue Type: Bug

Re: svn commits

2015-04-22 Thread Jimmy Xiang
I think it is better to not use the svn any more. Otherwise, we may need to find those changes in svn but not in git and apply them to git again, most likely, manually. On Wed, Apr 22, 2015 at 11:27 AM, Szehon Ho sze...@cloudera.com wrote: I spent about few days trying to merge spark branch,

Re: svn commits

2015-04-22 Thread Prasanth Jayachandran
Yes. SVN is made read only yesterday for git migration. Thanks Prasanth On Wed, Apr 22, 2015 at 10:24 AM -0700, Ashutosh Chauhan hashut...@apache.orgmailto:hashut...@apache.org wrote: are they already turned off because of git transition? I tried a commit on svn trunk today morning and got

Re: svn commits

2015-04-22 Thread Jimmy Xiang
The svn tree is locked due to migrating to git. See INFRA-9497 for more info. You can follow INFRA-9488 for the progress. On Wed, Apr 22, 2015 at 10:23 AM, Ashutosh Chauhan hashut...@apache.org wrote: are they already turned off because of git transition? I tried a commit on svn trunk today

Re: Preparation for Hive-1.2 release

2015-04-22 Thread Xuefu Zhang
Since the svn repo is locked for git migration and a queue of patches are waiting to be committed, I'm not sure if forking on this Friday is realistic. HIVE-9726 is part of the merge (HIVE-10347) which is also in the queue. Thanks, Xuefu On Wed, Apr 22, 2015 at 9:55 AM, Sushanth Sowmyan

[jira] [Created] (HIVE-10443) HIVE-9870 broke hadoop-1 build

2015-04-22 Thread Prasanth Jayachandran (JIRA)
Prasanth Jayachandran created HIVE-10443: Summary: HIVE-9870 broke hadoop-1 build Key: HIVE-10443 URL: https://issues.apache.org/jira/browse/HIVE-10443 Project: Hive Issue Type: Bug

Re: svn commits

2015-04-22 Thread Xuefu Zhang
I think it would be good if we can update the wiki on how to commit as we switch the tool. On Wed, Apr 22, 2015 at 3:54 PM, Prasanth Jayachandran pjayachand...@hortonworks.com wrote: I just tried committing a path to git master and it works. Try adding this URL to new remote for committing

[jira] [Created] (HIVE-10453) HS2 leaking open file descriptors when using UDFs

2015-04-22 Thread Yongzhi Chen (JIRA)
Yongzhi Chen created HIVE-10453: --- Summary: HS2 leaking open file descriptors when using UDFs Key: HIVE-10453 URL: https://issues.apache.org/jira/browse/HIVE-10453 Project: Hive Issue Type: Bug

[jira] [Created] (HIVE-10455) CBO (Calcite Return Path): Different data types at Reducer before JoinOp

2015-04-22 Thread Pengcheng Xiong (JIRA)
Pengcheng Xiong created HIVE-10455: -- Summary: CBO (Calcite Return Path): Different data types at Reducer before JoinOp Key: HIVE-10455 URL: https://issues.apache.org/jira/browse/HIVE-10455 Project:

Re: hive contributor meetup in bay area

2015-04-22 Thread Lefty Leverenz
Well now I've got audio and slides for most of the talks, so I'll get by. -- Lefty On Wed, Apr 22, 2015 at 9:45 PM, Prasanth Jayachandran pjayachand...@hortonworks.com wrote: Not yet. I will let Thejas know about it. Thanks Prasanth On Wed, Apr 22, 2015 at 6:37 PM -0700, Lefty

[jira] [Created] (HIVE-10458) Enable parallel order by for spark [Spark Branch]

2015-04-22 Thread Rui Li (JIRA)
Rui Li created HIVE-10458: - Summary: Enable parallel order by for spark [Spark Branch] Key: HIVE-10458 URL: https://issues.apache.org/jira/browse/HIVE-10458 Project: Hive Issue Type: Sub-task

[jira] [Created] (HIVE-10454) Query against partitioned table in strict mode failed with No partition predicate found even if partition predicate is specified.

2015-04-22 Thread Aihua Xu (JIRA)
Aihua Xu created HIVE-10454: --- Summary: Query against partitioned table in strict mode failed with No partition predicate found even if partition predicate is specified. Key: HIVE-10454 URL:

[jira] [Created] (HIVE-10452) Followup fix for HIVE-10202 to restrict it it for script mode.

2015-04-22 Thread Naveen Gangam (JIRA)
Naveen Gangam created HIVE-10452: Summary: Followup fix for HIVE-10202 to restrict it it for script mode. Key: HIVE-10452 URL: https://issues.apache.org/jira/browse/HIVE-10452 Project: Hive

[jira] [Created] (HIVE-10456) Grace Hash Join should not load spilled partitions on abort

2015-04-22 Thread Prasanth Jayachandran (JIRA)
Prasanth Jayachandran created HIVE-10456: Summary: Grace Hash Join should not load spilled partitions on abort Key: HIVE-10456 URL: https://issues.apache.org/jira/browse/HIVE-10456 Project:

Re: hive contributor meetup in bay area

2015-04-22 Thread Prasanth Jayachandran
Not yet. I will let Thejas know about it. Thanks Prasanth On Wed, Apr 22, 2015 at 6:37 PM -0700, Lefty Leverenz leftylever...@gmail.commailto:leftylever...@gmail.com wrote: I'm on hold for the audio. Have introductions started yet? -- Lefty On Wed, Apr 22, 2015 at 9:35 PM, Lefty Leverenz

Re: hive contributor meetup in bay area

2015-04-22 Thread Lefty Leverenz
I'm on hold for the audio. Have introductions started yet? -- Lefty On Wed, Apr 22, 2015 at 9:35 PM, Lefty Leverenz leftylever...@gmail.com wrote: Webex won't let me log in -- the login page seems to expect a Hortonworks employee, and the forgot-my-password page rejected both of my email

Re: hive contributor meetup in bay area

2015-04-22 Thread Lefty Leverenz
Webex won't let me log in -- the login page seems to expect a Hortonworks employee, and the forgot-my-password page rejected both of my email addresses. In 2013 we joined with a meeting password, but I don't see anywhere to enter it this time (and besides, I don't know what it is). -- Lefty On

[jira] [Created] (HIVE-10459) Add materialized views to Hive

2015-04-22 Thread Alan Gates (JIRA)
Alan Gates created HIVE-10459: - Summary: Add materialized views to Hive Key: HIVE-10459 URL: https://issues.apache.org/jira/browse/HIVE-10459 Project: Hive Issue Type: Improvement

[jira] [Created] (HIVE-10451) IdentityProjectRemover removed useful project

2015-04-22 Thread Ashutosh Chauhan (JIRA)
Ashutosh Chauhan created HIVE-10451: --- Summary: IdentityProjectRemover removed useful project Key: HIVE-10451 URL: https://issues.apache.org/jira/browse/HIVE-10451 Project: Hive Issue Type:

Re: hive contributor meetup in bay area

2015-04-22 Thread Lefty Leverenz
Woohoo! WebEx is starting, no login required. (I was trying to join too early before.) -- Lefty On Wed, Apr 22, 2015 at 9:54 PM, Lefty Leverenz leftylever...@gmail.com wrote: Well now I've got audio and slides for most of the talks, so I'll get by. -- Lefty On Wed, Apr 22, 2015 at 9:45

[jira] [Created] (HIVE-10457) Merge trunk to spark (4/22/15)

2015-04-22 Thread Szehon Ho (JIRA)
Szehon Ho created HIVE-10457: Summary: Merge trunk to spark (4/22/15) Key: HIVE-10457 URL: https://issues.apache.org/jira/browse/HIVE-10457 Project: Hive Issue Type: Bug Reporter:

Re: svn commits

2015-04-22 Thread Sergey Shelukhin
The git repo should be ready to use soon. On 15/4/22, 10:23, Ashutosh Chauhan hashut...@apache.org wrote: are they already turned off because of git transition? I tried a commit on svn trunk today morning and got access denied exception. Any one else getting this exception? If it indeed is

[jira] [Created] (HIVE-10448) LLAP: Consider replacing BytesBytesMultiHashMap with new fast hash table code of Native Vector Map Join

2015-04-22 Thread Matt McCline (JIRA)
Matt McCline created HIVE-10448: --- Summary: LLAP: Consider replacing BytesBytesMultiHashMap with new fast hash table code of Native Vector Map Join Key: HIVE-10448 URL:

Re: svn commits

2015-04-22 Thread Prasanth Jayachandran
I just tried committing a path to git master and it works. Try adding this URL to new remote for committing https://git-wip-us.apache.org/repos/asf/hive.git On Apr 22, 2015, at 2:46 PM, Sergey Shelukhin ser...@hortonworks.com wrote: Git repo should now be writeable. Thanks everyone, sorry

[jira] [Created] (HIVE-10450) More than one TableScan in MapWork not supported in Vectorization -- causes query to fail during vectorization

2015-04-22 Thread Matt McCline (JIRA)
Matt McCline created HIVE-10450: --- Summary: More than one TableScan in MapWork not supported in Vectorization -- causes query to fail during vectorization Key: HIVE-10450 URL:

Re: Preparation for Hive-1.2 release

2015-04-22 Thread Szehon Ho
Thanks Sushanth. Yes, blind applying the patch would work, but I believe it is a little more involved to keep the history (git merge, rebase, resolve, test, etc). I am doing the merge process again this time in git and regenerated what is hopefully an equivalent patch, hopefully I can get it

[jira] [Created] (HIVE-10445) Report error when dynamic partition insert is not following the correct syntax

2015-04-22 Thread Chao Sun (JIRA)
Chao Sun created HIVE-10445: --- Summary: Report error when dynamic partition insert is not following the correct syntax Key: HIVE-10445 URL: https://issues.apache.org/jira/browse/HIVE-10445 Project: Hive

[jira] [Created] (HIVE-10449) LLAP: Make new fast hash table for Native Vector Map Join work with Hybrid Grace

2015-04-22 Thread Matt McCline (JIRA)
Matt McCline created HIVE-10449: --- Summary: LLAP: Make new fast hash table for Native Vector Map Join work with Hybrid Grace Key: HIVE-10449 URL: https://issues.apache.org/jira/browse/HIVE-10449

Re: Preparation for Hive-1.2 release

2015-04-22 Thread Thejas Nair
The Apache Accumulo project has written a nice doc on git use reccomendations for their project based on experiences in Accumulo and a similar Kafka document - http://accumulo.apache.org/git.html. It describes the options for committing feature branches as well. I propose using that as the current

[jira] [Created] (HIVE-10447) Beeline JDBC Driver to support 2 way SSL

2015-04-22 Thread Hari Sankar Sivarama Subramaniyan (JIRA)
Hari Sankar Sivarama Subramaniyan created HIVE-10447: Summary: Beeline JDBC Driver to support 2 way SSL Key: HIVE-10447 URL: https://issues.apache.org/jira/browse/HIVE-10447

Re: svn commits

2015-04-22 Thread Sergey Shelukhin
Git repo should now be writeable. Thanks everyone, sorry for the disturbance. On 15/4/22, 13:39, Sergey Shelukhin ser...@hortonworks.com wrote: The git repo should be ready to use soon. On 15/4/22, 10:23, Ashutosh Chauhan hashut...@apache.org wrote: are they already turned off because of git

[jira] [Created] (HIVE-10446) Hybrid Hybrid Grace Hash Join : java.lang.IllegalArgumentException in Kryo while spilling big table

2015-04-22 Thread Mostafa Mokhtar (JIRA)
Mostafa Mokhtar created HIVE-10446: -- Summary: Hybrid Hybrid Grace Hash Join : java.lang.IllegalArgumentException in Kryo while spilling big table Key: HIVE-10446 URL:

Re: Preparation for Hive-1.2 release

2015-04-22 Thread Sushanth Sowmyan
Hi Szehon, Sergey tells me that he's verifying the repos, and expects to unblock it in another 20 minutes or so. I think that if HIVE-10347 is ready for commit now to svn, it should be committable to git as well, as-is as soon as that comes back up. In that scenario, there's no extra need to

[jira] [Created] (HIVE-10436) DEBUG lockmgr.DummyTxnManager: Adding database :mkt_incoming to list of lock outputs

2015-04-22 Thread ankush (JIRA)
ankush created HIVE-10436: - Summary: DEBUG lockmgr.DummyTxnManager: Adding database :mkt_incoming to list of lock outputs Key: HIVE-10436 URL: https://issues.apache.org/jira/browse/HIVE-10436 Project: Hive

[jira] [Created] (HIVE-10435) Make HiveSession implementation pluggable through configuration

2015-04-22 Thread Amareshwari Sriramadasu (JIRA)
Amareshwari Sriramadasu created HIVE-10435: -- Summary: Make HiveSession implementation pluggable through configuration Key: HIVE-10435 URL: https://issues.apache.org/jira/browse/HIVE-10435

hive on tez optimize MRR to MR?

2015-04-22 Thread r7raul1...@163.com
select userid,count(*) from u_data group by userid order by useridwill product MRR. I think when the result of userid,count(*) is small(one reduce can process the result) . This query plan can optimize to MR ? r7raul1...@163.com

Re: hive on tez optimize MRR to MR?

2015-04-22 Thread Gopal Vijayaraghavan
To prevent bad reducer merging, the reducer merging only kicks in when the optimizer thinks it gets a perf boost. MR - MRR is not a big win when it comes Tez, due to container-reuse - going wide on the large cardinality in case of missing map-side aggregation will be safer. If