Top-K optimization

Hi All, I'm a developer at Qubole (http://www.qubole.com) looking at Hadoop and Hive. In my past life, I was on the optimizer team of Greenplum Parallel Database. I'm a newbie to the Hive mailing list, so apologies for any missteps. I've done some searching in the Hive mailing list and JIRA

Re: Top-K optimization

Hi Siva, Take a look at https://issues.apache.org/jira/browse/HIVE-3562. It is in my todo list, but I have not been able to review this. I think, this addresses a very similar problem. If yes, can you also review the above patch ? Thanks, -namit On 11/19/12 3:10 PM, Sivaramakrishnan

[jira] [Commented] (HIVE-3562) Some limit can be pushed down to map stage

[ https://issues.apache.org/jira/browse/HIVE-3562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500118#comment-13500118 ] Sivaramakrishnan Narayanan commented on HIVE-3562: -- I'm interested in this

[jira] [Updated] (HIVE-3633) sort-merge join does not work with sub-queries

[ https://issues.apache.org/jira/browse/HIVE-3633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Namit Jain updated HIVE-3633: - Attachment: hive.3633.5.patch sort-merge join does not work with sub-queries

[jira] [Commented] (HIVE-3562) Some limit can be pushed down to map stage

[ https://issues.apache.org/jira/browse/HIVE-3562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500165#comment-13500165 ] Sivaramakrishnan Narayanan commented on HIVE-3562: -- Apologies, you can use

Build failed in Jenkins: Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false #203

See https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/203/ -- [...truncated 10343 lines...] compile-test: [echo] Project: serde [javac] Compiling 26 source files to

[jira] [Commented] (HIVE-3705) Adding authorization capability to the metastore

[ https://issues.apache.org/jira/browse/HIVE-3705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500375#comment-13500375 ] Rob Weltman commented on HIVE-3705: --- A new JIRA has been opened for the larger issues

[jira] [Assigned] (HIVE-3718) Add check to determine whether partition can be dropped at Semantic Analysis time

[ https://issues.apache.org/jira/browse/HIVE-3718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pamela Vagata reassigned HIVE-3718: --- Assignee: Pamela Vagata Add check to determine whether partition can be dropped at

[jira] [Commented] (HIVE-2206) add a new optimizer for query correlation discovery and optimization

[ https://issues.apache.org/jira/browse/HIVE-2206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500469#comment-13500469 ] Carl Steinbach commented on HIVE-2206: -- @Yin: The correlation optimizer is only

[jira] [Updated] (HIVE-3718) Add check to determine whether partition can be dropped at Semantic Analysis time

[ https://issues.apache.org/jira/browse/HIVE-3718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pamela Vagata updated HIVE-3718: Attachment: (was: HIVE-3718.1.patch.txt) Add check to determine whether partition can be

[jira] [Updated] (HIVE-3718) Add check to determine whether partition can be dropped at Semantic Analysis time

[ https://issues.apache.org/jira/browse/HIVE-3718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pamela Vagata updated HIVE-3718: Attachment: HIVE-3718.1.patch.txt Add check to determine whether partition can be dropped at

[jira] [Updated] (HIVE-3718) Add check to determine whether partition can be dropped at Semantic Analysis time

[ https://issues.apache.org/jira/browse/HIVE-3718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pamela Vagata updated HIVE-3718: Status: Patch Available (was: Open) Add check to determine whether partition can be dropped

[jira] [Commented] (HIVE-2206) add a new optimizer for query correlation discovery and optimization

[ https://issues.apache.org/jira/browse/HIVE-2206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500474#comment-13500474 ] David Inbar commented on HIVE-2206: --- I will be on vacation through Friday Nov 23rd, but

[jira] [Commented] (HIVE-3718) Add check to determine whether partition can be dropped at Semantic Analysis time

[ https://issues.apache.org/jira/browse/HIVE-3718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500493#comment-13500493 ] Kevin Wilfong commented on HIVE-3718: - +1 Add check to determine

[jira] [Updated] (HIVE-3647) map-side groupby wrongly due to HIVE-3432

[ https://issues.apache.org/jira/browse/HIVE-3647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Wilfong updated HIVE-3647: Resolution: Fixed Status: Resolved (was: Patch Available) Committed, thanks Namit.

[jira] [Commented] (HIVE-3678) Add metastore upgrade scripts for column stats schema changes

[ https://issues.apache.org/jira/browse/HIVE-3678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500500#comment-13500500 ] Carl Steinbach commented on HIVE-3678: -- The upgrade scripts look good to me. As for

[jira] [Commented] (HIVE-2206) add a new optimizer for query correlation discovery and optimization

[ https://issues.apache.org/jira/browse/HIVE-2206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500499#comment-13500499 ] Yin Huai commented on HIVE-2206: [~cwsteinbach] If the optimizer is enabled by default,

[jira] [Updated] (HIVE-3718) Add check to determine whether partition can be dropped at Semantic Analysis time

[ https://issues.apache.org/jira/browse/HIVE-3718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pamela Vagata updated HIVE-3718: Attachment: (was: HIVE-3718.1.patch.txt) Add check to determine whether partition can be

[jira] [Updated] (HIVE-3719) Improve HiveServer to support username/password authentication

[ https://issues.apache.org/jira/browse/HIVE-3719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-3719: --- Assignee: Yu Gao Improve HiveServer to support username/password authentication

[jira] [Commented] (HIVE-3678) Add metastore upgrade scripts for column stats schema changes

[ https://issues.apache.org/jira/browse/HIVE-3678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500526#comment-13500526 ] Shreepadma Venugopalan commented on HIVE-3678: -- With the changes from

[jira] [Updated] (HIVE-3709) Stop storing default ConfVars in temp file

[ https://issues.apache.org/jira/browse/HIVE-3709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-3709: - Status: Open (was: Patch Available) @Kevin: I still see errors in TestHiveServerSessions when I

[jira] [Commented] (HIVE-3678) Add metastore upgrade scripts for column stats schema changes

[ https://issues.apache.org/jira/browse/HIVE-3678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500570#comment-13500570 ] Carl Steinbach commented on HIVE-3678: -- Sorry for the confusion. When I wrote blob I

Re: Review Request: HIVE-2206: add a new optimizer for query correlation discovery and optimization

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/7126/ --- (Updated Nov. 19, 2012, 7:51 p.m.) Review request for hive. Changes ---

[jira] [Updated] (HIVE-3648) HiveMetaStoreFsImpl is not compatible with hadoop viewfs

[ https://issues.apache.org/jira/browse/HIVE-3648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arup Malakar updated HIVE-3648: --- Attachment: HIVE_3648_branch_0.patch HIVE_3648_trunk_1.patch Patch available for

[jira] [Created] (HIVE-3721) ALTER TABLE ADD PARTS should check for valid partition spec and throw a SemanticException if part spec is not valid

Pamela Vagata created HIVE-3721: --- Summary: ALTER TABLE ADD PARTS should check for valid partition spec and throw a SemanticException if part spec is not valid Key: HIVE-3721 URL:

[jira] [Updated] (HIVE-2206) add a new optimizer for query correlation discovery and optimization

[ https://issues.apache.org/jira/browse/HIVE-2206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated HIVE-2206: --- Attachment: HIVE-2206.19-r1410581.patch.txt I just integrate HIVE-3671 into this patch. At the beginning of

[jira] [Commented] (HIVE-3678) Add metastore upgrade scripts for column stats schema changes

[ https://issues.apache.org/jira/browse/HIVE-3678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500599#comment-13500599 ] Ashutosh Chauhan commented on HIVE-3678: I agree with Carl, making it easier to

[jira] [Commented] (HIVE-2206) add a new optimizer for query correlation discovery and optimization

[ https://issues.apache.org/jira/browse/HIVE-2206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500626#comment-13500626 ] Carl Steinbach commented on HIVE-2206: -- I'm surprised that auto_join26 is the only

Build failed in Jenkins: Hive-0.9.1-SNAPSHOT-h0.21 #203

See https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/203/ -- [...truncated 36981 lines...] [junit] POSTHOOK: Input: default@testhivedrivertable [junit] POSTHOOK: Output:

Re: hive 0.10 release

Another quick update. I have created a hive-0.10 branch. At this point, HIVE-3678 is a blocker to do a 0.10 release. There are few others nice to have which were there in my previous email. I will be happy to merge new patches between now and RC if folks request for it and are low risk. Thanks,

Re: hive 0.10 release

There are couple of enhancements that I have been working on mainly related to the hive/hbase integration. It would be awesome if it is possible at all to include them in this release. None of them should really be high risk. I have patches submitted for few of them. Will try to get for others

Hive-trunk-h0.21 - Build # 1805 - Still Failing

Changes for Build #1764 [kevinwilfong] HIVE-3610. Add a command Explain dependency ... (Sambavi Muthukrishnan via kevinwilfong) Changes for Build #1765 Changes for Build #1766 [hashutosh] HIVE-3441 : testcases escape1,escape2 fail on windows (Thejas Nair via Ashutosh Chauhan) [kevinwilfong]

[jira] [Commented] (HIVE-3722) Create index fails on CLI using remote metastore

[ https://issues.apache.org/jira/browse/HIVE-3722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500828#comment-13500828 ] Namit Jain commented on HIVE-3722: -- +1 Create index fails on CLI using

[jira] [Commented] (HIVE-3722) Create index fails on CLI using remote metastore

[ https://issues.apache.org/jira/browse/HIVE-3722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500834#comment-13500834 ] Ashutosh Chauhan commented on HIVE-3722: Kevin, I am not sure if you have looked at

[jira] [Commented] (HIVE-3722) Create index fails on CLI using remote metastore

[ https://issues.apache.org/jira/browse/HIVE-3722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500847#comment-13500847 ] Kevin Wilfong commented on HIVE-3722: - Ashutosh, I missed that JIRA. But based on

[jira] [Commented] (HIVE-3589) describe/show partition/show tblproperties command should accept database name

[ https://issues.apache.org/jira/browse/HIVE-3589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500857#comment-13500857 ] Phabricator commented on HIVE-3589: --- navis has commented on the revision HIVE-3589 [jira]

[jira] [Updated] (HIVE-3635) allow 't', 'T', '1', 'f', 'F', and '0' to be allowable true/false values for the boolean hive type

[ https://issues.apache.org/jira/browse/HIVE-3635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Alten-Lorenz updated HIVE-3635: - Attachment: (was: HIVE-3635.patch) allow 't', 'T', '1', 'f', 'F', and

[jira] [Updated] (HIVE-3635) allow 't', 'T', '1', 'f', 'F', and '0' to be allowable true/false values for the boolean hive type

[ https://issues.apache.org/jira/browse/HIVE-3635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Alten-Lorenz updated HIVE-3635: - Status: Patch Available (was: Open) allow 't', 'T', '1', 'f', 'F', and '0'

[jira] [Updated] (HIVE-3635) allow 't', 'T', '1', 'f', 'F', and '0' to be allowable true/false values for the boolean hive type

[ https://issues.apache.org/jira/browse/HIVE-3635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Alten-Lorenz updated HIVE-3635: - Attachment: HIVE-3635.patch allow 't', 'T', '1', 'f', 'F', and '0' to be

[jira] [Commented] (HIVE-3635) allow 't', 'T', '1', 'f', 'F', and '0' to be allowable true/false values for the boolean hive type

[ https://issues.apache.org/jira/browse/HIVE-3635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500878#comment-13500878 ] Alexander Alten-Lorenz commented on HIVE-3635: -- Replaced available patch here

Re: Review Request: allow 't', 'T', '1', 'f', 'F', and '0' to be allowable true/false values for the boolean hive type

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/7759/ --- (Updated Nov. 20, 2012, 7:11 a.m.) Review request for hive. Changes ---

[jira] [Updated] (HIVE-3073) Hive List Bucketing - DML support

[ https://issues.apache.org/jira/browse/HIVE-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gang Tim Liu updated HIVE-3073: --- Status: Patch Available (was: Open) Another patch. thanks Hive List Bucketing -

[jira] [Updated] (HIVE-3073) Hive List Bucketing - DML support

[ https://issues.apache.org/jira/browse/HIVE-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gang Tim Liu updated HIVE-3073: --- Attachment: HIVE-3073.patch.15 Hive List Bucketing - DML support