[jira] [Commented] (HIVE-6735) Make scalable dynamic partitioning work in vectorized mode

2014-03-27 Thread Harish Butani (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-6735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13950250#comment-13950250
 ] 

Harish Butani commented on HIVE-6735:
-

+1 for 0.13

> Make scalable dynamic partitioning work in vectorized mode
> --
>
> Key: HIVE-6735
> URL: https://issues.apache.org/jira/browse/HIVE-6735
> Project: Hive
>  Issue Type: Sub-task
>  Components: Query Processor
>Affects Versions: 0.13.0, 0.14.0
>Reporter: Prasanth J
>Assignee: Prasanth J
> Fix For: 0.13.0, 0.14.0
>
> Attachments: HIVE-6735.1.patch, HIVE-6735.2.patch, HIVE-6735.2.patch, 
> HIVE-6735.3.patch
>
>
> HIVE-6455 added support for scalable dynamic partitioning. This is subtask to 
> make HIVE-6455 work with vectorized operators.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-6735) Make scalable dynamic partitioning work in vectorized mode

2014-03-27 Thread Prasanth J (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-6735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13949851#comment-13949851
 ] 

Prasanth J commented on HIVE-6735:
--

The test failures are unrelated. They pass locally on my system.

> Make scalable dynamic partitioning work in vectorized mode
> --
>
> Key: HIVE-6735
> URL: https://issues.apache.org/jira/browse/HIVE-6735
> Project: Hive
>  Issue Type: Sub-task
>  Components: Query Processor
>Affects Versions: 0.13.0, 0.14.0
>Reporter: Prasanth J
>Assignee: Prasanth J
> Fix For: 0.13.0, 0.14.0
>
> Attachments: HIVE-6735.1.patch, HIVE-6735.2.patch, HIVE-6735.2.patch, 
> HIVE-6735.3.patch
>
>
> HIVE-6455 added support for scalable dynamic partitioning. This is subtask to 
> make HIVE-6455 work with vectorized operators.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-6735) Make scalable dynamic partitioning work in vectorized mode

2014-03-27 Thread Hive QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-6735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13949836#comment-13949836
 ] 

Hive QA commented on HIVE-6735:
---



{color:red}Overall{color}: -1 at least one tests failed

Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12637049/HIVE-6735.3.patch

{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 5492 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table
org.apache.hadoop.hive.metastore.TestRetryingHMSHandler.testRetryingHMSHandler
{noformat}

Test results: 
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1986/testReport
Console output: 
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1986/console

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12637049

> Make scalable dynamic partitioning work in vectorized mode
> --
>
> Key: HIVE-6735
> URL: https://issues.apache.org/jira/browse/HIVE-6735
> Project: Hive
>  Issue Type: Sub-task
>  Components: Query Processor
>Affects Versions: 0.13.0, 0.14.0
>Reporter: Prasanth J
>Assignee: Prasanth J
> Fix For: 0.13.0, 0.14.0
>
> Attachments: HIVE-6735.1.patch, HIVE-6735.2.patch, HIVE-6735.2.patch, 
> HIVE-6735.3.patch
>
>
> HIVE-6455 added support for scalable dynamic partitioning. This is subtask to 
> make HIVE-6455 work with vectorized operators.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-6735) Make scalable dynamic partitioning work in vectorized mode

2014-03-26 Thread Prasanth J (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-6735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13948647#comment-13948647
 ] 

Prasanth J commented on HIVE-6735:
--

[~gopalv] the reason for the lesser hashcode generation is mentioned in 
HIVE-6761. I will fix this patch to include all partition columns.

> Make scalable dynamic partitioning work in vectorized mode
> --
>
> Key: HIVE-6735
> URL: https://issues.apache.org/jira/browse/HIVE-6735
> Project: Hive
>  Issue Type: Sub-task
>  Components: Query Processor
>Affects Versions: 0.13.0, 0.14.0
>Reporter: Prasanth J
>Assignee: Prasanth J
> Fix For: 0.13.0, 0.14.0
>
> Attachments: HIVE-6735.1.patch, HIVE-6735.2.patch, HIVE-6735.2.patch
>
>
> HIVE-6455 added support for scalable dynamic partitioning. This is subtask to 
> make HIVE-6455 work with vectorized operators.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-6735) Make scalable dynamic partitioning work in vectorized mode

2014-03-26 Thread Gopal V (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-6735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13948046#comment-13948046
 ] 

Gopal V commented on HIVE-6735:
---

The updated .2 patch runs in 859 seconds, where the previous one ran in ~350 
seconds.

The code needs to loop through all partition columns when generating hashCode - 
I got 13 unique hashCodes for 78 different keys, with .1 patch I was getting 78 
different hashCodes.

> Make scalable dynamic partitioning work in vectorized mode
> --
>
> Key: HIVE-6735
> URL: https://issues.apache.org/jira/browse/HIVE-6735
> Project: Hive
>  Issue Type: Sub-task
>  Components: Query Processor
>Affects Versions: 0.13.0, 0.14.0
>Reporter: Prasanth J
>Assignee: Prasanth J
> Fix For: 0.13.0, 0.14.0
>
> Attachments: HIVE-6735.1.patch, HIVE-6735.2.patch, HIVE-6735.2.patch
>
>
> HIVE-6455 added support for scalable dynamic partitioning. This is subtask to 
> make HIVE-6455 work with vectorized operators.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-6735) Make scalable dynamic partitioning work in vectorized mode

2014-03-25 Thread Harish Butani (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-6735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13947356#comment-13947356
 ] 

Harish Butani commented on HIVE-6735:
-

+1 for 0.13

> Make scalable dynamic partitioning work in vectorized mode
> --
>
> Key: HIVE-6735
> URL: https://issues.apache.org/jira/browse/HIVE-6735
> Project: Hive
>  Issue Type: Sub-task
>  Components: Query Processor
>Affects Versions: 0.13.0, 0.14.0
>Reporter: Prasanth J
>Assignee: Prasanth J
> Fix For: 0.13.0, 0.14.0
>
> Attachments: HIVE-6735.1.patch, HIVE-6735.2.patch, HIVE-6735.2.patch
>
>
> HIVE-6455 added support for scalable dynamic partitioning. This is subtask to 
> make HIVE-6455 work with vectorized operators.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-6735) Make scalable dynamic partitioning work in vectorized mode

2014-03-25 Thread Prasanth J (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-6735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13947334#comment-13947334
 ] 

Prasanth J commented on HIVE-6735:
--

I ran bucket_num_reducers.q locally and it ran successfully. Also that test is 
not related as it doesn't use vectorization or dynamic partitioning. The 
failures are unrelated.

> Make scalable dynamic partitioning work in vectorized mode
> --
>
> Key: HIVE-6735
> URL: https://issues.apache.org/jira/browse/HIVE-6735
> Project: Hive
>  Issue Type: Sub-task
>  Components: Query Processor
>Affects Versions: 0.13.0, 0.14.0
>Reporter: Prasanth J
>Assignee: Prasanth J
> Fix For: 0.13.0, 0.14.0
>
> Attachments: HIVE-6735.1.patch, HIVE-6735.2.patch, HIVE-6735.2.patch
>
>
> HIVE-6455 added support for scalable dynamic partitioning. This is subtask to 
> make HIVE-6455 work with vectorized operators.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-6735) Make scalable dynamic partitioning work in vectorized mode

2014-03-25 Thread Hive QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-6735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13947329#comment-13947329
 ] 

Hive QA commented on HIVE-6735:
---



{color:red}Overall{color}: -1 at least one tests failed

Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12636757/HIVE-6735.2.patch

{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 5448 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucket_num_reducers
org.apache.hadoop.hive.metastore.TestRetryingHMSHandler.testRetryingHMSHandler
{noformat}

Test results: 
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1958/testReport
Console output: 
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1958/console

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12636757

> Make scalable dynamic partitioning work in vectorized mode
> --
>
> Key: HIVE-6735
> URL: https://issues.apache.org/jira/browse/HIVE-6735
> Project: Hive
>  Issue Type: Sub-task
>  Components: Query Processor
>Affects Versions: 0.13.0, 0.14.0
>Reporter: Prasanth J
>Assignee: Prasanth J
> Fix For: 0.13.0, 0.14.0
>
> Attachments: HIVE-6735.1.patch, HIVE-6735.2.patch, HIVE-6735.2.patch
>
>
> HIVE-6455 added support for scalable dynamic partitioning. This is subtask to 
> make HIVE-6455 work with vectorized operators.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-6735) Make scalable dynamic partitioning work in vectorized mode

2014-03-25 Thread Prasanth J (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-6735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13947025#comment-13947025
 ] 

Prasanth J commented on HIVE-6735:
--

[~rhbutani] this is a sub-task of feature HIVE-6455. Can we have this in 0.13 
as well?

> Make scalable dynamic partitioning work in vectorized mode
> --
>
> Key: HIVE-6735
> URL: https://issues.apache.org/jira/browse/HIVE-6735
> Project: Hive
>  Issue Type: Sub-task
>  Components: Query Processor
>Affects Versions: 0.13.0, 0.14.0
>Reporter: Prasanth J
>Assignee: Prasanth J
> Fix For: 0.13.0, 0.14.0
>
> Attachments: HIVE-6735.1.patch, HIVE-6735.2.patch, HIVE-6735.2.patch
>
>
> HIVE-6455 added support for scalable dynamic partitioning. This is subtask to 
> make HIVE-6455 work with vectorized operators.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-6735) Make scalable dynamic partitioning work in vectorized mode

2014-03-24 Thread Jitendra Nath Pandey (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-6735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13946085#comment-13946085
 ] 

Jitendra Nath Pandey commented on HIVE-6735:


+1 LGTM

> Make scalable dynamic partitioning work in vectorized mode
> --
>
> Key: HIVE-6735
> URL: https://issues.apache.org/jira/browse/HIVE-6735
> Project: Hive
>  Issue Type: Sub-task
>  Components: Query Processor
>Affects Versions: 0.13.0, 0.14.0
>Reporter: Prasanth J
>Assignee: Prasanth J
> Fix For: 0.13.0, 0.14.0
>
> Attachments: HIVE-6735.1.patch
>
>
> HIVE-6455 added support for scalable dynamic partitioning. This is subtask to 
> make HIVE-6455 work with vectorized operators.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (HIVE-6735) Make scalable dynamic partitioning work in vectorized mode

2014-03-24 Thread Gopal V (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-6735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13946061#comment-13946061
 ] 

Gopal V commented on HIVE-6735:
---

Testing this on a scale=200 data-load.

> Make scalable dynamic partitioning work in vectorized mode
> --
>
> Key: HIVE-6735
> URL: https://issues.apache.org/jira/browse/HIVE-6735
> Project: Hive
>  Issue Type: Sub-task
>  Components: Query Processor
>Affects Versions: 0.13.0, 0.14.0
>Reporter: Prasanth J
>Assignee: Prasanth J
> Fix For: 0.13.0, 0.14.0
>
> Attachments: HIVE-6735.1.patch
>
>
> HIVE-6455 added support for scalable dynamic partitioning. This is subtask to 
> make HIVE-6455 work with vectorized operators.



--
This message was sent by Atlassian JIRA
(v6.2#6252)