[jira] [Updated] (HIVE-18254) Use proper AVG Calcite primitive instead of Other_FUNCTION

2017-12-09 Thread Ashutosh Chauhan (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-18254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan updated HIVE-18254:

   Resolution: Fixed
 Assignee: slim bouguerra  (was: Ashutosh Chauhan)
Fix Version/s: 3.0.0
   Status: Resolved  (was: Patch Available)

Pushed to master. Thanks, Slim!

> Use proper AVG Calcite primitive instead of Other_FUNCTION
> --
>
> Key: HIVE-18254
> URL: https://issues.apache.org/jira/browse/HIVE-18254
> Project: Hive
>  Issue Type: Bug
>Reporter: slim bouguerra
>Assignee: slim bouguerra
> Fix For: 3.0.0
>
> Attachments: HIVE-18254.patch
>
>
> Currently Hive-Calcite operator tree treats AVG function as an unknown 
> function that has a Calcite Sql Kind of Other_FUNCTION. This is an issue that 
> can get into the way of rules like 
> {{org.apache.calcite.rel.rules.AggregateReduceFunctionsRule}}.
> This patch adds the avg function to the list of known aggregate function.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (HIVE-18254) Use proper AVG Calcite primitive instead of Other_FUNCTION

2017-12-09 Thread Ashutosh Chauhan (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-18254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16285119#comment-16285119
 ] 

Ashutosh Chauhan commented on HIVE-18254:
-

+1

> Use proper AVG Calcite primitive instead of Other_FUNCTION
> --
>
> Key: HIVE-18254
> URL: https://issues.apache.org/jira/browse/HIVE-18254
> Project: Hive
>  Issue Type: Bug
>Reporter: slim bouguerra
>Assignee: Ashutosh Chauhan
> Attachments: HIVE-18254.patch
>
>
> Currently Hive-Calcite operator tree treats AVG function as an unknown 
> function that has a Calcite Sql Kind of Other_FUNCTION. This is an issue that 
> can get into the way of rules like 
> {{org.apache.calcite.rel.rules.AggregateReduceFunctionsRule}}.
> This patch adds the avg function to the list of known aggregate function.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (HIVE-13567) Enable auto-gather column stats by default

2017-12-09 Thread Ashutosh Chauhan (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-13567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16285115#comment-16285115
 ] 

Ashutosh Chauhan commented on HIVE-13567:
-

+1 pending tests

> Enable auto-gather column stats by default
> --
>
> Key: HIVE-13567
> URL: https://issues.apache.org/jira/browse/HIVE-13567
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Pengcheng Xiong
>Assignee: Zoltan Haindrich
> Attachments: HIVE-13567.01.patch, HIVE-13567.02.patch, 
> HIVE-13567.03.patch, HIVE-13567.04.patch, HIVE-13567.05.patch, 
> HIVE-13567.06.patch, HIVE-13567.07.patch, HIVE-13567.08.patch, 
> HIVE-13567.09.patch, HIVE-13567.10.patch, HIVE-13567.11.patch, 
> HIVE-13567.12.patch, HIVE-13567.13.patch, HIVE-13567.14.patch, 
> HIVE-13567.15.patch, HIVE-13567.16.patch, HIVE-13567.17.patch, 
> HIVE-13567.18.patch, HIVE-13567.19.patch, HIVE-13567.20.patch, 
> HIVE-13567.21.patch, HIVE-13567.22.patch, HIVE-13567.23.patch, 
> HIVE-13567.23wip01.patch, HIVE-13567.23wip02.patch, HIVE-13567.23wip03.patch, 
> HIVE-13567.23wip04.patch, HIVE-13567.23wip05.patch, HIVE-13567.23wip06.patch, 
> HIVE-13567.23wip07.patch, HIVE-13567.23wip08.patch, HIVE-13567.23wip09.patch, 
> HIVE-13567.23wip10.patch, HIVE-13567.24.patch
>
>
> in phase 2, we are going to set auto-gather column on as default. This needs 
> to update golden files.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (HIVE-18112) show create for view having special char in where clause is not showing properly

2017-12-09 Thread Naresh P R (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-18112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Naresh P R updated HIVE-18112:
--
Attachment: HIVE-18112-branch-2.2.patch

> show create for view having special char in where clause is not showing 
> properly
> 
>
> Key: HIVE-18112
> URL: https://issues.apache.org/jira/browse/HIVE-18112
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 1.2.0
>Reporter: Naresh P R
>Assignee: Naresh P R
>Priority: Minor
> Fix For: 2.2.0
>
> Attachments: HIVE-18112-branch-2.2.patch
>
>
> e.g., 
> CREATE VIEW `v2` AS select `evil_byte1`.`a` from `default`.`EVIL_BYTE1` where 
> `evil_byte1`.`a` = 'abcÖdefÖgh';
> Output:
> ==
> 0: jdbc:hive2://172.26.122.227:1> show create table v2;
> ++--+
> | createtab_stmt  
>|
> ++--+
> | CREATE VIEW `v2` AS select `evil_byte1`.`a` from `default`.`EVIL_BYTE1` 
> where `evil_byte1`.`a` = 'abc�def�gh'  |
> ++--+
> Only show create output is having invalid characters, actual source table 
> content is displayed properly in the console.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (HIVE-18112) show create for view having special char in where clause is not showing properly

2017-12-09 Thread Naresh P R (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-18112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Naresh P R updated HIVE-18112:
--
Attachment: (was: HIVE-18112-branch-2.2.patch)

> show create for view having special char in where clause is not showing 
> properly
> 
>
> Key: HIVE-18112
> URL: https://issues.apache.org/jira/browse/HIVE-18112
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 1.2.0
>Reporter: Naresh P R
>Assignee: Naresh P R
>Priority: Minor
> Fix For: 2.2.0
>
>
> e.g., 
> CREATE VIEW `v2` AS select `evil_byte1`.`a` from `default`.`EVIL_BYTE1` where 
> `evil_byte1`.`a` = 'abcÖdefÖgh';
> Output:
> ==
> 0: jdbc:hive2://172.26.122.227:1> show create table v2;
> ++--+
> | createtab_stmt  
>|
> ++--+
> | CREATE VIEW `v2` AS select `evil_byte1`.`a` from `default`.`EVIL_BYTE1` 
> where `evil_byte1`.`a` = 'abc�def�gh'  |
> ++--+
> Only show create output is having invalid characters, actual source table 
> content is displayed properly in the console.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (HIVE-13567) Enable auto-gather column stats by default

2017-12-09 Thread Ashutosh Chauhan (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-13567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan updated HIVE-13567:

Status: Patch Available  (was: Open)

> Enable auto-gather column stats by default
> --
>
> Key: HIVE-13567
> URL: https://issues.apache.org/jira/browse/HIVE-13567
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Pengcheng Xiong
>Assignee: Zoltan Haindrich
> Attachments: HIVE-13567.01.patch, HIVE-13567.02.patch, 
> HIVE-13567.03.patch, HIVE-13567.04.patch, HIVE-13567.05.patch, 
> HIVE-13567.06.patch, HIVE-13567.07.patch, HIVE-13567.08.patch, 
> HIVE-13567.09.patch, HIVE-13567.10.patch, HIVE-13567.11.patch, 
> HIVE-13567.12.patch, HIVE-13567.13.patch, HIVE-13567.14.patch, 
> HIVE-13567.15.patch, HIVE-13567.16.patch, HIVE-13567.17.patch, 
> HIVE-13567.18.patch, HIVE-13567.19.patch, HIVE-13567.20.patch, 
> HIVE-13567.21.patch, HIVE-13567.22.patch, HIVE-13567.23.patch, 
> HIVE-13567.23wip01.patch, HIVE-13567.23wip02.patch, HIVE-13567.23wip03.patch, 
> HIVE-13567.23wip04.patch, HIVE-13567.23wip05.patch, HIVE-13567.23wip06.patch, 
> HIVE-13567.23wip07.patch, HIVE-13567.23wip08.patch, HIVE-13567.23wip09.patch, 
> HIVE-13567.23wip10.patch, HIVE-13567.24.patch
>
>
> in phase 2, we are going to set auto-gather column on as default. This needs 
> to update golden files.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (HIVE-13567) Enable auto-gather column stats by default

2017-12-09 Thread Ashutosh Chauhan (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-13567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan updated HIVE-13567:

Attachment: HIVE-13567.24.patch

> Enable auto-gather column stats by default
> --
>
> Key: HIVE-13567
> URL: https://issues.apache.org/jira/browse/HIVE-13567
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Pengcheng Xiong
>Assignee: Zoltan Haindrich
> Attachments: HIVE-13567.01.patch, HIVE-13567.02.patch, 
> HIVE-13567.03.patch, HIVE-13567.04.patch, HIVE-13567.05.patch, 
> HIVE-13567.06.patch, HIVE-13567.07.patch, HIVE-13567.08.patch, 
> HIVE-13567.09.patch, HIVE-13567.10.patch, HIVE-13567.11.patch, 
> HIVE-13567.12.patch, HIVE-13567.13.patch, HIVE-13567.14.patch, 
> HIVE-13567.15.patch, HIVE-13567.16.patch, HIVE-13567.17.patch, 
> HIVE-13567.18.patch, HIVE-13567.19.patch, HIVE-13567.20.patch, 
> HIVE-13567.21.patch, HIVE-13567.22.patch, HIVE-13567.23.patch, 
> HIVE-13567.23wip01.patch, HIVE-13567.23wip02.patch, HIVE-13567.23wip03.patch, 
> HIVE-13567.23wip04.patch, HIVE-13567.23wip05.patch, HIVE-13567.23wip06.patch, 
> HIVE-13567.23wip07.patch, HIVE-13567.23wip08.patch, HIVE-13567.23wip09.patch, 
> HIVE-13567.23wip10.patch, HIVE-13567.24.patch
>
>
> in phase 2, we are going to set auto-gather column on as default. This needs 
> to update golden files.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (HIVE-13567) Enable auto-gather column stats by default

2017-12-09 Thread Ashutosh Chauhan (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-13567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan updated HIVE-13567:

Status: Open  (was: Patch Available)

> Enable auto-gather column stats by default
> --
>
> Key: HIVE-13567
> URL: https://issues.apache.org/jira/browse/HIVE-13567
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Pengcheng Xiong
>Assignee: Zoltan Haindrich
> Attachments: HIVE-13567.01.patch, HIVE-13567.02.patch, 
> HIVE-13567.03.patch, HIVE-13567.04.patch, HIVE-13567.05.patch, 
> HIVE-13567.06.patch, HIVE-13567.07.patch, HIVE-13567.08.patch, 
> HIVE-13567.09.patch, HIVE-13567.10.patch, HIVE-13567.11.patch, 
> HIVE-13567.12.patch, HIVE-13567.13.patch, HIVE-13567.14.patch, 
> HIVE-13567.15.patch, HIVE-13567.16.patch, HIVE-13567.17.patch, 
> HIVE-13567.18.patch, HIVE-13567.19.patch, HIVE-13567.20.patch, 
> HIVE-13567.21.patch, HIVE-13567.22.patch, HIVE-13567.23.patch, 
> HIVE-13567.23wip01.patch, HIVE-13567.23wip02.patch, HIVE-13567.23wip03.patch, 
> HIVE-13567.23wip04.patch, HIVE-13567.23wip05.patch, HIVE-13567.23wip06.patch, 
> HIVE-13567.23wip07.patch, HIVE-13567.23wip08.patch, HIVE-13567.23wip09.patch, 
> HIVE-13567.23wip10.patch
>
>
> in phase 2, we are going to set auto-gather column on as default. This needs 
> to update golden files.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (HIVE-18061) q.outs: be more selective with masikng hdfs paths

2017-12-09 Thread Hive QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-18061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16285021#comment-16285021
 ] 

Hive QA commented on HIVE-18061:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  6m 
36s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
18s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  0m 
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
25s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
13s{color} | {color:red} itests/util: The patch generated 3 new + 193 unchanged 
- 0 fixed = 196 total (was 193) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
19s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
11s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}  9m 15s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / 5bbd864 |
| Default Java | 1.8.0_111 |
| checkstyle | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-8165/yetus/diff-checkstyle-itests_util.txt
 |
| modules | C: itests/util U: itests/util |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-8165/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> q.outs: be more selective with masikng hdfs paths
> -
>
> Key: HIVE-18061
> URL: https://issues.apache.org/jira/browse/HIVE-18061
> Project: Hive
>  Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Laszlo Bodor
> Attachments: HIVE-18061.01.patch
>
>
> currently any line which contains a path which looks like an hdfs location is 
> replaced with a "masked pattern was here"...
> it might be releavant to record these messages; since even an exception 
> message might contain an hdfs location
> noticed in
> HIVE-18012



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (HIVE-18149) Stats: rownum estimation from datasize underestimates in most cases

2017-12-09 Thread Hive QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-18149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16285012#comment-16285012
 ] 

Hive QA commented on HIVE-18149:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12900887/HIVE-18149.02.patch

{color:red}ERROR:{color} -1 due to build exiting with an error

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8164/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8164/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8164/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Tests exited with: ExecutionException: java.util.concurrent.ExecutionException: 
org.apache.hive.ptest.execution.ssh.SSHExecutionException: RSyncResult 
[localFile=/data/hiveptest/logs/PreCommit-HIVE-Build-8164/succeeded/205_UTBatch_service_8_tests,
 remoteFile=/home/hiveptest/104.198.217.87-hiveptest-0/logs/, 
getExitCode()=255, getException()=null, getUser()=hiveptest, 
getHost()=104.198.217.87, getInstance()=0]: 'Warning: Permanently added 
'104.198.217.87' (ECDSA) to the list of known hosts.
receiving incremental file list
./
TEST-205_UTBatch_service_8_tests-TEST-org.apache.hive.service.auth.TestLdapAtnProviderWithMiniDS.xml

  0   0%0.00kB/s0:00:00  
 91,994 100%2.58MB/s0:00:00 (xfr#1, to-chk=12/14)
TEST-205_UTBatch_service_8_tests-TEST-org.apache.hive.service.auth.ldap.TestCustomQueryFilter.xml

  0   0%0.00kB/s0:00:00  
 87,958 100%2.33MB/s0:00:00 (xfr#2, to-chk=11/14)
TEST-205_UTBatch_service_8_tests-TEST-org.apache.hive.service.auth.ldap.TestQuery.xml

  0   0%0.00kB/s0:00:00  
 87,939 100%1.22MB/s0:00:00 (xfr#3, to-chk=10/14)
TEST-205_UTBatch_service_8_tests-TEST-org.apache.hive.service.auth.ldap.TestUserFilter.xml

  0   0%0.00kB/s0:00:00  
 87,929 100%1.22MB/s0:00:00 (xfr#4, to-chk=9/14)
TEST-205_UTBatch_service_8_tests-TEST-org.apache.hive.service.auth.ldap.TestUserSearchFilter.xml

  0   0%0.00kB/s0:00:00  
 88,367 100%  837.82kB/s0:00:00 (xfr#5, to-chk=8/14)
TEST-205_UTBatch_service_8_tests-TEST-org.apache.hive.service.cli.TestCLIServiceConnectionLimits.xml

  0   0%0.00kB/s0:00:00  
 89,238 100%  837.95kB/s0:00:00 (xfr#6, to-chk=7/14)
TEST-205_UTBatch_service_8_tests-TEST-org.apache.hive.service.cli.TestCLIServiceRestore.xml

  0   0%0.00kB/s0:00:00  
 87,707 100%  823.57kB/s0:00:00 (xfr#7, to-chk=6/14)
TEST-205_UTBatch_service_8_tests-TEST-org.apache.hive.service.cli.TestHiveSQLException.xml

  0   0%0.00kB/s0:00:00  
 88,448 100%  822.62kB/s0:00:00 (xfr#8, to-chk=5/14)
maven-test.txt

  0   0%0.00kB/s0:00:00  
  6,086 100%   56.07kB/s0:00:00 (xfr#9, to-chk=4/14)
logs/
logs/derby.log

  0   0%0.00kB/s0:00:00  
989 100%9.11kB/s0:00:00 (xfr#10, to-chk=1/14)
logs/hive.log

  0   0%0.00kB/s0:00:00  
 35,487,744   0%   33.84MB/s0:02:01  
 91,553,792   2%   43.66MB/s0:01:32  
148,144,128   3%   47.09MB/s0:01:24  
205,651,968   4%   49.04MB/s0:01:20  
262,864,896   6%   54.22MB/s0:01:11  
319,324,160   7%   54.35MB/s0:01:10  
377,126,912   8%   54.63MB/s0:01:08  Timeout, server 104.198.217.87 not 
responding.

rsync: connection unexpectedly closed (391788893 bytes received so far) 
[receiver]
rsync error: error in rsync protocol data stream (code 12) at io.c(226) 
[receiver=3.1.1]
rsync: connection unexpectedly closed (904 bytes received so far) [generator]
rsync error: unexplained error (code 255) at io.c(226) [generator=3.1.1]
ssh: connect to host 104.198.217.87 port 22: Connection timed out
rsync: connection unexpectedly closed (0 bytes received so far) [Receiver]
rsync error: unexplained error (code 255) at io.c(226) [Receiver=3.1.1]
ssh: connect to host 104.198.217.87 port 22: Connection timed out
rsync: connection unexpectedly closed (0 bytes received so far) [Receiver]
rsync error: unexplained error (code 255) at io.c(226) [Receiver=3.1.1]
ssh: connect to host 104.198.217.87 port 22: Connection timed out
rsync: connection unexpectedly closed (0 bytes received so far) [Receiver]
rsync error: unexplained error (code 255) at io.c(226) [Receiver=3.1.1]
ssh: connect to host 104.198.217.87 port 22: Connection timed out
rsync: connection unexpectedly closed (0 bytes received so far) [Receiver]
rsync error: unexplained error (code 255) at io.c(226) [Receiver=3.1.1]
'
{noformat}

This message is automatically 

[jira] [Updated] (HIVE-17710) LockManager should only lock Managed tables

2017-12-09 Thread Eugene Koifman (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-17710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eugene Koifman updated HIVE-17710:
--
Attachment: HIVE-17710.03.patch

> LockManager should only lock Managed tables
> ---
>
> Key: HIVE-17710
> URL: https://issues.apache.org/jira/browse/HIVE-17710
> Project: Hive
>  Issue Type: New Feature
>  Components: Transactions
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
> Attachments: HIVE-17710.01.patch, HIVE-17710.02.patch, 
> HIVE-17710.03.patch
>
>
> should the LM take locks on External tables?  Out of the box Acid LM is being 
> conservative which can cause throughput issues.
> A better strategy may be to exclude External tables but enable explicit "lock 
> table/partition " command (only on external tables?).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (HIVE-18245) clean up acid_vectorization_original.q

2017-12-09 Thread Eugene Koifman (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-18245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eugene Koifman updated HIVE-18245:
--
Attachment: HIVE-18245.03.patch

> clean up acid_vectorization_original.q
> --
>
> Key: HIVE-18245
> URL: https://issues.apache.org/jira/browse/HIVE-18245
> Project: Hive
>  Issue Type: Sub-task
>  Components: Transactions
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
> Fix For: 3.0.0
>
> Attachments: HIVE-18245.01.patch, HIVE-18245.02.patch, 
> HIVE-18245.03.patch
>
>
> now that HIVE-17923 is fixed, 
> acid_vectorization_original_tez.q/acid_vectorization_original.q can be 
> cleaned up



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (HIVE-18221) test acid default

2017-12-09 Thread Eugene Koifman (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-18221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eugene Koifman updated HIVE-18221:
--
Attachment: HIVE-18221.09.patch

> test acid default
> -
>
> Key: HIVE-18221
> URL: https://issues.apache.org/jira/browse/HIVE-18221
> Project: Hive
>  Issue Type: Test
>  Components: Transactions
>Affects Versions: 3.0.0
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
> Attachments: HIVE-18221.01.patch, HIVE-18221.02.patch, 
> HIVE-18221.03.patch, HIVE-18221.04.patch, HIVE-18221.07.patch, 
> HIVE-18221.08.patch, HIVE-18221.09.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (HIVE-18133) Parametrize TestTxnNoBuckets wrt Vectorization

2017-12-09 Thread Eugene Koifman (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-18133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eugene Koifman updated HIVE-18133:
--
Attachment: HIVE-18133.03.patch

patch 2 had no related UT failures
patch 3 adds the missing ASF header, i.e. a comment - no need to run build bot

> Parametrize TestTxnNoBuckets wrt Vectorization
> --
>
> Key: HIVE-18133
> URL: https://issues.apache.org/jira/browse/HIVE-18133
> Project: Hive
>  Issue Type: Sub-task
>  Components: Transactions
>Affects Versions: 3.0.0
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
>Priority: Minor
> Attachments: HIVE-18133.01.patch, HIVE-18133.02.patch, 
> HIVE-18133.03.patch
>
>
> it currently runs in Vector mode only
> {noformat}
>   public void setUp() throws Exception {
> setUpInternal();
> hiveConf.setBoolVar(HiveConf.ConfVars.HIVE_VECTORIZATION_ENABLED, true);
>   }
> {noformat}
> would be good to run both modes



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (HIVE-18133) Parametrize TestTxnNoBuckets wrt Vectorization

2017-12-09 Thread Eugene Koifman (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-18133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16284840#comment-16284840
 ] 

Eugene Koifman commented on HIVE-18133:
---

[~prasanth_j] could you review please

> Parametrize TestTxnNoBuckets wrt Vectorization
> --
>
> Key: HIVE-18133
> URL: https://issues.apache.org/jira/browse/HIVE-18133
> Project: Hive
>  Issue Type: Sub-task
>  Components: Transactions
>Affects Versions: 3.0.0
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
>Priority: Minor
> Attachments: HIVE-18133.01.patch, HIVE-18133.02.patch, 
> HIVE-18133.03.patch
>
>
> it currently runs in Vector mode only
> {noformat}
>   public void setUp() throws Exception {
> setUpInternal();
> hiveConf.setBoolVar(HiveConf.ConfVars.HIVE_VECTORIZATION_ENABLED, true);
>   }
> {noformat}
> would be good to run both modes



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (HIVE-18133) Parametrize TestTxnNoBuckets wrt Vectorization

2017-12-09 Thread Eugene Koifman (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-18133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eugene Koifman updated HIVE-18133:
--
Status: Open  (was: Patch Available)

> Parametrize TestTxnNoBuckets wrt Vectorization
> --
>
> Key: HIVE-18133
> URL: https://issues.apache.org/jira/browse/HIVE-18133
> Project: Hive
>  Issue Type: Sub-task
>  Components: Transactions
>Affects Versions: 3.0.0
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
>Priority: Minor
> Attachments: HIVE-18133.01.patch, HIVE-18133.02.patch
>
>
> it currently runs in Vector mode only
> {noformat}
>   public void setUp() throws Exception {
> setUpInternal();
> hiveConf.setBoolVar(HiveConf.ConfVars.HIVE_VECTORIZATION_ENABLED, true);
>   }
> {noformat}
> would be good to run both modes



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (HIVE-18124) clean up isAcidTable() API vs isInsertOnlyTable()

2017-12-09 Thread Eugene Koifman (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-18124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eugene Koifman updated HIVE-18124:
--
Attachment: HIVE-18124.03.patch

>  clean up isAcidTable() API vs isInsertOnlyTable()
> --
>
> Key: HIVE-18124
> URL: https://issues.apache.org/jira/browse/HIVE-18124
> Project: Hive
>  Issue Type: Bug
>  Components: Transactions
>Affects Versions: 3.0.0
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
> Attachments: HIVE-18124.01.patch, HIVE-18124.02.patch, 
> HIVE-18124.03.patch
>
>
> With the addition of MM tables (_AcidUtils.isInsertOnlyTable(table)_) the 
> methods in AcidUtils and dependent places are very muddled.
> Need to clean it up so that there is a isTransactional(Table) that checks 
> transactional=true setting and isAcid(Table) to mean full ACID and 
> isInsertOnly(Table) to mean MM tables.
> This would accurately describe the semantics of the tables.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (HIVE-18149) Stats: rownum estimation from datasize underestimates in most cases

2017-12-09 Thread Hive QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-18149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16284829#comment-16284829
 ] 

Hive QA commented on HIVE-18149:


| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
56s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  5m 
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
49s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
29s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
20s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  2m 
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m 
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
32s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
12s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 19m 55s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / 5bbd864 |
| Default Java | 1.8.0_111 |
| modules | C: common ql contrib itests/hive-blobstore U: . |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-8164/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> Stats: rownum estimation from datasize underestimates in most cases
> ---
>
> Key: HIVE-18149
> URL: https://issues.apache.org/jira/browse/HIVE-18149
> Project: Hive
>  Issue Type: Sub-task
>  Components: Statistics
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
> Attachments: HIVE-18149.01.patch, HIVE-18149.01wip01.patch, 
> HIVE-18149.02.patch
>
>
> rownum estimation is based on the following fact as of now:
> * datasize being used from the following sources:
> ** basicstats aggregates the loaded "on-heap" row sizes ; other readers are 
> able to give "raw size" estimation - I've checked orc; but I'm sure others 
> will do the sameapi docs are a bit vague about the methods purpose...
> ** if the basicstats level info is not available; the filesystem level 
> "file-size-sums" are used as the "raw data size" ; which is multiplied by the 
> [deserialization 
> ratio|https://github.com/apache/hive/blob/d9924ab3e285536f7e2cc15ecbea36a78c59c66d/ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java#L261]
>  ; which is currently 1.
> the problem with all of this is that deser factor is 1; and that rowsize 
> counts in the online object headers..
> example; 20 rows are loaded into a partition 
> [columnstats_partlvl_dp.q|https://github.com/apache/hive/blob/d9924ab3e285536f7e2cc15ecbea36a78c59c66d/ql/src/test/queries/clientpositive/columnstats_partlvl_dp.q#L7]
> after HIVE-18108 [this 
> 

[jira] [Updated] (HIVE-18258) Vectorization: Reduce-Side GROUP BY MERGEPARTIAL with duplicate columns is broken

2017-12-09 Thread Matt McCline (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-18258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt McCline updated HIVE-18258:

Status: Patch Available  (was: Open)

> Vectorization: Reduce-Side GROUP BY MERGEPARTIAL with duplicate columns is 
> broken
> -
>
> Key: HIVE-18258
> URL: https://issues.apache.org/jira/browse/HIVE-18258
> Project: Hive
>  Issue Type: Bug
>  Components: Hive
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Fix For: 3.0.0
>
> Attachments: HIVE-18258.01.patch
>
>
> See Q file.  Duplicate columns in key are not handled correctly.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (HIVE-18258) Vectorization: Reduce-Side GROUP BY MERGEPARTIAL with duplicate columns is broken

2017-12-09 Thread Matt McCline (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-18258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt McCline updated HIVE-18258:

Attachment: HIVE-18258.01.patch

> Vectorization: Reduce-Side GROUP BY MERGEPARTIAL with duplicate columns is 
> broken
> -
>
> Key: HIVE-18258
> URL: https://issues.apache.org/jira/browse/HIVE-18258
> Project: Hive
>  Issue Type: Bug
>  Components: Hive
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Fix For: 3.0.0
>
> Attachments: HIVE-18258.01.patch
>
>
> See Q file.  Duplicate columns in key are not handled correctly.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (HIVE-13567) Enable auto-gather column stats by default

2017-12-09 Thread Hive QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-13567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16284818#comment-16284818
 ] 

Hive QA commented on HIVE-13567:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12900955/HIVE-13567.23.patch

{color:red}ERROR:{color} -1 due to build exiting with an error

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8163/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8163/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8163/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit 
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2017-12-09 16:22:41.730
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export 
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ 
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-8163/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2017-12-09 16:22:41.733
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 5bbd864 HIVE-14069 : update curator version to 2.12.0 (Jason 
Dere via Ashutosh Chauhan)
+ git clean -f -d
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 5bbd864 HIVE-14069 : update curator version to 2.12.0 (Jason 
Dere via Ashutosh Chauhan)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2017-12-09 16:22:42.347
+ rm -rf ../yetus
+ mkdir ../yetus
+ cp -R . ../yetus
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-8163/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh 
/data/hiveptest/working/scratch/build.patch
error: patch failed: 
ql/src/test/results/clientpositive/llap/subquery_exists.q.out:1327
error: ql/src/test/results/clientpositive/llap/subquery_exists.q.out: patch 
does not apply
The patch does not appear to apply with p0, p1, or p2
+ exit 1
'
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12900955 - PreCommit-HIVE-Build

> Enable auto-gather column stats by default
> --
>
> Key: HIVE-13567
> URL: https://issues.apache.org/jira/browse/HIVE-13567
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Pengcheng Xiong
>Assignee: Zoltan Haindrich
> Attachments: HIVE-13567.01.patch, HIVE-13567.02.patch, 
> HIVE-13567.03.patch, HIVE-13567.04.patch, HIVE-13567.05.patch, 
> HIVE-13567.06.patch, HIVE-13567.07.patch, HIVE-13567.08.patch, 
> HIVE-13567.09.patch, HIVE-13567.10.patch, HIVE-13567.11.patch, 
> HIVE-13567.12.patch, HIVE-13567.13.patch, HIVE-13567.14.patch, 
> HIVE-13567.15.patch, HIVE-13567.16.patch, HIVE-13567.17.patch, 
> HIVE-13567.18.patch, HIVE-13567.19.patch, HIVE-13567.20.patch, 
> HIVE-13567.21.patch, HIVE-13567.22.patch, HIVE-13567.23.patch, 
> HIVE-13567.23wip01.patch, HIVE-13567.23wip02.patch, HIVE-13567.23wip03.patch, 
> HIVE-13567.23wip04.patch, HIVE-13567.23wip05.patch, HIVE-13567.23wip06.patch, 
> HIVE-13567.23wip07.patch, HIVE-13567.23wip08.patch, HIVE-13567.23wip09.patch, 
> HIVE-13567.23wip10.patch
>
>
> in phase 2, we are going to set auto-gather column on as default. This needs 
> to update golden files.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (HIVE-18244) CachedStore: Fix UT when CachedStore is enabled

2017-12-09 Thread Hive QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-18244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16284815#comment-16284815
 ] 

Hive QA commented on HIVE-18244:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12901123/HIVE-18244.1.patch

{color:red}ERROR:{color} -1 due to build exiting with an error

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8162/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8162/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8162/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Tests exited with: ExecutionException: java.util.concurrent.ExecutionException: 
org.apache.hive.ptest.execution.ssh.SSHExecutionException: RSyncResult 
[localFile=/data/hiveptest/logs/PreCommit-HIVE-Build-8162/succeeded/47-TestCliDriver-fileformat_sequencefile.q-dynamic_semijoin_reduction.q-ppd_union.q-and-27-more,
 remoteFile=/home/hiveptest/104.198.248.189-hiveptest-0/logs/, 
getExitCode()=255, getException()=null, getUser()=hiveptest, 
getHost()=104.198.248.189, getInstance()=0]: 'ssh: connect to host 
104.198.248.189 port 22: Connection refused
rsync: connection unexpectedly closed (0 bytes received so far) [Receiver]
rsync error: unexplained error (code 255) at io.c(226) [Receiver=3.1.1]
ssh: connect to host 104.198.248.189 port 22: Connection timed out
rsync: connection unexpectedly closed (0 bytes received so far) [Receiver]
rsync error: unexplained error (code 255) at io.c(226) [Receiver=3.1.1]
ssh: connect to host 104.198.248.189 port 22: Connection timed out
rsync: connection unexpectedly closed (0 bytes received so far) [Receiver]
rsync error: unexplained error (code 255) at io.c(226) [Receiver=3.1.1]
ssh: connect to host 104.198.248.189 port 22: Connection timed out
rsync: connection unexpectedly closed (0 bytes received so far) [Receiver]
rsync error: unexplained error (code 255) at io.c(226) [Receiver=3.1.1]
ssh: connect to host 104.198.248.189 port 22: Connection timed out
rsync: connection unexpectedly closed (0 bytes received so far) [Receiver]
rsync error: unexplained error (code 255) at io.c(226) [Receiver=3.1.1]
'
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12901123 - PreCommit-HIVE-Build

> CachedStore: Fix UT when CachedStore is enabled
> ---
>
> Key: HIVE-18244
> URL: https://issues.apache.org/jira/browse/HIVE-18244
> Project: Hive
>  Issue Type: Bug
>Reporter: Vaibhav Gumashta
>Assignee: Vaibhav Gumashta
> Attachments: HIVE-18244.1.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Assigned] (HIVE-18258) Vectorization: Reduce-Side GROUP BY MERGEPARTIAL with duplicate columns is broken

2017-12-09 Thread Matt McCline (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-18258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt McCline reassigned HIVE-18258:
---


> Vectorization: Reduce-Side GROUP BY MERGEPARTIAL with duplicate columns is 
> broken
> -
>
> Key: HIVE-18258
> URL: https://issues.apache.org/jira/browse/HIVE-18258
> Project: Hive
>  Issue Type: Bug
>  Components: Hive
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Fix For: 3.0.0
>
>
> See Q file.  Duplicate columns in key are not handled correctly.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (HIVE-18244) CachedStore: Fix UT when CachedStore is enabled

2017-12-09 Thread Hive QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-18244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16284796#comment-16284796
 ] 

Hive QA commented on HIVE-18244:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  1m  
5s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  5m 
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  4m 
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
58s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  5m 
27s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
19s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  5m 
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  4m 
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  4m 
53s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
20s{color} | {color:red} common: The patch generated 1 new + 930 unchanged - 1 
fixed = 931 total (was 931) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  1m 
39s{color} | {color:red} root: The patch generated 1 new + 930 unchanged - 1 
fixed = 931 total (was 931) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red}  0m  
0s{color} | {color:red} The patch has 4 line(s) that end in whitespace. Use git 
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply 
{color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green}  0m  
4s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  5m 
31s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
11s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 37m  8s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  
xml  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / 5bbd864 |
| Default Java | 1.8.0_111 |
| checkstyle | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-8162/yetus/diff-checkstyle-common.txt
 |
| checkstyle | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-8162/yetus/diff-checkstyle-root.txt
 |
| whitespace | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-8162/yetus/whitespace-eol.txt 
|
| modules | C: common . U: . |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-8162/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> CachedStore: Fix UT when CachedStore is enabled
> ---
>
> Key: HIVE-18244
> URL: https://issues.apache.org/jira/browse/HIVE-18244
> Project: Hive
>  Issue Type: Bug
>Reporter: Vaibhav Gumashta
>Assignee: Vaibhav Gumashta
> Attachments: HIVE-18244.1.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (HIVE-18191) Vectorization: Add validation of TableScanOperator (gather statistics) back

2017-12-09 Thread Matt McCline (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-18191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt McCline updated HIVE-18191:

Attachment: HIVE-18191.06.patch

> Vectorization: Add validation of TableScanOperator (gather statistics) back
> ---
>
> Key: HIVE-18191
> URL: https://issues.apache.org/jira/browse/HIVE-18191
> Project: Hive
>  Issue Type: Bug
>  Components: Hive
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Attachments: HIVE-18191.01.patch, HIVE-18191.02.patch, 
> HIVE-18191.03.patch, HIVE-18191.04.patch, HIVE-18191.05.patch, 
> HIVE-18191.06.patch
>
>
> HIVE-17433 accidentally removed call to validateTableScanOperator.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (HIVE-18191) Vectorization: Add validation of TableScanOperator (gather statistics) back

2017-12-09 Thread Matt McCline (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-18191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt McCline updated HIVE-18191:

Status: Patch Available  (was: In Progress)

> Vectorization: Add validation of TableScanOperator (gather statistics) back
> ---
>
> Key: HIVE-18191
> URL: https://issues.apache.org/jira/browse/HIVE-18191
> Project: Hive
>  Issue Type: Bug
>  Components: Hive
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Attachments: HIVE-18191.01.patch, HIVE-18191.02.patch, 
> HIVE-18191.03.patch, HIVE-18191.04.patch, HIVE-18191.05.patch, 
> HIVE-18191.06.patch
>
>
> HIVE-17433 accidentally removed call to validateTableScanOperator.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (HIVE-18191) Vectorization: Add validation of TableScanOperator (gather statistics) back

2017-12-09 Thread Matt McCline (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-18191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt McCline updated HIVE-18191:

Status: In Progress  (was: Patch Available)

> Vectorization: Add validation of TableScanOperator (gather statistics) back
> ---
>
> Key: HIVE-18191
> URL: https://issues.apache.org/jira/browse/HIVE-18191
> Project: Hive
>  Issue Type: Bug
>  Components: Hive
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Attachments: HIVE-18191.01.patch, HIVE-18191.02.patch, 
> HIVE-18191.03.patch, HIVE-18191.04.patch, HIVE-18191.05.patch
>
>
> HIVE-17433 accidentally removed call to validateTableScanOperator.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (HIVE-18191) Vectorization: Add validation of TableScanOperator (gather statistics) back

2017-12-09 Thread Matt McCline (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-18191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16284784#comment-16284784
 ] 

Matt McCline commented on HIVE-18191:
-

Flaky Hive QA system.

> Vectorization: Add validation of TableScanOperator (gather statistics) back
> ---
>
> Key: HIVE-18191
> URL: https://issues.apache.org/jira/browse/HIVE-18191
> Project: Hive
>  Issue Type: Bug
>  Components: Hive
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Attachments: HIVE-18191.01.patch, HIVE-18191.02.patch, 
> HIVE-18191.03.patch, HIVE-18191.04.patch, HIVE-18191.05.patch
>
>
> HIVE-17433 accidentally removed call to validateTableScanOperator.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (HIVE-18061) q.outs: be more selective with masikng hdfs paths

2017-12-09 Thread Zoltan Haindrich (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-18061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16284766#comment-16284766
 ] 

Zoltan Haindrich commented on HIVE-18061:
-

+1 looks good; but I think there will be some qtests which will need some 
updating


> q.outs: be more selective with masikng hdfs paths
> -
>
> Key: HIVE-18061
> URL: https://issues.apache.org/jira/browse/HIVE-18061
> Project: Hive
>  Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Laszlo Bodor
> Attachments: HIVE-18061.01.patch
>
>
> currently any line which contains a path which looks like an hdfs location is 
> replaced with a "masked pattern was here"...
> it might be releavant to record these messages; since even an exception 
> message might contain an hdfs location
> noticed in
> HIVE-18012



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (HIVE-18191) Vectorization: Add validation of TableScanOperator (gather statistics) back

2017-12-09 Thread Hive QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-18191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16284750#comment-16284750
 ] 

Hive QA commented on HIVE-18191:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12901356/HIVE-18191.05.patch

{color:red}ERROR:{color} -1 due to build exiting with an error

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8161/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8161/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8161/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Tests exited with: ExecutionException: java.util.concurrent.ExecutionException: 
org.apache.hive.ptest.execution.ssh.SSHExecutionException: RSyncResult 
[localFile=/data/hiveptest/logs/PreCommit-HIVE-Build-8161/succeeded/7-TestCliDriver-orc_ppd_decimal.q-parquet_decimal.q-udf_when.q-and-27-more,
 remoteFile=/home/hiveptest/35.184.147.31-hiveptest-0/logs/, getExitCode()=255, 
getException()=null, getUser()=hiveptest, getHost()=35.184.147.31, 
getInstance()=0]: 'ssh: connect to host 35.184.147.31 port 22: Connection timed 
out
rsync: connection unexpectedly closed (0 bytes received so far) [Receiver]
rsync error: unexplained error (code 255) at io.c(226) [Receiver=3.1.1]
ssh: connect to host 35.184.147.31 port 22: Connection timed out
rsync: connection unexpectedly closed (0 bytes received so far) [Receiver]
rsync error: unexplained error (code 255) at io.c(226) [Receiver=3.1.1]
ssh: connect to host 35.184.147.31 port 22: Connection timed out
rsync: connection unexpectedly closed (0 bytes received so far) [Receiver]
rsync error: unexplained error (code 255) at io.c(226) [Receiver=3.1.1]
ssh: connect to host 35.184.147.31 port 22: Connection timed out
rsync: connection unexpectedly closed (0 bytes received so far) [Receiver]
rsync error: unexplained error (code 255) at io.c(226) [Receiver=3.1.1]
ssh: connect to host 35.184.147.31 port 22: Connection timed out
rsync: connection unexpectedly closed (0 bytes received so far) [Receiver]
rsync error: unexplained error (code 255) at io.c(226) [Receiver=3.1.1]
'
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12901356 - PreCommit-HIVE-Build

> Vectorization: Add validation of TableScanOperator (gather statistics) back
> ---
>
> Key: HIVE-18191
> URL: https://issues.apache.org/jira/browse/HIVE-18191
> Project: Hive
>  Issue Type: Bug
>  Components: Hive
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Attachments: HIVE-18191.01.patch, HIVE-18191.02.patch, 
> HIVE-18191.03.patch, HIVE-18191.04.patch, HIVE-18191.05.patch
>
>
> HIVE-17433 accidentally removed call to validateTableScanOperator.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (HIVE-18191) Vectorization: Add validation of TableScanOperator (gather statistics) back

2017-12-09 Thread Hive QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-18191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16284735#comment-16284735
 ] 

Hive QA commented on HIVE-18191:


| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  6m 
 3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
58s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
36s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
54s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
51s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
12s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 12m 40s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / 5bbd864 |
| Default Java | 1.8.0_111 |
| modules | C: ql U: ql |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-8161/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> Vectorization: Add validation of TableScanOperator (gather statistics) back
> ---
>
> Key: HIVE-18191
> URL: https://issues.apache.org/jira/browse/HIVE-18191
> Project: Hive
>  Issue Type: Bug
>  Components: Hive
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Attachments: HIVE-18191.01.patch, HIVE-18191.02.patch, 
> HIVE-18191.03.patch, HIVE-18191.04.patch, HIVE-18191.05.patch
>
>
> HIVE-17433 accidentally removed call to validateTableScanOperator.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (HIVE-18191) Vectorization: Add validation of TableScanOperator (gather statistics) back

2017-12-09 Thread Matt McCline (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-18191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16284723#comment-16284723
 ] 

Matt McCline commented on HIVE-18191:
-

Jenkins restart.

> Vectorization: Add validation of TableScanOperator (gather statistics) back
> ---
>
> Key: HIVE-18191
> URL: https://issues.apache.org/jira/browse/HIVE-18191
> Project: Hive
>  Issue Type: Bug
>  Components: Hive
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Attachments: HIVE-18191.01.patch, HIVE-18191.02.patch, 
> HIVE-18191.03.patch, HIVE-18191.04.patch, HIVE-18191.05.patch
>
>
> HIVE-17433 accidentally removed call to validateTableScanOperator.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (HIVE-18191) Vectorization: Add validation of TableScanOperator (gather statistics) back

2017-12-09 Thread Matt McCline (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-18191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt McCline updated HIVE-18191:

Status: Patch Available  (was: In Progress)

> Vectorization: Add validation of TableScanOperator (gather statistics) back
> ---
>
> Key: HIVE-18191
> URL: https://issues.apache.org/jira/browse/HIVE-18191
> Project: Hive
>  Issue Type: Bug
>  Components: Hive
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Attachments: HIVE-18191.01.patch, HIVE-18191.02.patch, 
> HIVE-18191.03.patch, HIVE-18191.04.patch, HIVE-18191.05.patch
>
>
> HIVE-17433 accidentally removed call to validateTableScanOperator.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (HIVE-18191) Vectorization: Add validation of TableScanOperator (gather statistics) back

2017-12-09 Thread Matt McCline (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-18191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt McCline updated HIVE-18191:

Attachment: HIVE-18191.05.patch

> Vectorization: Add validation of TableScanOperator (gather statistics) back
> ---
>
> Key: HIVE-18191
> URL: https://issues.apache.org/jira/browse/HIVE-18191
> Project: Hive
>  Issue Type: Bug
>  Components: Hive
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Attachments: HIVE-18191.01.patch, HIVE-18191.02.patch, 
> HIVE-18191.03.patch, HIVE-18191.04.patch, HIVE-18191.05.patch
>
>
> HIVE-17433 accidentally removed call to validateTableScanOperator.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (HIVE-18191) Vectorization: Add validation of TableScanOperator (gather statistics) back

2017-12-09 Thread Matt McCline (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-18191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt McCline updated HIVE-18191:

Status: In Progress  (was: Patch Available)

> Vectorization: Add validation of TableScanOperator (gather statistics) back
> ---
>
> Key: HIVE-18191
> URL: https://issues.apache.org/jira/browse/HIVE-18191
> Project: Hive
>  Issue Type: Bug
>  Components: Hive
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Attachments: HIVE-18191.01.patch, HIVE-18191.02.patch, 
> HIVE-18191.03.patch, HIVE-18191.04.patch
>
>
> HIVE-17433 accidentally removed call to validateTableScanOperator.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (HIVE-18203) change the way WM is enabled and allow dropping the last resource plan

2017-12-09 Thread Hive QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-18203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16284713#comment-16284713
 ] 

Hive QA commented on HIVE-18203:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12901270/HIVE-18203.01.patch

{color:red}ERROR:{color} -1 due to build exiting with an error

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8160/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8160/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8160/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Tests exited with: ExecutionException: java.util.concurrent.ExecutionException: 
org.apache.hive.ptest.execution.ssh.SSHExecutionException: RSyncResult 
[localFile=/data/hiveptest/logs/PreCommit-HIVE-Build-8160/succeeded/18-TestCliDriver-list_bucket_dml_1.q-ppd_join3.q-auto_join23.q-and-27-more,
 remoteFile=/home/hiveptest/35.184.14.247-hiveptest-0/logs/, getExitCode()=255, 
getException()=null, getUser()=hiveptest, getHost()=35.184.14.247, 
getInstance()=0]: 'Warning: Permanently added '35.184.14.247' (ECDSA) to the 
list of known hosts.
receiving incremental file list
./
TEST-18-TestCliDriver-list_bucket_dml_1.q-ppd_join3.q-auto_join23.q-and-27-more-TEST-org.apache.hadoop.hive.cli.TestCliDriver.xml

  0   0%0.00kB/s0:00:00  
114,519 100%1.61MB/s0:00:00 (xfr#1, to-chk=5/7)
maven-test.txt

  0   0%0.00kB/s0:00:00  
 56,493 100%  540.87kB/s0:00:00 (xfr#2, to-chk=4/7)
logs/
logs/derby.log

  0   0%0.00kB/s0:00:00  
 39,894 100%  284.37kB/s0:00:00 (xfr#3, to-chk=1/7)
logs/hive.log

  0   0%0.00kB/s0:00:00  
  1,966,080   2%1.83MB/s0:00:41  
  8,585,216  10%4.04MB/s0:00:16  
 25,690,112  32%8.10MB/s0:00:06  Timeout, server 35.184.14.247 not 
responding.

rsync: connection unexpectedly closed (55496090 bytes received so far) 
[receiver]
rsync error: error in rsync protocol data stream (code 12) at io.c(226) 
[receiver=3.1.1]
rsync: connection unexpectedly closed (428 bytes received so far) [generator]
rsync error: unexplained error (code 255) at io.c(226) [generator=3.1.1]
ssh: connect to host 35.184.14.247 port 22: Connection timed out
rsync: connection unexpectedly closed (0 bytes received so far) [Receiver]
rsync error: unexplained error (code 255) at io.c(226) [Receiver=3.1.1]
ssh: connect to host 35.184.14.247 port 22: Connection timed out
rsync: connection unexpectedly closed (0 bytes received so far) [Receiver]
rsync error: unexplained error (code 255) at io.c(226) [Receiver=3.1.1]
ssh: connect to host 35.184.14.247 port 22: Connection timed out
rsync: connection unexpectedly closed (0 bytes received so far) [Receiver]
rsync error: unexplained error (code 255) at io.c(226) [Receiver=3.1.1]
ssh: connect to host 35.184.14.247 port 22: Connection timed out
rsync: connection unexpectedly closed (0 bytes received so far) [Receiver]
rsync error: unexplained error (code 255) at io.c(226) [Receiver=3.1.1]
'
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12901270 - PreCommit-HIVE-Build

> change the way WM is enabled and allow dropping the last resource plan
> --
>
> Key: HIVE-18203
> URL: https://issues.apache.org/jira/browse/HIVE-18203
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Aswathy Chellammal Sreekumar
>Assignee: Sergey Shelukhin
> Attachments: HIVE-18203.01.patch, HIVE-18203.patch
>
>
> Currently it's impossible to drop the last active resource plan even if WM is 
> disabled. It should be possible to deactivate the last resource plan AND 
> disable WM in the same action. Activating a resource plan should enable WM in 
> this case.
> This should interact with the WM queue config in a sensible manner.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (HIVE-18203) change the way WM is enabled and allow dropping the last resource plan

2017-12-09 Thread Hive QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-18203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16284712#comment-16284712
 ] 

Hive QA commented on HIVE-18203:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
50s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  6m 
 5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  2m 
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  2m  
8s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
20s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  2m 
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  2m  
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  2m  
7s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
32s{color} | {color:red} standalone-metastore: The patch generated 6 new + 2375 
unchanged - 3 fixed = 2381 total (was 2378) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
37s{color} | {color:red} ql: The patch generated 4 new + 1074 unchanged - 2 
fixed = 1078 total (was 1076) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
11s{color} | {color:red} service: The patch generated 1 new + 31 unchanged - 2 
fixed = 32 total (was 33) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red}  0m  
0s{color} | {color:red} The patch has 2 line(s) that end in whitespace. Use git 
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply 
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  2m 
17s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
11s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m  5s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / 5bbd864 |
| Default Java | 1.8.0_111 |
| checkstyle | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-8160/yetus/diff-checkstyle-standalone-metastore.txt
 |
| checkstyle | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-8160/yetus/diff-checkstyle-ql.txt
 |
| checkstyle | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-8160/yetus/diff-checkstyle-service.txt
 |
| whitespace | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-8160/yetus/whitespace-eol.txt 
|
| modules | C: standalone-metastore ql service itests/hcatalog-unit U: . |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-8160/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> change the way WM is enabled and allow dropping the last resource plan
> --
>
> Key: HIVE-18203
> URL: https://issues.apache.org/jira/browse/HIVE-18203
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Aswathy Chellammal Sreekumar
>Assignee: Sergey Shelukhin
> Attachments: HIVE-18203.01.patch, HIVE-18203.patch
>
>
> Currently it's impossible to drop the last active resource plan even if WM is 
> disabled. It should be possible to deactivate the last 

[jira] [Assigned] (HIVE-18237) missing results for insert_only table after DP insert

2017-12-09 Thread Zoltan Haindrich (JIRA)

 [ 
https://issues.apache.org/jira/browse/HIVE-18237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zoltan Haindrich reassigned HIVE-18237:
---

Assignee: (was: Zoltan Haindrich)

Hello Steve!
Seems like there are quite a few tests which have proved that this solution is 
a no go..so I guess you might be better of handling this ticket... :)

> missing results for insert_only table after DP insert
> -
>
> Key: HIVE-18237
> URL: https://issues.apache.org/jira/browse/HIVE-18237
> Project: Hive
>  Issue Type: Bug
>  Components: Transactions
>Reporter: Zoltan Haindrich
> Attachments: HIVE-18237.01.patch
>
>
> {code}
> set hive.stats.column.autogather=false;
> set hive.exec.dynamic.partition.mode=nonstrict;
> set hive.exec.max.dynamic.partitions.pernode=200;
> set hive.exec.max.dynamic.partitions=200;
> set hive.support.concurrency=true;
> set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
> create table i0 (p int,v int);
> insert into i0 values
> (0,0),
> (2,2),
> (3,3);
> create table p0 (v int) partitioned by (p int) stored as orc 
>   tblproperties ("transactional"="true", 
> "transactional_properties"="insert_only");
> explain insert overwrite table p0 partition (p) select * from i0 where v < 3;
> insert overwrite table p0 partition (p) select * from i0 where v < 3;
> select count(*) from p0 where v!=1;
> {code}
> The table p0 should contain {{2}} rows at this point; but the result is {{0}}.
> * seems to be specific to insert_only tables
> * the existing data appears if an {{insert into}} is executed.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (HIVE-18240) support getClientInfo/setClientInfo in JDBC

2017-12-09 Thread Hive QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-18240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16284690#comment-16284690
 ] 

Hive QA commented on HIVE-18240:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12901269/HIVE-18240.01.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 6 failed/errored test(s), 11517 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_ppd_decimal] 
(batchId=9)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[hybridgrace_hashjoin_2]
 (batchId=157)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
 (batchId=165)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb] 
(batchId=160)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_multi] 
(batchId=113)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints 
(batchId=226)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8159/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8159/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8159/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 6 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12901269 - PreCommit-HIVE-Build

> support getClientInfo/setClientInfo in JDBC
> ---
>
> Key: HIVE-18240
> URL: https://issues.apache.org/jira/browse/HIVE-18240
> Project: Hive
>  Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: HIVE-18240.01.patch, HIVE-18240.patch
>
>
> These are JDBC APIs that allow the user of the driver to provide client info 
> to the server; the list of the fields supported by the driver is returned as 
> a result set by getClientInfoProperties API.
> I've looked at IBM, Oracle, MS etc. docs and it seems like ApplicationName is 
> a common one; there's also ClientHostname, etc. that we don't need because 
> HS2 derives them already.
> The client will then set these properties via setClientInfo if desired. 
> Whether it is desired by any BI tools of significance I've no idea. 
> The properties are sent to the server on connect (which is what Microsoft 
> seems to do, but in Hive model it's impossible because HiveConnection 
> connects in ctor), or on the next query (I don't recall where I've seen 
> this), or immediately (which is what I do in this patch).
> The getClientInfo API on the driver side seems completely pointless, so I 
> cache clientinfo locally for it.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (HIVE-18240) support getClientInfo/setClientInfo in JDBC

2017-12-09 Thread Hive QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-18240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16284682#comment-16284682
 ] 

Hive QA commented on HIVE-18240:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  1m 
23s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  5m 
21s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
21s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
38s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
58s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
21s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red}  0m 
18s{color} | {color:red} service in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red}  0m 
18s{color} | {color:red} service in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red}  0m 18s{color} 
| {color:red} service in the patch failed. {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
13s{color} | {color:red} service: The patch generated 35 new + 122 unchanged - 
0 fixed = 157 total (was 122) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
10s{color} | {color:red} jdbc: The patch generated 12 new + 67 unchanged - 0 
fixed = 79 total (was 67) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red}  0m  
0s{color} | {color:red} The patch has 17 line(s) that end in whitespace. Use 
git apply --whitespace=fix <>. Refer 
https://git-scm.com/docs/git-apply {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
57s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
12s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 15m 43s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / 5bbd864 |
| Default Java | 1.8.0_111 |
| mvninstall | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-8159/yetus/patch-mvninstall-service.txt
 |
| compile | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-8159/yetus/patch-compile-service.txt
 |
| javac | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-8159/yetus/patch-compile-service.txt
 |
| checkstyle | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-8159/yetus/diff-checkstyle-service.txt
 |
| checkstyle | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-8159/yetus/diff-checkstyle-jdbc.txt
 |
| whitespace | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-8159/yetus/whitespace-eol.txt 
|
| modules | C: service-rpc service jdbc itests/hive-unit U: . |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-8159/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> support getClientInfo/setClientInfo in JDBC
> ---
>
> Key: HIVE-18240
> URL: https://issues.apache.org/jira/browse/HIVE-18240
> Project: Hive
>  Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: HIVE-18240.01.patch, HIVE-18240.patch
>
>
> These are JDBC APIs that allow the user of the driver to provide client info 
> to the server; the list of the fields supported by the driver is returned as 
> a result set by getClientInfoProperties API.
> I've looked at 

[jira] [Commented] (HIVE-17710) LockManager should only lock Managed tables

2017-12-09 Thread Hive QA (JIRA)

[ 
https://issues.apache.org/jira/browse/HIVE-17710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16284663#comment-16284663
 ] 

Hive QA commented on HIVE-17710:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12901276/HIVE-17710.02.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 11498 tests 
executed
*Failed tests:*
{noformat}
TestMiniTezCliDriver - did not produce a TEST-*.xml file (likely timed out) 
(batchId=102)

[update_orig_table.q,explainanalyze_3.q,orc_vectorization_ppd.q,vectorization_limit.q,vector_non_string_partition.q,multi_count_distinct.q,explainanalyze_1.q,explainuser_3.q,hybridgrace_hashjoin_1.q,delete_orig_table.q,hybridgrace_hashjoin_2.q,tez-tag.q,orc_merge12.q,explainanalyze_5.q,vector_join_part_col_char.q]
org.apache.hadoop.hive.cli.TestContribNegativeCliDriver.org.apache.hadoop.hive.cli.TestContribNegativeCliDriver
 (batchId=243)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[hybridgrace_hashjoin_2]
 (batchId=157)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
 (batchId=165)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb] 
(batchId=160)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_multi] 
(batchId=113)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints 
(batchId=226)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8158/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8158/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8158/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 7 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12901276 - PreCommit-HIVE-Build

> LockManager should only lock Managed tables
> ---
>
> Key: HIVE-17710
> URL: https://issues.apache.org/jira/browse/HIVE-17710
> Project: Hive
>  Issue Type: New Feature
>  Components: Transactions
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
> Attachments: HIVE-17710.01.patch, HIVE-17710.02.patch
>
>
> should the LM take locks on External tables?  Out of the box Acid LM is being 
> conservative which can cause throughput issues.
> A better strategy may be to exclude External tables but enable explicit "lock 
> table/partition " command (only on external tables?).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)