[jira] [Commented] (HIVE-3907) Hive should support adding multiple resources at once
[ https://issues.apache.org/jira/browse/HIVE-3907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14009351#comment-14009351 ] Lefty Leverenz commented on HIVE-3907: -- No user doc, right? Hive should support adding multiple resources at once - Key: HIVE-3907 URL: https://issues.apache.org/jira/browse/HIVE-3907 Project: Hive Issue Type: Improvement Components: CLI Reporter: Navis Assignee: Navis Priority: Trivial Fix For: 0.14.0 Attachments: HIVE-3907.2.patch.txt, HIVE-3907.3.patch.txt, HIVE-3907.D7971.1.patch Currently hive adds resources in one by one manner. And for JAR resources, one classloader is created for each jar file, which seemed not good idea. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HIVE-7127) Handover more details on exception in hiveserver2
Navis created HIVE-7127:
---
Summary: Handover more details on exception in hiveserver2
Key: HIVE-7127
URL: https://issues.apache.org/jira/browse/HIVE-7127
Project: Hive
Issue Type: Improvement
Components: JDBC
Reporter: Navis
Assignee: Navis
Priority: Trivial
Currently, JDBC hands over exception message and error codes. But it's not
helpful for debugging.
{noformat}
org.apache.hive.service.cli.HiveSQLException: Error while compiling statement:
FAILED: ParseException line 1:0 cannot recognize input near 'createa' 'asd'
'EOF'
at org.apache.hive.jdbc.Utils.verifySuccess(Utils.java:121)
at org.apache.hive.jdbc.Utils.verifySuccessWithInfo(Utils.java:109)
at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:231)
at org.apache.hive.beeline.Commands.execute(Commands.java:736)
at org.apache.hive.beeline.Commands.sql(Commands.java:657)
at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:889)
at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:744)
at
org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:459)
at org.apache.hive.beeline.BeeLine.main(BeeLine.java:442)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:160)
{noformat}
With this patch, JDBC client can get more details on hiveserver2.
{noformat}
Caused by: org.apache.hive.service.cli.HiveSQLException: Error while compiling
statement: FAILED: ParseException line 1:0 cannot recognize input near
'createa' 'asd' 'EOF'
at org.apache.hive.service.cli.operation.SQLOperation.prepare(Unknown
Source)
at org.apache.hive.service.cli.operation.SQLOperation.run(Unknown
Source)
at
org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(Unknown
Source)
at
org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(Unknown
Source)
at org.apache.hive.service.cli.CLIService.executeStatementAsync(Unknown
Source)
at
org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(Unknown
Source)
at
org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(Unknown
Source)
at
org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(Unknown
Source)
at org.apache.thrift.ProcessFunction.process(Unknown Source)
at org.apache.thrift.TBaseProcessor.process(Unknown Source)
at org.apache.hive.service.auth.TSetIpAddressProcessor.process(Unknown
Source)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(Unknown
Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
Review Request 21922: Handover more details on exception in hiveserver2
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/21922/
---
Review request for hive.
Bugs: HIVE-7127
https://issues.apache.org/jira/browse/HIVE-7127
Repository: hive-git
Description
---
NO_PRECOMMIT_TEST
Currently, JDBC hands over exception message and error codes. But it's not
helpful for debugging.
{noformat}
org.apache.hive.service.cli.HiveSQLException: Error while compiling statement:
FAILED: ParseException line 1:0 cannot recognize input near 'createa' 'asd'
'EOF'
at org.apache.hive.jdbc.Utils.verifySuccess(Utils.java:121)
at org.apache.hive.jdbc.Utils.verifySuccessWithInfo(Utils.java:109)
at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:231)
at org.apache.hive.beeline.Commands.execute(Commands.java:736)
at org.apache.hive.beeline.Commands.sql(Commands.java:657)
at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:889)
at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:744)
at
org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:459)
at org.apache.hive.beeline.BeeLine.main(BeeLine.java:442)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:160)
{noformat}
With this patch, JDBC client can get more details on hiveserver2.
{noformat}
Caused by: org.apache.hive.service.cli.HiveSQLException: Error while compiling
statement: FAILED: ParseException line 1:0 cannot recognize input near
'createa' 'asd' 'EOF'
at org.apache.hive.service.cli.operation.SQLOperation.prepare(Unknown
Source)
at org.apache.hive.service.cli.operation.SQLOperation.run(Unknown
Source)
at
org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(Unknown
Source)
at
org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(Unknown
Source)
at org.apache.hive.service.cli.CLIService.executeStatementAsync(Unknown
Source)
at
org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(Unknown
Source)
at
org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(Unknown
Source)
at
org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(Unknown
Source)
at org.apache.thrift.ProcessFunction.process(Unknown Source)
at org.apache.thrift.TBaseProcessor.process(Unknown Source)
at org.apache.hive.service.auth.TSetIpAddressProcessor.process(Unknown
Source)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(Unknown
Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
{noformat}
Diffs
-
beeline/src/java/org/apache/hive/beeline/BeeLine.java dcdd6f2
beeline/src/main/resources/BeeLine.properties 697c29a
jdbc/src/java/org/apache/hive/jdbc/Utils.java 87fec11
service/src/java/org/apache/hive/service/cli/HiveSQLException.java 8c9496e
service/src/test/org/apache/hive/service/cli/TestHiveSQLException.java
PRE-CREATION
shims/0.20S/src/main/java/org/apache/hadoop/hive/shims/Hadoop20SShims.java
e6493eb
Diff: https://reviews.apache.org/r/21922/diff/
Testing
---
Thanks,
Navis Ryu
[jira] [Updated] (HIVE-7127) Handover more details on exception in hiveserver2
[
https://issues.apache.org/jira/browse/HIVE-7127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-7127:
Status: Patch Available (was: Open)
Handover more details on exception in hiveserver2
-
Key: HIVE-7127
URL: https://issues.apache.org/jira/browse/HIVE-7127
Project: Hive
Issue Type: Improvement
Components: JDBC
Reporter: Navis
Assignee: Navis
Priority: Trivial
Attachments: HIVE-7127.1.patch.txt
NO_PRECOMMIT_TEST
Currently, JDBC hands over exception message and error codes. But it's not
helpful for debugging.
{noformat}
org.apache.hive.service.cli.HiveSQLException: Error while compiling
statement: FAILED: ParseException line 1:0 cannot recognize input near
'createa' 'asd' 'EOF'
at org.apache.hive.jdbc.Utils.verifySuccess(Utils.java:121)
at org.apache.hive.jdbc.Utils.verifySuccessWithInfo(Utils.java:109)
at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:231)
at org.apache.hive.beeline.Commands.execute(Commands.java:736)
at org.apache.hive.beeline.Commands.sql(Commands.java:657)
at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:889)
at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:744)
at
org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:459)
at org.apache.hive.beeline.BeeLine.main(BeeLine.java:442)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:160)
{noformat}
With this patch, JDBC client can get more details on hiveserver2.
{noformat}
Caused by: org.apache.hive.service.cli.HiveSQLException: Error while
compiling statement: FAILED: ParseException line 1:0 cannot recognize input
near 'createa' 'asd' 'EOF'
at org.apache.hive.service.cli.operation.SQLOperation.prepare(Unknown
Source)
at org.apache.hive.service.cli.operation.SQLOperation.run(Unknown
Source)
at
org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(Unknown
Source)
at
org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(Unknown
Source)
at org.apache.hive.service.cli.CLIService.executeStatementAsync(Unknown
Source)
at
org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(Unknown
Source)
at
org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(Unknown
Source)
at
org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(Unknown
Source)
at org.apache.thrift.ProcessFunction.process(Unknown Source)
at org.apache.thrift.TBaseProcessor.process(Unknown Source)
at org.apache.hive.service.auth.TSetIpAddressProcessor.process(Unknown
Source)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(Unknown
Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-7127) Handover more details on exception in hiveserver2
[
https://issues.apache.org/jira/browse/HIVE-7127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-7127:
Description:
NO_PRECOMMIT_TEST
Currently, JDBC hands over exception message and error codes. But it's not
helpful for debugging.
{noformat}
org.apache.hive.service.cli.HiveSQLException: Error while compiling statement:
FAILED: ParseException line 1:0 cannot recognize input near 'createa' 'asd'
'EOF'
at org.apache.hive.jdbc.Utils.verifySuccess(Utils.java:121)
at org.apache.hive.jdbc.Utils.verifySuccessWithInfo(Utils.java:109)
at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:231)
at org.apache.hive.beeline.Commands.execute(Commands.java:736)
at org.apache.hive.beeline.Commands.sql(Commands.java:657)
at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:889)
at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:744)
at
org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:459)
at org.apache.hive.beeline.BeeLine.main(BeeLine.java:442)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:160)
{noformat}
With this patch, JDBC client can get more details on hiveserver2.
{noformat}
Caused by: org.apache.hive.service.cli.HiveSQLException: Error while compiling
statement: FAILED: ParseException line 1:0 cannot recognize input near
'createa' 'asd' 'EOF'
at org.apache.hive.service.cli.operation.SQLOperation.prepare(Unknown
Source)
at org.apache.hive.service.cli.operation.SQLOperation.run(Unknown
Source)
at
org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(Unknown
Source)
at
org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(Unknown
Source)
at org.apache.hive.service.cli.CLIService.executeStatementAsync(Unknown
Source)
at
org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(Unknown
Source)
at
org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(Unknown
Source)
at
org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(Unknown
Source)
at org.apache.thrift.ProcessFunction.process(Unknown Source)
at org.apache.thrift.TBaseProcessor.process(Unknown Source)
at org.apache.hive.service.auth.TSetIpAddressProcessor.process(Unknown
Source)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(Unknown
Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
{noformat}
was:
Currently, JDBC hands over exception message and error codes. But it's not
helpful for debugging.
{noformat}
org.apache.hive.service.cli.HiveSQLException: Error while compiling statement:
FAILED: ParseException line 1:0 cannot recognize input near 'createa' 'asd'
'EOF'
at org.apache.hive.jdbc.Utils.verifySuccess(Utils.java:121)
at org.apache.hive.jdbc.Utils.verifySuccessWithInfo(Utils.java:109)
at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:231)
at org.apache.hive.beeline.Commands.execute(Commands.java:736)
at org.apache.hive.beeline.Commands.sql(Commands.java:657)
at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:889)
at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:744)
at
org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:459)
at org.apache.hive.beeline.BeeLine.main(BeeLine.java:442)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:160)
{noformat}
With this patch, JDBC client can get more details on hiveserver2.
{noformat}
Caused by: org.apache.hive.service.cli.HiveSQLException: Error while compiling
statement: FAILED: ParseException line 1:0 cannot recognize input near
'createa' 'asd' 'EOF'
at org.apache.hive.service.cli.operation.SQLOperation.prepare(Unknown
Source)
at org.apache.hive.service.cli.operation.SQLOperation.run(Unknown
Source)
at
org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(Unknown
Source)
at
[jira] [Updated] (HIVE-7127) Handover more details on exception in hiveserver2
[
https://issues.apache.org/jira/browse/HIVE-7127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-7127:
Attachment: HIVE-7127.1.patch.txt
Handover more details on exception in hiveserver2
-
Key: HIVE-7127
URL: https://issues.apache.org/jira/browse/HIVE-7127
Project: Hive
Issue Type: Improvement
Components: JDBC
Reporter: Navis
Assignee: Navis
Priority: Trivial
Attachments: HIVE-7127.1.patch.txt
NO_PRECOMMIT_TEST
Currently, JDBC hands over exception message and error codes. But it's not
helpful for debugging.
{noformat}
org.apache.hive.service.cli.HiveSQLException: Error while compiling
statement: FAILED: ParseException line 1:0 cannot recognize input near
'createa' 'asd' 'EOF'
at org.apache.hive.jdbc.Utils.verifySuccess(Utils.java:121)
at org.apache.hive.jdbc.Utils.verifySuccessWithInfo(Utils.java:109)
at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:231)
at org.apache.hive.beeline.Commands.execute(Commands.java:736)
at org.apache.hive.beeline.Commands.sql(Commands.java:657)
at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:889)
at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:744)
at
org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:459)
at org.apache.hive.beeline.BeeLine.main(BeeLine.java:442)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:160)
{noformat}
With this patch, JDBC client can get more details on hiveserver2.
{noformat}
Caused by: org.apache.hive.service.cli.HiveSQLException: Error while
compiling statement: FAILED: ParseException line 1:0 cannot recognize input
near 'createa' 'asd' 'EOF'
at org.apache.hive.service.cli.operation.SQLOperation.prepare(Unknown
Source)
at org.apache.hive.service.cli.operation.SQLOperation.run(Unknown
Source)
at
org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(Unknown
Source)
at
org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(Unknown
Source)
at org.apache.hive.service.cli.CLIService.executeStatementAsync(Unknown
Source)
at
org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(Unknown
Source)
at
org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(Unknown
Source)
at
org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(Unknown
Source)
at org.apache.thrift.ProcessFunction.process(Unknown Source)
at org.apache.thrift.TBaseProcessor.process(Unknown Source)
at org.apache.hive.service.auth.TSetIpAddressProcessor.process(Unknown
Source)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(Unknown
Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-5799) session/operation timeout for hiveserver2
[
https://issues.apache.org/jira/browse/HIVE-5799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14009485#comment-14009485
]
Hive QA commented on HIVE-5799:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12646742/HIVE-5799.9.patch.txt
{color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 5464 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_parquet_decimal1
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_ctas
org.apache.hive.hcatalog.pig.TestOrcHCatPigStorer.testWriteDecimal
org.apache.hive.hcatalog.pig.TestOrcHCatPigStorer.testWriteDecimalX
org.apache.hive.hcatalog.pig.TestOrcHCatPigStorer.testWriteDecimalXY
org.apache.hive.jdbc.miniHS2.TestHiveServer2.testConnection
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/306/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/306/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-Build-306/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 7 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12646742
session/operation timeout for hiveserver2
-
Key: HIVE-5799
URL: https://issues.apache.org/jira/browse/HIVE-5799
Project: Hive
Issue Type: Improvement
Components: HiveServer2
Reporter: Navis
Assignee: Navis
Priority: Minor
Attachments: HIVE-5799.1.patch.txt, HIVE-5799.2.patch.txt,
HIVE-5799.3.patch.txt, HIVE-5799.4.patch.txt, HIVE-5799.5.patch.txt,
HIVE-5799.6.patch.txt, HIVE-5799.7.patch.txt, HIVE-5799.8.patch.txt,
HIVE-5799.9.patch.txt
Need some timeout facility for preventing resource leakages from instable or
bad clients.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6438) Sort query result for test, removing order by clause
[
https://issues.apache.org/jira/browse/HIVE-6438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14009486#comment-14009486
]
Hive QA commented on HIVE-6438:
---
{color:red}Overall{color}: -1 no tests executed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12646744/HIVE-6438.6.patch.txt
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/307/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/307/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-Build-307/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hive-ptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ [[ -n /usr/java/jdk1.7.0_45-cloudera ]]
+ export JAVA_HOME=/usr/java/jdk1.7.0_45-cloudera
+ JAVA_HOME=/usr/java/jdk1.7.0_45-cloudera
+ export
PATH=/usr/java/jdk1.7.0_45-cloudera/bin/:/usr/java/jdk1.6.0_34/bin:/usr/local/apache-maven-3.0.5/bin:/usr/local/apache-maven-3.0.5/bin:/usr/java/jdk1.6.0_34/bin:/usr/local/apache-ant-1.9.1/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/hiveptest/bin
+
PATH=/usr/java/jdk1.7.0_45-cloudera/bin/:/usr/java/jdk1.6.0_34/bin:/usr/local/apache-maven-3.0.5/bin:/usr/local/apache-maven-3.0.5/bin:/usr/java/jdk1.6.0_34/bin:/usr/local/apache-ant-1.9.1/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/hiveptest/bin
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'M2_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ M2_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ cd /data/hive-ptest/working/
+ tee /data/hive-ptest/logs/PreCommit-HIVE-Build-307/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ svn = \s\v\n ]]
+ [[ -n '' ]]
+ [[ -d apache-svn-trunk-source ]]
+ [[ ! -d apache-svn-trunk-source/.svn ]]
+ [[ ! -d apache-svn-trunk-source ]]
+ cd apache-svn-trunk-source
+ svn revert -R .
Reverted 'conf/hive-default.xml.template'
Reverted 'common/src/java/org/apache/hadoop/hive/conf/HiveConf.java'
Reverted 'service/src/java/org/apache/hive/service/cli/OperationState.java'
Reverted 'service/src/java/org/apache/hive/service/cli/session/HiveSession.java'
Reverted
'service/src/java/org/apache/hive/service/cli/session/HiveSessionImpl.java'
Reverted
'service/src/java/org/apache/hive/service/cli/session/HiveSessionBase.java'
Reverted
'service/src/java/org/apache/hive/service/cli/session/SessionManager.java'
Reverted 'service/src/java/org/apache/hive/service/cli/operation/Operation.java'
Reverted
'service/src/java/org/apache/hive/service/cli/operation/OperationManager.java'
++ awk '{print $2}'
++ egrep -v '^X|^Performing status on external'
++ svn status --no-ignore
+ rm -rf target datanucleus.log ant/target shims/target shims/0.20/target
shims/0.20S/target shims/0.23/target shims/aggregator/target
shims/common/target shims/common-secure/target packaging/target
hbase-handler/target testutils/target jdbc/target metastore/target
itests/target itests/hcatalog-unit/target itests/test-serde/target
itests/qtest/target itests/hive-minikdc/target itests/hive-unit/target
itests/custom-serde/target itests/util/target hcatalog/target
hcatalog/core/target hcatalog/streaming/target
hcatalog/server-extensions/target hcatalog/hcatalog-pig-adapter/target
hcatalog/webhcat/svr/target hcatalog/webhcat/java-client/target hwi/target
common/target common/src/gen service/target contrib/target serde/target
beeline/target odbc/target cli/target ql/dependency-reduced-pom.xml ql/target
+ svn update
U
contrib/src/java/org/apache/hadoop/hive/contrib/util/typedbytes/TypedBytesWritableOutput.java
U
contrib/src/java/org/apache/hadoop/hive/contrib/util/typedbytes/TypedBytesRecordInput.java
U
contrib/src/java/org/apache/hadoop/hive/contrib/util/typedbytes/TypedBytesInput.java
U
contrib/src/java/org/apache/hadoop/hive/contrib/util/typedbytes/TypedBytesRecordOutput.java
U
contrib/src/java/org/apache/hadoop/hive/contrib/util/typedbytes/TypedBytesOutput.java
Fetching external item into 'hcatalog/src/test/e2e/harness'
Updated external to revision 1597716.
Updated to revision 1597716.
+ patchCommandPath=/data/hive-ptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hive-ptest/working/scratch/build.patch
+ [[ -f /data/hive-ptest/working/scratch/build.patch ]]
+ chmod +x /data/hive-ptest/working/scratch/smart-apply-patch.sh
+ /data/hive-ptest/working/scratch/smart-apply-patch.sh
/data/hive-ptest/working/scratch/build.patch
The patch does not appear to apply with p0, p1, or p2
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT
[jira] [Commented] (HIVE-7113) Support fully managed partitioned table, which does not allow partitions to have different properties with table
[
https://issues.apache.org/jira/browse/HIVE-7113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14009487#comment-14009487
]
Hive QA commented on HIVE-7113:
---
{color:red}Overall{color}: -1 no tests executed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12646753/HIVE-7113.2.patch.txt
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/308/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/308/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-Build-308/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hive-ptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ [[ -n /usr/java/jdk1.7.0_45-cloudera ]]
+ export JAVA_HOME=/usr/java/jdk1.7.0_45-cloudera
+ JAVA_HOME=/usr/java/jdk1.7.0_45-cloudera
+ export
PATH=/usr/java/jdk1.7.0_45-cloudera/bin/:/usr/java/jdk1.6.0_34/bin:/usr/local/apache-maven-3.0.5/bin:/usr/local/apache-maven-3.0.5/bin:/usr/java/jdk1.6.0_34/bin:/usr/local/apache-ant-1.9.1/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/hiveptest/bin
+
PATH=/usr/java/jdk1.7.0_45-cloudera/bin/:/usr/java/jdk1.6.0_34/bin:/usr/local/apache-maven-3.0.5/bin:/usr/local/apache-maven-3.0.5/bin:/usr/java/jdk1.6.0_34/bin:/usr/local/apache-ant-1.9.1/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/hiveptest/bin
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'M2_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ M2_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ cd /data/hive-ptest/working/
+ tee /data/hive-ptest/logs/PreCommit-HIVE-Build-308/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ svn = \s\v\n ]]
+ [[ -n '' ]]
+ [[ -d apache-svn-trunk-source ]]
+ [[ ! -d apache-svn-trunk-source/.svn ]]
+ [[ ! -d apache-svn-trunk-source ]]
+ cd apache-svn-trunk-source
+ svn revert -R .
++ awk '{print $2}'
++ egrep -v '^X|^Performing status on external'
++ svn status --no-ignore
+ rm -rf
+ svn update
Fetching external item into 'hcatalog/src/test/e2e/harness'
External at revision 1597716.
At revision 1597716.
+ patchCommandPath=/data/hive-ptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hive-ptest/working/scratch/build.patch
+ [[ -f /data/hive-ptest/working/scratch/build.patch ]]
+ chmod +x /data/hive-ptest/working/scratch/smart-apply-patch.sh
+ /data/hive-ptest/working/scratch/smart-apply-patch.sh
/data/hive-ptest/working/scratch/build.patch
The patch does not appear to apply with p0, p1, or p2
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12646753
Support fully managed partitioned table, which does not allow partitions to
have different properties with table
Key: HIVE-7113
URL: https://issues.apache.org/jira/browse/HIVE-7113
Project: Hive
Issue Type: New Feature
Components: Query Processor
Reporter: Navis
Assignee: Navis
Priority: Minor
Attachments: HIVE-7113.1.patch.txt, HIVE-7113.2.patch.txt
Mostly, partitioned table has the same storage description with mother table
just except data location. If partitions can be forced not to have different
schema with table by configuration, the time for retrieving partition objects
from metastore and for creating serde for each partition could be nullified.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-5469) support nullif
[ https://issues.apache.org/jira/browse/HIVE-5469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-5469: Attachment: HIVE-5469.1.patch.txt support nullif -- Key: HIVE-5469 URL: https://issues.apache.org/jira/browse/HIVE-5469 Project: Hive Issue Type: Improvement Affects Versions: 0.11.0 Reporter: N Campbell Priority: Minor Attachments: HIVE-5469.1.patch.txt Have to express case expression to work around lack of NULLIF select nullif(cint, 1) from tint select cint, case when cint = 1 then null else cint end from tint -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-5469) support nullif
[ https://issues.apache.org/jira/browse/HIVE-5469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-5469: Assignee: Navis Status: Patch Available (was: Open) Migrating mssql to hive, met huge use case of nullif. support nullif -- Key: HIVE-5469 URL: https://issues.apache.org/jira/browse/HIVE-5469 Project: Hive Issue Type: Improvement Affects Versions: 0.11.0 Reporter: N Campbell Assignee: Navis Priority: Minor Attachments: HIVE-5469.1.patch.txt Have to express case expression to work around lack of NULLIF select nullif(cint, 1) from tint select cint, case when cint = 1 then null else cint end from tint -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-4317) StackOverflowError when add jar concurrently
[ https://issues.apache.org/jira/browse/HIVE-4317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14009499#comment-14009499 ] dima machlin commented on HIVE-4317: I can confirm that this also happens in Hive 0.12 and is fully reproducible. StackOverflowError when add jar concurrently - Key: HIVE-4317 URL: https://issues.apache.org/jira/browse/HIVE-4317 Project: Hive Issue Type: Bug Affects Versions: 0.9.0, 0.10.0 Reporter: wangwenli Attachments: hive-4317.1.patch scenario: multiple thread add jar and do select operation by jdbc concurrently , when hiveserver serializeMapRedWork sometimes, it will throw StackOverflowError from XMLEncoder. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Resolved] (HIVE-5980) MapJoin does not take effect
[ https://issues.apache.org/jira/browse/HIVE-5980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis resolved HIVE-5980. - Resolution: Not a Problem Release Note: From hive-0.12.0, hive.ignore.mapjoin.hint=true by default. MapJoin does not take effect Key: HIVE-5980 URL: https://issues.apache.org/jira/browse/HIVE-5980 Project: Hive Issue Type: Bug Components: CLI Affects Versions: 0.12.0 Reporter: ruish li Priority: Trivial Labels: patch step1: hive-site.xml property namehive.auto.convert.join/name valuetrue/value /property step2: set the parameter in console: cli: set hive.auto.convert.join=false; step3:execute sql, mapjoin Does not take effect select /*+ mapjoin(t2)*/ * from t1 join t2 on t1.f=t2.f; step4: explain sql, we can see that the join does not conver to mapjoin select /*+ mapjoin(t2)*/ * from t1 join t2 on t1.f=t2.f; -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-5980) MapJoin does not take effect
[ https://issues.apache.org/jira/browse/HIVE-5980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-5980: Release Note: From hive-0.11.0, hive.ignore.mapjoin.hint=true by default. (was: From hive-0.12.0, hive.ignore.mapjoin.hint=true by default.) MapJoin does not take effect Key: HIVE-5980 URL: https://issues.apache.org/jira/browse/HIVE-5980 Project: Hive Issue Type: Bug Components: CLI Affects Versions: 0.12.0 Reporter: ruish li Priority: Trivial Labels: patch step1: hive-site.xml property namehive.auto.convert.join/name valuetrue/value /property step2: set the parameter in console: cli: set hive.auto.convert.join=false; step3:execute sql, mapjoin Does not take effect select /*+ mapjoin(t2)*/ * from t1 join t2 on t1.f=t2.f; step4: explain sql, we can see that the join does not conver to mapjoin select /*+ mapjoin(t2)*/ * from t1 join t2 on t1.f=t2.f; -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-2627) NPE on MAP-JOIN with a UDF in an external JAR
[ https://issues.apache.org/jira/browse/HIVE-2627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14009503#comment-14009503 ] dima machlin commented on HIVE-2627: I can confirm that this still happens in hive 0.12. Getting : java.lang.ClassNotFoundException: com.some.class.used.by.UDF Continuing ... java.lang.NullPointerException: target should not be null java.lang.NullPointerException: target should not be null Continuing ... and eventually ERROR mr.MapredLocalTask: Hive Runtime Error: Map local work failed java.lang.NullPointerException at org.apache.hadoop.hive.ql.exec.FunctionRegistry.isStateful(FunctionRegistry.java:1415) at org.apache.hadoop.hive.ql.exec.FunctionRegistry.isDeterministic(FunctionRegistry.java:1385) at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.isDeterministic(ExprNodeGenericFuncEvaluator.java:132) at org.apache.hadoop.hive.ql.exec.FunctionRegistry.isDeterministic(FunctionRegistry.java:1385) at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.isDeterministic(ExprNodeGenericFuncEvaluator.java:132) at org.apache.hadoop.hive.ql.exec.ExprNodeEvaluatorFactory.iterate(ExprNodeEvaluatorFactory.java:83) at org.apache.hadoop.hive.ql.exec.ExprNodeEvaluatorFactory.iterate(ExprNodeEvaluatorFactory.java:83) at org.apache.hadoop.hive.ql.exec.ExprNodeEvaluatorFactory.toCachedEval(ExprNodeEvaluatorFactory.java:73) at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:57) at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:57) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:377)at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:377) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:453)at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:453) at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:409) at org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:188) at org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:188) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:377) at org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask.initializeOperators(MapredLocalTask.java:419) at org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask.executeFromChildJVM(MapredLocalTask.java:305) at org.apache.hadoop.hive.ql.exec.mr.ExecDriver.main(ExecDriver.java:722) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.util.RunJar.main(RunJar.java:197) NPE on MAP-JOIN with a UDF in an external JAR - Key: HIVE-2627 URL: https://issues.apache.org/jira/browse/HIVE-2627 Project: Hive Issue Type: Bug Reporter: Jonathan Chang When a query is converted into a map join, and it depends on some UDF (ADD JAR...; CREATE TEMPORARY FUNCTION...), then an NPE may happen. Here is an example. SELECT some_udf(dummy1) as dummies FROM ( SELECT a.dummy as dummy1, b.dummy as dummy2 FROM test a LEFT OUTER JOIN test b ON a.dummy = b.dummy ) c; My guess is that the JAR classes are not getting propagated to the hashmapjoin operator. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-5469) support nullif
[ https://issues.apache.org/jira/browse/HIVE-5469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-5469: Attachment: HIVE-5469.2.patch.txt support nullif -- Key: HIVE-5469 URL: https://issues.apache.org/jira/browse/HIVE-5469 Project: Hive Issue Type: Improvement Affects Versions: 0.11.0 Reporter: N Campbell Assignee: Navis Priority: Minor Attachments: HIVE-5469.1.patch.txt, HIVE-5469.2.patch.txt Have to express case expression to work around lack of NULLIF select nullif(cint, 1) from tint select cint, case when cint = 1 then null else cint end from tint -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6316) Document support for new types in HCat
[ https://issues.apache.org/jira/browse/HIVE-6316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14009548#comment-14009548 ] Lefty Leverenz commented on HIVE-6316: -- [~ekoifman], here's a first pass ready for review: * [HCatLoader Data Types | https://cwiki.apache.org/confluence/display/Hive/HCatalog+LoadStore#HCatalogLoadStore-HCatLoaderDataTypes] * [HCatStorer Data Types | https://cwiki.apache.org/confluence/display/Hive/HCatalog+LoadStore#HCatalogLoadStore-HCatStorerDataTypes] More information about onOutOfRangeValue for HCatStorer might be good. Questions: * Are Null and Throw the only possible values? * Where/how does it get specified? * Should hcat.pig.store.onoutofrangevalue be documented in https://cwiki.apache.org/confluence/display/Hive/HCatalog+Config+Properties? * If so, does it belong in the Data Promotion Behaviour section? You also said https://cwiki.apache.org/confluence/display/Hive/HCatalog+InputOutput is out of date. What needs to be changed there? Document support for new types in HCat -- Key: HIVE-6316 URL: https://issues.apache.org/jira/browse/HIVE-6316 Project: Hive Issue Type: Sub-task Components: Documentation, HCatalog Affects Versions: 0.13.0 Reporter: Eugene Koifman Assignee: Lefty Leverenz HIVE-5814 added support for new types in HCat. The PDF file in that bug explains exactly how these map to Pig types. This should be added to the Wiki somewhere (probably here https://cwiki.apache.org/confluence/display/Hive/HCatalog+LoadStore). In particular it should be highlighted that copying data from Hive TIMESTAMP to Pig DATETIME, any 'nanos' in the timestamp will be lost. Also, HCatStorer now takes new parameter which is described in the PDF doc. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-3925) dependencies of fetch task are not shown by explain
[
https://issues.apache.org/jira/browse/HIVE-3925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14009614#comment-14009614
]
Hive QA commented on HIVE-3925:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12646831/HIVE-3925.7.patch.txt
{color:red}ERROR:{color} -1 due to 19 failed/errored test(s), 5464 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_groupby
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join_without_localtask
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_sort_1_23
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_sort_skew_1_23
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_parquet_decimal1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_plan_json
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats19
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subq_where_serialization
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subquery_in_having
org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_pushdown
org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_ppd_key_ranges
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_ctas
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_limit_partition_stats
org.apache.hadoop.hive.common.metrics.TestMetrics.testScopeConcurrency
org.apache.hive.hcatalog.pig.TestOrcHCatPigStorer.testWriteDecimal
org.apache.hive.hcatalog.pig.TestOrcHCatPigStorer.testWriteDecimalX
org.apache.hive.hcatalog.pig.TestOrcHCatPigStorer.testWriteDecimalXY
org.apache.hive.jdbc.miniHS2.TestHiveServer2.testConnection
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/309/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/309/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-Build-309/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 19 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12646831
dependencies of fetch task are not shown by explain
---
Key: HIVE-3925
URL: https://issues.apache.org/jira/browse/HIVE-3925
Project: Hive
Issue Type: Bug
Components: Query Processor
Reporter: Namit Jain
Assignee: Navis
Attachments: HIVE-3925.4.patch.txt, HIVE-3925.5.patch.txt,
HIVE-3925.6.patch.txt, HIVE-3925.7.patch.txt, HIVE-3925.D8577.1.patch,
HIVE-3925.D8577.2.patch, HIVE-3925.D8577.3.patch
A simple query like:
hive explain select * from src order by key;
OK
ABSTRACT SYNTAX TREE:
(TOK_QUERY (TOK_FROM (TOK_TABREF (TOK_TABNAME src))) (TOK_INSERT
(TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR
TOK_ALLCOLREF)) (TOK_ORDERBY (TOK_TABSORTCOLNAMEASC (TOK_TABLE_OR_COL key)
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 is a root stage
Stage: Stage-0
Fetch Operator
limit: -1
Stage-0 is not a root stage and depends on stage-1.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-600) Running TPC-H queries on Hive
[ https://issues.apache.org/jira/browse/HIVE-600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14009633#comment-14009633 ] xuanjinlee commented on HIVE-600: - Hi all: Which database I choose to generate data ? SQL SEVER ?DB2 or ORACLE? or change the default setting of config? The README of HIVE-600 does not tell me which database should I set to, Running TPC-H queries on Hive - Key: HIVE-600 URL: https://issues.apache.org/jira/browse/HIVE-600 Project: Hive Issue Type: New Feature Reporter: Yuntao Jia Assignee: Yuntao Jia Attachments: TPC-H_on_Hive_2009-08-11.pdf, TPC-H_on_Hive_2009-08-11.tar.gz, TPC-H_on_Hive_2009-08-14.tar.gz The goal is to run all TPC-H (http://www.tpc.org/tpch/) benchmark queries on Hive for two reasons. First, through those queries, we would like to find the new features that we need to put into Hive so that Hive supports common SQL queries. Second, we would like to measure the performance of Hive to find out what Hive is not good at. We can then improve Hive based on those information. For queries that are not supported now in Hive, I will try to rewrite them to one or more Hive-supported queries. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6189) Support top level union all statements
[ https://issues.apache.org/jira/browse/HIVE-6189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14009674#comment-14009674 ] Lefty Leverenz commented on HIVE-6189: -- [~hagleitn], how's this wiki revision? * [Language Manual - Union | https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Union] * [diffs | https://cwiki.apache.org/confluence/pages/diffpages.action?pageId=27362049originalId=41812730] Support top level union all statements -- Key: HIVE-6189 URL: https://issues.apache.org/jira/browse/HIVE-6189 Project: Hive Issue Type: Bug Reporter: Gunther Hagleitner Assignee: Gunther Hagleitner Fix For: 0.13.0 Attachments: HIVE-6189.1.patch, HIVE-6189.2.patch, HIVE-6189.3.patch I've always wondered why union all has to be in subqueries in hive. After looking at it, problems are: - Hive Parser: - Union happens at the wrong place (insert ... select ... union all select ...) is parsed as (insert select) union select. - There are many rewrite rules in the parser to force any query into the a from - insert -select form. No doubt for historical reasons. - Plan generation/semantic analysis assumes top level TOK_QUERY and not top level TOK_UNION. The rewrite rules don't work when we move the UNION ALL recursion into the select statements. However, it's not hard to do that in code. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6936) Provide table properties to InputFormats
[ https://issues.apache.org/jira/browse/HIVE-6936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley updated HIVE-6936: Attachment: HIVE-6936.patch Reattaching for jenkins. Provide table properties to InputFormats Key: HIVE-6936 URL: https://issues.apache.org/jira/browse/HIVE-6936 Project: Hive Issue Type: Bug Components: File Formats Reporter: Owen O'Malley Assignee: Owen O'Malley Fix For: 0.14.0 Attachments: HIVE-6936.patch, HIVE-6936.patch, HIVE-6936.patch, HIVE-6936.patch, HIVE-6936.patch, HIVE-6936.patch, HIVE-6936.patch, HIVE-6936.patch Some advanced file formats need the table properties made available to them. Additionally, it would be convenient to provide a unique id for fetch operators and the complete list of directories. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HIVE-7128) Add direct support for creating and managing salted hbase tables
Swarnim Kulkarni created HIVE-7128: -- Summary: Add direct support for creating and managing salted hbase tables Key: HIVE-7128 URL: https://issues.apache.org/jira/browse/HIVE-7128 Project: Hive Issue Type: New Feature Components: HBase Handler Affects Versions: 0.13.0 Reporter: Swarnim Kulkarni Assignee: Swarnim Kulkarni Salting is a very important technique in order to avoid hot-spotting in hbase. It will be very beneficial if with current hbase integration we can provide a direct support for salting. More information on salting can be found here[1] [1] http://blog.sematext.com/2012/04/09/hbasewd-avoid-regionserver-hotspotting-despite-writing-records-with-sequential-keys/ -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-7123) Follow-up of HIVE-6367
[
https://issues.apache.org/jira/browse/HIVE-7123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14009841#comment-14009841
]
Hive QA commented on HIVE-7123:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12646789/HIVE-7123.1.patch
{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 5465 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_ctas
org.apache.hive.hcatalog.pig.TestOrcHCatPigStorer.testWriteDecimal
org.apache.hive.hcatalog.pig.TestOrcHCatPigStorer.testWriteDecimalX
org.apache.hive.hcatalog.pig.TestOrcHCatPigStorer.testWriteDecimalXY
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/310/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/310/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-Build-310/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12646789
Follow-up of HIVE-6367
--
Key: HIVE-7123
URL: https://issues.apache.org/jira/browse/HIVE-7123
Project: Hive
Issue Type: Bug
Components: Serializers/Deserializers
Affects Versions: 0.14.0
Reporter: Xuefu Zhang
Assignee: Xuefu Zhang
Attachments: HIVE-7123.1.patch, HIVE-7123.patch
HIVE-6367 provides initial decimal support in Parquet serde. The are a few
minor items left over:
1. parquet_decimal.q seems failing
2. will use fixed length binary to encode decimal instead of variable length
binary.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-7123) Follow-up of HIVE-6367
[ https://issues.apache.org/jira/browse/HIVE-7123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14009899#comment-14009899 ] Xuefu Zhang commented on HIVE-7123: --- The above test failures are unrelated. Patch is ready for review. [~brocknoland] Would you mind doing so? Follow-up of HIVE-6367 -- Key: HIVE-7123 URL: https://issues.apache.org/jira/browse/HIVE-7123 Project: Hive Issue Type: Bug Components: Serializers/Deserializers Affects Versions: 0.14.0 Reporter: Xuefu Zhang Assignee: Xuefu Zhang Attachments: HIVE-7123.1.patch, HIVE-7123.patch HIVE-6367 provides initial decimal support in Parquet serde. The are a few minor items left over: 1. parquet_decimal.q seems failing 2. will use fixed length binary to encode decimal instead of variable length binary. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-1643) support range scans and non-key columns in HBase filter pushdown
[
https://issues.apache.org/jira/browse/HIVE-1643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14009938#comment-14009938
]
Craig Condit commented on HIVE-1643:
The patch as-is has a few issues...
First, at least in Hive 0.12, it interacts badly when multiple tables are
joined. I've seen cases where it was clear that Hive was attempting to push
down predicates for the wrong table, leading to NullPointerExceptions when the
column is looked up and not found since the HBase storage handler assumes that
any predicate that it receives will be for a valid column. I suspect this must
be a bug in the query optimizer, but have not been able to determine exactly
where.
Second, the fallback behavior when a complex query predicate is passed down is
to punt on the entire expression, even if it could be partially evaluated (for
example rowkey = 'A' AND rowkey 'B' AND ([complex bit])). This leads to
unexpected full table scans in HBase. At the very least, the code should try
really hard to at least handle the rowkey parts if possible. This can happen
unexpectedly, if a single term uses an operator that the storage handler does
not have a case for.
Third, even if the predicate pushdown works, this often results in secondary
issues when interacting with HBase. In a case where no rowkey expression
exists, it's possible to run very high CPU usage on HBase to evaluate the
filters, and even get HBase RPC timeouts if enough rows are filtered out to
result in no data being returned quickly enough. It would be nice to be able to
control (somehow) which expressions the code tries to push down.
At our location, we didn't even try to port the patch to Hive 0.13 when we
upgraded, mainly due to issues #2 and #3. Fortunately, CTEs have allowed us to
ensure that only rowkey predicates get pushed down like so:
{noformat}
with a as (select ... from hbase_table where rowkey = 'start' and rowkey
'end') do select * from a where ...;
{noformat}
It might be more useful for Hive-HBase integration to focus on ensuring that
rowkey predicates are always pushed down (except for things like OR/NOT
expressions, etc.) rather than trying to push down other types of expressions.
support range scans and non-key columns in HBase filter pushdown
Key: HIVE-1643
URL: https://issues.apache.org/jira/browse/HIVE-1643
Project: Hive
Issue Type: Improvement
Components: HBase Handler
Affects Versions: 0.9.0
Reporter: John Sichi
Assignee: bharath v
Labels: patch
Attachments: HIVE-1643.patch, Hive-1643.2.patch, hbase_handler.patch
HIVE-1226 added support for WHERE rowkey=3. We would like to support WHERE
rowkey BETWEEN 10 and 20, as well as predicates on non-rowkeys (plus
conjunctions etc). Non-rowkey conditions can't be used to filter out entire
ranges, but they can be used to push the per-row filter processing as far
down as possible.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-7123) Follow-up of HIVE-6367
[ https://issues.apache.org/jira/browse/HIVE-7123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuefu Zhang updated HIVE-7123: -- Attachment: HIVE-7123.2.patch Follow-up of HIVE-6367 -- Key: HIVE-7123 URL: https://issues.apache.org/jira/browse/HIVE-7123 Project: Hive Issue Type: Bug Components: Serializers/Deserializers Affects Versions: 0.14.0 Reporter: Xuefu Zhang Assignee: Xuefu Zhang Attachments: HIVE-7123.1.patch, HIVE-7123.2.patch, HIVE-7123.patch HIVE-6367 provides initial decimal support in Parquet serde. The are a few minor items left over: 1. parquet_decimal.q seems failing 2. will use fixed length binary to encode decimal instead of variable length binary. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-7123) Follow-up of HIVE-6367
[ https://issues.apache.org/jira/browse/HIVE-7123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14009947#comment-14009947 ] Xuefu Zhang commented on HIVE-7123: --- Patch #2 slightly changed the way that the number of bytes needed to hold a decimal value of a given precision. Follow-up of HIVE-6367 -- Key: HIVE-7123 URL: https://issues.apache.org/jira/browse/HIVE-7123 Project: Hive Issue Type: Bug Components: Serializers/Deserializers Affects Versions: 0.14.0 Reporter: Xuefu Zhang Assignee: Xuefu Zhang Attachments: HIVE-7123.1.patch, HIVE-7123.2.patch, HIVE-7123.patch HIVE-6367 provides initial decimal support in Parquet serde. The are a few minor items left over: 1. parquet_decimal.q seems failing 2. will use fixed length binary to encode decimal instead of variable length binary. -- This message was sent by Atlassian JIRA (v6.2#6252)
Review Request 21933: HIVE-7123: Follow-up of HIVE-6367
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/21933/ --- Review request for hive. Bugs: HIVE-7123 https://issues.apache.org/jira/browse/HIVE-7123 Repository: hive-git Description --- Please refer to HIVE-7123. Diffs - common/src/test/org/apache/hadoop/hive/common/type/TestHiveDecimal.java 3e666e5 data/files/dec_comp.txt PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/io/parquet/convert/HiveSchemaConverter.java 1243585 ql/src/java/org/apache/hadoop/hive/ql/io/parquet/serde/ParquetHiveSerDe.java 6b23fbe ql/src/test/org/apache/hadoop/hive/ql/io/parquet/TestHiveSchemaConverter.java ff604ab Diff: https://reviews.apache.org/r/21933/diff/ Testing --- Tests are updated to reflect the changes. New tests are added also. Thanks, Xuefu Zhang
Re: Review Request 21933: HIVE-7123: Follow-up of HIVE-6367
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/21933/ --- (Updated May 27, 2014, 5:54 p.m.) Review request for hive. Bugs: HIVE-7123 https://issues.apache.org/jira/browse/HIVE-7123 Repository: hive-git Description --- Please refer to HIVE-7123. Diffs (updated) - common/src/test/org/apache/hadoop/hive/common/type/TestHiveDecimal.java 3e666e5 data/files/dec_comp.txt PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/io/parquet/convert/HiveSchemaConverter.java 1243585 ql/src/java/org/apache/hadoop/hive/ql/io/parquet/serde/ParquetHiveSerDe.java 6b23fbe ql/src/test/org/apache/hadoop/hive/ql/io/parquet/TestHiveSchemaConverter.java ff604ab Diff: https://reviews.apache.org/r/21933/diff/ Testing --- Tests are updated to reflect the changes. New tests are added also. Thanks, Xuefu Zhang
[jira] [Updated] (HIVE-7123) Follow-up of HIVE-6367
[ https://issues.apache.org/jira/browse/HIVE-7123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuefu Zhang updated HIVE-7123: -- Attachment: HIVE-7123.2.patch Follow-up of HIVE-6367 -- Key: HIVE-7123 URL: https://issues.apache.org/jira/browse/HIVE-7123 Project: Hive Issue Type: Bug Components: Serializers/Deserializers Affects Versions: 0.14.0 Reporter: Xuefu Zhang Assignee: Xuefu Zhang Attachments: HIVE-7123.1.patch, HIVE-7123.2.patch, HIVE-7123.patch HIVE-6367 provides initial decimal support in Parquet serde. The are a few minor items left over: 1. parquet_decimal.q seems failing 2. will use fixed length binary to encode decimal instead of variable length binary. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-7123) Follow-up of HIVE-6367
[ https://issues.apache.org/jira/browse/HIVE-7123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuefu Zhang updated HIVE-7123: -- Attachment: (was: HIVE-7123.2.patch) Follow-up of HIVE-6367 -- Key: HIVE-7123 URL: https://issues.apache.org/jira/browse/HIVE-7123 Project: Hive Issue Type: Bug Components: Serializers/Deserializers Affects Versions: 0.14.0 Reporter: Xuefu Zhang Assignee: Xuefu Zhang Attachments: HIVE-7123.1.patch, HIVE-7123.2.patch, HIVE-7123.patch HIVE-6367 provides initial decimal support in Parquet serde. The are a few minor items left over: 1. parquet_decimal.q seems failing 2. will use fixed length binary to encode decimal instead of variable length binary. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-7089) StorageBasedAuthorizationProvider fails to allow non-admin users to create databases in writable directories
[
https://issues.apache.org/jira/browse/HIVE-7089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14010026#comment-14010026
]
Craig Condit commented on HIVE-7089:
Created patch which does what the previous comment described. Building now.
StorageBasedAuthorizationProvider fails to allow non-admin users to create
databases in writable directories
Key: HIVE-7089
URL: https://issues.apache.org/jira/browse/HIVE-7089
Project: Hive
Issue Type: Bug
Affects Versions: 0.13.0
Reporter: Craig Condit
When attempting to create a database with a custom location and using
hive.security.authorizationmanager=org.apache.hadoop.hive.ql.security.authorization.StorageBasedAuthorizationProvider,
an AccessControlException is generated for the default warehouse location,
not the location which was given in the create database command.
{noformat}
hive create database test LOCATION '/user/ccondit/test';
Authorization failed:java.security.AccessControlException: action WRITE not
permitted on path hdfs://example.com:8020/apps/hive/warehouse for user
ccondit. Use SHOW GRANT to get more details.
14/05/19 09:50:59 ERROR ql.Driver: Authorization
failed:java.security.AccessControlException: action WRITE not permitted on
path hdfs://example.com:8020/apps/hive/warehouse for user ccondit. Use SHOW
GRANT to get more details.
{noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-3925) dependencies of fetch task are not shown by explain
[ https://issues.apache.org/jira/browse/HIVE-3925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-3925: --- Resolution: Fixed Fix Version/s: 0.14.0 Status: Resolved (was: Patch Available) Committed to trunk. Thanks, Navis! dependencies of fetch task are not shown by explain --- Key: HIVE-3925 URL: https://issues.apache.org/jira/browse/HIVE-3925 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Namit Jain Assignee: Navis Fix For: 0.14.0 Attachments: HIVE-3925.4.patch.txt, HIVE-3925.5.patch.txt, HIVE-3925.6.patch.txt, HIVE-3925.7.patch.txt, HIVE-3925.D8577.1.patch, HIVE-3925.D8577.2.patch, HIVE-3925.D8577.3.patch A simple query like: hive explain select * from src order by key; OK ABSTRACT SYNTAX TREE: (TOK_QUERY (TOK_FROM (TOK_TABREF (TOK_TABNAME src))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR TOK_ALLCOLREF)) (TOK_ORDERBY (TOK_TABSORTCOLNAMEASC (TOK_TABLE_OR_COL key) STAGE DEPENDENCIES: Stage-1 is a root stage Stage-0 is a root stage Stage: Stage-0 Fetch Operator limit: -1 Stage-0 is not a root stage and depends on stage-1. -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: Review Request 21846: HIVE-7117: Partitions not inheriting table permissions after alter rename partition
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/21846/ --- (Updated May 27, 2014, 6:38 p.m.) Review request for hive and Szehon Ho. Summary (updated) - HIVE-7117: Partitions not inheriting table permissions after alter rename partition Bugs: HIVE-7117 https://issues.apache.org/jira/browse/HIVE-7117 Repository: hive-git Description --- Partitions not inheriting table permissions after alter rename partition Avoid using same table in two different tests Diffs - common/src/java/org/apache/hadoop/hive/common/FileUtils.java ee61350ab1d32db96a234f6444836f76e23f3251 itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestFolderPermissions.java 4f566d2a2999eecea7018836505208ae80662e97 metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java 221b01043b3520211643847c00b17b328ad2b9c0 metastore/src/java/org/apache/hadoop/hive/metastore/Warehouse.java c1790b427e7fa64780fdf0a580f28135d6a4ff8d Diff: https://reviews.apache.org/r/21846/diff/ Testing --- Tested with itests. Thanks, Ashish Singh
Re: Review Request 21846: Avoid using same table in two different tests
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/21846/ --- (Updated May 27, 2014, 6:38 p.m.) Review request for hive and Szehon Ho. Changes --- Avoid using same table in two different tests Summary (updated) - Avoid using same table in two different tests Bugs: HIVE-7117 https://issues.apache.org/jira/browse/HIVE-7117 Repository: hive-git Description (updated) --- Partitions not inheriting table permissions after alter rename partition Avoid using same table in two different tests Diffs (updated) - common/src/java/org/apache/hadoop/hive/common/FileUtils.java ee61350ab1d32db96a234f6444836f76e23f3251 itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/security/TestFolderPermissions.java 4f566d2a2999eecea7018836505208ae80662e97 metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java 221b01043b3520211643847c00b17b328ad2b9c0 metastore/src/java/org/apache/hadoop/hive/metastore/Warehouse.java c1790b427e7fa64780fdf0a580f28135d6a4ff8d Diff: https://reviews.apache.org/r/21846/diff/ Testing --- Tested with itests. Thanks, Ashish Singh
[jira] [Updated] (HIVE-7089) StorageBasedAuthorizationProvider fails to allow non-admin users to create databases in writable directories
[
https://issues.apache.org/jira/browse/HIVE-7089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Craig Condit updated HIVE-7089:
---
Attachment: HIVE-7089.patch
StorageBasedAuthorizationProvider fails to allow non-admin users to create
databases in writable directories
Key: HIVE-7089
URL: https://issues.apache.org/jira/browse/HIVE-7089
Project: Hive
Issue Type: Bug
Affects Versions: 0.13.0
Reporter: Craig Condit
Attachments: HIVE-7089.patch
When attempting to create a database with a custom location and using
hive.security.authorizationmanager=org.apache.hadoop.hive.ql.security.authorization.StorageBasedAuthorizationProvider,
an AccessControlException is generated for the default warehouse location,
not the location which was given in the create database command.
{noformat}
hive create database test LOCATION '/user/ccondit/test';
Authorization failed:java.security.AccessControlException: action WRITE not
permitted on path hdfs://example.com:8020/apps/hive/warehouse for user
ccondit. Use SHOW GRANT to get more details.
14/05/19 09:50:59 ERROR ql.Driver: Authorization
failed:java.security.AccessControlException: action WRITE not permitted on
path hdfs://example.com:8020/apps/hive/warehouse for user ccondit. Use SHOW
GRANT to get more details.
{noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-7117) Partitions not inheriting table permissions after alter rename partition
[ https://issues.apache.org/jira/browse/HIVE-7117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Kumar Singh updated HIVE-7117: - Attachment: HIVE-7117.5.patch Avoid using same table in two different tests, which caused test error because of previous patch. Partitions not inheriting table permissions after alter rename partition Key: HIVE-7117 URL: https://issues.apache.org/jira/browse/HIVE-7117 Project: Hive Issue Type: Bug Components: Security Reporter: Ashish Kumar Singh Assignee: Ashish Kumar Singh Attachments: HIVE-7117.2.patch, HIVE-7117.3.patch, HIVE-7117.4.patch, HIVE-7117.5.patch, HIVE-7117.patch On altering/renaming a partition it must inherit permission of the parent directory, if the flag hive.warehouse.subdir.inherit.perms is set. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-7117) Partitions not inheriting table permissions after alter rename partition
[ https://issues.apache.org/jira/browse/HIVE-7117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Kumar Singh updated HIVE-7117: - Status: Patch Available (was: Open) Fixed test failure due to previous patch. Partitions not inheriting table permissions after alter rename partition Key: HIVE-7117 URL: https://issues.apache.org/jira/browse/HIVE-7117 Project: Hive Issue Type: Bug Components: Security Reporter: Ashish Kumar Singh Assignee: Ashish Kumar Singh Attachments: HIVE-7117.2.patch, HIVE-7117.3.patch, HIVE-7117.4.patch, HIVE-7117.5.patch, HIVE-7117.patch On altering/renaming a partition it must inherit permission of the parent directory, if the flag hive.warehouse.subdir.inherit.perms is set. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-7117) Partitions not inheriting table permissions after alter rename partition
[ https://issues.apache.org/jira/browse/HIVE-7117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Kumar Singh updated HIVE-7117: - Status: Open (was: Patch Available) Avoid using same table in two different tests. Partitions not inheriting table permissions after alter rename partition Key: HIVE-7117 URL: https://issues.apache.org/jira/browse/HIVE-7117 Project: Hive Issue Type: Bug Components: Security Reporter: Ashish Kumar Singh Assignee: Ashish Kumar Singh Attachments: HIVE-7117.2.patch, HIVE-7117.3.patch, HIVE-7117.4.patch, HIVE-7117.5.patch, HIVE-7117.patch On altering/renaming a partition it must inherit permission of the parent directory, if the flag hive.warehouse.subdir.inherit.perms is set. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-7117) Partitions not inheriting table permissions after alter rename partition
[ https://issues.apache.org/jira/browse/HIVE-7117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14010093#comment-14010093 ] Ashish Kumar Singh commented on HIVE-7117: -- testAlterPartitionsPerms was using same table name as of testAlterSinglePartitionPerm, which led to failure while creating table in testAlterPartitionPerms. .5 patch fixes this. Updated RB. Partitions not inheriting table permissions after alter rename partition Key: HIVE-7117 URL: https://issues.apache.org/jira/browse/HIVE-7117 Project: Hive Issue Type: Bug Components: Security Reporter: Ashish Kumar Singh Assignee: Ashish Kumar Singh Attachments: HIVE-7117.2.patch, HIVE-7117.3.patch, HIVE-7117.4.patch, HIVE-7117.5.patch, HIVE-7117.patch On altering/renaming a partition it must inherit permission of the parent directory, if the flag hive.warehouse.subdir.inherit.perms is set. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6756) alter table set fileformat should set serde too
[ https://issues.apache.org/jira/browse/HIVE-6756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-6756: --- Resolution: Fixed Fix Version/s: 0.14.0 Status: Resolved (was: Patch Available) Committed to trunk. Thanks, Chinna! alter table set fileformat should set serde too --- Key: HIVE-6756 URL: https://issues.apache.org/jira/browse/HIVE-6756 Project: Hive Issue Type: Bug Affects Versions: 0.13.0 Reporter: Owen O'Malley Assignee: Chinna Rao Lalam Fix For: 0.14.0 Attachments: HIVE-6756.1.patch, HIVE-6756.2.patch, HIVE-6756.3.patch, HIVE-6756.patch Currently doing alter table set fileformat doesn't change the serde. This is unexpected by customers because the serdes are largely file format specific. -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: [VOTE] Apache Hive 0.13.1 Release Candidate 2
The Hive ByLaws require that a release vote should remain open for at least 72 hours, so as to give people time to test. I would like, however to make allowances for the fact that this was a long weekend, and it's likely people were off on vacation. Thus, I think I will count working days to mark the cutoff time for 3 days, thus excluding yesterday and the weekend, thus making the cutoff after tomorrow night (thus allowing Friday, Tuesday and Wednesday for perusal). If I get 3 +1s and no -1s by 1:45AM Thursday May 29th (that's thursday early morning, not thursday afternoon), I will consider this vote as having passed, and continue with the remaining steps outlined in the wiki. In this, I should also note that I skipped the end date declaration for in this vote mail as per Edward's response to my previous vote mail that said that that statement specifying end date was incorrect because a vote will remain open for as long as it has to. I see his point, and it is incorrect to assert that it concludes at 3 days, it's more that 3 days is a minimum time granted to allow people to register disapproval after testing if they desire to. That line was picked directly from the HowToRelease wiki, however, so I would suggest that the PMC clarify the rules on that and edit it appropriately. On Tue, May 27, 2014 at 11:37 AM, Thejas Nair the...@hortonworks.com wrote: Sushanth, Can you specify an end date for the vote ? The vote has been out for 4 days already (more than the mandatory minimum of 3 days), maybe set the end date to two days from now ? Thanks, Thejas On Tue, May 27, 2014 at 11:30 AM, Sushanth Sowmyan khorg...@gmail.com wrote: Hello all, Please test out this Release candidate and cast your vote. I understand I posted it up before a three-day weekend, so it's likely to have not gathered enough eyeballs, but I figured I'd send a reminder out so people can look at it. :) Thanks, -Sushanth On Fri, May 23, 2014 at 1:45 AM, Sushanth Sowmyan khorg...@apache.org wrote: Apache Hive 0.13.1 Release Candidate 2 is available here: http://people.apache.org/~khorgath/releases/0.13.1_RC2/ Maven artifacts are available here: https://repository.apache.org/content/repositories/orgapachehive-1014 Source tag for RC2 is at : https://svn.apache.org/viewvc/hive/tags/release-0.13.1-rc2/ Hive PMC Members: Please test and vote. Thanks, -Sushanth -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
[jira] [Commented] (HIVE-7123) Follow-up of HIVE-6367
[ https://issues.apache.org/jira/browse/HIVE-7123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14010127#comment-14010127 ] Brock Noland commented on HIVE-7123: LGTM. The only item is that precisionToBytes is a constant and thus should be final and all caps. Follow-up of HIVE-6367 -- Key: HIVE-7123 URL: https://issues.apache.org/jira/browse/HIVE-7123 Project: Hive Issue Type: Bug Components: Serializers/Deserializers Affects Versions: 0.14.0 Reporter: Xuefu Zhang Assignee: Xuefu Zhang Attachments: HIVE-7123.1.patch, HIVE-7123.2.patch, HIVE-7123.patch HIVE-6367 provides initial decimal support in Parquet serde. The are a few minor items left over: 1. parquet_decimal.q seems failing 2. will use fixed length binary to encode decimal instead of variable length binary. -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: [VOTE] Apache Hive 0.13.1 Release Candidate 2
That line was picked directly from the HowToRelease wiki, however, so I would suggest that the PMC clarify the rules on that and edit it appropriately. Edward is right that the 3 days is minimum required duration for the vote. That does not contradict the by-law. However, I think having an end date to a vote is important, as without a vote end time, the meaning of 'lazy majority' is not clear. Also, people work better with deadlines! On Tue, May 27, 2014 at 11:49 AM, Sushanth Sowmyan khorg...@gmail.com wrote: The Hive ByLaws require that a release vote should remain open for at least 72 hours, so as to give people time to test. I would like, however to make allowances for the fact that this was a long weekend, and it's likely people were off on vacation. Thus, I think I will count working days to mark the cutoff time for 3 days, thus excluding yesterday and the weekend, thus making the cutoff after tomorrow night (thus allowing Friday, Tuesday and Wednesday for perusal). If I get 3 +1s and no -1s by 1:45AM Thursday May 29th (that's thursday early morning, not thursday afternoon), I will consider this vote as having passed, and continue with the remaining steps outlined in the wiki. In this, I should also note that I skipped the end date declaration for in this vote mail as per Edward's response to my previous vote mail that said that that statement specifying end date was incorrect because a vote will remain open for as long as it has to. I see his point, and it is incorrect to assert that it concludes at 3 days, it's more that 3 days is a minimum time granted to allow people to register disapproval after testing if they desire to. That line was picked directly from the HowToRelease wiki, however, so I would suggest that the PMC clarify the rules on that and edit it appropriately. On Tue, May 27, 2014 at 11:37 AM, Thejas Nair the...@hortonworks.com wrote: Sushanth, Can you specify an end date for the vote ? The vote has been out for 4 days already (more than the mandatory minimum of 3 days), maybe set the end date to two days from now ? Thanks, Thejas On Tue, May 27, 2014 at 11:30 AM, Sushanth Sowmyan khorg...@gmail.com wrote: Hello all, Please test out this Release candidate and cast your vote. I understand I posted it up before a three-day weekend, so it's likely to have not gathered enough eyeballs, but I figured I'd send a reminder out so people can look at it. :) Thanks, -Sushanth On Fri, May 23, 2014 at 1:45 AM, Sushanth Sowmyan khorg...@apache.org wrote: Apache Hive 0.13.1 Release Candidate 2 is available here: http://people.apache.org/~khorgath/releases/0.13.1_RC2/ Maven artifacts are available here: https://repository.apache.org/content/repositories/orgapachehive-1014 Source tag for RC2 is at : https://svn.apache.org/viewvc/hive/tags/release-0.13.1-rc2/ Hive PMC Members: Please test and vote. Thanks, -Sushanth -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
[jira] [Commented] (HIVE-7121) Use murmur hash to distribute HiveKey
[ https://issues.apache.org/jira/browse/HIVE-7121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14010149#comment-14010149 ] Gunther Hagleitner commented on HIVE-7121: -- [~appodictic] I think you're right. This definitely affects bucketing. Options I see are: - Only do it for queries that do not enter into bucketed tables, i.e.: leave the bucketing hash function as badly distributed as it is, but fix shuffle joins, group bys and inserts into other tables. - Remember the hash function in table metadata. This is slightly tricky because we probably don't want a mix of hash functions in the same table (different partitions have different bucketing schemes - that would probably destroy any chance of SMB on that table.) Maybe we even want only one function per DB to make sure different tables in a DB can be joined without looking at the hash function used for each. How come though these unit tests are failing? I didn't think we changed the bucketing scheme between hive 12 and 13. Did we? Use murmur hash to distribute HiveKey - Key: HIVE-7121 URL: https://issues.apache.org/jira/browse/HIVE-7121 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Gopal V Assignee: Gopal V Attachments: HIVE-7121.1.patch, HIVE-7121.WIP.patch The current hashCode implementation produces poor parallelism when dealing with single integers or doubles. And for partitioned inserts into a 1 bucket table, there is a significant hotspot on Reducer #31. Removing the magic number 31 and using a more normal hash algorithm would help fix these hotspots. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-7043) When using the tez session pool via hive, once sessions time out, all queries go to the default queue
[ https://issues.apache.org/jira/browse/HIVE-7043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vikram Dixit K updated HIVE-7043: - Resolution: Fixed Status: Resolved (was: Patch Available) Committed to trunk. When using the tez session pool via hive, once sessions time out, all queries go to the default queue - Key: HIVE-7043 URL: https://issues.apache.org/jira/browse/HIVE-7043 Project: Hive Issue Type: Bug Components: HiveServer2 Affects Versions: 0.13.0 Reporter: Vikram Dixit K Assignee: Vikram Dixit K Fix For: 0.14.0 Attachments: HIVE-7043.2.patch, HIVE-7043.3.patch, HIVE-7043.4.patch, HIVE-7043.4.patch When using a tez session pool to run multiple queries, once the sessions time out, we always end up using the default queue to launch queries. The load balancing doesn't work in this case. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-7043) When using the tez session pool via hive, once sessions time out, all queries go to the default queue
[ https://issues.apache.org/jira/browse/HIVE-7043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14010196#comment-14010196 ] Vikram Dixit K commented on HIVE-7043: -- Thanks [~hagleitn] for the review. When using the tez session pool via hive, once sessions time out, all queries go to the default queue - Key: HIVE-7043 URL: https://issues.apache.org/jira/browse/HIVE-7043 Project: Hive Issue Type: Bug Components: HiveServer2 Affects Versions: 0.13.0 Reporter: Vikram Dixit K Assignee: Vikram Dixit K Fix For: 0.14.0 Attachments: HIVE-7043.2.patch, HIVE-7043.3.patch, HIVE-7043.4.patch, HIVE-7043.4.patch When using a tez session pool to run multiple queries, once the sessions time out, we always end up using the default queue to launch queries. The load balancing doesn't work in this case. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-7121) Use murmur hash to distribute HiveKey
[
https://issues.apache.org/jira/browse/HIVE-7121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14010197#comment-14010197
]
Gopal V commented on HIVE-7121:
---
[~hagleitn]: The unit tests are failing because I'm applying the same insert
mechanic for flat partitioned tables.
The patch works correctly when the following code fragment is hit
{code}
// replace bucketing columns with hashcode % numBuckets
int buckNum = 0;
if (bucketEval != null) {
buckNum = computeBucketNumber(row, conf.getNumBuckets());
cachedKeys[0][buckColIdxInKey] = new IntWritable(buckNum);
}
{code}
This is indeed setup correctly when doing dynamic partitioned inserts. Looks
like this optimization is missed for the flat table inserts.
Use murmur hash to distribute HiveKey
-
Key: HIVE-7121
URL: https://issues.apache.org/jira/browse/HIVE-7121
Project: Hive
Issue Type: Bug
Components: Query Processor
Reporter: Gopal V
Assignee: Gopal V
Attachments: HIVE-7121.1.patch, HIVE-7121.WIP.patch
The current hashCode implementation produces poor parallelism when dealing
with single integers or doubles.
And for partitioned inserts into a 1 bucket table, there is a significant
hotspot on Reducer #31.
Removing the magic number 31 and using a more normal hash algorithm would
help fix these hotspots.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-7087) Remove lineage information after query completion
[
https://issues.apache.org/jira/browse/HIVE-7087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14010204#comment-14010204
]
Hive QA commented on HIVE-7087:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12646832/HIVE-7087.3.patch.txt
{color:red}ERROR:{color} -1 due to 87 failed/errored test(s), 5464 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_numbuckets_partitioned_table2_h23
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_numbuckets_partitioned_table_h23
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_authorization_insert
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join14
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_combine2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_create_like_view
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ctas
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_dynpart_sort_opt_vectorization
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_filter_numeric
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_sort_1_23
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_sort_skew_1_23
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_infer_bucket_sort_list_bucket
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_input12
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_input39
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join14
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_lb_fs_stats
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_11
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_12
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_13
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_14
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_3
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_4
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_5
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_6
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_7
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_8
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_9
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_query_multiskew_1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_query_multiskew_2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_query_multiskew_3
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_query_oneskew_1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_query_oneskew_2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_query_oneskew_3
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_parquet_decimal
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_parquet_decimal1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_recursive_dir
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_sample8
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_sample_islocalmode_hook
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_show_create_table_serde
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_skewjoin_union_remove_1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_list_bucket
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_partscan_1_23
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_transform_ppr1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_transform_ppr2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_truncate_column_list_bucket
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udaf_percentile_approx_23
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_ppr
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_10
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_11
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_12
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_13
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_14
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_15
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_16
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_17
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_18
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_19
[jira] [Commented] (HIVE-6928) Beeline should not chop off describe extended results by default
[ https://issues.apache.org/jira/browse/HIVE-6928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14010240#comment-14010240 ] Szehon Ho commented on HIVE-6928: - Thanks Chinna, +1 (non-binding). Beeline should not chop off describe extended results by default -- Key: HIVE-6928 URL: https://issues.apache.org/jira/browse/HIVE-6928 Project: Hive Issue Type: Bug Components: CLI Reporter: Szehon Ho Assignee: Chinna Rao Lalam Attachments: HIVE-6928.1.patch, HIVE-6928.patch By default, beeline truncates long results based on the console width like: +-+--+ | col_name | | +-+--+ | pat_id | string | | score | float | | acutes | float | | | | | Detailed Table Information | Table(tableName:refills, dbName:default, owner:hdadmin, createTime:1393882396, lastAccessTime:0, retention:0, sd:Sto | +-+--+ 5 rows selected (0.4 seconds) This can be changed by !outputformat, but the default should behave better to give a better experience to the first-time beeline user. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6316) Document support for new types in HCat
[
https://issues.apache.org/jira/browse/HIVE-6316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14010341#comment-14010341
]
Eugene Koifman commented on HIVE-6316:
--
[~leftylev],
Null and Throw are the only possible values. The description of HIVE-5814 has
a usage example:
{noformat}
HCatStorer('','', '-onOutOfRangeValue Throw')
{noformat}
hcat.pig.store.onoutofrangevalue does NOT need to be documented, it's internal.
This only applies when using HCat fro Pig, where the user is expected to use
'onOutOfRangeValue in HCatStorer. Is not really related to Data Promotion
Behavior.
The HCatInputFormat and HCatOutputFormat section need the same update to the
type mapping tables as HCatLoader/HCatStorer. I think it would be easier to
just create link from all 4 current tables to a single page that has the whole
table in
https://issues.apache.org/jira/secure/attachment/12626251/HCat-Pig%20Type%20Mapping%20Hive%200.13.pdf
exactly. The headers in the table actually indicate a mapping of Hive
Type/Value system to Pig Type/Value system.
Logically speaking there is no such thing as HCatalog type/value system.
HCatalog connects Hive tables to Pig/Map Reduce. Pig has it's own type/value
system; MR does not as such and is expected to use (in HCatRecord) the same
classes as used in Hive internally.
so the data type mapping is really Hive-Pig (HCatLoader/Storer) and Hive-MR
(HCatInput/OutputFormat) which is why it's all summarized in a single table in
my document.
Document support for new types in HCat
--
Key: HIVE-6316
URL: https://issues.apache.org/jira/browse/HIVE-6316
Project: Hive
Issue Type: Sub-task
Components: Documentation, HCatalog
Affects Versions: 0.13.0
Reporter: Eugene Koifman
Assignee: Lefty Leverenz
HIVE-5814 added support for new types in HCat. The PDF file in that bug
explains exactly how these map to Pig types. This should be added to the
Wiki somewhere (probably here
https://cwiki.apache.org/confluence/display/Hive/HCatalog+LoadStore).
In particular it should be highlighted that copying data from Hive TIMESTAMP
to Pig DATETIME, any 'nanos' in the timestamp will be lost. Also, HCatStorer
now takes new parameter which is described in the PDF doc.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (HIVE-7129) Change datanucleus.fixedDatastore config to true
Ashutosh Chauhan created HIVE-7129: -- Summary: Change datanucleus.fixedDatastore config to true Key: HIVE-7129 URL: https://issues.apache.org/jira/browse/HIVE-7129 Project: Hive Issue Type: Improvement Components: Metastore Reporter: Ashutosh Chauhan Much safer in production environment to have this as false. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6950) Parsing Error in GROUPING SETS
[
https://issues.apache.org/jira/browse/HIVE-6950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14010449#comment-14010449
]
Jason Rosendale commented on HIVE-6950:
---
I don't know if my issue is identical to this one or just very closely related,
but I can recreate the same error with the following query:
{code}
select a, left(b,5), count(1) from temp_table
group by a, left(b,5)
grouping sets
((left(b,5),a))
{code}
The error does not occur if I just switch the order of the two elements in the
grouping set:
{code}
select a, left(b,5), count(1) from temp_table
group by a, left(b,5)
grouping sets
((a,left(b,5)))
{code}
The error occurs whenever the LEFT function is not the very last element in its
grouping set. The error also occurs when I replace LEFT with other functions.
Parsing Error in GROUPING SETS
--
Key: HIVE-6950
URL: https://issues.apache.org/jira/browse/HIVE-6950
Project: Hive
Issue Type: Bug
Reporter: Rohit Agarwal
The following query:
{code}
SELECT tab1.a,
tab1.b,
SUM(tab1.c)
FROM tab1
GROUP BY tab1.a,
tab1.b
GROUPING SETS ((tab1.a, tab1.b))
{code}
results in the following error:
{code}
ParseException line 7:22 missing ) at ',' near 'EOF'
line 7:31 extraneous input ')' expecting EOF near 'EOF'
{code}
Changing the query to:
{code}
SELECT tab1.a,
tab1.b,
SUM(tab1.c)
FROM tab1
GROUP BY tab1.a,
tab1.b
GROUPING SETS ((a, tab1.b))
{code}
makes it work.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-7123) Follow-up of HIVE-6367
[ https://issues.apache.org/jira/browse/HIVE-7123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuefu Zhang updated HIVE-7123: -- Attachment: HIVE-7123.3.patch Patch #3 addressed above Brock's comment. Follow-up of HIVE-6367 -- Key: HIVE-7123 URL: https://issues.apache.org/jira/browse/HIVE-7123 Project: Hive Issue Type: Bug Components: Serializers/Deserializers Affects Versions: 0.14.0 Reporter: Xuefu Zhang Assignee: Xuefu Zhang Attachments: HIVE-7123.1.patch, HIVE-7123.2.patch, HIVE-7123.3.patch, HIVE-7123.patch HIVE-6367 provides initial decimal support in Parquet serde. The are a few minor items left over: 1. parquet_decimal.q seems failing 2. will use fixed length binary to encode decimal instead of variable length binary. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-7129) Change datanucleus.fixedDatastore config to true
[ https://issues.apache.org/jira/browse/HIVE-7129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-7129: --- Description: Much safer in production environment to have this as true. (was: Much safer in production environment to have this as false.) Change datanucleus.fixedDatastore config to true Key: HIVE-7129 URL: https://issues.apache.org/jira/browse/HIVE-7129 Project: Hive Issue Type: Improvement Components: Metastore Reporter: Ashutosh Chauhan Much safer in production environment to have this as true. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6638) Hive needs to implement recovery for Application Master restart
[ https://issues.apache.org/jira/browse/HIVE-6638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14010504#comment-14010504 ] Ashutosh Chauhan commented on HIVE-6638: +1 Hive needs to implement recovery for Application Master restart Key: HIVE-6638 URL: https://issues.apache.org/jira/browse/HIVE-6638 Project: Hive Issue Type: Improvement Components: Query Processor Affects Versions: 0.11.0, 0.12.0, 0.13.0 Reporter: Ashutosh Chauhan Assignee: Mohammad Kamrul Islam Attachments: HIVE-6638.1.patch, HIVE-6638.2.patch Currently, if AM restarts, whole job is restarted. Although, job and subsequently query would still finish to completion, it would be nice if Hive don't need to redo all the work done under previous AM. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6598) Importing the project into eclipse as maven project have some issues
[
https://issues.apache.org/jira/browse/HIVE-6598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14010510#comment-14010510
]
Ashutosh Chauhan commented on HIVE-6598:
When I run
{code}
mvn eclipse:eclipse -DdownloadSources -DdownloadJavadocs -Phadoop-2
{code}
after applying this patch, I get following warning:
{code}
[WARNING] The POM for org.eclipse.m2e:lifecycle-mapping:jar:1.0.0 is missing,
no dependency information available
[WARNING] Failed to retrieve plugin descriptor for
org.eclipse.m2e:lifecycle-mapping:1.0.0: Plugin
org.eclipse.m2e:lifecycle-mapping:1.0.0 or one of its dependencies could not be
resolved: Failure to find org.eclipse.m2e:lifecycle-mapping:jar:1.0.0 in
http://repo.maven.apache.org/maven2 was cached in the local repository,
resolution will not be reattempted until the update interval of central has
elapsed or updates are forced
{code}
Is that a cause for concern ? Though, I see that .classpath and .project files
being created for different modules, which seem to indicate that patch is
working.
Importing the project into eclipse as maven project have some issues
Key: HIVE-6598
URL: https://issues.apache.org/jira/browse/HIVE-6598
Project: Hive
Issue Type: Bug
Affects Versions: 0.13.0
Environment: Windows 8 ,Eclipse Kepler and Maven 3.1.1
Reporter: Chinna Rao Lalam
Assignee: Chinna Rao Lalam
Fix For: 0.14.0
Attachments: HIVE-6598.patch
Importing the project into eclipse as maven project throwing these problems.
Plugin execution not covered by lifecycle configuration:
org.apache.maven.plugins:maven-antrun-plugin:1.7:run (execution:
setup-test-dirs, phase: process-test-resources)
--
This message was sent by Atlassian JIRA
(v6.2#6252)
Re: [VOTE] Apache Hive 0.13.1 Release Candidate 2
+1 (non-binding) - Built it, checked the signature and md5, and ran some basic tests. Alan. On May 23, 2014, at 1:45 AM, Sushanth Sowmyan khorg...@apache.org wrote: Apache Hive 0.13.1 Release Candidate 2 is available here: http://people.apache.org/~khorgath/releases/0.13.1_RC2/ Maven artifacts are available here: https://repository.apache.org/content/repositories/orgapachehive-1014 Source tag for RC2 is at : https://svn.apache.org/viewvc/hive/tags/release-0.13.1-rc2/ Hive PMC Members: Please test and vote. Thanks, -Sushanth -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
[jira] [Commented] (HIVE-4561) Column stats : LOW_VALUE (or HIGH_VALUE) will always be 0.0000 ,if all the column values larger than 0.0 (or if all column values smaller than 0.0)
[
https://issues.apache.org/jira/browse/HIVE-4561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14010535#comment-14010535
]
Hive QA commented on HIVE-4561:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12646836/HIVE-4561.7.patch.txt
{color:red}ERROR:{color} -1 due to 16 failed/errored test(s), 5466 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_nullgroup
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_nullgroup4
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_nullgroup4_multi_distinct
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_ppd_decimal
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_parquet_decimal1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_partition_wise_fileformat6
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udf_count
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_schemeAuthority2
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_ctas
org.apache.hadoop.hive.ql.exec.tez.TestTezTask.testSubmit
org.apache.hive.hcatalog.pig.TestOrcHCatLoader.testReadDataPrimitiveTypes
org.apache.hive.hcatalog.pig.TestOrcHCatPigStorer.testWriteDecimal
org.apache.hive.hcatalog.pig.TestOrcHCatPigStorer.testWriteDecimalX
org.apache.hive.hcatalog.pig.TestOrcHCatPigStorer.testWriteDecimalXY
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/313/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/313/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-Build-313/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 16 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12646836
Column stats : LOW_VALUE (or HIGH_VALUE) will always be 0. ,if all the
column values larger than 0.0 (or if all column values smaller than 0.0)
Key: HIVE-4561
URL: https://issues.apache.org/jira/browse/HIVE-4561
Project: Hive
Issue Type: Bug
Components: Statistics
Affects Versions: 0.12.0, 0.13.0
Reporter: caofangkun
Assignee: Navis
Attachments: HIVE-4561.1.patch, HIVE-4561.2.patch, HIVE-4561.3.patch,
HIVE-4561.4.patch.txt, HIVE-4561.5.patch.txt, HIVE-4561.6.patch.txt,
HIVE-4561.7.patch.txt
if all column values larger than 0.0 DOUBLE_LOW_VALUE always will be 0.0
or if all column values less than 0.0, DOUBLE_HIGH_VALUE will always be
hive (default) create table src_test (price double);
hive (default) load data local inpath './test.txt' into table src_test;
hive (default) select * from src_test;
OK
1.0
2.0
3.0
Time taken: 0.313 seconds, Fetched: 3 row(s)
hive (default) analyze table src_test compute statistics for columns price;
mysql select * from TAB_COL_STATS \G;
CS_ID: 16
DB_NAME: default
TABLE_NAME: src_test
COLUMN_NAME: price
COLUMN_TYPE: double
TBL_ID: 2586
LONG_LOW_VALUE: 0
LONG_HIGH_VALUE: 0
DOUBLE_LOW_VALUE: 0. # Wrong Result ! Expected is 1.
DOUBLE_HIGH_VALUE: 3.
BIG_DECIMAL_LOW_VALUE: NULL
BIG_DECIMAL_HIGH_VALUE: NULL
NUM_NULLS: 0
NUM_DISTINCTS: 1
AVG_COL_LEN: 0.
MAX_COL_LEN: 0
NUM_TRUES: 0
NUM_FALSES: 0
LAST_ANALYZED: 1368596151
2 rows in set (0.00 sec)
--
This message was sent by Atlassian JIRA
(v6.2#6252)
Re: [VOTE] Apache Hive 0.13.1 Release Candidate 2
+1 On Tue, May 27, 2014 at 7:54 PM, Alan Gates ga...@hortonworks.com wrote: +1 (non-binding) - Built it, checked the signature and md5, and ran some basic tests. Alan. On May 23, 2014, at 1:45 AM, Sushanth Sowmyan khorg...@apache.org wrote: Apache Hive 0.13.1 Release Candidate 2 is available here: http://people.apache.org/~khorgath/releases/0.13.1_RC2/ Maven artifacts are available here: https://repository.apache.org/content/repositories/orgapachehive-1014 Source tag for RC2 is at : https://svn.apache.org/viewvc/hive/tags/release-0.13.1-rc2/ Hive PMC Members: Please test and vote. Thanks, -Sushanth -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: [VOTE] Apache Hive 0.13.1 Release Candidate 2
Short version: SchemaTool is broken in this RC. There are workarounds of not using SchemaTool, but it is a better way of upgrading hive, and I think we should fix it before releasing. Long Version: I ran the unit test suite against the RC and found two failures in TestSchemaTool. There are real issues. The schema initialization fails with schema tool, with the message that Unknown version specified for initialization: 0.13.1. The upgrade with schema tool works (ie it runs the upgrade scripts), but then errors out at the end, when it finds that the version in metastore is 0.13.0 (not 0.13.1). On Tue, May 27, 2014 at 5:16 PM, Edward Capriolo edlinuxg...@gmail.com wrote: +1 On Tue, May 27, 2014 at 7:54 PM, Alan Gates ga...@hortonworks.com wrote: +1 (non-binding) - Built it, checked the signature and md5, and ran some basic tests. Alan. On May 23, 2014, at 1:45 AM, Sushanth Sowmyan khorg...@apache.org wrote: Apache Hive 0.13.1 Release Candidate 2 is available here: http://people.apache.org/~khorgath/releases/0.13.1_RC2/ Maven artifacts are available here: https://repository.apache.org/content/repositories/orgapachehive-1014 Source tag for RC2 is at : https://svn.apache.org/viewvc/hive/tags/release-0.13.1-rc2/ Hive PMC Members: Please test and vote. Thanks, -Sushanth -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
[jira] [Created] (HIVE-7130) schematool is broken for minor version upgrades (eg 0.13.x)
Thejas M Nair created HIVE-7130: --- Summary: schematool is broken for minor version upgrades (eg 0.13.x) Key: HIVE-7130 URL: https://issues.apache.org/jira/browse/HIVE-7130 Project: Hive Issue Type: Bug Affects Versions: 0.13.1 Reporter: Thejas M Nair Assignee: Thejas M Nair The schema initialization fails with schema tool in 0.13.1, with the message that Unknown version specified for initialization: 0.13.1. The upgrade with schema tool works (ie it runs the upgrade scripts), but then errors out at the end, when it finds that the version in metastore is 0.13.0 (not 0.13.1). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-7130) schematool is broken for minor version upgrades (eg 0.13.x)
[
https://issues.apache.org/jira/browse/HIVE-7130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14010673#comment-14010673
]
Thejas M Nair commented on HIVE-7130:
-
{code}
[apache-hive-0.13.1-bin18:52]$ bin/schematool -dbType derby -initSchema
Picked up JAVA_TOOL_OPTIONS: -Djava.awt.headless=true
Picked up JAVA_TOOL_OPTIONS: -Djava.awt.headless=true
Metastore connection URL:
jdbc:derby:;databaseName=metastore_db;create=true
Metastore Connection Driver :org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User: APP
Starting metastore schema initialization to 0.13.1
org.apache.hadoop.hive.metastore.HiveMetaException: Unknown version specified
for initialization: 0.13.1
*** schemaTool failed ***
[apache-hive-0.13.1-bin18:35]$ bin/schematool -dbType derby -initSchemaTo
0.12.0
Picked up JAVA_TOOL_OPTIONS: -Djava.awt.headless=true
Picked up JAVA_TOOL_OPTIONS: -Djava.awt.headless=true
Metastore connection URL:
jdbc:derby:;databaseName=metastore_db;create=true
Metastore Connection Driver :org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User: APP
Starting metastore schema initialization to 0.12.0
Initialization script hive-schema-0.12.0.derby.sql
Initialization script completed
schemaTool completeted
[apache-hive-0.13.1-bin18:35]$ bin/schematool -dbType derby -upgradeSchema
Picked up JAVA_TOOL_OPTIONS: -Djava.awt.headless=true
Picked up JAVA_TOOL_OPTIONS: -Djava.awt.headless=true
Metastore connection URL:
jdbc:derby:;databaseName=metastore_db;create=true
Metastore Connection Driver :org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User: APP
Starting upgrade metastore schema from version 0.12.0 to 0.13.1
Upgrade script upgrade-0.12.0-to-0.13.0.derby.sql
Completed upgrade-0.12.0-to-0.13.0.derby.sql
org.apache.hadoop.hive.metastore.HiveMetaException: Found unexpected schema
version 0.13.0
*** schemaTool failed ***
{code}
schematool is broken for minor version upgrades (eg 0.13.x)
---
Key: HIVE-7130
URL: https://issues.apache.org/jira/browse/HIVE-7130
Project: Hive
Issue Type: Bug
Affects Versions: 0.13.1
Reporter: Thejas M Nair
Assignee: Thejas M Nair
The schema initialization fails with schema tool in 0.13.1, with the message
that Unknown version specified for initialization: 0.13.1.
The upgrade with schema tool works (ie it runs the upgrade scripts),
but then errors out at the end, when it finds that the version in
metastore is 0.13.0 (not 0.13.1).
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Comment Edited] (HIVE-7130) schematool is broken for minor version upgrades (eg 0.13.x)
[
https://issues.apache.org/jira/browse/HIVE-7130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14010673#comment-14010673
]
Thejas M Nair edited comment on HIVE-7130 at 5/28/14 2:18 AM:
--
Schema tool when used for schema initialization -
{code}
[apache-hive-0.13.1-bin18:52]$ bin/schematool -dbType derby -initSchema
Picked up JAVA_TOOL_OPTIONS: -Djava.awt.headless=true
Picked up JAVA_TOOL_OPTIONS: -Djava.awt.headless=true
Metastore connection URL:
jdbc:derby:;databaseName=metastore_db;create=true
Metastore Connection Driver :org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User: APP
Starting metastore schema initialization to 0.13.1
org.apache.hadoop.hive.metastore.HiveMetaException: Unknown version specified
for initialization: 0.13.1
*** schemaTool failed ***
{code}
Schema tool when used for upgrade -
{code}
[apache-hive-0.13.1-bin18:35]$ bin/schematool -dbType derby -initSchemaTo
0.12.0
Picked up JAVA_TOOL_OPTIONS: -Djava.awt.headless=true
Picked up JAVA_TOOL_OPTIONS: -Djava.awt.headless=true
Metastore connection URL:
jdbc:derby:;databaseName=metastore_db;create=true
Metastore Connection Driver :org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User: APP
Starting metastore schema initialization to 0.12.0
Initialization script hive-schema-0.12.0.derby.sql
Initialization script completed
schemaTool completeted
[apache-hive-0.13.1-bin18:35]$ bin/schematool -dbType derby -upgradeSchema
Picked up JAVA_TOOL_OPTIONS: -Djava.awt.headless=true
Picked up JAVA_TOOL_OPTIONS: -Djava.awt.headless=true
Metastore connection URL:
jdbc:derby:;databaseName=metastore_db;create=true
Metastore Connection Driver :org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User: APP
Starting upgrade metastore schema from version 0.12.0 to 0.13.1
Upgrade script upgrade-0.12.0-to-0.13.0.derby.sql
Completed upgrade-0.12.0-to-0.13.0.derby.sql
org.apache.hadoop.hive.metastore.HiveMetaException: Found unexpected schema
version 0.13.0
*** schemaTool failed ***
{code}
was (Author: thejas):
{code}
[apache-hive-0.13.1-bin18:52]$ bin/schematool -dbType derby -initSchema
Picked up JAVA_TOOL_OPTIONS: -Djava.awt.headless=true
Picked up JAVA_TOOL_OPTIONS: -Djava.awt.headless=true
Metastore connection URL:
jdbc:derby:;databaseName=metastore_db;create=true
Metastore Connection Driver :org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User: APP
Starting metastore schema initialization to 0.13.1
org.apache.hadoop.hive.metastore.HiveMetaException: Unknown version specified
for initialization: 0.13.1
*** schemaTool failed ***
[apache-hive-0.13.1-bin18:35]$ bin/schematool -dbType derby -initSchemaTo
0.12.0
Picked up JAVA_TOOL_OPTIONS: -Djava.awt.headless=true
Picked up JAVA_TOOL_OPTIONS: -Djava.awt.headless=true
Metastore connection URL:
jdbc:derby:;databaseName=metastore_db;create=true
Metastore Connection Driver :org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User: APP
Starting metastore schema initialization to 0.12.0
Initialization script hive-schema-0.12.0.derby.sql
Initialization script completed
schemaTool completeted
[apache-hive-0.13.1-bin18:35]$ bin/schematool -dbType derby -upgradeSchema
Picked up JAVA_TOOL_OPTIONS: -Djava.awt.headless=true
Picked up JAVA_TOOL_OPTIONS: -Djava.awt.headless=true
Metastore connection URL:
jdbc:derby:;databaseName=metastore_db;create=true
Metastore Connection Driver :org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User: APP
Starting upgrade metastore schema from version 0.12.0 to 0.13.1
Upgrade script upgrade-0.12.0-to-0.13.0.derby.sql
Completed upgrade-0.12.0-to-0.13.0.derby.sql
org.apache.hadoop.hive.metastore.HiveMetaException: Found unexpected schema
version 0.13.0
*** schemaTool failed ***
{code}
schematool is broken for minor version upgrades (eg 0.13.x)
---
Key: HIVE-7130
URL: https://issues.apache.org/jira/browse/HIVE-7130
Project: Hive
Issue Type: Bug
Affects Versions: 0.13.1
Reporter: Thejas M Nair
Assignee: Thejas M Nair
The schema initialization fails with schema tool in 0.13.1, with the message
that Unknown version specified for initialization: 0.13.1.
The upgrade with schema tool works (ie it runs the upgrade scripts),
but then errors out at the end, when it finds that the version in
metastore is 0.13.0 (not 0.13.1).
--
This message was sent by Atlassian JIRA
(v6.2#6252)
Re: [VOTE] Apache Hive 0.13.1 Release Candidate 2
Created jira with more information - https://issues.apache.org/jira/browse/HIVE-7130. On Tue, May 27, 2014 at 7:13 PM, Thejas Nair the...@hortonworks.com wrote: Short version: SchemaTool is broken in this RC. There are workarounds of not using SchemaTool, but it is a better way of upgrading hive, and I think we should fix it before releasing. Long Version: I ran the unit test suite against the RC and found two failures in TestSchemaTool. There are real issues. The schema initialization fails with schema tool, with the message that Unknown version specified for initialization: 0.13.1. The upgrade with schema tool works (ie it runs the upgrade scripts), but then errors out at the end, when it finds that the version in metastore is 0.13.0 (not 0.13.1). On Tue, May 27, 2014 at 5:16 PM, Edward Capriolo edlinuxg...@gmail.com wrote: +1 On Tue, May 27, 2014 at 7:54 PM, Alan Gates ga...@hortonworks.com wrote: +1 (non-binding) - Built it, checked the signature and md5, and ran some basic tests. Alan. On May 23, 2014, at 1:45 AM, Sushanth Sowmyan khorg...@apache.org wrote: Apache Hive 0.13.1 Release Candidate 2 is available here: http://people.apache.org/~khorgath/releases/0.13.1_RC2/ Maven artifacts are available here: https://repository.apache.org/content/repositories/orgapachehive-1014 Source tag for RC2 is at : https://svn.apache.org/viewvc/hive/tags/release-0.13.1-rc2/ Hive PMC Members: Please test and vote. Thanks, -Sushanth -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
[jira] [Commented] (HIVE-7130) schematool is broken for minor version upgrades (eg 0.13.x)
[
https://issues.apache.org/jira/browse/HIVE-7130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14010675#comment-14010675
]
Thejas M Nair commented on HIVE-7130:
-
Also after upgrade with schematool, the command fails if you turn on
hive.metastore.schema.verification
{code}
bin/hive -hiveconf hive.metastore.schema.verification=true
..
..
Caused by: MetaException(message:Hive Schema version 0.13.1 does not match
metastore's schema version 0.13.0 Metastore is not upgraded or corrupt)
at
org.apache.hadoop.hive.metastore.ObjectStore.checkSchema(ObjectStore.java:6306)
at
org.apache.hadoop.hive.metastore.ObjectStore.verifySchema(ObjectStore.java:6277)
{code}
schematool is broken for minor version upgrades (eg 0.13.x)
---
Key: HIVE-7130
URL: https://issues.apache.org/jira/browse/HIVE-7130
Project: Hive
Issue Type: Bug
Affects Versions: 0.13.1
Reporter: Thejas M Nair
Assignee: Thejas M Nair
The schema initialization fails with schema tool in 0.13.1, with the message
that Unknown version specified for initialization: 0.13.1.
The upgrade with schema tool works (ie it runs the upgrade scripts),
but then errors out at the end, when it finds that the version in
metastore is 0.13.0 (not 0.13.1).
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-7123) Follow-up of HIVE-6367
[ https://issues.apache.org/jira/browse/HIVE-7123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14010688#comment-14010688 ] Brock Noland commented on HIVE-7123: Thank you!! +1 pending tests Follow-up of HIVE-6367 -- Key: HIVE-7123 URL: https://issues.apache.org/jira/browse/HIVE-7123 Project: Hive Issue Type: Bug Components: Serializers/Deserializers Affects Versions: 0.14.0 Reporter: Xuefu Zhang Assignee: Xuefu Zhang Attachments: HIVE-7123.1.patch, HIVE-7123.2.patch, HIVE-7123.3.patch, HIVE-7123.patch HIVE-6367 provides initial decimal support in Parquet serde. The are a few minor items left over: 1. parquet_decimal.q seems failing 2. will use fixed length binary to encode decimal instead of variable length binary. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-7087) Remove lineage information after query completion
[ https://issues.apache.org/jira/browse/HIVE-7087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-7087: --- Resolution: Fixed Fix Version/s: 0.14.0 Status: Resolved (was: Patch Available) Committed to trunk. Thanks, Navis! Remove lineage information after query completion - Key: HIVE-7087 URL: https://issues.apache.org/jira/browse/HIVE-7087 Project: Hive Issue Type: Bug Components: Logging Reporter: Navis Assignee: Navis Priority: Minor Fix For: 0.14.0 Attachments: HIVE-7087.1.patch.txt, HIVE-7087.2.patch.txt, HIVE-7087.3.patch.txt Lineage information is stacked in session and is not cleared before the session is closed. That also makes redundant lineage logs in q.out files for all of the queries after any inserts, which should be available only for insert queries. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6313) Minimr tests in hadoop-1 hangs on shutdown
[ https://issues.apache.org/jira/browse/HIVE-6313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-6313: --- Resolution: Fixed Fix Version/s: 0.14.0 Status: Resolved (was: Patch Available) Committed to trunk. Thanks, Navis! Minimr tests in hadoop-1 hangs on shutdown -- Key: HIVE-6313 URL: https://issues.apache.org/jira/browse/HIVE-6313 Project: Hive Issue Type: Bug Components: Tests Reporter: Navis Assignee: Navis Priority: Trivial Fix For: 0.14.0 Attachments: HIVE-6313.1.patch.txt, HIVE-6313.2.patch.txt It takes minutes after all tests run waiting for all task trackers shutdown. Just shutting down JobTracker after killing pending jobs seemed enough. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6367) Implement Decimal in ParquetSerde
[ https://issues.apache.org/jira/browse/HIVE-6367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14010722#comment-14010722 ] Navis commented on HIVE-6367: - [~xuefuz] [~brocknoland] dec_comp.txt is missing, making parquet_decimal1 fail. Implement Decimal in ParquetSerde - Key: HIVE-6367 URL: https://issues.apache.org/jira/browse/HIVE-6367 Project: Hive Issue Type: Sub-task Components: Serializers/Deserializers Affects Versions: 0.13.0 Reporter: Brock Noland Assignee: Xuefu Zhang Labels: Parquet Fix For: 0.14.0 Attachments: HIVE-6367.patch, dec.parq Some code in the Parquet Serde deals with decimal and other does not. For example in ETypeConverter we convert Decimal to double (which is invalid) whereas in DataWritableWriter and other locations we throw an exception if decimal is used. This JIRA is to implement decimal support. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-4561) Column stats : LOW_VALUE (or HIGH_VALUE) will always be 0.0000 ,if all the column values larger than 0.0 (or if all column values smaller than 0.0)
[ https://issues.apache.org/jira/browse/HIVE-4561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-4561: Attachment: HIVE-4561.8.patch.txt Column stats : LOW_VALUE (or HIGH_VALUE) will always be 0. ,if all the column values larger than 0.0 (or if all column values smaller than 0.0) Key: HIVE-4561 URL: https://issues.apache.org/jira/browse/HIVE-4561 Project: Hive Issue Type: Bug Components: Statistics Affects Versions: 0.12.0, 0.13.0 Reporter: caofangkun Assignee: Navis Attachments: HIVE-4561.1.patch, HIVE-4561.2.patch, HIVE-4561.3.patch, HIVE-4561.4.patch.txt, HIVE-4561.5.patch.txt, HIVE-4561.6.patch.txt, HIVE-4561.7.patch.txt, HIVE-4561.8.patch.txt if all column values larger than 0.0 DOUBLE_LOW_VALUE always will be 0.0 or if all column values less than 0.0, DOUBLE_HIGH_VALUE will always be hive (default) create table src_test (price double); hive (default) load data local inpath './test.txt' into table src_test; hive (default) select * from src_test; OK 1.0 2.0 3.0 Time taken: 0.313 seconds, Fetched: 3 row(s) hive (default) analyze table src_test compute statistics for columns price; mysql select * from TAB_COL_STATS \G; CS_ID: 16 DB_NAME: default TABLE_NAME: src_test COLUMN_NAME: price COLUMN_TYPE: double TBL_ID: 2586 LONG_LOW_VALUE: 0 LONG_HIGH_VALUE: 0 DOUBLE_LOW_VALUE: 0. # Wrong Result ! Expected is 1. DOUBLE_HIGH_VALUE: 3. BIG_DECIMAL_LOW_VALUE: NULL BIG_DECIMAL_HIGH_VALUE: NULL NUM_NULLS: 0 NUM_DISTINCTS: 1 AVG_COL_LEN: 0. MAX_COL_LEN: 0 NUM_TRUES: 0 NUM_FALSES: 0 LAST_ANALYZED: 1368596151 2 rows in set (0.00 sec) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-7087) Remove lineage information after query completion
[ https://issues.apache.org/jira/browse/HIVE-7087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14010726#comment-14010726 ] Navis commented on HIVE-7087: - [~ashutoshc] Oops, this needs one more diff file. I'm on it. Remove lineage information after query completion - Key: HIVE-7087 URL: https://issues.apache.org/jira/browse/HIVE-7087 Project: Hive Issue Type: Bug Components: Logging Reporter: Navis Assignee: Navis Priority: Minor Fix For: 0.14.0 Attachments: HIVE-7087.1.patch.txt, HIVE-7087.2.patch.txt, HIVE-7087.3.patch.txt Lineage information is stacked in session and is not cleared before the session is closed. That also makes redundant lineage logs in q.out files for all of the queries after any inserts, which should be available only for insert queries. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6367) Implement Decimal in ParquetSerde
[ https://issues.apache.org/jira/browse/HIVE-6367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14010727#comment-14010727 ] Xuefu Zhang commented on HIVE-6367: --- It's addressed in the followiup jira, HIVE-7123. Implement Decimal in ParquetSerde - Key: HIVE-6367 URL: https://issues.apache.org/jira/browse/HIVE-6367 Project: Hive Issue Type: Sub-task Components: Serializers/Deserializers Affects Versions: 0.13.0 Reporter: Brock Noland Assignee: Xuefu Zhang Labels: Parquet Fix For: 0.14.0 Attachments: HIVE-6367.patch, dec.parq Some code in the Parquet Serde deals with decimal and other does not. For example in ETypeConverter we convert Decimal to double (which is invalid) whereas in DataWritableWriter and other locations we throw an exception if decimal is used. This JIRA is to implement decimal support. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-7127) Handover more details on exception in hiveserver2
[
https://issues.apache.org/jira/browse/HIVE-7127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14010745#comment-14010745
]
Hive QA commented on HIVE-7127:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12646847/HIVE-7127.1.patch.txt
{color:red}ERROR:{color} -1 due to 10 failed/errored test(s), 5467 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_combine2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_parquet_decimal1
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_ctas
org.apache.hadoop.hive.ql.exec.tez.TestTezTask.testSubmit
org.apache.hive.beeline.TestBeelineArgParsing.testBeelineOpts
org.apache.hive.hcatalog.pig.TestOrcHCatPigStorer.testWriteDecimal
org.apache.hive.hcatalog.pig.TestOrcHCatPigStorer.testWriteDecimalX
org.apache.hive.hcatalog.pig.TestOrcHCatPigStorer.testWriteDecimalXY
org.apache.hive.jdbc.TestJdbcDriver2.testErrorMessages
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/314/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/314/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-Build-314/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 10 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12646847
Handover more details on exception in hiveserver2
-
Key: HIVE-7127
URL: https://issues.apache.org/jira/browse/HIVE-7127
Project: Hive
Issue Type: Improvement
Components: JDBC
Reporter: Navis
Assignee: Navis
Priority: Trivial
Attachments: HIVE-7127.1.patch.txt
NO_PRECOMMIT_TEST
Currently, JDBC hands over exception message and error codes. But it's not
helpful for debugging.
{noformat}
org.apache.hive.service.cli.HiveSQLException: Error while compiling
statement: FAILED: ParseException line 1:0 cannot recognize input near
'createa' 'asd' 'EOF'
at org.apache.hive.jdbc.Utils.verifySuccess(Utils.java:121)
at org.apache.hive.jdbc.Utils.verifySuccessWithInfo(Utils.java:109)
at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:231)
at org.apache.hive.beeline.Commands.execute(Commands.java:736)
at org.apache.hive.beeline.Commands.sql(Commands.java:657)
at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:889)
at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:744)
at
org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:459)
at org.apache.hive.beeline.BeeLine.main(BeeLine.java:442)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:160)
{noformat}
With this patch, JDBC client can get more details on hiveserver2.
{noformat}
Caused by: org.apache.hive.service.cli.HiveSQLException: Error while
compiling statement: FAILED: ParseException line 1:0 cannot recognize input
near 'createa' 'asd' 'EOF'
at org.apache.hive.service.cli.operation.SQLOperation.prepare(Unknown
Source)
at org.apache.hive.service.cli.operation.SQLOperation.run(Unknown
Source)
at
org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(Unknown
Source)
at
org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(Unknown
Source)
at org.apache.hive.service.cli.CLIService.executeStatementAsync(Unknown
Source)
at
org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(Unknown
Source)
at
org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(Unknown
Source)
at
org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(Unknown
Source)
at org.apache.thrift.ProcessFunction.process(Unknown Source)
at org.apache.thrift.TBaseProcessor.process(Unknown Source)
at org.apache.hive.service.auth.TSetIpAddressProcessor.process(Unknown
Source)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(Unknown
[jira] [Updated] (HIVE-7087) Remove lineage information after query completion
[ https://issues.apache.org/jira/browse/HIVE-7087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-7087: Attachment: HIVE-7087.3_2.patch.txt Remove lineage information after query completion - Key: HIVE-7087 URL: https://issues.apache.org/jira/browse/HIVE-7087 Project: Hive Issue Type: Bug Components: Logging Reporter: Navis Assignee: Navis Priority: Minor Fix For: 0.14.0 Attachments: HIVE-7087.1.patch.txt, HIVE-7087.2.patch.txt, HIVE-7087.3.patch.txt, HIVE-7087.3_2.patch.txt Lineage information is stacked in session and is not cleared before the session is closed. That also makes redundant lineage logs in q.out files for all of the queries after any inserts, which should be available only for insert queries. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-7127) Handover more details on exception in hiveserver2
[
https://issues.apache.org/jira/browse/HIVE-7127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-7127:
Attachment: HIVE-7127.2.patch.txt
Handover more details on exception in hiveserver2
-
Key: HIVE-7127
URL: https://issues.apache.org/jira/browse/HIVE-7127
Project: Hive
Issue Type: Improvement
Components: JDBC
Reporter: Navis
Assignee: Navis
Priority: Trivial
Attachments: HIVE-7127.1.patch.txt, HIVE-7127.2.patch.txt
NO_PRECOMMIT_TEST
Currently, JDBC hands over exception message and error codes. But it's not
helpful for debugging.
{noformat}
org.apache.hive.service.cli.HiveSQLException: Error while compiling
statement: FAILED: ParseException line 1:0 cannot recognize input near
'createa' 'asd' 'EOF'
at org.apache.hive.jdbc.Utils.verifySuccess(Utils.java:121)
at org.apache.hive.jdbc.Utils.verifySuccessWithInfo(Utils.java:109)
at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:231)
at org.apache.hive.beeline.Commands.execute(Commands.java:736)
at org.apache.hive.beeline.Commands.sql(Commands.java:657)
at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:889)
at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:744)
at
org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:459)
at org.apache.hive.beeline.BeeLine.main(BeeLine.java:442)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:160)
{noformat}
With this patch, JDBC client can get more details on hiveserver2.
{noformat}
Caused by: org.apache.hive.service.cli.HiveSQLException: Error while
compiling statement: FAILED: ParseException line 1:0 cannot recognize input
near 'createa' 'asd' 'EOF'
at org.apache.hive.service.cli.operation.SQLOperation.prepare(Unknown
Source)
at org.apache.hive.service.cli.operation.SQLOperation.run(Unknown
Source)
at
org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(Unknown
Source)
at
org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(Unknown
Source)
at org.apache.hive.service.cli.CLIService.executeStatementAsync(Unknown
Source)
at
org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(Unknown
Source)
at
org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(Unknown
Source)
at
org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(Unknown
Source)
at org.apache.thrift.ProcessFunction.process(Unknown Source)
at org.apache.thrift.TBaseProcessor.process(Unknown Source)
at org.apache.hive.service.auth.TSetIpAddressProcessor.process(Unknown
Source)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(Unknown
Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6367) Implement Decimal in ParquetSerde
[ https://issues.apache.org/jira/browse/HIVE-6367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14010775#comment-14010775 ] Navis commented on HIVE-6367: - [~xuefuz] Ah, thanks. Implement Decimal in ParquetSerde - Key: HIVE-6367 URL: https://issues.apache.org/jira/browse/HIVE-6367 Project: Hive Issue Type: Sub-task Components: Serializers/Deserializers Affects Versions: 0.13.0 Reporter: Brock Noland Assignee: Xuefu Zhang Labels: Parquet Fix For: 0.14.0 Attachments: HIVE-6367.patch, dec.parq Some code in the Parquet Serde deals with decimal and other does not. For example in ETypeConverter we convert Decimal to double (which is invalid) whereas in DataWritableWriter and other locations we throw an exception if decimal is used. This JIRA is to implement decimal support. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-7043) When using the tez session pool via hive, once sessions time out, all queries go to the default queue
[ https://issues.apache.org/jira/browse/HIVE-7043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-7043: Attachment: HIVE-7043.followup.patch.txt [~vikram.dixit] TestTezTask#testSubmit fails with NPE. Consider this patch. When using the tez session pool via hive, once sessions time out, all queries go to the default queue - Key: HIVE-7043 URL: https://issues.apache.org/jira/browse/HIVE-7043 Project: Hive Issue Type: Bug Components: HiveServer2 Affects Versions: 0.13.0 Reporter: Vikram Dixit K Assignee: Vikram Dixit K Fix For: 0.14.0 Attachments: HIVE-7043.2.patch, HIVE-7043.3.patch, HIVE-7043.4.patch, HIVE-7043.4.patch, HIVE-7043.followup.patch.txt When using a tez session pool to run multiple queries, once the sessions time out, we always end up using the default queue to launch queries. The load balancing doesn't work in this case. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-7129) Change datanucleus.fixedDatastore config to true
[ https://issues.apache.org/jira/browse/HIVE-7129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14010784#comment-14010784 ] Navis commented on HIVE-7129: - Seemed duplicates HIVE-1841. We should check comment of [~slider] first (https://issues.apache.org/jira/browse/HIVE-1841?focusedCommentId=13063474). Change datanucleus.fixedDatastore config to true Key: HIVE-7129 URL: https://issues.apache.org/jira/browse/HIVE-7129 Project: Hive Issue Type: Improvement Components: Metastore Reporter: Ashutosh Chauhan Much safer in production environment to have this as true. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-7087) Remove lineage information after query completion
[ https://issues.apache.org/jira/browse/HIVE-7087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14010803#comment-14010803 ] Ashutosh Chauhan commented on HIVE-7087: Thanks, Navis. I updated all other .q.out files where reported by Hive QA but missed tez files. Remove lineage information after query completion - Key: HIVE-7087 URL: https://issues.apache.org/jira/browse/HIVE-7087 Project: Hive Issue Type: Bug Components: Logging Reporter: Navis Assignee: Navis Priority: Minor Fix For: 0.14.0 Attachments: HIVE-7087.1.patch.txt, HIVE-7087.2.patch.txt, HIVE-7087.3.patch.txt, HIVE-7087.3_2.patch.txt Lineage information is stacked in session and is not cleared before the session is closed. That also makes redundant lineage logs in q.out files for all of the queries after any inserts, which should be available only for insert queries. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-7062) Support Streaming mode in Windowing
[ https://issues.apache.org/jira/browse/HIVE-7062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harish Butani updated HIVE-7062: Attachment: HIVE-7062.4.patch Has Framework changes + Streaming for Sum and Avg functions. Still need to do Streaming for Min, Max, Lead, Lag, FirstVal, LastVal Support Streaming mode in Windowing --- Key: HIVE-7062 URL: https://issues.apache.org/jira/browse/HIVE-7062 Project: Hive Issue Type: Bug Reporter: Harish Butani Assignee: Harish Butani Attachments: HIVE-7062.1.patch, HIVE-7062.4.patch 1. Have the Windowing Table Function support streaming mode. 2. Have special handling for Ranking UDAFs. 3. Have special handling for Sum/Avg for fixed size Wdws. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-4867) Deduplicate columns appearing in both the key list and value list of ReduceSinkOperator
[
https://issues.apache.org/jira/browse/HIVE-4867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Chauhan updated HIVE-4867:
---
Status: Open (was: Patch Available)
[~navis] Do you want to update this patch, now that HIVE-7087 is in.
Deduplicate columns appearing in both the key list and value list of
ReduceSinkOperator
---
Key: HIVE-4867
URL: https://issues.apache.org/jira/browse/HIVE-4867
Project: Hive
Issue Type: Improvement
Reporter: Yin Huai
Assignee: Navis
Attachments: HIVE-4867.1.patch.txt, source_only.txt
A ReduceSinkOperator emits data in the format of keys and values. Right now,
a column may appear in both the key list and value list, which result in
unnecessary overhead for shuffling.
Example:
We have a query shown below ...
{code:sql}
explain select ss_ticket_number from store_sales cluster by ss_ticket_number;
{\code}
The plan is ...
{code}
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 is a root stage
STAGE PLANS:
Stage: Stage-1
Map Reduce
Alias - Map Operator Tree:
store_sales
TableScan
alias: store_sales
Select Operator
expressions:
expr: ss_ticket_number
type: int
outputColumnNames: _col0
Reduce Output Operator
key expressions:
expr: _col0
type: int
sort order: +
Map-reduce partition columns:
expr: _col0
type: int
tag: -1
value expressions:
expr: _col0
type: int
Reduce Operator Tree:
Extract
File Output Operator
compressed: false
GlobalTableId: 0
table:
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
Stage: Stage-0
Fetch Operator
limit: -1
{\code}
The column 'ss_ticket_number' is in both the key list and value list of the
ReduceSinkOperator. The type of ss_ticket_number is int. For this case,
BinarySortableSerDe will introduce 1 byte more for every int in the key.
LazyBinarySerDe will also introduce overhead when recording the length of a
int. For every int, 10 bytes should be a rough estimation of the size of data
emitted from the Map phase.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
Review Request 21970: HIVE-7062: Support Streaming mode in Windowing
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/21970/ --- Review request for hive and Ashutosh Chauhan. Bugs: HIVE-7062 https://issues.apache.org/jira/browse/HIVE-7062 Repository: hive-git Description --- 1. Have the Windowing Table Function support streaming mode. 2. Have special handling for Ranking UDAFs. 3. Have special handling for Sum/Avg for fixed size Wdws. Diffs - ql/src/java/org/apache/hadoop/hive/ql/exec/PTFOperator.java d3800c2 ql/src/java/org/apache/hadoop/hive/ql/exec/PTFPartition.java b5adb11 ql/src/java/org/apache/hadoop/hive/ql/exec/PTFRollingPartition.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFAverage.java 814ae37 ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFCumeDist.java 18c8c8d ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFDenseRank.java c1d43d8 ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFEvaluator.java 5668a3b ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFPercentRank.java aab1922 ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFRank.java 5c8f1e0 ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFStreamingEnhancer.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFSum.java 8508ffb ql/src/java/org/apache/hadoop/hive/ql/udf/generic/ISupportStreamingModeForWindowing.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/udf/ptf/NoopStreaming.java d50a542 ql/src/java/org/apache/hadoop/hive/ql/udf/ptf/NoopWithMapStreaming.java be1f9ab ql/src/java/org/apache/hadoop/hive/ql/udf/ptf/TableFunctionEvaluator.java 8a1e085 ql/src/java/org/apache/hadoop/hive/ql/udf/ptf/WindowingTableFunction.java cdb5624 ql/src/test/org/apache/hadoop/hive/ql/udaf/TestStreamingAvg.java PRE-CREATION ql/src/test/org/apache/hadoop/hive/ql/udaf/TestStreamingSum.java PRE-CREATION ql/src/test/results/clientpositive/ptf.q.out eb4997d ql/src/test/results/clientpositive/windowing.q.out 7e23497 ql/src/test/results/clientpositive/windowing_windowspec.q.out 6ea068c Diff: https://reviews.apache.org/r/21970/diff/ Testing --- run existing windowing and ptf tests Add unit tests for StreamingSum and StreamingAvg evaluators. Thanks, Harish Butani
[jira] [Commented] (HIVE-4561) Column stats : LOW_VALUE (or HIGH_VALUE) will always be 0.0000 ,if all the column values larger than 0.0 (or if all column values smaller than 0.0)
[ https://issues.apache.org/jira/browse/HIVE-4561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14010813#comment-14010813 ] Ashutosh Chauhan commented on HIVE-4561: Can you also update RB with latest patch ? Column stats : LOW_VALUE (or HIGH_VALUE) will always be 0. ,if all the column values larger than 0.0 (or if all column values smaller than 0.0) Key: HIVE-4561 URL: https://issues.apache.org/jira/browse/HIVE-4561 Project: Hive Issue Type: Bug Components: Statistics Affects Versions: 0.12.0, 0.13.0 Reporter: caofangkun Assignee: Navis Attachments: HIVE-4561.1.patch, HIVE-4561.2.patch, HIVE-4561.3.patch, HIVE-4561.4.patch.txt, HIVE-4561.5.patch.txt, HIVE-4561.6.patch.txt, HIVE-4561.7.patch.txt, HIVE-4561.8.patch.txt if all column values larger than 0.0 DOUBLE_LOW_VALUE always will be 0.0 or if all column values less than 0.0, DOUBLE_HIGH_VALUE will always be hive (default) create table src_test (price double); hive (default) load data local inpath './test.txt' into table src_test; hive (default) select * from src_test; OK 1.0 2.0 3.0 Time taken: 0.313 seconds, Fetched: 3 row(s) hive (default) analyze table src_test compute statistics for columns price; mysql select * from TAB_COL_STATS \G; CS_ID: 16 DB_NAME: default TABLE_NAME: src_test COLUMN_NAME: price COLUMN_TYPE: double TBL_ID: 2586 LONG_LOW_VALUE: 0 LONG_HIGH_VALUE: 0 DOUBLE_LOW_VALUE: 0. # Wrong Result ! Expected is 1. DOUBLE_HIGH_VALUE: 3. BIG_DECIMAL_LOW_VALUE: NULL BIG_DECIMAL_HIGH_VALUE: NULL NUM_NULLS: 0 NUM_DISTINCTS: 1 AVG_COL_LEN: 0. MAX_COL_LEN: 0 NUM_TRUES: 0 NUM_FALSES: 0 LAST_ANALYZED: 1368596151 2 rows in set (0.00 sec) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-7043) When using the tez session pool via hive, once sessions time out, all queries go to the default queue
[ https://issues.apache.org/jira/browse/HIVE-7043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14010814#comment-14010814 ] Vikram Dixit K commented on HIVE-7043: -- +1 How should this be committed? Do you want to raise another jira or revert the commit and re-apply? Thanks Vikram. When using the tez session pool via hive, once sessions time out, all queries go to the default queue - Key: HIVE-7043 URL: https://issues.apache.org/jira/browse/HIVE-7043 Project: Hive Issue Type: Bug Components: HiveServer2 Affects Versions: 0.13.0 Reporter: Vikram Dixit K Assignee: Vikram Dixit K Fix For: 0.14.0 Attachments: HIVE-7043.2.patch, HIVE-7043.3.patch, HIVE-7043.4.patch, HIVE-7043.4.patch, HIVE-7043.followup.patch.txt When using a tez session pool to run multiple queries, once the sessions time out, we always end up using the default queue to launch queries. The load balancing doesn't work in this case. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Comment Edited] (HIVE-7062) Support Streaming mode in Windowing
[ https://issues.apache.org/jira/browse/HIVE-7062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14010809#comment-14010809 ] Harish Butani edited comment on HIVE-7062 at 5/28/14 5:43 AM: -- Has Framework changes + Streaming for Sum and Avg functions + Streaming for Ranking functions. Still need to do Streaming for Min, Max, Lead, Lag, FirstVal, LastVal was (Author: rhbutani): Has Framework changes + Streaming for Sum and Avg functions. Still need to do Streaming for Min, Max, Lead, Lag, FirstVal, LastVal Support Streaming mode in Windowing --- Key: HIVE-7062 URL: https://issues.apache.org/jira/browse/HIVE-7062 Project: Hive Issue Type: Bug Reporter: Harish Butani Assignee: Harish Butani Attachments: HIVE-7062.1.patch, HIVE-7062.4.patch 1. Have the Windowing Table Function support streaming mode. 2. Have special handling for Ranking UDAFs. 3. Have special handling for Sum/Avg for fixed size Wdws. -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: Review Request 21886: Column stats : LOW_VALUE (or HIGH_VALUE) will always be 0.0000 , if all the column values larger than 0.0 (or if all column values smaller than 0.0)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/21886/ --- (Updated May 28, 2014, 5:45 a.m.) Review request for hive. Changes --- Fixed test fails Refactoring Bugs: HIVE-4561 https://issues.apache.org/jira/browse/HIVE-4561 Repository: hive-git Description --- if all column values larger than 0.0 DOUBLE_LOW_VALUE always will be 0.0 or if all column values less than 0.0, DOUBLE_HIGH_VALUE will always be hive (default) create table src_test (price double); hive (default) load data local inpath './test.txt' into table src_test; hive (default) select * from src_test; OK 1.0 2.0 3.0 Time taken: 0.313 seconds, Fetched: 3 row(s) hive (default) analyze table src_test compute statistics for columns price; mysql select * from TAB_COL_STATS \G; CS_ID: 16 DB_NAME: default TABLE_NAME: src_test COLUMN_NAME: price COLUMN_TYPE: double TBL_ID: 2586 LONG_LOW_VALUE: 0 LONG_HIGH_VALUE: 0 DOUBLE_LOW_VALUE: 0. # Wrong Result ! Expected is 1. DOUBLE_HIGH_VALUE: 3. BIG_DECIMAL_LOW_VALUE: NULL BIG_DECIMAL_HIGH_VALUE: NULL NUM_NULLS: 0 NUM_DISTINCTS: 1 AVG_COL_LEN: 0. MAX_COL_LEN: 0 NUM_TRUES: 0 NUM_FALSES: 0 LAST_ANALYZED: 1368596151 2 rows in set (0.00 sec) Diffs (updated) - metastore/if/hive_metastore.thrift eef1b80 metastore/src/gen/thrift/gen-cpp/hive_metastore_types.h 43869c2 metastore/src/gen/thrift/gen-cpp/hive_metastore_types.cpp 9e440bb metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/DecimalColumnStatsData.java 5661252 metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/DoubleColumnStatsData.java d3f3f68 metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/LongColumnStatsData.java 2cf4380 metastore/src/gen/thrift/gen-py/hive_metastore/ttypes.py c4b583b metastore/src/gen/thrift/gen-rb/hive_metastore_types.rb 79b7a1a metastore/src/java/org/apache/hadoop/hive/metastore/StatObjectConverter.java dc0e266 metastore/src/model/org/apache/hadoop/hive/metastore/model/MPartitionColumnStatistics.java f61cdf0 metastore/src/model/org/apache/hadoop/hive/metastore/model/MTableColumnStatistics.java 85f6427 ql/src/java/org/apache/hadoop/hive/ql/exec/ColumnStatsTask.java 3dc02f0 ql/src/java/org/apache/hadoop/hive/ql/optimizer/StatsOptimizer.java ee4d56c ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFComputeStats.java 3b063eb ql/src/java/org/apache/hadoop/hive/ql/udf/generic/NumDistinctValueEstimator.java 24159b8 ql/src/test/queries/clientpositive/metadata_only_queries.q b549a56 ql/src/test/results/clientpositive/compute_stats_empty_table.q.out 50d6c8d ql/src/test/results/clientpositive/compute_stats_long.q.out 2f5cbdd ql/src/test/results/clientpositive/metadata_only_queries.q.out 531ea41 ql/src/test/results/clientpositive/metadata_only_queries_with_filters.q.out c8e2c0c Diff: https://reviews.apache.org/r/21886/diff/ Testing --- Thanks, Navis Ryu
Re: Review Request 21886: Column stats : LOW_VALUE (or HIGH_VALUE) will always be 0.0000 , if all the column values larger than 0.0 (or if all column values smaller than 0.0)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/21886/#review44095 --- Ship it! Thanks, Looks good to me! - Zhuoluo Yang On May 28, 2014, 5:45 a.m., Navis Ryu wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/21886/ --- (Updated May 28, 2014, 5:45 a.m.) Review request for hive. Bugs: HIVE-4561 https://issues.apache.org/jira/browse/HIVE-4561 Repository: hive-git Description --- if all column values larger than 0.0 DOUBLE_LOW_VALUE always will be 0.0 or if all column values less than 0.0, DOUBLE_HIGH_VALUE will always be hive (default) create table src_test (price double); hive (default) load data local inpath './test.txt' into table src_test; hive (default) select * from src_test; OK 1.0 2.0 3.0 Time taken: 0.313 seconds, Fetched: 3 row(s) hive (default) analyze table src_test compute statistics for columns price; mysql select * from TAB_COL_STATS \G; CS_ID: 16 DB_NAME: default TABLE_NAME: src_test COLUMN_NAME: price COLUMN_TYPE: double TBL_ID: 2586 LONG_LOW_VALUE: 0 LONG_HIGH_VALUE: 0 DOUBLE_LOW_VALUE: 0. # Wrong Result ! Expected is 1. DOUBLE_HIGH_VALUE: 3. BIG_DECIMAL_LOW_VALUE: NULL BIG_DECIMAL_HIGH_VALUE: NULL NUM_NULLS: 0 NUM_DISTINCTS: 1 AVG_COL_LEN: 0. MAX_COL_LEN: 0 NUM_TRUES: 0 NUM_FALSES: 0 LAST_ANALYZED: 1368596151 2 rows in set (0.00 sec) Diffs - metastore/if/hive_metastore.thrift eef1b80 metastore/src/gen/thrift/gen-cpp/hive_metastore_types.h 43869c2 metastore/src/gen/thrift/gen-cpp/hive_metastore_types.cpp 9e440bb metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/DecimalColumnStatsData.java 5661252 metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/DoubleColumnStatsData.java d3f3f68 metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/LongColumnStatsData.java 2cf4380 metastore/src/gen/thrift/gen-py/hive_metastore/ttypes.py c4b583b metastore/src/gen/thrift/gen-rb/hive_metastore_types.rb 79b7a1a metastore/src/java/org/apache/hadoop/hive/metastore/StatObjectConverter.java dc0e266 metastore/src/model/org/apache/hadoop/hive/metastore/model/MPartitionColumnStatistics.java f61cdf0 metastore/src/model/org/apache/hadoop/hive/metastore/model/MTableColumnStatistics.java 85f6427 ql/src/java/org/apache/hadoop/hive/ql/exec/ColumnStatsTask.java 3dc02f0 ql/src/java/org/apache/hadoop/hive/ql/optimizer/StatsOptimizer.java ee4d56c ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFComputeStats.java 3b063eb ql/src/java/org/apache/hadoop/hive/ql/udf/generic/NumDistinctValueEstimator.java 24159b8 ql/src/test/queries/clientpositive/metadata_only_queries.q b549a56 ql/src/test/results/clientpositive/compute_stats_empty_table.q.out 50d6c8d ql/src/test/results/clientpositive/compute_stats_long.q.out 2f5cbdd ql/src/test/results/clientpositive/metadata_only_queries.q.out 531ea41 ql/src/test/results/clientpositive/metadata_only_queries_with_filters.q.out c8e2c0c Diff: https://reviews.apache.org/r/21886/diff/ Testing --- Thanks, Navis Ryu
