[jira] [Updated] (HIVE-19733) RemoteSparkJobStatus#getSparkStageProgress inefficient implementation
[
https://issues.apache.org/jira/browse/HIVE-19733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bharathkrishna Guruvayoor Murali updated HIVE-19733:
Attachment: HIVE-19733.2.patch
> RemoteSparkJobStatus#getSparkStageProgress inefficient implementation
> -
>
> Key: HIVE-19733
> URL: https://issues.apache.org/jira/browse/HIVE-19733
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
>Reporter: Sahil Takiar
>Assignee: Bharathkrishna Guruvayoor Murali
>Priority: Major
> Attachments: HIVE-19733.1.patch, HIVE-19733.2.patch
>
>
> The implementation of {{RemoteSparkJobStatus#getSparkStageProgress}} is a bit
> inefficient. There is one RPC call to get the {{SparkJobInfo}} and then for
> every stage there is another RPC call to get each {{SparkStageInfo}}. This
> could all be done in a single RPC call.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20072) Write access being requested when performing select on a table
[
https://issues.apache.org/jira/browse/HIVE-20072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thejas M Nair updated HIVE-20072:
-
Description:
hive.exec.pre.hooks=org.apache.hadoop.hive.ql.hooks.UpdateInputAccessTimeHook$PreExec
along with hive doAs=true
which is causing to perform Alter table operation internally when accessed
Hive table, and results in HiveException with Write access to table directory.
Select access is provided to a table and when performing select * for that
table, below exception is seen.
Steps to reproduce. (with ranger enabled)
1. Create test table
create table lastaccesstime (id int);
2. Enable Hive doAs
3. update
hive.exec.pre.hooks=org.apache.hadoop.hive.ql.hooks.ATSHook,org.apache.hadoop.hive.ql.hooks.UpdateInputAccessTimeHook$PreExec
4. Perform select on table lastaccesstime as non-hive user and exception will
be thrown.
{code:java}
0: jdbc:hive2://example.com:> select * from lastaccesstime;
Error: Error while processing statement: FAILED: Hive Internal Error:
org.apache.hadoop.hive.ql.metadata.HiveException(Unable to alter table.
java.security.AccessControlException: Permission denied: user=kafka,
access=WRITE,
inode="/apps/hive/warehouse/testdb.db/lastaccesstime":hive:hadoop:drwxr-x---
at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:353)
at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:252)
at
org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer$RangerAccessControlEnforcer.checkDefaultEnforcer(RangerHdfsAuthorizer.java:427)
at
org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer$RangerAccessControlEnforcer.checkPermission(RangerHdfsAuthorizer.java:303)
at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:190)
at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1950)
at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1934)
at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPathAccess(FSDirectory.java:1908)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkAccess(FSNamesystem.java:8800)
at
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.checkAccess(NameNodeRpcServer.java:2089)
at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.checkAccess(ClientNamenodeProtocolServerSideTranslatorPB.java:1466)
at
org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:640)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:982)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2351)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2347)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1869)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2347)
) (state=08S01,code=12)
{code}
was:
hive.exec.pre.hooks=org.apache.hadoop.hive.ql.hooks.UpdateInputAccessTimeHook$PreExec
along with hive doAs=true
which is causing to perform Alter table operation internally when accessed
Hive table, and results in HiveException with Write access to table directory.
Select access is provided to a table and when performing select * for that
table, below exception is seen.
Steps to reproduce. (with ranger enabled)
1. Create test table
create table lastaccesstime (id int);
2. Enable Hive doAs
3. update
hive.exec.pre.hooks=org.apache.hadoop.hive.ql.hooks.ATSHook,org.apache.hadoop.hive.ql.hooks.UpdateInputAccessTimeHook$PreExec
4. Perform select on table lastaccesstime as non-hive user and exception will
be thrown.
{code:java}
0: jdbc:hive2://c470-node4.squadron-labs.com:> select * from lastaccesstime;
Error: Error while processing statement: FAILED: Hive Internal Error:
org.apache.hadoop.hive.ql.metadata.HiveException(Unable to alter table.
java.security.AccessControlException: Permission denied: user=kafka,
access=WRITE,
inode="/apps/hive/warehouse/testdb.db/lastaccesstime":hive:hadoop:drwxr-x---
at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:353)
at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:252)
at
org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer$RangerAccessControlEnforcer.checkDefaultEnforcer(RangerHdfsAuthorizer.java:427)
at
org.apache.ranger.authorization.
[jira] [Commented] (HIVE-20072) Write access being requested when performing select on a table
[
https://issues.apache.org/jira/browse/HIVE-20072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16533329#comment-16533329
]
Thejas M Nair commented on HIVE-20072:
--
Patch looks good.
Can you also add a debug level log message when the exception is ignored ?
> Write access being requested when performing select on a table
> --
>
> Key: HIVE-20072
> URL: https://issues.apache.org/jira/browse/HIVE-20072
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Reporter: mahesh kumar behera
>Assignee: mahesh kumar behera
>Priority: Critical
> Attachments: HIVE-20072.01.patch
>
>
> hive.exec.pre.hooks=org.apache.hadoop.hive.ql.hooks.UpdateInputAccessTimeHook$PreExec
> along with hive doAs=true
> which is causing to perform Alter table operation internally when accessed
> Hive table, and results in HiveException with Write access to table directory.
> Select access is provided to a table and when performing select * for that
> table, below exception is seen.
>
> Steps to reproduce. (with ranger enabled)
> 1. Create test table
> create table lastaccesstime (id int);
> 2. Enable Hive doAs
> 3. update
> hive.exec.pre.hooks=org.apache.hadoop.hive.ql.hooks.ATSHook,org.apache.hadoop.hive.ql.hooks.UpdateInputAccessTimeHook$PreExec
> 4. Perform select on table lastaccesstime as non-hive user and exception will
> be thrown.
> {code:java}
> 0: jdbc:hive2://example.com:> select * from lastaccesstime;
> Error: Error while processing statement: FAILED: Hive Internal Error:
> org.apache.hadoop.hive.ql.metadata.HiveException(Unable to alter table.
> java.security.AccessControlException: Permission denied: user=kafka,
> access=WRITE,
> inode="/apps/hive/warehouse/testdb.db/lastaccesstime":hive:hadoop:drwxr-x---
> at
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:353)
> at
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:252)
> at
> org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer$RangerAccessControlEnforcer.checkDefaultEnforcer(RangerHdfsAuthorizer.java:427)
> at
> org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer$RangerAccessControlEnforcer.checkPermission(RangerHdfsAuthorizer.java:303)
> at
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:190)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1950)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1934)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPathAccess(FSDirectory.java:1908)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkAccess(FSNamesystem.java:8800)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.checkAccess(NameNodeRpcServer.java:2089)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.checkAccess(ClientNamenodeProtocolServerSideTranslatorPB.java:1466)
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:640)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:982)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2351)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2347)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1869)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2347)
> ) (state=08S01,code=12)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20039) Bucket pruning: Left Outer Join on bucketed table gives wrong result
[ https://issues.apache.org/jira/browse/HIVE-20039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deepak Jaiswal updated HIVE-20039: -- Resolution: Fixed Status: Resolved (was: Patch Available) Committed to branch-3. > Bucket pruning: Left Outer Join on bucketed table gives wrong result > - > > Key: HIVE-20039 > URL: https://issues.apache.org/jira/browse/HIVE-20039 > Project: Hive > Issue Type: Bug >Affects Versions: 3.0.0, 2.3.2 >Reporter: Deepak Jaiswal >Assignee: Deepak Jaiswal >Priority: Major > Fix For: 4.0.0, 3.2.0 > > Attachments: HIVE-20039.01-branch-3.patch, HIVE-20039.1.patch, > HIVE-20039.2.patch, HIVE-20039.3.patch, HIVE-20039.4.patch > > > Left outer join on bucketed table on certain cases gives wrong results. > Depending on the order in which the table-scans are walked through, the > FilterPruner might end up using the wrong table scan's table properties on > the other table. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20066) hive.load.data.owner is compared to full principal
[
https://issues.apache.org/jira/browse/HIVE-20066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16533350#comment-16533350
]
Daniel Voros commented on HIVE-20066:
-
Thank you all!
> hive.load.data.owner is compared to full principal
> --
>
> Key: HIVE-20066
> URL: https://issues.apache.org/jira/browse/HIVE-20066
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.0, 4.0.0
>Reporter: Daniel Voros
>Assignee: Daniel Voros
>Priority: Major
> Fix For: 4.0.0, 3.2.0
>
> Attachments: HIVE-20066.1.patch
>
>
> HIVE-19928 compares the user running HS2 to the configured owner
> (hive.load.data.owner) to check if we're able to move the file with LOAD DATA
> or need to copy.
> This check compares the full username (that may contain the full kerberos
> principal) to hive.load.data.owner. We should compare to the short username
> ({{UGI.getShortUserName()}}) instead. That's used in similar context
> [here|https://github.com/apache/hive/blob/f519db7eafacb4b4d2d9fe2a9e10e908d8077224/common/src/java/org/apache/hadoop/hive/common/FileUtils.java#L398].
> cc [~djaiswal]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20025) Clean-up of event files created by HiveProtoLoggingHook.
[
https://issues.apache.org/jira/browse/HIVE-20025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16533396#comment-16533396
]
Hive QA commented on HIVE-20025:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12930254/HIVE-20025.01-branch-3.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 17 failed/errored test(s), 14358 tests
executed
*Failed tests:*
{noformat}
TestAlterTableMetadata - did not produce a TEST-*.xml file (likely timed out)
(batchId=236)
TestAutoPurgeTables - did not produce a TEST-*.xml file (likely timed out)
(batchId=236)
TestBeeLineDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=257)
TestClearDanglingScratchDir - did not produce a TEST-*.xml file (likely timed
out) (batchId=236)
TestCopyUtils - did not produce a TEST-*.xml file (likely timed out)
(batchId=236)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=257)
TestLocationQueries - did not produce a TEST-*.xml file (likely timed out)
(batchId=236)
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=257)
TestMiniDruidKafkaCliDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=257)
TestReplicationScenariosAcidTables - did not produce a TEST-*.xml file (likely
timed out) (batchId=236)
TestReplicationScenariosAcrossInstances - did not produce a TEST-*.xml file
(likely timed out) (batchId=236)
TestSemanticAnalyzerHookLoading - did not produce a TEST-*.xml file (likely
timed out) (batchId=236)
TestSparkStatistics - did not produce a TEST-*.xml file (likely timed out)
(batchId=236)
TestTezPerfCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=257)
org.apache.hadoop.hive.ql.TestWarehouseExternalDir.testManagedPaths
(batchId=234)
org.apache.hive.service.TestHS2ImpersonationWithRemoteMS.testImpersonation
(batchId=242)
org.apache.hive.spark.client.rpc.TestRpc.testServerPort (batchId=309)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/12396/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12396/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12396/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 17 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12930254 - PreCommit-HIVE-Build
> Clean-up of event files created by HiveProtoLoggingHook.
>
>
> Key: HIVE-20025
> URL: https://issues.apache.org/jira/browse/HIVE-20025
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 3.0.0
>Reporter: Sankar Hariappan
>Assignee: Sankar Hariappan
>Priority: Major
> Labels: Hive, hooks, pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-20025.01-branch-3.patch, HIVE-20025.01.patch,
> HIVE-20025.02.patch, HIVE-20025.03.patch, HIVE-20025.04.patch
>
>
> Currently, HiveProtoLoggingHook write event data to hdfs. The number of files
> can grow to very large numbers.
> Since the files are created under a folder with Date being a part of the
> path, hive should have a way to clean up data older than a certain configured
> time / date. This can be a job that can run with as little frequency as just
> once a day.
> This time should be set to 1 week default. There should also be a sane upper
> bound of # of files so that when a large cluster generates a lot of files
> during a spike, we don't force the cluster fall over.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20089) CTAS doesn't work into nonexisting /tmp/... directory while CT works
[
https://issues.apache.org/jira/browse/HIVE-20089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Laszlo Bodor updated HIVE-20089:
Description:
While checking negative qtests I've found some strange behavior according to CT
and CTAS statements.
ct_noperm_loc.q
ctas_noperm_loc.q
The common part these tests are initialization:
{code:java}
set hive.test.authz.sstd.hs2.mode=true;
set
hive.security.authorization.manager=org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.SQLStdHiveAuthorizerFactoryForTest;
set
hive.security.authenticator.manager=org.apache.hadoop.hive.ql.security.SessionStateConfigUserAuthenticator;
set hive.security.authorization.enabled=true;
set user.name=user1;
{code}
But while simple 'create table' works to a nonexisting dir...
{code:java}
create table foo0(id int) location 'hdfs:///tmp/ct_noperm_loc_foo0';
{code}
...'create table as select' doesn't work:
{code:java}
create table foo0 location 'hdfs:///tmp/ctas_noperm_loc_foo0' as select 1 as c1;
{code}
expected result in a negative test is:
{code:java}
FAILED: HiveAccessControlException Permission denied: Principal [name=user1,
type=USER] does not have following privileges for operation
CREATETABLE_AS_SELECT [[INSERT, DELETE] on Object [type=DFS_URI,
name=hdfs://### HDFS PATH ###]]
{code}
So it passes, however it should work same as simple CT, isn't it?
Is it by design, or am I missing something here?
{code:java}
mvn test -Dtest=TestNegativeMinimrCliDriver -Dqfile=ct_noperm_loc.q
-Pitests,hadoop-2 -pl itests/qtest
mvn test -Dtest=TestNegativeMinimrCliDriver -Dqfile=ctas_noperm_loc.q
-Pitests,hadoop-2 -pl itests/qtest
{code}
was:
While checking negative qtests I've found some strange behavior according to CT
and CTAS statements.
ct_noperm_loc.q
ctas_noperm_loc.q
The common part these tests are initialization:
{code}
set hive.test.authz.sstd.hs2.mode=true;
set
hive.security.authorization.manager=org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.SQLStdHiveAuthorizerFactoryForTest;
set
hive.security.authenticator.manager=org.apache.hadoop.hive.ql.security.SessionStateConfigUserAuthenticator;
set hive.security.authorization.enabled=true;
set user.name=user1;
{code}
But while simple 'create table' works to a nonexisting dir...
{code}
create table foo0(id int) location 'hdfs:///tmp/ct_noperm_loc_foo0';
{code}
...'create table as select' doesn't work:
{code}
create table foo0 location 'hdfs:///tmp/ctas_noperm_loc_foo0' as select 1 as c1;
{code}
expected result is:
{code}
FAILED: HiveAccessControlException Permission denied: Principal [name=user1,
type=USER] does not have following privileges for operation
CREATETABLE_AS_SELECT [[INSERT, DELETE] on Object [type=DFS_URI,
name=hdfs://### HDFS PATH ###]]
{code}
Is it by design, am I missing something here?
{code}
mvn test -Dtest=TestNegativeMinimrCliDriver -Dqfile=ct_noperm_loc.q
-Pitests,hadoop-2 -pl itests/qtest
mvn test -Dtest=TestNegativeMinimrCliDriver -Dqfile=ctas_noperm_loc.q
-Pitests,hadoop-2 -pl itests/qtest
{code}
> CTAS doesn't work into nonexisting /tmp/... directory while CT works
>
>
> Key: HIVE-20089
> URL: https://issues.apache.org/jira/browse/HIVE-20089
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Laszlo Bodor
>Priority: Major
>
> While checking negative qtests I've found some strange behavior according to
> CT and CTAS statements.
> ct_noperm_loc.q
> ctas_noperm_loc.q
> The common part these tests are initialization:
> {code:java}
> set hive.test.authz.sstd.hs2.mode=true;
> set
> hive.security.authorization.manager=org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.SQLStdHiveAuthorizerFactoryForTest;
> set
> hive.security.authenticator.manager=org.apache.hadoop.hive.ql.security.SessionStateConfigUserAuthenticator;
> set hive.security.authorization.enabled=true;
> set user.name=user1;
> {code}
>
> But while simple 'create table' works to a nonexisting dir...
> {code:java}
> create table foo0(id int) location 'hdfs:///tmp/ct_noperm_loc_foo0';
> {code}
> ...'create table as select' doesn't work:
> {code:java}
> create table foo0 location 'hdfs:///tmp/ctas_noperm_loc_foo0' as select 1 as
> c1;
> {code}
> expected result in a negative test is:
> {code:java}
> FAILED: HiveAccessControlException Permission denied: Principal [name=user1,
> type=USER] does not have following privileges for operation
> CREATETABLE_AS_SELECT [[INSERT, DELETE] on Object [type=DFS_URI,
> name=hdfs://### HDFS PATH ###]]
> {code}
> So it passes, however it should work same as simple CT, isn't it?
> Is it by design, or am I missing something here?
> {code:java}
> mvn test -Dtest=TestNegativeMinimrCliDriver -Dqfile=ct_noperm_loc.q
> -Pitests,hadoop-2 -pl itests/qtest
> mvn t
[jira] [Commented] (HIVE-20041) ResultsCache: Improve loggng for concurrent queries
[
https://issues.apache.org/jira/browse/HIVE-20041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16533413#comment-16533413
]
Hive QA commented on HIVE-20041:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
6s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
15s{color} | {color:blue} ql in master has 2287 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 44s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-12397/dev-support/hive-personality.sh
|
| git revision | master / e865b44 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12397/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> ResultsCache: Improve loggng for concurrent queries
> ---
>
> Key: HIVE-20041
> URL: https://issues.apache.org/jira/browse/HIVE-20041
> Project: Hive
> Issue Type: Improvement
> Components: Diagnosability
>Reporter: Gopal V
>Assignee: Laszlo Bodor
>Priority: Minor
> Attachments: HIVE-20041.01.patch, HIVE-20041.02.patch,
> HIVE-20041.03.patch
>
>
> The logging for QueryResultsCache ends up printing information without
> context, like
> {code}
> 2018-06-30T17:48:45,502 INFO [HiveServer2-Background-Pool: Thread-166]
> results.QueryResultsCache: Waiting on pending cacheEntry
> {code}
> {code}
> 2018-06-30T17:50:17,963 INFO [HiveServer2-Background-Pool: Thread-145]
> ql.Driver: savedToCache: true
> {code}
> The previous lines for this are in DEBUG level, so the logging ends up being
> useless at INFO level to debug.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20041) ResultsCache: Improve loggng for concurrent queries
[
https://issues.apache.org/jira/browse/HIVE-20041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16533448#comment-16533448
]
Hive QA commented on HIVE-20041:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12930258/HIVE-20041.03.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 14633 tests
executed
*Failed tests:*
{noformat}
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=190)
[druidmini_dynamic_partition.q,druidmini_expressions.q,druidmini_test_alter.q,druidmini_test1.q,druidmini_test_insert.q]
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/12397/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12397/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12397/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12930258 - PreCommit-HIVE-Build
> ResultsCache: Improve loggng for concurrent queries
> ---
>
> Key: HIVE-20041
> URL: https://issues.apache.org/jira/browse/HIVE-20041
> Project: Hive
> Issue Type: Improvement
> Components: Diagnosability
>Reporter: Gopal V
>Assignee: Laszlo Bodor
>Priority: Minor
> Attachments: HIVE-20041.01.patch, HIVE-20041.02.patch,
> HIVE-20041.03.patch
>
>
> The logging for QueryResultsCache ends up printing information without
> context, like
> {code}
> 2018-06-30T17:48:45,502 INFO [HiveServer2-Background-Pool: Thread-166]
> results.QueryResultsCache: Waiting on pending cacheEntry
> {code}
> {code}
> 2018-06-30T17:50:17,963 INFO [HiveServer2-Background-Pool: Thread-145]
> ql.Driver: savedToCache: true
> {code}
> The previous lines for this are in DEBUG level, so the logging ends up being
> useless at INFO level to debug.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-20093) LlapOutputFomatService: Use ArrowBuf with Netty for Accounting
[
https://issues.apache.org/jira/browse/HIVE-20093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eric Wohlstadter reassigned HIVE-20093:
---
> LlapOutputFomatService: Use ArrowBuf with Netty for Accounting
> --
>
> Key: HIVE-20093
> URL: https://issues.apache.org/jira/browse/HIVE-20093
> Project: Hive
> Issue Type: Bug
>Reporter: Eric Wohlstadter
>Assignee: Eric Wohlstadter
>Priority: Major
>
> Combining {{Unpooled.wrappedBuffer}} with Arrow buffers can create corrupted
> buffers from buffer reuse race-condition.
> This change ensures Arrow memory to be accounted by the same BufferAllocator.
> RootAllocator will return an ArrowBuf which cooperates with Arrow memory
> arrow accounting after Netty {{release(1)}} the buffer.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20065) metastore should not rely on jackson 1.x
[
https://issues.apache.org/jira/browse/HIVE-20065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16533472#comment-16533472
]
Hive QA commented on HIVE-20065:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
58s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
45s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
20s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
19s{color} | {color:blue} standalone-metastore in master has 228 extant
Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
30s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
22s{color} | {color:green} standalone-metastore: The patch generated 0 new +
231 unchanged - 27 fixed = 231 total (was 258) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m
43s{color} | {color:red} standalone-metastore generated 1 new + 227 unchanged -
1 fixed = 228 total (was 228) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
15s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 21m 51s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:standalone-metastore |
| | input must be non-null but is marked as nullable At
JSONMessageFactory.java:is marked as nullable At JSONMessageFactory.java:[line
397] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-12398/dev-support/hive-personality.sh
|
| git revision | master / e865b44 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12398/yetus/new-findbugs-standalone-metastore.html
|
| modules | C: standalone-metastore U: standalone-metastore |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12398/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> metastore should not rely on jackson 1.x
>
>
> Key: HIVE-20065
> URL: https://issues.apache.org/jira/browse/HIVE-20065
> Project: Hive
> Issue Type: Bug
> Components: Standalone Metastore
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-20065.01.patch, HIVE-20065.02.patch
>
>
> somehow jackson 1.x is on the classpath in some ide-s ...and somehow 1.x
> org.codehaus jackson is being used from a dozen of classes - meanwhile the
> pom.xml doesn't mention it at all; but only a fasterxml's 2.9.0
> I don't know where it gets the jackson 1.x implementation; but I think it
> shouldn't rely on that...
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20093) LlapOutputFomatService: Use ArrowBuf with Netty for Accounting
[
https://issues.apache.org/jira/browse/HIVE-20093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eric Wohlstadter updated HIVE-20093:
Attachment: HIVE-20093.1.patch
> LlapOutputFomatService: Use ArrowBuf with Netty for Accounting
> --
>
> Key: HIVE-20093
> URL: https://issues.apache.org/jira/browse/HIVE-20093
> Project: Hive
> Issue Type: Bug
>Reporter: Eric Wohlstadter
>Assignee: Eric Wohlstadter
>Priority: Major
> Attachments: HIVE-20093.1.patch
>
>
> Combining {{Unpooled.wrappedBuffer}} with Arrow buffers can create corrupted
> buffers from buffer reuse race-condition.
> This change ensures Arrow memory to be accounted by the same BufferAllocator.
> RootAllocator will return an ArrowBuf which cooperates with Arrow memory
> arrow accounting after Netty {{release(1)}} the buffer.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20093) LlapOutputFomatService: Use ArrowBuf with Netty for Accounting
[
https://issues.apache.org/jira/browse/HIVE-20093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eric Wohlstadter updated HIVE-20093:
Status: Patch Available (was: Open)
> LlapOutputFomatService: Use ArrowBuf with Netty for Accounting
> --
>
> Key: HIVE-20093
> URL: https://issues.apache.org/jira/browse/HIVE-20093
> Project: Hive
> Issue Type: Bug
>Reporter: Eric Wohlstadter
>Assignee: Eric Wohlstadter
>Priority: Major
> Attachments: HIVE-20093.1.patch
>
>
> Combining {{Unpooled.wrappedBuffer}} with Arrow buffers can create corrupted
> buffers from buffer reuse race-condition.
> This change ensures Arrow memory to be accounted by the same BufferAllocator.
> RootAllocator will return an ArrowBuf which cooperates with Arrow memory
> arrow accounting after Netty {{release(1)}} the buffer.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20085) Druid-Hive (managed) table creation fails with strict managed table checks: Table is marked as a managed table but is not transactional
[
https://issues.apache.org/jira/browse/HIVE-20085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16533495#comment-16533495
]
Nishant Bangarwa commented on HIVE-20085:
-
[~ashutoshc] https://reviews.apache.org/r/67833/
> Druid-Hive (managed) table creation fails with strict managed table checks:
> Table is marked as a managed table but is not transactional
> ---
>
> Key: HIVE-20085
> URL: https://issues.apache.org/jira/browse/HIVE-20085
> Project: Hive
> Issue Type: Bug
> Components: Hive, StorageHandler
>Affects Versions: 3.0.0
>Reporter: Dileep Kumar Chiguruvada
>Assignee: Nishant Bangarwa
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-20085.1.patch, HIVE-20085.2.patch,
> HIVE-20085.3.patch, HIVE-20085.patch
>
>
> Druid-Hive (managed) table creation fails with strict managed table checks:
> Table is marked as a managed table but is not transactional
> {code}
> drop table if exists calcs;
> create table calcs
> STORED BY 'org.apache.hadoop.hive.druid.DruidStorageHandler'
> TBLPROPERTIES (
> "druid.segment.granularity" = "MONTH",
> "druid.query.granularity" = "DAY")
> AS SELECT
> cast(datetime0 as timestamp with local time zone) `__time`,
> key,
> str0, str1, str2, str3,
> date0, date1, date2, date3,
> time0, time1,
> datetime0, datetime1,
> zzz,
> cast(bool0 as string) bool0,
> cast(bool1 as string) bool1,
> cast(bool2 as string) bool2,
> cast(bool3 as string) bool3,
> int0, int1, int2, int3,
> num0, num1, num2, num3, num4
> from tableau_orc.calcs;
> 2018-07-03 04:57:31,911|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : Status: Running
> (Executing on YARN cluster with App id application_1530592209763_0009)
> ...
> ...
> 2018-07-03 04:57:36,334|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : SHUFFLE_BYTES_TO_MEM: > 0
> 2018-07-03 04:57:36,334|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : SHUFFLE_PHASE_TIME:
> 330
> 2018-07-03 04:57:36,334|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : SPILLED_RECORDS: 17
> 2018-07-03 04:57:36,334|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO :
> TaskCounter_Reducer_2_OUTPUT_out_Reducer_2:
> 2018-07-03 04:57:36,335|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : OUTPUT_RECORDS: 0
> 2018-07-03 04:57:36,335|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO :
> org.apache.hadoop.hive.llap.counters.LlapWmCounters:
> 2018-07-03 04:57:36,335|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : GUARANTEED_QUEUED_NS: > 0
> 2018-07-03 04:57:36,335|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO :
> GUARANTEED_RUNNING_NS: 0
> 2018-07-03 04:57:36,335|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO :
> SPECULATIVE_QUEUED_NS: 2162643606
> 2018-07-03 04:57:36,335|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO :
> SPECULATIVE_RUNNING_NS: 12151664909
> 2018-07-03 04:57:36,335|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : Starting task
> [Stage-2:DEPENDENCY_COLLECTION] in serial mode
> 2018-07-03 04:57:36,335|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : Starting task
> [Stage-0:MOVE] in serial mode
> 2018-07-03 04:57:36,336|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : Moving data to
> directory
> hdfs://mycluster/warehouse/tablespace/managed/hive/druid_tableau.db/calcs
> from
> hdfs://mycluster/warehouse/tablespace/managed/hive/druid_tableau.db/.hive-staging_hive_2018-07-03_04-57-27_351_7124633902209008283-3/-ext-10002
> 2018-07-03 04:57:36,336|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : Starting task
> [Stage-4:DDL] in serial mode
> 2018-07-03 04:57:36,336|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|ERROR : FAILED: Execution
> Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask.
> MetaException(message:Table druid_tableau.calcs failed strict managed table
> checks due to the following reason: Table is marked as a managed table but is
> not transactional.)
> 2018-07-03 04:57:36,336|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : Completed executing
> command(queryId=hive_201
[jira] [Updated] (HIVE-20035) write booleans as long when serializing to druid
[ https://issues.apache.org/jira/browse/HIVE-20035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishant Bangarwa updated HIVE-20035: Attachment: HIVE-20035.1.patch > write booleans as long when serializing to druid > > > Key: HIVE-20035 > URL: https://issues.apache.org/jira/browse/HIVE-20035 > Project: Hive > Issue Type: Bug >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Attachments: HIVE-20035.1.patch, HIVE-20035.1.patch, HIVE-20035.patch > > > Druid expressions do not support booleans yet. > In druid expressions booleans are treated and parsed from longs, however when > we store booleans from hive they are serialized as 'true' and 'false' string > values. > Need to make serialization consistent with deserialization and write long > values when sending data to druid. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20035) write booleans as long when serializing to druid
[ https://issues.apache.org/jira/browse/HIVE-20035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishant Bangarwa updated HIVE-20035: Attachment: HIVE-20035.2.patch > write booleans as long when serializing to druid > > > Key: HIVE-20035 > URL: https://issues.apache.org/jira/browse/HIVE-20035 > Project: Hive > Issue Type: Bug >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Attachments: HIVE-20035.1.patch, HIVE-20035.1.patch, > HIVE-20035.2.patch, HIVE-20035.patch > > > Druid expressions do not support booleans yet. > In druid expressions booleans are treated and parsed from longs, however when > we store booleans from hive they are serialized as 'true' and 'false' string > values. > Need to make serialization consistent with deserialization and write long > values when sending data to druid. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20035) write booleans as long when serializing to druid
[ https://issues.apache.org/jira/browse/HIVE-20035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishant Bangarwa updated HIVE-20035: Status: Patch Available (was: Open) > write booleans as long when serializing to druid > > > Key: HIVE-20035 > URL: https://issues.apache.org/jira/browse/HIVE-20035 > Project: Hive > Issue Type: Bug >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Attachments: HIVE-20035.1.patch, HIVE-20035.1.patch, > HIVE-20035.2.patch, HIVE-20035.patch > > > Druid expressions do not support booleans yet. > In druid expressions booleans are treated and parsed from longs, however when > we store booleans from hive they are serialized as 'true' and 'false' string > values. > Need to make serialization consistent with deserialization and write long > values when sending data to druid. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20035) write booleans as long when serializing to druid
[ https://issues.apache.org/jira/browse/HIVE-20035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishant Bangarwa updated HIVE-20035: Status: Open (was: Patch Available) > write booleans as long when serializing to druid > > > Key: HIVE-20035 > URL: https://issues.apache.org/jira/browse/HIVE-20035 > Project: Hive > Issue Type: Bug >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Attachments: HIVE-20035.1.patch, HIVE-20035.1.patch, > HIVE-20035.2.patch, HIVE-20035.patch > > > Druid expressions do not support booleans yet. > In druid expressions booleans are treated and parsed from longs, however when > we store booleans from hive they are serialized as 'true' and 'false' string > values. > Need to make serialization consistent with deserialization and write long > values when sending data to druid. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20085) Druid-Hive (managed) table creation fails with strict managed table checks: Table is marked as a managed table but is not transactional
[
https://issues.apache.org/jira/browse/HIVE-20085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nishant Bangarwa updated HIVE-20085:
Attachment: HIVE-20085.3.patch
> Druid-Hive (managed) table creation fails with strict managed table checks:
> Table is marked as a managed table but is not transactional
> ---
>
> Key: HIVE-20085
> URL: https://issues.apache.org/jira/browse/HIVE-20085
> Project: Hive
> Issue Type: Bug
> Components: Hive, StorageHandler
>Affects Versions: 3.0.0
>Reporter: Dileep Kumar Chiguruvada
>Assignee: Nishant Bangarwa
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-20085.1.patch, HIVE-20085.2.patch,
> HIVE-20085.3.patch, HIVE-20085.patch
>
>
> Druid-Hive (managed) table creation fails with strict managed table checks:
> Table is marked as a managed table but is not transactional
> {code}
> drop table if exists calcs;
> create table calcs
> STORED BY 'org.apache.hadoop.hive.druid.DruidStorageHandler'
> TBLPROPERTIES (
> "druid.segment.granularity" = "MONTH",
> "druid.query.granularity" = "DAY")
> AS SELECT
> cast(datetime0 as timestamp with local time zone) `__time`,
> key,
> str0, str1, str2, str3,
> date0, date1, date2, date3,
> time0, time1,
> datetime0, datetime1,
> zzz,
> cast(bool0 as string) bool0,
> cast(bool1 as string) bool1,
> cast(bool2 as string) bool2,
> cast(bool3 as string) bool3,
> int0, int1, int2, int3,
> num0, num1, num2, num3, num4
> from tableau_orc.calcs;
> 2018-07-03 04:57:31,911|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : Status: Running
> (Executing on YARN cluster with App id application_1530592209763_0009)
> ...
> ...
> 2018-07-03 04:57:36,334|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : SHUFFLE_BYTES_TO_MEM: > 0
> 2018-07-03 04:57:36,334|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : SHUFFLE_PHASE_TIME:
> 330
> 2018-07-03 04:57:36,334|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : SPILLED_RECORDS: 17
> 2018-07-03 04:57:36,334|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO :
> TaskCounter_Reducer_2_OUTPUT_out_Reducer_2:
> 2018-07-03 04:57:36,335|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : OUTPUT_RECORDS: 0
> 2018-07-03 04:57:36,335|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO :
> org.apache.hadoop.hive.llap.counters.LlapWmCounters:
> 2018-07-03 04:57:36,335|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : GUARANTEED_QUEUED_NS: > 0
> 2018-07-03 04:57:36,335|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO :
> GUARANTEED_RUNNING_NS: 0
> 2018-07-03 04:57:36,335|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO :
> SPECULATIVE_QUEUED_NS: 2162643606
> 2018-07-03 04:57:36,335|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO :
> SPECULATIVE_RUNNING_NS: 12151664909
> 2018-07-03 04:57:36,335|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : Starting task
> [Stage-2:DEPENDENCY_COLLECTION] in serial mode
> 2018-07-03 04:57:36,335|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : Starting task
> [Stage-0:MOVE] in serial mode
> 2018-07-03 04:57:36,336|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : Moving data to
> directory
> hdfs://mycluster/warehouse/tablespace/managed/hive/druid_tableau.db/calcs
> from
> hdfs://mycluster/warehouse/tablespace/managed/hive/druid_tableau.db/.hive-staging_hive_2018-07-03_04-57-27_351_7124633902209008283-3/-ext-10002
> 2018-07-03 04:57:36,336|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : Starting task
> [Stage-4:DDL] in serial mode
> 2018-07-03 04:57:36,336|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|ERROR : FAILED: Execution
> Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask.
> MetaException(message:Table druid_tableau.calcs failed strict managed table
> checks due to the following reason: Table is marked as a managed table but is
> not transactional.)
> 2018-07-03 04:57:36,336|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : Completed executing
> command(queryId=hive_20180703045727_c39c40d2-7d4a-46c7-a36d-7925e7c4a788);
> Time taken: 6.794
[jira] [Commented] (HIVE-20065) metastore should not rely on jackson 1.x
[
https://issues.apache.org/jira/browse/HIVE-20065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16533513#comment-16533513
]
Hive QA commented on HIVE-20065:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12930263/HIVE-20065.02.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 14638 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/12398/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12398/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12398/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12930263 - PreCommit-HIVE-Build
> metastore should not rely on jackson 1.x
>
>
> Key: HIVE-20065
> URL: https://issues.apache.org/jira/browse/HIVE-20065
> Project: Hive
> Issue Type: Bug
> Components: Standalone Metastore
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-20065.01.patch, HIVE-20065.02.patch
>
>
> somehow jackson 1.x is on the classpath in some ide-s ...and somehow 1.x
> org.codehaus jackson is being used from a dozen of classes - meanwhile the
> pom.xml doesn't mention it at all; but only a fasterxml's 2.9.0
> I don't know where it gets the jackson 1.x implementation; but I think it
> shouldn't rely on that...
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20041) ResultsCache: Improve loggng for concurrent queries
[
https://issues.apache.org/jira/browse/HIVE-20041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16533526#comment-16533526
]
Laszlo Bodor commented on HIVE-20041:
-
Druid test timeouts. What do you think [~gopalv] ? Is it ready to commit or
need to keep on patching?
> ResultsCache: Improve loggng for concurrent queries
> ---
>
> Key: HIVE-20041
> URL: https://issues.apache.org/jira/browse/HIVE-20041
> Project: Hive
> Issue Type: Improvement
> Components: Diagnosability
>Reporter: Gopal V
>Assignee: Laszlo Bodor
>Priority: Minor
> Attachments: HIVE-20041.01.patch, HIVE-20041.02.patch,
> HIVE-20041.03.patch
>
>
> The logging for QueryResultsCache ends up printing information without
> context, like
> {code}
> 2018-06-30T17:48:45,502 INFO [HiveServer2-Background-Pool: Thread-166]
> results.QueryResultsCache: Waiting on pending cacheEntry
> {code}
> {code}
> 2018-06-30T17:50:17,963 INFO [HiveServer2-Background-Pool: Thread-145]
> ql.Driver: savedToCache: true
> {code}
> The previous lines for this are in DEBUG level, so the logging ends up being
> useless at INFO level to debug.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20067) fix InsertEvent on mm tables to not cause failing capability checks
[
https://issues.apache.org/jira/browse/HIVE-20067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16533550#comment-16533550
]
Hive QA commented on HIVE-20067:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
31s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
6s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
47s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
57s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
54s{color} | {color:blue} standalone-metastore in master has 228 extant
Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
59s{color} | {color:blue} ql in master has 2287 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
12s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
22s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
40s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
20s{color} | {color:red} standalone-metastore: The patch generated 1 new + 381
unchanged - 1 fixed = 382 total (was 382) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 7m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
8s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 34m 58s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-12399/dev-support/hive-personality.sh
|
| git revision | master / e865b44 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12399/yetus/diff-checkstyle-standalone-metastore.txt
|
| modules | C: standalone-metastore ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12399/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> fix InsertEvent on mm tables to not cause failing capability checks
> ---
>
> Key: HIVE-20067
> URL: https://issues.apache.org/jira/browse/HIVE-20067
> Project: Hive
> Issue Type: Bug
> Components: Standalone Metastore
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-20067.01.patch, HIVE-20067.02.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20025) Clean-up of event files created by HiveProtoLoggingHook.
[ https://issues.apache.org/jira/browse/HIVE-20025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16533559#comment-16533559 ] Sankar Hariappan commented on HIVE-20025: - Test failures are irrelevant. 01-branch-3.patch is committed to branch-3. > Clean-up of event files created by HiveProtoLoggingHook. > > > Key: HIVE-20025 > URL: https://issues.apache.org/jira/browse/HIVE-20025 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Affects Versions: 3.0.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan >Priority: Major > Labels: Hive, hooks, pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-20025.01-branch-3.patch, HIVE-20025.01.patch, > HIVE-20025.02.patch, HIVE-20025.03.patch, HIVE-20025.04.patch > > > Currently, HiveProtoLoggingHook write event data to hdfs. The number of files > can grow to very large numbers. > Since the files are created under a folder with Date being a part of the > path, hive should have a way to clean up data older than a certain configured > time / date. This can be a job that can run with as little frequency as just > once a day. > This time should be set to 1 week default. There should also be a sane upper > bound of # of files so that when a large cluster generates a lot of files > during a spike, we don't force the cluster fall over. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20025) Clean-up of event files created by HiveProtoLoggingHook.
[ https://issues.apache.org/jira/browse/HIVE-20025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-20025: Resolution: Fixed Status: Resolved (was: Patch Available) > Clean-up of event files created by HiveProtoLoggingHook. > > > Key: HIVE-20025 > URL: https://issues.apache.org/jira/browse/HIVE-20025 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Affects Versions: 3.0.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan >Priority: Major > Labels: Hive, hooks, pull-request-available > Fix For: 4.0.0, 3.2.0 > > Attachments: HIVE-20025.01-branch-3.patch, HIVE-20025.01.patch, > HIVE-20025.02.patch, HIVE-20025.03.patch, HIVE-20025.04.patch > > > Currently, HiveProtoLoggingHook write event data to hdfs. The number of files > can grow to very large numbers. > Since the files are created under a folder with Date being a part of the > path, hive should have a way to clean up data older than a certain configured > time / date. This can be a job that can run with as little frequency as just > once a day. > This time should be set to 1 week default. There should also be a sane upper > bound of # of files so that when a large cluster generates a lot of files > during a spike, we don't force the cluster fall over. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20025) Clean-up of event files created by HiveProtoLoggingHook.
[ https://issues.apache.org/jira/browse/HIVE-20025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-20025: Fix Version/s: 3.2.0 > Clean-up of event files created by HiveProtoLoggingHook. > > > Key: HIVE-20025 > URL: https://issues.apache.org/jira/browse/HIVE-20025 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Affects Versions: 3.0.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan >Priority: Major > Labels: Hive, hooks, pull-request-available > Fix For: 4.0.0, 3.2.0 > > Attachments: HIVE-20025.01-branch-3.patch, HIVE-20025.01.patch, > HIVE-20025.02.patch, HIVE-20025.03.patch, HIVE-20025.04.patch > > > Currently, HiveProtoLoggingHook write event data to hdfs. The number of files > can grow to very large numbers. > Since the files are created under a folder with Date being a part of the > path, hive should have a way to clean up data older than a certain configured > time / date. This can be a job that can run with as little frequency as just > once a day. > This time should be set to 1 week default. There should also be a sane upper > bound of # of files so that when a large cluster generates a lot of files > during a spike, we don't force the cluster fall over. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20067) fix InsertEvent on mm tables to not cause failing capability checks
[
https://issues.apache.org/jira/browse/HIVE-20067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16533575#comment-16533575
]
Hive QA commented on HIVE-20067:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12930275/HIVE-20067.02.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 14638 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/12399/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12399/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12399/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12930275 - PreCommit-HIVE-Build
> fix InsertEvent on mm tables to not cause failing capability checks
> ---
>
> Key: HIVE-20067
> URL: https://issues.apache.org/jira/browse/HIVE-20067
> Project: Hive
> Issue Type: Bug
> Components: Standalone Metastore
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-20067.01.patch, HIVE-20067.02.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20015) Populate ArrayList with Constructor
[
https://issues.apache.org/jira/browse/HIVE-20015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16533580#comment-16533580
]
Daniel Voros commented on HIVE-20015:
-
Failed tests are unrelated and they're all passing locally.
> Populate ArrayList with Constructor
> ---
>
> Key: HIVE-20015
> URL: https://issues.apache.org/jira/browse/HIVE-20015
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 3.0.0, 4.0.0
>Reporter: BELUGA BEHR
>Assignee: Daniel Voros
>Priority: Trivial
> Labels: newbie, noob
> Attachments: HIVE-20015.1.patch
>
>
> {code:title=MapWork.java}
> public ArrayList> getWorks() {
> return new ArrayList>(aliasToWork.values());
> }
> public ArrayList getPaths() {
> ArrayList ret=new ArrayList<>();
> ret.addAll(pathToAliases.keySet());
> return ret;
> }
> {code}
> {{getWorks}} method correctly uses the constructor to populate the
> {{ArrayList}}. Please update {{getPaths}} method to do the same, instead of
> creating an empty array, then creating a new array to accommodate the
> {{addAll}} request.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20088) Beeline config location path is assembled incorrectly
[
https://issues.apache.org/jira/browse/HIVE-20088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16533586#comment-16533586
]
Hive QA commented on HIVE-20088:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
28s{color} | {color:blue} beeline in master has 56 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
12s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
11s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 10m 40s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-12400/dev-support/hive-personality.sh
|
| git revision | master / e865b44 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: beeline U: beeline |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12400/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Beeline config location path is assembled incorrectly
> -
>
> Key: HIVE-20088
> URL: https://issues.apache.org/jira/browse/HIVE-20088
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Affects Versions: 3.0.0
>Reporter: Denes Bodo
>Assignee: Denes Bodo
>Priority: Critical
> Labels: easyfix, usability
> Attachments: HIVE-20088_001.patch
>
>
> Checking the code in
> [https://github.com/apache/hive/blob/branch-3/beeline/src/java/org/apache/hive/beeline/hs2connection/UserHS2ConnectionFileParser.java]
> or in
> [https://github.com/apache/hive/blob/branch-3/beeline/src/java/org/apache/hive/beeline/hs2connection/BeelineSiteParser.java]
> I see {code}locations.add(ETC_HIVE_CONF_LOCATION +
> DEFAULT_BEELINE_SITE_FILE_NAME);{code}
> whee file separator shall be used:
> {code}locations.add(ETC_HIVE_CONF_LOCATION + File.separator +
> DEFAULT_BEELINE_SITE_FILE_NAME);{code}
> Due to this, BeeLine cannot use configuration in case if this would be the
> only way.
> In my hadoop-3 setup, the locations list contains the following:
> {code}
> /home/myuser/.beeline/beeline-site.xml
> /etc/hive/confbeeline-site.xml
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20015) Populate ArrayList with Constructor
[
https://issues.apache.org/jira/browse/HIVE-20015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16533599#comment-16533599
]
Zoltan Haindrich commented on HIVE-20015:
-
+1
> Populate ArrayList with Constructor
> ---
>
> Key: HIVE-20015
> URL: https://issues.apache.org/jira/browse/HIVE-20015
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 3.0.0, 4.0.0
>Reporter: BELUGA BEHR
>Assignee: Daniel Voros
>Priority: Trivial
> Labels: newbie, noob
> Attachments: HIVE-20015.1.patch
>
>
> {code:title=MapWork.java}
> public ArrayList> getWorks() {
> return new ArrayList>(aliasToWork.values());
> }
> public ArrayList getPaths() {
> ArrayList ret=new ArrayList<>();
> ret.addAll(pathToAliases.keySet());
> return ret;
> }
> {code}
> {{getWorks}} method correctly uses the constructor to populate the
> {{ArrayList}}. Please update {{getPaths}} method to do the same, instead of
> creating an empty array, then creating a new array to accommodate the
> {{addAll}} request.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20041) ResultsCache: Improve loggng for concurrent queries
[
https://issues.apache.org/jira/browse/HIVE-20041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Laszlo Bodor updated HIVE-20041:
Attachment: HIVE-20041.04.patch
> ResultsCache: Improve loggng for concurrent queries
> ---
>
> Key: HIVE-20041
> URL: https://issues.apache.org/jira/browse/HIVE-20041
> Project: Hive
> Issue Type: Improvement
> Components: Diagnosability
>Reporter: Gopal V
>Assignee: Laszlo Bodor
>Priority: Minor
> Attachments: HIVE-20041.01.patch, HIVE-20041.02.patch,
> HIVE-20041.03.patch, HIVE-20041.04.patch
>
>
> The logging for QueryResultsCache ends up printing information without
> context, like
> {code}
> 2018-06-30T17:48:45,502 INFO [HiveServer2-Background-Pool: Thread-166]
> results.QueryResultsCache: Waiting on pending cacheEntry
> {code}
> {code}
> 2018-06-30T17:50:17,963 INFO [HiveServer2-Background-Pool: Thread-145]
> ql.Driver: savedToCache: true
> {code}
> The previous lines for this are in DEBUG level, so the logging ends up being
> useless at INFO level to debug.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work started] (HIVE-19927) Last Repl ID set by bootstrap dump is not proper and may cause loss of data if have ACID tables.
[ https://issues.apache.org/jira/browse/HIVE-19927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-19927 started by Sankar Hariappan. --- > Last Repl ID set by bootstrap dump is not proper and may cause loss of data > if have ACID tables. > > > Key: HIVE-19927 > URL: https://issues.apache.org/jira/browse/HIVE-19927 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2, Transactions >Affects Versions: 3.1.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan >Priority: Major > Labels: DR, replication > > During bootstrap dump of ACID tables, let's consider the below sequence. > - Current session (REPL DUMP), Open txn (Txn1) - Event-10 > - Another session (Session-2), Open txn (Txn2) - Event-11 > - Session-2 -> Insert data (T1.D1) to ACID table. - Event-12 > - Get lastReplId = last event ID logged. (Event-12) > - Session-2 -> Commit Txn (Txn2) - Event-13 > - Dump ACID tables based on validTxnList based on Txn1. --> This step skips > all the data written by txns > Txn1. So, T1.D1 will be missing. > - Commit Txn (Txn1) > - REPL LOAD from bootstrap dump will skip T1.D1. > - Incremental REPL DUMP will start from Event-13 and hence lose Txn2 which is > opened after Txn1. So, data T1.D1 will be lost for ever. > Proposed to capture the lastReplId of bootstrap before opening current txn > (Txn1) and store it in Driver context and use it for dump. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20088) Beeline config location path is assembled incorrectly
[
https://issues.apache.org/jira/browse/HIVE-20088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16533638#comment-16533638
]
Hive QA commented on HIVE-20088:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12930292/HIVE-20088_001.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 14635 tests
executed
*Failed tests:*
{noformat}
TestAutoPurgeTables - did not produce a TEST-*.xml file (likely timed out)
(batchId=240)
TestClearDanglingScratchDir - did not produce a TEST-*.xml file (likely timed
out) (batchId=240)
TestLocationQueries - did not produce a TEST-*.xml file (likely timed out)
(batchId=240)
TestSparkStatistics - did not produce a TEST-*.xml file (likely timed out)
(batchId=240)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druid_timestamptz]
(batchId=191)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_joins]
(batchId=191)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_masking]
(batchId=191)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/12400/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12400/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12400/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 7 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12930292 - PreCommit-HIVE-Build
> Beeline config location path is assembled incorrectly
> -
>
> Key: HIVE-20088
> URL: https://issues.apache.org/jira/browse/HIVE-20088
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Affects Versions: 3.0.0
>Reporter: Denes Bodo
>Assignee: Denes Bodo
>Priority: Critical
> Labels: easyfix, usability
> Attachments: HIVE-20088_001.patch
>
>
> Checking the code in
> [https://github.com/apache/hive/blob/branch-3/beeline/src/java/org/apache/hive/beeline/hs2connection/UserHS2ConnectionFileParser.java]
> or in
> [https://github.com/apache/hive/blob/branch-3/beeline/src/java/org/apache/hive/beeline/hs2connection/BeelineSiteParser.java]
> I see {code}locations.add(ETC_HIVE_CONF_LOCATION +
> DEFAULT_BEELINE_SITE_FILE_NAME);{code}
> whee file separator shall be used:
> {code}locations.add(ETC_HIVE_CONF_LOCATION + File.separator +
> DEFAULT_BEELINE_SITE_FILE_NAME);{code}
> Due to this, BeeLine cannot use configuration in case if this would be the
> only way.
> In my hadoop-3 setup, the locations list contains the following:
> {code}
> /home/myuser/.beeline/beeline-site.xml
> /etc/hive/confbeeline-site.xml
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20077) hcat command should follow same pattern as hive cli for getting HBase jars
[
https://issues.apache.org/jira/browse/HIVE-20077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16533644#comment-16533644
]
Sean Busbey commented on HIVE-20077:
bq. ERROR: -1 due to no test(s) being added or modified.
I didn't see a way to expressly check the runtime environment in a way that
would reflect this change. Any suggestions?
> hcat command should follow same pattern as hive cli for getting HBase jars
> --
>
> Key: HIVE-20077
> URL: https://issues.apache.org/jira/browse/HIVE-20077
> Project: Hive
> Issue Type: Bug
> Components: HCatalog
>Affects Versions: 0.14.0, 2.3.2
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Major
> Attachments: HIVE-20077.0.patch
>
>
> Currently the {{hcat}} command adds HBase jars to the classpath by using find
> to walk the directories under {{$HBASE_HOME/lib}}.
> {code}
> # Look for HBase in a BigTop-compatible way. Avoid thrift version
> # conflict with modern versions of HBase.
> HBASE_HOME=${HBASE_HOME:-"/usr/lib/hbase"}
> HBASE_CONF_DIR=${HBASE_CONF_DIR:-"${HBASE_HOME}/conf"}
> if [ -d ${HBASE_HOME} ] ; then
>for jar in $(find $HBASE_HOME -name '*.jar' -not -name '*thrift*'); do
> HBASE_CLASSPATH=$HBASE_CLASSPATH:${jar}
>done
>export HADOOP_CLASSPATH="${HADOOP_CLASSPATH}:${HBASE_CLASSPATH}"
> fi
> if [ -d $HBASE_CONF_DIR ] ; then
> HADOOP_CLASSPATH="${HADOOP_CLASSPATH}:${HBASE_CONF_DIR}"
> fi
> {code}
> This is incorrect as that path contains jars for a mixture of purposes; hbase
> client jars, hbase server jars, and hbase shell specific jars. The inclusion
> of unneeded jars is mostly innocuous until the upcoming HBase 2.1.0 release.
> That release will have HBASE-20615 and HBASE-19735, which will mean most
> client facing installations will have a number of shaded client artifacts
> present.
> With those changes in place, the current implementation will include in the
> hcat runtime a mix of shaded and non-shaded hbase artifacts that include some
> Hadoop classes rewritten to use a shaded version of protobuf. When these mix
> with other Hadoop classes in the classpath that have not been rewritten hcat
> will fail with errors that look like this:
> {code}
> Exception in thread "main" java.lang.ClassCastException:
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$GetFileInfoRequestProto
> cannot be cast to org.apache.hadoop.hbase.shaded.com.google.protobuf.Message
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:225)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116)
> at com.sun.proxy.$Proxy28.getFileInfo(Unknown Source)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:875)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:422)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:165)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:157)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:359)
> at com.sun.proxy.$Proxy29.getFileInfo(Unknown Source)
> at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1643)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$29.doCall(DistributedFileSystem.java:1495)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$29.doCall(DistributedFileSystem.java:1492)
> at
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1507)
> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1668)
> at
> org.apache.hadoop.hive.ql.session.SessionState.createRootHDFSDir(SessionState.java:686)
> at
> org.apache.hadoop.hive.ql.session.SessionState.createSessionDirs(SessionState.java:625)
> at
> org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:557)
> at
> org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.jav
[jira] [Commented] (HIVE-20069) Fix reoptimization in case of DPP
[ https://issues.apache.org/jira/browse/HIVE-20069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16533670#comment-16533670 ] Zoltan Haindrich commented on HIVE-20069: - so far signatures was added during stats annotation; and by doing it there - it also "silently" also handled the case of multiple tablescans about the same thing... Although this should be changed later; but right now I've moved the link to a walker (as I've done it for semijoin - because for that I saw no other way) - so from now on this patch will incorporate HIVE-20087 as well.. > Fix reoptimization in case of DPP > - > > Key: HIVE-20069 > URL: https://issues.apache.org/jira/browse/HIVE-20069 > Project: Hive > Issue Type: Bug > Components: Query Planning >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-20069.01.patch > > > reported by [~t3rmin4t0r] > In case dynamic partition pruning; the operator statistics became partial; to > only reflect the actually scanned partitions; but they are being used as an > information about the "full" table...which leads to the exchange of the 2 > tables being joined...which is really unfortunate... -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20069) Fix reoptimization in case of DPP
[ https://issues.apache.org/jira/browse/HIVE-20069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-20069: Attachment: HIVE-20069.02.patch > Fix reoptimization in case of DPP > - > > Key: HIVE-20069 > URL: https://issues.apache.org/jira/browse/HIVE-20069 > Project: Hive > Issue Type: Bug > Components: Query Planning >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-20069.01.patch, HIVE-20069.02.patch > > > reported by [~t3rmin4t0r] > In case dynamic partition pruning; the operator statistics became partial; to > only reflect the actually scanned partitions; but they are being used as an > information about the "full" table...which leads to the exchange of the 2 > tables being joined...which is really unfortunate... -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20069) Fix reoptimization in case of DPP
[ https://issues.apache.org/jira/browse/HIVE-20069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-20069: Attachment: (was: HIVE-20069.02.patch) > Fix reoptimization in case of DPP > - > > Key: HIVE-20069 > URL: https://issues.apache.org/jira/browse/HIVE-20069 > Project: Hive > Issue Type: Bug > Components: Query Planning >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-20069.01.patch, HIVE-20069.02.patch > > > reported by [~t3rmin4t0r] > In case dynamic partition pruning; the operator statistics became partial; to > only reflect the actually scanned partitions; but they are being used as an > information about the "full" table...which leads to the exchange of the 2 > tables being joined...which is really unfortunate... -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20069) Fix reoptimization in case of DPP
[ https://issues.apache.org/jira/browse/HIVE-20069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-20069: Attachment: HIVE-20069.02.patch > Fix reoptimization in case of DPP > - > > Key: HIVE-20069 > URL: https://issues.apache.org/jira/browse/HIVE-20069 > Project: Hive > Issue Type: Bug > Components: Query Planning >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-20069.01.patch, HIVE-20069.02.patch > > > reported by [~t3rmin4t0r] > In case dynamic partition pruning; the operator statistics became partial; to > only reflect the actually scanned partitions; but they are being used as an > information about the "full" table...which leads to the exchange of the 2 > tables being joined...which is really unfortunate... -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20043) HiveServer2: SessionState has a static sync block around an AtomicBoolean
[
https://issues.apache.org/jira/browse/HIVE-20043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Laszlo Bodor updated HIVE-20043:
Attachment: HIVE-20043.01.patch
> HiveServer2: SessionState has a static sync block around an AtomicBoolean
> -
>
> Key: HIVE-20043
> URL: https://issues.apache.org/jira/browse/HIVE-20043
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 3.0.0
>Reporter: Gopal V
>Assignee: Laszlo Bodor
>Priority: Major
> Labels: Concurrency
> Attachments: HIVE-20043.01.patch
>
>
> {code}
> private static void start(SessionState startSs, boolean isAsync, LogHelper
> console) {
> ...
> synchronized(SessionState.class) {
> if (!startSs.isStarted.compareAndSet(false, true)) {
> return;
> }
> }
> {code}
> startSs.isStarted is an AtomicBoolean, which makes it hard to know why this
> code is locked with a static lock.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20043) HiveServer2: SessionState has a static sync block around an AtomicBoolean
[
https://issues.apache.org/jira/browse/HIVE-20043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Laszlo Bodor updated HIVE-20043:
Attachment: HIVE-20043.01.patch
> HiveServer2: SessionState has a static sync block around an AtomicBoolean
> -
>
> Key: HIVE-20043
> URL: https://issues.apache.org/jira/browse/HIVE-20043
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 3.0.0
>Reporter: Gopal V
>Assignee: Laszlo Bodor
>Priority: Major
> Labels: Concurrency
> Attachments: HIVE-20043.01.patch
>
>
> {code}
> private static void start(SessionState startSs, boolean isAsync, LogHelper
> console) {
> ...
> synchronized(SessionState.class) {
> if (!startSs.isStarted.compareAndSet(false, true)) {
> return;
> }
> }
> {code}
> startSs.isStarted is an AtomicBoolean, which makes it hard to know why this
> code is locked with a static lock.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20043) HiveServer2: SessionState has a static sync block around an AtomicBoolean
[
https://issues.apache.org/jira/browse/HIVE-20043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Laszlo Bodor updated HIVE-20043:
Attachment: (was: HIVE-20043.01.patch)
> HiveServer2: SessionState has a static sync block around an AtomicBoolean
> -
>
> Key: HIVE-20043
> URL: https://issues.apache.org/jira/browse/HIVE-20043
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 3.0.0
>Reporter: Gopal V
>Assignee: Laszlo Bodor
>Priority: Major
> Labels: Concurrency
> Attachments: HIVE-20043.01.patch
>
>
> {code}
> private static void start(SessionState startSs, boolean isAsync, LogHelper
> console) {
> ...
> synchronized(SessionState.class) {
> if (!startSs.isStarted.compareAndSet(false, true)) {
> return;
> }
> }
> {code}
> startSs.isStarted is an AtomicBoolean, which makes it hard to know why this
> code is locked with a static lock.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20043) HiveServer2: SessionState has a static sync block around an AtomicBoolean
[
https://issues.apache.org/jira/browse/HIVE-20043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Laszlo Bodor updated HIVE-20043:
Attachment: (was: HIVE-20043.01.patch)
> HiveServer2: SessionState has a static sync block around an AtomicBoolean
> -
>
> Key: HIVE-20043
> URL: https://issues.apache.org/jira/browse/HIVE-20043
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 3.0.0
>Reporter: Gopal V
>Assignee: Laszlo Bodor
>Priority: Major
> Labels: Concurrency
> Attachments: HIVE-20043.01.patch
>
>
> {code}
> private static void start(SessionState startSs, boolean isAsync, LogHelper
> console) {
> ...
> synchronized(SessionState.class) {
> if (!startSs.isStarted.compareAndSet(false, true)) {
> return;
> }
> }
> {code}
> startSs.isStarted is an AtomicBoolean, which makes it hard to know why this
> code is locked with a static lock.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20043) HiveServer2: SessionState has a static sync block around an AtomicBoolean
[
https://issues.apache.org/jira/browse/HIVE-20043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Laszlo Bodor updated HIVE-20043:
Attachment: HIVE-20043.01.patch
> HiveServer2: SessionState has a static sync block around an AtomicBoolean
> -
>
> Key: HIVE-20043
> URL: https://issues.apache.org/jira/browse/HIVE-20043
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 3.0.0
>Reporter: Gopal V
>Assignee: Laszlo Bodor
>Priority: Major
> Labels: Concurrency
> Attachments: HIVE-20043.01.patch
>
>
> {code}
> private static void start(SessionState startSs, boolean isAsync, LogHelper
> console) {
> ...
> synchronized(SessionState.class) {
> if (!startSs.isStarted.compareAndSet(false, true)) {
> return;
> }
> }
> {code}
> startSs.isStarted is an AtomicBoolean, which makes it hard to know why this
> code is locked with a static lock.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20077) hcat command should follow same pattern as hive cli for getting HBase jars
[
https://issues.apache.org/jira/browse/HIVE-20077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16533679#comment-16533679
]
Sean Busbey commented on HIVE-20077:
{quote}
Failed tests:
TestAlterTableMetadata - did not produce a TEST-*.xml file (likely timed out)
(batchId=240)
TestAutoPurgeTables - did not produce a TEST-*.xml file (likely timed out)
(batchId=240)
TestClearDanglingScratchDir - did not produce a TEST-*.xml file (likely timed
out) (batchId=240)
TestLocationQueries - did not produce a TEST-*.xml file (likely timed out)
(batchId=240)
TestReplicationScenariosAcidTables - did not produce a TEST-*.xml file (likely
timed out) (batchId=240)
TestSemanticAnalyzerHookLoading - did not produce a TEST-*.xml file (likely
timed out) (batchId=240)
TestSparkStatistics - did not produce a TEST-*.xml file (likely timed out)
(batchId=240)
{quote}
The only connection to HCatalog code I can find in the failed tests is itests'
{{WarehouseInstance}} makes use of:
{code}
import org.apache.hive.hcatalog.api.repl.ReplicationV1CompatRule;
import org.apache.hive.hcatalog.listener.DbNotificationListener;
{code}
looking at those classes I have a hard time guessing how HBase classpath
changes could reasonably impact them.
Is there an existing automated tool to get a rerun of just the failed tests? Or
perhaps just batchId 240, since it seems like that one failed due to
{{TestReplicationScenariosAcidTables}} hanging?
> hcat command should follow same pattern as hive cli for getting HBase jars
> --
>
> Key: HIVE-20077
> URL: https://issues.apache.org/jira/browse/HIVE-20077
> Project: Hive
> Issue Type: Bug
> Components: HCatalog
>Affects Versions: 0.14.0, 2.3.2
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Major
> Attachments: HIVE-20077.0.patch
>
>
> Currently the {{hcat}} command adds HBase jars to the classpath by using find
> to walk the directories under {{$HBASE_HOME/lib}}.
> {code}
> # Look for HBase in a BigTop-compatible way. Avoid thrift version
> # conflict with modern versions of HBase.
> HBASE_HOME=${HBASE_HOME:-"/usr/lib/hbase"}
> HBASE_CONF_DIR=${HBASE_CONF_DIR:-"${HBASE_HOME}/conf"}
> if [ -d ${HBASE_HOME} ] ; then
>for jar in $(find $HBASE_HOME -name '*.jar' -not -name '*thrift*'); do
> HBASE_CLASSPATH=$HBASE_CLASSPATH:${jar}
>done
>export HADOOP_CLASSPATH="${HADOOP_CLASSPATH}:${HBASE_CLASSPATH}"
> fi
> if [ -d $HBASE_CONF_DIR ] ; then
> HADOOP_CLASSPATH="${HADOOP_CLASSPATH}:${HBASE_CONF_DIR}"
> fi
> {code}
> This is incorrect as that path contains jars for a mixture of purposes; hbase
> client jars, hbase server jars, and hbase shell specific jars. The inclusion
> of unneeded jars is mostly innocuous until the upcoming HBase 2.1.0 release.
> That release will have HBASE-20615 and HBASE-19735, which will mean most
> client facing installations will have a number of shaded client artifacts
> present.
> With those changes in place, the current implementation will include in the
> hcat runtime a mix of shaded and non-shaded hbase artifacts that include some
> Hadoop classes rewritten to use a shaded version of protobuf. When these mix
> with other Hadoop classes in the classpath that have not been rewritten hcat
> will fail with errors that look like this:
> {code}
> Exception in thread "main" java.lang.ClassCastException:
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$GetFileInfoRequestProto
> cannot be cast to org.apache.hadoop.hbase.shaded.com.google.protobuf.Message
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:225)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116)
> at com.sun.proxy.$Proxy28.getFileInfo(Unknown Source)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:875)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:422)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:165)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:157)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95)
> at
> org.apache.ha
[jira] [Commented] (HIVE-20043) HiveServer2: SessionState has a static sync block around an AtomicBoolean
[
https://issues.apache.org/jira/browse/HIVE-20043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16533680#comment-16533680
]
Laszlo Bodor commented on HIVE-20043:
-
i think AtomicBoolean is enough, checked patch history of HIVE-19813, first
synchronized was moved from method signature to isStarted checker block, and
then AtomicBoolean was added, however the latter seems to be enough in this case
attached test case
> HiveServer2: SessionState has a static sync block around an AtomicBoolean
> -
>
> Key: HIVE-20043
> URL: https://issues.apache.org/jira/browse/HIVE-20043
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 3.0.0
>Reporter: Gopal V
>Assignee: Laszlo Bodor
>Priority: Major
> Labels: Concurrency
> Attachments: HIVE-20043.01.patch
>
>
> {code}
> private static void start(SessionState startSs, boolean isAsync, LogHelper
> console) {
> ...
> synchronized(SessionState.class) {
> if (!startSs.isStarted.compareAndSet(false, true)) {
> return;
> }
> }
> {code}
> startSs.isStarted is an AtomicBoolean, which makes it hard to know why this
> code is locked with a static lock.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20085) Druid-Hive (managed) table creation fails with strict managed table checks: Table is marked as a managed table but is not transactional
[
https://issues.apache.org/jira/browse/HIVE-20085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16533708#comment-16533708
]
Hive QA commented on HIVE-20085:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
30s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
23s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
31s{color} | {color:blue} common in master has 64 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
57s{color} | {color:blue} standalone-metastore in master has 228 extant
Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
52s{color} | {color:blue} ql in master has 2287 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
27s{color} | {color:blue} druid-handler in master has 13 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
31s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
23s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
19s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
11s{color} | {color:red} druid-handler: The patch generated 3 new + 64
unchanged - 1 fixed = 67 total (was 65) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 8m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
38s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 41m 17s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-12401/dev-support/hive-personality.sh
|
| git revision | master / e865b44 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12401/yetus/diff-checkstyle-druid-handler.txt
|
| modules | C: common standalone-metastore ql druid-handler U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12401/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Druid-Hive (managed) table creation fails with strict managed table checks:
> Table is marked as a managed table but is not transactional
> ---
>
> Key: HIVE-20085
> URL: https://issues.apache.org/jira/browse/HIVE-20085
> Project: Hive
> Issue Type: Bug
> Components: Hive, StorageHandler
>Affects Versions: 3.0.0
>Reporter: Dileep Kumar Chiguruvada
>Assignee: Nishant Bangarwa
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-20
[jira] [Commented] (HIVE-19765) Add Parquet specific tests to BlobstoreCliDriver
[ https://issues.apache.org/jira/browse/HIVE-19765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16533719#comment-16533719 ] Marta Kuczora commented on HIVE-19765: -- Thanks for the patch [~stakiar], these tests are very useful. +1 > Add Parquet specific tests to BlobstoreCliDriver > > > Key: HIVE-19765 > URL: https://issues.apache.org/jira/browse/HIVE-19765 > Project: Hive > Issue Type: Sub-task >Reporter: Sahil Takiar >Assignee: Sahil Takiar >Priority: Major > Attachments: HIVE-19765.1.patch, HIVE-19765.2.patch, > HIVE-19765.3.patch, HIVE-19765.4.patch > > > Similar to what was done for RC and ORC files. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-20094) Update Druid to 0.12.1 version
[ https://issues.apache.org/jira/browse/HIVE-20094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] slim bouguerra reassigned HIVE-20094: - > Update Druid to 0.12.1 version > -- > > Key: HIVE-20094 > URL: https://issues.apache.org/jira/browse/HIVE-20094 > Project: Hive > Issue Type: Bug >Reporter: slim bouguerra >Assignee: slim bouguerra >Priority: Minor > > As per Jira title. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work stopped] (HIVE-20094) Update Druid to 0.12.1 version
[ https://issues.apache.org/jira/browse/HIVE-20094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-20094 stopped by slim bouguerra. - > Update Druid to 0.12.1 version > -- > > Key: HIVE-20094 > URL: https://issues.apache.org/jira/browse/HIVE-20094 > Project: Hive > Issue Type: Bug >Reporter: slim bouguerra >Assignee: slim bouguerra >Priority: Minor > > As per Jira title. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20094) Update Druid to 0.12.1 version
[ https://issues.apache.org/jira/browse/HIVE-20094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] slim bouguerra updated HIVE-20094: -- Status: Patch Available (was: Open) > Update Druid to 0.12.1 version > -- > > Key: HIVE-20094 > URL: https://issues.apache.org/jira/browse/HIVE-20094 > Project: Hive > Issue Type: Bug >Reporter: slim bouguerra >Assignee: slim bouguerra >Priority: Minor > > As per Jira title. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work started] (HIVE-20094) Update Druid to 0.12.1 version
[ https://issues.apache.org/jira/browse/HIVE-20094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-20094 started by slim bouguerra. - > Update Druid to 0.12.1 version > -- > > Key: HIVE-20094 > URL: https://issues.apache.org/jira/browse/HIVE-20094 > Project: Hive > Issue Type: Bug >Reporter: slim bouguerra >Assignee: slim bouguerra >Priority: Minor > > As per Jira title. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20085) Druid-Hive (managed) table creation fails with strict managed table checks: Table is marked as a managed table but is not transactional
[
https://issues.apache.org/jira/browse/HIVE-20085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16533753#comment-16533753
]
Hive QA commented on HIVE-20085:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12930377/HIVE-20085.3.patch
{color:green}SUCCESS:{color} +1 due to 14 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 14622 tests
executed
*Failed tests:*
{noformat}
TestAlterTableMetadata - did not produce a TEST-*.xml file (likely timed out)
(batchId=240)
TestAutoPurgeTables - did not produce a TEST-*.xml file (likely timed out)
(batchId=240)
TestClearDanglingScratchDir - did not produce a TEST-*.xml file (likely timed
out) (batchId=240)
TestLocationQueries - did not produce a TEST-*.xml file (likely timed out)
(batchId=240)
TestReplicationScenariosAcidTables - did not produce a TEST-*.xml file (likely
timed out) (batchId=240)
TestSemanticAnalyzerHookLoading - did not produce a TEST-*.xml file (likely
timed out) (batchId=240)
TestSparkStatistics - did not produce a TEST-*.xml file (likely timed out)
(batchId=240)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/12401/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12401/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12401/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 7 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12930377 - PreCommit-HIVE-Build
> Druid-Hive (managed) table creation fails with strict managed table checks:
> Table is marked as a managed table but is not transactional
> ---
>
> Key: HIVE-20085
> URL: https://issues.apache.org/jira/browse/HIVE-20085
> Project: Hive
> Issue Type: Bug
> Components: Hive, StorageHandler
>Affects Versions: 3.0.0
>Reporter: Dileep Kumar Chiguruvada
>Assignee: Nishant Bangarwa
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-20085.1.patch, HIVE-20085.2.patch,
> HIVE-20085.3.patch, HIVE-20085.patch
>
>
> Druid-Hive (managed) table creation fails with strict managed table checks:
> Table is marked as a managed table but is not transactional
> {code}
> drop table if exists calcs;
> create table calcs
> STORED BY 'org.apache.hadoop.hive.druid.DruidStorageHandler'
> TBLPROPERTIES (
> "druid.segment.granularity" = "MONTH",
> "druid.query.granularity" = "DAY")
> AS SELECT
> cast(datetime0 as timestamp with local time zone) `__time`,
> key,
> str0, str1, str2, str3,
> date0, date1, date2, date3,
> time0, time1,
> datetime0, datetime1,
> zzz,
> cast(bool0 as string) bool0,
> cast(bool1 as string) bool1,
> cast(bool2 as string) bool2,
> cast(bool3 as string) bool3,
> int0, int1, int2, int3,
> num0, num1, num2, num3, num4
> from tableau_orc.calcs;
> 2018-07-03 04:57:31,911|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : Status: Running
> (Executing on YARN cluster with App id application_1530592209763_0009)
> ...
> ...
> 2018-07-03 04:57:36,334|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : SHUFFLE_BYTES_TO_MEM: > 0
> 2018-07-03 04:57:36,334|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : SHUFFLE_PHASE_TIME:
> 330
> 2018-07-03 04:57:36,334|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : SPILLED_RECORDS: 17
> 2018-07-03 04:57:36,334|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO :
> TaskCounter_Reducer_2_OUTPUT_out_Reducer_2:
> 2018-07-03 04:57:36,335|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : OUTPUT_RECORDS: 0
> 2018-07-03 04:57:36,335|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO :
> org.apache.hadoop.hive.llap.counters.LlapWmCounters:
> 2018-07-03 04:57:36,335|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : GUARANTEED_QUEUED_NS: > 0
> 2018-07-03 04:57:36,335|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO :
> GUARANTEED_RUNNING_NS: 0
> 2018-07-03 04:57:36,335|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO :
> SPECULATIVE_QU
[jira] [Updated] (HIVE-20094) Update Druid to 0.12.1 version
[ https://issues.apache.org/jira/browse/HIVE-20094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] slim bouguerra updated HIVE-20094: -- Attachment: HIVE-20094.patch > Update Druid to 0.12.1 version > -- > > Key: HIVE-20094 > URL: https://issues.apache.org/jira/browse/HIVE-20094 > Project: Hive > Issue Type: Bug >Reporter: slim bouguerra >Assignee: slim bouguerra >Priority: Minor > Attachments: HIVE-20094.patch > > > As per Jira title. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20085) Druid-Hive (managed) table creation fails with strict managed table checks: Table is marked as a managed table but is not transactional
[
https://issues.apache.org/jira/browse/HIVE-20085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16533755#comment-16533755
]
Hive QA commented on HIVE-20085:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12930377/HIVE-20085.3.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/12402/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12402/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12402/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12930377/HIVE-20085.3.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12930377 - PreCommit-HIVE-Build
> Druid-Hive (managed) table creation fails with strict managed table checks:
> Table is marked as a managed table but is not transactional
> ---
>
> Key: HIVE-20085
> URL: https://issues.apache.org/jira/browse/HIVE-20085
> Project: Hive
> Issue Type: Bug
> Components: Hive, StorageHandler
>Affects Versions: 3.0.0
>Reporter: Dileep Kumar Chiguruvada
>Assignee: Nishant Bangarwa
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-20085.1.patch, HIVE-20085.2.patch,
> HIVE-20085.3.patch, HIVE-20085.patch
>
>
> Druid-Hive (managed) table creation fails with strict managed table checks:
> Table is marked as a managed table but is not transactional
> {code}
> drop table if exists calcs;
> create table calcs
> STORED BY 'org.apache.hadoop.hive.druid.DruidStorageHandler'
> TBLPROPERTIES (
> "druid.segment.granularity" = "MONTH",
> "druid.query.granularity" = "DAY")
> AS SELECT
> cast(datetime0 as timestamp with local time zone) `__time`,
> key,
> str0, str1, str2, str3,
> date0, date1, date2, date3,
> time0, time1,
> datetime0, datetime1,
> zzz,
> cast(bool0 as string) bool0,
> cast(bool1 as string) bool1,
> cast(bool2 as string) bool2,
> cast(bool3 as string) bool3,
> int0, int1, int2, int3,
> num0, num1, num2, num3, num4
> from tableau_orc.calcs;
> 2018-07-03 04:57:31,911|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : Status: Running
> (Executing on YARN cluster with App id application_1530592209763_0009)
> ...
> ...
> 2018-07-03 04:57:36,334|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : SHUFFLE_BYTES_TO_MEM: > 0
> 2018-07-03 04:57:36,334|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : SHUFFLE_PHASE_TIME:
> 330
> 2018-07-03 04:57:36,334|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : SPILLED_RECORDS: 17
> 2018-07-03 04:57:36,334|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO :
> TaskCounter_Reducer_2_OUTPUT_out_Reducer_2:
> 2018-07-03 04:57:36,335|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : OUTPUT_RECORDS: 0
> 2018-07-03 04:57:36,335|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO :
> org.apache.hadoop.hive.llap.counters.LlapWmCounters:
> 2018-07-03 04:57:36,335|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : GUARANTEED_QUEUED_NS: > 0
> 2018-07-03 04:57:36,335|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO :
> GUARANTEED_RUNNING_NS: 0
> 2018-07-03 04:57:36,335|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO :
> SPECULATIVE_QUEUED_NS: 2162643606
> 2018-07-03 04:57:36,335|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO :
> SPECULATIVE_RUNNING_NS: 12151664909
> 2018-07-03 04:57:36,335|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : Starting task
> [Stage-2:DEPENDENCY_COLLECTION] in serial mode
> 2018-07-03 04:57:36,335|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : Starting task
> [Stage-0:MOVE] in serial mode
> 2018-07-03 04:57:36,336|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : Moving data to
> directory
> hdfs://mycluster/warehouse/tablespace/managed/hive/druid_tableau.db/calcs
> from
> hdfs://mycluster/warehouse/tablespace/managed/hive/druid_tableau.db/.hive-staging_hive_2018-07-03_04-57-27_3
[jira] [Commented] (HIVE-20085) Druid-Hive (managed) table creation fails with strict managed table checks: Table is marked as a managed table but is not transactional
[
https://issues.apache.org/jira/browse/HIVE-20085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16533756#comment-16533756
]
Hive QA commented on HIVE-20085:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12930377/HIVE-20085.3.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/12403/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12403/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12403/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12930377/HIVE-20085.3.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12930377 - PreCommit-HIVE-Build
> Druid-Hive (managed) table creation fails with strict managed table checks:
> Table is marked as a managed table but is not transactional
> ---
>
> Key: HIVE-20085
> URL: https://issues.apache.org/jira/browse/HIVE-20085
> Project: Hive
> Issue Type: Bug
> Components: Hive, StorageHandler
>Affects Versions: 3.0.0
>Reporter: Dileep Kumar Chiguruvada
>Assignee: Nishant Bangarwa
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-20085.1.patch, HIVE-20085.2.patch,
> HIVE-20085.3.patch, HIVE-20085.patch
>
>
> Druid-Hive (managed) table creation fails with strict managed table checks:
> Table is marked as a managed table but is not transactional
> {code}
> drop table if exists calcs;
> create table calcs
> STORED BY 'org.apache.hadoop.hive.druid.DruidStorageHandler'
> TBLPROPERTIES (
> "druid.segment.granularity" = "MONTH",
> "druid.query.granularity" = "DAY")
> AS SELECT
> cast(datetime0 as timestamp with local time zone) `__time`,
> key,
> str0, str1, str2, str3,
> date0, date1, date2, date3,
> time0, time1,
> datetime0, datetime1,
> zzz,
> cast(bool0 as string) bool0,
> cast(bool1 as string) bool1,
> cast(bool2 as string) bool2,
> cast(bool3 as string) bool3,
> int0, int1, int2, int3,
> num0, num1, num2, num3, num4
> from tableau_orc.calcs;
> 2018-07-03 04:57:31,911|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : Status: Running
> (Executing on YARN cluster with App id application_1530592209763_0009)
> ...
> ...
> 2018-07-03 04:57:36,334|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : SHUFFLE_BYTES_TO_MEM: > 0
> 2018-07-03 04:57:36,334|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : SHUFFLE_PHASE_TIME:
> 330
> 2018-07-03 04:57:36,334|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : SPILLED_RECORDS: 17
> 2018-07-03 04:57:36,334|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO :
> TaskCounter_Reducer_2_OUTPUT_out_Reducer_2:
> 2018-07-03 04:57:36,335|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : OUTPUT_RECORDS: 0
> 2018-07-03 04:57:36,335|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO :
> org.apache.hadoop.hive.llap.counters.LlapWmCounters:
> 2018-07-03 04:57:36,335|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : GUARANTEED_QUEUED_NS: > 0
> 2018-07-03 04:57:36,335|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO :
> GUARANTEED_RUNNING_NS: 0
> 2018-07-03 04:57:36,335|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO :
> SPECULATIVE_QUEUED_NS: 2162643606
> 2018-07-03 04:57:36,335|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO :
> SPECULATIVE_RUNNING_NS: 12151664909
> 2018-07-03 04:57:36,335|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : Starting task
> [Stage-2:DEPENDENCY_COLLECTION] in serial mode
> 2018-07-03 04:57:36,335|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : Starting task
> [Stage-0:MOVE] in serial mode
> 2018-07-03 04:57:36,336|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : Moving data to
> directory
> hdfs://mycluster/warehouse/tablespace/managed/hive/druid_tableau.db/calcs
> from
> hdfs://mycluster/warehouse/tablespace/managed/hive/druid_tableau.db/.hive-staging_hive_2018-07-03_04-57-27_3
[jira] [Updated] (HIVE-20088) Beeline config location path is assembled incorrectly
[
https://issues.apache.org/jira/browse/HIVE-20088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Denes Bodo updated HIVE-20088:
--
Attachment: HIVE-20088_002.patch
> Beeline config location path is assembled incorrectly
> -
>
> Key: HIVE-20088
> URL: https://issues.apache.org/jira/browse/HIVE-20088
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Affects Versions: 3.0.0
>Reporter: Denes Bodo
>Assignee: Denes Bodo
>Priority: Critical
> Labels: easyfix, usability
> Attachments: HIVE-20088_001.patch, HIVE-20088_002.patch
>
>
> Checking the code in
> [https://github.com/apache/hive/blob/branch-3/beeline/src/java/org/apache/hive/beeline/hs2connection/UserHS2ConnectionFileParser.java]
> or in
> [https://github.com/apache/hive/blob/branch-3/beeline/src/java/org/apache/hive/beeline/hs2connection/BeelineSiteParser.java]
> I see {code}locations.add(ETC_HIVE_CONF_LOCATION +
> DEFAULT_BEELINE_SITE_FILE_NAME);{code}
> whee file separator shall be used:
> {code}locations.add(ETC_HIVE_CONF_LOCATION + File.separator +
> DEFAULT_BEELINE_SITE_FILE_NAME);{code}
> Due to this, BeeLine cannot use configuration in case if this would be the
> only way.
> In my hadoop-3 setup, the locations list contains the following:
> {code}
> /home/myuser/.beeline/beeline-site.xml
> /etc/hive/confbeeline-site.xml
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-20095) Fix jdbc external table feature
[ https://issues.apache.org/jira/browse/HIVE-20095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Doron reassigned HIVE-20095: - > Fix jdbc external table feature > --- > > Key: HIVE-20095 > URL: https://issues.apache.org/jira/browse/HIVE-20095 > Project: Hive > Issue Type: Bug >Reporter: Jonathan Doron >Assignee: Jonathan Doron >Priority: Major > > It seems like the committed code for HIVE-19161 > (7584b3276bebf64aa006eaa162c0a6264d8fcb56) reverted some of HIVE-18423 > updates, and therefore some of the external table queries are not working > correctly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20095) Fix jdbc external table feature
[ https://issues.apache.org/jira/browse/HIVE-20095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Doron updated HIVE-20095: -- Status: Patch Available (was: In Progress) > Fix jdbc external table feature > --- > > Key: HIVE-20095 > URL: https://issues.apache.org/jira/browse/HIVE-20095 > Project: Hive > Issue Type: Bug >Reporter: Jonathan Doron >Assignee: Jonathan Doron >Priority: Major > Attachments: HIVE-20095.1.patch > > > It seems like the committed code for HIVE-19161 > (7584b3276bebf64aa006eaa162c0a6264d8fcb56) reverted some of HIVE-18423 > updates, and therefore some of the external table queries are not working > correctly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work started] (HIVE-20095) Fix jdbc external table feature
[ https://issues.apache.org/jira/browse/HIVE-20095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-20095 started by Jonathan Doron. - > Fix jdbc external table feature > --- > > Key: HIVE-20095 > URL: https://issues.apache.org/jira/browse/HIVE-20095 > Project: Hive > Issue Type: Bug >Reporter: Jonathan Doron >Assignee: Jonathan Doron >Priority: Major > Attachments: HIVE-20095.1.patch > > > It seems like the committed code for HIVE-19161 > (7584b3276bebf64aa006eaa162c0a6264d8fcb56) reverted some of HIVE-18423 > updates, and therefore some of the external table queries are not working > correctly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20095) Fix jdbc external table feature
[ https://issues.apache.org/jira/browse/HIVE-20095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Doron updated HIVE-20095: -- Attachment: HIVE-20095.1.patch > Fix jdbc external table feature > --- > > Key: HIVE-20095 > URL: https://issues.apache.org/jira/browse/HIVE-20095 > Project: Hive > Issue Type: Bug >Reporter: Jonathan Doron >Assignee: Jonathan Doron >Priority: Major > Attachments: HIVE-20095.1.patch > > > It seems like the committed code for HIVE-19161 > (7584b3276bebf64aa006eaa162c0a6264d8fcb56) reverted some of HIVE-18423 > updates, and therefore some of the external table queries are not working > correctly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20090) Extend creation of semijoin reduction filters to be able to discover new opportunities
[
https://issues.apache.org/jira/browse/HIVE-20090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16533792#comment-16533792
]
Hive QA commented on HIVE-20090:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
22s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
54s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
32s{color} | {color:blue} common in master has 64 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
1s{color} | {color:blue} ql in master has 2287 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
11s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
23s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
21s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
39s{color} | {color:red} ql: The patch generated 10 new + 43 unchanged - 0
fixed = 53 total (was 43) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
10s{color} | {color:red} ql generated 2 new + 2287 unchanged - 0 fixed = 2289
total (was 2287) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 27m 1s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | Should
org.apache.hadoop.hive.ql.parse.TezCompiler$RedundantSemijoinAndDppContext be a
_static_ inner class? At TezCompiler.java:inner class? At
TezCompiler.java:[lines 1150-1159] |
| | Redundant nullcheck of currentOp, which is known to be non-null in
org.apache.hadoop.hive.ql.ppd.SyntheticJoinPredicate$JoinSynthetic.createDerivatives(List,
Operator, ExprNodeDesc, ExprNodeDesc) Redundant null check at
SyntheticJoinPredicate.java:is known to be non-null in
org.apache.hadoop.hive.ql.ppd.SyntheticJoinPredicate$JoinSynthetic.createDerivatives(List,
Operator, ExprNodeDesc, ExprNodeDesc) Redundant null check at
SyntheticJoinPredicate.java:[line 302] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-12404/dev-support/hive-personality.sh
|
| git revision | master / e865b44 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12404/yetus/diff-checkstyle-ql.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12404/yetus/new-findbugs-ql.html
|
| modules | C: common ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12404/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Extend creation of semijoin reduction filters to be able to discover new
> opportunities
> --
>
> Key: HIVE-20090
>
[jira] [Assigned] (HIVE-1696) Add delegation token support to metastore
[ https://issues.apache.org/jira/browse/HIVE-1696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhangbutao reassigned HIVE-1696: Assignee: luhuachao (was: Devaraj Das) > Add delegation token support to metastore > - > > Key: HIVE-1696 > URL: https://issues.apache.org/jira/browse/HIVE-1696 > Project: Hive > Issue Type: Sub-task > Components: Authentication, Metastore, Security, Server > Infrastructure >Reporter: Todd Lipcon >Assignee: luhuachao >Priority: Major > Fix For: 0.7.0 > > Attachments: hive-1696-1-with-gen-code.patch, hive-1696-1.patch, > hive-1696-3-with-gen-code.patch, hive-1696-3.patch, > hive-1696-4-with-gen-code.1.patch, hive-1696-4-with-gen-code.patch, > hive-1696-4.patch, hive-1696-4.patch, hive_1696.patch, hive_1696.patch, > hive_1696_no-thrift.patch > > > As discussed in HIVE-842, kerberos authentication is only sufficient for > authentication of a hive user client to the metastore. There are other cases > where thrift calls need to be authenticated when the caller is running in an > environment without kerberos credentials. For example, an MR task running as > part of a hive job may want to report statistics to the metastore, or a job > may be running within the context of Oozie or Hive Server. > This JIRA is to implement support of delegation tokens for the metastore. The > concept of a delegation token is borrowed from the Hadoop security design - > the quick summary is that a kerberos-authenticated client may retrieve a > binary token from the server. This token can then be passed to other clients > which can use it to achieve authentication as the original user in lieu of a > kerberos ticket. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-1696) Add delegation token support to metastore
[ https://issues.apache.org/jira/browse/HIVE-1696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhangbutao reassigned HIVE-1696: Assignee: Devaraj Das (was: luhuachao) > Add delegation token support to metastore > - > > Key: HIVE-1696 > URL: https://issues.apache.org/jira/browse/HIVE-1696 > Project: Hive > Issue Type: Sub-task > Components: Authentication, Metastore, Security, Server > Infrastructure >Reporter: Todd Lipcon >Assignee: Devaraj Das >Priority: Major > Fix For: 0.7.0 > > Attachments: hive-1696-1-with-gen-code.patch, hive-1696-1.patch, > hive-1696-3-with-gen-code.patch, hive-1696-3.patch, > hive-1696-4-with-gen-code.1.patch, hive-1696-4-with-gen-code.patch, > hive-1696-4.patch, hive-1696-4.patch, hive_1696.patch, hive_1696.patch, > hive_1696_no-thrift.patch > > > As discussed in HIVE-842, kerberos authentication is only sufficient for > authentication of a hive user client to the metastore. There are other cases > where thrift calls need to be authenticated when the caller is running in an > environment without kerberos credentials. For example, an MR task running as > part of a hive job may want to report statistics to the metastore, or a job > may be running within the context of Oozie or Hive Server. > This JIRA is to implement support of delegation tokens for the metastore. The > concept of a delegation token is borrowed from the Hadoop security design - > the quick summary is that a kerberos-authenticated client may retrieve a > binary token from the server. This token can then be passed to other clients > which can use it to achieve authentication as the original user in lieu of a > kerberos ticket. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20090) Extend creation of semijoin reduction filters to be able to discover new opportunities
[
https://issues.apache.org/jira/browse/HIVE-20090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16533839#comment-16533839
]
Hive QA commented on HIVE-20090:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12930318/HIVE-20090.01.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 417 failed/errored test(s), 14639 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dynamic_semijoin_reduction_sw2]
(batchId=80)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druid_basic2]
(batchId=189)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_joins]
(batchId=191)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[dynamic_partition_pruning_2]
(batchId=151)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[dynamic_semijoin_user_level]
(batchId=152)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[except_distinct]
(batchId=152)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[explainuser_2]
(batchId=153)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[intersect_all]
(batchId=151)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[intersect_distinct]
(batchId=153)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_nullscan]
(batchId=149)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llapdecider]
(batchId=149)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[mm_all]
(batchId=152)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_ppd_basic]
(batchId=149)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[unionDistinct_1]
(batchId=151)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[acid_no_buckets]
(batchId=172)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[autoColumnStats_1]
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[autoColumnStats_2]
(batchId=177)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[auto_join1]
(batchId=175)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[auto_join21]
(batchId=176)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[auto_join29]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[auto_join30]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[auto_join_filters]
(batchId=170)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[auto_join_nulls]
(batchId=172)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[auto_smb_mapjoin_14]
(batchId=170)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[auto_sortmerge_join_10]
(batchId=173)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[auto_sortmerge_join_11]
(batchId=177)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[auto_sortmerge_join_12]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[auto_sortmerge_join_13]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[auto_sortmerge_join_14]
(batchId=157)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[auto_sortmerge_join_15]
(batchId=157)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[auto_sortmerge_join_16]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[auto_sortmerge_join_1]
(batchId=166)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[auto_sortmerge_join_2]
(batchId=167)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[auto_sortmerge_join_3]
(batchId=155)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[auto_sortmerge_join_4]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[auto_sortmerge_join_5]
(batchId=178)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[auto_sortmerge_join_6]
(batchId=158)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[auto_sortmerge_join_7]
(batchId=178)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[auto_sortmerge_join_8]
(batchId=174)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[auto_sortmerge_join_9]
(batchId=172)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez2]
(batchId=157)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez_empty]
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucketmapjoin1]
(batchId=177)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucketmapjoin2
[jira] [Commented] (HIVE-18882) Do Not Hide Exception in Hive Metastore Client Connection
[
https://issues.apache.org/jira/browse/HIVE-18882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16533840#comment-16533840
]
Naveen Gangam commented on HIVE-18882:
--
I do not see a reason why debug needs to be enabled to be able to see the full
stack trace. Just a quick followup question though. With the retrying metastore
client in HS2, does this exception get logged many times in the HS2 log when
HMS is down? Could you test with starting a HS2 or even Hive CLI without a
running HMS to see what gets spit out?
Otherwise, change looks good to me. Thanks
> Do Not Hide Exception in Hive Metastore Client Connection
> -
>
> Key: HIVE-18882
> URL: https://issues.apache.org/jira/browse/HIVE-18882
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore
>Affects Versions: 3.0.0
>Reporter: BELUGA BEHR
>Assignee: Manoj Narayanan
>Priority: Minor
> Labels: noob
> Attachments: HIVE-18882.1.patch, HIVE-18882.2.patch, HIVE-18882.patch
>
>
> [https://github.com/apache/hive/blob/4047befe48c8f762c58d8854e058385c1df151c6/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java#L526-L531]
>
> {code:java}
> if (LOG.isDebugEnabled()) {
> LOG.warn("Failed to connect to the MetaStore Server...", e);
> } else {
> // Don't print full exception trace if DEBUG is not on.
> LOG.warn("Failed to connect to the MetaStore Server...");
> }
> {code}
> I do not understand the logic here. I always want to see the reason for the
> failure. Otherwise, I do not know why it is failing unless I restart the
> server with debug logging enabled. By that point, the error may have
> cleared. Please just use the Exception in the WARN output without adding
> confusing logging for debugging. This is never an expected behavior... that
> enabling debug would change a _warn_ level log message.
> Also... please remove the ellipsis, they add no value.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20073) Additional tests for to_utc_timestamp function based on HIVE-20068
[
https://issues.apache.org/jira/browse/HIVE-20073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16533851#comment-16533851
]
Hive QA commented on HIVE-20073:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
21s{color} | {color:blue} Maven dependency ordering for branch {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
40s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
10s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 2m 41s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-12405/dev-support/hive-personality.sh
|
| git revision | master / e865b44 |
| modules | C: ql itests U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12405/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Additional tests for to_utc_timestamp function based on HIVE-20068
> --
>
> Key: HIVE-20073
> URL: https://issues.apache.org/jira/browse/HIVE-20073
> Project: Hive
> Issue Type: Bug
> Environment: MapR running on Linux I believe. Client is DBeaver on
> Windows 7.
>Reporter: JAMES J STEINBUGL
>Assignee: Jesus Camacho Rodriguez
>Priority: Minor
> Attachments: HIVE-20073.01.patch, HIVE-20073.patch,
> image-2018-07-03-08-50-42-390.png
>
>
> I have the following script and I'm at loss to explain the behavior.
> Possibly it's an older bug as we are using the 2.1.1 drivers (?). We noticed
> this issue when converting from US/Eastern into UTC and then back to
> US/Eastern. Everything that was in Status Date / Status Hour on 3/11/17
> 21:00:00 shifted 6 hours ahead into UTC ... then shifted back to 3/11/17
> 22:00:00 back in US/Eastern. The behavior appears to be the same using the
> constant EST5EDT. EDT was effective on 3/12 2 am, so the issue appears only
> at this boundary condition when we "spring ahead", but it at least on the
> surface seems incorrect.
> --
> -- Potential Issue with to_utc_timestamp
> ---
> SELECT '2017-03-11 18:00:00', to_utc_timestamp(timestamp '2017-03-11
> 18:00:00','US/Eastern'); -- Shifts ahead 5 hours as expected
> SELECT '2017-03-11 19:00:00', to_utc_timestamp(timestamp '2017-03-11
> 19:00:00','US/Eastern'); -- Shifts ahead 5 hours as expected
> SELECT '2017-03-11 20:00:00', to_utc_timestamp(timestamp '2017-03-11
> 20:00:00','US/Eastern'); -- Shifts ahead 5 hours as expected
> {color:#FF}SELECT '2017-03-11 21:00:00', to_utc_timestamp(timestamp
> '2017-03-11 21:00:00','US/Eastern'); -- Shifts ahead 6 hours (???){color}
> {color:#FF}_c0 _c1
> 2017-03-11 21:00:00 2017-03-12 03:00:00{color}
> SELECT '2017-03-11 22:00:00', to_utc_timestamp(timestamp '2017-03-11
> 22:00:00','US/Eastern'); -- Shifts ahead 5 hours as expected
> SELECT '2017-03-11 23:00:00', to_utc_timestamp(timestamp '2017-03-11
> 23:00:00','US/Eastern'); -- Shifts ahead 5 hours as expected
> SELECT '2017-03-12 00:00:00', to_utc_timestamp(timestamp '2017-03-12
> 00:00:00','US/Eastern'); -- Shifts ahead 5 hours as expected
> SELECT '2017-03-12 01:00:00', to_utc_timestamp(timestamp '2017-03-12
> 01:00:00','US/Eastern'); -- Shifts ahead 5 hours as expected
> SELECT '2017-03-12 02:00:00', to_utc_timestamp(timestamp '2017-03-12
> 02:00:00','US/Eastern'); -- Shifts ahead 5 hours as expected
> SELECT '2017-03-12 03:00:00', to_utc_timestamp(timestamp '2017-03-12
> 03:00:00','US/Eastern
[jira] [Commented] (HIVE-20096) dynamic partition pruning results in java.lang.UnsupportedOperationException: Not expecting to handle any events
[
https://issues.apache.org/jira/browse/HIVE-20096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16533871#comment-16533871
]
Gopal V commented on HIVE-20096:
bq. at
org.apache.tez.mapreduce.common.MRInputAMSplitGenerator.handleInputInitializerEvent(MRInputAMSplitGenerator.java:170)
This is supposed to be using HiveSplitGenerator in Hive.
https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/HiveSplitGenerator.java#L377
Looks like a configuration issue with the split-generation, where the
CombineInputFormat splits are generated in the CLI & sent over RPC (instead of
being generated on the AM from HiveInputFormat).
Check {{hive.tez.input.format}} and {{hive.compute.splits.in.am}}.
> dynamic partition pruning results in java.lang.UnsupportedOperationException:
> Not expecting to handle any events
>
>
> Key: HIVE-20096
> URL: https://issues.apache.org/jira/browse/HIVE-20096
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.0.0
> Environment: Hive 3.0.0 and Tez 0.91 .
>Reporter: Karthik
>Priority: Major
>
> * Setting hive.tez.dynamic.partition.pruning=true partition results in error
> "java.lang.UnsupportedOperationException: Not expecting to handle any events"
> for some queries.
> * Below is stack trace as shown in Hive CLI and JDBC.
> --
> VERTICES MODE STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED
> --
> Map 3 llap INITED 1 0 0 1 0 0
> Map 4 .. llap SUCCEEDED 1 1 0 0 0 0
> Map 1 llap INITIALIZING -1 0 0 -1 0 0
> Reducer 2 llap INITED 3 0 0 3 0 0
> --
> VERTICES: 01/04 [=>>-] 20% ELAPSED TIME: 0.56 s
> --
> 2018-07-05 06:41:22,813 INFO [32a75ad3-f0de-4255-b1d5-0c919e3bc672 main]
> SessionState (SessionState.java:logInfo(1163)) - Map 1: -/- Map 3: 0/1 Map 4:
> 1/1 Reducer 2: 0/3
> Status: Failed
> 2018-07-05 06:41:22,814 ERROR [32a75ad3-f0de-4255-b1d5-0c919e3bc672 main]
> SessionState (SessionState.java:printError(1237)) - Status: Failed
> Vertex failed, vertexName=Map 1, vertexId=vertex_1530776596475_0020_4_02,
> diagnostics=[Vertex vertex_1530776596475_0020_4_02 [Map 1] killed/failed due
> to:ROOT_INPUT_INIT_FAILURE, Vertex Input: my_fact_table initializer failed,
> vertex=vertex_1530776596475_0020_4_02 [Map 1],
> java.lang.UnsupportedOperationException: Not expecting to handle any events
> at
> org.apache.tez.mapreduce.common.MRInputAMSplitGenerator.handleInputInitializerEvent(MRInputAMSplitGenerator.java:170)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InitializerWrapper.sendEvents(RootInputInitializerManager.java:518)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InitializerWrapper.handleInputInitializerEvents(RootInputInitializerManager.java:511)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager.handleInitializerEvents(RootInputInitializerManager.java:201)
> at
> org.apache.tez.dag.app.dag.impl.VertexImpl.handleRoutedTezEvents(VertexImpl.java:4009)
> at
> org.apache.tez.dag.app.dag.impl.VertexImpl.access$4000(VertexImpl.java:205)
> at
> org.apache.tez.dag.app.dag.impl.VertexImpl$RouteEventTransition.transition(VertexImpl.java:3738)
> at
> org.apache.tez.dag.app.dag.impl.VertexImpl$RouteEventTransition.transition(VertexImpl.java:3731)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory$MultipleInternalArc.doTransition(StateMachineFactory.java:385)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:302)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory.access$500(StateMachineFactory.java:46)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:487)
> at org.apache.tez.state.StateMachineTez.doTransition(StateMachineTez.java:59)
> at org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:1939)
> at org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:204)
> at
> org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:2317)
> at
> org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:2303)
> at org.apache.tez.common.AsyncDispatcher.dispatch(AsyncDispatcher.java:180)
> at org.apache.tez.common.AsyncDispatcher$1.run(AsyncDispatcher.java:115)
> at java.lang.Thread.run(Thread.java:748)
> ]
> 2018-07-0
[jira] [Commented] (HIVE-19940) Push predicates with deterministic UDFs with RBO
[
https://issues.apache.org/jira/browse/HIVE-19940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16533873#comment-16533873
]
Janaki Lahorani commented on HIVE-19940:
Thanks [~ngangam]. The same test case is added as part of the patch in
ppd_deterministic_expr.q. The new plan is part of
ppd_deterministic_expr.q.out.
Query:
{code}
explain
select vcol1, vcol2, vcol3, vcola, vcolb
from viewDeterministicUDFA a inner join viewDeterministicUDFB b
on a.vpart1 = b.vpart1
and a.vpart2 = b.vpart2
and a.vpart3 = b.vpart3
and a.vpart1 = 'US'
and a.vpart2 = 'DEF'
and a.vpart3 = '200';
{code}
New Plan:
{code}
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
STAGE PLANS:
Stage: Stage-1
Map Reduce
Map Operator Tree:
TableScan
alias: testa
filterExpr: (part1) IN ('US', 'CA') (type: boolean)
Statistics: Num rows: 2 Data size: 4580 Basic stats: COMPLETE
Column stats: NONE
Select Operator
expressions: CAST( col1 AS decimal(38,18)) (type:
decimal(38,18)), CAST( col2 AS decimal(38,18)) (type: decimal(38,18)), CAST(
col3 AS decimal(38,18)) (type: decimal(38,18))
outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 2 Data size: 4580 Basic stats: COMPLETE
Column stats: NONE
Reduce Output Operator
key expressions: 'US' (type: char(2)), 'DEF' (type: char(3)),
'200' (type: char(3))
sort order: +++
Map-reduce partition columns: 'US' (type: char(2)), 'DEF'
(type: char(3)), '200' (type: char(3))
Statistics: Num rows: 2 Data size: 4580 Basic stats: COMPLETE
Column stats: NONE
value expressions: _col0 (type: decimal(38,18)), _col1 (type:
decimal(38,18)), _col2 (type: decimal(38,18))
TableScan
alias: testb
filterExpr: (part1) IN ('US', 'CA') (type: boolean)
Statistics: Num rows: 2 Data size: 3180 Basic stats: COMPLETE
Column stats: NONE
Select Operator
expressions: CAST( cola AS decimal(38,18)) (type:
decimal(38,18)), CAST( colb AS decimal(38,18)) (type: decimal(38,18)), CAST(
part1 AS CHAR(2)) (type: char(2)), CAST( part2 AS CHAR(3)) (type: char(3))
outputColumnNames: _col0, _col1, _col3, _col4
Statistics: Num rows: 2 Data size: 3180 Basic stats: COMPLETE
Column stats: NONE
Reduce Output Operator
key expressions: _col3 (type: char(2)), _col4 (type: char(3)),
'200' (type: char(3))
sort order: +++
Map-reduce partition columns: _col3 (type: char(2)), _col4
(type: char(3)), '200' (type: char(3))
Statistics: Num rows: 2 Data size: 3180 Basic stats: COMPLETE
Column stats: NONE
value expressions: _col0 (type: decimal(38,18)), _col1 (type:
decimal(38,18))
Reduce Operator Tree:
Join Operator
condition map:
Inner Join 0 to 1
keys:
0 _col5 (type: char(2)), _col6 (type: char(3)), _col7 (type:
char(3))
1 _col3 (type: char(2)), _col4 (type: char(3)), _col5 (type:
char(3))
outputColumnNames: _col0, _col1, _col2, _col8, _col9
Statistics: Num rows: 2 Data size: 5038 Basic stats: COMPLETE Column
stats: NONE
Select Operator
expressions: _col0 (type: decimal(38,18)), _col1 (type:
decimal(38,18)), _col2 (type: decimal(38,18)), _col8 (type: decimal(38,18)),
_col9 (type: decimal(38,18))
outputColumnNames: _col0, _col1, _col2, _col3, _col4
Statistics: Num rows: 2 Data size: 5038 Basic stats: COMPLETE
Column stats: NONE
File Output Operator
compressed: false
Statistics: Num rows: 2 Data size: 5038 Basic stats: COMPLETE
Column stats: NONE
table:
input format: org.apache.hadoop.mapred.SequenceFileInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
ListSink
{code}
> Push predicates with deterministic UDFs with RBO
>
>
> Key: HIVE-19940
> URL: https://issues.apache.org/jira/browse/HIVE-19940
> Project: Hive
> Issue Type: Improvement
>Reporter: Janaki Lahorani
>Assignee: Janaki Lahorani
>Priority: Major
> Attachments: HIVE-19940.1.patch, HIVE-19940.2.patch
>
>
> With RBO, predicates with any UDF doesn't get pushed down. It makes sense to
> not pushdown the predicates with non-deterministic function as the meaning of
> the query changes after the predic
[jira] [Commented] (HIVE-20041) ResultsCache: Improve loggng for concurrent queries
[
https://issues.apache.org/jira/browse/HIVE-20041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16533882#comment-16533882
]
Gopal V commented on HIVE-20041:
The patch is ready, will look at filing a flaky-test JIRA for the druid timeout.
{code}
2018-07-05T02:33:22,883 WARN [a1043130-e1f6-4bdc-a880-9f641efce907 main]
common.RetryUtils: Failed on try 7, retrying in 36,587ms.
org.skife.jdbi.v2.exceptions.UnableToObtainConnectionException:
java.sql.SQLException: Cannot create PoolableConnectionFactory
(java.net.ConnectException : Error connecting to server localhost on port 1,527
with message Connection refused.)
at org.skife.jdbi.v2.DBI.open(DBI.java:230)
~[hive-druid-handler-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at org.skife.jdbi.v2.DBI.withHandle(DBI.java:279)
~[hive-druid-handler-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at
org.apache.hive.druid.io.druid.metadata.SQLMetadataConnector$2.call(SQLMetadataConnector.java:135)
~[hive-druid-handler-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at
org.apache.hive.druid.io.druid.java.util.common.RetryUtils.retry(RetryUtils.java:63)
[hive-druid-handler-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at
org.apache.hive.druid.io.druid.java.util.common.RetryUtils.retry(RetryUtils.java:81)
[hive-druid-handler-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
{code}
Looks to be a derby connection failure, not an actual result failure.
> ResultsCache: Improve loggng for concurrent queries
> ---
>
> Key: HIVE-20041
> URL: https://issues.apache.org/jira/browse/HIVE-20041
> Project: Hive
> Issue Type: Improvement
> Components: Diagnosability
>Reporter: Gopal V
>Assignee: Laszlo Bodor
>Priority: Minor
> Attachments: HIVE-20041.01.patch, HIVE-20041.02.patch,
> HIVE-20041.03.patch, HIVE-20041.04.patch
>
>
> The logging for QueryResultsCache ends up printing information without
> context, like
> {code}
> 2018-06-30T17:48:45,502 INFO [HiveServer2-Background-Pool: Thread-166]
> results.QueryResultsCache: Waiting on pending cacheEntry
> {code}
> {code}
> 2018-06-30T17:50:17,963 INFO [HiveServer2-Background-Pool: Thread-145]
> ql.Driver: savedToCache: true
> {code}
> The previous lines for this are in DEBUG level, so the logging ends up being
> useless at INFO level to debug.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20073) Additional tests for to_utc_timestamp function based on HIVE-20068
[
https://issues.apache.org/jira/browse/HIVE-20073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16533910#comment-16533910
]
Hive QA commented on HIVE-20073:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12930319/HIVE-20073.01.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 14639 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/12405/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12405/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12405/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12930319 - PreCommit-HIVE-Build
> Additional tests for to_utc_timestamp function based on HIVE-20068
> --
>
> Key: HIVE-20073
> URL: https://issues.apache.org/jira/browse/HIVE-20073
> Project: Hive
> Issue Type: Bug
> Environment: MapR running on Linux I believe. Client is DBeaver on
> Windows 7.
>Reporter: JAMES J STEINBUGL
>Assignee: Jesus Camacho Rodriguez
>Priority: Minor
> Attachments: HIVE-20073.01.patch, HIVE-20073.patch,
> image-2018-07-03-08-50-42-390.png
>
>
> I have the following script and I'm at loss to explain the behavior.
> Possibly it's an older bug as we are using the 2.1.1 drivers (?). We noticed
> this issue when converting from US/Eastern into UTC and then back to
> US/Eastern. Everything that was in Status Date / Status Hour on 3/11/17
> 21:00:00 shifted 6 hours ahead into UTC ... then shifted back to 3/11/17
> 22:00:00 back in US/Eastern. The behavior appears to be the same using the
> constant EST5EDT. EDT was effective on 3/12 2 am, so the issue appears only
> at this boundary condition when we "spring ahead", but it at least on the
> surface seems incorrect.
> --
> -- Potential Issue with to_utc_timestamp
> ---
> SELECT '2017-03-11 18:00:00', to_utc_timestamp(timestamp '2017-03-11
> 18:00:00','US/Eastern'); -- Shifts ahead 5 hours as expected
> SELECT '2017-03-11 19:00:00', to_utc_timestamp(timestamp '2017-03-11
> 19:00:00','US/Eastern'); -- Shifts ahead 5 hours as expected
> SELECT '2017-03-11 20:00:00', to_utc_timestamp(timestamp '2017-03-11
> 20:00:00','US/Eastern'); -- Shifts ahead 5 hours as expected
> {color:#FF}SELECT '2017-03-11 21:00:00', to_utc_timestamp(timestamp
> '2017-03-11 21:00:00','US/Eastern'); -- Shifts ahead 6 hours (???){color}
> {color:#FF}_c0 _c1
> 2017-03-11 21:00:00 2017-03-12 03:00:00{color}
> SELECT '2017-03-11 22:00:00', to_utc_timestamp(timestamp '2017-03-11
> 22:00:00','US/Eastern'); -- Shifts ahead 5 hours as expected
> SELECT '2017-03-11 23:00:00', to_utc_timestamp(timestamp '2017-03-11
> 23:00:00','US/Eastern'); -- Shifts ahead 5 hours as expected
> SELECT '2017-03-12 00:00:00', to_utc_timestamp(timestamp '2017-03-12
> 00:00:00','US/Eastern'); -- Shifts ahead 5 hours as expected
> SELECT '2017-03-12 01:00:00', to_utc_timestamp(timestamp '2017-03-12
> 01:00:00','US/Eastern'); -- Shifts ahead 5 hours as expected
> SELECT '2017-03-12 02:00:00', to_utc_timestamp(timestamp '2017-03-12
> 02:00:00','US/Eastern'); -- Shifts ahead 5 hours as expected
> SELECT '2017-03-12 03:00:00', to_utc_timestamp(timestamp '2017-03-12
> 03:00:00','US/Eastern'); -- Shifts ahead 4 hours as expected
> SELECT '2017-03-12 04:00:00', to_utc_timestamp(timestamp '2017-03-12
> 04:00:00','US/Eastern'); -- Shifts ahead 4 hours as expected
> SELECT '2017-03-12 05:00:00', to_utc_timestamp(timestamp '2017-03-12
> 05:00:00','US/Eastern'); -- Shifts ahead 4 hours as expected
> !image-2018-07-03-08-50-42-390.png!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20090) Extend creation of semijoin reduction filters to be able to discover new opportunities
[
https://issues.apache.org/jira/browse/HIVE-20090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16533911#comment-16533911
]
Hive QA commented on HIVE-20090:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12930318/HIVE-20090.01.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/12406/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12406/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12406/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12930318/HIVE-20090.01.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12930318 - PreCommit-HIVE-Build
> Extend creation of semijoin reduction filters to be able to discover new
> opportunities
> --
>
> Key: HIVE-20090
> URL: https://issues.apache.org/jira/browse/HIVE-20090
> Project: Hive
> Issue Type: Improvement
> Components: Physical Optimizer
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-20090.01.patch
>
>
> Assume the following plan:
> {noformat}
> TS[0] - RS[1] - JOIN[4] - RS[5] - JOIN[8] - FS[9]
> TS[2] - RS[3] - JOIN[4]
> TS[6] - RS[7] - JOIN[8]
> {noformat}
> Currently, {{TS\[6\]}} may only be reduced with the output of {{RS\[5\]}},
> i.e., input to join between both subplans.
> However, it may be useful to consider other possibilities too, e.g., reduced
> by the output of {{RS\[1\]}} or {{RS\[3\]}}. For instance, this is important
> when, given a large plan, an edge between {{RS[5]}} and {{TS[0]}} would
> create a cycle, while an edge between {{RS[1]}} and {{TS[6]}} would not.
> This patch comprises two parts. First, it creates additional predicates when
> possible. Secondly, it removes duplicate semijoin reduction
> branches/predicates, e.g., if another semijoin that consumes the output of
> the same expression already reduces a certain table scan operator (heuristic,
> since this may not result in most efficient plan in all cases). Ultimately,
> the decision on whether to use one or another should be cost-driven
> (follow-up).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20095) Fix jdbc external table feature
[ https://issues.apache.org/jira/browse/HIVE-20095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16533913#comment-16533913 ] Jesus Camacho Rodriguez commented on HIVE-20095: [~daijy], it seems HIVE-19161 broke the integration with JDBC pushdown from Calcite side. Can you comment on this patch? [~msydoron], I did not see any test related to what was broken. Can you add tests to the patch? In addition, HIVE-19299 still needs to be addressed. Without proper testing coverage, this situation will continue repeating itself. > Fix jdbc external table feature > --- > > Key: HIVE-20095 > URL: https://issues.apache.org/jira/browse/HIVE-20095 > Project: Hive > Issue Type: Bug >Reporter: Jonathan Doron >Assignee: Jonathan Doron >Priority: Major > Attachments: HIVE-20095.1.patch > > > It seems like the committed code for HIVE-19161 > (7584b3276bebf64aa006eaa162c0a6264d8fcb56) reverted some of HIVE-18423 > updates, and therefore some of the external table queries are not working > correctly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20073) Additional tests for to_utc_timestamp function based on HIVE-20068
[
https://issues.apache.org/jira/browse/HIVE-20073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16533916#comment-16533916
]
Jesus Camacho Rodriguez commented on HIVE-20073:
[~ashutoshc], could you review? Thanks
> Additional tests for to_utc_timestamp function based on HIVE-20068
> --
>
> Key: HIVE-20073
> URL: https://issues.apache.org/jira/browse/HIVE-20073
> Project: Hive
> Issue Type: Bug
> Environment: MapR running on Linux I believe. Client is DBeaver on
> Windows 7.
>Reporter: JAMES J STEINBUGL
>Assignee: Jesus Camacho Rodriguez
>Priority: Minor
> Attachments: HIVE-20073.01.patch, HIVE-20073.patch,
> image-2018-07-03-08-50-42-390.png
>
>
> I have the following script and I'm at loss to explain the behavior.
> Possibly it's an older bug as we are using the 2.1.1 drivers (?). We noticed
> this issue when converting from US/Eastern into UTC and then back to
> US/Eastern. Everything that was in Status Date / Status Hour on 3/11/17
> 21:00:00 shifted 6 hours ahead into UTC ... then shifted back to 3/11/17
> 22:00:00 back in US/Eastern. The behavior appears to be the same using the
> constant EST5EDT. EDT was effective on 3/12 2 am, so the issue appears only
> at this boundary condition when we "spring ahead", but it at least on the
> surface seems incorrect.
> --
> -- Potential Issue with to_utc_timestamp
> ---
> SELECT '2017-03-11 18:00:00', to_utc_timestamp(timestamp '2017-03-11
> 18:00:00','US/Eastern'); -- Shifts ahead 5 hours as expected
> SELECT '2017-03-11 19:00:00', to_utc_timestamp(timestamp '2017-03-11
> 19:00:00','US/Eastern'); -- Shifts ahead 5 hours as expected
> SELECT '2017-03-11 20:00:00', to_utc_timestamp(timestamp '2017-03-11
> 20:00:00','US/Eastern'); -- Shifts ahead 5 hours as expected
> {color:#FF}SELECT '2017-03-11 21:00:00', to_utc_timestamp(timestamp
> '2017-03-11 21:00:00','US/Eastern'); -- Shifts ahead 6 hours (???){color}
> {color:#FF}_c0 _c1
> 2017-03-11 21:00:00 2017-03-12 03:00:00{color}
> SELECT '2017-03-11 22:00:00', to_utc_timestamp(timestamp '2017-03-11
> 22:00:00','US/Eastern'); -- Shifts ahead 5 hours as expected
> SELECT '2017-03-11 23:00:00', to_utc_timestamp(timestamp '2017-03-11
> 23:00:00','US/Eastern'); -- Shifts ahead 5 hours as expected
> SELECT '2017-03-12 00:00:00', to_utc_timestamp(timestamp '2017-03-12
> 00:00:00','US/Eastern'); -- Shifts ahead 5 hours as expected
> SELECT '2017-03-12 01:00:00', to_utc_timestamp(timestamp '2017-03-12
> 01:00:00','US/Eastern'); -- Shifts ahead 5 hours as expected
> SELECT '2017-03-12 02:00:00', to_utc_timestamp(timestamp '2017-03-12
> 02:00:00','US/Eastern'); -- Shifts ahead 5 hours as expected
> SELECT '2017-03-12 03:00:00', to_utc_timestamp(timestamp '2017-03-12
> 03:00:00','US/Eastern'); -- Shifts ahead 4 hours as expected
> SELECT '2017-03-12 04:00:00', to_utc_timestamp(timestamp '2017-03-12
> 04:00:00','US/Eastern'); -- Shifts ahead 4 hours as expected
> SELECT '2017-03-12 05:00:00', to_utc_timestamp(timestamp '2017-03-12
> 05:00:00','US/Eastern'); -- Shifts ahead 4 hours as expected
> !image-2018-07-03-08-50-42-390.png!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-17751) Separate HMS Client and HMS server into separate sub-modules
[ https://issues.apache.org/jira/browse/HIVE-17751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Kolbasov updated HIVE-17751: -- Attachment: HIVE-17751.11.patch > Separate HMS Client and HMS server into separate sub-modules > > > Key: HIVE-17751 > URL: https://issues.apache.org/jira/browse/HIVE-17751 > Project: Hive > Issue Type: Sub-task > Components: Standalone Metastore >Reporter: Vihang Karajgaonkar >Assignee: Alexander Kolbasov >Priority: Major > Attachments: HIVE-17751.01.patch, HIVE-17751.02.patch, > HIVE-17751.03.patch, HIVE-17751.04.patch, > HIVE-17751.06-standalone-metastore.patch, HIVE-17751.07.patch, > HIVE-17751.08.patch, HIVE-17751.09.patch, HIVE-17751.10.patch, > HIVE-17751.11.patch > > > external applications which are interfacing with HMS should ideally only > include HMSClient library instead of one big library containing server as > well. We should ideally have a thin client library so that cross version > support for external applications is easier. We should sub-divide the > standalone module into possibly 3 modules (one for common classes, one for > client classes and one for server) or 2 sub-modules (one for client and one > for server) so that we can generate separate jars for HMS client and server. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-17751) Separate HMS Client and HMS server into separate sub-modules
[
https://issues.apache.org/jira/browse/HIVE-17751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16533935#comment-16533935
]
Alexander Kolbasov commented on HIVE-17751:
---
Patch 11 merged with
{code}
* commit e865b449efca04b02922bbc42693a1e8f91825a9 (origin/master, origin/HEAD)
| Author: Ganesha Shreedhara
| Date: Wed Jul 4 23:03:10 2018 -0700
|
| HIVE-19850 : Dynamic partition pruning in Tez is leading to 'No work
found for tablescan' error (Ganesha Shreedhara via Ashutosh Chauhan)
{code}
> Separate HMS Client and HMS server into separate sub-modules
>
>
> Key: HIVE-17751
> URL: https://issues.apache.org/jira/browse/HIVE-17751
> Project: Hive
> Issue Type: Sub-task
> Components: Standalone Metastore
>Reporter: Vihang Karajgaonkar
>Assignee: Alexander Kolbasov
>Priority: Major
> Attachments: HIVE-17751.01.patch, HIVE-17751.02.patch,
> HIVE-17751.03.patch, HIVE-17751.04.patch,
> HIVE-17751.06-standalone-metastore.patch, HIVE-17751.07.patch,
> HIVE-17751.08.patch, HIVE-17751.09.patch, HIVE-17751.10.patch,
> HIVE-17751.11.patch
>
>
> external applications which are interfacing with HMS should ideally only
> include HMSClient library instead of one big library containing server as
> well. We should ideally have a thin client library so that cross version
> support for external applications is easier. We should sub-divide the
> standalone module into possibly 3 modules (one for common classes, one for
> client classes and one for server) or 2 sub-modules (one for client and one
> for server) so that we can generate separate jars for HMS client and server.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-17751) Separate HMS Client and HMS server into separate sub-modules
[ https://issues.apache.org/jira/browse/HIVE-17751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16533947#comment-16533947 ] Alan Gates commented on HIVE-17751: --- Regarding the endless merge issue, I can sympathize. Slowing down other metastore patches likely won't be an option as others have their work to get done too. The best approach I found is to keep the patches small so that any given change has a lower chance of being broken by others' changes, or at least so that the breakage was small enough it was easy to do the merges. Would it be possible to break this up into multiple patches where the first just moves stuff into standalone-metastore/common, then one for standalone-metastore/client, and finally one for standalone-metastore/server? Would that help? > Separate HMS Client and HMS server into separate sub-modules > > > Key: HIVE-17751 > URL: https://issues.apache.org/jira/browse/HIVE-17751 > Project: Hive > Issue Type: Sub-task > Components: Standalone Metastore >Reporter: Vihang Karajgaonkar >Assignee: Alexander Kolbasov >Priority: Major > Attachments: HIVE-17751.01.patch, HIVE-17751.02.patch, > HIVE-17751.03.patch, HIVE-17751.04.patch, > HIVE-17751.06-standalone-metastore.patch, HIVE-17751.07.patch, > HIVE-17751.08.patch, HIVE-17751.09.patch, HIVE-17751.10.patch, > HIVE-17751.11.patch > > > external applications which are interfacing with HMS should ideally only > include HMSClient library instead of one big library containing server as > well. We should ideally have a thin client library so that cross version > support for external applications is easier. We should sub-divide the > standalone module into possibly 3 modules (one for common classes, one for > client classes and one for server) or 2 sub-modules (one for client and one > for server) so that we can generate separate jars for HMS client and server. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-20080) TxnHandler checkLock direct sql fail with ORA-01795 , if the table has more than 1000 partitions
[
https://issues.apache.org/jira/browse/HIVE-20080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rajkumar Singh reassigned HIVE-20080:
-
Assignee: Rajkumar Singh
> TxnHandler checkLock direct sql fail with ORA-01795 , if the table has more
> than 1000 partitions
>
>
> Key: HIVE-20080
> URL: https://issues.apache.org/jira/browse/HIVE-20080
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 2.1.0
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
>
> with Oracle as Metastore, txnhandler checkLock fail with
> "checkLockWithRetry(181398,34773) : ORA-01795: maximum number of expressions
> in a list is 1000" if the write table has more than 1000 partitions.
> complete stacktrace
> {code}
> txn.TxnHandler (TxnHandler.java:checkRetryable(2099)) - Non-retryable error
> in checkLockWithRetry(181398,34773) : ORA-01795: maximum number of
> expressions in a list is 1000
> (SQLState=42000, ErrorCode=1795)
> 2018-06-25 15:09:35,999 ERROR [pool-7-thread-197]:
> metastore.RetryingHMSHandler (RetryingHMSHandler.java:invokeInternal(203)) -
> MetaException(message:Unable to update transaction database
> java.sql.SQLSyntaxErrorException: ORA-01795: maximum number of expressions in
> a list is 1000
>
> at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:447)
> at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:396)
> at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:951)
> at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:513)
> at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:227)
> at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:531)
> at oracle.jdbc.driver.T4CStatement.doOall8(T4CStatement.java:195)
> at
> oracle.jdbc.driver.T4CStatement.executeForDescribe(T4CStatement.java:876)
> at
> oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:1175)
> at
> oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1296)
> at
> oracle.jdbc.driver.OracleStatement.executeQuery(OracleStatement.java:1498)
> at
> oracle.jdbc.driver.OracleStatementWrapper.executeQuery(OracleStatementWrapper.java:406)
> at
> com.jolbox.bonecp.StatementHandle.executeQuery(StatementHandle.java:464)
> at
> org.apache.hadoop.hive.metastore.txn.TxnHandler.checkLock(TxnHandler.java:2649)
> at
> org.apache.hadoop.hive.metastore.txn.TxnHandler.checkLockWithRetry(TxnHandler.java:1126)
> at
> org.apache.hadoop.hive.metastore.txn.TxnHandler.lock(TxnHandler.java:895)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.lock(HiveMetaStore.java:6123)
> at sun.reflect.GeneratedMethodAccessor90.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:105)
> at com.sun.proxy.$Proxy11.lock(Unknown Source)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$lock.getResult(ThriftHiveMetastore.java:12012)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$lock.getResult(ThriftHiveMetastore.java:11996)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> at
> org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor$1.run(HadoopThriftAuthBridge.java:551)
> at
> org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor$1.run(HadoopThriftAuthBridge.java:546)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866)
> at
> org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge.java:546)
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:748)
> )
> at
> org.apache.hadoop.hive.metastore.txn.TxnHandler.checkLockWithRetry(TxnHandler.java:1131)
> at
> org.apache.hadoop.hive.metastore.txn.TxnHandler.lock(TxnHandler.java:895)
> at
> org.apache.hadoop.hive.m
[jira] [Work started] (HIVE-20080) TxnHandler checkLock direct sql fail with ORA-01795 , if the table has more than 1000 partitions
[
https://issues.apache.org/jira/browse/HIVE-20080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HIVE-20080 started by Rajkumar Singh.
-
> TxnHandler checkLock direct sql fail with ORA-01795 , if the table has more
> than 1000 partitions
>
>
> Key: HIVE-20080
> URL: https://issues.apache.org/jira/browse/HIVE-20080
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 2.1.0
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
>
> with Oracle as Metastore, txnhandler checkLock fail with
> "checkLockWithRetry(181398,34773) : ORA-01795: maximum number of expressions
> in a list is 1000" if the write table has more than 1000 partitions.
> complete stacktrace
> {code}
> txn.TxnHandler (TxnHandler.java:checkRetryable(2099)) - Non-retryable error
> in checkLockWithRetry(181398,34773) : ORA-01795: maximum number of
> expressions in a list is 1000
> (SQLState=42000, ErrorCode=1795)
> 2018-06-25 15:09:35,999 ERROR [pool-7-thread-197]:
> metastore.RetryingHMSHandler (RetryingHMSHandler.java:invokeInternal(203)) -
> MetaException(message:Unable to update transaction database
> java.sql.SQLSyntaxErrorException: ORA-01795: maximum number of expressions in
> a list is 1000
>
> at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:447)
> at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:396)
> at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:951)
> at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:513)
> at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:227)
> at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:531)
> at oracle.jdbc.driver.T4CStatement.doOall8(T4CStatement.java:195)
> at
> oracle.jdbc.driver.T4CStatement.executeForDescribe(T4CStatement.java:876)
> at
> oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:1175)
> at
> oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1296)
> at
> oracle.jdbc.driver.OracleStatement.executeQuery(OracleStatement.java:1498)
> at
> oracle.jdbc.driver.OracleStatementWrapper.executeQuery(OracleStatementWrapper.java:406)
> at
> com.jolbox.bonecp.StatementHandle.executeQuery(StatementHandle.java:464)
> at
> org.apache.hadoop.hive.metastore.txn.TxnHandler.checkLock(TxnHandler.java:2649)
> at
> org.apache.hadoop.hive.metastore.txn.TxnHandler.checkLockWithRetry(TxnHandler.java:1126)
> at
> org.apache.hadoop.hive.metastore.txn.TxnHandler.lock(TxnHandler.java:895)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.lock(HiveMetaStore.java:6123)
> at sun.reflect.GeneratedMethodAccessor90.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:105)
> at com.sun.proxy.$Proxy11.lock(Unknown Source)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$lock.getResult(ThriftHiveMetastore.java:12012)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$lock.getResult(ThriftHiveMetastore.java:11996)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> at
> org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor$1.run(HadoopThriftAuthBridge.java:551)
> at
> org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor$1.run(HadoopThriftAuthBridge.java:546)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866)
> at
> org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge.java:546)
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:748)
> )
> at
> org.apache.hadoop.hive.metastore.txn.TxnHandler.checkLockWithRetry(TxnHandler.java:1131)
> at
> org.apache.hadoop.hive.metastore.txn.TxnHandler.lock(TxnHandler.java:895)
> at
> org.apache.hadoop.hive.metastore.HiveMe
[jira] [Commented] (HIVE-20091) Tez: Add security credentials for FileSinkOperator output
[
https://issues.apache.org/jira/browse/HIVE-20091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16533955#comment-16533955
]
Hive QA commented on HIVE-20091:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
40s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
55s{color} | {color:blue} ql in master has 2287 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
39s{color} | {color:red} ql: The patch generated 2 new + 40 unchanged - 0 fixed
= 42 total (was 40) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
4s{color} | {color:red} ql generated 1 new + 2287 unchanged - 0 fixed = 2288
total (was 2287) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 59s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | Should
org.apache.hadoop.hive.ql.exec.tez.DagUtils$CollectFileSinkUrisNodeProcessor be
a _static_ inner class? At DagUtils.java:inner class? At DagUtils.java:[lines
192-214] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-12407/dev-support/hive-personality.sh
|
| git revision | master / e865b44 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12407/yetus/diff-checkstyle-ql.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12407/yetus/new-findbugs-ql.html
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12407/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Tez: Add security credentials for FileSinkOperator output
> -
>
> Key: HIVE-20091
> URL: https://issues.apache.org/jira/browse/HIVE-20091
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Attachments: HIVE-20091.01.patch
>
>
> DagUtils needs to add security credentials for the output for the
> FileSinkOperator.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20080) TxnHandler checkLock direct sql fail with ORA-01795 , if the table has more than 1000 partitions
[
https://issues.apache.org/jira/browse/HIVE-20080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16533967#comment-16533967
]
ASF GitHub Bot commented on HIVE-20080:
---
GitHub user rajkrrsingh opened a pull request:
https://github.com/apache/hive/pull/390
HIVE-20080: TxnHandler checkLock direct sql fail with ORA-01795 , if …
check if the dbType is oracle break the query IN clause to include
hl_partition in batch of 1000.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/rajkrrsingh/hive HIVE-20080
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/hive/pull/390.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #390
commit 097c42a8e54ca48cd634dd5c777021eebab076bb
Author: Rajkumar singh
Date: 2018-07-05T17:52:30Z
HIVE-20080: TxnHandler checkLock direct sql fail with ORA-01795 , if the
table has more than 1000 partitions
> TxnHandler checkLock direct sql fail with ORA-01795 , if the table has more
> than 1000 partitions
>
>
> Key: HIVE-20080
> URL: https://issues.apache.org/jira/browse/HIVE-20080
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 2.1.0
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Labels: pull-request-available
>
> with Oracle as Metastore, txnhandler checkLock fail with
> "checkLockWithRetry(181398,34773) : ORA-01795: maximum number of expressions
> in a list is 1000" if the write table has more than 1000 partitions.
> complete stacktrace
> {code}
> txn.TxnHandler (TxnHandler.java:checkRetryable(2099)) - Non-retryable error
> in checkLockWithRetry(181398,34773) : ORA-01795: maximum number of
> expressions in a list is 1000
> (SQLState=42000, ErrorCode=1795)
> 2018-06-25 15:09:35,999 ERROR [pool-7-thread-197]:
> metastore.RetryingHMSHandler (RetryingHMSHandler.java:invokeInternal(203)) -
> MetaException(message:Unable to update transaction database
> java.sql.SQLSyntaxErrorException: ORA-01795: maximum number of expressions in
> a list is 1000
>
> at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:447)
> at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:396)
> at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:951)
> at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:513)
> at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:227)
> at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:531)
> at oracle.jdbc.driver.T4CStatement.doOall8(T4CStatement.java:195)
> at
> oracle.jdbc.driver.T4CStatement.executeForDescribe(T4CStatement.java:876)
> at
> oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:1175)
> at
> oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1296)
> at
> oracle.jdbc.driver.OracleStatement.executeQuery(OracleStatement.java:1498)
> at
> oracle.jdbc.driver.OracleStatementWrapper.executeQuery(OracleStatementWrapper.java:406)
> at
> com.jolbox.bonecp.StatementHandle.executeQuery(StatementHandle.java:464)
> at
> org.apache.hadoop.hive.metastore.txn.TxnHandler.checkLock(TxnHandler.java:2649)
> at
> org.apache.hadoop.hive.metastore.txn.TxnHandler.checkLockWithRetry(TxnHandler.java:1126)
> at
> org.apache.hadoop.hive.metastore.txn.TxnHandler.lock(TxnHandler.java:895)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.lock(HiveMetaStore.java:6123)
> at sun.reflect.GeneratedMethodAccessor90.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:105)
> at com.sun.proxy.$Proxy11.lock(Unknown Source)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$lock.getResult(ThriftHiveMetastore.java:12012)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$lock.getResult(ThriftHiveMetastore.java:11996)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> at
> org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor$1.run(HadoopThriftAuthBridge.java:551)
> at
> org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssum
[jira] [Updated] (HIVE-20080) TxnHandler checkLock direct sql fail with ORA-01795 , if the table has more than 1000 partitions
[
https://issues.apache.org/jira/browse/HIVE-20080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HIVE-20080:
--
Labels: pull-request-available (was: )
> TxnHandler checkLock direct sql fail with ORA-01795 , if the table has more
> than 1000 partitions
>
>
> Key: HIVE-20080
> URL: https://issues.apache.org/jira/browse/HIVE-20080
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 2.1.0
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Labels: pull-request-available
>
> with Oracle as Metastore, txnhandler checkLock fail with
> "checkLockWithRetry(181398,34773) : ORA-01795: maximum number of expressions
> in a list is 1000" if the write table has more than 1000 partitions.
> complete stacktrace
> {code}
> txn.TxnHandler (TxnHandler.java:checkRetryable(2099)) - Non-retryable error
> in checkLockWithRetry(181398,34773) : ORA-01795: maximum number of
> expressions in a list is 1000
> (SQLState=42000, ErrorCode=1795)
> 2018-06-25 15:09:35,999 ERROR [pool-7-thread-197]:
> metastore.RetryingHMSHandler (RetryingHMSHandler.java:invokeInternal(203)) -
> MetaException(message:Unable to update transaction database
> java.sql.SQLSyntaxErrorException: ORA-01795: maximum number of expressions in
> a list is 1000
>
> at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:447)
> at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:396)
> at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:951)
> at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:513)
> at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:227)
> at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:531)
> at oracle.jdbc.driver.T4CStatement.doOall8(T4CStatement.java:195)
> at
> oracle.jdbc.driver.T4CStatement.executeForDescribe(T4CStatement.java:876)
> at
> oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:1175)
> at
> oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1296)
> at
> oracle.jdbc.driver.OracleStatement.executeQuery(OracleStatement.java:1498)
> at
> oracle.jdbc.driver.OracleStatementWrapper.executeQuery(OracleStatementWrapper.java:406)
> at
> com.jolbox.bonecp.StatementHandle.executeQuery(StatementHandle.java:464)
> at
> org.apache.hadoop.hive.metastore.txn.TxnHandler.checkLock(TxnHandler.java:2649)
> at
> org.apache.hadoop.hive.metastore.txn.TxnHandler.checkLockWithRetry(TxnHandler.java:1126)
> at
> org.apache.hadoop.hive.metastore.txn.TxnHandler.lock(TxnHandler.java:895)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.lock(HiveMetaStore.java:6123)
> at sun.reflect.GeneratedMethodAccessor90.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:105)
> at com.sun.proxy.$Proxy11.lock(Unknown Source)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$lock.getResult(ThriftHiveMetastore.java:12012)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$lock.getResult(ThriftHiveMetastore.java:11996)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> at
> org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor$1.run(HadoopThriftAuthBridge.java:551)
> at
> org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor$1.run(HadoopThriftAuthBridge.java:546)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866)
> at
> org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge.java:546)
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:748)
> )
> at
> org.apache.hadoop.hive.metastore.txn.TxnHandler.checkLockWithRetry(TxnHandler.java:1131)
> at
> org.apache.hadoop.hive.metastore.txn.TxnHandler.lock(Txn
[jira] [Commented] (HIVE-20093) LlapOutputFomatService: Use ArrowBuf with Netty for Accounting
[
https://issues.apache.org/jira/browse/HIVE-20093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16533971#comment-16533971
]
Jason Dere commented on HIVE-20093:
---
+1 pending tests
> LlapOutputFomatService: Use ArrowBuf with Netty for Accounting
> --
>
> Key: HIVE-20093
> URL: https://issues.apache.org/jira/browse/HIVE-20093
> Project: Hive
> Issue Type: Bug
>Reporter: Eric Wohlstadter
>Assignee: Eric Wohlstadter
>Priority: Major
> Attachments: HIVE-20093.1.patch
>
>
> Combining {{Unpooled.wrappedBuffer}} with Arrow buffers can create corrupted
> buffers from buffer reuse race-condition.
> This change ensures Arrow memory to be accounted by the same BufferAllocator.
> RootAllocator will return an ArrowBuf which cooperates with Arrow memory
> arrow accounting after Netty {{release(1)}} the buffer.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20080) TxnHandler checkLock direct sql fail with ORA-01795 , if the table has more than 1000 partitions
[
https://issues.apache.org/jira/browse/HIVE-20080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rajkumar Singh updated HIVE-20080:
--
Attachment: HIVE-20080.patch
Status: Patch Available (was: In Progress)
> TxnHandler checkLock direct sql fail with ORA-01795 , if the table has more
> than 1000 partitions
>
>
> Key: HIVE-20080
> URL: https://issues.apache.org/jira/browse/HIVE-20080
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 2.1.0
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-20080.patch
>
>
> with Oracle as Metastore, txnhandler checkLock fail with
> "checkLockWithRetry(181398,34773) : ORA-01795: maximum number of expressions
> in a list is 1000" if the write table has more than 1000 partitions.
> complete stacktrace
> {code}
> txn.TxnHandler (TxnHandler.java:checkRetryable(2099)) - Non-retryable error
> in checkLockWithRetry(181398,34773) : ORA-01795: maximum number of
> expressions in a list is 1000
> (SQLState=42000, ErrorCode=1795)
> 2018-06-25 15:09:35,999 ERROR [pool-7-thread-197]:
> metastore.RetryingHMSHandler (RetryingHMSHandler.java:invokeInternal(203)) -
> MetaException(message:Unable to update transaction database
> java.sql.SQLSyntaxErrorException: ORA-01795: maximum number of expressions in
> a list is 1000
>
> at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:447)
> at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:396)
> at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:951)
> at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:513)
> at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:227)
> at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:531)
> at oracle.jdbc.driver.T4CStatement.doOall8(T4CStatement.java:195)
> at
> oracle.jdbc.driver.T4CStatement.executeForDescribe(T4CStatement.java:876)
> at
> oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:1175)
> at
> oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1296)
> at
> oracle.jdbc.driver.OracleStatement.executeQuery(OracleStatement.java:1498)
> at
> oracle.jdbc.driver.OracleStatementWrapper.executeQuery(OracleStatementWrapper.java:406)
> at
> com.jolbox.bonecp.StatementHandle.executeQuery(StatementHandle.java:464)
> at
> org.apache.hadoop.hive.metastore.txn.TxnHandler.checkLock(TxnHandler.java:2649)
> at
> org.apache.hadoop.hive.metastore.txn.TxnHandler.checkLockWithRetry(TxnHandler.java:1126)
> at
> org.apache.hadoop.hive.metastore.txn.TxnHandler.lock(TxnHandler.java:895)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.lock(HiveMetaStore.java:6123)
> at sun.reflect.GeneratedMethodAccessor90.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:105)
> at com.sun.proxy.$Proxy11.lock(Unknown Source)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$lock.getResult(ThriftHiveMetastore.java:12012)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$lock.getResult(ThriftHiveMetastore.java:11996)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> at
> org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor$1.run(HadoopThriftAuthBridge.java:551)
> at
> org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor$1.run(HadoopThriftAuthBridge.java:546)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866)
> at
> org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge.java:546)
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:748)
> )
> at
> org.apache.hadoop.hive.metastore.txn.TxnHandler.checkLockWithRetry(TxnHand
[jira] [Commented] (HIVE-20076) Delete on a partitioned table removes more rows than expected
[ https://issues.apache.org/jira/browse/HIVE-20076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16533986#comment-16533986 ] Eugene Koifman commented on HIVE-20076: --- [~teddy.choi] the patch didn't apply. Also, how long does it take to run these tests? The q tests include multiple queries producing 30K rows each all of which is recored in the .q.out file. > Delete on a partitioned table removes more rows than expected > - > > Key: HIVE-20076 > URL: https://issues.apache.org/jira/browse/HIVE-20076 > Project: Hive > Issue Type: Bug > Components: Transactions >Reporter: Teddy Choi >Assignee: Teddy Choi >Priority: Major > Attachments: HIVE-20076.2.patch, HIVE-20076.patch > > > Delete on a partitioned table removes more rows than expected -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20085) Druid-Hive (managed) table creation fails with strict managed table checks: Table is marked as a managed table but is not transactional
[
https://issues.apache.org/jira/browse/HIVE-20085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nishant Bangarwa updated HIVE-20085:
Attachment: HIVE-20085.4.patch
> Druid-Hive (managed) table creation fails with strict managed table checks:
> Table is marked as a managed table but is not transactional
> ---
>
> Key: HIVE-20085
> URL: https://issues.apache.org/jira/browse/HIVE-20085
> Project: Hive
> Issue Type: Bug
> Components: Hive, StorageHandler
>Affects Versions: 3.0.0
>Reporter: Dileep Kumar Chiguruvada
>Assignee: Nishant Bangarwa
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-20085.1.patch, HIVE-20085.2.patch,
> HIVE-20085.3.patch, HIVE-20085.4.patch, HIVE-20085.patch
>
>
> Druid-Hive (managed) table creation fails with strict managed table checks:
> Table is marked as a managed table but is not transactional
> {code}
> drop table if exists calcs;
> create table calcs
> STORED BY 'org.apache.hadoop.hive.druid.DruidStorageHandler'
> TBLPROPERTIES (
> "druid.segment.granularity" = "MONTH",
> "druid.query.granularity" = "DAY")
> AS SELECT
> cast(datetime0 as timestamp with local time zone) `__time`,
> key,
> str0, str1, str2, str3,
> date0, date1, date2, date3,
> time0, time1,
> datetime0, datetime1,
> zzz,
> cast(bool0 as string) bool0,
> cast(bool1 as string) bool1,
> cast(bool2 as string) bool2,
> cast(bool3 as string) bool3,
> int0, int1, int2, int3,
> num0, num1, num2, num3, num4
> from tableau_orc.calcs;
> 2018-07-03 04:57:31,911|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : Status: Running
> (Executing on YARN cluster with App id application_1530592209763_0009)
> ...
> ...
> 2018-07-03 04:57:36,334|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : SHUFFLE_BYTES_TO_MEM: > 0
> 2018-07-03 04:57:36,334|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : SHUFFLE_PHASE_TIME:
> 330
> 2018-07-03 04:57:36,334|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : SPILLED_RECORDS: 17
> 2018-07-03 04:57:36,334|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO :
> TaskCounter_Reducer_2_OUTPUT_out_Reducer_2:
> 2018-07-03 04:57:36,335|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : OUTPUT_RECORDS: 0
> 2018-07-03 04:57:36,335|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO :
> org.apache.hadoop.hive.llap.counters.LlapWmCounters:
> 2018-07-03 04:57:36,335|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : GUARANTEED_QUEUED_NS: > 0
> 2018-07-03 04:57:36,335|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO :
> GUARANTEED_RUNNING_NS: 0
> 2018-07-03 04:57:36,335|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO :
> SPECULATIVE_QUEUED_NS: 2162643606
> 2018-07-03 04:57:36,335|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO :
> SPECULATIVE_RUNNING_NS: 12151664909
> 2018-07-03 04:57:36,335|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : Starting task
> [Stage-2:DEPENDENCY_COLLECTION] in serial mode
> 2018-07-03 04:57:36,335|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : Starting task
> [Stage-0:MOVE] in serial mode
> 2018-07-03 04:57:36,336|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : Moving data to
> directory
> hdfs://mycluster/warehouse/tablespace/managed/hive/druid_tableau.db/calcs
> from
> hdfs://mycluster/warehouse/tablespace/managed/hive/druid_tableau.db/.hive-staging_hive_2018-07-03_04-57-27_351_7124633902209008283-3/-ext-10002
> 2018-07-03 04:57:36,336|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : Starting task
> [Stage-4:DDL] in serial mode
> 2018-07-03 04:57:36,336|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|ERROR : FAILED: Execution
> Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask.
> MetaException(message:Table druid_tableau.calcs failed strict managed table
> checks due to the following reason: Table is marked as a managed table but is
> not transactional.)
> 2018-07-03 04:57:36,336|INFO|Thread-721|machine.py:111 -
> tee_pipe()||aa121a45-29eb-48a8-8628-ae5368aa172d|INFO : Completed executing
> command(queryId=hive_20180703045727_c39c40d2-7d4a-46c7-a36d-7925e7c4a788);
[jira] [Commented] (HIVE-20091) Tez: Add security credentials for FileSinkOperator output
[
https://issues.apache.org/jira/browse/HIVE-20091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16534002#comment-16534002
]
Hive QA commented on HIVE-20091:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12930321/HIVE-20091.01.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 14636 tests
executed
*Failed tests:*
{noformat}
TestClearDanglingScratchDir - did not produce a TEST-*.xml file (likely timed
out) (batchId=240)
TestSparkStatistics - did not produce a TEST-*.xml file (likely timed out)
(batchId=240)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druid_timestamptz]
(batchId=191)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_joins]
(batchId=191)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_masking]
(batchId=191)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/12407/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12407/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12407/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12930321 - PreCommit-HIVE-Build
> Tez: Add security credentials for FileSinkOperator output
> -
>
> Key: HIVE-20091
> URL: https://issues.apache.org/jira/browse/HIVE-20091
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Attachments: HIVE-20091.01.patch
>
>
> DagUtils needs to add security credentials for the output for the
> FileSinkOperator.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20091) Tez: Add security credentials for FileSinkOperator output
[ https://issues.apache.org/jira/browse/HIVE-20091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-20091: Status: In Progress (was: Patch Available) > Tez: Add security credentials for FileSinkOperator output > - > > Key: HIVE-20091 > URL: https://issues.apache.org/jira/browse/HIVE-20091 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-20091.01.patch > > > DagUtils needs to add security credentials for the output for the > FileSinkOperator. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20091) Tez: Add security credentials for FileSinkOperator output
[ https://issues.apache.org/jira/browse/HIVE-20091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-20091: Status: Patch Available (was: In Progress) > Tez: Add security credentials for FileSinkOperator output > - > > Key: HIVE-20091 > URL: https://issues.apache.org/jira/browse/HIVE-20091 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-20091.01.patch, HIVE-20091.02.patch > > > DagUtils needs to add security credentials for the output for the > FileSinkOperator. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20091) Tez: Add security credentials for FileSinkOperator output
[ https://issues.apache.org/jira/browse/HIVE-20091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16534005#comment-16534005 ] Matt McCline commented on HIVE-20091: - Again. > Tez: Add security credentials for FileSinkOperator output > - > > Key: HIVE-20091 > URL: https://issues.apache.org/jira/browse/HIVE-20091 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-20091.01.patch, HIVE-20091.02.patch > > > DagUtils needs to add security credentials for the output for the > FileSinkOperator. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20091) Tez: Add security credentials for FileSinkOperator output
[ https://issues.apache.org/jira/browse/HIVE-20091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-20091: Attachment: HIVE-20091.02.patch > Tez: Add security credentials for FileSinkOperator output > - > > Key: HIVE-20091 > URL: https://issues.apache.org/jira/browse/HIVE-20091 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-20091.01.patch, HIVE-20091.02.patch > > > DagUtils needs to add security credentials for the output for the > FileSinkOperator. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20076) Delete on a partitioned table removes more rows than expected
[
https://issues.apache.org/jira/browse/HIVE-20076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16534007#comment-16534007
]
Gopal V commented on HIVE-20076:
The order by in the queries make it very hard to tell if the bug is happening
or not, because this is about sequential numbering.
{code}
+{"writeid":### Masked writeid ###,"bucketid":536870912,"rowid":513}jessica
garcia 59 3.5220110926
+{"writeid":### Masked writeid ###,"bucketid":536870912,"rowid":6633} jessica
garcia 59 3.6920110925
{code}
I think the fix to getRowNumber() is necessary, which is
{code}
if (rowInBatch >= batch.size) {
+ baseRow = super.getRowNumber();
+ rowInBatch = 0;
return super.nextBatch(theirBatch);
}
{code}
This has a side-effect of reading batch.size again the next time around (if
batch.size !=0, then the first batch will be repeated between every fast-path
batch).
Ideally, at that point it should reset the batch, if the batch.size is > 0 (the
invariant is that it has already been consumed by rowInBatch).
> Delete on a partitioned table removes more rows than expected
> -
>
> Key: HIVE-20076
> URL: https://issues.apache.org/jira/browse/HIVE-20076
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Reporter: Teddy Choi
>Assignee: Teddy Choi
>Priority: Major
> Attachments: HIVE-20076.2.patch, HIVE-20076.patch
>
>
> Delete on a partitioned table removes more rows than expected
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20001) With doas set to true, running select query as hrt_qa user on external table fails due to permission denied to read /warehouse/tablespace/managed directory.
[
https://issues.apache.org/jira/browse/HIVE-20001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16534015#comment-16534015
]
ASF GitHub Bot commented on HIVE-20001:
---
Github user beltran closed the pull request at:
https://github.com/apache/hive/pull/380
> With doas set to true, running select query as hrt_qa user on external table
> fails due to permission denied to read /warehouse/tablespace/managed
> directory.
>
>
> Key: HIVE-20001
> URL: https://issues.apache.org/jira/browse/HIVE-20001
> Project: Hive
> Issue Type: Bug
>Reporter: Jaume M
>Assignee: Jaume M
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-20001.1.patch, HIVE-20001.1.patch,
> HIVE-20001.2.patch, HIVE-20001.3.patch, HIVE-20001.4.patch
>
>
> Hive: With doas set to true, running select query as hrt_qa user on external
> table fails due to permission denied to read /warehouse/tablespace/managed
> directory.
> Steps:
> 1. Create a external table.
> 2. Set doas to true.
> 3. run select count(*) using user hrt_qa.
> Table creation query.
> {code}
> beeline -n hrt_qa -p pwd -u
> "jdbc:hive2://ctr-e138-1518143905142-375925-01-06.hwx.site:2181,ctr-e138-1518143905142-375925-01-05.hwx.site:2181,ctr-e138-1518143905142-375925-01-07.hwx.site:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2;principal=hive/_h...@example.com;transportMode=http;httpPath=cliservice;ssl=true;sslTrustStore=/etc/security/serverKeys/hivetruststore.jks;trustStorePassword=changeit"
> --outputformat=tsv -e "drop table if exists test_table purge;
> create external table test_table(id int, age int) row format delimited fields
> terminated by '|' stored as textfile;
> load data inpath '/tmp/table1.dat' overwrite into table test_table;
> {code}
> select count(*) query execution fails
> {code}
> beeline -n hrt_qa -p pwd -u
> "jdbc:hive2://ctr-e138-1518143905142-375925-01-06.hwx.site:2181,ctr-e138-1518143905142-375925-01-05.hwx.site:2181,ctr-e138-1518143905142-375925-01-07.hwx.site:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2;principal=hive/_h...@example.com;transportMode=http;httpPath=cliservice;ssl=true;sslTrustStore=/etc/security/serverKeys/hivetruststore.jks;trustStorePassword=changeit"
> --outputformat=tsv -e "select count(*) from test_table where age>30 and
> id<10100;"
> 2018-06-22 10:22:29,328|INFO|Thread-126|machine.py:111 -
> tee_pipe()||b3a493ec-99be-483e-91fe-4b701ec27ebc|SLF4J: Class path contains
> multiple SLF4J bindings.
> 2018-06-22 10:22:29,330|INFO|Thread-126|machine.py:111 -
> tee_pipe()||b3a493ec-99be-483e-91fe-4b701ec27ebc|SLF4J: See
> http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
> 2018-06-22 10:22:29,335|INFO|Thread-126|machine.py:111 -
> tee_pipe()||b3a493ec-99be-483e-91fe-4b701ec27ebc|SLF4J: Actual binding is of
> type [org.apache.logging.slf4j.Log4jLoggerFactory]
> 2018-06-22 10:22:31,408|INFO|Thread-126|machine.py:111 -
> tee_pipe()||b3a493ec-99be-483e-91fe-4b701ec27ebc|Format tsv is deprecated,
> please use tsv2
> 2018-06-22 10:22:31,529|INFO|Thread-126|machine.py:111 -
> tee_pipe()||b3a493ec-99be-483e-91fe-4b701ec27ebc|Connecting to
> jdbc:hive2://ctr-e138-1518143905142-375925-01-06.hwx.site:2181,ctr-e138-1518143905142-375925-01-05.hwx.site:2181,ctr-e138-1518143905142-375925-01-07.hwx.site:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2;principal=hive/_h...@example.com;transportMode=http;httpPath=cliservice;ssl=true;sslTrustStore=/etc/security/serverKeys/hivetruststore.jks;trustStorePassword=changeit
> 2018-06-22 10:22:32,031|INFO|Thread-126|machine.py:111 -
> tee_pipe()||b3a493ec-99be-483e-91fe-4b701ec27ebc|18/06/22 10:22:32 [main]:
> INFO jdbc.HiveConnection: Connected to
> ctr-e138-1518143905142-375925-01-04.hwx.site:10001
> 2018-06-22 10:22:34,130|INFO|Thread-126|machine.py:111 -
> tee_pipe()||b3a493ec-99be-483e-91fe-4b701ec27ebc|18/06/22 10:22:34 [main]:
> WARN jdbc.HiveConnection: Failed to connect to
> ctr-e138-1518143905142-375925-01-04.hwx.site:10001
> 2018-06-22 10:22:34,244|INFO|Thread-126|machine.py:111 -
> tee_pipe()||b3a493ec-99be-483e-91fe-4b701ec27ebc|18/06/22 10:22:34 [main]:
> WARN jdbc.HiveConnection: Could not open client transport with JDBC Uri:
> jdbc:hive2://ctr-e138-1518143905142-375925-01-04.hwx.site:10001/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2;principal=hive/_h...@example.com;transportMode=http;httpPath=cliservice;ssl=true;sslTrustStore=/etc/security/serverKeys/hivetruststore.jks;trustStorePassword=changeit:
> Failed to open new session:
> org.apache.hadoop.hive.ql.
[jira] [Commented] (HIVE-20001) With doas set to true, running select query as hrt_qa user on external table fails due to permission denied to read /warehouse/tablespace/managed directory.
[
https://issues.apache.org/jira/browse/HIVE-20001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16534014#comment-16534014
]
ASF GitHub Bot commented on HIVE-20001:
---
Github user beltran closed the pull request at:
https://github.com/apache/hive/pull/389
> With doas set to true, running select query as hrt_qa user on external table
> fails due to permission denied to read /warehouse/tablespace/managed
> directory.
>
>
> Key: HIVE-20001
> URL: https://issues.apache.org/jira/browse/HIVE-20001
> Project: Hive
> Issue Type: Bug
>Reporter: Jaume M
>Assignee: Jaume M
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-20001.1.patch, HIVE-20001.1.patch,
> HIVE-20001.2.patch, HIVE-20001.3.patch, HIVE-20001.4.patch
>
>
> Hive: With doas set to true, running select query as hrt_qa user on external
> table fails due to permission denied to read /warehouse/tablespace/managed
> directory.
> Steps:
> 1. Create a external table.
> 2. Set doas to true.
> 3. run select count(*) using user hrt_qa.
> Table creation query.
> {code}
> beeline -n hrt_qa -p pwd -u
> "jdbc:hive2://ctr-e138-1518143905142-375925-01-06.hwx.site:2181,ctr-e138-1518143905142-375925-01-05.hwx.site:2181,ctr-e138-1518143905142-375925-01-07.hwx.site:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2;principal=hive/_h...@example.com;transportMode=http;httpPath=cliservice;ssl=true;sslTrustStore=/etc/security/serverKeys/hivetruststore.jks;trustStorePassword=changeit"
> --outputformat=tsv -e "drop table if exists test_table purge;
> create external table test_table(id int, age int) row format delimited fields
> terminated by '|' stored as textfile;
> load data inpath '/tmp/table1.dat' overwrite into table test_table;
> {code}
> select count(*) query execution fails
> {code}
> beeline -n hrt_qa -p pwd -u
> "jdbc:hive2://ctr-e138-1518143905142-375925-01-06.hwx.site:2181,ctr-e138-1518143905142-375925-01-05.hwx.site:2181,ctr-e138-1518143905142-375925-01-07.hwx.site:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2;principal=hive/_h...@example.com;transportMode=http;httpPath=cliservice;ssl=true;sslTrustStore=/etc/security/serverKeys/hivetruststore.jks;trustStorePassword=changeit"
> --outputformat=tsv -e "select count(*) from test_table where age>30 and
> id<10100;"
> 2018-06-22 10:22:29,328|INFO|Thread-126|machine.py:111 -
> tee_pipe()||b3a493ec-99be-483e-91fe-4b701ec27ebc|SLF4J: Class path contains
> multiple SLF4J bindings.
> 2018-06-22 10:22:29,330|INFO|Thread-126|machine.py:111 -
> tee_pipe()||b3a493ec-99be-483e-91fe-4b701ec27ebc|SLF4J: See
> http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
> 2018-06-22 10:22:29,335|INFO|Thread-126|machine.py:111 -
> tee_pipe()||b3a493ec-99be-483e-91fe-4b701ec27ebc|SLF4J: Actual binding is of
> type [org.apache.logging.slf4j.Log4jLoggerFactory]
> 2018-06-22 10:22:31,408|INFO|Thread-126|machine.py:111 -
> tee_pipe()||b3a493ec-99be-483e-91fe-4b701ec27ebc|Format tsv is deprecated,
> please use tsv2
> 2018-06-22 10:22:31,529|INFO|Thread-126|machine.py:111 -
> tee_pipe()||b3a493ec-99be-483e-91fe-4b701ec27ebc|Connecting to
> jdbc:hive2://ctr-e138-1518143905142-375925-01-06.hwx.site:2181,ctr-e138-1518143905142-375925-01-05.hwx.site:2181,ctr-e138-1518143905142-375925-01-07.hwx.site:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2;principal=hive/_h...@example.com;transportMode=http;httpPath=cliservice;ssl=true;sslTrustStore=/etc/security/serverKeys/hivetruststore.jks;trustStorePassword=changeit
> 2018-06-22 10:22:32,031|INFO|Thread-126|machine.py:111 -
> tee_pipe()||b3a493ec-99be-483e-91fe-4b701ec27ebc|18/06/22 10:22:32 [main]:
> INFO jdbc.HiveConnection: Connected to
> ctr-e138-1518143905142-375925-01-04.hwx.site:10001
> 2018-06-22 10:22:34,130|INFO|Thread-126|machine.py:111 -
> tee_pipe()||b3a493ec-99be-483e-91fe-4b701ec27ebc|18/06/22 10:22:34 [main]:
> WARN jdbc.HiveConnection: Failed to connect to
> ctr-e138-1518143905142-375925-01-04.hwx.site:10001
> 2018-06-22 10:22:34,244|INFO|Thread-126|machine.py:111 -
> tee_pipe()||b3a493ec-99be-483e-91fe-4b701ec27ebc|18/06/22 10:22:34 [main]:
> WARN jdbc.HiveConnection: Could not open client transport with JDBC Uri:
> jdbc:hive2://ctr-e138-1518143905142-375925-01-04.hwx.site:10001/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2;principal=hive/_h...@example.com;transportMode=http;httpPath=cliservice;ssl=true;sslTrustStore=/etc/security/serverKeys/hivetruststore.jks;trustStorePassword=changeit:
> Failed to open new session:
> org.apache.hadoop.hive.ql.
[jira] [Commented] (HIVE-17751) Separate HMS Client and HMS server into separate sub-modules
[ https://issues.apache.org/jira/browse/HIVE-17751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16534016#comment-16534016 ] Alexander Kolbasov commented on HIVE-17751: --- [~alangates] I think your suggestion to separate this in stages makes sense - at least it will break down things at least a bit. I'll open separate sub-issues for that. > Separate HMS Client and HMS server into separate sub-modules > > > Key: HIVE-17751 > URL: https://issues.apache.org/jira/browse/HIVE-17751 > Project: Hive > Issue Type: Sub-task > Components: Standalone Metastore >Reporter: Vihang Karajgaonkar >Assignee: Alexander Kolbasov >Priority: Major > Attachments: HIVE-17751.01.patch, HIVE-17751.02.patch, > HIVE-17751.03.patch, HIVE-17751.04.patch, > HIVE-17751.06-standalone-metastore.patch, HIVE-17751.07.patch, > HIVE-17751.08.patch, HIVE-17751.09.patch, HIVE-17751.10.patch, > HIVE-17751.11.patch > > > external applications which are interfacing with HMS should ideally only > include HMSClient library instead of one big library containing server as > well. We should ideally have a thin client library so that cross version > support for external applications is easier. We should sub-divide the > standalone module into possibly 3 modules (one for common classes, one for > client classes and one for server) or 2 sub-modules (one for client and one > for server) so that we can generate separate jars for HMS client and server. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-17751) Separate HMS Client and HMS server into separate sub-modules
[ https://issues.apache.org/jira/browse/HIVE-17751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Kolbasov updated HIVE-17751: -- Issue Type: Improvement (was: Sub-task) Parent: (was: HIVE-17159) > Separate HMS Client and HMS server into separate sub-modules > > > Key: HIVE-17751 > URL: https://issues.apache.org/jira/browse/HIVE-17751 > Project: Hive > Issue Type: Improvement > Components: Standalone Metastore >Reporter: Vihang Karajgaonkar >Assignee: Alexander Kolbasov >Priority: Major > Attachments: HIVE-17751.01.patch, HIVE-17751.02.patch, > HIVE-17751.03.patch, HIVE-17751.04.patch, > HIVE-17751.06-standalone-metastore.patch, HIVE-17751.07.patch, > HIVE-17751.08.patch, HIVE-17751.09.patch, HIVE-17751.10.patch, > HIVE-17751.11.patch > > > external applications which are interfacing with HMS should ideally only > include HMSClient library instead of one big library containing server as > well. We should ideally have a thin client library so that cross version > support for external applications is easier. We should sub-divide the > standalone module into possibly 3 modules (one for common classes, one for > client classes and one for server) or 2 sub-modules (one for client and one > for server) so that we can generate separate jars for HMS client and server. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19733) RemoteSparkJobStatus#getSparkStageProgress inefficient implementation
[
https://issues.apache.org/jira/browse/HIVE-19733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16534020#comment-16534020
]
Hive QA commented on HIVE-19733:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
49s{color} | {color:blue} ql in master has 2287 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} ql: The patch generated 0 new + 2 unchanged - 1
fixed = 2 total (was 3) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 28s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-12408/dev-support/hive-personality.sh
|
| git revision | master / e865b44 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12408/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> RemoteSparkJobStatus#getSparkStageProgress inefficient implementation
> -
>
> Key: HIVE-19733
> URL: https://issues.apache.org/jira/browse/HIVE-19733
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
>Reporter: Sahil Takiar
>Assignee: Bharathkrishna Guruvayoor Murali
>Priority: Major
> Attachments: HIVE-19733.1.patch, HIVE-19733.2.patch
>
>
> The implementation of {{RemoteSparkJobStatus#getSparkStageProgress}} is a bit
> inefficient. There is one RPC call to get the {{SparkJobInfo}} and then for
> every stage there is another RPC call to get each {{SparkStageInfo}}. This
> could all be done in a single RPC call.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-20097) Convert standalone-metastore to a submodule
[ https://issues.apache.org/jira/browse/HIVE-20097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Kolbasov reassigned HIVE-20097: - > Convert standalone-metastore to a submodule > --- > > Key: HIVE-20097 > URL: https://issues.apache.org/jira/browse/HIVE-20097 > Project: Hive > Issue Type: Sub-task > Components: Hive, Metastore, Standalone Metastore >Affects Versions: 3.1.0, 4.0.0 >Reporter: Alexander Kolbasov >Assignee: Alexander Kolbasov >Priority: Major > > This is a subtask to stage HIVE-17751 changes into several smaller phases. > The first part is moving existing code in hive-standalone-metastore to a > sub-module. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19806) Several tests do not properly sort their output
[ https://issues.apache.org/jira/browse/HIVE-19806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated HIVE-19806: -- Attachment: HIVE-19806.4.patch > Several tests do not properly sort their output > --- > > Key: HIVE-19806 > URL: https://issues.apache.org/jira/browse/HIVE-19806 > Project: Hive > Issue Type: Bug > Components: Test >Affects Versions: 3.0.0 >Reporter: Alan Gates >Assignee: Alan Gates >Priority: Major > Attachments: HIVE-19806.2.patch, HIVE-19806.3.patch, > HIVE-19806.4.patch, HIVE-19806.patch > > > A number of the tests produce unsorted output that happens to come out the > same on people's laptops and the ptest infrastructure. But when run on a > separate linux box the sort differences show up. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20097) Convert standalone-metastore to a submodule
[
https://issues.apache.org/jira/browse/HIVE-20097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Kolbasov updated HIVE-20097:
--
Status: Patch Available (was: Open)
Patch 01 is based on
{code}
* commit e865b449efca04b02922bbc42693a1e8f91825a9 (origin/master, origin/HEAD,
master, HIVE-17751)
| Author: Ganesha Shreedhara
| Date: Wed Jul 4 23:03:10 2018 -0700
|
| HIVE-19850 : Dynamic partition pruning in Tez is leading to 'No work
found for tablescan' error (Ganesha Shreedhara via Ashutosh Chauhan)
{code}
> Convert standalone-metastore to a submodule
> ---
>
> Key: HIVE-20097
> URL: https://issues.apache.org/jira/browse/HIVE-20097
> Project: Hive
> Issue Type: Sub-task
> Components: Hive, Metastore, Standalone Metastore
>Affects Versions: 3.1.0, 4.0.0
>Reporter: Alexander Kolbasov
>Assignee: Alexander Kolbasov
>Priority: Major
> Attachments: HIVE-20097.01.patch
>
>
> This is a subtask to stage HIVE-17751 changes into several smaller phases.
> The first part is moving existing code in hive-standalone-metastore to a
> sub-module.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20097) Convert standalone-metastore to a submodule
[ https://issues.apache.org/jira/browse/HIVE-20097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Kolbasov updated HIVE-20097: -- Attachment: HIVE-20097.01.patch > Convert standalone-metastore to a submodule > --- > > Key: HIVE-20097 > URL: https://issues.apache.org/jira/browse/HIVE-20097 > Project: Hive > Issue Type: Sub-task > Components: Hive, Metastore, Standalone Metastore >Affects Versions: 3.1.0, 4.0.0 >Reporter: Alexander Kolbasov >Assignee: Alexander Kolbasov >Priority: Major > Attachments: HIVE-20097.01.patch > > > This is a subtask to stage HIVE-17751 changes into several smaller phases. > The first part is moving existing code in hive-standalone-metastore to a > sub-module. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20097) Convert standalone-metastore to a submodule
[ https://issues.apache.org/jira/browse/HIVE-20097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16534029#comment-16534029 ] Alexander Kolbasov commented on HIVE-20097: --- This change doesn't involve any code changes. The top-level pom file for standalone is converted to a pom packaging and it now only defines versions of various dependencies. All the rest is moved into metastore-common directory. Other pom files are updated to reference new hive-standalone-metastore-common jars. > Convert standalone-metastore to a submodule > --- > > Key: HIVE-20097 > URL: https://issues.apache.org/jira/browse/HIVE-20097 > Project: Hive > Issue Type: Sub-task > Components: Hive, Metastore, Standalone Metastore >Affects Versions: 3.1.0, 4.0.0 >Reporter: Alexander Kolbasov >Assignee: Alexander Kolbasov >Priority: Major > Attachments: HIVE-20097.01.patch > > > This is a subtask to stage HIVE-17751 changes into several smaller phases. > The first part is moving existing code in hive-standalone-metastore to a > sub-module. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
