[jira] [Created] (HIVE-12327) WebHCat e2e tests TestJob_1 and TestJob_2 fail
Daniel Dai created HIVE-12327: - Summary: WebHCat e2e tests TestJob_1 and TestJob_2 fail Key: HIVE-12327 URL: https://issues.apache.org/jira/browse/HIVE-12327 Project: Hive Issue Type: Bug Components: WebHCat Reporter: Daniel Dai Assignee: Daniel Dai Fix For: 1.3.0, 2.0.0 The tests are added in HIVE-7035. Both are negative tests and check if the http status code is 400. The original patch capture the exception containing specific message. However, in latter version of Hadoop, the message change so the exception is not contained. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-12583) HS2 ShutdownHookManager holds extra of Driver instance
Daniel Dai created HIVE-12583: - Summary: HS2 ShutdownHookManager holds extra of Driver instance Key: HIVE-12583 URL: https://issues.apache.org/jira/browse/HIVE-12583 Project: Hive Issue Type: Bug Components: Locking Affects Versions: 1.3.0 Reporter: Daniel Dai Assignee: Daniel Dai HIVE-12266 add a shutdown hook for every Driver instance to release the lock th session holds in case Driver does not exist elegantly. However, that holds all Driver instances and HS2 may run out of memory. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-13429) Tool to remove dangling scratch dir
Daniel Dai created HIVE-13429: - Summary: Tool to remove dangling scratch dir Key: HIVE-13429 URL: https://issues.apache.org/jira/browse/HIVE-13429 Project: Hive Issue Type: Improvement Reporter: Daniel Dai Assignee: Daniel Dai We have seen in some cases, user will leave the scratch dir behind, and eventually eat out hdfs storage. This could happen when vm restarts and leave no chance for Hive to run shutdown hook. This is applicable for both HiveCli and HiveServer2. Here we provide an external tool to clear dead scratch dir as needed. We need a way to identify which scratch dir is in use. We will rely on HDFS write lock for that. Here is how HDFS write lock works: 1. A HDFS client open HDFS file for write and only close at the time of shutdown 2. Cleanup process can try to open HDFS file for write. If the client holding this file is still running, we will get exception. Otherwise, we know the client is dead 3. If the HDFS client dies without closing the HDFS file, NN will reclaim the lease after 10 min, ie, the HDFS file hold by the dead client is writable again after 10 min So here is how we remove dangling scratch directory in Hive: 1. HiveCli/HiveServer2 opens a well-named lock file in scratch directory and only close it when we about to drop scratch directory 2. A command line tool cleardanglingscratchdir will check every scratch directory and try open the lock file for write. If it does not get exception, meaning the owner is dead and we can safely remove the scratch directory 3. The 10 min window means it is possible a HiveCli/HiveServer2 is dead but we still cannot reclaim the scratch directory for another 10 min. But this should be tolerable -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-13476) HS2 ShutdownHookManager holds extra of Driver instance in nested compile
Daniel Dai created HIVE-13476:
-
Summary: HS2 ShutdownHookManager holds extra of Driver instance in
nested compile
Key: HIVE-13476
URL: https://issues.apache.org/jira/browse/HIVE-13476
Project: Hive
Issue Type: Bug
Reporter: Daniel Dai
Assignee: Daniel Dai
For some SQL statement, Hive will do nested compile. In this case, Hive will
create a Driver instance to do the nested compile, but not calling destroy.
That left Driver instance in the shudownhook:
{code}
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:402)
at
org.apache.hadoop.hive.ql.optimizer.IndexUtils.createRootTask(IndexUtils.java:223)
at
org.apache.hadoop.hive.ql.index.compact.CompactIndexHandler.getIndexBuilderMapRedTask(CompactIndexHandler.java:151)
at
org.apache.hadoop.hive.ql.index.TableBasedIndexHandler.getIndexBuilderMapRedTask(TableBasedIndexHandler.java:108)
at
org.apache.hadoop.hive.ql.index.TableBasedIndexHandler.generateIndexBuildTaskList(TableBasedIndexHandler.java:92)
at
org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.getIndexBuilderMapRed(DDLSemanticAnalyzer.java:1228)
at
org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeAlterIndexRebuild(DDLSemanticAnalyzer.java:1175)
at
org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeInternal(DDLSemanticAnalyzer.java:408)
at
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:227)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:464)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:318)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1194)
at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1188)
at
org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:110)
at
org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:181)
at
org.apache.hive.service.cli.operation.Operation.run(Operation.java:257)
at
org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:419)
at
org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:406)
at sun.reflect.GeneratedMethodAccessor26.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at
org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:78)
at
org.apache.hive.service.cli.session.HiveSessionProxy.access$000(HiveSessionProxy.java:36)
at
org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:63)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1709)
at
org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:59)
at com.sun.proxy.$Proxy20.executeStatementAsync(Unknown Source)
at
org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:276)
at
org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:486)
at
org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1317)
at
org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1302)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at
org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56)
at
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:285)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HIVE-13498) cleardanglingscratchdir does not work if scratchdir is not on defaultFs
Daniel Dai created HIVE-13498: - Summary: cleardanglingscratchdir does not work if scratchdir is not on defaultFs Key: HIVE-13498 URL: https://issues.apache.org/jira/browse/HIVE-13498 Project: Hive Issue Type: Bug Reporter: Daniel Dai Assignee: Daniel Dai Fix For: 1.3.0, 2.1.0 The cleardanglingscratchdir utility need a fix to make it work if scratchdir is not on defaultFs, such as on Azure. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-13513) cleardanglingscratchdir does not work in some version of HDFS
Daniel Dai created HIVE-13513:
-
Summary: cleardanglingscratchdir does not work in some version of
HDFS
Key: HIVE-13513
URL: https://issues.apache.org/jira/browse/HIVE-13513
Project: Hive
Issue Type: Bug
Reporter: Daniel Dai
Assignee: Daniel Dai
Fix For: 1.3.0, 2.1.0
On some Hadoop version, we keep getting "lease recovery" message at the time we
check for scratchdir by opening for appending:
{code}
Failed to APPEND_FILE xxx for DFSClient_NONMAPREDUCE_785768631_1 on 10.0.0.18
because lease recovery is in progress. Try again later.
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.recoverLeaseInternal(FSNamesystem.java:2917)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.appendFileInternal(FSNamesystem.java:2677)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.appendFileInt(FSNamesystem.java:2984)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.appendFile(FSNamesystem.java:2953)
at
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.append(NameNodeRpcServer.java:655)
at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.append(ClientNamenodeProtocolServerSideTranslatorPB.java:421)
at
org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2137)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2133)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2131)
{code}
and
{code}
16/04/14 04:51:56 ERROR hdfs.DFSClient: Failed to close inode 18963
java.io.IOException: Failed to replace a bad datanode on the existing pipeline
due to no more good datanodes being available to try. (Nodes:
current=[DatanodeInfoWithStorage[10.0.0.12:30010,DS-b355ac2a-a23a-418a-af9b-4c1b4e26afe8,DISK]],
original=[DatanodeInfoWithStorage[10.0.0.12:30010,DS-b355ac2a-a23a-418a-af9b-4c1b4e26afe8,DISK]]).
The current failed datanode replacement policy is DEFAULT, and a client may
configure this via 'dfs.client.block.write.replace-datanode-on-failure.policy'
in its configuration.
at
org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.findNewDatanode(DFSOutputStream.java:951)
at
org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.addDatanode2ExistingPipeline(DFSOutputStream.java:1017)
at
org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.setupPipelineForAppendOrRecovery(DFSOutputStream.java:1165)
at
org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:470)
{code}
The reason is not clear. However, if we remove hsync from SessionState,
everything works as expected. Attach patch to remove hsync call for now.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HIVE-13514) TestClearDanglingScratchDir fail on branch-1
Daniel Dai created HIVE-13514: - Summary: TestClearDanglingScratchDir fail on branch-1 Key: HIVE-13514 URL: https://issues.apache.org/jira/browse/HIVE-13514 Project: Hive Issue Type: Bug Reporter: Daniel Dai Assignee: Daniel Dai TestClearDanglingScratchDir fail on branch-1 due to branch-1 is using log4j. Attach a patch to let test pass on both master and branch-1. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-13551) Make cleardanglingscratchdir work on Windows
Daniel Dai created HIVE-13551: - Summary: Make cleardanglingscratchdir work on Windows Key: HIVE-13551 URL: https://issues.apache.org/jira/browse/HIVE-13551 Project: Hive Issue Type: Bug Reporter: Daniel Dai Assignee: Daniel Dai Attachments: HIVE-13551.1.patch See a couple of issues when running cleardanglingscratchdir on Windows, includes: 1. dfs.support.append is set to false in Azure cluster, need an alternative way when append is disabled 2. fix for cmd scripts 3. fix UT on Windows -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-13560) Adding Omid as connection manager for HBase Metastore
Daniel Dai created HIVE-13560: - Summary: Adding Omid as connection manager for HBase Metastore Key: HIVE-13560 URL: https://issues.apache.org/jira/browse/HIVE-13560 Project: Hive Issue Type: Improvement Components: HBase Metastore Reporter: Daniel Dai Assignee: Daniel Dai Adding Omid as a transaction manager to HBase Metastore. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-13631) Support index in HBase Metastore
Daniel Dai created HIVE-13631: - Summary: Support index in HBase Metastore Key: HIVE-13631 URL: https://issues.apache.org/jira/browse/HIVE-13631 Project: Hive Issue Type: Improvement Components: HBase Metastore Reporter: Daniel Dai Assignee: Daniel Dai Currently all index related methods in HBaseStore is not implemented. We need to add those missing methods and index support in hbaseimport tool. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-13729) FileSystem$Cache leaks in FileUtils.checkFileAccessWithImpersonation
Daniel Dai created HIVE-13729: - Summary: FileSystem$Cache leaks in FileUtils.checkFileAccessWithImpersonation Key: HIVE-13729 URL: https://issues.apache.org/jira/browse/HIVE-13729 Project: Hive Issue Type: Bug Components: HiveServer2 Reporter: Daniel Dai Assignee: Daniel Dai Didn't invoke FileSystem.closeAllForUGI after checkFileAccess. This results leak in FileSystem$Cache and eventually OOM for HS2. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-13981) Operation.toSQLException eats full exception stack
Daniel Dai created HIVE-13981:
-
Summary: Operation.toSQLException eats full exception stack
Key: HIVE-13981
URL: https://issues.apache.org/jira/browse/HIVE-13981
Project: Hive
Issue Type: Bug
Reporter: Daniel Dai
Assignee: Daniel Dai
Operation.toSQLException eats half of the exception stack and make debug hard.
For example, we saw an exception:
{code}
org.apache.hive.service.cli.HiveSQL Exception : Error while compiling
statement: FAILED : NullPointer Exception null
at org.apache.hive.service.cli.operation.Operation.toSQL Exception
(Operation.java:336)
at
org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:113)
at
org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:182)
at org.apache.hive.service.cli.operation.Operation.run(Operation.java:278)
at
org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:421)
at
org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:408)
at
org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:276)
at
org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:505)
at
org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1317)
at
org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1302)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at
org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge.java:562)
at
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang. NullPointer Exception
{code}
The real stack causing the NPE is lost.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HIVE-14097) Fix TestCliDriver for hbase metastore
Daniel Dai created HIVE-14097: - Summary: Fix TestCliDriver for hbase metastore Key: HIVE-14097 URL: https://issues.apache.org/jira/browse/HIVE-14097 Project: Hive Issue Type: Bug Components: HBase Metastore Reporter: Daniel Dai Assignee: Daniel Dai More than half of TestCliDriver fail with hbasemetastore, we need to fix them. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-14101) Adding type/event notification/version/constraints to hbase metastore
Daniel Dai created HIVE-14101: - Summary: Adding type/event notification/version/constraints to hbase metastore Key: HIVE-14101 URL: https://issues.apache.org/jira/browse/HIVE-14101 Project: Hive Issue Type: Improvement Components: HBase Metastore Reporter: Daniel Dai Assignee: Daniel Dai type/event notification/version/constraints are missing in hbase metastore, we need to add the missing piece. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-14103) DateColumnStatsAggregator is missing in hbase metastore
Daniel Dai created HIVE-14103: - Summary: DateColumnStatsAggregator is missing in hbase metastore Key: HIVE-14103 URL: https://issues.apache.org/jira/browse/HIVE-14103 Project: Hive Issue Type: Improvement Reporter: Daniel Dai Assignee: Daniel Dai Currently throws exception if getting aggregate stats of date column. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-14104) addPartitions with PartitionSpecProxy is not implemented in hbase metastore
Daniel Dai created HIVE-14104: - Summary: addPartitions with PartitionSpecProxy is not implemented in hbase metastore Key: HIVE-14104 URL: https://issues.apache.org/jira/browse/HIVE-14104 Project: Hive Issue Type: Improvement Reporter: Daniel Dai Assignee: Daniel Dai This seems only be used by hcat. Need to implement properly in hbase metastore. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-14105) listTableNamesByFilter/listPartitionNamesByFilter is missing in hbase metastore
Daniel Dai created HIVE-14105: - Summary: listTableNamesByFilter/listPartitionNamesByFilter is missing in hbase metastore Key: HIVE-14105 URL: https://issues.apache.org/jira/browse/HIVE-14105 Project: Hive Issue Type: Improvement Components: HBase Metastore Reporter: Daniel Dai Assignee: Daniel Dai Both take a JDO filter which is not relevant in hbase metastore. Need to revisit and find a solution for both. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-14106) Retry when hbase transaction conflict happen
Daniel Dai created HIVE-14106: - Summary: Retry when hbase transaction conflict happen Key: HIVE-14106 URL: https://issues.apache.org/jira/browse/HIVE-14106 Project: Hive Issue Type: Improvement Components: HBase Metastore Reporter: Daniel Dai Assignee: Daniel Dai With HBase transaction manager (Omid), it is possible metastore operation aborted due to a new cause: transaction conflict. In this case, a concurrent transaction is underway and we need to implement retry logic. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-14107) Complete HBaseStore.setConf
Daniel Dai created HIVE-14107: - Summary: Complete HBaseStore.setConf Key: HIVE-14107 URL: https://issues.apache.org/jira/browse/HIVE-14107 Project: Hive Issue Type: Improvement Components: HBase Metastore Reporter: Daniel Dai Assignee: Daniel Dai HBaseStore.setConf currently has a barebone implementation and missing some features in ObjectStore.setConf. We need to review and complete it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-14108) Add missing objects in hbaseimport
Daniel Dai created HIVE-14108: - Summary: Add missing objects in hbaseimport Key: HIVE-14108 URL: https://issues.apache.org/jira/browse/HIVE-14108 Project: Hive Issue Type: Improvement Components: HBase Metastore Reporter: Daniel Dai Assignee: Daniel Dai The following objects are not imported with hbaseimport: privs (table/partition/column) column stats type/constraint/version -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-14110) Implement a better ObjectStore in hbase metastore
Daniel Dai created HIVE-14110: - Summary: Implement a better ObjectStore in hbase metastore Key: HIVE-14110 URL: https://issues.apache.org/jira/browse/HIVE-14110 Project: Hive Issue Type: Improvement Components: HBase Metastore Reporter: Daniel Dai Assignee: Daniel Dai The ObjectStore in hbase metastore is very naive and we need to enhance it to a decent one. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-14152) datanucleus.autoStartMechanismMode should set to 'Ignored' to allow rolling downgrade
Daniel Dai created HIVE-14152:
-
Summary: datanucleus.autoStartMechanismMode should set to
'Ignored' to allow rolling downgrade
Key: HIVE-14152
URL: https://issues.apache.org/jira/browse/HIVE-14152
Project: Hive
Issue Type: Bug
Components: Metastore
Reporter: Daniel Dai
Assignee: Daniel Dai
We see the following issue when downgrading metastore:
1. Run some query using new tables
2. Downgrade metastore
3. Restart metastore will complain the new table does not exist
In particular, constaints tables does not exist in branch-1. If we run Hive 2
and create a constraint, then downgrade metastore to Hive 1, datanucleus will
complain:
{code}
javax.jdo.JDOFatalUserException: Error starting up DataNucleus : a class
"org.apache.hadoop.hive.metastore.model.MConstraint" was listed as being
persisted previously in this datastore, yet the class wasnt found. Perhaps it
is used by a different DataNucleus-enabled application in this datastore, or
you have changed your class names.
at
org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:528)
at
org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:788)
at
org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
at
org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
at java.security.AccessController.doPrivileged(Native Method)
at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
at
javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
at
org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:377)
at
org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:406)
at
org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:299)
at
org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:266)
at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
at
org.apache.hadoop.hive.metastore.RawStoreProxy.(RawStoreProxy.java:60)
at
org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:69)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:650)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:628)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:677)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:484)
at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.(RetryingHMSHandler.java:77)
at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:83)
at
org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5905)
at
org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5900)
at
org.apache.hadoop.hive.metastore.HiveMetaStore.startMetaStore(HiveMetaStore.java:6159)
at
org.apache.hadoop.hive.metastore.HiveMetaStore.main(HiveMetaStore.java:6084)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
{code}
Apparently datanucleus cache some trace about the new table and retry to
reinstantiate later. This breaks downgrading we shall disable this behavior.
We need to set "datanucleus.autoStartMechanismMode" to "Ignored" to disable the
check since it becomes a norm in downgrading case.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HIVE-14293) PerfLogger.openScopes should be transient
Daniel Dai created HIVE-14293:
-
Summary: PerfLogger.openScopes should be transient
Key: HIVE-14293
URL: https://issues.apache.org/jira/browse/HIVE-14293
Project: Hive
Issue Type: Bug
Affects Versions: 2.1.0
Reporter: Daniel Dai
Assignee: Daniel Dai
Attachments: HIVE-14293.1.patch
See the following exception when running Hive e2e tests:
{code}
0: jdbc:hive2://nat-r6-ojss-hsihs2-1.openstac> SELECT s.name, s2.age, s.gpa,
v.registration, v2.contributions FROM student s INNER JOIN voter v ON (s.name =
v.name) INNER JOIN student s2 ON (s2.age = v.age and s.name = s2.name) INNER
JOIN voter v2 ON (v2.name = s2.name and v2.age = s2.age) WHERE v2.age = s.age
ORDER BY s.name, s2.age, s.gpa, v.registration, v2.contributions;
INFO : Compiling
command(queryId=hive_20160717224915_3a52719f-539f-4f82-a9cd-0c0af4e09ef8):
SELECT s.name, s2.age, s.gpa, v.registration, v2.contributions FROM student s
INNER JOIN voter v ON (s.name = v.name) INNER JOIN student s2 ON (s2.age =
v.age and s.name = s2.name) INNER JOIN voter v2 ON (v2.name = s2.name and
v2.age = s2.age) WHERE v2.age = s.age ORDER BY s.name, s2.age, s.gpa,
v.registration, v2.contributions
INFO : Semantic Analysis Completed
INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:s.name,
type:string, comment:null), FieldSchema(name:s2.age, type:int, comment:null),
FieldSchema(name:s.gpa, type:double, comment:null),
FieldSchema(name:v.registration, type:string, comment:null),
FieldSchema(name:v2.contributions, type:float, comment:null)], properties:null)
INFO : Completed compiling
command(queryId=hive_20160717224915_3a52719f-539f-4f82-a9cd-0c0af4e09ef8); Time
taken: 1.165 seconds
INFO : Executing
command(queryId=hive_20160717224915_3a52719f-539f-4f82-a9cd-0c0af4e09ef8):
SELECT s.name, s2.age, s.gpa, v.registration, v2.contributions FROM student s
INNER JOIN voter v ON (s.name = v.name) INNER JOIN student s2 ON (s2.age =
v.age and s.name = s2.name) INNER JOIN voter v2 ON (v2.name = s2.name and
v2.age = s2.age) WHERE v2.age = s.age ORDER BY s.name, s2.age, s.gpa,
v.registration, v2.contributions
INFO : Query ID = hive_20160717224915_3a52719f-539f-4f82-a9cd-0c0af4e09ef8
INFO : Total jobs = 1
INFO : Launching Job 1 out of 1
INFO : Starting task [Stage-1:MAPRED] in serial mode
INFO : Session is already open
INFO : Dag name: SELECT s.name, s2.age, sv2.contributions(Stage-1)
ERROR : Failed to execute tez graph.
java.lang.RuntimeException: Error caching map.xml:
org.apache.hive.com.esotericsoftware.kryo.KryoException:
java.util.ConcurrentModificationException
Serialization trace:
classes (sun.misc.Launcher$AppClassLoader)
classloader (java.security.ProtectionDomain)
context (java.security.AccessControlContext)
acc (org.apache.hadoop.hive.ql.exec.UDFClassLoader)
classLoader (org.apache.hadoop.hive.conf.HiveConf)
conf (org.apache.hadoop.hive.common.metrics.metrics2.CodahaleMetrics)
metrics
(org.apache.hadoop.hive.common.metrics.metrics2.CodahaleMetrics$CodahaleMetricsScope)
openScopes (org.apache.hadoop.hive.ql.log.PerfLogger)
perfLogger (org.apache.hadoop.hive.ql.exec.MapJoinOperator)
childOperators (org.apache.hadoop.hive.ql.exec.MapJoinOperator)
childOperators (org.apache.hadoop.hive.ql.exec.MapJoinOperator)
childOperators (org.apache.hadoop.hive.ql.exec.SelectOperator)
childOperators (org.apache.hadoop.hive.ql.exec.FilterOperator)
childOperators (org.apache.hadoop.hive.ql.exec.TableScanOperator)
aliasToWork (org.apache.hadoop.hive.ql.plan.MapWork)
at
org.apache.hadoop.hive.ql.exec.Utilities.setBaseWork(Utilities.java:582)
~[hive-exec-2.1.0.2.5.0.0-1009.jar:2.1.0.2.5.0.0-1009]
at
org.apache.hadoop.hive.ql.exec.Utilities.setMapWork(Utilities.java:516)
~[hive-exec-2.1.0.2.5.0.0-1009.jar:2.1.0.2.5.0.0-1009]
at
org.apache.hadoop.hive.ql.exec.tez.DagUtils.createVertex(DagUtils.java:601)
~[hive-exec-2.1.0.2.5.0.0-1009.jar:2.1.0.2.5.0.0-1009]
at
org.apache.hadoop.hive.ql.exec.tez.DagUtils.createVertex(DagUtils.java:1147)
~[hive-exec-2.1.0.2.5.0.0-1009.jar:2.1.0.2.5.0.0-1009]
at org.apache.hadoop.hive.ql.exec.tez.TezTask.build(TezTask.java:390)
~[hive-exec-2.1.0.2.5.0.0-1009.jar:2.1.0.2.5.0.0-1009]
at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:164)
[hive-exec-2.1.0.2.5.0.0-1009.jar:2.1.0.2.5.0.0-1009]
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197)
[hive-exec-2.1.0.2.5.0.0-1009.jar:2.1.0.2.5.0.0-1009]
at
org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
[hive-exec-2.1.0.2.5.0.0-1009.jar:2.1.0.2.5.0.0-1009]
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1865)
[hive-exec-2.1.0.2.5.0.0-1009.jar:2.1.0.2.5.0.0-1009]
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1569)
[hive-exec-2.1.0.2.5.0.0-1009.jar:2.1.0.2.5.0.0-1
[jira] [Created] (HIVE-14399) Fix test flakiness of org.apache.hive.hcatalog.listener.TestDbNotificationListener.cleanupNotifs
Daniel Dai created HIVE-14399: - Summary: Fix test flakiness of org.apache.hive.hcatalog.listener.TestDbNotificationListener.cleanupNotifs Key: HIVE-14399 URL: https://issues.apache.org/jira/browse/HIVE-14399 Project: Hive Issue Type: Bug Components: HCatalog Reporter: Daniel Dai Assignee: Daniel Dai We get intermittent test failure of TestDbNotificationListener.cleanupNotifs. We shall make it stable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-14690) Query fail when hive.exec.parallel=true, with conflicting session dir
Daniel Dai created HIVE-14690:
-
Summary: Query fail when hive.exec.parallel=true, with conflicting
session dir
Key: HIVE-14690
URL: https://issues.apache.org/jira/browse/HIVE-14690
Project: Hive
Issue Type: Bug
Affects Versions: 2.1.0, 1.3.0
Reporter: Daniel Dai
Assignee: Daniel Dai
This happens when hive.scratchdir.lock=true. Error message:
{code}
/hive/scratch/343hdirdp/cab907fc-5e1d-4d69-aa72-d7b442495c7a/inuse.info (inode
19537): File does not exist. [Lease. Holder:
DFSClient_NONMAPREDUCE_1572639975_1, pendingcreates: 2]
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:3430)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.analyzeFileState(FSNamesystem.java:3235)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getNewBlockTargets(FSNamesystem.java:3073)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:3033)
at
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:725)
at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:492)
at
org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2137)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2133)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1668)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2131)
at
org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:535)
at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:74)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HIVE-14968) Fix compilation failure on branch-1
Daniel Dai created HIVE-14968: - Summary: Fix compilation failure on branch-1 Key: HIVE-14968 URL: https://issues.apache.org/jira/browse/HIVE-14968 Project: Hive Issue Type: Bug Reporter: Daniel Dai Assignee: Daniel Dai Fix For: 1.3.0 branch-1 compilation failure due to: HIVE-14436: Hive 1.2.1/Hitting "ql.Driver: FAILED: IllegalArgumentException Error: , expected at the end of 'decimal(9'" after enabling hive.optimize.skewjoin and with MR engine HIVE-14483 : java.lang.ArrayIndexOutOfBoundsException org.apache.orc.impl.TreeReaderFactory.commonReadByteArrays 1.2 branch is fine. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-15049) Fix unit test failures on branch-1
Daniel Dai created HIVE-15049:
-
Summary: Fix unit test failures on branch-1
Key: HIVE-15049
URL: https://issues.apache.org/jira/browse/HIVE-15049
Project: Hive
Issue Type: Bug
Components: Test
Affects Versions: 1.3.0
Reporter: Daniel Dai
When working on HIVE-14968, I notice there are 36 test failures and quite a few
other tests did not produce a TEST-*.xml file. At least some of them are valid.
Here is one of the stack:
{code}
java.lang.Exception: java.lang.RuntimeException: Error in configuring object
at
org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:462)
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:529)
Caused by: java.lang.RuntimeException: Error in configuring object
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109)
at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:75)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133)
at
org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:409)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:392)
at
org.apache.hadoop.mapred.LocalJobRunner$Job$ReduceTaskRunnable.run(LocalJobRunner.java:319)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.GeneratedMethodAccessor16.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
... 10 more
Caused by: java.lang.RuntimeException: Reduce operator initialization failed
at
org.apache.hadoop.hive.ql.exec.mr.ExecReducer.configure(ExecReducer.java:157)
... 14 more
Caused by: java.lang.RuntimeException: Cannot find ExprNodeEvaluator for the
exprNodeDesc = null
at
org.apache.hadoop.hive.ql.exec.ExprNodeEvaluatorFactory.get(ExprNodeEvaluatorFactory.java:57)
at
org.apache.hadoop.hive.ql.exec.GroupByOperator.initializeOp(GroupByOperator.java:272)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:363)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:482)
at
org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:439)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
at
org.apache.hadoop.hive.ql.exec.mr.ExecReducer.configure(ExecReducer.java:150)
... 14 more
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HIVE-15068) Run ClearDanglingScratchDir periodically inside HS2
Daniel Dai created HIVE-15068: - Summary: Run ClearDanglingScratchDir periodically inside HS2 Key: HIVE-15068 URL: https://issues.apache.org/jira/browse/HIVE-15068 Project: Hive Issue Type: Improvement Components: HiveServer2 Reporter: Daniel Dai Assignee: Daniel Dai In HIVE-13429, we introduce a tool which clear dangling scratch directory. In this ticket, we want to invoke the tool automatically on a Hive cluster. Options are: 1. cron job, which would involve manual cron job setup 2. As a metastore thread. However, it is possible we run metastore without hdfs in the future (eg, managing s3 files). ClearDanglingScratchDir needs support which only exists in hdfs, it won't work if the above scenario happens 3. As a HS2 thread. The downside is if no HS2 is running, the tool will not run automatically. But we expect HS2 will be a required component down the road Here I choose approach 3 in the implementation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-15322) Skipping "hbase mapredcp" in hive script for certain services
Daniel Dai created HIVE-15322: - Summary: Skipping "hbase mapredcp" in hive script for certain services Key: HIVE-15322 URL: https://issues.apache.org/jira/browse/HIVE-15322 Project: Hive Issue Type: Improvement Reporter: Daniel Dai Assignee: Daniel Dai "hbase mapredcp" is intended to append hbase classpath to hive. However, the command can take some time when the system is heavy loaded. In some extreme cases, we saw ~20s delay due to it. For certain commands, such as "schemaTool", hbase classpath is certainly useless, and we can safely skip invoking it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-15448) ChangeManager for replication
Daniel Dai created HIVE-15448: - Summary: ChangeManager for replication Key: HIVE-15448 URL: https://issues.apache.org/jira/browse/HIVE-15448 Project: Hive Issue Type: Sub-task Components: repl Reporter: Daniel Dai Assignee: Daniel Dai The change manager implementation as described in https://cwiki.apache.org/confluence/display/Hive/HiveReplicationv2Development#HiveReplicationv2Development-Changemanagement. This issue tracks the infrastructure code. Hooking to actions will be tracked in other ticket. ReplChangeManager includes: * method to generate checksum * method to convert file path to cm path * method to move table/partition/file into cm * thread to clear cm files if expires -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-15525) Hooking ChangeManager to "drop table", "drop partition"
Daniel Dai created HIVE-15525:
-
Summary: Hooking ChangeManager to "drop table", "drop partition"
Key: HIVE-15525
URL: https://issues.apache.org/jira/browse/HIVE-15525
Project: Hive
Issue Type: Sub-task
Components: repl
Reporter: Daniel Dai
Assignee: Daniel Dai
When Hive "drop table"/"drop partition", we will move data files into cmroot in
case the replication destination will need it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HIVE-15587) Using ChangeManager to copy files in ReplCopyTask
Daniel Dai created HIVE-15587: - Summary: Using ChangeManager to copy files in ReplCopyTask Key: HIVE-15587 URL: https://issues.apache.org/jira/browse/HIVE-15587 Project: Hive Issue Type: Sub-task Components: repl Reporter: Daniel Dai Assignee: Daniel Dai Currently ReplCopyTask copy files directly from source repo. The files in the source repo may have been dropped or change. We shall use checksum transferred to ReplCopyTask to verify. If different, retrieve file from cmroot instead. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-15724) getPrimaryKeys and getForeignKeys in metastore does not normalize db and table name
Daniel Dai created HIVE-15724: - Summary: getPrimaryKeys and getForeignKeys in metastore does not normalize db and table name Key: HIVE-15724 URL: https://issues.apache.org/jira/browse/HIVE-15724 Project: Hive Issue Type: Bug Components: Metastore Reporter: Daniel Dai Assignee: Daniel Dai In db, everything is lower case. When we retrieve constraints back, we need to normalize dbname/tablename. Otherwise, the following sample script fail: alter table Table9 add constraint pk1 primary key (a) disable novalidate; ALTER TABLE Table9 drop constraint pk1; Error message: InvalidObjectException(message:The constraint: pk1 does not exist for the associated table: default.Table9 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-15740) Include hive-hcatalog-core.jar and hive-hcatalog-server-extensions.jar in binary distribution
Daniel Dai created HIVE-15740: - Summary: Include hive-hcatalog-core.jar and hive-hcatalog-server-extensions.jar in binary distribution Key: HIVE-15740 URL: https://issues.apache.org/jira/browse/HIVE-15740 Project: Hive Issue Type: Bug Components: distribution Reporter: Daniel Dai Assignee: Daniel Dai Currently both jars are in hcatalog/share/hcatalog and not in classpath. Metastore using DbNotificationListener will fail with CNF exception. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HIVE-15840) Webhcat test TestPig_5 failing with Pig on Tez at check for percent complete of job
Daniel Dai created HIVE-15840: - Summary: Webhcat test TestPig_5 failing with Pig on Tez at check for percent complete of job Key: HIVE-15840 URL: https://issues.apache.org/jira/browse/HIVE-15840 Project: Hive Issue Type: Bug Components: HCatalog Reporter: Daniel Dai Assignee: Daniel Dai TestPig_5 is failing at percentage check if the job is Pig on Tez: check_job_percent_complete failed. got percentComplete , expected 100% complete Test command: curl -d user.name=daijy -d arg=-p -d arg=INPDIR=/tmp/templeton_test_data -d arg=-p -d arg=OUTDIR=/tmp/output -d file=loadstore.pig -X POST http://localhost:50111/templeton/v1/pig curl http://localhost:50111/templeton/v1/jobs/job_1486502484681_0003?user.name=daijy This is similar to HIVE-9351, which fixes Hive on Tez. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HIVE-15935) ACL is not set in ATS data
Daniel Dai created HIVE-15935: - Summary: ACL is not set in ATS data Key: HIVE-15935 URL: https://issues.apache.org/jira/browse/HIVE-15935 Project: Hive Issue Type: Bug Reporter: Daniel Dai Assignee: Daniel Dai When publishing ATS info, Hive does not set ACL, that make Hive ATS entries visible to all users. On the other hand, Tez ATS entires is using Tez DAG ACL which limit both view/modify ACL to end user only. We shall make them consistent. In the Jira, I am going to limit ACL to end user for both Tez ATS and Hive ATS, also provide config "hive.view.acls" and "hive.modify.acls" if user need to overridden. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HIVE-15936) ConcurrentModificationException in ATSHook
Daniel Dai created HIVE-15936:
-
Summary: ConcurrentModificationException in ATSHook
Key: HIVE-15936
URL: https://issues.apache.org/jira/browse/HIVE-15936
Project: Hive
Issue Type: Bug
Reporter: Daniel Dai
Assignee: Daniel Dai
Attachments: HIVE-15936.1.patch
See ATSHook error:
{noformat}
java.util.ConcurrentModificationException
at java.util.HashMap$HashIterator.nextNode(HashMap.java:1437)
~[?:1.8.0_112]
at java.util.HashMap$EntryIterator.next(HashMap.java:1471)
~[?:1.8.0_112]
at java.util.HashMap$EntryIterator.next(HashMap.java:1469)
~[?:1.8.0_112]
at java.util.AbstractCollection.toArray(AbstractCollection.java:196)
~[?:1.8.0_112]
at com.google.common.collect.ImmutableMap.copyOf(ImmutableMap.java:290)
~[guava-14.0.1.jar:?]
at
org.apache.hadoop.hive.ql.log.PerfLogger.getEndTimes(PerfLogger.java:219)
~[hive-common-2.1.0.2.6.0.0-457.jar:2.1.0.2.6.0.0-457]
at
org.apache.hadoop.hive.ql.hooks.ATSHook.createPostHookEvent(ATSHook.java:347)
~[hive-exec-2.1.0.2.6.0.0-457.jar:2.1.0.2.6.0.0-457]
at org.apache.hadoop.hive.ql.hooks.ATSHook$2.run(ATSHook.java:206)
[hive-exec-2.1.0.2.6.0.0-457.jar:2.1.0.2.6.0.0-457]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[?:1.8.0_112]
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[?:1.8.0_112]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[?:1.8.0_112]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[?:1.8.0_112]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_112]
{noformat}
According to [~jdere], ATSHook is currently accessing the PerfLogger on a
separate thread, which means the main query thread can potentially write to the
PerfLogger at the same time.
The ATSHook should access the PerfLogger on the main query thread, before it
sends the execution to the ATS Logger thread.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (HIVE-16305) Additional Datanucleus ClassLoaderResolverImpl leaks causing HS2 OOM
Daniel Dai created HIVE-16305: - Summary: Additional Datanucleus ClassLoaderResolverImpl leaks causing HS2 OOM Key: HIVE-16305 URL: https://issues.apache.org/jira/browse/HIVE-16305 Project: Hive Issue Type: Bug Components: HiveServer2 Reporter: Daniel Dai Assignee: Daniel Dai This is a followup for HIVE-16160. We see additional ClassLoaderResolverImpl leaks even with the patch. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HIVE-16323) HS2 JDOPersistenceManagerFactory.pmCache leaks after HIVE-14204
Daniel Dai created HIVE-16323: - Summary: HS2 JDOPersistenceManagerFactory.pmCache leaks after HIVE-14204 Key: HIVE-16323 URL: https://issues.apache.org/jira/browse/HIVE-16323 Project: Hive Issue Type: Bug Components: HiveServer2 Reporter: Daniel Dai Assignee: Daniel Dai Hive.loadDynamicPartitions creates threads with new embedded rawstore, but never close them, thus we leak PersistenceManager one per such thread. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HIVE-16520) Cache hive metadata in metastore
Daniel Dai created HIVE-16520: - Summary: Cache hive metadata in metastore Key: HIVE-16520 URL: https://issues.apache.org/jira/browse/HIVE-16520 Project: Hive Issue Type: New Feature Components: Metastore Reporter: Daniel Dai Assignee: Daniel Dai During Hive 2 benchmark, we find Hive metastore operation take a lot of time and thus slow down Hive compilation. In some extreme case, it takes much longer than the actual query run time. Especially, we find the latency of cloud db is very high and 90% of total query runtime is waiting for metastore SQL database operations. Based on this observation, the metastore operation performance will be greatly enhanced if we have a memory structure which cache the database query result. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (HIVE-21379) Mask password in DDL commands for table properties
Daniel Dai created HIVE-21379: - Summary: Mask password in DDL commands for table properties Key: HIVE-21379 URL: https://issues.apache.org/jira/browse/HIVE-21379 Project: Hive Issue Type: Improvement Reporter: Daniel Dai Assignee: Daniel Dai Attachments: HIVE-21379.1.patch We need to mask password related table properties (such as hive.sql.dbcp.password) in DDL output, such as describe extended/describe formatted/show create table/show tblproperties. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-21389) Hive distribution miss javax.ws.rs-api.jar after HIVE-21247
Daniel Dai created HIVE-21389: - Summary: Hive distribution miss javax.ws.rs-api.jar after HIVE-21247 Key: HIVE-21389 URL: https://issues.apache.org/jira/browse/HIVE-21389 Project: Hive Issue Type: Bug Reporter: Daniel Dai Assignee: Daniel Dai -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-21478) Metastore cache update shall capture exception
Daniel Dai created HIVE-21478: - Summary: Metastore cache update shall capture exception Key: HIVE-21478 URL: https://issues.apache.org/jira/browse/HIVE-21478 Project: Hive Issue Type: Bug Components: Standalone Metastore Reporter: Daniel Dai Assignee: Daniel Dai Attachments: HIVE-21478.1.patch We definitely need to capture any exception during CacheUpdateMasterWork.update(), otherwise, Java would refuse to schedule future update(). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-21479) NPE during metastore cache update
Daniel Dai created HIVE-21479:
-
Summary: NPE during metastore cache update
Key: HIVE-21479
URL: https://issues.apache.org/jira/browse/HIVE-21479
Project: Hive
Issue Type: Bug
Reporter: Daniel Dai
Assignee: Daniel Dai
Saw the following stack during a long periodical update:
{code}
2019-03-12T10:01:43,015 ERROR [CachedStore-CacheUpdateService: Thread-36]
cache.CachedStore: Update failure:java.lang.NullPointerException
at
org.apache.hadoop.hive.metastore.cache.CachedStore$CacheUpdateMasterWork.updateTableColStats(CachedStore.java:508)
at
org.apache.hadoop.hive.metastore.cache.CachedStore$CacheUpdateMasterWork.update(CachedStore.java:461)
at
org.apache.hadoop.hive.metastore.cache.CachedStore$CacheUpdateMasterWork.run(CachedStore.java:396)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
{code}
The reason is we get the table list at very early stage and then refresh table
one by one. It is likely table is removed during the interim. We need to deal
with this case during cache update.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HIVE-21583) KillTriggerActionHandler should use "hive" credential
Daniel Dai created HIVE-21583: - Summary: KillTriggerActionHandler should use "hive" credential Key: HIVE-21583 URL: https://issues.apache.org/jira/browse/HIVE-21583 Project: Hive Issue Type: Bug Reporter: Daniel Dai Assignee: Daniel Dai Currently SessionState.username is set to null, which is invalid as KillQueryImplementation will valid the user privilege. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-21625) Fix TxnIdUtils.checkEquivalentWriteIds, also provides a comparison method
Daniel Dai created HIVE-21625:
-
Summary: Fix TxnIdUtils.checkEquivalentWriteIds, also provides a
comparison method
Key: HIVE-21625
URL: https://issues.apache.org/jira/browse/HIVE-21625
Project: Hive
Issue Type: Bug
Environment: TxnIdUtils.checkEquivalentWriteIds has a bug which thinks
({1,2,3,4}, 6) and ({1,2,3,4,5,6}, 8) compatible (the notation is (invalidlist,
hwm)). Here is a patch to fix it, also provide a comparison method to check
which is newer.
Reporter: Daniel Dai
Assignee: Daniel Dai
Attachments: HIVE-21625.1.patch
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HIVE-21637) Synchronized metastore cache
Daniel Dai created HIVE-21637: - Summary: Synchronized metastore cache Key: HIVE-21637 URL: https://issues.apache.org/jira/browse/HIVE-21637 Project: Hive Issue Type: New Feature Reporter: Daniel Dai Assignee: Daniel Dai Currently, HMS has a cache implemented by CachedStore. The cache is asynchronized and in HMS HA setting, we can only get eventual consistency. In this Jira, we try to make it synchronized. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-21697) Remove periodical full refresh in HMS cache
Daniel Dai created HIVE-21697: - Summary: Remove periodical full refresh in HMS cache Key: HIVE-21697 URL: https://issues.apache.org/jira/browse/HIVE-21697 Project: Hive Issue Type: Improvement Components: Standalone Metastore Reporter: Daniel Dai Assignee: Daniel Dai In HIVE-18661, we added periodical notification based refresh in HMS cache. We shall remove periodical full refresh to simplify the code as it will no longer be used. In the mean time, we introduced mechanism to provide monotonic reads through the CachedStore.commitTransaction. This will no longer be needed after HIVE-21637. So I will remove related code as well. This will provide some performance benefits include: 1. We don't have to slow down write to catch up notification logs. Write can be done immediately and tag the cache with writeids 2. We can read from cache even if updateUsingNotificationEvents is running. Read will compare the writeids of the cache so monotonic reads will be guaranteed I'd like to put a patch separately with HIVE-21637 so it can be tested independently. HMW will use periodical notification based refresh to update cache. And it will temporarily lift the monotonic reads guarantee until HIVE-21637 checkin. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HIVE-22014) Tear down locks in CachedStore
Daniel Dai created HIVE-22014: - Summary: Tear down locks in CachedStore Key: HIVE-22014 URL: https://issues.apache.org/jira/browse/HIVE-22014 Project: Hive Issue Type: Sub-task Reporter: Daniel Dai There's a lot of locks in CachedStore. After HIVE-21637, only notification log puller thread will update the cache. And when it process event, the first thing is to mark the entry invalid. The only exception may be TableWrapperSizeUpdater, but we can also make it synchronous (maybe run it once after every iteration of notification log puller). There should be no synchronization issue and we can tear down existing locks to simplify the code. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (HIVE-22015) Cache table constraints in CachedStore
Daniel Dai created HIVE-22015: - Summary: Cache table constraints in CachedStore Key: HIVE-22015 URL: https://issues.apache.org/jira/browse/HIVE-22015 Project: Hive Issue Type: Sub-task Reporter: Daniel Dai Currently table constraints are not cached. Hive will pull all constraints from tables involved in query, which results multiple db reads (including get_primary_keys, get_foreign_keys, get_unique_constraints, etc). The effort to cache this is small as it's just another table component. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (HIVE-22016) Do not open transaction for readonly query
Daniel Dai created HIVE-22016: - Summary: Do not open transaction for readonly query Key: HIVE-22016 URL: https://issues.apache.org/jira/browse/HIVE-22016 Project: Hive Issue Type: Sub-task Reporter: Daniel Dai Open/abort/commit transaction would increment transaction id which is a burden unnecessarily. In addition, it spams the notification log and make CachedStore (and of cause other components rely on notification log) harder to catch up. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (HIVE-22017) HMS interface backward compatible after HIVE-21637
Daniel Dai created HIVE-22017: - Summary: HMS interface backward compatible after HIVE-21637 Key: HIVE-22017 URL: https://issues.apache.org/jira/browse/HIVE-22017 Project: Hive Issue Type: Sub-task Reporter: Daniel Dai HIVE-21637 changes a bunch HMS interface to add writeid into all get_xxx calls. Ideally we shall provide original version and forward it to the new api to make the change backward compatible. The downside is double the size of HMS methods. We shall mark those deprecated and remove in future version. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (HIVE-22018) Add table id to HMS get methods
Daniel Dai created HIVE-22018: - Summary: Add table id to HMS get methods Key: HIVE-22018 URL: https://issues.apache.org/jira/browse/HIVE-22018 Project: Hive Issue Type: Sub-task Reporter: Daniel Dai It is possible we remove a table and immediately move another table to occupy the same name. CachedStore may retrieve the wrong table in this case. We shall add tableid in every get_(table/partition) api, so we can compare the one stored in TBLS (tableid is part of Table object) and check if the same id, if not, HMS shall fail the read request. The initial table id can be retrieved along with writeid (in DbTxnManager.getValidWriteIds call, to join the TBLS table) -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (HIVE-22019) alter_table_update_status/alter_table_update_status_disable_bitvector/alter_partition_update_status fail when DbNotificationListener is installed
Daniel Dai created HIVE-22019:
-
Summary:
alter_table_update_status/alter_table_update_status_disable_bitvector/alter_partition_update_status
fail when DbNotificationListener is installed
Key: HIVE-22019
URL: https://issues.apache.org/jira/browse/HIVE-22019
Project: Hive
Issue Type: Sub-task
Reporter: Daniel Dai
Statement like:
ALTER TABLE src_stat_n0 UPDATE STATISTICS for column key SET
('numDVs'='','avgColLen'='1.111')
fail when DbNotificationListener is installed with the message:
{code}
See ./ql/target/tmp/log/hive.log or ./itests/qtest/target/tmp/log/hive.log, or
check ./ql/target/surefire-reports or ./itests/qtest/target/surefire-reports/
for specific test cases logs.
org.apache.hadoop.hive.ql.metadata.HiveException:
java.lang.IllegalArgumentException: Could not serialize
JSONUpdateTableColumnStatMessage :
at
org.apache.hadoop.hive.ql.metadata.Hive.setPartitionColumnStatistics(Hive.java:5350)
at
org.apache.hadoop.hive.ql.exec.ColumnStatsUpdateTask.persistColumnStats(ColumnStatsUpdateTask.java:339)
at
org.apache.hadoop.hive.ql.exec.ColumnStatsUpdateTask.execute(ColumnStatsUpdateTask.java:347)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:212)
at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:103)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:2343)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1995)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1662)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1422)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1416)
at org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:162)
at org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:223)
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:242)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:189)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:408)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:340)
at
org.apache.hadoop.hive.ql.QTestUtil.executeClientInternal(QTestUtil.java:680)
at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:651)
at
org.apache.hadoop.hive.cli.control.CoreCliDriver.runTest(CoreCliDriver.java:182)
at org.apache.hadoop.hive.cli.control.CliAdapter.runTest(CliAdapter.java:104)
at
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver(TestCliDriver.java:59)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.apache.hadoop.hive.cli.control.CliAdapter$2$1.evaluate(CliAdapter.java:92)
at org.junit.rules.RunRules.evaluate(RunRules.java:20)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
at org.junit.runners.Suite.runChild(Suite.java:127)
at org.junit.runners.Suite.runChild(Suite.java:26)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
at
org.apache.hadoop.hive.cli.control.CliAdapter$1$1.evaluate(CliAdapter.java:73)
at org.junit.rules.RunRules.evaluate(RunRules.java:20)
at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
at
org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
at
org.apache.maven.surefire.junit4.JUnit
[jira] [Created] (HIVE-2800) NPE in "create index" without comment clause in external metastore
NPE in "create index" without comment clause in external metastore -- Key: HIVE-2800 URL: https://issues.apache.org/jira/browse/HIVE-2800 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 0.8.1, 0.9.0 Reporter: Daniel Dai Priority: Minor This happens only when using external metastore (with --hiveconf hive.metastore.uris=thrift://localhost:8088 --hiveconf hive.metastore.local=false). Also if I gave a comment in the statement, this exception go away. Here is the statement: create index test111 on table hcat_test(name) as 'compact' with deferred rebuild; Here is the stack: 2012-02-10 17:07:42,612 ERROR exec.Task (SessionState.java:printError(380)) - FAILED: Error in metadata: java.lang.NullPointerException org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.NullPointerException at org.apache.hadoop.hive.ql.metadata.Hive.createIndex(Hive.java:725) at org.apache.hadoop.hive.ql.exec.DDLTask.createIndex(DDLTask.java:822) at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:231) at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:134) at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57) at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1291) at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1082) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:933) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:255) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:212) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:671) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:554) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.util.RunJar.main(RunJar.java:156) Caused by: java.lang.NullPointerException at org.apache.thrift.protocol.TBinaryProtocol.writeString(TBinaryProtocol.java:185) at org.apache.hadoop.hive.metastore.api.Index.write(Index.java:1032) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$add_index_args.write(ThriftHiveMetastore.java:47518) at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:63) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.send_add_index(ThriftHiveMetastore.java:1675) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.add_index(ThriftHiveMetastore.java:1666) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createIndex(HiveMetaStoreClient.java:853) at org.apache.hadoop.hive.ql.metadata.Hive.createIndex(Hive.java:722) ... 17 more -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
