Daniel Dai created HIVE-18530:
---------------------------------
Summary: Replication should skip MM table (for now)
Key: HIVE-18530
URL: https://issues.apache.org/jira/browse/HIVE-18530
Project: Hive
Issue Type: Bug
Components: repl
Reporter: Daniel Dai
Assignee: Daniel Dai
Currently replication cannot handle transactional table (including MM table)
until HIVE-18320. HIVE-17504 skips table with transactional=true explicitly.
HIVE-18352 changes the logic to use AcidUtils.isAcidTable for the same purpose.
However, isAcidTable returns false for mm table, thus Hive still dump mm table
during replication.
Here is an error message during dump mm table:
{code}
ERROR : FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.repl.ReplDumpTask.
java.io.FileNotFoundException: Path is not a file:
/apps/hive/warehouse/testrepldb5.db/test1/delta_0000261_0000261_0000
at
org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:89)
at
org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:75)
at
org.apache.hadoop.hdfs.server.namenode.FSDirStatAndListingOp.getBlockLocations(FSDirStatAndListingOp.java:154)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1920)
at
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:731)
at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getBlockLocations(ClientNamenodeProtocolServerSideTranslatorPB.java:424)
at
org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:523)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:991)
at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:869)
at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:815)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1965)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2675)
INFO : Completed executing
command(queryId=hive_20180119203438_293813df-7630-47fa-bc30-5ef7cbb42842); Time
taken: 1.119 seconds
Error: Error while processing statement: FAILED: Execution Error, return code 1
from org.apache.hadoop.hive.ql.exec.repl.ReplDumpTask.
java.io.FileNotFoundException: Path is not a file:
/apps/hive/warehouse/testrepldb5.db/test1/delta_0000261_0000261_0000
at
org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:89)
at
org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:75)
at
org.apache.hadoop.hdfs.server.namenode.FSDirStatAndListingOp.getBlockLocations(FSDirStatAndListingOp.java:154)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1920)
at
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:731)
at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getBlockLocations(ClientNamenodeProtocolServerSideTranslatorPB.java:424)
at
org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:523)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:991)
at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:869)
at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:815)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1965)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2675)
(state=08S01,code=1)
0: jdbc:hive2://ctr-e137-1514896590304-25219-> Closing: 0:
jdbc:hive2://ctr-e137-1514896590304-25219-02-000005.hwx.site:2181,ctr-e137-1514896590304-25219-02-000012.hwx.site:2181,ctr-e137-1514896590304-25219-02-000009.hwx.site:2181,ctr-e137-1514896590304-25219-02-000004.hwx.site:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2;principal=hive/[email protected]
{code}
We shall switch to use AcidUtils.isTransactionalTable.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
