Manish Gupta created CARBONDATA-1277:
----------------------------------------
Summary: Dictionary generation failure if there is failure in
closing output steam in HDFS
Key: CARBONDATA-1277
URL: https://issues.apache.org/jira/browse/CARBONDATA-1277
Project: CarbonData
Issue Type: Bug
Reporter: Manish Gupta
Assignee: Manish Gupta
Fix For: 1.2.0
If there is any failure while closing the output stream of dictionary file in
HDFS then on next data load, update or insert into operation dictionary
generation fails. This is because we open the dictionary file in append mode
and when we try to get the output stream for that file HDFS throws an exception
that Lease is already acquired by some other client. Exception trace as below.
java.io.IOException: Failed to APPEND_FILE
/user/hive/warehouse/carbon.store/test/t12/Metadata/010497af-1833-4804-a7ec-849ab7b9bf10.dictmeta
for DFSClient_NONMAPREDUCE_-1904161438_159 on 172.168.100.212 because lease
recovery is in progress. Try again later.
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.recoverLeaseInternal(FSNamesystem.java:2901)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.appendFileInternal(FSNamesystem.java:2655)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.appendFileInt(FSNamesystem.java:2968)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.appendFile(FSNamesystem.java:2937)
at
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.append(NameNodeRpcServer.java:776)
at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.append(ClientNamenodeProtocolServerSideTranslatorPB.java:458)
at
org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
