[
https://issues.apache.org/jira/browse/HDFS-8475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15175054#comment-15175054
]
Harshal Joshi commented on HDFS-8475:
-------------------------------------
Hi Team,
Has anyone got a clue about this issue. Currently I am facing the same issue
while inserting data into table hive table with ORC file formats.
ooks like this is a generic issue for below case :
1. create external table T1(col A , B , C) with partition on (col A) stored as
ORC . Load table with substantial data. in my case around 85 GB data.
2. Create external table T2(col A,B,C) with partition on (Col B) stored as ORC.
Load table T2 from T1 with dynamic partition.
Output :- Premature EOF exception
Thanks
> Exception in createBlockOutputStream java.io.EOFException: Premature EOF: no
> length prefix available
> ----------------------------------------------------------------------------------------------------
>
> Key: HDFS-8475
> URL: https://issues.apache.org/jira/browse/HDFS-8475
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 2.2.0
> Reporter: Vinod Valecha
> Priority: Blocker
>
> Scenraio:
> =========
> write a file
> corrupt block manually
> Exception stack trace-
> 2015-05-24 02:31:55.291 INFO [T-33716795]
> [org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer] Exception in
> createBlockOutputStream
> java.io.EOFException: Premature EOF: no length prefix available
> at
> org.apache.hadoop.hdfs.protocolPB.PBHelper.vintPrefixed(PBHelper.java:1492)
> at
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:1155)
> at
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1088)
> at
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:514)
> [5/24/15 2:31:55:291 UTC] 02027a3b DFSClient I
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer createBlockOutputStream
> Exception in createBlockOutputStream
> java.io.EOFException: Premature EOF: no
> length prefix available
> at
> org.apache.hadoop.hdfs.protocolPB.PBHelper.vintPrefixed(PBHelper.java:1492)
> at
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:1155)
> at
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1088)
> at
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:514)
> 2015-05-24 02:31:55.291 INFO [T-33716795]
> [org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer] Abandoning
> BP-176676314-10.108.106.59-1402620296713:blk_1404621403_330880579
> [5/24/15 2:31:55:291 UTC] 02027a3b DFSClient I
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer nextBlockOutputStream
> Abandoning BP-176676314-10.108.106.59-1402620296713:blk_1404621403_330880579
> 2015-05-24 02:31:55.299 INFO [T-33716795]
> [org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer] Excluding datanode
> 10.108.106.59:50010
> [5/24/15 2:31:55:299 UTC] 02027a3b DFSClient I
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer nextBlockOutputStream
> Excluding datanode 10.108.106.59:50010
> 2015-05-24 02:31:55.300 WARNING [T-33716795]
> [org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer] DataStreamer Exception
> org.apache.hadoop.ipc.RemoteException(java.io.IOException): File
> /var/db/opera/files/B4889CCDA75F9751DDBB488E5AAB433E/BE4DAEF290B7136ED6EF3D4B157441A2/BE4DAEF290B7136ED6EF3D4B157441A2-4.pag
> could only be replicated to 0 nodes instead of minReplication (=1). There
> are 1 datanode(s) running and 1 node(s) are excluded in this operation.
> at
> org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.chooseTarget(BlockManager.java:1384)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:2477)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:555)
> [5/24/15 2:31:55:300 UTC] 02027a3b DFSClient W
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer run DataStreamer Exception
>
> org.apache.hadoop.ipc.RemoteException(java.io.IOException): File
> /var/db/opera/files/B4889CCDA75F9751DDBB488E5AAB433E/BE4DAEF290B7136ED6EF3D4B157441A2/BE4DAEF290B7136ED6EF3D4B157441A2-4.pag
> could only be replicated to 0 nodes instead of minReplication (=1). There
> are 1 datanode(s) running and 1 node(s) are excluded in this operation.
> at
> org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.chooseTarget(BlockManager.java:1384)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:2477)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:555)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:387)
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java:59582)
>
>
> 2015-05-24 02:31:55.301 WARNING [T-880] [E-AA380B730CF751508DC9163BAC8E4D1D]
> [job:B94FEC9411E2C8563C842833D78142CF]
> [org.apache.hadoop.hdfs.DFSOutputStream] Error while syncing
> org.apache.hadoop.ipc.RemoteException(java.io.IOException): File
> /var/db/opera/files/B4889CCDA75F9751DDBB488E5AAB433E/BE4DAEF290B7136ED6EF3D4B157441A2/BE4DAEF290B7136ED6EF3D4B157441A2-4.pag
> could only be replicated to 0 nodes instead of minReplication (=1). There
> are 1 datanode(s) running and 1 node(s) are excluded in this operation.
> at
> org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.chooseTarget(BlockManager.java:1384)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:2477)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:555)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:387)
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java:59582)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:585)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:928)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2048)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
