Wei-Chiu Chuang created HDFS-12293: -------------------------------------- Summary: DataNode should log file name on disk error Key: HDFS-12293 URL: https://issues.apache.org/jira/browse/HDFS-12293 Project: Hadoop HDFS Issue Type: Bug Components: datanode Reporter: Wei-Chiu Chuang
Found the following error message in precommit build https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/488/testReport/junit/org.apache.hadoop.hdfs.server.datanode/TestDataNodeVolumeFailureReporting/testSuccessiveVolumeFailures/ {noformat} 2017-08-10 09:36:53,619 [DataXceiver for client DFSClient_NONMAPREDUCE_670847838_18 at /127.0.0.1:55851 [Receiving block BP-219227751-172.17.0.2-1502357801473:blk_1073741829_1005]] WARN datanode.DataNode (BlockReceiver.java:<init>(287)) - IOException in BlockReceiver constructor. Cause is java.io.IOException: Not a directory at java.io.UnixFileSystem.createFileExclusively(Native Method) at java.io.File.createNewFile(File.java:1012) at org.apache.hadoop.hdfs.server.datanode.FileIoProvider.createFile(FileIoProvider.java:302) at org.apache.hadoop.hdfs.server.datanode.DatanodeUtil.createFileWithExistsCheck(DatanodeUtil.java:69) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.BlockPoolSlice.createRbwFile(BlockPoolSlice.java:306) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsVolumeImpl.createRbwFile(FsVolumeImpl.java:933) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsVolumeImpl.createRbw(FsVolumeImpl.java:1202) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.createRbw(FsDatasetImpl.java:1356) at org.apache.hadoop.hdfs.server.datanode.BlockReceiver.<init>(BlockReceiver.java:215) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.getBlockReceiver(DataXceiver.java:1291) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:758) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:173) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:107) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) {noformat} It is not known what file was being created. What's interesting is that {{DatanodeUtil#createFileWithExistsCheck}} does carry file name in its log message, but the exception handler at {{DataTransfer#run()}} and {{BlockReceiver#BlockReceiver}} ignores it: {code:title=BlockReceiver#BlockReceiver} // check if there is a disk error IOException cause = DatanodeUtil.getCauseIfDiskError(ioe); DataNode.LOG.warn("IOException in BlockReceiver constructor" + (cause == null ? "" : ". Cause is "), cause); if (cause != null) { ioe = cause; // Volume error check moved to FileIoProvider } {code} The logger should print the file name in addition to the cause. -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org