[
https://issues.apache.org/jira/browse/HDFS-1846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13026049#comment-13026049
]
Hadoop QA commented on HDFS-1846:
---------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12477582/hdfs-1846.1.patch
against trunk revision 1097252.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 2 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac
compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9)
warnings.
+1 release audit. The applied patch does not increase the total number of
release audit warnings.
-1 core tests. The patch failed these core unit tests:
org.apache.hadoop.hdfs.server.namenode.TestEditLogFileOutputStream
org.apache.hadoop.hdfs.TestFileConcurrentReader
org.apache.hadoop.hdfs.tools.offlineEditsViewer.TestOfflineEditsViewer
+1 contrib tests. The patch passed contrib unit tests.
+1 system test framework. The patch passed system test framework compile.
Test results:
https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/421//testReport/
Findbugs warnings:
https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/421//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output:
https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/421//console
This message is automatically generated.
> Don't fill preallocated portion of edits log with 0x00
> ------------------------------------------------------
>
> Key: HDFS-1846
> URL: https://issues.apache.org/jira/browse/HDFS-1846
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: name-node
> Affects Versions: 0.23.0
> Reporter: Aaron T. Myers
> Assignee: Aaron T. Myers
> Attachments: hdfs-1846-perf-analysis.0.patch, hdfs-1846.0.txt,
> hdfs-1846.1.patch
>
>
> HADOOP-2330 added a feature to preallocate space in the local file system for
> the NN transaction log. That change seeks past the current end of the file
> and writes out some data, which on most systems results in the intervening
> data in the file being filled with zeros. Most underlying file systems have
> special handling for sparse files, and don't actually allocate blocks on disk
> for blocks of a file which consist completely of 0x00.
> I've seen cases in the wild where the volume an edits dir is on fills up,
> resulting in a partial final transaction being written out to disk. If you
> examine the bytes of this (now corrupt) edits file, you'll see the partial
> final transaction followed by a lot of zeros, suggesting that the
> preallocation previously succeeded before the volume ran out of space. If we
> fill the preallocated space with something other than zeros, we'd likely see
> the failure at preallocation time, rather than transaction-writing time, and
> so cause the NN to crash earlier, without a partial transaction being written
> out.
> I also hypothesize that filling the preallocated space in the edits log with
> something other than 0x00 will result in a performance improvement in NN
> throughput. I haven't tested this yet, but I intend to as part of this JIRA.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
