[ https://issues.apache.org/jira/browse/HDFS-8964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14729882#comment-14729882 ]
Hudson commented on HDFS-8964: ------------------------------ SUCCESS: Integrated in Hadoop-Hdfs-trunk #2269 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/2269/]) HDFS-8964. When validating the edit log, do not read at or beyond the file offset that is being written (Zhe Zhang via Colin P. McCabe) (cmccabe: rev 53c38cc89ab979ec47557dcfa7affbad20578c0a) * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLog.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/qjournal/server/Journal.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSEditLogLoader.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/SecondaryNameNode.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java * hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogTestUtil.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/EditLogFileInputStream.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestCheckPointForSecurityTokens.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogLoader.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FileJournalManager.java > When validating the edit log, do not read at or beyond the file offset that > is being written > -------------------------------------------------------------------------------------------- > > Key: HDFS-8964 > URL: https://issues.apache.org/jira/browse/HDFS-8964 > Project: Hadoop HDFS > Issue Type: Bug > Components: journal-node, namenode > Affects Versions: 2.7.1 > Reporter: Zhe Zhang > Assignee: Zhe Zhang > Fix For: 2.8.0 > > Attachments: HDFS-8964.00.patch, HDFS-8964.01.patch, > HDFS-8964.02.patch, HDFS-8964.03.patch, HDFS-8964.04.patch, > HDFS-8964.05.patch, HDFS-8964.06.patch > > > NN/JN validates in-progress edit log files in multiple scenarios, via > {{EditLogFile#validateLog}}. The method scans through the edit log file to > find the last transaction ID. > However, an in-progress edit log file could be actively written to, which > creates a race condition and causes incorrect data to be read (and later we > attempt to interpret the data as ops). This causes problems for INotify, > which reads edit log entries while the edit log is still being written. > Currently {{validateLog}} is used in 3 places: > # NN {{getEditsFromTxid}} > # JN {{getEditLogManifest}} > # NN/JN {{recoverUnfinalizedSegments}} > In the first two scenarios we should provide a maximum TxId to validate in > the in-progress file. The 3rd scenario won't cause a race condition because > only non-current in-progress edit log files are validated. > {{validateLog}} is actually only used with in-progress files, and could use a > better name and Javadoc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)