[ https://issues.apache.org/jira/browse/HDFS-1104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hairong Kuang updated HDFS-1104: -------------------------------- Status: Patch Available (was: Open) > Fsck triggers full GC on NameNode > --------------------------------- > > Key: HDFS-1104 > URL: https://issues.apache.org/jira/browse/HDFS-1104 > Project: Hadoop HDFS > Issue Type: Bug > Components: name-node > Affects Versions: 0.21.0 > Reporter: Hairong Kuang > Assignee: Hairong Kuang > Fix For: 0.22.0 > > Attachments: fsckATime.patch, fsckATime1.patch, fsckATime2.patch > > > A NameNode at one of our clusters fell into full GC while fsck was performed. > Digging into the problem shows that it is caused by how NameNode handles the > access time of a file. > Fsck calls open on every file in the checked directory to get the file's > block locations. Each open changes the file's access time and then leads to > writing a transaction entry to the edit log. The current code optimizes open > so that it returns without issuing synchronizing the edit log to the disk. It > happened that in our cluster no other jobs were running while fsck was > performed. No edit log sync was ever called. So all open transactions were > kept in memory. When the edit log buffer got full, it automatically doubled > its space by allocating a new buffer. Full GC happened when no contiguous > space were found when allocating a new bigger buffer. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.