[jira] Created: (HDFS-1590) Decommissioning never ends when node to decommission has blocks that are under-replicated and cannot be replicated to the expected level of replication
Decommissioning never ends when node to decommission has blocks that are under-replicated and cannot be replicated to the expected level of replication --- Key: HDFS-1590 URL: https://issues.apache.org/jira/browse/HDFS-1590 Project: Hadoop HDFS Issue Type: Bug Components: name-node Affects Versions: 0.20.2 Environment: Linux Reporter: Mathias Herberts Priority: Minor On a test cluster with 4 DNs and a default repl level of 3, I recently attempted to decommission one of the DNs. Right after the modification of the dfs.hosts.exclude file and the 'dfsadmin -refreshNodes', I could see the blocks being replicated to other nodes. After a while, the replication stopped but the node was not marked as decommissioned. When running an 'fsck -files -blocks -locations' I saw that all files had a replication of 4 (which is logical given there are 4 DNs), but some of the files had an expected replication set to 10 (those were job.jar files from M/R jobs). I ran 'fs -setrep 3' on those files and shortly after the namenode reported the DN as decommissioned. Shouldn't this case be checked by the NameNode when decommissioning a node? I.e considere a node decommissioned if either one of the following is true for each block on the node being decommissioned: 1. It is replicated more than the expected replication level. 2. It is replicated as much as possible given the available nodes, even though it is less replicated than expected. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-1148) Convert FSDataset to ReadWriteLock
[ https://issues.apache.org/jira/browse/HDFS-1148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12984851#action_12984851 ] dhruba borthakur commented on HDFS-1148: hi todd, if you have this patch, may we have a look? thanks. Convert FSDataset to ReadWriteLock -- Key: HDFS-1148 URL: https://issues.apache.org/jira/browse/HDFS-1148 Project: Hadoop HDFS Issue Type: Improvement Components: data-node Reporter: Todd Lipcon Assignee: Todd Lipcon In benchmarking HDFS-941 I noticed that for the random read workload, the FSDataset lock is highly contended. After converting it to a ReentrantReadWriteLock, I saw a ~25% improvement on both latency and ops/second. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (HDFS-1591) Fix javac, javadoc, findbugs warnings
Fix javac, javadoc, findbugs warnings - Key: HDFS-1591 URL: https://issues.apache.org/jira/browse/HDFS-1591 Project: Hadoop HDFS Issue Type: Bug Reporter: Po Cheung Assignee: Po Cheung Fix For: 0.22.0 Split from HADOOP-6642 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-1557) Separate Storage from FSImage
[ https://issues.apache.org/jira/browse/HDFS-1557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12984855#action_12984855 ] Jitendra Nath Pandey commented on HDFS-1557: Couple of things I missed before FSEditLog.java - In errorOccurred method, the LOG.error is being printed for all editStreams. - In formatOccurred, getStorageFile is static, but is being accessed non-statically. Separate Storage from FSImage - Key: HDFS-1557 URL: https://issues.apache.org/jira/browse/HDFS-1557 Project: Hadoop HDFS Issue Type: Sub-task Components: name-node Affects Versions: 0.21.0 Reporter: Ivan Kelly Assignee: Ivan Kelly Fix For: 0.23.0 Attachments: HDFS-1557-branch-0.22.diff, HDFS-1557-branch-0.22.diff, HDFS-1557-trunk.diff, HDFS-1557-trunk.diff, HDFS-1557-trunk.diff, HDFS-1557.diff, HDFS-1557.diff, HDFS-1557.diff FSImage currently derives from Storage and FSEditLog has to call methods directly on FSImage to access the filesystem. This JIRA is to separate the Storage class out into NNStorage so that FSEditLog is less dependent on FSImage. From this point, the other parts of the circular dependency should be easy to fix. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-1148) Convert FSDataset to ReadWriteLock
[ https://issues.apache.org/jira/browse/HDFS-1148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon updated HDFS-1148: -- Attachment: hdfs-1148-old.txt Here's the old patch - it's against some old branch of mine, probably will need rework either for branch-20-append or for trunk Convert FSDataset to ReadWriteLock -- Key: HDFS-1148 URL: https://issues.apache.org/jira/browse/HDFS-1148 Project: Hadoop HDFS Issue Type: Improvement Components: data-node Reporter: Todd Lipcon Assignee: Todd Lipcon Attachments: hdfs-1148-old.txt In benchmarking HDFS-941 I noticed that for the random read workload, the FSDataset lock is highly contended. After converting it to a ReentrantReadWriteLock, I saw a ~25% improvement on both latency and ops/second. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-1591) Fix javac, javadoc, findbugs warnings
[ https://issues.apache.org/jira/browse/HDFS-1591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Po Cheung updated HDFS-1591: Attachment: hdfs-1591-trunk.patch This patch is dependent on the common-6642-hdfs.patch from HADOOP-6642. Fix javac, javadoc, findbugs warnings - Key: HDFS-1591 URL: https://issues.apache.org/jira/browse/HDFS-1591 Project: Hadoop HDFS Issue Type: Bug Reporter: Po Cheung Assignee: Po Cheung Fix For: 0.22.0 Attachments: hdfs-1591-trunk.patch Split from HADOOP-6642 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-1586) Add InterfaceAudience annotation to MiniDFSCluster
[ https://issues.apache.org/jira/browse/HDFS-1586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12984915#action_12984915 ] Suresh Srinivas commented on HDFS-1586: --- This patch just adds annotations to MiniDFSCluster. I will commit it without adding unit tests or hudson validation. Add InterfaceAudience annotation to MiniDFSCluster -- Key: HDFS-1586 URL: https://issues.apache.org/jira/browse/HDFS-1586 Project: Hadoop HDFS Issue Type: Improvement Reporter: Suresh Srinivas Assignee: Suresh Srinivas Attachments: HDFS-1586.1.patch, HDFS-1586.patch MiniDFSCluster is used both by hdfs and mapreduce. Annotation needs to be added to this class to reflect this. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HDFS-1586) Add InterfaceAudience annotation to MiniDFSCluster
[ https://issues.apache.org/jira/browse/HDFS-1586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas resolved HDFS-1586. --- Resolution: Fixed I committed the patch. Add InterfaceAudience annotation to MiniDFSCluster -- Key: HDFS-1586 URL: https://issues.apache.org/jira/browse/HDFS-1586 Project: Hadoop HDFS Issue Type: Improvement Reporter: Suresh Srinivas Assignee: Suresh Srinivas Attachments: HDFS-1586.1.patch, HDFS-1586.patch MiniDFSCluster is used both by hdfs and mapreduce. Annotation needs to be added to this class to reflect this. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-1588) Add dfs.hosts.exclude to DFSConfigKeys and use constant in stead of hardcoded string
[ https://issues.apache.org/jira/browse/HDFS-1588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12984922#action_12984922 ] Suresh Srinivas commented on HDFS-1588: --- +1 for the patch. Add dfs.hosts.exclude to DFSConfigKeys and use constant in stead of hardcoded string Key: HDFS-1588 URL: https://issues.apache.org/jira/browse/HDFS-1588 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 0.23.0 Reporter: Erik Steffl Assignee: Erik Steffl Fix For: 0.23.0 Attachments: HDFS-1588-0.23.patch -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-1588) Add dfs.hosts.exclude to DFSConfigKeys and use constant in stead of hardcoded string
[ https://issues.apache.org/jira/browse/HDFS-1588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-1588: -- Issue Type: Improvement (was: Bug) Add dfs.hosts.exclude to DFSConfigKeys and use constant in stead of hardcoded string Key: HDFS-1588 URL: https://issues.apache.org/jira/browse/HDFS-1588 Project: Hadoop HDFS Issue Type: Improvement Affects Versions: 0.23.0 Reporter: Erik Steffl Assignee: Erik Steffl Fix For: 0.23.0 Attachments: HDFS-1588-0.23.patch -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HDFS-1588) Add dfs.hosts.exclude to DFSConfigKeys and use constant in stead of hardcoded string
[ https://issues.apache.org/jira/browse/HDFS-1588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas resolved HDFS-1588. --- Resolution: Fixed Hadoop Flags: [Reviewed] I committed the patch. Thank you Erik. Add dfs.hosts.exclude to DFSConfigKeys and use constant in stead of hardcoded string Key: HDFS-1588 URL: https://issues.apache.org/jira/browse/HDFS-1588 Project: Hadoop HDFS Issue Type: Improvement Affects Versions: 0.23.0 Reporter: Erik Steffl Assignee: Erik Steffl Fix For: 0.23.0 Attachments: HDFS-1588-0.23.patch -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-1561) BackupNode listens on default host
[ https://issues.apache.org/jira/browse/HDFS-1561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12984955#action_12984955 ] Tom White commented on HDFS-1561: - Looks like MR test compilation is failing now. I filed MAPREDUCE-2282. BackupNode listens on default host -- Key: HDFS-1561 URL: https://issues.apache.org/jira/browse/HDFS-1561 Project: Hadoop HDFS Issue Type: Bug Components: name-node Affects Versions: 0.21.0 Reporter: Konstantin Shvachko Assignee: Konstantin Shvachko Priority: Blocker Fix For: 0.22.0 Attachments: BNAddress.patch, BNAddress.patch Currently BackupNode uses DNS to find its default host name, and then starts RPC server listening on that address ignoring the address specified in the configuration. Therefore, there is no way to start BackupNode on a particular ip or host address. BackupNode should use the address specified in the configuration instead. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-1561) BackupNode listens on default host
[ https://issues.apache.org/jira/browse/HDFS-1561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12984960#action_12984960 ] Tom White commented on HDFS-1561: - I looked at trying to fix this by replacing references to NAME_NODE_HTTP and NAME_NODE_HTTP_HOST with THIS_HOST (all in TestHDFSServerPorts), but THIS_HOST is not accessible from TestMRServerPorts. So should we revert this while we come up with a fix? Thanks. BackupNode listens on default host -- Key: HDFS-1561 URL: https://issues.apache.org/jira/browse/HDFS-1561 Project: Hadoop HDFS Issue Type: Bug Components: name-node Affects Versions: 0.21.0 Reporter: Konstantin Shvachko Assignee: Konstantin Shvachko Priority: Blocker Fix For: 0.22.0 Attachments: BNAddress.patch, BNAddress.patch Currently BackupNode uses DNS to find its default host name, and then starts RPC server listening on that address ignoring the address specified in the configuration. Therefore, there is no way to start BackupNode on a particular ip or host address. BackupNode should use the address specified in the configuration instead. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-1448) Create multi-format parser for edits logs file, support binary and XML formats initially
[ https://issues.apache.org/jira/browse/HDFS-1448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo (Nicholas), SZE updated HDFS-1448: - Hadoop Flags: [Reviewed] +1 patch looks good. Create multi-format parser for edits logs file, support binary and XML formats initially Key: HDFS-1448 URL: https://issues.apache.org/jira/browse/HDFS-1448 Project: Hadoop HDFS Issue Type: New Feature Components: tools Affects Versions: 0.22.0 Reporter: Erik Steffl Assignee: Erik Steffl Fix For: 0.23.0 Attachments: editsStored, HDFS-1448-0.22-1.patch, HDFS-1448-0.22-2.patch, HDFS-1448-0.22-3.patch, HDFS-1448-0.22-4.patch, HDFS-1448-0.22-5.patch, HDFS-1448-0.22.patch, Viewer hierarchy.pdf Create multi-format parser for edits logs file, support binary and XML formats initially. Parsing should work from any supported format to any other supported format (e.g. from binary to XML and from XML to binary). The binary format is the format used by FSEditLog class to read/write edits file. Primary reason to develop this tool is to help with troubleshooting, the binary format is hard to read and edit (for human troubleshooters). Longer term it could be used to clean up and minimize parsers for fsimage and edits files. Edits parser OfflineEditsViewer is written in a very similar fashion to OfflineImageViewer. Next step would be to merge OfflineImageViewer and OfflineEditsViewer and use the result in both FSImage and FSEditLog. This is subject to change, specifically depending on adoption of avro (which would completely change how objects are serialized as well as provide ways to convert files to different formats). -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-1561) BackupNode listens on default host
[ https://issues.apache.org/jira/browse/HDFS-1561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12984962#action_12984962 ] Konstantin Shvachko commented on HDFS-1561: --- Yes, go ahead revert/reopen, I'll check on mapreduce. BackupNode listens on default host -- Key: HDFS-1561 URL: https://issues.apache.org/jira/browse/HDFS-1561 Project: Hadoop HDFS Issue Type: Bug Components: name-node Affects Versions: 0.21.0 Reporter: Konstantin Shvachko Assignee: Konstantin Shvachko Priority: Blocker Fix For: 0.22.0 Attachments: BNAddress.patch, BNAddress.patch Currently BackupNode uses DNS to find its default host name, and then starts RPC server listening on that address ignoring the address specified in the configuration. Therefore, there is no way to start BackupNode on a particular ip or host address. BackupNode should use the address specified in the configuration instead. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HDFS-1448) Create multi-format parser for edits logs file, support binary and XML formats initially
[ https://issues.apache.org/jira/browse/HDFS-1448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo (Nicholas), SZE resolved HDFS-1448. -- Resolution: Fixed I have committed it. Thanks, Erik! Erik, please add Release Note for the new feature. Create multi-format parser for edits logs file, support binary and XML formats initially Key: HDFS-1448 URL: https://issues.apache.org/jira/browse/HDFS-1448 Project: Hadoop HDFS Issue Type: New Feature Components: tools Affects Versions: 0.22.0 Reporter: Erik Steffl Assignee: Erik Steffl Fix For: 0.23.0 Attachments: editsStored, HDFS-1448-0.22-1.patch, HDFS-1448-0.22-2.patch, HDFS-1448-0.22-3.patch, HDFS-1448-0.22-4.patch, HDFS-1448-0.22-5.patch, HDFS-1448-0.22.patch, Viewer hierarchy.pdf Create multi-format parser for edits logs file, support binary and XML formats initially. Parsing should work from any supported format to any other supported format (e.g. from binary to XML and from XML to binary). The binary format is the format used by FSEditLog class to read/write edits file. Primary reason to develop this tool is to help with troubleshooting, the binary format is hard to read and edit (for human troubleshooters). Longer term it could be used to clean up and minimize parsers for fsimage and edits files. Edits parser OfflineEditsViewer is written in a very similar fashion to OfflineImageViewer. Next step would be to merge OfflineImageViewer and OfflineEditsViewer and use the result in both FSImage and FSEditLog. This is subject to change, specifically depending on adoption of avro (which would completely change how objects are serialized as well as provide ways to convert files to different formats). -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-1561) BackupNode listens on default host
[ https://issues.apache.org/jira/browse/HDFS-1561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12984975#action_12984975 ] Tsz Wo (Nicholas), SZE commented on HDFS-1561: -- Yes, go ahead revert/reopen, I'll check on mapreduce. Unfortunately, it seems the HDFS test codes, which are supposed to be unstable, are used in other projects. Not a big deal. Just fix it in either HDFS or MapReduce but not necessarily reverting this. BackupNode listens on default host -- Key: HDFS-1561 URL: https://issues.apache.org/jira/browse/HDFS-1561 Project: Hadoop HDFS Issue Type: Bug Components: name-node Affects Versions: 0.21.0 Reporter: Konstantin Shvachko Assignee: Konstantin Shvachko Priority: Blocker Fix For: 0.22.0 Attachments: BNAddress.patch, BNAddress.patch Currently BackupNode uses DNS to find its default host name, and then starts RPC server listening on that address ignoring the address specified in the configuration. Therefore, there is no way to start BackupNode on a particular ip or host address. BackupNode should use the address specified in the configuration instead. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-1448) Create multi-format parser for edits logs file, support binary and XML formats initially
[ https://issues.apache.org/jira/browse/HDFS-1448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Steffl updated HDFS-1448: -- Release Note: Offline edits viewer feature adds oev tool to hdfs script. Oev makes it possible to convert edits logs to/from native binary and XML formats. It uses the same framework as Offline image viewer. Example usage: $HADOOP_HOME/bin/hdfs oev -i edits -o output.xml Create multi-format parser for edits logs file, support binary and XML formats initially Key: HDFS-1448 URL: https://issues.apache.org/jira/browse/HDFS-1448 Project: Hadoop HDFS Issue Type: New Feature Components: tools Affects Versions: 0.22.0 Reporter: Erik Steffl Assignee: Erik Steffl Fix For: 0.23.0 Attachments: editsStored, HDFS-1448-0.22-1.patch, HDFS-1448-0.22-2.patch, HDFS-1448-0.22-3.patch, HDFS-1448-0.22-4.patch, HDFS-1448-0.22-5.patch, HDFS-1448-0.22.patch, Viewer hierarchy.pdf Create multi-format parser for edits logs file, support binary and XML formats initially. Parsing should work from any supported format to any other supported format (e.g. from binary to XML and from XML to binary). The binary format is the format used by FSEditLog class to read/write edits file. Primary reason to develop this tool is to help with troubleshooting, the binary format is hard to read and edit (for human troubleshooters). Longer term it could be used to clean up and minimize parsers for fsimage and edits files. Edits parser OfflineEditsViewer is written in a very similar fashion to OfflineImageViewer. Next step would be to merge OfflineImageViewer and OfflineEditsViewer and use the result in both FSImage and FSEditLog. This is subject to change, specifically depending on adoption of avro (which would completely change how objects are serialized as well as provide ways to convert files to different formats). -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-1448) Create multi-format parser for edits logs file, support binary and XML formats initially
[ https://issues.apache.org/jira/browse/HDFS-1448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12984984#action_12984984 ] Erik Steffl commented on HDFS-1448: --- Release note added. Create multi-format parser for edits logs file, support binary and XML formats initially Key: HDFS-1448 URL: https://issues.apache.org/jira/browse/HDFS-1448 Project: Hadoop HDFS Issue Type: New Feature Components: tools Affects Versions: 0.22.0 Reporter: Erik Steffl Assignee: Erik Steffl Fix For: 0.23.0 Attachments: editsStored, HDFS-1448-0.22-1.patch, HDFS-1448-0.22-2.patch, HDFS-1448-0.22-3.patch, HDFS-1448-0.22-4.patch, HDFS-1448-0.22-5.patch, HDFS-1448-0.22.patch, Viewer hierarchy.pdf Create multi-format parser for edits logs file, support binary and XML formats initially. Parsing should work from any supported format to any other supported format (e.g. from binary to XML and from XML to binary). The binary format is the format used by FSEditLog class to read/write edits file. Primary reason to develop this tool is to help with troubleshooting, the binary format is hard to read and edit (for human troubleshooters). Longer term it could be used to clean up and minimize parsers for fsimage and edits files. Edits parser OfflineEditsViewer is written in a very similar fashion to OfflineImageViewer. Next step would be to merge OfflineImageViewer and OfflineEditsViewer and use the result in both FSImage and FSEditLog. This is subject to change, specifically depending on adoption of avro (which would completely change how objects are serialized as well as provide ways to convert files to different formats). -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-1585) HDFS-1547 broke MR build
[ https://issues.apache.org/jira/browse/HDFS-1585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12984990#action_12984990 ] Tsz Wo (Nicholas), SZE commented on HDFS-1585: -- Todd, thanks for fixing this. HDFS-1547 broke MR build Key: HDFS-1585 URL: https://issues.apache.org/jira/browse/HDFS-1585 Project: Hadoop HDFS Issue Type: Bug Components: test Affects Versions: 0.23.0 Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Blocker Fix For: 0.23.0 Attachments: hdfs-1585.txt Added a parameter to startDatanodes without maintaining old API -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HDFS-1566) Test that covers full partition
[ https://issues.apache.org/jira/browse/HDFS-1566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Boudnik reassigned HDFS-1566: Assignee: Konstantin Boudnik Test that covers full partition - Key: HDFS-1566 URL: https://issues.apache.org/jira/browse/HDFS-1566 Project: Hadoop HDFS Issue Type: Test Components: name-node Affects Versions: 0.20.2 Reporter: Eli Collins Assignee: Konstantin Boudnik Fix For: 0.23.0 We've seen the following bug, hdfs needs a test to reproduce this: * /var filled up * 2NN failed checkpoint due to no space left on device * NN log hit end of disk * NN seems to have exited on the spot, mid-log-message * NN edits are left corrupted ** Half of a rename made it into the log ** valid data appears to end on a sector boundary ** this is true across all of the edit dirs -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-1566) Test that covers full partition
[ https://issues.apache.org/jira/browse/HDFS-1566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12985016#action_12985016 ] Konstantin Boudnik commented on HDFS-1566: -- Is there any logs/traces available from this incident? Would be mighty helpful. Test that covers full partition - Key: HDFS-1566 URL: https://issues.apache.org/jira/browse/HDFS-1566 Project: Hadoop HDFS Issue Type: Test Components: name-node Affects Versions: 0.20.2 Reporter: Eli Collins Assignee: Konstantin Boudnik Fix For: 0.23.0 We've seen the following bug, hdfs needs a test to reproduce this: * /var filled up * 2NN failed checkpoint due to no space left on device * NN log hit end of disk * NN seems to have exited on the spot, mid-log-message * NN edits are left corrupted ** Half of a rename made it into the log ** valid data appears to end on a sector boundary ** this is true across all of the edit dirs -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
