[jira] [Commented] (HDFS-2704) NameNodeResouceChecker#checkAvailableResources should check for inodes
[ https://issues.apache.org/jira/browse/HDFS-2704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13254700#comment-13254700 ] Robert Joseph Evans commented on HDFS-2704: --- Good point Aaron :) NameNodeResouceChecker#checkAvailableResources should check for inodes -- Key: HDFS-2704 URL: https://issues.apache.org/jira/browse/HDFS-2704 Project: Hadoop HDFS Issue Type: Improvement Components: name-node Affects Versions: 0.24.0 Reporter: Eli Collins Assignee: SreeHari NameNodeResouceChecker#checkAvailableResources currently just checks for free space. However inodes are also a file system resource that needs to be available (you can run out of inodes but have free space). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2704) NameNodeResouceChecker#checkAvailableResources should check for inodes
[ https://issues.apache.org/jira/browse/HDFS-2704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13253392#comment-13253392 ] Robert Joseph Evans commented on HDFS-2704: --- I personally would like to see it go into safemode, just because you never know what in the system will start to have problems when you run out of inodes, or any other resource for that matter. I would hope most standard things would fail gracefully, but you never know and having a corrupted edit log because we tried to roll them when there were no inodes available is a very bad thing. NameNodeResouceChecker#checkAvailableResources should check for inodes -- Key: HDFS-2704 URL: https://issues.apache.org/jira/browse/HDFS-2704 Project: Hadoop HDFS Issue Type: Improvement Components: name-node Affects Versions: 0.24.0 Reporter: Eli Collins Assignee: SreeHari NameNodeResouceChecker#checkAvailableResources currently just checks for free space. However inodes are also a file system resource that needs to be available (you can run out of inodes but have free space). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2704) NameNodeResouceChecker#checkAvailableResources should check for inodes
[ https://issues.apache.org/jira/browse/HDFS-2704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13253501#comment-13253501 ] Harsh J commented on HDFS-2704: --- NameNode also does create local image/edits files upon every checkpointing and such - so the inode monitoring is surely relevant. NameNodeResouceChecker#checkAvailableResources should check for inodes -- Key: HDFS-2704 URL: https://issues.apache.org/jira/browse/HDFS-2704 Project: Hadoop HDFS Issue Type: Improvement Components: name-node Affects Versions: 0.24.0 Reporter: Eli Collins Assignee: SreeHari NameNodeResouceChecker#checkAvailableResources currently just checks for free space. However inodes are also a file system resource that needs to be available (you can run out of inodes but have free space). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2704) NameNodeResouceChecker#checkAvailableResources should check for inodes
[ https://issues.apache.org/jira/browse/HDFS-2704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13253742#comment-13253742 ] Aaron T. Myers commented on HDFS-2704: -- bq. having a corrupted edit log because we tried to roll them when there were no inodes available is a very bad thing. Certainly true, though it seems unlikely that this would result in a truly corrupt edit log that the NN couldn't restart from, since running out of inodes would presumably result in no file being created at all, which isn't an invalid edit log. bq. NameNode also does create local image/edits files upon every checkpointing and such - so the inode monitoring is surely relevant. +1 NameNodeResouceChecker#checkAvailableResources should check for inodes -- Key: HDFS-2704 URL: https://issues.apache.org/jira/browse/HDFS-2704 Project: Hadoop HDFS Issue Type: Improvement Components: name-node Affects Versions: 0.24.0 Reporter: Eli Collins Assignee: SreeHari NameNodeResouceChecker#checkAvailableResources currently just checks for free space. However inodes are also a file system resource that needs to be available (you can run out of inodes but have free space). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2704) NameNodeResouceChecker#checkAvailableResources should check for inodes
[ https://issues.apache.org/jira/browse/HDFS-2704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13253149#comment-13253149 ] SreeHari commented on HDFS-2704: Is a low number of fs inodes a problem for namenode ? Since namenode doesn't create new files in the local filesystem (unlike datanode) while it is running , will it be fine if we put nn into safe mode when system run out of inodes ? Or are we talking about just logging a warning ? NameNodeResouceChecker#checkAvailableResources should check for inodes -- Key: HDFS-2704 URL: https://issues.apache.org/jira/browse/HDFS-2704 Project: Hadoop HDFS Issue Type: Improvement Components: name-node Affects Versions: 0.24.0 Reporter: Eli Collins Assignee: SreeHari NameNodeResouceChecker#checkAvailableResources currently just checks for free space. However inodes are also a file system resource that needs to be available (you can run out of inodes but have free space). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2704) NameNodeResouceChecker#checkAvailableResources should check for inodes
[ https://issues.apache.org/jira/browse/HDFS-2704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13172326#comment-13172326 ] Robert Joseph Evans commented on HDFS-2704: --- That is a great point. I would suggest that we output a warning/error the first time this happens for each disk and possibly include a metric for it. For all file systems that I know of that have a static number of inodes the ratio of inodes to disk space is configurable and even if it requires reformatting the drive, knowing that it is a problem will allow operations to possibly adjust that ratio appropriately. NameNodeResouceChecker#checkAvailableResources should check for inodes -- Key: HDFS-2704 URL: https://issues.apache.org/jira/browse/HDFS-2704 Project: Hadoop HDFS Issue Type: Improvement Components: name-node Affects Versions: 0.24.0 Reporter: Eli Collins Assignee: SreeHari NameNodeResouceChecker#checkAvailableResources currently just checks for free space. However inodes are also a file system resource that needs to be available (you can run out of inodes but have free space). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2704) NameNodeResouceChecker#checkAvailableResources should check for inodes
[ https://issues.apache.org/jira/browse/HDFS-2704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13172046#comment-13172046 ] SreeHari commented on HDFS-2704: Will give a patch for this . NameNodeResouceChecker#checkAvailableResources should check for inodes -- Key: HDFS-2704 URL: https://issues.apache.org/jira/browse/HDFS-2704 Project: Hadoop HDFS Issue Type: Improvement Components: name-node Affects Versions: 0.24.0 Reporter: Eli Collins NameNodeResouceChecker#checkAvailableResources currently just checks for free space. However inodes are also a file system resource that needs to be available (you can run out of inodes but have free space). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
