[jira] [Updated] (HDFS-2964) No notice in any logs if dfs.datanode.du.reserved is greater than available disk space
[ https://issues.apache.org/jira/browse/HDFS-2964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] madhukara phatak updated HDFS-2964: --- Status: Patch Available (was: Open) > No notice in any logs if dfs.datanode.du.reserved is greater than available > disk space > -- > > Key: HDFS-2964 > URL: https://issues.apache.org/jira/browse/HDFS-2964 > Project: Hadoop HDFS > Issue Type: Bug > Components: data-node, name-node >Affects Versions: 0.20.2 >Reporter: Robert J Berger >Priority: Minor > Attachments: HDFS-2964-1.patch, HDFS-2964.patch > > > We spent a long time tracking down why a test hdfs cluster seemed to be > running fine, but would not allow the mapred system to come up complaining > that "could only be replicated to 0 nodes, instead of 1". > There were no namenode or datanode errors in any of the logs. hadoop fsck > said everything was good. At first glance dfsadmin -report looked good. It > wasn't until I realized that there was 0 Capacity available that we poked > around and found > https://groups.google.com/a/cloudera.org/group/scm-users/msg/a4252d6623adbc2d > which mentioned that the "reserverd space" might be greater than the disk > space available. And we did find that our dfs.datanode.du.reserved was indeed > higher than our actual since we were only testing a small cluster. > It seems that there should be some warning or error in the logs that say that. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2964) No notice in any logs if dfs.datanode.du.reserved is greater than available disk space
[ https://issues.apache.org/jira/browse/HDFS-2964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] madhukara phatak updated HDFS-2964: --- Attachment: HDFS-2964-1.patch Fixed the patch file to have only changed code and converted new lines into UNIX > No notice in any logs if dfs.datanode.du.reserved is greater than available > disk space > -- > > Key: HDFS-2964 > URL: https://issues.apache.org/jira/browse/HDFS-2964 > Project: Hadoop HDFS > Issue Type: Bug > Components: data-node, name-node >Affects Versions: 0.20.2 >Reporter: Robert J Berger >Priority: Minor > Attachments: HDFS-2964-1.patch, HDFS-2964.patch > > > We spent a long time tracking down why a test hdfs cluster seemed to be > running fine, but would not allow the mapred system to come up complaining > that "could only be replicated to 0 nodes, instead of 1". > There were no namenode or datanode errors in any of the logs. hadoop fsck > said everything was good. At first glance dfsadmin -report looked good. It > wasn't until I realized that there was 0 Capacity available that we poked > around and found > https://groups.google.com/a/cloudera.org/group/scm-users/msg/a4252d6623adbc2d > which mentioned that the "reserverd space" might be greater than the disk > space available. And we did find that our dfs.datanode.du.reserved was indeed > higher than our actual since we were only testing a small cluster. > It seems that there should be some warning or error in the logs that say that. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2964) No notice in any logs if dfs.datanode.du.reserved is greater than available disk space
[ https://issues.apache.org/jira/browse/HDFS-2964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] madhukara phatak updated HDFS-2964: --- Attachment: HDFS-2964.patch Added a warning message in FsVolumeImpl.java where it reads the latest capacity. I m creating the patch first time so not sure its the correct place to add the warning message. > No notice in any logs if dfs.datanode.du.reserved is greater than available > disk space > -- > > Key: HDFS-2964 > URL: https://issues.apache.org/jira/browse/HDFS-2964 > Project: Hadoop HDFS > Issue Type: Bug > Components: data-node, name-node >Affects Versions: 0.20.2 >Reporter: Robert J Berger >Priority: Minor > Attachments: HDFS-2964.patch > > > We spent a long time tracking down why a test hdfs cluster seemed to be > running fine, but would not allow the mapred system to come up complaining > that "could only be replicated to 0 nodes, instead of 1". > There were no namenode or datanode errors in any of the logs. hadoop fsck > said everything was good. At first glance dfsadmin -report looked good. It > wasn't until I realized that there was 0 Capacity available that we poked > around and found > https://groups.google.com/a/cloudera.org/group/scm-users/msg/a4252d6623adbc2d > which mentioned that the "reserverd space" might be greater than the disk > space available. And we did find that our dfs.datanode.du.reserved was indeed > higher than our actual since we were only testing a small cluster. > It seems that there should be some warning or error in the logs that say that. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
