> 2013-07-12 11:04:26,002 WARN > org.apache.hadoop.hdfs.server.namenode.NameNodeResourceChecker: Space > available on volume 'null' is 0, which is below the configured reserved > amount 104857600
This is interesting. Its calling your volume "null", which may be more of a superficial bug. What is your dfs.namenode.name.dir set to? From /tmp/hadoop-dsadm/dfs/name I'd expect you haven't set it up and /tmp is being used off of the out-of-box defaults. Could you try to set it to a specific directory thats not on /tmp? On Mon, Jul 15, 2013 at 2:43 PM, Krishna Kishore Bonagiri <write2kish...@gmail.com> wrote: > I don't have it in my hdfs-site.xml, in which case probably the default > value is taken.. > > > On Mon, Jul 15, 2013 at 2:29 PM, Azuryy Yu <azury...@gmail.com> wrote: >> >> please check dfs.datanode.du.reserved in the hdfs-site.xml >> >> On Jul 15, 2013 4:30 PM, "Aditya exalter" <adityaexal...@gmail.com> wrote: >>> >>> Hi Krishna, >>> >>> Can you please send screenshots of namenode web UI. >>> >>> Thanks Aditya. >>> >>> >>> On Mon, Jul 15, 2013 at 1:54 PM, Krishna Kishore Bonagiri >>> <write2kish...@gmail.com> wrote: >>>> >>>> I have had enough space on the disk that is used, like around 30 Gigs >>>> >>>> Thanks, >>>> Kishore >>>> >>>> >>>> On Mon, Jul 15, 2013 at 1:30 PM, Venkatarami Netla >>>> <venkatarami.ne...@cloudwick.com> wrote: >>>>> >>>>> Hi, >>>>> pls see the available space for NN storage directory. >>>>> >>>>> Thanks & Regards >>>>> >>>>> Venkat >>>>> >>>>> >>>>> On Mon, Jul 15, 2013 at 12:14 PM, Krishna Kishore Bonagiri >>>>> <write2kish...@gmail.com> wrote: >>>>>> >>>>>> Hi, >>>>>> >>>>>> I am doing no activity on my single node cluster which is using >>>>>> 2.1.0-beta, and still observed that it has gone to safe mode by itself >>>>>> after >>>>>> a while. I was looking at the name node log and see many of these kinds >>>>>> of >>>>>> entries.. Can anything be interpreted from these? >>>>>> >>>>>> 2013-07-12 09:06:11,256 INFO >>>>>> org.apache.hadoop.hdfs.server.namenode.FSEditLog: Starting log segment at >>>>>> 561 >>>>>> 2013-07-12 09:07:11,290 INFO >>>>>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Roll Edit Log from >>>>>> 9.70.137.114 >>>>>> 2013-07-12 09:07:11,290 INFO >>>>>> org.apache.hadoop.hdfs.server.namenode.FSEditLog: Rolling edit logs >>>>>> 2013-07-12 09:07:11,290 INFO >>>>>> org.apache.hadoop.hdfs.server.namenode.FSEditLog: Ending log segment 561 >>>>>> 2013-07-12 09:07:11,291 INFO >>>>>> org.apache.hadoop.hdfs.server.namenode.FSEditLog: Number of >>>>>> transactions: 2 >>>>>> Total time for transactions(ms): 1 Number of transactions batched in >>>>>> Syncs: >>>>>> 0 Number of syncs: 2 SyncTimes(ms): 14 >>>>>> 2013-07-12 09:07:11,292 INFO >>>>>> org.apache.hadoop.hdfs.server.namenode.FSEditLog: Number of >>>>>> transactions: 2 >>>>>> Total time for transactions(ms): 1 Number of transactions batched in >>>>>> Syncs: >>>>>> 0 Number of syncs: 3 SyncTimes(ms): 15 >>>>>> 2013-07-12 09:07:11,293 INFO >>>>>> org.apache.hadoop.hdfs.server.namenode.FileJournalManager: Finalizing >>>>>> edits >>>>>> file >>>>>> /tmp/hadoop-dsadm/dfs/name/current/edits_inprogress_0000000000000000561 >>>>>> -> >>>>>> /tmp/hadoop-dsadm/dfs/name/current/edits_0000000000000000561-0000000000000000562 >>>>>> 2013-07-12 09:07:11,294 INFO >>>>>> org.apache.hadoop.hdfs.server.namenode.FSEditLog: Starting log segment at >>>>>> 563 >>>>>> 2013-07-12 09:08:11,397 INFO >>>>>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Roll Edit Log from >>>>>> 9.70.137.114 >>>>>> 2013-07-12 09:08:11,398 INFO >>>>>> org.apache.hadoop.hdfs.server.namenode.FSEditLog: Rolling edit logs >>>>>> 2013-07-12 09:08:11,398 INFO >>>>>> org.apache.hadoop.hdfs.server.namenode.FSEditLog: Ending log segment 563 >>>>>> 2013-07-12 09:08:11,399 INFO >>>>>> org.apache.hadoop.hdfs.server.namenode.FSEditLog: Number of >>>>>> transactions: 2 >>>>>> Total time for transactions(ms): 2 Number of transactions batched in >>>>>> Syncs: >>>>>> 0 Number of syncs: 2 SyncTimes(ms): 11 >>>>>> 2013-07-12 09:08:11,400 INFO >>>>>> org.apache.hadoop.hdfs.server.namenode.FSEditLog: Number of >>>>>> transactions: 2 >>>>>> Total time for transactions(ms): 2 Number of transactions batched in >>>>>> Syncs: >>>>>> 0 Number of syncs: 3 SyncTimes(ms): 12 >>>>>> 2013-07-12 09:08:11,402 INFO >>>>>> org.apache.hadoop.hdfs.server.namenode.FileJournalManager: Finalizing >>>>>> edits >>>>>> file >>>>>> /tmp/hadoop-dsadm/dfs/name/current/edits_inprogress_0000000000000000563 >>>>>> -> >>>>>> /tmp/hadoop-dsadm/dfs/name/current/edits_0000000000000000563-0000000000000000564 >>>>>> 2013-07-12 09:08:11,402 INFO >>>>>> org.apache.hadoop.hdfs.server.namenode.FSEditLog: Starting log segment at >>>>>> 565 >>>>>> 2013-07-12 09:09:11,440 INFO >>>>>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Roll Edit Log from >>>>>> 9.70.137.114 >>>>>> 2013-07-12 09:09:11,440 INFO >>>>>> org.apache.hadoop.hdfs.server.namenode.FSEditLog: Rolling edit logs >>>>>> 2013-07-12 09:09:11,440 INFO >>>>>> org.apache.hadoop.hdfs.server.namenode.FSEditLog: Ending log segment 565 >>>>>> 2013-07-12 09:09:11,440 INFO >>>>>> org.apache.hadoop.hdfs.server.namenode.FSEditLog: Number of >>>>>> transactions: 2 >>>>>> Total time for transactions(ms): 0 Number of transactions batched in >>>>>> Syncs: >>>>>> 0 Number of syncs: 2 SyncTimes(ms): 13 >>>>>> 2013-07-12 09:09:11,441 INFO >>>>>> org.apache.hadoop.hdfs.server.namenode.FSEditLog: Number of >>>>>> transactions: 2 >>>>>> Total time for transactions(ms): 0 Number of transactions batched in >>>>>> Syncs: >>>>>> 0 Number of syncs: 3 SyncTimes(ms): 13 >>>>>> >>>>>> >>>>>> And after sometime it said: >>>>>> >>>>>> 2013-07-12 11:03:19,799 INFO >>>>>> org.apache.hadoop.hdfs.server.namenode.FSEditLog: Starting log segment at >>>>>> 795 >>>>>> 2013-07-12 11:04:19,826 INFO >>>>>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Roll Edit Log from >>>>>> 9.70.137.114 >>>>>> 2013-07-12 11:04:19,826 INFO >>>>>> org.apache.hadoop.hdfs.server.namenode.FSEditLog: Rolling edit logs >>>>>> 2013-07-12 11:04:19,827 INFO >>>>>> org.apache.hadoop.hdfs.server.namenode.FSEditLog: Ending log segment 795 >>>>>> 2013-07-12 11:04:19,827 INFO >>>>>> org.apache.hadoop.hdfs.server.namenode.FSEditLog: Number of >>>>>> transactions: 2 >>>>>> Total time for transactions(ms): 0 Number of transactions batched in >>>>>> Syncs: >>>>>> 0 Number of syncs: 2 SyncTimes(ms): 12 >>>>>> 2013-07-12 11:04:19,827 INFO >>>>>> org.apache.hadoop.hdfs.server.namenode.FSEditLog: Number of >>>>>> transactions: 2 >>>>>> Total time for transactions(ms): 0 Number of transactions batched in >>>>>> Syncs: >>>>>> 0 Number of syncs: 3 SyncTimes(ms): 12 >>>>>> 2013-07-12 11:04:19,829 INFO >>>>>> org.apache.hadoop.hdfs.server.namenode.FileJournalManager: Finalizing >>>>>> edits >>>>>> file >>>>>> /tmp/hadoop-dsadm/dfs/name/current/edits_inprogress_0000000000000000795 >>>>>> -> >>>>>> /tmp/hadoop-dsadm/dfs/name/current/edits_0000000000000000795-0000000000000000796 >>>>>> 2013-07-12 11:04:19,829 INFO >>>>>> org.apache.hadoop.hdfs.server.namenode.FSEditLog: Starting log segment at >>>>>> 797 >>>>>> 2013-07-12 11:04:26,002 WARN >>>>>> org.apache.hadoop.hdfs.server.namenode.NameNodeResourceChecker: Space >>>>>> available on volume 'null' is 0, which is below the configured reserved >>>>>> amount 104857600 >>>>>> 2013-07-12 11:04:26,003 WARN >>>>>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: NameNode low on >>>>>> available disk space. Entering safe mode. >>>>>> 2013-07-12 11:04:26,004 INFO org.apache.hadoop.hdfs.StateChange: >>>>>> STATE* Safe mode is ON. >>>>>> Resources are low on NN. Please add or free up more resources then >>>>>> turn off safe mode manually. NOTE: If you turn off safe mode before >>>>>> adding >>>>>> resources, the NN will immediately return to safe mode.. >>>>>> 2013-07-12 11:04:31,004 WARN >>>>>> org.apache.hadoop.hdfs.server.namenode.NameNodeResourceChecker: Space >>>>>> available on volume 'null' is 0, which is below the configured reserved >>>>>> amount 104857600 >>>>>> 2013-07-12 11:04:31,005 WARN >>>>>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: NameNode low on >>>>>> available disk space. Already in safe mode. >>>>>> 2013-07-12 11:04:31,005 INFO org.apache.hadoop.hdfs.StateChange: >>>>>> STATE* Safe mode is ONResources are low on NN. Please add or free up more >>>>>> resources then turn off safe mode manually. NOTE: If you turn off safe >>>>>> mode before adding resources, the NN will immediately return to safe >>>>>> mode.. >>>>>> 2013-07-12 11:04:36,005 WARN >>>>>> org.apache.hadoop.hdfs.server.namenode.NameNodeResourceChecker: Space >>>>>> available on volume 'null' is 0, which is below the configured reserved >>>>>> amount 104857600 >>>>>> >>>>>> >>>>>> Thanks, >>>>>> Kishore >>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> N Venkata Rami Reddy >>>>> Hadoop Admin >>>>> Cloudwick Technologies >>>> >>>> >>> > -- Harsh J
