Filesystem is considered corrupt if there are any missing blocks. do you see 
MISSING in your output? and also we see missing blocks for files not closed 
yet. When u stopped MR cluster where there any jobs running? 




On May 15, 2008, at 12:15 PM, C G <[EMAIL PROTECTED]> wrote:

Earlier this week I wrote about a master node crash and our efforts to recover 
from the crash.  We recovered from the crash and all systems are normal.  
However, I have a concern about what fsck is reporting and what it really means 
for a filesystem to be marked "corrupt."

 With the mapred engine shut down, I ran fsck / -files -blocks -locations to 
inspect the file system.  The output looks clean with the exception of this at 
the end of the output:

 Status: CORRUPT
Total size:    5113667836544 B
Total blocks:  1070996 (avg. block size 4774684 B)
Total dirs:    50012
Total files:   1027089
Over-replicated blocks:        0 (0.0 %)
Under-replicated blocks:       0 (0.0 %)
Target replication factor:     3
Real replication factor:       3.0

The filesystem under path '/' is CORRUPT

 In reviewing the fsck output, there are no obvious errors being reported.  I 
see tons of output like this:

 /foo/bar/part-00005 3387058, 6308 block(s):  OK
0. 4958936159429948772 len=3387058 repl=3 [10.2.14.5:50010, 10.2.14.20:50010, 
10.2.14.8:50010]

 and the only status ever reported is "OK."

 So this begs the question about what causes HDFS to declare the FS is 
"corrupt" and how do I clear this up?

 The second question, assuming that I can't make the "corrupt" state go away, 
concerns running an upgrade.  If every file in HDFS reports "OK" but the FS 
reports "corrupt", is it safe to undertake an upgrade from 0.15.x to 0.16.4 ?

 Thanks for any help....
 C G



Reply via email to