But my files are still there. As an example -rw-r--r-- 10 User hadmin 6392159 2013-02-05 21:32 /var/lib/hadoop-hdfs/cache/mapred/mapred/staging/User/.staging/job_201302050715_0032/job.jar
Thanks, On Fri, Apr 19, 2013 at 2:51 PM, Bejoy Ks <bejoy.had...@gmail.com> wrote: > This basically happens while running a mapreduce job. When a map reduce > job is triggered the job files are put in hdfs with high replication ( > replication is controlled by - 'mapred.submit.replication' default value > is 10). > > The job files are cleaned up after the job is completed and hence that > could be the reason you are seeing the hdfs file system status as healthy > after running the job. > > > On Fri, Apr 19, 2013 at 1:04 PM, Mohit Vadhera < > project.linux.p...@gmail.com> wrote: > >> its one (1). Output is below. >> >> .......Status: HEALTHY >> Total size: 903709673179 B >> Total dirs: 2906 >> Total files: 11110 >> Total blocks (validated): 20906 (avg. block size 43227287 B) >> Minimally replicated blocks: 20906 (100.0 %) >> Over-replicated blocks: 0 (0.0 %) >> Under-replicated blocks: 248 (1.1862624 %) >> Mis-replicated blocks: 0 (0.0 %) >> Default replication factor: 1 >> Average block replication: 1.0 >> Corrupt blocks: 0 >> Missing replicas: 2232 (9.646469 %) >> Number of data-nodes: 1 >> Number of racks: 1 >> FSCK ended at Fri Apr 19 03:47:04 EDT 2013 in 2224 milliseconds >> >> >> The filesystem under path '/' is HEALTHY >> >> >> >> On Fri, Apr 19, 2013 at 12:28 PM, S, Manoj <mano...@intel.com> wrote: >> >>> It means that some of your data blocks are not replicated as intended. >>> What is the value of “dfs.replication” in your hadoop-site.xml file?**** >>> >>> ** ** >>> >>> Can you paste the output of **** >>> >>> ** ** >>> >>> *bin/hadoop fsck /**** >>> >>> ** ** >>> >>> --**** >>> >>> Manoj**** >>> >>> ** ** >>> >>> *From:* Mohit Vadhera [mailto:project.linux.p...@gmail.com] >>> *Sent:* Friday, April 19, 2013 12:09 PM >>> *To:* <user@hadoop.apache.org> >>> *Subject:* log**** >>> >>> ** ** >>> >>> Can anybody let me know the meaning of the below log plz " Target >>> Replicas is 10 but found 1 replica(s)." ? >>> >>> /var/lib/hadoop-hdfs/cache/mapred/mapred/staging/test_user/.staging/job_201302180313_0623/job.split: >>> Under replicated >>> BP-2091347308-172.20.3.119-1356632249303:blk_6297333561560198850_70720. >>> Target >>> Replicas is 10 but found 1 replica(s). >>> >>> **** >>> >>> Thanks,**** >>> >> >> >
