Re: hdfs fsck -locations
Got it, Harsh, now I see it all. Thank you, Mark On Fri, Jan 24, 2014 at 7:49 PM, Harsh J wrote: > The right syntax is to use "-files -blocks -locations", so it drills > down all the way. You are not "missing" a feature - this has existed > for as long as I've known HDFS. > > In Rodrigo's output, he's seeing a BlockPool ID, which is not > equivalent to a location, but just carries an IP in it for > identification purposes. > > On Sat, Jan 25, 2014 at 3:53 AM, Mark Kerzner > wrote: > > Yes, Rodrigo, > > > > that's what I was looking for. So in my install I somehow don't have it > at > > all. Was asked by my students, so I got the answer. > > > > Mark > > > > > > On Fri, Jan 24, 2014 at 4:00 PM, Nascimento, Rodrigo > > wrote: > >> > >> Mark, > >> > >> there we go ;-) > >> > >> Rodrigo Nascimento > >> Systems Engineer @ Brazil > >> Mobile +55 11 991.873.810 > >> > >> Sent from my iPhone > >> > >> Begin forwarded message: > >> > >> From: "Nascimento, Rodrigo" > >> Date: 24 de janeiro de 2014 15:59:33 BRST > >> To: "user@hadoop.apache.org" > >> Subject: RE: hdfs fsck -locations > >> > >> Hi Mark, > >> > >> It is a sample from my sandbox. Your question is about the part that is > in > >> RED at the output below, right? > >> > >> [root@sandbox ~]# hdfs fsck /user/ambari-qa/passwd -locations > >> Connecting to namenode via http://sandbox.hortonworks.com:50070 > >> FSCK started by root (auth:SIMPLE) from /172.16.13.30 for path > >> /user/ambari-qa/passwd at Fri Jan 24 09:53:43 PST 2014 > >> . > >> /user/ambari-qa/passwd: Under replicated > >> BP-1578958328-10.0.2.15-1382306880516:blk_1073742464_1640. Target > Replicas > >> is 3 but found 1 replica(s). > >> Status: HEALTHY > >> Total size:1708 B > >> Total dirs:0 > >> Total files:1 > >> Total symlinks:0 > >> Total blocks (validated):1 (avg. block size 1708 B) > >> Minimally replicated blocks:1 (100.0 %) > >> Over-replicated blocks:0 (0.0 %) > >> Under-replicated blocks:1 (100.0 %) > >> Mis-replicated blocks:0 (0.0 %) > >> Default replication factor:3 > >> Average block replication:1.0 > >> Corrupt blocks:0 > >> Missing replicas:2 (66.64 %) > >> Number of data-nodes:1 > >> Number of racks:1 > >> FSCK ended at Fri Jan 24 09:53:43 PST 2014 in 1 milliseconds > >> > >> > >> The filesystem under path '/user/ambari-qa/passwd' is HEALTHY > >> [root@sandbox ~]# > >> > >> Rod Nascimento > >> > >> > >> From: Nascimento, Rodrigo [rodrigo.nascime...@netapp.com] > >> Sent: Friday, January 24, 2014 3:34 PM > >> To: user@hadoop.apache.org > >> Subject: RE: hdfs fsck -locations > >> > >> I’m not seeing locations flag yet. > >> > >> > >> > >> Rod Nascimento > >> > >> Systems Engineer @ Brazil > >> > >> > >> > >> People don’t buy WHAT you do. They buy WHY you do it. > >> > >> > >> > >> From: Mark Kerzner [mailto:mark.kerz...@shmsoft.com] > >> Sent: Friday, January 24, 2014 3:16 PM > >> To: Hadoop User > >> Subject: Re: hdfs fsck -locations > >> > >> > >> > >> Sorry, did not copy the full command > >> > >> > >> > >> hdfs fsck /user/mark/data/word_count.csv -locations > >> > >> Connecting to namenode via http://mark-7:50070 > >> > >> FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path > >> /user/mark/data/word_count.csv at Fri Jan 24 11:15:17 CST 2014 > >> > >> .Status: HEALTHY > >> > >> Total size: 7217 B > >> > >> Total dirs: 0 > >> > >> Total files: 1 > >> > >> Total blocks (validated):1 (avg. block size 7217 B) > >> > >> Minimally replicated blocks: 1 (100.0 %) > >> > >> Over-replicated blocks: 0 (0.0 %) > >> > >> Under-replicated blocks:0 (0.0 %) > >> > >> Mis-replicated blocks:0 (0.0 %) >
Re: hdfs fsck -locations
The right syntax is to use "-files -blocks -locations", so it drills down all the way. You are not "missing" a feature - this has existed for as long as I've known HDFS. In Rodrigo's output, he's seeing a BlockPool ID, which is not equivalent to a location, but just carries an IP in it for identification purposes. On Sat, Jan 25, 2014 at 3:53 AM, Mark Kerzner wrote: > Yes, Rodrigo, > > that's what I was looking for. So in my install I somehow don't have it at > all. Was asked by my students, so I got the answer. > > Mark > > > On Fri, Jan 24, 2014 at 4:00 PM, Nascimento, Rodrigo > wrote: >> >> Mark, >> >> there we go ;-) >> >> Rodrigo Nascimento >> Systems Engineer @ Brazil >> Mobile +55 11 991.873.810 >> >> Sent from my iPhone >> >> Begin forwarded message: >> >> From: "Nascimento, Rodrigo" >> Date: 24 de janeiro de 2014 15:59:33 BRST >> To: "user@hadoop.apache.org" >> Subject: RE: hdfs fsck -locations >> >> Hi Mark, >> >> It is a sample from my sandbox. Your question is about the part that is in >> RED at the output below, right? >> >> [root@sandbox ~]# hdfs fsck /user/ambari-qa/passwd -locations >> Connecting to namenode via http://sandbox.hortonworks.com:50070 >> FSCK started by root (auth:SIMPLE) from /172.16.13.30 for path >> /user/ambari-qa/passwd at Fri Jan 24 09:53:43 PST 2014 >> . >> /user/ambari-qa/passwd: Under replicated >> BP-1578958328-10.0.2.15-1382306880516:blk_1073742464_1640. Target Replicas >> is 3 but found 1 replica(s). >> Status: HEALTHY >> Total size:1708 B >> Total dirs:0 >> Total files:1 >> Total symlinks:0 >> Total blocks (validated):1 (avg. block size 1708 B) >> Minimally replicated blocks:1 (100.0 %) >> Over-replicated blocks:0 (0.0 %) >> Under-replicated blocks:1 (100.0 %) >> Mis-replicated blocks:0 (0.0 %) >> Default replication factor:3 >> Average block replication:1.0 >> Corrupt blocks:0 >> Missing replicas:2 (66.64 %) >> Number of data-nodes:1 >> Number of racks: 1 >> FSCK ended at Fri Jan 24 09:53:43 PST 2014 in 1 milliseconds >> >> >> The filesystem under path '/user/ambari-qa/passwd' is HEALTHY >> [root@sandbox ~]# >> >> Rod Nascimento >> >> >> From: Nascimento, Rodrigo [rodrigo.nascime...@netapp.com] >> Sent: Friday, January 24, 2014 3:34 PM >> To: user@hadoop.apache.org >> Subject: RE: hdfs fsck -locations >> >> I’m not seeing locations flag yet. >> >> >> >> Rod Nascimento >> >> Systems Engineer @ Brazil >> >> >> >> People don’t buy WHAT you do. They buy WHY you do it. >> >> >> >> From: Mark Kerzner [mailto:mark.kerz...@shmsoft.com] >> Sent: Friday, January 24, 2014 3:16 PM >> To: Hadoop User >> Subject: Re: hdfs fsck -locations >> >> >> >> Sorry, did not copy the full command >> >> >> >> hdfs fsck /user/mark/data/word_count.csv -locations >> >> Connecting to namenode via http://mark-7:50070 >> >> FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path >> /user/mark/data/word_count.csv at Fri Jan 24 11:15:17 CST 2014 >> >> .Status: HEALTHY >> >> Total size: 7217 B >> >> Total dirs: 0 >> >> Total files: 1 >> >> Total blocks (validated):1 (avg. block size 7217 B) >> >> Minimally replicated blocks: 1 (100.0 %) >> >> Over-replicated blocks: 0 (0.0 %) >> >> Under-replicated blocks:0 (0.0 %) >> >> Mis-replicated blocks:0 (0.0 %) >> >> Default replication factor: 1 >> >> Average block replication: 1.0 >> >> Corrupt blocks: 0 >> >> Missing replicas: 0 (0.0 %) >> >> Number of data-nodes: 1 >> >> Number of racks: 1 >> >> FSCK ended at Fri Jan 24 11:15:17 CST 2014 in 1 milliseconds >> >> >> >> >> >> The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY >> >> >> >> >> >> On Fri, Jan 24, 2014 at 11:08 AM, Harsh J wrote: >> >> Sorry, but what was the question? I also do not see a locations option >> flag. >> >
Re: hdfs fsck -locations
Yes, Rodrigo, that's what I was looking for. So in my install I somehow don't have it at all. Was asked by my students, so I got the answer. Mark On Fri, Jan 24, 2014 at 4:00 PM, Nascimento, Rodrigo < rodrigo.nascime...@netapp.com> wrote: > Mark, > > there we go ;-) > > Rodrigo Nascimento > Systems Engineer @ Brazil > Mobile +55 11 991.873.810 > > Sent from my iPhone > > Begin forwarded message: > > *From:* "Nascimento, Rodrigo" > *Date:* 24 de janeiro de 2014 15:59:33 BRST > *To:* "user@hadoop.apache.org" > *Subject:* *RE: hdfs fsck -locations* > > Hi Mark, > > It is a sample from my sandbox. Your question is about the part that is in > RED at the output below, right? > > [root@sandbox ~]# hdfs fsck /user/ambari-qa/passwd -locations > Connecting to namenode via http://sandbox.hortonworks.com:50070 > FSCK started by root (auth:SIMPLE) from /172.16.13.30 for path > /user/ambari-qa/passwd at Fri Jan 24 09:53:43 PST 2014 > . > */user/ambari-qa/passwd: Under replicated > BP-1578958328-10.0.2.15-1382306880516:blk_1073742464_1640. Target Replicas > is 3 but found 1 replica(s).* > Status: HEALTHY > Total size:1708 B > Total dirs:0 > Total files:1 > Total symlinks:0 > Total blocks (validated):1 (avg. block size 1708 B) > Minimally replicated blocks:1 (100.0 %) > Over-replicated blocks:0 (0.0 %) > Under-replicated blocks:1 (100.0 %) > Mis-replicated blocks:0 (0.0 %) > Default replication factor:3 > Average block replication:1.0 > Corrupt blocks:0 > Missing replicas:2 (66.64 %) > Number of data-nodes:1 > Number of racks:1 > FSCK ended at Fri Jan 24 09:53:43 PST 2014 in 1 milliseconds > > > The filesystem under path '/user/ambari-qa/passwd' is HEALTHY > [root@sandbox ~]# > > Rod Nascimento > > -- > *From:* Nascimento, Rodrigo [rodrigo.nascime...@netapp.com] > *Sent:* Friday, January 24, 2014 3:34 PM > *To:* user@hadoop.apache.org > *Subject:* RE: hdfs fsck -locations > > I’m not seeing locations flag yet. > > > > *Rod Nascimento* > > *Systems Engineer @ Brazil* > > > > *People **don’t** buy **WHAT** you do. They buy **WHY** you do it.* > > > > *From:* Mark Kerzner > [mailto:mark.kerz...@shmsoft.com] > > *Sent:* Friday, January 24, 2014 3:16 PM > *To:* Hadoop User > *Subject:* Re: hdfs fsck -locations > > > > Sorry, did not copy the full command > > > > hdfs fsck /user/mark/data/word_count.csv -locations > > Connecting to namenode via http://mark-7:50070 > > FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path > /user/mark/data/word_count.csv at Fri Jan 24 11:15:17 CST 2014 > > .Status: HEALTHY > > Total size: 7217 B > > Total dirs: 0 > > Total files: 1 > > Total blocks (validated):1 (avg. block size 7217 B) > > Minimally replicated blocks: 1 (100.0 %) > > Over-replicated blocks: 0 (0.0 %) > > Under-replicated blocks:0 (0.0 %) > > Mis-replicated blocks:0 (0.0 %) > > Default replication factor: 1 > > Average block replication: 1.0 > > Corrupt blocks: 0 > > Missing replicas: 0 (0.0 %) > > Number of data-nodes: 1 > > Number of racks: 1 > > FSCK ended at Fri Jan 24 11:15:17 CST 2014 in 1 milliseconds > > > > > > The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY > > > > > > On Fri, Jan 24, 2014 at 11:08 AM, Harsh J wrote: > > Sorry, but what was the question? I also do not see a locations option > flag. > > On Jan 24, 2014 7:17 PM, "Mark Kerzner" wrote: > > Here is an example > > > > hdfs fsck /user/mark/data/word_count.csv > > Connecting to namenode via http://mark-7:50070 > > FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path > /user/mark/data/word_count.csv at Fri Jan 24 07:45:24 CST 2014 > > .Status: HEALTHY > > Total size: 7217 B > > Total dirs: 0 > > Total files: 1 > > Total blocks (validated): 1 (avg. block size 7217 B) > > Minimally replicated blocks: 1 (100.0 %) > > Over-replicated blocks: 0 (0.0 %) > > Under-replicated blocks: 0 (0.0 %) > > Mis-replicated blocks: 0 (0.0 %) > > Default replication factor: 1 > > Average block replication: 1.0 > > Corrupt blocks: 0 > > Missing replicas: 0 (0.0 %) > > Number of data-nodes: 1 > > Number of racks: 1 > > FSCK ended at Fri Jan 24 07:45:24 CST 2014 in 0 milliseconds > > > > > > On Fri, Jan 24, 2014 at 4:34 AM, Harsh J wrote: > > Hi Mark, > > Yes, the locations are shown as IP. > > > On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner > wrote: > > Hi, > > > > hdfs fsck -locations > > > > is supposed to show every block with its location? Is location the ip of > the > > datanode? > > > > Thank you, > > Mark > > > -- > Harsh J > > > > > >
Fwd: hdfs fsck -locations
Mark, there we go ;-) Rodrigo Nascimento Systems Engineer @ Brazil Mobile +55 11 991.873.810 Sent from my iPhone Begin forwarded message: From: "Nascimento, Rodrigo" mailto:rodrigo.nascime...@netapp.com>> Date: 24 de janeiro de 2014 15:59:33 BRST To: "user@hadoop.apache.org<mailto:user@hadoop.apache.org>" mailto:user@hadoop.apache.org>> Subject: RE: hdfs fsck -locations Hi Mark, It is a sample from my sandbox. Your question is about the part that is in RED at the output below, right? [root@sandbox ~]# hdfs fsck /user/ambari-qa/passwd -locations Connecting to namenode via http://sandbox.hortonworks.com:50070 FSCK started by root (auth:SIMPLE) from /172.16.13.30 for path /user/ambari-qa/passwd at Fri Jan 24 09:53:43 PST 2014 . /user/ambari-qa/passwd: Under replicated BP-1578958328-10.0.2.15-1382306880516:blk_1073742464_1640. Target Replicas is 3 but found 1 replica(s). Status: HEALTHY Total size:1708 B Total dirs:0 Total files:1 Total symlinks:0 Total blocks (validated):1 (avg. block size 1708 B) Minimally replicated blocks:1 (100.0 %) Over-replicated blocks:0 (0.0 %) Under-replicated blocks:1 (100.0 %) Mis-replicated blocks:0 (0.0 %) Default replication factor:3 Average block replication:1.0 Corrupt blocks:0 Missing replicas:2 (66.64 %) Number of data-nodes:1 Number of racks:1 FSCK ended at Fri Jan 24 09:53:43 PST 2014 in 1 milliseconds The filesystem under path '/user/ambari-qa/passwd' is HEALTHY [root@sandbox ~]# Rod Nascimento From: Nascimento, Rodrigo [rodrigo.nascime...@netapp.com<mailto:rodrigo.nascime...@netapp.com>] Sent: Friday, January 24, 2014 3:34 PM To: user@hadoop.apache.org<mailto:user@hadoop.apache.org> Subject: RE: hdfs fsck -locations I’m not seeing locations flag yet. Rod Nascimento Systems Engineer @ Brazil People don’t buy WHAT you do. They buy WHY you do it. From: Mark Kerzner [mailto:mark.kerz...@shmsoft.com] Sent: Friday, January 24, 2014 3:16 PM To: Hadoop User Subject: Re: hdfs fsck -locations Sorry, did not copy the full command hdfs fsck /user/mark/data/word_count.csv -locations Connecting to namenode via http://mark-7:50070 FSCK started by mark (auth:SIMPLE) from /192.168.1.232<http://192.168.1.232> for path /user/mark/data/word_count.csv at Fri Jan 24 11:15:17 CST 2014 .Status: HEALTHY Total size: 7217 B Total dirs: 0 Total files: 1 Total blocks (validated):1 (avg. block size 7217 B) Minimally replicated blocks: 1 (100.0 %) Over-replicated blocks: 0 (0.0 %) Under-replicated blocks:0 (0.0 %) Mis-replicated blocks:0 (0.0 %) Default replication factor: 1 Average block replication: 1.0 Corrupt blocks: 0 Missing replicas: 0 (0.0 %) Number of data-nodes: 1 Number of racks: 1 FSCK ended at Fri Jan 24 11:15:17 CST 2014 in 1 milliseconds The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY On Fri, Jan 24, 2014 at 11:08 AM, Harsh J mailto:ha...@cloudera.com>> wrote: Sorry, but what was the question? I also do not see a locations option flag. On Jan 24, 2014 7:17 PM, "Mark Kerzner" mailto:mark.kerz...@shmsoft.com>> wrote: Here is an example hdfs fsck /user/mark/data/word_count.csv Connecting to namenode via http://mark-7:50070 FSCK started by mark (auth:SIMPLE) from /192.168.1.232<http://192.168.1.232> for path /user/mark/data/word_count.csv at Fri Jan 24 07:45:24 CST 2014 .Status: HEALTHY Total size: 7217 B Total dirs: 0 Total files: 1 Total blocks (validated): 1 (avg. block size 7217 B) Minimally replicated blocks: 1 (100.0 %) Over-replicated blocks: 0 (0.0 %) Under-replicated blocks: 0 (0.0 %) Mis-replicated blocks: 0 (0.0 %) Default replication factor: 1 Average block replication: 1.0 Corrupt blocks: 0 Missing replicas: 0 (0.0 %) Number of data-nodes: 1 Number of racks: 1 FSCK ended at Fri Jan 24 07:45:24 CST 2014 in 0 milliseconds On Fri, Jan 24, 2014 at 4:34 AM, Harsh J mailto:ha...@cloudera.com>> wrote: Hi Mark, Yes, the locations are shown as IP. On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner mailto:mark.kerz...@shmsoft.com>> wrote: > Hi, > > hdfs fsck -locations > > is supposed to show every block with its location? Is location the ip of the > datanode? > > Thank you, > Mark -- Harsh J
Re: hdfs fsck -locations
Can you send me your output? hadoop version Hadoop 2.0.0-cdh4.5.0 Subversion git://ubuntu64-12-04-mk1/var/lib/jenkins/workspace/generic-package-ubuntu64-12-04/CDH4.5.0-Packaging-Hadoop-2013-11-20_14-31-53/hadoop-2.0.0+1518-1.cdh4.5.0.p0.24~precise/src/hadoop-common-project/hadoop-common -r 8e266e052e423af592871e2dfe09d54c03f6a0e8 Compiled by jenkins on Wed Nov 20 15:10:35 PST 2013 >From source with checksum 9848b0f85b461913ed63fa19c2b79ccc This command was run using /usr/lib/hadoop/hadoop-common-2.0.0-cdh4.5.0.jar On Fri, Jan 24, 2014 at 3:14 PM, Nascimento, Rodrigo < rodrigo.nascime...@netapp.com> wrote: > Mark, > > Did you see that your output is different from mine? > > which Is your hadoop versioks > > Rodrigo Nascimento > Systems Engineer @ Brazil > Mobile +55 11 991.873.810 > > Sent from my iPhone > > On 24/01/2014, at 18:31, "Mark Kerzner" wrote: > > HI, Rodrigo, I am fine thank you :) > > Here is the _complete_ output > > mark@mark-7:~$ hdfs fsck /user/mark/data/word_count.csv -locations > Connecting to namenode via http://mark-7:50070 > FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path > /user/mark/data/word_count.csv at Fri Jan 24 14:30:23 CST 2014 > .Status: HEALTHY > Total size: 7217 B > Total dirs: 0 > Total files: 1 > Total blocks (validated): 1 (avg. block size 7217 B) > Minimally replicated blocks: 1 (100.0 %) > Over-replicated blocks: 0 (0.0 %) > Under-replicated blocks: 0 (0.0 %) > Mis-replicated blocks: 0 (0.0 %) > Default replication factor: 1 > Average block replication: 1.0 > Corrupt blocks: 0 > Missing replicas: 0 (0.0 %) > Number of data-nodes: 1 > Number of racks: 1 > FSCK ended at Fri Jan 24 14:30:23 CST 2014 in 0 milliseconds > > > The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY > mark@mark-7:~$ > > > On Fri, Jan 24, 2014 at 1:35 PM, Nascimento, Rodrigo < > rodrigo.nascime...@netapp.com> wrote: > >> Hi Mark, >> >> How are you? >> >> at your output from word_count.csv, the portion related to the nlock >> locations is missing. This is the reason why I told you in my previous >> message that "I'm not seeing locations yet." >> >> please, take a look at my last e-mail. >> >> All the best, >> >> Rodrigo Nascimento >> Systems Engineer @ Brazil >> >> Sent from my iPhone >> >> On 24/01/2014, at 16:40, "Mark Kerzner" wrote: >> >> >> hdfs fsck /user/mark/data/word_count.csv *-locations* >> >> >> On Fri, Jan 24, 2014 at 11:34 AM, Nascimento, Rodrigo < >> rodrigo.nascime...@netapp.com> wrote: >> >>> I’m not seeing locations flag yet. >>> >>> >>> >>> *Rod Nascimento* >>> >>> *Systems Engineer @ Brazil* >>> >>> >>> >>> *People **don’t** buy **WHAT** you do. They buy **WHY** you do it.* >>> >>> >>> >>> *From:* Mark Kerzner [mailto:mark.kerz...@shmsoft.com] >>> *Sent:* Friday, January 24, 2014 3:16 PM >>> *To:* Hadoop User >>> *Subject:* Re: hdfs fsck -locations >>> >>> >>> >>> Sorry, did not copy the full command >>> >>> >>> >>> hdfs fsck /user/mark/data/word_count.csv -locations >>> >>> Connecting to namenode via http://mark-7:50070 >>> >>> FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path >>> /user/mark/data/word_count.csv at Fri Jan 24 11:15:17 CST 2014 >>> >>> .Status: HEALTHY >>> >>> Total size: 7217 B >>> >>> Total dirs: 0 >>> >>> Total files: 1 >>> >>> Total blocks (validated):1 (avg. block size 7217 B) >>> >>> Minimally replicated blocks: 1 (100.0 %) >>> >>> Over-replicated blocks: 0 (0.0 %) >>> >>> Under-replicated blocks:0 (0.0 %) >>> >>> Mis-replicated blocks:0 (0.0 %) >>> >>> Default replication factor: 1 >>> >>> Average block replication: 1.0 >>> >>> Corrupt blocks: 0 >>> >>> Missing replicas: 0 (0.0 %) >>> >>> Number of data-nodes: 1 >>> >>> Number of racks: 1 >>> >>> FSCK ended at Fri Jan 24 11:15:17 CST 2014 in 1 milliseconds >>> >>> >>> >>> >>> >>> The fil
Re: hdfs fsck -locations
Mark, Did you see that your output is different from mine? which Is your hadoop versioks Rodrigo Nascimento Systems Engineer @ Brazil Mobile +55 11 991.873.810 Sent from my iPhone On 24/01/2014, at 18:31, "Mark Kerzner" mailto:mark.kerz...@shmsoft.com>> wrote: HI, Rodrigo, I am fine thank you :) Here is the _complete_ output mark@mark-7:~$ hdfs fsck /user/mark/data/word_count.csv -locations Connecting to namenode via http://mark-7:50070 FSCK started by mark (auth:SIMPLE) from /192.168.1.232<http://192.168.1.232> for path /user/mark/data/word_count.csv at Fri Jan 24 14:30:23 CST 2014 .Status: HEALTHY Total size: 7217 B Total dirs: 0 Total files: 1 Total blocks (validated): 1 (avg. block size 7217 B) Minimally replicated blocks: 1 (100.0 %) Over-replicated blocks: 0 (0.0 %) Under-replicated blocks: 0 (0.0 %) Mis-replicated blocks: 0 (0.0 %) Default replication factor: 1 Average block replication: 1.0 Corrupt blocks: 0 Missing replicas: 0 (0.0 %) Number of data-nodes: 1 Number of racks: 1 FSCK ended at Fri Jan 24 14:30:23 CST 2014 in 0 milliseconds The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY mark@mark-7:~$ On Fri, Jan 24, 2014 at 1:35 PM, Nascimento, Rodrigo mailto:rodrigo.nascime...@netapp.com>> wrote: Hi Mark, How are you? at your output from word_count.csv, the portion related to the nlock locations is missing. This is the reason why I told you in my previous message that "I'm not seeing locations yet." please, take a look at my last e-mail. All the best, Rodrigo Nascimento Systems Engineer @ Brazil Sent from my iPhone On 24/01/2014, at 16:40, "Mark Kerzner" mailto:mark.kerz...@shmsoft.com>> wrote: hdfs fsck /user/mark/data/word_count.csv -locations On Fri, Jan 24, 2014 at 11:34 AM, Nascimento, Rodrigo mailto:rodrigo.nascime...@netapp.com>> wrote: I’m not seeing locations flag yet. Rod Nascimento Systems Engineer @ Brazil People don’t buy WHAT you do. They buy WHY you do it. From: Mark Kerzner [mailto:mark.kerz...@shmsoft.com<mailto:mark.kerz...@shmsoft.com>] Sent: Friday, January 24, 2014 3:16 PM To: Hadoop User Subject: Re: hdfs fsck -locations Sorry, did not copy the full command hdfs fsck /user/mark/data/word_count.csv -locations Connecting to namenode via http://mark-7:50070 FSCK started by mark (auth:SIMPLE) from /192.168.1.232<http://192.168.1.232> for path /user/mark/data/word_count.csv at Fri Jan 24 11:15:17 CST 2014 .Status: HEALTHY Total size: 7217 B Total dirs: 0 Total files: 1 Total blocks (validated):1 (avg. block size 7217 B) Minimally replicated blocks: 1 (100.0 %) Over-replicated blocks: 0 (0.0 %) Under-replicated blocks:0 (0.0 %) Mis-replicated blocks:0 (0.0 %) Default replication factor: 1 Average block replication: 1.0 Corrupt blocks: 0 Missing replicas: 0 (0.0 %) Number of data-nodes: 1 Number of racks: 1 FSCK ended at Fri Jan 24 11:15:17 CST 2014 in 1 milliseconds The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY On Fri, Jan 24, 2014 at 11:08 AM, Harsh J mailto:ha...@cloudera.com>> wrote: Sorry, but what was the question? I also do not see a locations option flag. On Jan 24, 2014 7:17 PM, "Mark Kerzner" mailto:mark.kerz...@shmsoft.com>> wrote: Here is an example hdfs fsck /user/mark/data/word_count.csv Connecting to namenode via http://mark-7:50070 FSCK started by mark (auth:SIMPLE) from /192.168.1.232<http://192.168.1.232> for path /user/mark/data/word_count.csv at Fri Jan 24 07:45:24 CST 2014 .Status: HEALTHY Total size: 7217 B Total dirs: 0 Total files: 1 Total blocks (validated): 1 (avg. block size 7217 B) Minimally replicated blocks: 1 (100.0 %) Over-replicated blocks: 0 (0.0 %) Under-replicated blocks: 0 (0.0 %) Mis-replicated blocks: 0 (0.0 %) Default replication factor: 1 Average block replication: 1.0 Corrupt blocks: 0 Missing replicas: 0 (0.0 %) Number of data-nodes: 1 Number of racks: 1 FSCK ended at Fri Jan 24 07:45:24 CST 2014 in 0 milliseconds On Fri, Jan 24, 2014 at 4:34 AM, Harsh J mailto:ha...@cloudera.com>> wrote: Hi Mark, Yes, the locations are shown as IP. On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner mailto:mark.kerz...@shmsoft.com>> wrote: > Hi, > > hdfs fsck -locations > > is supposed to show every block with its location? Is location the ip of the > datanode? > > Thank you, > Mark -- Harsh J
Re: hdfs fsck -locations
HI, Rodrigo, I am fine thank you :) Here is the _complete_ output mark@mark-7:~$ hdfs fsck /user/mark/data/word_count.csv -locations Connecting to namenode via http://mark-7:50070 FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path /user/mark/data/word_count.csv at Fri Jan 24 14:30:23 CST 2014 .Status: HEALTHY Total size: 7217 B Total dirs: 0 Total files: 1 Total blocks (validated): 1 (avg. block size 7217 B) Minimally replicated blocks: 1 (100.0 %) Over-replicated blocks: 0 (0.0 %) Under-replicated blocks: 0 (0.0 %) Mis-replicated blocks: 0 (0.0 %) Default replication factor: 1 Average block replication: 1.0 Corrupt blocks: 0 Missing replicas: 0 (0.0 %) Number of data-nodes: 1 Number of racks: 1 FSCK ended at Fri Jan 24 14:30:23 CST 2014 in 0 milliseconds The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY mark@mark-7:~$ On Fri, Jan 24, 2014 at 1:35 PM, Nascimento, Rodrigo < rodrigo.nascime...@netapp.com> wrote: > Hi Mark, > > How are you? > > at your output from word_count.csv, the portion related to the nlock > locations is missing. This is the reason why I told you in my previous > message that "I'm not seeing locations yet." > > please, take a look at my last e-mail. > > All the best, > > Rodrigo Nascimento > Systems Engineer @ Brazil > > Sent from my iPhone > > On 24/01/2014, at 16:40, "Mark Kerzner" wrote: > > > hdfs fsck /user/mark/data/word_count.csv *-locations* > > > On Fri, Jan 24, 2014 at 11:34 AM, Nascimento, Rodrigo < > rodrigo.nascime...@netapp.com> wrote: > >> I’m not seeing locations flag yet. >> >> >> >> *Rod Nascimento* >> >> *Systems Engineer @ Brazil* >> >> >> >> *People **don’t** buy **WHAT** you do. They buy **WHY** you do it.* >> >> >> >> *From:* Mark Kerzner [mailto:mark.kerz...@shmsoft.com] >> *Sent:* Friday, January 24, 2014 3:16 PM >> *To:* Hadoop User >> *Subject:* Re: hdfs fsck -locations >> >> >> >> Sorry, did not copy the full command >> >> >> >> hdfs fsck /user/mark/data/word_count.csv -locations >> >> Connecting to namenode via http://mark-7:50070 >> >> FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path >> /user/mark/data/word_count.csv at Fri Jan 24 11:15:17 CST 2014 >> >> .Status: HEALTHY >> >> Total size: 7217 B >> >> Total dirs: 0 >> >> Total files: 1 >> >> Total blocks (validated):1 (avg. block size 7217 B) >> >> Minimally replicated blocks: 1 (100.0 %) >> >> Over-replicated blocks: 0 (0.0 %) >> >> Under-replicated blocks:0 (0.0 %) >> >> Mis-replicated blocks:0 (0.0 %) >> >> Default replication factor: 1 >> >> Average block replication: 1.0 >> >> Corrupt blocks: 0 >> >> Missing replicas: 0 (0.0 %) >> >> Number of data-nodes: 1 >> >> Number of racks: 1 >> >> FSCK ended at Fri Jan 24 11:15:17 CST 2014 in 1 milliseconds >> >> >> >> >> >> The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY >> >> >> >> >> >> On Fri, Jan 24, 2014 at 11:08 AM, Harsh J wrote: >> >> Sorry, but what was the question? I also do not see a locations option >> flag. >> >> On Jan 24, 2014 7:17 PM, "Mark Kerzner" wrote: >> >> Here is an example >> >> >> >> hdfs fsck /user/mark/data/word_count.csv >> >> Connecting to namenode via http://mark-7:50070 >> >> FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path >> /user/mark/data/word_count.csv at Fri Jan 24 07:45:24 CST 2014 >> >> .Status: HEALTHY >> >> Total size: 7217 B >> >> Total dirs: 0 >> >> Total files: 1 >> >> Total blocks (validated): 1 (avg. block size 7217 B) >> >> Minimally replicated blocks: 1 (100.0 %) >> >> Over-replicated blocks: 0 (0.0 %) >> >> Under-replicated blocks: 0 (0.0 %) >> >> Mis-replicated blocks: 0 (0.0 %) >> >> Default replication factor: 1 >> >> Average block replication: 1.0 >> >> Corrupt blocks: 0 >> >> Missing replicas: 0 (0.0 %) >> >> Number of data-nodes: 1 >> >> Number of racks: 1 >> >> FSCK ended at Fri Jan 24 07:45:24 CST 2014 in 0 milliseconds >> >> >> >> >> >> On Fri, Jan 24, 2014 at 4:34 AM, Harsh J wrote: >> >> Hi Mark, >> >> Yes, the locations are shown as IP. >> >> >> On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner >> wrote: >> > Hi, >> > >> > hdfs fsck -locations >> > >> > is supposed to show every block with its location? Is location the ip >> of the >> > datanode? >> > >> > Thank you, >> > Mark >> >> >> -- >> Harsh J >> >> >> >> >> > >
Re: hdfs fsck -locations
Hi Mark, How are you? at your output from word_count.csv, the portion related to the nlock locations is missing. This is the reason why I told you in my previous message that "I'm not seeing locations yet." please, take a look at my last e-mail. All the best, Rodrigo Nascimento Systems Engineer @ Brazil Sent from my iPhone On 24/01/2014, at 16:40, "Mark Kerzner" mailto:mark.kerz...@shmsoft.com>> wrote: hdfs fsck /user/mark/data/word_count.csv -locations On Fri, Jan 24, 2014 at 11:34 AM, Nascimento, Rodrigo mailto:rodrigo.nascime...@netapp.com>> wrote: I’m not seeing locations flag yet. Rod Nascimento Systems Engineer @ Brazil People don’t buy WHAT you do. They buy WHY you do it. From: Mark Kerzner [mailto:mark.kerz...@shmsoft.com<mailto:mark.kerz...@shmsoft.com>] Sent: Friday, January 24, 2014 3:16 PM To: Hadoop User Subject: Re: hdfs fsck -locations Sorry, did not copy the full command hdfs fsck /user/mark/data/word_count.csv -locations Connecting to namenode via http://mark-7:50070 FSCK started by mark (auth:SIMPLE) from /192.168.1.232<http://192.168.1.232> for path /user/mark/data/word_count.csv at Fri Jan 24 11:15:17 CST 2014 .Status: HEALTHY Total size: 7217 B Total dirs: 0 Total files: 1 Total blocks (validated):1 (avg. block size 7217 B) Minimally replicated blocks: 1 (100.0 %) Over-replicated blocks: 0 (0.0 %) Under-replicated blocks:0 (0.0 %) Mis-replicated blocks:0 (0.0 %) Default replication factor: 1 Average block replication: 1.0 Corrupt blocks: 0 Missing replicas: 0 (0.0 %) Number of data-nodes: 1 Number of racks: 1 FSCK ended at Fri Jan 24 11:15:17 CST 2014 in 1 milliseconds The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY On Fri, Jan 24, 2014 at 11:08 AM, Harsh J mailto:ha...@cloudera.com>> wrote: Sorry, but what was the question? I also do not see a locations option flag. On Jan 24, 2014 7:17 PM, "Mark Kerzner" mailto:mark.kerz...@shmsoft.com>> wrote: Here is an example hdfs fsck /user/mark/data/word_count.csv Connecting to namenode via http://mark-7:50070 FSCK started by mark (auth:SIMPLE) from /192.168.1.232<http://192.168.1.232> for path /user/mark/data/word_count.csv at Fri Jan 24 07:45:24 CST 2014 .Status: HEALTHY Total size: 7217 B Total dirs: 0 Total files: 1 Total blocks (validated): 1 (avg. block size 7217 B) Minimally replicated blocks: 1 (100.0 %) Over-replicated blocks: 0 (0.0 %) Under-replicated blocks: 0 (0.0 %) Mis-replicated blocks: 0 (0.0 %) Default replication factor: 1 Average block replication: 1.0 Corrupt blocks: 0 Missing replicas: 0 (0.0 %) Number of data-nodes: 1 Number of racks: 1 FSCK ended at Fri Jan 24 07:45:24 CST 2014 in 0 milliseconds On Fri, Jan 24, 2014 at 4:34 AM, Harsh J mailto:ha...@cloudera.com>> wrote: Hi Mark, Yes, the locations are shown as IP. On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner mailto:mark.kerz...@shmsoft.com>> wrote: > Hi, > > hdfs fsck -locations > > is supposed to show every block with its location? Is location the ip of the > datanode? > > Thank you, > Mark -- Harsh J
Re: hdfs fsck -locations
hdfs fsck /user/mark/data/word_count.csv *-locations* On Fri, Jan 24, 2014 at 11:34 AM, Nascimento, Rodrigo < rodrigo.nascime...@netapp.com> wrote: > I’m not seeing locations flag yet. > > > > *Rod Nascimento* > > *Systems Engineer @ Brazil* > > > > *People **don’t** buy **WHAT** you do. They buy **WHY** you do it.* > > > > *From:* Mark Kerzner [mailto:mark.kerz...@shmsoft.com] > *Sent:* Friday, January 24, 2014 3:16 PM > *To:* Hadoop User > *Subject:* Re: hdfs fsck -locations > > > > Sorry, did not copy the full command > > > > hdfs fsck /user/mark/data/word_count.csv -locations > > Connecting to namenode via http://mark-7:50070 > > FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path > /user/mark/data/word_count.csv at Fri Jan 24 11:15:17 CST 2014 > > .Status: HEALTHY > > Total size: 7217 B > > Total dirs: 0 > > Total files: 1 > > Total blocks (validated):1 (avg. block size 7217 B) > > Minimally replicated blocks: 1 (100.0 %) > > Over-replicated blocks: 0 (0.0 %) > > Under-replicated blocks:0 (0.0 %) > > Mis-replicated blocks:0 (0.0 %) > > Default replication factor: 1 > > Average block replication: 1.0 > > Corrupt blocks: 0 > > Missing replicas: 0 (0.0 %) > > Number of data-nodes: 1 > > Number of racks: 1 > > FSCK ended at Fri Jan 24 11:15:17 CST 2014 in 1 milliseconds > > > > > > The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY > > > > > > On Fri, Jan 24, 2014 at 11:08 AM, Harsh J wrote: > > Sorry, but what was the question? I also do not see a locations option > flag. > > On Jan 24, 2014 7:17 PM, "Mark Kerzner" wrote: > > Here is an example > > > > hdfs fsck /user/mark/data/word_count.csv > > Connecting to namenode via http://mark-7:50070 > > FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path > /user/mark/data/word_count.csv at Fri Jan 24 07:45:24 CST 2014 > > .Status: HEALTHY > > Total size: 7217 B > > Total dirs: 0 > > Total files: 1 > > Total blocks (validated): 1 (avg. block size 7217 B) > > Minimally replicated blocks: 1 (100.0 %) > > Over-replicated blocks: 0 (0.0 %) > > Under-replicated blocks: 0 (0.0 %) > > Mis-replicated blocks: 0 (0.0 %) > > Default replication factor: 1 > > Average block replication: 1.0 > > Corrupt blocks: 0 > > Missing replicas: 0 (0.0 %) > > Number of data-nodes: 1 > > Number of racks: 1 > > FSCK ended at Fri Jan 24 07:45:24 CST 2014 in 0 milliseconds > > > > > > On Fri, Jan 24, 2014 at 4:34 AM, Harsh J wrote: > > Hi Mark, > > Yes, the locations are shown as IP. > > > On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner > wrote: > > Hi, > > > > hdfs fsck -locations > > > > is supposed to show every block with its location? Is location the ip of > the > > datanode? > > > > Thank you, > > Mark > > > -- > Harsh J > > > > >
RE: hdfs fsck -locations
Hi Mark, It is a sample from my sandbox. Your question is about the part that is in RED at the output below, right? [root@sandbox ~]# hdfs fsck /user/ambari-qa/passwd -locations Connecting to namenode via http://sandbox.hortonworks.com:50070 FSCK started by root (auth:SIMPLE) from /172.16.13.30 for path /user/ambari-qa/passwd at Fri Jan 24 09:53:43 PST 2014 . /user/ambari-qa/passwd: Under replicated BP-1578958328-10.0.2.15-1382306880516:blk_1073742464_1640. Target Replicas is 3 but found 1 replica(s). Status: HEALTHY Total size:1708 B Total dirs:0 Total files:1 Total symlinks:0 Total blocks (validated):1 (avg. block size 1708 B) Minimally replicated blocks:1 (100.0 %) Over-replicated blocks:0 (0.0 %) Under-replicated blocks:1 (100.0 %) Mis-replicated blocks:0 (0.0 %) Default replication factor:3 Average block replication:1.0 Corrupt blocks:0 Missing replicas:2 (66.64 %) Number of data-nodes:1 Number of racks:1 FSCK ended at Fri Jan 24 09:53:43 PST 2014 in 1 milliseconds The filesystem under path '/user/ambari-qa/passwd' is HEALTHY [root@sandbox ~]# Rod Nascimento From: Nascimento, Rodrigo [rodrigo.nascime...@netapp.com] Sent: Friday, January 24, 2014 3:34 PM To: user@hadoop.apache.org Subject: RE: hdfs fsck -locations I’m not seeing locations flag yet. Rod Nascimento Systems Engineer @ Brazil People don’t buy WHAT you do. They buy WHY you do it. From: Mark Kerzner [mailto:mark.kerz...@shmsoft.com] Sent: Friday, January 24, 2014 3:16 PM To: Hadoop User Subject: Re: hdfs fsck -locations Sorry, did not copy the full command hdfs fsck /user/mark/data/word_count.csv -locations Connecting to namenode via http://mark-7:50070 FSCK started by mark (auth:SIMPLE) from /192.168.1.232<http://192.168.1.232> for path /user/mark/data/word_count.csv at Fri Jan 24 11:15:17 CST 2014 .Status: HEALTHY Total size: 7217 B Total dirs: 0 Total files: 1 Total blocks (validated):1 (avg. block size 7217 B) Minimally replicated blocks: 1 (100.0 %) Over-replicated blocks: 0 (0.0 %) Under-replicated blocks:0 (0.0 %) Mis-replicated blocks:0 (0.0 %) Default replication factor: 1 Average block replication: 1.0 Corrupt blocks: 0 Missing replicas: 0 (0.0 %) Number of data-nodes: 1 Number of racks: 1 FSCK ended at Fri Jan 24 11:15:17 CST 2014 in 1 milliseconds The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY On Fri, Jan 24, 2014 at 11:08 AM, Harsh J mailto:ha...@cloudera.com>> wrote: Sorry, but what was the question? I also do not see a locations option flag. On Jan 24, 2014 7:17 PM, "Mark Kerzner" mailto:mark.kerz...@shmsoft.com>> wrote: Here is an example hdfs fsck /user/mark/data/word_count.csv Connecting to namenode via http://mark-7:50070 FSCK started by mark (auth:SIMPLE) from /192.168.1.232<http://192.168.1.232> for path /user/mark/data/word_count.csv at Fri Jan 24 07:45:24 CST 2014 .Status: HEALTHY Total size: 7217 B Total dirs: 0 Total files: 1 Total blocks (validated): 1 (avg. block size 7217 B) Minimally replicated blocks: 1 (100.0 %) Over-replicated blocks: 0 (0.0 %) Under-replicated blocks: 0 (0.0 %) Mis-replicated blocks: 0 (0.0 %) Default replication factor: 1 Average block replication: 1.0 Corrupt blocks: 0 Missing replicas: 0 (0.0 %) Number of data-nodes: 1 Number of racks: 1 FSCK ended at Fri Jan 24 07:45:24 CST 2014 in 0 milliseconds On Fri, Jan 24, 2014 at 4:34 AM, Harsh J mailto:ha...@cloudera.com>> wrote: Hi Mark, Yes, the locations are shown as IP. On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner mailto:mark.kerz...@shmsoft.com>> wrote: > Hi, > > hdfs fsck -locations > > is supposed to show every block with its location? Is location the ip of the > datanode? > > Thank you, > Mark -- Harsh J
RE: hdfs fsck -locations
I'm not seeing locations flag yet. Rod Nascimento Systems Engineer @ Brazil People don't buy WHAT you do. They buy WHY you do it. From: Mark Kerzner [mailto:mark.kerz...@shmsoft.com] Sent: Friday, January 24, 2014 3:16 PM To: Hadoop User Subject: Re: hdfs fsck -locations Sorry, did not copy the full command hdfs fsck /user/mark/data/word_count.csv -locations Connecting to namenode via http://mark-7:50070 FSCK started by mark (auth:SIMPLE) from /192.168.1.232<http://192.168.1.232> for path /user/mark/data/word_count.csv at Fri Jan 24 11:15:17 CST 2014 .Status: HEALTHY Total size: 7217 B Total dirs: 0 Total files: 1 Total blocks (validated):1 (avg. block size 7217 B) Minimally replicated blocks: 1 (100.0 %) Over-replicated blocks: 0 (0.0 %) Under-replicated blocks:0 (0.0 %) Mis-replicated blocks:0 (0.0 %) Default replication factor: 1 Average block replication: 1.0 Corrupt blocks: 0 Missing replicas: 0 (0.0 %) Number of data-nodes: 1 Number of racks: 1 FSCK ended at Fri Jan 24 11:15:17 CST 2014 in 1 milliseconds The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY On Fri, Jan 24, 2014 at 11:08 AM, Harsh J mailto:ha...@cloudera.com>> wrote: Sorry, but what was the question? I also do not see a locations option flag. On Jan 24, 2014 7:17 PM, "Mark Kerzner" mailto:mark.kerz...@shmsoft.com>> wrote: Here is an example hdfs fsck /user/mark/data/word_count.csv Connecting to namenode via http://mark-7:50070 FSCK started by mark (auth:SIMPLE) from /192.168.1.232<http://192.168.1.232> for path /user/mark/data/word_count.csv at Fri Jan 24 07:45:24 CST 2014 .Status: HEALTHY Total size: 7217 B Total dirs: 0 Total files: 1 Total blocks (validated): 1 (avg. block size 7217 B) Minimally replicated blocks: 1 (100.0 %) Over-replicated blocks: 0 (0.0 %) Under-replicated blocks: 0 (0.0 %) Mis-replicated blocks: 0 (0.0 %) Default replication factor: 1 Average block replication: 1.0 Corrupt blocks: 0 Missing replicas: 0 (0.0 %) Number of data-nodes: 1 Number of racks: 1 FSCK ended at Fri Jan 24 07:45:24 CST 2014 in 0 milliseconds On Fri, Jan 24, 2014 at 4:34 AM, Harsh J mailto:ha...@cloudera.com>> wrote: Hi Mark, Yes, the locations are shown as IP. On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner mailto:mark.kerz...@shmsoft.com>> wrote: > Hi, > > hdfs fsck -locations > > is supposed to show every block with its location? Is location the ip of the > datanode? > > Thank you, > Mark -- Harsh J
Re: hdfs fsck -locations
Sorry, did not copy the full command hdfs fsck /user/mark/data/word_count.csv -locations Connecting to namenode via http://mark-7:50070 FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path /user/mark/data/word_count.csv at Fri Jan 24 11:15:17 CST 2014 .Status: HEALTHY Total size: 7217 B Total dirs: 0 Total files: 1 Total blocks (validated): 1 (avg. block size 7217 B) Minimally replicated blocks: 1 (100.0 %) Over-replicated blocks: 0 (0.0 %) Under-replicated blocks: 0 (0.0 %) Mis-replicated blocks: 0 (0.0 %) Default replication factor: 1 Average block replication: 1.0 Corrupt blocks: 0 Missing replicas: 0 (0.0 %) Number of data-nodes: 1 Number of racks: 1 FSCK ended at Fri Jan 24 11:15:17 CST 2014 in 1 milliseconds The filesystem under path '/user/mark/data/word_count.csv' is HEALTHY On Fri, Jan 24, 2014 at 11:08 AM, Harsh J wrote: > Sorry, but what was the question? I also do not see a locations option > flag. > On Jan 24, 2014 7:17 PM, "Mark Kerzner" wrote: > >> Here is an example >> >> hdfs fsck /user/mark/data/word_count.csv >> Connecting to namenode via http://mark-7:50070 >> FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path >> /user/mark/data/word_count.csv at Fri Jan 24 07:45:24 CST 2014 >> .Status: HEALTHY >> Total size: 7217 B >> Total dirs: 0 >> Total files: 1 >> Total blocks (validated): 1 (avg. block size 7217 B) >> Minimally replicated blocks: 1 (100.0 %) >> Over-replicated blocks: 0 (0.0 %) >> Under-replicated blocks: 0 (0.0 %) >> Mis-replicated blocks: 0 (0.0 %) >> Default replication factor: 1 >> Average block replication: 1.0 >> Corrupt blocks: 0 >> Missing replicas: 0 (0.0 %) >> Number of data-nodes: 1 >> Number of racks: 1 >> FSCK ended at Fri Jan 24 07:45:24 CST 2014 in 0 milliseconds >> >> >> >> On Fri, Jan 24, 2014 at 4:34 AM, Harsh J wrote: >> >>> Hi Mark, >>> >>> Yes, the locations are shown as IP. >>> >>> On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner >>> wrote: >>> > Hi, >>> > >>> > hdfs fsck -locations >>> > >>> > is supposed to show every block with its location? Is location the ip >>> of the >>> > datanode? >>> > >>> > Thank you, >>> > Mark >>> >>> >>> >>> -- >>> Harsh J >>> >> >>
Re: hdfs fsck -locations
Sorry, but what was the question? I also do not see a locations option flag. On Jan 24, 2014 7:17 PM, "Mark Kerzner" wrote: > Here is an example > > hdfs fsck /user/mark/data/word_count.csv > Connecting to namenode via http://mark-7:50070 > FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path > /user/mark/data/word_count.csv at Fri Jan 24 07:45:24 CST 2014 > .Status: HEALTHY > Total size: 7217 B > Total dirs: 0 > Total files: 1 > Total blocks (validated): 1 (avg. block size 7217 B) > Minimally replicated blocks: 1 (100.0 %) > Over-replicated blocks: 0 (0.0 %) > Under-replicated blocks: 0 (0.0 %) > Mis-replicated blocks: 0 (0.0 %) > Default replication factor: 1 > Average block replication: 1.0 > Corrupt blocks: 0 > Missing replicas: 0 (0.0 %) > Number of data-nodes: 1 > Number of racks: 1 > FSCK ended at Fri Jan 24 07:45:24 CST 2014 in 0 milliseconds > > > > On Fri, Jan 24, 2014 at 4:34 AM, Harsh J wrote: > >> Hi Mark, >> >> Yes, the locations are shown as IP. >> >> On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner >> wrote: >> > Hi, >> > >> > hdfs fsck -locations >> > >> > is supposed to show every block with its location? Is location the ip >> of the >> > datanode? >> > >> > Thank you, >> > Mark >> >> >> >> -- >> Harsh J >> > >
Re: hdfs fsck -locations
Here is an example hdfs fsck /user/mark/data/word_count.csv Connecting to namenode via http://mark-7:50070 FSCK started by mark (auth:SIMPLE) from /192.168.1.232 for path /user/mark/data/word_count.csv at Fri Jan 24 07:45:24 CST 2014 .Status: HEALTHY Total size: 7217 B Total dirs: 0 Total files: 1 Total blocks (validated): 1 (avg. block size 7217 B) Minimally replicated blocks: 1 (100.0 %) Over-replicated blocks: 0 (0.0 %) Under-replicated blocks: 0 (0.0 %) Mis-replicated blocks: 0 (0.0 %) Default replication factor: 1 Average block replication: 1.0 Corrupt blocks: 0 Missing replicas: 0 (0.0 %) Number of data-nodes: 1 Number of racks: 1 FSCK ended at Fri Jan 24 07:45:24 CST 2014 in 0 milliseconds On Fri, Jan 24, 2014 at 4:34 AM, Harsh J wrote: > Hi Mark, > > Yes, the locations are shown as IP. > > On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner > wrote: > > Hi, > > > > hdfs fsck -locations > > > > is supposed to show every block with its location? Is location the ip of > the > > datanode? > > > > Thank you, > > Mark > > > > -- > Harsh J >
Re: hdfs fsck -locations
Hi Mark, Yes, the locations are shown as IP. On Fri, Jan 24, 2014 at 12:09 AM, Mark Kerzner wrote: > Hi, > > hdfs fsck -locations > > is supposed to show every block with its location? Is location the ip of the > datanode? > > Thank you, > Mark -- Harsh J
hdfs fsck -locations
Hi, hdfs fsck -locations is supposed to show every block with its location? Is location the ip of the datanode? Thank you, Mark