you can use the fsck command to find the block locations, here's one example
hadoop fsck /user/hadoop/graph_data.txt -blocks -locations -files On Sun, Jan 26, 2014 at 2:48 PM, EdwardKing <zhan...@neusoft.com> wrote: > hdfs-site.xm is follows: > <configuration> > <property> > <name>dfs.name.dir</name> > <value>file:/home/software/name</value> > <description> </description> > </property> > <property> > <name>dfs.namenode.secondary.http-address</name> > <value>master:9001</value> > </property> > <property> > <name>dfs.data.dir</name> > <value>file:/home/software/data</value> > </property> > <property> > <name>dfs.http.address</name> > <value>master:9002</value> > </property> > <property> > <name>dfs.replication</name> > <value>2</value> > </property> > <property> > <name>dfs.datanode.du.reserved</name> > <value>1073741824</value> > </property> > <property> > <name>dfs.block.size</name> > <value>134217728</value> > </property> > <property> > <name>dfs.permissions</name> > <value>false</value> > </property> > </configuration> > > [root@master ~]# cd /home > [root@master home]# cd software/ > [root@master software]# ls > data hadoop-2.2.0 jdk1.7.0_02 name test.txt > file: hadoop-2.2.0.tar.gz jdk-7u2-linux-i586.tar.gz temp tmp > > [root@master name]# pwd > /home/software/name > [root@master name]# ls > current in_use.lock > [root@master name]# > > [root@master software]# pwd > /home/software > [root@master software]# cd data > [root@master data]# ls > current in_use.lock > > >> the meadata(file name, file path and block location) is in master, the > file data itself is in datanode. > > Where I can find abc.txt meadata,such as file name, file path and block > location? The abc.txt file data itself is in master 172.11.12.6 or node1 > 172.11.12.7,which directory it locate? > > Thanks. > > > > ----- Original Message ----- > From: Jeff Zhang > To: user@hadoop.apache.org > Sent: Sunday, January 26, 2014 2:30 PM > Subject: Re: question about hadoop dfs > > > 1. Is supergroup a directory? Where does it locate? > supergroup is user group rather than directory just like the user > group of linux > > > 2. I search abc.txt on master 172.11.12.6 and node1 172.11.12.7 by > following command: > the meadata(file name, file path and block location) is in master, the > file data itself is in datanode. > > > > > > On Sun, Jan 26, 2014 at 2:22 PM, EdwardKing <zhan...@neusoft.com> wrote: > > I use Hadoop2.2.0 to create a master node and a sub node,like follows: > > Live Datanodes : 2 > Node Transferring Address Last Contact Admin State Configured Capacity > (GB) Used(GB) Non DFS Used (GB) Remaining(GB) Used(%) > master 172.11.12.6:50010 1 In Service > 16.15 0.00 2.76 > 13.39 0.00 > node1 172.11.12.7:50010 0 In Service > 16.15 0.00 2.75 > 13.40 0.00 > > Then I create a abc.txt file on master 172.11.12.6 > [hadoop@master ~]$ pwd > /home/hadoop > [hadoop@master ~]$ echo "This is a test." >> abc.txt > [hadoop@master ~]$ hadoop dfs -copyFromLocal test.txt > [hadoop@master ~]$ hadoop dfs -ls > DEPRECATED: Use of this script to execute hdfs command is deprecated. > Instead use the hdfs command for it. > > 14/01/25 22:07:00 WARN util.NativeCodeLoader: Unable to load native-hadoop > library for your platform... using builtin-java classes where applicable > Found 1 items > -rw-r--r-- 2 hadoop supergroup 16 2014-01-25 21:36 abc.txt > > [hadoop@master ~]$ rm abc.txt > [hadoop@master ~]$ hadoop dfs -cat abc.txt > This is a test. > > My question is: > 1. Is supergroup a directory? Where does it locate? > 2. I search abc.txt on master 172.11.12.6 and node1 172.11.12.7 by > following command: > [hadoop@master ~]$ find / -name abc.txt > But I don't find abc.txt file. Where is the file abc.txt? After I erase it > by rm command, I still cat this file? Where is it? My OS is CentOS-5.8. > > Thanks. > > --------------------------------------------------------------------------------------------------- > Confidentiality Notice: The information contained in this e-mail and any > accompanying attachment(s) > is intended only for the use of the intended recipient and may be > confidential and/or privileged of > Neusoft Corporation, its subsidiaries and/or its affiliates. If any reader > of this communication is > not the intended recipient, unauthorized use, forwarding, printing, > storing, disclosure or copying > is strictly prohibited, and may be unlawful.If you have received this > communication in error,please > immediately notify the sender by return e-mail, and delete the original > message and all copies from > your system. Thank you. > > --------------------------------------------------------------------------------------------------- > > --------------------------------------------------------------------------------------------------- > Confidentiality Notice: The information contained in this e-mail and any > accompanying attachment(s) > is intended only for the use of the intended recipient and may be > confidential and/or privileged of > Neusoft Corporation, its subsidiaries and/or its affiliates. If any reader > of this communication is > not the intended recipient, unauthorized use, forwarding, printing, > storing, disclosure or copying > is strictly prohibited, and may be unlawful.If you have received this > communication in error,please > immediately notify the sender by return e-mail, and delete the original > message and all copies from > your system. Thank you. > > --------------------------------------------------------------------------------------------------- >
