thanks. i verified, datanode is up and running i ran the below command:
Administrator@DFS-DC /cygdrive/c/hadoop-1.1.2/hadoop-1.1.2/bin $ ./hadoop dfs -copyFromLocal C:\\Users\\Administrator\\Desktop\\hadoop-1.1.2.tar /wksp copyFromLocal: File C:/Users/Administrator/Desktop/hadoop-1.1.2.tar does not exist. it says , file does not exist. as my cluster is windows based , i dont know how dir path needs to be used i am using cygwin for linux style formatting i tried below as well : Administrator@DFS-DC /cygdrive/c/hadoop-1.1.2/hadoop-1.1.2/bin $ ./hadoop dfs -copyFromLocal /cygdrive/c/Users/Administrator/Desktop/hadoop-1.1.2.tar /wksp copyFromLocal: File /cygdrive/c/Users/Administrator/Desktop/hadoop-1.1.2.tar does not exist. Administrator@DFS-DC /cygdrive/c/hadoop-1.1.2/hadoop-1.1.2/bin $ ./hadoop dfs -copyFromLocal /cygdrive/c/Users/Administrator/Desktop/hadoop-1.1.2.tar.gz /wksp copyFromLocal: File /cygdrive/c/Users/Administrator/Desktop/hadoop-1.1.2.tar.gz does not exist. please suggest regards On Mon, Aug 5, 2013 at 6:00 PM, manish dunani <manishd...@gmail.com> wrote: > You can not physically access the datanode.You have to understand it to > logically and it really happens. > > Type "jps" command to check ur datanode was started or not. > > when user stores the file into hdfs ,the request is goes to datanode and > datanode will divide the file into number of blocks. > Each block size is 64mb or 128mb.but,default is 64 mb. > > The default replication factor is 1 that was u already set in ur > hdfs-site.xml .If u want to change ur replication factor from 1 to 3 for > particular file or particular directory in hdfs then use the below commands > appropriately. > > First load any file to hdfs using command: > > bin/hadoop dfs -copyFromLocal /ur/local/directory/path/to/file > /ur/hdfs/directory/path. > > *commands:* > > To set replication of an individual file to 4: > > ./bin/hadoop dfs -setrep -w 4 /path/to/file > > You can also do this recursively. To change replication of entire HDFS to > 1: > > ./bin/hadoop dfs -setrep -R -w 1 / > * > * > If the replication factor is 3 then data will divided into 3 blocks and > replicated to various data nodes of ur production cluster(multi node). > > Here Replication means same file copied three times on data nodes for > handling hardware failure. > > > On Mon, Aug 5, 2013 at 5:29 PM, Irfan Sayed <irfu.sa...@gmail.com> wrote: > >> thanks. >> please refer below: >> >> Administrator@DFS-DC /cygdrive/c/hadoop-1.1.2/hadoop-1.1.2/bin >> $ ./hadoop dfs -ls /wksp >> Found 1 items >> drwxr-xr-x - Administrator Domain 0 2013-08-05 16:58 /wksp/New >> folder >> >> Administrator@DFS-DC /cygdrive/c/hadoop-1.1.2/hadoop-1.1.2/bin >> $ >> >> >> same command if i run on the datanode then it says: >> >> Administrator@DFS-1 /cygdrive/c/hadoop-1.1.2/hadoop-1.1.2/bin >> $ ./hadoop dfs -ls /wksp >> ls: Cannot access /wksp: No such file or directory. >> >> does it mean that replication is not yet started ...? >> >> please suggest >> >> regards >> >> >> >> On Mon, Aug 5, 2013 at 5:18 PM, Mohammad Tariq <donta...@gmail.com>wrote: >> >>> You cannot physically see the HDFS files and directories through local >>> FS. Either use HDFS shell or HDFS webUI(namenode_machine:50070). >>> >>> Warm Regards, >>> Tariq >>> cloudfront.blogspot.com >>> >>> >>> On Mon, Aug 5, 2013 at 4:46 PM, Irfan Sayed <irfu.sa...@gmail.com>wrote: >>> >>>> thanks mohammad >>>> i ran the below command on NameNode >>>> >>>> $ ./hadoop dfs -mkdir /wksp >>>> >>>> and the "wksp" dir got created in c:\ ( as i have windows environment) >>>> >>>> now , when i log in to one of the DataNode , then i am not able to see >>>> c:\wksp >>>> >>>> any issue ? >>>> please suggest >>>> >>>> regards >>>> >>>> >>>> >>>> >>>> >>>> On Mon, Aug 5, 2013 at 3:03 PM, Mohammad Tariq <donta...@gmail.com>wrote: >>>> >>>>> Hello Irfan, >>>>> >>>>> You can find all the answers from HDFS architecture >>>>> guide<http://hadoop.apache.org/docs/stable/hdfs_design.html>. >>>>> See the section Data >>>>> Organization<http://hadoop.apache.org/docs/stable/hdfs_design.html#Data+Organization>in >>>>> particular for this question. >>>>> >>>>> Warm Regards, >>>>> Tariq >>>>> cloudfront.blogspot.com >>>>> >>>>> >>>>> On Mon, Aug 5, 2013 at 12:23 PM, Irfan Sayed <irfu.sa...@gmail.com>wrote: >>>>> >>>>>> hi, >>>>>> >>>>>> i have setup the two node apache hadoop cluster on windows environment >>>>>> one is namenode and another is datanode >>>>>> >>>>>> everything is working fine. >>>>>> one thing which i need to know is , how the replication starts >>>>>> if i create a.txt in namenode , how it will be appeared in datanodes >>>>>> >>>>>> please suggest >>>>>> >>>>>> regards >>>>>> >>>>>> >>>>> >>>> >>> >> > > > -- > MANISH DUNANI > -THANX > +91 9426881954,+91 8460656443 > manishd...@gmail.com >
