Re: Uploading a file to HDFS

2013-10-01 Thread Jay Vyas
> > HTH > Ravi > > > -- > *From:* Karim Awara > *To:* user > *Sent:* Thursday, September 26, 2013 7:51 AM > *Subject:* Re: Uploading a file to HDFS > > > Thanks for the reply. when the client caches 64KB of data on its own side, > do you know which set o

Re: Uploading a file to HDFS

2013-10-01 Thread Ravi Prakash
Karim! Look at DFSOutputStream.java:DataStreamer HTH Ravi From: Karim Awara To: user Sent: Thursday, September 26, 2013 7:51 AM Subject: Re: Uploading a file to HDFS Thanks for the reply. when the client caches 64KB of data on its own side, do you

Re: Uploading a file to HDFS

2013-09-26 Thread Karim Awara
Thanks for the reply. when the client caches 64KB of data on its own side, do you know which set of major java classes/files responsible for such action? -- Best Regards, Karim Ahmed Awara On Thu, Sep 26, 2013 at 2:25 PM, Jitendra Yadav wrote: > Case 2: > > While selecting target DN in case of

Re: Uploading a file to HDFS

2013-09-26 Thread Jitendra Yadav
Case 2: While selecting target DN in case of write operations, NN will always prefers first DN as same DN from where client sending the data, in some cases NN ignore that DN when there is some disk space issues or some other health symptoms found,rest of things will same. Thanks Jitendra On Th

Re: Uploading a file to HDFS

2013-09-26 Thread Shekhar Sharma
Its not the namenode that does the reading or breaking of the file.. When you run the command hadoop fs -put . Here "hadoop" is a script file which is default client for hadoop..and when the client contacts the namenode for writing, then NN creates a block id and ask 3 dN to host the block ( r

Uploading a file to HDFS

2013-09-26 Thread Karim Awara
Hi, I have a couple of questions about the process of uploading a large file (> 10GB) to HDFS. To make sure my understanding is correct, assuming I have a cluster of N machines. - What happens in the following: Case 1: assuming i want to uppload a file (input.txt) of size K