Re: Who splits the file into blocks

2013-03-31 Thread Harsh J
The client splits the file. The block size attribute is a per-file one, sent along with the file creation request. Blocks are requested as-it-goes, not pre-allocated. On Sun, Mar 31, 2013 at 2:15 PM, Sai Sai wrote: > Here is my understanding about putting a file into hdfs: > A client contacts nam

Re: Who splits the file into blocks

2013-03-31 Thread Rahul Bhattacharjee
I think what Sai was asking is when client asks namenode to give it a list of data nodes then how does the namenode knows as how many blocks would be required to store the entire file. I think the way it works is client requests the NN for list of blocks and then the client writes the first block

Re: Who splits the file into blocks

2013-03-31 Thread Jens Scheidtmann
Dear Sai Sai, "Hadoop, the definitive guide" says regarding default replica placement: - first replica is placed on the same node as the client (lowest bandwidth penalty). - second replica is placed off-rack, at a random node of the other rack (avoiding busy racks). - third replicate is placed on

Re: Who splits the file into blocks

2013-03-31 Thread Sai Sai
Here is my understanding about putting a file into hdfs: A client contacts name node and gets the location of blocks where it needs to put the blocks in data nodes. But before this how does the name node know how many blocks it needs to split a file into. Who splits the file is it the client itse