HDFS development on eclipse

2013-09-29 Thread Karim Awara
Hi, is there a guide that guides me on how to run hdfs commands through eclipse for debugging? I was using this guide ( http://wiki.apache.org/hadoop/EclipseEnvironment), but it only mentions until importing a project only. -- Best Regards, Karim Ahmed Awara -- -- T

HDFS splitting file to 64MB blocks

2013-09-24 Thread Karim Awara
Hi, I want to know when I upload a file from the local disk to hdfs in a distributed environment (local cluster), the file gets split into blocks of 64MB each. assuming the file resides on the namenode, who splits the file (namenode)? what if the file resides on a datanode, does the scenario cha

Uploading a file to HDFS

2013-09-24 Thread Karim Awara
Hi, I have a couple of questions about the process of uploading a large file (> 10GB) to HDFS. To make sure my understanding is correct, assuming I have a cluster of N machines. - What happens in the following: Case 1: assuming i want to uppload a file (input.txt) of size K