Re: HDFS single datanode cluster issues

2013-11-07 Thread Colin McCabe
First of all, HDFS isn't really the right choice for single-node environments. I would recommend using LocalFileSystem in this case. If you're evaluating HDFS and only have one computer, it will really be better to run several VMs to see how it works, rather than running just one Datanode. You ar

Re: HDFS single datanode cluster issues

2013-10-30 Thread David Mankellow
We are mapping a 1:1 replication. We have tried setting dfs.client.block.write.replace-datanode-on-failure.enable to NEVER but it seems to be ignored. We have tried the following: === dfs.client.block.write.replace-datanode-on-failure.enable false dfs.client.block.write.replace-d

Re: HDFS single datanode cluster issues

2013-10-30 Thread Allan Wilson
Hi David How does your block replica count compare to the number of datanodes in your cluster? Anyway...I found this in the online doc. You may want to use the NEVER policy. dfs.client.block.write.replace-datanode-on-failure.enable true If there is a datanode/network failure in the write pipeli

HDFS single datanode cluster issues

2013-10-30 Thread David Mankellow
Hi all, We are having difficulty writing any logs to a HDFS cluster of less than 3 nodes. This has been since the update between cdh4.2 and 4.3 (4.4 is also the same). Has anything changed that may make this occur and is there anything that can be done to rectify the situation, so we can use