How to keep data consistency?

2014-02-19 Thread EdwardKing
Hadoop 2.2.0, two computer, one is master,another is node1. I want to know following scene: If node1 is down by some reason, but I don't know node1 can't work, then I use hadoop command to put a file,such as: $ hadoop fs -put graph.txtgraphin/graph.txt I know graph.txt file will be put master m

Re: How to keep data consistency?

2014-02-19 Thread Sergey Murylev
Hi Edward, You can't achieve data consistency on your cluster configuration. To do this you need at least 3 data nodes and enabled replication with level 3 ( dfs.replication property in hdfs-site.xml). On 19/02/14 13:02, EdwardKing wrote: > Hadoop 2.2.0, two computer, one is master,another is nod

Re: How to keep data consistency?

2014-02-19 Thread Devin Suiter RDX
Edward, It doesn't seem like your "hadoop -put ..." command will even complete - the master isn't receiving the file at any point. It instructs the node1 to connect to the client, after asking the node1 if it is in a state that it can receive data to be written, which depends on several other daem