Hi Hadoop folks:
I have a 8 nodes cluster and use copyFromLocal to write some text workload
into dfs. I tried to write it on different nodes in the same cluster. That
is ,I log on to each machine in the same cluster after hadoop boots. In each
machine, I use: bin/hadoop dfs -copyFromLocal workload.txt. I was expecting
in this way, each node's local disk will be reduce because its contributed
disk space for dfs is consumed.
However, after trying a few times, I found that only the disk space of
namenode is consumed whether or not copyFromLocal is doing from namenode
machine or data node machines. Thus, it seems to have a bottleneck on the
total bytes that can write in dfs. Is this a design setting of hadoop or
there some ways to let the workload spread in non-name node's disk?
Thanks.
Richard

Reply via email to