Hi,
Current volume choosing policy is round robin fashion, Since the DN got new
disk, balancer will balance some blocks to this node. But the volume choosing
will be same when palcing the block. AFAIK, it wont do any special balancing
between disks in the same node. please correct me if i
Thanks for Help Joey!
Does just copying block files from one drive to another work?
Isn't there metadata maintained at datanode about block locations on that
datanode? If not, then how does datanode know about blocks stored on it?
-Ajit.
On Tue, Nov 22, 2011 at 5:25 PM, Joey Echeverria
This is an interesting usecase based on my understanding data nodes send
block information to name node so if you move the block files around old
data node should stop sending and new nodes would start sending. each block
is a seperate file.
it would be better to try this but i dont think this is
Ajit / Inder,
Please see
http://wiki.apache.org/hadoop/FAQ#On_an_individual_data_node.2C_how_do_you_balance_the_blocks_on_the_disk.3F
On Tue, Nov 22, 2011 at 9:44 PM, Ajit Ratnaparkhi
ajit.ratnapar...@gmail.com wrote:
Thanks for Help Joey!
Does just copying block files from one drive to
Thanks Harsh!
On Tue, Nov 22, 2011 at 10:05 PM, Harsh J ha...@cloudera.com wrote:
Ajit / Inder,
Please see
http://wiki.apache.org/hadoop/FAQ#On_an_individual_data_node.2C_how_do_you_balance_the_blocks_on_the_disk.3F
On Tue, Nov 22, 2011 at 9:44 PM, Ajit Ratnaparkhi
Sorry for this novice question. I am trying to find the best way of moving
(Copying) data in and out of HDFS. There are bunch of tools available and I
need to pick the one which offers the easiest way. I have seen MapR
presentation, who claim to offer direct NFS mounts to feed data into HDFS.
You can use the linux command hadoop fs -put to push files from local
filesystem, and -get to retrieve files
http://hadoop.apache.org/common/docs/r0.20.0/hdfs_shell.html#put
http://hadoop.apache.org/common/docs/r0.20.0/hdfs_shell.html#get
These work fine for single files or one-offs- if you need
Hi Steve,
You can use Chukwa (A Hadoop sub project that aims to provide a
flexible and powerful platform for distributed data collection).
Chukwa makes our data collection quite simple and efficient.
Regards,
Mohammad Tariq
On Wed, Nov 23, 2011 at 12:58 AM, Steve Ed sediso...@gmail.com
