Push jars into userchache without running a MR job

2016-03-04 Thread Shashank Prabhakara
Hi all, I have some job dependency jars that need to be pushed to all the node managers' usercache ahead of the job that will be using it. I understand that the job will distribute the jars at runtime, but if possible I want to have it done beforehand. Thanks in advance. Regards, Shashank

RE: change HDFS disks on each node

2016-03-04 Thread Joseph Naegele
Thanks Anu, that makes sense. Assuming that works, allow me to complicate the problem a bit further: Say I didn't use LVM and I needed to move from 2x3 TB volumes to 6x1 TB volumes. The problem now is that the contents of each 3TB volume won't fit on a single 1TB volume. Is this still

Re: change HDFS disks on each node

2016-03-04 Thread Anu Engineer
Yes, it is possible, but If I were you and someone asked me to do it, I would say No. May I suggest that you decommission one node, let the data migrate to other machines, replace the disks and bring that machine back in and let the cluster heal itself. Not as efficient as moving data around

Re: change HDFS disks on each node

2016-03-04 Thread Anu Engineer
Hi Joe, As long as you copy all the data in the old disk without altering the paths ( ie. Structure and layout of the data dirs) and you use the same version of datanode software then it should work. Here is the Apache documentation that implies that this will work,

change HDFS disks on each node

2016-03-04 Thread Joseph Naegele
Hi all, Each of our N datanodes has attached two 3TB disks. I want to attach new *replacement* storage to each node, move the HDFS contents to the new storage, and remove the old volumes. We're using Hadoop 2.7.1. 1. What is the simplest, correct way to do this? Does hot-swapping move data from

Hadoop 2.8 Release Data

2016-03-04 Thread Benjamin Kim
I have a general question about Hadoop 2.8. Is it being prepped for release anytime soon? I am awaiting HADOOP-5732 bringing SFTP support natively. Thanks, Ben - To unsubscribe, e-mail: user-unsubscr...@hadoop.apache.org For

Error While copying file from local to dfs

2016-03-04 Thread Vinodh Nagaraj
Hi All, I am new bee to Hadoop. I installed hadoop 2.7.1 on windows 32 bit machine ( windows 7 ) for learning purpose. I can execute start-all.cmd successfully. When i execute jps,i got the below output. 28544 NameNode 35728 36308 DataNode 43828 Jps 40688 NodeManager 33820 ResourceManager My