RE: moving data

2012-08-01 Thread Dave Shine
>From the command line you can hadoop fs -copyFromLocal Dave Shine Sr. Software Engineer 321.939.5093 direct | 407.314.0122 mobile CI Boost(tm) Clients Outperform Online(tm) www.ciboost.com From: Soheila Dehghanzadeh [mailto:sally...@gmail.com] Sent: Wednesday, Augu

Re: moving data

2012-08-01 Thread Soheila Dehghanzadeh
Great! thanks :) On Wed, Aug 1, 2012 at 1:12 PM, Dave Shine < dave.sh...@channelintelligence.com> wrote: > From the command line you can hadoop fs –copyFromLocal > > > ** ** > > *Dave Shine* > > Sr. Software Engineer > > 321.939.5093 direct | 407.314.0122 mobile > > *CI Boost™

Re: Moving data into HDFS

2011-11-22 Thread Chris Shain
You can use the linux command hadoop fs -put to push files from local filesystem, and -get to retrieve files http://hadoop.apache.org/common/docs/r0.20.0/hdfs_shell.html#put http://hadoop.apache.org/common/docs/r0.20.0/hdfs_shell.html#get These work fine for single files or one-offs- if you need

Re: Moving data into HDFS

2011-11-22 Thread Mohammad Tariq
Hi Steve, You can use Chukwa (A Hadoop sub project that aims to provide a flexible and powerful platform for distributed data collection). Chukwa makes our data collection quite simple and efficient. Regards,     Mohammad Tariq On Wed, Nov 23, 2011 at 12:58 AM, Steve Ed wrote: > Sorry for

Re: Moving data from a remote machine into the HDFS

2012-01-24 Thread Ronald Petty
Mohammed, Is this remote machine part of the HDFS system? Ron On Tue, Jan 24, 2012 at 7:30 AM, Mohammad Tariq wrote: > Hello list, > >I have a situation wherein I have to move large binary files(~TB) > from remote machines into the HDFS.While looking for some way to do > this I came across

Re: Moving data from a remote machine into the HDFS

2012-01-24 Thread Mohammad Tariq
Hey Ron, Thanks for the response.No, the remote machine is not a part of our Hadoop ecosystem. Regards,     Mohammad Tariq On Tue, Jan 24, 2012 at 10:23 PM, Ronald Petty wrote: > Mohammed, > > Is this remote machine part of the HDFS system? > > Ron > > > On Tue, Jan 24, 2012 at 7:30 AM, Mo

Re: Moving data from a remote machine into the HDFS

2012-01-24 Thread Harsh J
You have two ways: A. If remote node has access to all HDFS machines (NN + all DNs). Simply do a "hadoop dfs -put" operation to push in data. B. If remote node has no access to HDFS, setup a bastion box with Hoop and write to HDFS via Hoop. Hoop provides a REST API to do this. Some examples to

Re: Moving data from a remote machine into the HDFS

2012-01-24 Thread Mohammad Tariq
Hello Harsh, Thanks a lot for the valuable reply.I have to go with the 2nd option as there is no access for the remote machine to our HDFS.I'll go through the link specified by you and act accordingly. Regards,     Mohammad Tariq On Wed, Jan 25, 2012 at 12:49 AM, Harsh J wrote: > You have t