In order to do anything other than a tar transfer (which is a kludge, of course), you'll need to open up the relevant ports between the client and the hadoop cluster. I may miss a few here, but I believe these would include port 50010 for the datanodes and whatever port the namenode is listening on.
Once you've done this, install hadoop on your client machine (but don't start it) and use the command line tools directly from there. On Wed, Aug 27, 2008 at 12:59 AM, Victor Samoylov < [EMAIL PROTECTED]> wrote: > Jeff, > > Thanks for help, I want to clarify several details: > > 1. I know this way to import files to HDFS, but this is connected with > direct accessing HDFS nodes by user. > Does exist another way export all data files from data server side to > remote > HDFS nodes without tar invocation? > > 2. I've setup replication factor as 2. How to setup 50 GB size of FS on one > data node? > > Thanks, > Victor Samoylov > > On Wed, Aug 27, 2008 at 3:10 AM, Jeff Payne <[EMAIL PROTECTED]> wrote: > > > Victor: > > > > I think in your use case the best way to move the data into hadoop would > > either be to tar it up and move it to the same network the HDFS machines > > are > > on, untar it and then run... > > > > hadoop dfs -put /contents-path /dfs-path > > > > If you only want a replication factor of 2 (the default is 3), open up > the > > hadoop.site.xml file and add this snippet... > > > > <property> > > <name>dfs.replication</name> > > <value>2</value> > > </property> > > > > -- > > Jeffrey Payne > > Lead Software Engineer > > Eyealike, Inc. > > [EMAIL PROTECTED] > > www.eyealike.com > > (206) 257-8708 > > > > > > "Anything worth doing is worth overdoing." > > -H. Lifter > > > > On Tue, Aug 26, 2008 at 2:54 PM, Victor Samoylov < > > [EMAIL PROTECTED] > > > wrote: > > > > > Hi, > > > > > > I want to use HDFS as DFS to store files. I have one data server with > > 50Gb > > > data and I plan to use 3 new machines with installed HDFS to duplicate > > this > > > data. > > > These 3 machines are: 1 name node, 2 data nodes. The duplication factor > > for > > > all files is 2. > > > > > > My questions are: > > > 1. How could I create 50 GB data node on one server? Actually I'm very > > > insteresting with setting 50 GB size for data node. > > > 2. What is the best way to export all data files from external server > > (ssh > > > access) to new ones with HDFS? > > > > > > Thanks, > > > Victor Samoylov > > > > > > -- Jeffrey Payne Lead Software Engineer Eyealike, Inc. [EMAIL PROTECTED] www.eyealike.com (206) 257-8708 "Anything worth doing is worth overdoing." -H. Lifter
