Hi Bill, Thanks for that. If the NameNode is unavailable, how do we find the secondary name node? Is there a way to deal with this in the code or should a load balancer of some type sit above each and only direct traffic to the name node if its listening?
-sd On Mon, May 18, 2009 at 2:09 PM, Bill Habermaas <b...@habermaas.us> wrote: > Sasha, > > Connecting to the namenode is the proper way to establish the hdfs > connection. Afterwards the Hadoop client handler that is called by your > code will go directly to the datanodes. There is no reason for you to > communicate directly with a datanode nor is there a way for you to even know > where the data nodes are located. That is all done by the Hadoop client code > and done silently under the covers by Hadoop itself. > > Bill > > -----Original Message----- > From: sdo...@gmail.com [mailto:sdo...@gmail.com] On Behalf Of Sasha Dolgy > Sent: Sunday, May 17, 2009 10:55 AM > To: core-user@hadoop.apache.org > Subject: proper method for writing files to hdfs > > The following graphic outlines the architecture for HDFS: > http://hadoop.apache.org/core/docs/current/images/hdfsarchitecture.gif > > If one is to write a client that adds data into HDFS, it needs to add it > through the Data Node. Now, from the graphic I am to understand that the > client doesn't communicate with the NameNode, and only the Data Node. > > In the examples I've seen and the playing I am doing, I am connecting to the > hdfs url as a configuration parameter before I create a file. Is this the > incorrect way to create files in HDFS? > > Configuration config = new Configuration(); > config.set("fs.default.name","hdfs://foo.bar.com:9000/"); > String path = "/tmp/i/am/a/path/to/a/file.name" > Path hdfsPath = new Path(path); > FileSystem fileSystem = FileSystem.get(config); > FSDataOutputStream os = fileSystem.create(hdfsPath, false); > os.write("something".getBytes()); > os.close(); > > Should the client be connecting to a data node to create the file as > indicated in the graphic above? > > If connecting to a data node is possible and suggested, where can I find > more details about this process? > > Thanks in advance, > -sasha > > -- > Sasha Dolgy > sasha.do...@gmail.com > > > -- Sasha Dolgy sasha.do...@gmail.com
