Hello Group, I've hadoop setup locally running.
Now I want to use Amazon s3://<mybucket> as my data store, so i changed like " dfs.data.dir=s3://<mybucket>/hadoop/ " in my hdfs-site.xml, Is it the correct way? I'm getting error : WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Invalid directory in dfs.data.dir: can not create directory: s3://<mybucket>/hadoop 2012-07-23 13:15:06,260 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: All directories in dfs.data.dir are invalid. and when i changed like " dfs.data.dir=s3://<mybucket>/ " I got error : ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: java.lang.IllegalArgumentException: Wrong FS: s3://<mybucket>/, expected: file:/// at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:381) at org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:55) at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:393) at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:251) at org.apache.hadoop.util.DiskChecker.mkdirsWithExistsAndPermissionCheck(DiskChecker.java:146) at org.apache.hadoop.util.DiskChecker.checkDir(DiskChecker.java:162) at org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:1574) at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1521) at org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:1539) at org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:1665) at org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:1682) Also, When I'm changing fs.default.name=s3://<<mybucket> , Namenode is not coming up with error : ERROR org.apache.hadoop.hdfs.server.namenode.NameNode: java.net.BindException: (Any way I want to run namenode locally, so I reverted it back to hdfs://localhost:9000 ) Your help is highly appreciated! Thanks -- Alok Kumar