Hi folks, (Resending to this group, sent to common-dev before, pretty sure that's for Hadoop internal development - sorry for that..)
I'm pretty stuck here. I've been researching for hours and I haven't made any forward progress on this one. I have a vmWare installation of Cloudera Hadoop 0.20. The following commands to create a directory and copy a file from the shared folder *work fine*, so I'm confident everything is setup correctly: [cloudera@localhost bin]$ hadoop fs -mkdir /user/cloudera/testdir [cloudera@localhost bin]$ hadoop fs -put /mnt/hgfs/shared_folder/file1.txt /user/cloudera/testdir/file1.txt The file shows up fine in the HDFS doing it this way on the Linux VM. *However*, when I try doing the equivalent operation in Java everything works great until I try to close() FSDataOutputStream. I'm left with the new directory and a zero byte size file. One suspicious thing is that the user is "admin" instead of "cloudera" which I haven't figured out why. Here is the error: 12/05/19 09:45:46 INFO hdfs.DFSClient: Exception in createBlockOutputStream 127.0.0.1:50010 java.net.ConnectException: Connection refused: no further information 12/05/19 09:45:46 INFO hdfs.DFSClient: Abandoning block blk_1931357292676354131_1068 12/05/19 09:45:46 INFO hdfs.DFSClient: Excluding datanode 127.0.0.1:50010 12/05/19 09:45:46 WARN hdfs.DFSClient: DataStreamer Exception: org.apache.hadoop.ipc.RemoteException: java.io.IOException: File /user/admin/testdir/file1.txt could only be replicated to 0 nodes, instead of 1 at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1533) at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:667) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) There are certainly lots of search references to "*could only be replicated to 0 nodes, instead of 1*" but chasing down those suggestions hasn't helped. I have run *jps* and* netstat* and that looks good. All services are running, all port seem to be good. The *health check* looks good, plenty of disk space, no failed nodes... Here is the java (it fails when it hits "fs.close()": import java.io.BufferedReader; import java.io.FileInputStream; import java.io.FileReader; import java.io.IOException; import org.apache.hadoop.conf.Configuration; import org.apache.hadoop.fs.FSDataInputStream; import org.apache.hadoop.fs.FSDataOutputStream; import org.apache.hadoop.fs.FileSystem; import org.apache.hadoop.fs.Path; public class TestFileTrans { public static void main(String[] args) { Configuration config = new Configuration(); config.addResource(new Path("c:/_bigdata/client_libs/core-site.xml")); config.addResource(new Path("c:/_bigdata/client_libs/hdfs-site.xml")); System.out.println("hadoop.tmp.dir: " + config.get("hadoop.tmp.dir")); try{ FileSystem dfs = FileSystem.get(config); // this will default to admin unless the workingDirectory is explicitly set.. System.out.println("HDFS Working Directory: " + dfs.getWorkingDirectory().toString()); String dirName = "testdir"; Path src = new Path(dfs.getWorkingDirectory()+"/"+dirName); dfs.mkdirs(src); System.out.println("HDFS Directory created: " + dfs.getWorkingDirectory().toString()); loadFile(dfs, src); }catch(IOException e){ System.out.println("Error" + e.getMessage()); } } private static void loadFile(FileSystem dfs, Path src) throws IOException{ FileInputStream fis = new FileInputStream("c:/_bigdata/shared_folder/file1.txt"); int len = fis.available(); byte[] btr = new byte[len]; fis.read(btr); FSDataOutputStream fs = dfs.create(new Path(src.toString() +"/file1.txt")); fs.write(btr); fs.flush(); fs.close(); } } Any help would be greatly appreciated!