Hi - I’m seeing the following exception while trying to start MiniDFSCluster in some environment. The stack trace is:
Starting DataNode 0 with dfs.data.dir: build/test/data/dfs/data/data1,build/test/data/dfs/data/data2 java.lang.NullPointerException at org.apache.hadoop.hdfs.MiniDFSCluster.startDataNodes(MiniDFSCluster.java:413) at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:274) at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:122) at com.yahoo.ads.ngdstone.tpbdm.BDMTestCase.oneTimeSetUp(BDMTestCase.java:69) My code is: System.setProperty("hadoop.log.dir", "/tmp"); System.setProperty("dfs.permissions.supergroup", "su"); conf = new Configuration(); dfsCluster = new MiniDFSCluster(conf, 1, true, null); // <== Line 69 in BDMTestCase.java fs = dfsCluster.getFileSystem(); ... Line 413 in MiniDFSCluster is: String ipAddr = dn.getSelfAddr().getAddress().getHostAddress(); So I’m guessing it has something to do with the network setup?? - I see just above in MiniDFSCluster: conf.set("dfs.datanode.address", "127.0.0.1:0"); conf.set("dfs.datanode.http.address", "127.0.0.1:0"); conf.set("dfs.datanode.ipc.address", "127.0.0.1:0"); Do I need to have those ports available/enabled in my environment? Is that the problem here? Thanks! Frank