Re: dfs put fails
Got a similar error when doing a mapreduce job on the master machine. Mapping job is ok and in the end there are the right results in my output folder, but the reduce hangs at 17% a very long time. Found this in one of the task logs a view times: ... 2008-06-18 17:31:02,297 INFO org.apache.hadoop.mapred.ReduceTask: task_200806181716_0001_r_00_0: Got 0 new map-outputs & 0 obsolete map-outputs from tasktracker and 0 map-outputs from previous failures 2008-06-18 17:31:02,297 INFO org.apache.hadoop.mapred.ReduceTask: task_200806181716_0001_r_00_0 Got 0 known map output location(s); scheduling... 2008-06-18 17:31:02,297 INFO org.apache.hadoop.mapred.ReduceTask: task_200806181716_0001_r_00_0 Scheduled 0 of 0 known outputs (0 slow hosts and 0 dup hosts) 2008-06-18 17:31:03,276 WARN org.apache.hadoop.mapred.ReduceTask: task_200806181716_0001_r_00_0 copy failed: task_200806181716_0001_m_01_0 from koeln 2008-06-18 17:31:03,276 WARN org.apache.hadoop.mapred.ReduceTask: java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333) at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195) at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182) at java.net.Socket.connect(Socket.java:519) at sun.net.NetworkClient.doConnect(NetworkClient.java:152) at sun.net.www.http.HttpClient.openServer(HttpClient.java:394) at sun.net.www.http.HttpClient.openServer(HttpClient.java:529) at sun.net.www.http.HttpClient.(HttpClient.java:233) at sun.net.www.http.HttpClient.New(HttpClient.java:306) at sun.net.www.http.HttpClient.New(HttpClient.java:323) at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:788) at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:729) at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:654) at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:977) at org.apache.hadoop.mapred.MapOutputLocation.getFile(MapOutputLocation.java:139) at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$MapOutputCopier.copyOutput(ReduceTask.java:815) at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$MapOutputCopier.run(ReduceTask.java:764) 2008-06-18 17:31:03,276 INFO org.apache.hadoop.mapred.ReduceTask: Task task_200806181716_0001_r_00_0: Failed fetch #7 from task_200806181716_0001_m_01_0 2008-06-18 17:31:03,276 INFO org.apache.hadoop.mapred.ReduceTask: Failed to fetch map-output from task_200806181716_0001_m_01_0 even after MAX_FETCH_RETRIES_PER_MAP retries... reporting to the JobTracker 2008-06-18 17:31:03,276 WARN org.apache.hadoop.mapred.ReduceTask: task_200806181716_0001_r_00_0 adding host koeln to penalty box, next contact in 150 seconds 2008-06-18 17:31:03,277 INFO org.apache.hadoop.mapred.ReduceTask: task_200806181716_0001_r_00_0 Need 1 map output(s) 2008-06-18 17:31:03,317 INFO org.apache.hadoop.mapred.ReduceTask: task_200806181716_0001_r_00_0: Got 0 new map-outputs & 0 obsolete map-outputs from tasktracker and 1 map-outputs from previous failures 2008-06-18 17:31:03,317 INFO org.apache.hadoop.mapred.ReduceTask: task_200806181716_0001_r_00_0 Got 1 known map output location(s); scheduling... 2008-06-18 17:31:03,317 INFO org.apache.hadoop.mapred.ReduceTask: task_200806181716_0001_r_00_0 Scheduled 0 of 1 known outputs (1 slow hosts and 0 dup hosts) 2008-06-18 17:31:08,336 INFO org.apache.hadoop.mapred.ReduceTask: task_200806181716_0001_r_00_0 Need 1 map output(s) 2008-06-18 17:31:08,337 INFO org.apache.hadoop.mapred.ReduceTask: task_200806181716_0001_r_00_0: Got 0 new map-outputs & 0 obsolete map-outputs from tasktracker and 0 map-outputs from previous failures 2008-06-18 17:31:08,337 INFO org.apache.hadoop.mapred.ReduceTask: task_200806181716_0001_r_00_0 Got 1 known map output location(s); scheduling... 2008-06-18 17:31:08,337 INFO org.apache.hadoop.mapred.ReduceTask: task_200806181716_0001_r_00_0 Scheduled 0 of 1 known outputs (1 slow hosts and 0 dup hosts) 2008-06-18 17:31:13,356 INFO org.apache.hadoop.mapred.ReduceTask: task_200806181716_0001_r_00_0 Need 1 map output(s) ... Did i forget to open some ports? i opened 50010 for datanode and the ports for dfs and jobtracker as specified in hadoop-site.xml. If its a firewall problem, woudnt hadoop recognize that at startup, i.e. that connections would be refused? On Wed, 2008-06-18 at 11:32 +0200, Alexander Arimond wrote: > Thank you, first tried the put from the master machine, which leads to > the error. The put from the slave machine works. Guess youre right with > the configuration parameters. Appears a bit strange to me, because the > firewall
Re: dfs put fails
Thank you, first tried the put from the master machine, which leads to the error. The put from the slave machine works. Guess youre right with the configuration parameters. Appears a bit strange to me, because the firewall settings and the hadoop-site.xml on both machines are equal. On Tue, 2008-06-17 at 14:08 -0700, Konstantin Shvachko wrote: > Looks like the client machine from which you call -put cannot connect to the > data-nodes. > It could be firewall or wrong configuration parameters that you use for the > client. > > Alexander Arimond wrote: > > hi, > > > > i'm new in hadoop and im just testing it at the moment. > > i set up a cluster with 2 nodes and it seems like they are running > > normally, > > the log files of the namenode and the datanodes dont show errors. > > Firewall should be set right. > > but when i try to upload a file to the dfs i get following message: > > > > [EMAIL PROTECTED]:~/hadoop$ bin/hadoop dfs -put file.txt file.txt > > 08/06/12 14:44:19 INFO dfs.DFSClient: Exception in > > createBlockOutputStream java.net.ConnectException: Connection refused > > 08/06/12 14:44:19 INFO dfs.DFSClient: Abandoning block > > blk_5837981856060447217 > > 08/06/12 14:44:28 INFO dfs.DFSClient: Exception in > > createBlockOutputStream java.net.ConnectException: Connection refused > > 08/06/12 14:44:28 INFO dfs.DFSClient: Abandoning block > > blk_2573458924311304120 > > 08/06/12 14:44:37 INFO dfs.DFSClient: Exception in > > createBlockOutputStream java.net.ConnectException: Connection refused > > 08/06/12 14:44:37 INFO dfs.DFSClient: Abandoning block > > blk_1207459436305221119 > > 08/06/12 14:44:46 INFO dfs.DFSClient: Exception in > > createBlockOutputStream java.net.ConnectException: Connection refused > > 08/06/12 14:44:46 INFO dfs.DFSClient: Abandoning block > > blk_-8263828216969765661 > > 08/06/12 14:44:52 WARN dfs.DFSClient: DataStreamer Exception: > > java.io.IOException: Unable to create new block. > > 08/06/12 14:44:52 WARN dfs.DFSClient: Error Recovery for block > > blk_-8263828216969765661 bad datanode[0] > > > > > > dont know what that means and didnt found something about that.. > > Hope somebody can help with that. > > > > Thank you! > > > > >
dfs copyFromLocal/put fails
hi, i'm new in hadoop and im just testing it at the moment. i set up a cluster with 2 nodes and it seems like they are running normally, the log files of the namenode and the datanodes dont show errors. Firewall should be set right. but when i try to upload a file to the dfs i get following message: [EMAIL PROTECTED]:~/hadoop$ bin/hadoop dfs -put file.txt file.txt 08/06/12 14:44:19 INFO dfs.DFSClient: Exception in createBlockOutputStream java.net.ConnectException: Connection refused 08/06/12 14:44:19 INFO dfs.DFSClient: Abandoning block blk_5837981856060447217 08/06/12 14:44:28 INFO dfs.DFSClient: Exception in createBlockOutputStream java.net.ConnectException: Connection refused 08/06/12 14:44:28 INFO dfs.DFSClient: Abandoning block blk_2573458924311304120 08/06/12 14:44:37 INFO dfs.DFSClient: Exception in createBlockOutputStream java.net.ConnectException: Connection refused 08/06/12 14:44:37 INFO dfs.DFSClient: Abandoning block blk_1207459436305221119 08/06/12 14:44:46 INFO dfs.DFSClient: Exception in createBlockOutputStream java.net.ConnectException: Connection refused 08/06/12 14:44:46 INFO dfs.DFSClient: Abandoning block blk_-8263828216969765661 08/06/12 14:44:52 WARN dfs.DFSClient: DataStreamer Exception: java.io.IOException: Unable to create new block. 08/06/12 14:44:52 WARN dfs.DFSClient: Error Recovery for block blk_-8263828216969765661 bad datanode[0] dont know what that means and didnt found something about that.. Hope somebody can help with that. Thank you!
dfs put fails
hi, i'm new in hadoop and im just testing it at the moment. i set up a cluster with 2 nodes and it seems like they are running normally, the log files of the namenode and the datanodes dont show errors. Firewall should be set right. but when i try to upload a file to the dfs i get following message: [EMAIL PROTECTED]:~/hadoop$ bin/hadoop dfs -put file.txt file.txt 08/06/12 14:44:19 INFO dfs.DFSClient: Exception in createBlockOutputStream java.net.ConnectException: Connection refused 08/06/12 14:44:19 INFO dfs.DFSClient: Abandoning block blk_5837981856060447217 08/06/12 14:44:28 INFO dfs.DFSClient: Exception in createBlockOutputStream java.net.ConnectException: Connection refused 08/06/12 14:44:28 INFO dfs.DFSClient: Abandoning block blk_2573458924311304120 08/06/12 14:44:37 INFO dfs.DFSClient: Exception in createBlockOutputStream java.net.ConnectException: Connection refused 08/06/12 14:44:37 INFO dfs.DFSClient: Abandoning block blk_1207459436305221119 08/06/12 14:44:46 INFO dfs.DFSClient: Exception in createBlockOutputStream java.net.ConnectException: Connection refused 08/06/12 14:44:46 INFO dfs.DFSClient: Abandoning block blk_-8263828216969765661 08/06/12 14:44:52 WARN dfs.DFSClient: DataStreamer Exception: java.io.IOException: Unable to create new block. 08/06/12 14:44:52 WARN dfs.DFSClient: Error Recovery for block blk_-8263828216969765661 bad datanode[0] dont know what that means and didnt found something about that.. Hope somebody can help with that. Thank you!
