Can you check $HADOOP_HOME/logs/userlogs/ subdirectories for attempt
logs. They can contain map-reduce related errors.

2009/10/16 Stephane Brossier <[email protected]>:
> I installed hadoop on my workstation today (pseudo distributed mode) using
> the instructions:
> http://hadoop.apache.org/common/docs/r0.19.1/quickstart.html#Download
>
> The installation is straightforward, but then when i try to run the example,
> all the map tasks complete,
> but the reduce task never completes. It looks like there is some connection
> problems??
>
> I made sure that the user 'viking' running both the hadoop java processes
> and the job can ssh into the machine
> without needed keys/passphrase.
>
>
> From looking at the code, in ReduceTask.java, it looks like the reduce task
> is trying to contact the TaskTracker, but from the syslog below,
> that seems to fail:
>   URL mapOutputLocation = new URL(event.getTaskTrackerHttp() +
>                                    "/mapOutput?job=" + taskId.getJobID() +
>                                    "&map=" + taskId +
>                                    "&reduce=" + getPartition());
>
> My TaskTracker is listening on port 50060 and i can access the url.
>
> What can be wrong???
>
>
> Thanks,
>
> S.
>
>
> # Test ran:
> vik...@stephane-desktop:~/hadoop$ hadoop jar hadoop-*-examples.jar grep
> input output 'dfs[a-z.]+'
> 09/10/15 18:50:00 INFO mapred.FileInputFormat: Total input paths to process
> : 10
> 09/10/15 18:50:00 INFO mapred.JobClient: Running job: job_200910151506_0001
> 09/10/15 18:50:01 INFO mapred.JobClient:  map 0% reduce 0%
> 09/10/15 18:50:07 INFO mapred.JobClient:  map 9% reduce 0%
> 09/10/15 18:50:08 INFO mapred.JobClient:  map 18% reduce 0%
> 09/10/15 18:50:09 INFO mapred.JobClient:  map 27% reduce 0%
> 09/10/15 18:50:10 INFO mapred.JobClient:  map 36% reduce 0%
> 09/10/15 18:50:11 INFO mapred.JobClient:  map 54% reduce 0%
> 09/10/15 18:50:12 INFO mapred.JobClient:  map 63% reduce 0%
> 09/10/15 18:50:13 INFO mapred.JobClient:  map 72% reduce 0%
> 09/10/15 18:50:14 INFO mapred.JobClient:  map 81% reduce 0%
> 09/10/15 18:50:15 INFO mapred.JobClient:  map 90% reduce 0%
> 09/10/15 18:50:16 INFO mapred.JobClient:  map 100% reduce 0%
>
> # Syslog from the reduce task
> syslog logs
> 2009-10-15 18:50:16,212 INFO org.apache.hadoop.metrics.jvm.JvmMetrics:
> Initializing JVM Metrics with processName=SHUFFLE, sessionId=
> 2009-10-15 18:50:16,281 INFO org.apache.hadoop.mapred.ReduceTask:
> ShuffleRamManager: MemoryLimit=668126400, MaxSingleShuffleLimit=167031600
> 2009-10-15 18:50:16,287 INFO org.apache.hadoop.mapred.ReduceTask:
> attempt_200910151506_0001_r_000000_0 Thread started: Thread for merging
> on-disk files
> 2009-10-15 18:50:16,287 INFO org.apache.hadoop.mapred.ReduceTask:
> attempt_200910151506_0001_r_000000_0 Thread started: Thread for merging in
> memory files
> 2009-10-15 18:50:16,287 INFO org.apache.hadoop.mapred.ReduceTask:
> attempt_200910151506_0001_r_000000_0 Thread waiting: Thread for merging
> on-disk files
> 2009-10-15 18:50:16,288 INFO org.apache.hadoop.mapred.ReduceTask:
> attempt_200910151506_0001_r_000000_0 Need another 11 map output(s) where 0
> is already in progress
> 2009-10-15 18:50:16,295 INFO org.apache.hadoop.mapred.ReduceTask:
> attempt_200910151506_0001_r_000000_0: Got 11 new map-outputs
> 2009-10-15 18:50:16,295 INFO org.apache.hadoop.mapred.ReduceTask:
> attempt_200910151506_0001_r_000000_0 Scheduled 1 outputs (0 slow hosts and0
> dup hosts)
> 2009-10-15 18:50:46,354 WARN org.apache.hadoop.mapred.ReduceTask:
> attempt_200910151506_0001_r_000000_0 copy failed:
> attempt_200910151506_0001_m_000000_0 from stephane-desktop
> 2009-10-15 18:50:46,355 WARN org.apache.hadoop.mapred.ReduceTask:
> java.net.SocketTimeoutException: connect timed out
>        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
>        at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
>        at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
>        at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
>        at
> sun.net.www.protocol.http.HttpURLConnection$6.run(HttpURLConnection.java:1360)
>        at java.security.AccessController.doPrivileged(Native Method)
>        at
> sun.net.www.protocol.http.HttpURLConnection.getChainedException(HttpURLConnection.java:1354)
>        at
> sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1008)
>        at
> org.apache.hadoop.mapred.ReduceTask$ReduceCopier$MapOutputCopier.getInputStream(ReduceTask.java:1293)
>        at
> org.apache.hadoop.mapred.ReduceTask$ReduceCopier$MapOutputCopier.getMapOutput(ReduceTask.java:1231)
>        at
> org.apache.hadoop.mapred.ReduceTask$ReduceCopier$MapOutputCopier.copyOutput(ReduceTask.java:1144)
>        at
> org.apache.hadoop.mapred.ReduceTask$ReduceCopier$MapOutputCopier.run(ReduceTask.java:1084)
> Caused by: java.net.SocketTimeoutException: connect timed out
>        at java.net.PlainSocketImpl.socketConnect(Native Method)
>        at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
>        at
> java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
>        at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
>        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
>        at java.net.Socket.connect(Socket.java:519)
>        at sun.net.NetworkClient.doConnect(NetworkClient.java:158)
>        at sun.net.www.http.HttpClient.openServer(HttpClient.java:394)
>        at sun.net.www.http.HttpClient.openServer(HttpClient.java:529)
>        at sun.net.www.http.HttpClient.<init>(HttpClient.java:233)
>        at sun.net.www.http.HttpClient.New(HttpClient.java:306)
>        at sun.net.www.http.HttpClient.New(HttpClient.java:323)
>        at
> sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:852)
>        at
> sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:793)
>        at
> sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:718)
>        at
> sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1041)
>        ... 4 more
>
>

Reply via email to