I found it similiar to HADDOP-3027(https://issues.apache.org/jira/browse/HADOOP-3027), but its error msg is: org.apache.hadoop.mapred.JobTracker: problem cleaning system directory: /tmp/hadoop/mapred/system
while mine is: org.apache.hadoop.mapred.JobTracker: problem cleaning system directory: null I don't know if its a key issue. On Mon, Apr 11, 2011 at 10:06 PM, 杨杰 <xtyang...@gmail.com> wrote: > HI, > > Some mr tasks upon small files had run on our hadoop cluster these > days, with not such high > work load. While when i check it tonight, the cluster refused to > response. So i restart the hdfs > & mapred. > > BUT unexcepted exceptions were thrown when starting~ and even the > "hadoop fs -ls" command > could not get any response. The system seems to "hang" there. > > btw: at start, there is a piece of message telling "22 port connection > timeout". I checked the > $HADOOP_SSH_OPT in hadoop-env.sh. There, connection time out is set to > 1, so i changed it > to 1000. Then the error disappeared, but the following exception goes on~ > > Thanks for help ! > > 2011-04-11 21:39:57,028 INFO org.apache.hadoop.mapred.JobTracker: > problem cleaning system directory: null > java.io.IOException: Call to job-tracker/192.168.14.210:8020 failed on > local exception: java.io.IOException: Connection reset by peer > at org.apache.hadoop.ipc.Client.wrapException(Client.java:774) > at org.apache.hadoop.ipc.Client.call(Client.java:742) > at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:220) > at $Proxy4.getProtocolVersion(Unknown Source) > at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:359) > at > org.apache.hadoop.hdfs.DFSClient.createRPCNamenode(DFSClient.java:105) > at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:177) > at > org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:82) > at > org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1373) > at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:66) > at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1385) > at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:191) > at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:95) > at org.apache.hadoop.mapred.JobTracker.<init>(JobTracker.java:1607) > at > org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:174) > at org.apache.hadoop.mapred.JobTracker.main(JobTracker.java:3528) > Caused by: java.io.IOException: Connection reset by peer > at sun.nio.ch.FileDispatcher.read0(Native Method) > at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:21) > at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:233) > at sun.nio.ch.IOUtil.read(IOUtil.java:206) > at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:236) > at > org.apache.hadoop.net.SocketInputStream$Reader.performIO(SocketInputStream.java:55) > at > org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142) > at > org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:155) > at > org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:128) > at java.io.FilterInputStream.read(FilterInputStream.java:116) > at > org.apache.hadoop.ipc.Client$Connection$PingInputStream.read(Client.java:276) > at java.io.BufferedInputStream.fill(BufferedInputStream.java:218) > at java.io.BufferedInputStream.read(BufferedInputStream.java:237) > at java.io.DataInputStream.readInt(DataInputStream.java:370) > at > org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:501) > at org.apache.hadoop.ipc.Client$Connection.run(Client.java:446) > > > -- > Yang Jie（杨杰） > hi.baidu.com/thinkdifferent > > Group of CLOUD, Xi'an Jiaotong University > Department of Computer Science and Technology, Xi’an Jiaotong University > > PHONE: 86 1346888 3723 > TEL: 86 29 82665263 EXT. 608 > MSN: xtyangjie2...@yahoo.com.cn > > freedom is not free. > -- Yang Jie（杨杰） hi.baidu.com/thinkdifferent Group of CLOUD, Xi'an Jiaotong University Department of Computer Science and Technology, Xi’an Jiaotong University PHONE: 86 1346888 3723 TEL: 86 29 82665263 EXT. 608 MSN: xtyangjie2...@yahoo.com.cn freedom is not free.
