Were other region servers functioning normally around 17:33 ? Which hbase release are you using ?
Can you pastebin more of the region server log ? Thanks On Fri, Oct 23, 2015 at 8:28 AM, 聪聪 <175998...@qq.com> wrote: > hi，all： > > > This afternoon,The whole Hbase cluster is suddenly unable to respond.after > I restart a regionserver after，the cluster has recovered.I don't know the > cause of the trouble.I hope I can get help from you. > > > Regionserver on the log is as follows： > 2015-10-23 17:28:49,335 INFO [regionserver60020.logRoller] wal.FSHLog: > moving old hlog file /hbase/WALs/l-hbase30.data.cn8.qunar.com > ,60020,1442810406218/l-hbase30.data.cn8.qunar.com%2C60020%2C1442810406218.1445580462689 > whose highest sequenceid is 9071525521 to /hbase/oldWALs/ > l-hbase30.data.cn8.qunar.com%2C60020%2C1442810406218.1445580462689 > 2015-10-23 17:33:31,375 WARN [RpcServer.reader=8,port=60020] > ipc.RpcServer: RpcServer.listener,port=60020: count of bytes read: 0 > java.io.IOException: Connection reset by peer > at sun.nio.ch.FileDispatcherImpl.read0(Native Method) > at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) > at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223) > at sun.nio.ch.IOUtil.read(IOUtil.java:197) > at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:379) > at > org.apache.hadoop.hbase.ipc.RpcServer.channelRead(RpcServer.java:2368) > at > org.apache.hadoop.hbase.ipc.RpcServer$Connection.readAndProcess(RpcServer.java:1403) > at > org.apache.hadoop.hbase.ipc.RpcServer$Listener.doRead(RpcServer.java:770) > at > org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:563) > at > org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:538) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:744) > 2015-10-23 17:33:31,779 WARN [RpcServer.reader=2,port=60020] > ipc.RpcServer: RpcServer.listener,port=60020: count of bytes read: 0 > java.io.IOException: Connection reset by peer > at sun.nio.ch.FileDispatcherImpl.read0(Native Method) > at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) > at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223) > at sun.nio.ch.IOUtil.read(IOUtil.java:197) > at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:379) > at > org.apache.hadoop.hbase.ipc.RpcServer.channelRead(RpcServer.java:2368) > at > org.apache.hadoop.hbase.ipc.RpcServer$Connection.readAndProcess(RpcServer.java:1403) > at > org.apache.hadoop.hbase.ipc.RpcServer$Listener.doRead(RpcServer.java:770) > at > org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:563) > at > org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:538) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:744)
