Re: Re: Hbase Dead region Server

2014-11-04 Thread Nishanth S
Thanks Every one.It turned out that I there were a few empty wal
directories corresponding to the dead region servers.Moved them out of
/hbase and  failed over the master.Things started working fine after that.

-Nishanth

On Mon, Nov 3, 2014 at 10:25 PM, yeweichen2...@gmail.com <
yeweichen2...@gmail.com> wrote:

> Nishanth,
>   What version of HBase you are using?
>
>   You can try clear the ZNode about regionserver list in zookeeper
> /hbase/ and then restart HMaster.
>
> --
> yeweichen2...@gmail.com
>
>
> *From:* Nishanth S 
> *Date:* 2014-11-04 02:32
> *To:* user 
> *Subject:* Re: Hbase Dead region Server
> Thanks Pere. I just did that and still  has the dead region server  showing
> up in Master UI as well as  in status command.I have replication turned on
>  in hbase and seeing few issues.Below is the stack trace I am seeing.
>
> 2014-11-03 18:31:00,215 WARN
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Can't
> replicate because of a local or network error:
> java.io.IOException: No replication sinks are available
> at
>
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSinkManager.getReplicationSink(ReplicationSinkManager.java:117)
> at
>
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.shipEdits(ReplicationSource.java:652)
> at
>
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.run(ReplicationSource.java:350)
> 2014-11-03 18:31:00,459 WARN
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Can't
> replicate because of a local or network error:
> java.io.IOException: No replication sinks are available
> at
>
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSinkManager.getReplicationSink(ReplicationSinkManager.java:117)
> at
>
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.shipEdits(ReplicationSource.java:652)
> at
>
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.run(ReplicationSource.java:350)
>
> On Mon, Nov 3, 2014 at 11:18 AM, Pere Kyle  wrote:
>
> > Nishanth,
> >
> > In my experience the only way I have been able to clear the dead region
> > servers is to restart the master daemon.
> >
> > -Pere
> >
> > On Mon, Nov 3, 2014 at 9:49 AM, Nishanth S 
> > wrote:
> >
> > > Hey folks,
> > >
> > > How do I remove a dead region server?.I manually failed over the hbase
> > > master but this is still appearing in master UI and also on the status
> > > command that I run.
> > >
> > > Thanks,
> > > Nishan
> > >
> >
>
>


Re: Re: Hbase Dead region Server

2014-11-03 Thread yeweichen2...@gmail.com
Thanks Pere. I just did that and still  has the dead region server  showing
up in Master UI as well as  in status command.I have replication turned on
 in hbase and seeing few issues.Below is the stack trace I am seeing.

2014-11-03 18:31:00,215 WARN
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Can't
replicate because of a local or network error:
java.io.IOException: No replication sinks are available
at
org.apache.hadoop.hbase.replication.regionserver.ReplicationSinkManager.getReplicationSink(ReplicationSinkManager.java:117)
at
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.shipEdits(ReplicationSource.java:652)
at
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.run(ReplicationSource.java:350)
2014-11-03 18:31:00,459 WARN
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Can't
replicate because of a local or network error:
java.io.IOException: No replication sinks are available
at
org.apache.hadoop.hbase.replication.regionserver.ReplicationSinkManager.getReplicationSink(ReplicationSinkManager.java:117)
at
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.shipEdits(ReplicationSource.java:652)
at
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.run(ReplicationSource.java:350)

On Mon, Nov 3, 2014 at 11:18 AM, Pere Kyle  wrote:

> Nishanth,
>
> In my experience the only way I have been able to clear the dead region
> servers is to restart the master daemon.
>
> -Pere
>
> On Mon, Nov 3, 2014 at 9:49 AM, Nishanth S 
> wrote:
>
> > Hey folks,
> >
> > How do I remove a dead region server?.I manually failed over the hbase
> > master but this is still appearing in master UI and also on the status
> > command that I run.
> >
> > Thanks,
> > Nishan
> >
>