Re: Is it normal that a dead data node causes 3 region servers to go down?

[Peter Naudus]

Sure thing, thanks so much for looking into this.

I uploaded the files (pastebin didn't like the size of the files) here:

http://www.linuxlefty.com/hbase-hbase-regionserver-003.log
http://www.linuxlefty.com/hbase-hbase-regionserver-008.log
http://www.linuxlefty.com/hbase-hbase-regionserver-009.log

These are the log files for each of the three regionservers that died.

Just in case you don't want to slog through those files I uploaded the last 1000 lines of each log file to:

http://www.linuxlefty.com/hbase-hbase-regionserver-003.log.1000
http://www.linuxlefty.com/hbase-hbase-regionserver-008.log.1000
http://www.linuxlefty.com/hbase-hbase-regionserver-009.log.1000

~Peter

On Mon, 25 Jun 2012 12:23:51 -0400, Stack <st...@duboce.net> wrote:

> On Mon, Jun 25, 2012 at 4:17 PM, Peter Naudus <pnau...@dataraker.com>
> wrote:
>> We're running CDH3 (hbase: 0.90.6, hadoop: 0.20.2).
>>
>> As far as I'm aware, no one tried to shut down the server. I read online
>> that that the "user requested stop" error is sometimes logged on unknown
>> exceptions, not necessarily during an explicit shutdown
>> (http://apache-hbase.679495.n3.nabble.com/Disable-table-causing-regionserver-shutdown-td2838463.html)
>>
>
> That should work. Mind posting more of the log around shutdown? Put
> it up in pastebin?
> Thanks,
> St.Ack