Looks like a datanode went down. InterruptedException is how java uses to interrupt IO in threads, its similar to the EINTR errno. That means the actual source of the abort is higher up...
So back to how InterruptedException works... at some point a thread in the JVM decides that the VM should abort. So it calls thread.interrupt() on all the threads it knows/cares about to interrupt their IO. That is what you are seeing in the logs. The root cause lies above I think. Look for the first "Exception" string or any FATAL or ERROR strings in the datanode logfiles. -ryan On Thu, Mar 10, 2011 at 1:03 PM, Jack Levin <magn...@gmail.com> wrote: > http://pastebin.com/ZmsyvcVc Here is the regionserver log, they all have > similar stuff, > > On Thu, Mar 10, 2011 at 11:34 AM, Stack <st...@duboce.net> wrote: > >> Whats in the regionserver logs? Please put up regionserver and >> datanode excerpts. >> Thanks Jack, >> St.Ack >> >> On Thu, Mar 10, 2011 at 10:31 AM, Jack Levin <magn...@gmail.com> wrote: >> > All was well, until this happen: >> > >> > http://pastebin.com/iM1niwrS >> > >> > and all regionservers went down, is this xciever issue? >> > >> > <property> >> > <name>dfs.datanode.max.xcievers</name> >> > <value>12047</value> >> > </property> >> > >> > this is what I have, should I set it higher? >> > >> > -Jack >> > >> >
