Maybe you can dig up master / region server logs to see if there was any of the following happening at time of merge ? region server failover load balancer run zookeeper session expiration.
I feel we should make region merging more robust. Cheers On Wed, Aug 29, 2012 at 9:44 AM, Shrijeet Paliwal <[email protected]>wrote: > Ted, > Most likely Yes. > > On Wed, Aug 29, 2012 at 9:41 AM, Ted Yu <[email protected]> wrote: > > > Thanks for sharing your experience, Shrijeet. > > > > Was the region cited below involved in your previous merging activity ? > > > > Cheers > > > > On Wed, Aug 29, 2012 at 9:36 AM, Shrijeet Paliwal > > <[email protected]>wrote: > > > > > Hi Devs, > > > > > > I did an upgrade to HBase 0.92.2 recently. It ran fine for couple of > > days, > > > then I did online merge of regions. This got into a bit of inconsistent > > > state. To fix it the corruption I ran hbck and managed to get into > > > a situation described in > > > > > > > > > http://search-hadoop.com/m/sm2APK8FMo1&subj=Re+Wondering+what+hbck+should+do+in+this+situation > > > . > > > After a bit of manual tinkering I got issues resolved. > > > > > > But since then I see hbck reporting following errors (for different > > regions > > > at different times, not always the same region) consistently: > > > > > > ERROR: Region { meta => > > > > > > table,c638103773063741088,1346207285319.d8051f85ab65b2e680ba5ebd383eaa08., > > > hdfs => hdfs:// > > > > > > master-hadoop.example.net:8020/hbase/table/d8051f85ab65b2e680ba5ebd383eaa08 > > > < > > > > > > http://master-hadoop.ine-arp.rfiserve.net:8020/hbase/ine-arp/userprofile/d8051f85ab65b2e680ba5ebd383eaa08 > > > >, > > > deployed => } is a split parent in META, in HDFS, and not deployed on > > any > > > region server. This could be transient. > > > > > > It does get back to 'Status: OK' state, but keeps flip flopping. > > > > > > Any experiences? > > > > > > -Shrijeet > > > > > >
