The only problem with this approach is that if RS responsible for closing the region did not flush the data properly then explicitly setting it to CLOSED in hbase:meta can result in data loss until you killed the RS so that the data can be replayed through WAL.
Thanks, Ankit Singhal On Wed, Sep 4, 2019 at 6:15 PM Syni Guo <guoshengy...@outlook.com> wrote: > > HI , > > Thanks for your suggection , I used a extreme method solved the issue . > > 1. Change the RIT region 'info:state’ value to 'CLOSED' of the meta table > ‘hbase:mata’ > 2. Switch over the master node . > > > > On 4 Sep 2019, at 00:05, Wellington Chevreuil < > wellington.chevre...@gmail.com> wrote: > > > > The "unassign" procs for each of those regions are waiting on a lock > that's > > currently held by another proc whose id is 21075720: > >> > >> 2019-09-02 19:51:29,365 INFO [PEWorker-1] > >> procedure.MasterProcedureScheduler: Waiting on xlock for > >> pid=21097026...held by pid=21075720 > >> > > > > You need to figure out why this pid=21075720 is stuck while it keeps the > > lock(s) needed by the two "unassign" procs. You can figure out which proc > > is this via Web UI or hbase shell list_procedures command. From past > > experience, I would bet this is a rogue "disable_table" command who had > > timed out long ago. A meta scan for this 'alpha_daas:device_data_details' > > table regions would also help determine if this indeed the case: > > $ echo "scan 'hbase:meta'" | hbase shell | grep > > "alpha_daas:device_data_details" > > > > Em ter, 3 de set de 2019 às 00:12, Syni Guo <guoshengy...@outlook.com> > > escreveu: > > > >> Hi Ankit, > >> > >> How can i find out the block session or process and kill them. it does > not > >> work even reboot the RS > >> > >> > >> 获取 Outlook for Android<https://aka.ms/ghei36> > >> > >> ________________________________ > >> From: Ankit Singhal <ankitsingha...@gmail.com> > >> Sent: Monday, September 2, 2019 11:47:06 PM > >> To: dev@hbase.apache.org <dev@hbase.apache.org> > >> Subject: Re: Region in RIT (CLOSING) , How to fix it ? > >> > >> you may check the RS logs of tx-220-70-27.h.chinabank.com.cn for > >> exceptions(like sync failure > >> while writing marker in the WAL during closure etc ) which are keeping > >> these regions stuck in the > >> CLOSING state and you may also try killing this server for > >> ServerCrashProcedure to take care of this. > >> > >> Regards, > >> Ankit Singhal > >> > >> On Mon, Sep 2, 2019 at 6:52 AM Jean-Marc Spaggiari < > >> jean-m...@spaggiari.org> > >> wrote: > >> > >>> Hi Syni, > >>> > >>> Have you tried using HBCK2? > >>> > >>> JMS > >>> > >>> Le lun. 2 sept. 2019 07 h 57, Syni Guo <guoshengy...@outlook.com> a > >> écrit > >>> : > >>> > >>>> > >>>> > >>>> Hbase version : 2.1.3 > >>>> > >>>> > >>>> There are 2 region in RIT (CLOSING) , How to fix it ? , I try to > >> unassign > >>>> it ,but timeout failed . > >>>> > >>>> > >>>> hbase(main):032:0> unassign '444405785869685e6ec948c03c2076b8',true > >>>> > >>>> ERROR: Call id=13259, waitTime=10009, rpcTimeout=10000 > >>>> > >>>> For usage try 'help "unassign”' > >>>> > >>>> > >>>> > >>>> Logs; > >>>> > >>>> 2019-09-02 19:50:54,453 WARN [ProcExecTimeout] > >>>> assignment.AssignmentManager: STUCK Region-In-Transition rit=CLOSING, > >>>> location=tx-220-70-27.h.chinabank.com.cn,60020,1567410218074, > >>>> table=alpha_daas:device_data_details, > >>>> region=444405785869685e6ec948c03c2076b8 > >>>> 2019-09-02 19:50:54,453 WARN [ProcExecTimeout] > >>>> assignment.AssignmentManager: STUCK Region-In-Transition rit=CLOSING, > >>>> location=tx-220-70-27.h.chinabank.com.cn,60020,1567410218074, > >>>> table=alpha_daas:poi_unicom_stat, > >> region=42de1052551760e45cb7ba2684d586f8 > >>>> > >>>> > >>>> > >>>> 2019-09-02 19:51:29,365 INFO [PEWorker-1] > >>>> procedure.MasterProcedureScheduler: Waiting on xlock for pid=21097026, > >>>> state=RUNNABLE:REGION_TRANSITION_DISPATCH; UnassignProcedure > >>>> table=alpha_daas:device_data_details, > >>>> region=444405785869685e6ec948c03c2076b8, server= > >>>> tx-220-70-27.h.chinabank.com.cn,60020,1567410218074 held by > >> pid=21075720 > >>>> 2019-09-02 19:51:39,581 INFO [PEWorker-11] > >>>> procedure.MasterProcedureScheduler: Waiting on xlock for pid=21097027, > >>>> state=RUNNABLE:REGION_TRANSITION_DISPATCH; UnassignProcedure > >>>> table=alpha_daas:device_data_details, > >>>> region=444405785869685e6ec948c03c2076b8, server= > >>>> tx-220-70-27.h.chinabank.com.cn,60020,1567410218074 held by > >> pid=21075720 > >>>> > >>>> > >>>> > >>>> > >>> > >> > >
