[ https://issues.apache.org/jira/browse/PHOENIX-3977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16065594#comment-16065594 ]
Sergey Soldatov commented on PHOENIX-3977: ------------------------------------------ [~mini666] Could you give some more details about the configuration you are using. Does table has only local index or global index as well? Such behavior usually happen when there is a global index on the scene and this problem is discussed in PHOENIX-3970 and related issues. It would be nice if you provide several jstacks for the stuck region server. > Region is not closed when moving region or balancing while writing to table > with local index > -------------------------------------------------------------------------------------------- > > Key: PHOENIX-3977 > URL: https://issues.apache.org/jira/browse/PHOENIX-3977 > Project: Phoenix > Issue Type: Bug > Affects Versions: 4.10.0 > Reporter: JeongMin Ju > > Region is not closed when moving region or balancing while writing to table > with local index. > If the regionserver is forcibly killed and restart and then balancing is > performed during write operation perform using YCSB. The region is not moved > properly, so it becomes jammed. > This is also true when moving a specific region. > {panel:title=RegionServer1} > 2017-06-26 17:18:49,096 INFO > org.apache.phoenix.hbase.index.util.IndexManagementUtil: Rethrowing > org.apache.hadoop.hbase.DoNotRetryIOException: ERROR 2008 (INT10): ERROR 2008 > (INT10): Unable to find cached index metadata. key=-4998016164816556219 > region=PHOENIX_TABLE,&,1498464285648.df3f3da32306361ce5e38e5193f5e5d6.host=juke-cdh-36f531b4.s2.krane.9rum.cc,60020,1498464965204 > Index update failed > 2017-06-26 17:18:49,531 INFO > org.apache.phoenix.hbase.index.util.IndexManagementUtil: Rethrowing > org.apache.hadoop.hbase.DoNotRetryIOException: ERROR 2008 (INT10): ERROR 2008 > (INT10): Unable to find cached index metadata. key=6036765365681157624 > region=PHOENIX_TABLE,&,1498464285648.df3f3da32306361ce5e38e5193f5e5d6.host=juke-cdh-36f531b4.s2.krane.9rum.cc,60020,1498464965204 > Index update failed > 2017-06-26 17:18:49,536 INFO > org.apache.phoenix.hbase.index.util.IndexManagementUtil: Rethrowing > org.apache.hadoop.hbase.DoNotRetryIOException: ERROR 2008 (INT10): ERROR 2008 > (INT10): Unable to find cached index metadata. key=-428088766346522675 > region=PHOENIX_TABLE,&,1498464285648.df3f3da32306361ce5e38e5193f5e5d6.host=juke-cdh-36f531b4.s2.krane.9rum.cc,60020,1498464965204 > Index update failed > {panel} > At other regionserver > {panel:title=RegionServer2} > 2017-06-26 17:19:49,755 INFO org.apache.hadoop.hbase.client.AsyncProcess: > #1476, waiting for 62 actions to finish on table: PHOENIX_TABLE > 2017-06-26 17:19:49,755 INFO org.apache.hadoop.hbase.client.AsyncProcess: > #1476, waiting for 27 actions to finish on table: PHOENIX_TABLE > 2017-06-26 17:19:49,755 INFO org.apache.hadoop.hbase.client.AsyncProcess: > #1476, waiting for 52 actions to finish on table: PHOENIX_TABLE > {panel} > phoenix.coprocessor.maxServerCacheTimeToLiveMs & > phoenix.coprocessor.maxMetaDataCacheSize was not effected. -- This message was sent by Atlassian JIRA (v6.4.14#64029)