[
https://issues.apache.org/jira/browse/PHOENIX-3977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
JeongMin Ju updated PHOENIX-3977:
---------------------------------
Description:
Region is not closed when moving region or balancing while writing to table
with local index.
If the regionserver is forcibly killed and restart and then balancing is
performed during write operation perform using YCSB. The region is not moved
properly, so it becomes jammed.
This is also true when moving a specific region.
{panel:title=RegionServer1}
2017-06-26 17:18:49,096 INFO
org.apache.phoenix.hbase.index.util.IndexManagementUtil: Rethrowing
org.apache.hadoop.hbase.DoNotRetryIOException: ERROR 2008 (INT10): ERROR 2008
(INT10): Unable to find cached index metadata. key=-4998016164816556219
region=PHOENIX_TABLE,&,1498464285648.df3f3da32306361ce5e38e5193f5e5d6.host=juke-cdh-36f531b4.s2.krane.9rum.cc,60020,1498464965204
Index update failed
2017-06-26 17:18:49,531 INFO
org.apache.phoenix.hbase.index.util.IndexManagementUtil: Rethrowing
org.apache.hadoop.hbase.DoNotRetryIOException: ERROR 2008 (INT10): ERROR 2008
(INT10): Unable to find cached index metadata. key=6036765365681157624
region=PHOENIX_TABLE,&,1498464285648.df3f3da32306361ce5e38e5193f5e5d6.host=juke-cdh-36f531b4.s2.krane.9rum.cc,60020,1498464965204
Index update failed
2017-06-26 17:18:49,536 INFO
org.apache.phoenix.hbase.index.util.IndexManagementUtil: Rethrowing
org.apache.hadoop.hbase.DoNotRetryIOException: ERROR 2008 (INT10): ERROR 2008
(INT10): Unable to find cached index metadata. key=-428088766346522675
region=PHOENIX_TABLE,&,1498464285648.df3f3da32306361ce5e38e5193f5e5d6.host=juke-cdh-36f531b4.s2.krane.9rum.cc,60020,1498464965204
Index update failed
{panel}
At other regionserver
{panel:title=RegionServer2}
2017-06-26 17:19:49,755 INFO org.apache.hadoop.hbase.client.AsyncProcess:
#1476, waiting for 62 actions to finish on table: PHOENIX_TABLE
2017-06-26 17:19:49,755 INFO org.apache.hadoop.hbase.client.AsyncProcess:
#1476, waiting for 27 actions to finish on table: PHOENIX_TABLE
2017-06-26 17:19:49,755 INFO org.apache.hadoop.hbase.client.AsyncProcess:
#1476, waiting for 52 actions to finish on table: PHOENIX_TABLE
{panel}
phoenix.coprocessor.maxServerCacheTimeToLiveMs &
phoenix.coprocessor.maxMetaDataCacheSize was not effected.
was:
Region is not closed when moving region or balancing while writing to table
with local index.
If the regionserver is forcibly killed and restart and then balancing is
performed during write operation perform using YCSB. The region is not moved
properly, so it becomes jammed.
This is also true when moving a specific region.
{panel:title=My title}
2017-06-26 17:18:49,096 INFO
org.apache.phoenix.hbase.index.util.IndexManagementUtil: Rethrowing
org.apache.hadoop.hbase.DoNotRetryIOException: ERROR 2008 (INT10): ERROR 2008
(INT10): Unable to find cached index metadata. key=-4998016164816556219
region=PHOENIX_TABLE,&,1498464285648.df3f3da32306361ce5e38e5193f5e5d6.host=juke-cdh-36f531b4.s2.krane.9rum.cc,60020,1498464965204
Index update failed
2017-06-26 17:18:49,531 INFO
org.apache.phoenix.hbase.index.util.IndexManagementUtil: Rethrowing
org.apache.hadoop.hbase.DoNotRetryIOException: ERROR 2008 (INT10): ERROR 2008
(INT10): Unable to find cached index metadata. key=6036765365681157624
region=PHOENIX_TABLE,&,1498464285648.df3f3da32306361ce5e38e5193f5e5d6.host=juke-cdh-36f531b4.s2.krane.9rum.cc,60020,1498464965204
Index update failed
2017-06-26 17:18:49,536 INFO
org.apache.phoenix.hbase.index.util.IndexManagementUtil: Rethrowing
org.apache.hadoop.hbase.DoNotRetryIOException: ERROR 2008 (INT10): ERROR 2008
(INT10): Unable to find cached index metadata. key=-428088766346522675
region=PHOENIX_TABLE,&,1498464285648.df3f3da32306361ce5e38e5193f5e5d6.host=juke-cdh-36f531b4.s2.krane.9rum.cc,60020,1498464965204
Index update failed
{panel}
At other regionserver
{panel:title=My title}
2017-06-26 17:19:49,755 INFO org.apache.hadoop.hbase.client.AsyncProcess:
#1476, waiting for 62 actions to finish on table: PHOENIX_TABLE
2017-06-26 17:19:49,755 INFO org.apache.hadoop.hbase.client.AsyncProcess:
#1476, waiting for 27 actions to finish on table: PHOENIX_TABLE
2017-06-26 17:19:49,755 INFO org.apache.hadoop.hbase.client.AsyncProcess:
#1476, waiting for 52 actions to finish on table: PHOENIX_TABLE
{panel}
phoenix.coprocessor.maxServerCacheTimeToLiveMs &
phoenix.coprocessor.maxMetaDataCacheSize was not effected.
> Region is not closed when moving region or balancing while writing to table
> with local index
> --------------------------------------------------------------------------------------------
>
> Key: PHOENIX-3977
> URL: https://issues.apache.org/jira/browse/PHOENIX-3977
> Project: Phoenix
> Issue Type: Bug
> Affects Versions: 4.10.0
> Reporter: JeongMin Ju
>
> Region is not closed when moving region or balancing while writing to table
> with local index.
> If the regionserver is forcibly killed and restart and then balancing is
> performed during write operation perform using YCSB. The region is not moved
> properly, so it becomes jammed.
> This is also true when moving a specific region.
> {panel:title=RegionServer1}
> 2017-06-26 17:18:49,096 INFO
> org.apache.phoenix.hbase.index.util.IndexManagementUtil: Rethrowing
> org.apache.hadoop.hbase.DoNotRetryIOException: ERROR 2008 (INT10): ERROR 2008
> (INT10): Unable to find cached index metadata. key=-4998016164816556219
> region=PHOENIX_TABLE,&,1498464285648.df3f3da32306361ce5e38e5193f5e5d6.host=juke-cdh-36f531b4.s2.krane.9rum.cc,60020,1498464965204
> Index update failed
> 2017-06-26 17:18:49,531 INFO
> org.apache.phoenix.hbase.index.util.IndexManagementUtil: Rethrowing
> org.apache.hadoop.hbase.DoNotRetryIOException: ERROR 2008 (INT10): ERROR 2008
> (INT10): Unable to find cached index metadata. key=6036765365681157624
> region=PHOENIX_TABLE,&,1498464285648.df3f3da32306361ce5e38e5193f5e5d6.host=juke-cdh-36f531b4.s2.krane.9rum.cc,60020,1498464965204
> Index update failed
> 2017-06-26 17:18:49,536 INFO
> org.apache.phoenix.hbase.index.util.IndexManagementUtil: Rethrowing
> org.apache.hadoop.hbase.DoNotRetryIOException: ERROR 2008 (INT10): ERROR 2008
> (INT10): Unable to find cached index metadata. key=-428088766346522675
> region=PHOENIX_TABLE,&,1498464285648.df3f3da32306361ce5e38e5193f5e5d6.host=juke-cdh-36f531b4.s2.krane.9rum.cc,60020,1498464965204
> Index update failed
> {panel}
> At other regionserver
> {panel:title=RegionServer2}
> 2017-06-26 17:19:49,755 INFO org.apache.hadoop.hbase.client.AsyncProcess:
> #1476, waiting for 62 actions to finish on table: PHOENIX_TABLE
> 2017-06-26 17:19:49,755 INFO org.apache.hadoop.hbase.client.AsyncProcess:
> #1476, waiting for 27 actions to finish on table: PHOENIX_TABLE
> 2017-06-26 17:19:49,755 INFO org.apache.hadoop.hbase.client.AsyncProcess:
> #1476, waiting for 52 actions to finish on table: PHOENIX_TABLE
> {panel}
> phoenix.coprocessor.maxServerCacheTimeToLiveMs &
> phoenix.coprocessor.maxMetaDataCacheSize was not effected.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
