Hbase Master logs:
2016-10-03 13:37:48,950 INFO [AM.ZK.Worker-pool2-t64] master.RegionStates:
Transition {8ea153077a9750d342506849ba9a0e9c state=PENDING_OPEN,
ts=1475501868940,
server=ip-172-31-17-32.us-west-2.compute.internal,16020,1475498064563} to
{8ea153077a9750d342506849ba9a0e9c state=OPENING, ts=1475501868950,
server=ip-172-31-17-32.us-west-2.compute.internal,16020,1475498064563}
2016-10-03 13:37:49,085 INFO [AM.ZK.Worker-pool2-t67] master.RegionStates:
Transition {8ea153077a9750d342506849ba9a0e9c state=OPENING, ts=1475501868950,
server=ip-172-31-17-32.us-west-2.compute.internal,16020,1475498064563} to
{8ea153077a9750d342506849ba9a0e9c state=OPEN, ts=1475501869085,
server=ip-172-31-17-32.us-west-2.compute.internal,16020,1475498064563}
2016-10-03 13:40:20,400 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache:
totalSize=417.43 KB, freeSize=395.89 MB, max=396.30 MB, blockCount=0,
accesses=0, hits=0, hitRatio=0, cachingAccesses=0, cachingHits=0,
cachingHitsRatio=0,evictions=389, evicted=0, evictedPerRun=0.0
2016-10-03 13:44:45,814 INFO [B.defaultRpcServer.handler=3,queue=0,port=16000]
master.HMaster: Client=ec2-user//172.31.21.233 modify nodes
2016-10-03 13:44:46,156 INFO [ProcedureExecutor-1] util.FSTableDescriptors:
Updated
tableinfo=hdfs://ip-172-31-20-113.us-west-2.compute.internal:8020/user/hbase/data/default/nodes/.tabledesc/.tableinfo.0000000217
2016-10-03 13:44:46,468 INFO [ProcedureExecutor-1]
procedure.MasterDDLOperationHelper: Bucketing regions by region server...
2016-10-03 13:44:46,471 INFO [ProcedureExecutor-1]
procedure.MasterDDLOperationHelper: Reopening 5 regions on 4 region servers.
Region Server 1 logs:
2016-10-03 13:42:17,705 INFO [MemStoreFlusher.0] regionserver.HRegion:
Finished memstore flush of ~128.00 MB/134218312, currentsize=182.53 KB/186912
for region
nodes-p_type-index,\x04\x00\x00,1475499172973.17a95c591e69416cdb8a5afca966ea69.
in 1497ms, sequenceid=610564, compaction requested=false
2016-10-03 13:42:32,478 INFO [HBase-Metrics2-1] impl.MetricsSystemImpl:
Stopping HBase metrics system...
2016-10-03 13:42:32,481 INFO [HBase-Metrics2-1] impl.MetricsSystemImpl: HBase
metrics system stopped.
2016-10-03 13:42:32,982 INFO [HBase-Metrics2-1] impl.MetricsConfig: loaded
properties from hadoop-metrics2-hbase.properties
2016-10-03 13:42:32,984 INFO [HBase-Metrics2-1] impl.MetricsSystemImpl:
Scheduled snapshot period at 10 second(s).
2016-10-03 13:42:32,984 INFO [HBase-Metrics2-1] impl.MetricsSystemImpl: HBase
metrics system started
2016-10-03 13:42:50,831 INFO [MemStoreFlusher.1] regionserver.HRegion:
Flushing 1/1 column families, memstore=128.00 MB
2016-10-03 13:42:51,585 INFO [MemStoreFlusher.0] regionserver.HRegion:
Flushing 1/1 column families, memstore=128.00 MB
Region Server 2 logs:
2016-10-03 13:42:44,404 ERROR
[B.defaultRpcServer.handler=10,queue=1,port=16020]
coprocessor.MetaDataEndpointImpl: getTable failed
java.lang.AssertionError: we should never remove a different context
at
org.apache.hadoop.hbase.regionserver.HRegion$RowLockContext.cleanUp(HRegion.java:5227)
at
org.apache.hadoop.hbase.regionserver.HRegion$RowLockImpl.release(HRegion.java:5272)
at
org.apache.phoenix.coprocessor.MetaDataEndpointImpl.doGetTable(MetaDataEndpointImpl.java:2489)
Thanks,
Krishna.
> On 03-Oct-2016, at 21:46, Ted Yu <[email protected]> wrote:
>
> Can you pastebin region server log around the time the assertion was raised
> ?
>
> Thanks
>
> On Mon, Oct 3, 2016 at 9:12 AM, Shiva Krishna <[email protected]>
> wrote:
>
>> Hi All,
>> when I am trying to insert the data into HBase through Phoenix.
>> I am getting the below exception
>> Caused by: java.lang.AssertionError: we should never remove a different
>> context
>> at org.apache.hadoop.hbase.regionserver.HRegion$RowLockContext.cleanUp(
>> HRegion.java:5227)
>> at org.apache.hadoop.hbase.regionserver.HRegion$
>> RowLockImpl.release(HRegion.java:5272)
>> at org.apache.phoenix.coprocessor.MetaDataEndpointImpl.doGetTable(
>> MetaDataEndpointImpl.java:2489)
>> at org.apache.phoenix.coprocessor.MetaDataEndpointImpl.doGetTable(
>> MetaDataEndpointImpl.java:2426)
>> at org.apache.phoenix.coprocessor.MetaDataEndpointImpl.getTable(
>> MetaDataEndpointImpl.java:451)
>> ... 10 more
>>
>> Phoenix : 4.7.0-HBase1.1 jar
>> HBase: 1.2.2
>>
>> Cluster : 1 master 4 cores
>>
>> OS : Amazon EMR 5.0.0
>>
>> Thanks,
>> Krishna.
