Hbase Master logs: 2016-10-03 13:37:48,950 INFO [AM.ZK.Worker-pool2-t64] master.RegionStates: Transition {8ea153077a9750d342506849ba9a0e9c state=PENDING_OPEN, ts=1475501868940, server=ip-172-31-17-32.us-west-2.compute.internal,16020,1475498064563} to {8ea153077a9750d342506849ba9a0e9c state=OPENING, ts=1475501868950, server=ip-172-31-17-32.us-west-2.compute.internal,16020,1475498064563} 2016-10-03 13:37:49,085 INFO [AM.ZK.Worker-pool2-t67] master.RegionStates: Transition {8ea153077a9750d342506849ba9a0e9c state=OPENING, ts=1475501868950, server=ip-172-31-17-32.us-west-2.compute.internal,16020,1475498064563} to {8ea153077a9750d342506849ba9a0e9c state=OPEN, ts=1475501869085, server=ip-172-31-17-32.us-west-2.compute.internal,16020,1475498064563} 2016-10-03 13:40:20,400 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=417.43 KB, freeSize=395.89 MB, max=396.30 MB, blockCount=0, accesses=0, hits=0, hitRatio=0, cachingAccesses=0, cachingHits=0, cachingHitsRatio=0,evictions=389, evicted=0, evictedPerRun=0.0 2016-10-03 13:44:45,814 INFO [B.defaultRpcServer.handler=3,queue=0,port=16000] master.HMaster: Client=ec2-user//172.31.21.233 modify nodes 2016-10-03 13:44:46,156 INFO [ProcedureExecutor-1] util.FSTableDescriptors: Updated tableinfo=hdfs://ip-172-31-20-113.us-west-2.compute.internal:8020/user/hbase/data/default/nodes/.tabledesc/.tableinfo.0000000217 2016-10-03 13:44:46,468 INFO [ProcedureExecutor-1] procedure.MasterDDLOperationHelper: Bucketing regions by region server... 2016-10-03 13:44:46,471 INFO [ProcedureExecutor-1] procedure.MasterDDLOperationHelper: Reopening 5 regions on 4 region servers.
Region Server 1 logs: 2016-10-03 13:42:17,705 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished memstore flush of ~128.00 MB/134218312, currentsize=182.53 KB/186912 for region nodes-p_type-index,\x04\x00\x00,1475499172973.17a95c591e69416cdb8a5afca966ea69. in 1497ms, sequenceid=610564, compaction requested=false 2016-10-03 13:42:32,478 INFO [HBase-Metrics2-1] impl.MetricsSystemImpl: Stopping HBase metrics system... 2016-10-03 13:42:32,481 INFO [HBase-Metrics2-1] impl.MetricsSystemImpl: HBase metrics system stopped. 2016-10-03 13:42:32,982 INFO [HBase-Metrics2-1] impl.MetricsConfig: loaded properties from hadoop-metrics2-hbase.properties 2016-10-03 13:42:32,984 INFO [HBase-Metrics2-1] impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s). 2016-10-03 13:42:32,984 INFO [HBase-Metrics2-1] impl.MetricsSystemImpl: HBase metrics system started 2016-10-03 13:42:50,831 INFO [MemStoreFlusher.1] regionserver.HRegion: Flushing 1/1 column families, memstore=128.00 MB 2016-10-03 13:42:51,585 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 1/1 column families, memstore=128.00 MB Region Server 2 logs: 2016-10-03 13:42:44,404 ERROR [B.defaultRpcServer.handler=10,queue=1,port=16020] coprocessor.MetaDataEndpointImpl: getTable failed java.lang.AssertionError: we should never remove a different context at org.apache.hadoop.hbase.regionserver.HRegion$RowLockContext.cleanUp(HRegion.java:5227) at org.apache.hadoop.hbase.regionserver.HRegion$RowLockImpl.release(HRegion.java:5272) at org.apache.phoenix.coprocessor.MetaDataEndpointImpl.doGetTable(MetaDataEndpointImpl.java:2489) Thanks, Krishna. > On 03-Oct-2016, at 21:46, Ted Yu <yuzhih...@gmail.com> wrote: > > Can you pastebin region server log around the time the assertion was raised > ? > > Thanks > > On Mon, Oct 3, 2016 at 9:12 AM, Shiva Krishna <shiva.krishn...@gmail.com> > wrote: > >> Hi All, >> when I am trying to insert the data into HBase through Phoenix. >> I am getting the below exception >> Caused by: java.lang.AssertionError: we should never remove a different >> context >> at org.apache.hadoop.hbase.regionserver.HRegion$RowLockContext.cleanUp( >> HRegion.java:5227) >> at org.apache.hadoop.hbase.regionserver.HRegion$ >> RowLockImpl.release(HRegion.java:5272) >> at org.apache.phoenix.coprocessor.MetaDataEndpointImpl.doGetTable( >> MetaDataEndpointImpl.java:2489) >> at org.apache.phoenix.coprocessor.MetaDataEndpointImpl.doGetTable( >> MetaDataEndpointImpl.java:2426) >> at org.apache.phoenix.coprocessor.MetaDataEndpointImpl.getTable( >> MetaDataEndpointImpl.java:451) >> ... 10 more >> >> Phoenix : 4.7.0-HBase1.1 jar >> HBase: 1.2.2 >> >> Cluster : 1 master 4 cores >> >> OS : Amazon EMR 5.0.0 >> >> Thanks, >> Krishna.