[
https://issues.apache.org/jira/browse/HBASE-13087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14341131#comment-14341131
]
Rajesh Nishtala commented on HBASE-13087:
-----------------------------------------
So Elliot and I spent some time digging into this over the past few days and we
found a couple of interesting things. The META table descriptions are correct
in Master and in HDFS. So we thought forcing the RegionServer to reload META
would fix it. However, when we tried this we found that it was still reading
the old META table descriptor. After diving deeper into the code Elliot found
that FSTableDescriptors::get(final TableName tablename) used a hard coded
value for the HTableDescriptor of META. As a result no amount of reloading and
forced reloading would fix this.
So there are two not so great paths forward:
(1) Patch 0.98 so that 0.98 can be hot upgradable to branch-1
(2) Change the upgrade process to tell users to upgrade the region servers
first and then master.
Neither of these options are particularly pleasing but I wanted to open this up
to the broader group to get people's thoughts.
> branch-1 isn't rolling upgradable from 0.98
> -------------------------------------------
>
> Key: HBASE-13087
> URL: https://issues.apache.org/jira/browse/HBASE-13087
> Project: HBase
> Issue Type: Bug
> Reporter: Elliott Clark
> Assignee: Rajesh Nishtala
> Priority: Blocker
> Fix For: 2.0.0, 1.1.0
>
>
> {code}org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException:
> Failed 1 action:
> org.apache.hadoop.hbase.regionserver.NoSuchColumnFamilyException: Column
> family table does not exist in region hbase:meta,,1.1588230740 in table
> 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 =>
> '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'},
> {NAME => 'info', BLOOMFILTER => 'NONE', VERSIONS => '10', IN_MEMORY =>
> 'true', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL =>
> 'FOREVER', COMPRESSION => 'NONE', MIN_VERSIONS => '0', BLOCKCACHE => 'true',
> BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.doBatchOp(HRegionServer.java:4513)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.doNonAtomicRegionMutation(HRegionServer.java:3687)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.multi(HRegionServer.java:3576)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:30816)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2029)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
> at java.lang.Thread.run(Thread.java:745)
> : 1 time,
> at
> org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.makeException(AsyncProcess.java:228)
> at
> org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.access$1700(AsyncProcess.java:208)
> at
> org.apache.hadoop.hbase.client.AsyncProcess.waitForAllPreviousOpsAndReset(AsyncProcess.java:1689)
> at
> org.apache.hadoop.hbase.client.BufferedMutatorImpl.backgroundFlushCommits(BufferedMutatorImpl.java:208)
> at
> org.apache.hadoop.hbase.client.BufferedMutatorImpl.flush(BufferedMutatorImpl.java:183)
> at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:1404)
> at org.apache.hadoop.hbase.client.HTable.put(HTable.java:1017)
> at
> org.apache.hadoop.hbase.MetaTableAccessor.put(MetaTableAccessor.java:1123)
> at
> org.apache.hadoop.hbase.MetaTableAccessor.putToMetaTable(MetaTableAccessor.java:1113)
> at
> org.apache.hadoop.hbase.MetaTableAccessor.updateTableState(MetaTableAccessor.java:1436)
> at
> org.apache.hadoop.hbase.MetaTableAccessor.updateTableState(MetaTableAccessor.java:948)
> at
> org.apache.hadoop.hbase.master.TableStateManager.writeMetaState(TableStateManager.java:195)
> at
> org.apache.hadoop.hbase.master.TableStateManager.setTableState(TableStateManager.java:69)
> at
> org.apache.hadoop.hbase.master.AssignmentManager.setEnabledTable(AssignmentManager.java:3427)
> at org.apache.hadoop.hbase.master.HMaster.assignMeta(HMaster.java:903)
> at
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:698)
> at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:166)
> at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1494)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
