[
https://issues.apache.org/jira/browse/HBASE-1573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-1573:
-------------------------
Fix Version/s: 0.20.0
Bringing into 0.20.0.
Rapleaf just got it on TRUNK.
I saw it last night at pset on a 0.19 install:
{code}
[st...@aa0-000-8 hbase]$ grep 'nyt,1xNceOaojLkfi-31ZTQSK-==,1248216593259'
hbase-Powerset-master-aa0-000-8.u.powerset.com.log.2009-07-*|grep -v
PROCESS_OPEN
hbase-Powerset-master-aa0-000-8.u.powerset.com.log.2009-07-21:2009-07-21
22:49:56,674 INFO org.apache.hadoop.hbase.master.ServerManager: Received
MSG_REPORT_SPLIT: nyt,0yGU-ywdW8iPTKOVcFFxuV==,1248214827405:
nyt,0yGU-ywdW8iPTKOVcFFxuV==,1248214827405 split; daughters:
nyt,0yGU-ywdW8iPTKOVcFFxuV==,1248216593259,
nyt,1xNceOaojLkfi-31ZTQSK-==,1248216593259: safeMode=false from
XX.XX.45.124:20020
hbase-Powerset-master-aa0-000-8.u.powerset.com.log.2009-07-21:2009-07-21
22:49:56,674 INFO org.apache.hadoop.hbase.master.RegionManager: Assigning
region nyt,1xNceOaojLkfi-31ZTQSK-==,1248216593259 to XX.XX.45.124:20020
hbase-Powerset-master-aa0-000-8.u.powerset.com.log.2009-07-21:2009-07-21
22:49:59,698 INFO org.apache.hadoop.hbase.master.ServerManager: Received
MSG_REPORT_OPEN: nyt,1xNceOaojLkfi-31ZTQSK-==,1248216593259: safeMode=false
from XX.XX.45.124:20020
hbase-Powerset-master-aa0-000-8.u.powerset.com.log.2009-07-21:2009-07-21
22:49:59,701 INFO org.apache.hadoop.hbase.master.ProcessRegionOpen:
nyt,1xNceOaojLkfi-31ZTQSK-==,1248216593259 open on XX.XX.45.124:20020
hbase-Powerset-master-aa0-000-8.u.powerset.com.log.2009-07-21:2009-07-21
22:49:59,701 INFO org.apache.hadoop.hbase.master.ProcessRegionOpen: updating
row nyt,1xNceOaojLkfi-31ZTQSK-==,1248216593259 in region .META.,,1 with with
startcode 1248174024124 and server XX.XX.45.124:20020
hbase-Powerset-master-aa0-000-8.u.powerset.com.log.2009-07-21:2009-07-21
22:50:00,625 DEBUG org.apache.hadoop.hbase.master.BaseScanner: Current
assignment of nyt,1xNceOaojLkfi-31ZTQSK-==,1248216593259 is not valid; Server
'' unknown.
hbase-Powerset-master-aa0-000-8.u.powerset.com.log.2009-07-21:2009-07-21
22:50:02,723 INFO org.apache.hadoop.hbase.master.RegionManager: Assigning
region nyt,1xNceOaojLkfi-31ZTQSK-==,1248216593259 to XX.XX.45.124:20020
hbase-Powerset-master-aa0-000-8.u.powerset.com.log.2009-07-21:2009-07-21
22:50:05,756 INFO org.apache.hadoop.hbase.master.ServerManager: Received
MSG_REPORT_OPEN: nyt,1xNceOaojLkfi-31ZTQSK-==,1248216593259: safeMode=false
from XX.XX.45.124:20020
hbase-Powerset-master-aa0-000-8.u.powerset.com.log.2009-07-21:2009-07-21
22:50:05,757 INFO org.apache.hadoop.hbase.master.ProcessRegionOpen:
nyt,1xNceOaojLkfi-31ZTQSK-==,1248216593259 open on XX.XX.45.124:20020
hbase-Powerset-master-aa0-000-8.u.powerset.com.log.2009-07-21:2009-07-21
22:50:05,757 INFO org.apache.hadoop.hbase.master.ProcessRegionOpen: updating
row nyt,1xNceOaojLkfi-31ZTQSK-==,1248216593259 in region .META.,,1 with with
startcode 1248174024124 and server XX.XX.45.124:20020
hbase-Powerset-master-aa0-000-8.u.powerset.com.log.2009-07-21:2009-07-21
22:51:02,053 DEBUG org.apache.hadoop.hbase.master.BaseScanner:
nyt,1xNceOaojLkfi-31ZTQSK-==,1248216593259 no longer has references to
nyt,0yGU-ywdW8iPTKOVcFFxuV==,1248214827405
hbase-Powerset-master-aa0-000-8.u.powerset.com.log.2009-07-22:2009-07-22
16:54:20,359 DEBUG org.apache.hadoop.hbase.master.BaseScanner: Current
assignment of nyt,1xNceOaojLkfi-31ZTQSK-==,1248216593259 is not valid; Server
'XX.XX.45.124:20020' unknown.
hbase-Powerset-master-aa0-000-8.u.powerset.com.log.2009-07-22:2009-07-22
17:06:23,114 INFO org.apache.hadoop.hbase.master.RegionManager: Assigning
region nyt,1xNceOaojLkfi-31ZTQSK-==,1248216593259 to XX.XX.44.95:20020
hbase-Powerset-master-aa0-000-8.u.powerset.com.log.2009-07-22:2009-07-22
17:36:00,303 DEBUG org.apache.hadoop.hbase.master.BaseScanner: Current
assignment of nyt,1xNceOaojLkfi-31ZTQSK-==,1248216593259 is not valid;
serverInfo: address: XX.XX.45.124:20020, startcode: 1248283082287, load:
(requests=0, regions=24, usedHeap=95, maxHeap=1820), passed startCode:
1248174024124, storedInfo.startCode: 1248283082287
hbase-Powerset-master-aa0-000-8.u.powerset.com.log.2009-07-22:2009-07-22
17:36:01,033 INFO org.apache.hadoop.hbase.master.RegionManager: Assigning
region nyt,1xNceOaojLkfi-31ZTQSK-==,1248216593259 to XX.XX.44.133:20020
hbase-Powerset-master-aa0-000-8.u.powerset.com.log.2009-07-22:2009-07-22
17:36:37,427 INFO org.apache.hadoop.hbase.master.ServerManager: Received
MSG_REPORT_OPEN: nyt,1xNceOaojLkfi-31ZTQSK-==,1248216593259: safeMode=false
from XX.XX.44.133:20020
hbase-Powerset-master-aa0-000-8.u.powerset.com.log.2009-07-22:2009-07-22
19:53:24,884 DEBUG org.apache.hadoop.hbase.master.BaseScanner: Current
assignment of nyt,1xNceOaojLkfi-31ZTQSK-==,1248216593259 is not valid;
serverInfo: address: XX.XX.45.124:20020, startcode: 1248286712292, load:
(requests=0, regions=28, usedHeap=113, maxHeap=1820), passed startCode:
1248174024124, storedInfo.startCode: 1248286712292
hbase-Powerset-master-aa0-000-8.u.powerset.com.log.2009-07-22:2009-07-22
19:53:24,886 INFO org.apache.hadoop.hbase.master.RegionManager: Assigning
region nyt,1xNceOaojLkfi-31ZTQSK-==,1248216593259 to XX.XX.44.185:20020
hbase-Powerset-master-aa0-000-8.u.powerset.com.log.2009-07-22:2009-07-22
19:53:28,708 INFO org.apache.hadoop.hbase.master.ServerManager: Received
MSG_REPORT_OPEN: nyt,1xNceOaojLkfi-31ZTQSK-==,1248216593259: safeMode=false
from XX.XX.44.185:20020
hbase-Powerset-master-aa0-000-8.u.powerset.com.log.2009-07-22:2009-07-22
19:58:36,599 INFO org.apache.hadoop.hbase.master.ProcessRegionOpen:
nyt,1xNceOaojLkfi-31ZTQSK-==,1248216593259 open on XX.XX.44.185:20020
hbase-Powerset-master-aa0-000-8.u.powerset.com.log.2009-07-22:2009-07-22
19:58:36,599 INFO org.apache.hadoop.hbase.master.ProcessRegionOpen: updating
row nyt,1xNceOaojLkfi-31ZTQSK-==,1248216593259 in region .META.,,1 with with
startcode 1248286712265 and server XX.XX.44.185:20020
hbase-Powerset-master-aa0-000-8.u.powerset.com.log.2009-07-22:2009-07-22
19:58:40,560 DEBUG org.apache.hadoop.hbase.master.BaseScanner: Current
assignment of nyt,1xNceOaojLkfi-31ZTQSK-==,1248216593259 is not valid;
serverInfo: address: XX.XX.45.124:20020, startcode: 1248286712292, load:
(requests=0, regions=49, usedHeap=101, maxHeap=1820), passed startCode:
1248174024124, storedInfo.startCode: 1248286712292
hbase-Powerset-master-aa0-000-8.u.powerset.com.log.2009-07-22:2009-07-22
19:58:41,823 INFO org.apache.hadoop.hbase.master.RegionManager: Assigning
region nyt,1xNceOaojLkfi-31ZTQSK-==,1248216593259 to XX.XX.44.49:20020
hbase-Powerset-master-aa0-000-8.u.powerset.com.log.2009-07-22:2009-07-22
19:59:00,842 INFO org.apache.hadoop.hbase.master.ServerManager: Received
MSG_REPORT_OPEN: nyt,1xNceOaojLkfi-31ZTQSK-==,1248216593259: safeMode=false
from XX.XX.44.49:20020
hbase-Powerset-master-aa0-000-8.u.powerset.com.log.2009-07-22:2009-07-22
20:04:08,569 INFO org.apache.hadoop.hbase.master.ProcessRegionOpen:
nyt,1xNceOaojLkfi-31ZTQSK-==,1248216593259 open on XX.XX.44.49:20020
hbase-Powerset-master-aa0-000-8.u.powerset.com.log.2009-07-22:2009-07-22
20:04:08,569 INFO org.apache.hadoop.hbase.master.ProcessRegionOpen: updating
row nyt,1xNceOaojLkfi-31ZTQSK-==,1248216593259 in region .META.,,1 with with
startcode 1248292287873 and server XX.XX.44.49:20020
{code}
> Holes in master state change; updated startcode and server go into .META. but
> catalog scanner just got old values
> -----------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-1573
> URL: https://issues.apache.org/jira/browse/HBASE-1573
> Project: Hadoop HBase
> Issue Type: Bug
> Reporter: stack
> Priority: Blocker
> Fix For: 0.20.0
>
> Attachments: 1573-v2.patch, 1573-v3.patch, 1573.patch
>
>
> Here is example of a scan that takes a while because 6k regions acting on
> stale data resulting in double assignment of region:
> {code}
> hbase-Powerset-master-aa0-000-8.u.powerset.com.log.2009-06-22:2009-06-22
> 10:56:06,220 INFO org.apache.hadoop.hbase.master.ServerManager: Received
> MSG_REPORT_OPEN: enwikibase,Cwj1sehVeEbnrUDR_j0xok==,1245604540748:
> safeMode=false from XX.XX.45.121:20020
> hbase-Powerset-master-aa0-000-8.u.powerset.com.log.2009-06-22:2009-06-22
> 10:56:06,220 INFO org.apache.hadoop.hbase.master.RegionServerOperation:
> enwikibase,Cwj1sehVeEbnrUDR_j0xok==,1245604540748 open on XX.XX.45.121:20020
> hbase-Powerset-master-aa0-000-8.u.powerset.com.log.2009-06-22:2009-06-22
> 10:56:06,220 INFO org.apache.hadoop.hbase.master.RegionServerOperation:
> updating row enwikibase,Cwj1sehVeEbnrUDR_j0xok==,1245604540748 in region
> .META.,,1 with with startcode !?B and server XX.XX.45.121:20020
> hbase-Powerset-master-aa0-000-8.u.powerset.com.log.2009-06-22:2009-06-22
> 10:56:06,397 DEBUG org.apache.hadoop.hbase.master.BaseScanner: Current
> assignment of enwikibase,Cwj0sehVeEbnrUDR_j0xok==,1245604540748 is not valid;
> Server 'XX.XX.44.95:20020' unknown.
> hbase-Powerset-master-aa0-000-8.u.powerset.com.log.2009-06-22:2009-06-22
> 10:56:06,582 INFO org.apache.hadoop.hbase.master.RegionManager: Assigning
> region enwikibase,Cwj1sehVeEbnrUDR_j0xok==,1245604540748 to XX.XX.45.97:20020
> hbase-Powerset-master-aa0-000-8.u.powerset.com.log.2009-06-22:2009-06-22
> 10:56:09,587 INFO org.apache.hadoop.hbase.master.ServerManager: Received
> MSG_REPORT_PROCESS_OPEN: enwikibase,Cwj1sehVeEbnrUDR_j0xok==,1245604540748:
> safeMode=false from XX.XX.45.97:20020
> hbase-Powerset-master-aa0-000-8.u.powerset.com.log.2009-06-22:2009-06-22
> 10:56:12,614 INFO org.apache.hadoop.hbase.master.ServerManager: Received
> MSG_REPORT_OPEN: enwikibase,Cwj1sehVeEbnrUDR_j0xok==,1245604540748:
> safeMode=false from XX.XX.45.97:20020
> hbase-Powerset-master-aa0-000-8.u.powerset.com.log.2009-06-22:2009-06-22
> 10:56:13,549 INFO org.apache.hadoop.hbase.master.RegionServerOperation:
> enwikibase,Cwj1sehVeEbnrUDR_j0xok==,1245604540748 open on XX.XX.45.97:20020
> hbase-Powerset-master-aa0-000-8.u.powerset.com.log.2009-06-22:2009-06-22
> 10:56:13,549 INFO org.apache.hadoop.hbase.master.RegionServerOperation:
> updating row enwikibase,Cwj1sehVeEbnrUDR_j0xok==,1245604540748 in region
> .META.,,1 with with startcode !?? and server XX.XX.45.97:20020
> {code}
> We've just updated the server info in the master because of the region open
> message but the scan sees old info in the .META. table though .META. was just
> updated.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
