[jira] [Updated] (HBASE-5820) hbck should check fs owner/permissions.
[ https://issues.apache.org/jira/browse/HBASE-5820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5820: -- Summary: hbck should check fs owner/permissions. (was: hbck should check fs permissions.) hbck should check fs owner/permissions. --- Key: HBASE-5820 URL: https://issues.apache.org/jira/browse/HBASE-5820 Project: HBase Issue Type: New Feature Components: hbck Reporter: Jonathan Hsieh In some cases, hbck needs to be run as a user that has write perms to the file system. If it writes data to hbase's directories, it may write new files/dirs that the hbase processes's user does not have permissions to. We should alert the user of this situation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5634) document how to use uberhbck
[ https://issues.apache.org/jira/browse/HBASE-5634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5634: -- Status: Patch Available (was: In Progress) document how to use uberhbck Key: HBASE-5634 URL: https://issues.apache.org/jira/browse/HBASE-5634 Project: HBase Issue Type: Improvement Components: documentation, hbck Affects Versions: 0.90.7, 0.92.2, 0.94.0, 0.96.0 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Attachments: uber hbck docs.pdf The updated hbck from HBASE-5128 introduces many new repair options and, as a side effect, offers many new opportunities to durably shoot oneself in the foot. Docs need to be written and added to the ref guide to explain its usage and ramifications and discuss repair strategies. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5634) document how to use uberhbck
[ https://issues.apache.org/jira/browse/HBASE-5634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5634: -- Attachment: uber hbck docs.pdf I've attached documentation how to use the new uber hbck features and options. This is not meant to be included as a pdf -- however, the content is meant to be integrated into the hbase reference guide. (I haven't spent the time to figure out docbook yet). document how to use uberhbck Key: HBASE-5634 URL: https://issues.apache.org/jira/browse/HBASE-5634 Project: HBase Issue Type: Improvement Components: documentation, hbck Affects Versions: 0.90.7, 0.92.2, 0.94.0, 0.96.0 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Attachments: uber hbck docs.pdf The updated hbck from HBASE-5128 introduces many new repair options and, as a side effect, offers many new opportunities to durably shoot oneself in the foot. Docs need to be written and added to the ref guide to explain its usage and ramifications and discuss repair strategies. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5781) Zookeeper session got closed while trying to assign the region to RS using hbck -fix
[
https://issues.apache.org/jira/browse/HBASE-5781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hsieh updated HBASE-5781:
--
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Anoop, Kristam, thanks for finding this and hunting down the root cause of the
problem. Thanks for the quick review Lars.
Committed to 0.90/0.92/0.94. Trunk is broken currently because of HBASE-5747,
HBASE-5793.
Zookeeper session got closed while trying to assign the region to RS using
hbck -fix
Key: HBASE-5781
URL: https://issues.apache.org/jira/browse/HBASE-5781
Project: HBase
Issue Type: Bug
Components: hbck
Affects Versions: 0.90.7, 0.92.1, 0.94.0, 0.96.0
Reporter: Kristam Subba Swathi
Assignee: Jonathan Hsieh
Priority: Critical
Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0
Attachments: hbase-5781.patch
After running the hbck in the cluster ,it is found that one region is not
assigned
So the hbck -fix is used to fix this
But the assignment didnt happen since the zookeeper session is closed
Please find the attached trace for more details
-
Trying to fix unassigned region...
12/04/03 11:02:57 INFO util.HBaseFsckRepair: Region still in transition,
waiting for it to become assigned: {NAME =
'ufdr,002300,179123498.00871fbd7583512e12c4eb38e900be8d.', STARTKEY =
'002300', ENDKEY = '002311', ENCODED = 00871fbd7583512e12c4eb38e900be8d,}
12/04/03 11:02:58 INFO client.HConnectionManager$HConnectionImplementation:
Closed zookeeper sessionid=0x236738a263a
12/04/03 11:02:58 INFO zookeeper.ZooKeeper: Session: 0x236738a263a closed
ERROR: Region { meta =
ufdr,010444,179123857.01594219211d0035b9586f98954462e1., hdfs =
hdfs://10.18.40.25:9000/hbase/ufdr/01594219211d0035b9586f98954462e1, deployed
= } not deployed on any region server.
Trying to fix unassigned region...
12/04/03 11:02:58 INFO zookeeper.ClientCnxn: EventThread shut down
12/04/03 11:02:58 WARN zookeeper.ZKUtil: hconnection-0x236738a263a Unable
to set watcher on znode (/hbase)
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode
= Session expired for /hbase
at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1021)
at
org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:150)
at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:263)
at
org.apache.hadoop.hbase.zookeeper.ZooKeeperNodeTracker.checkIfBaseNodeAvailable(ZooKeeperNodeTracker.java:208)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.checkIfBaseNodeAvailable(HConnectionManager.java:695)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getMaster(HConnectionManager.java:626)
at org.apache.hadoop.hbase.client.HBaseAdmin.getMaster(HBaseAdmin.java:211)
at org.apache.hadoop.hbase.client.HBaseAdmin.assign(HBaseAdmin.java:1325)
at
org.apache.hadoop.hbase.util.HBaseFsckRepair.forceOfflineInZK(HBaseFsckRepair.java:109)
at
org.apache.hadoop.hbase.util.HBaseFsckRepair.fixUnassigned(HBaseFsckRepair.java:92)
at
org.apache.hadoop.hbase.util.HBaseFsck.tryAssignmentRepair(HBaseFsck.java:1235)
at
org.apache.hadoop.hbase.util.HBaseFsck.checkRegionConsistency(HBaseFsck.java:1351)
at
org.apache.hadoop.hbase.util.HBaseFsck.checkAndFixConsistency(HBaseFsck.java:1114)
at
org.apache.hadoop.hbase.util.HBaseFsck.onlineConsistencyRepair(HBaseFsck.java:356)
at org.apache.hadoop.hbase.util.HBaseFsck.onlineHbck(HBaseFsck.java:375)
at org.apache.hadoop.hbase.util.HBaseFsck.main(HBaseFsck.java:2894)
12/04/03 11:02:58 ERROR zookeeper.ZooKeeperWatcher:
hconnection-0x236738a263a Received unexpected KeeperException,
re-throwing exception
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode
= Session expired for /hbase
at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1021)
at
org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:150)
at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:263)
at
org.apache.hadoop.hbase.zookeeper.ZooKeeperNodeTracker.checkIfBaseNodeAvailable(ZooKeeperNodeTracker.java:208)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.checkIfBaseNodeAvailable(HConnectionManager.java:695)
at
[jira] [Updated] (HBASE-5793) TestHBaseFsck#TestNoHdfsTable test hangs after client retries increased
[
https://issues.apache.org/jira/browse/HBASE-5793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hsieh updated HBASE-5793:
--
Summary: TestHBaseFsck#TestNoHdfsTable test hangs after client retries
increased (was: TestHBaseFsck#TestNoHdfsTable test hangs after client retires
increased)
TestHBaseFsck#TestNoHdfsTable test hangs after client retries increased
---
Key: HBASE-5793
URL: https://issues.apache.org/jira/browse/HBASE-5793
Project: HBase
Issue Type: Sub-task
Affects Versions: 0.96.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Attachments: hbase-5793.patch
After the HBASE-5747 modification, this one particular case hangs.
{code}
mvn test -PlocalTests -Dtest=TestHBaseFsck
{code}
It was hanging on a scan of a table that the test deleted. It expected a call
to thrown an exception after a timeout. HBASE-5747 changed the timeout to a
larger number of retries which caused mvn to fail the test.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5793) TestHBaseFsck#TestNoHdfsTable test hangs after client retires increased
[
https://issues.apache.org/jira/browse/HBASE-5793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hsieh updated HBASE-5793:
--
Summary: TestHBaseFsck#TestNoHdfsTable test hangs after client retires
increased (was: TestHBaseFsck#TestNoHdfsTable test hangs after HBASE-5747)
TestHBaseFsck#TestNoHdfsTable test hangs after client retires increased
---
Key: HBASE-5793
URL: https://issues.apache.org/jira/browse/HBASE-5793
Project: HBase
Issue Type: Sub-task
Affects Versions: 0.96.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Attachments: hbase-5793.patch
After the HBASE-5747 modification, this one particular case hangs.
{code}
mvn test -PlocalTests -Dtest=TestHBaseFsck
{code}
It was hanging on a scan of a table that the test deleted. It expected a call
to thrown an exception after a timeout. HBASE-5747 changed the timeout to a
larger number of retries which caused mvn to fail the test.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5793) TestHBaseFsck#TestNoHdfsTable test hangs after client retries increased
[
https://issues.apache.org/jira/browse/HBASE-5793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hsieh updated HBASE-5793:
--
Resolution: Fixed
Fix Version/s: 0.96.0
0.94.0
0.92.2
0.90.7
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Committed to 0.90/0.92/0.94/0.96/
TestHBaseFsck#TestNoHdfsTable test hangs after client retries increased
---
Key: HBASE-5793
URL: https://issues.apache.org/jira/browse/HBASE-5793
Project: HBase
Issue Type: Sub-task
Affects Versions: 0.96.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0
Attachments: hbase-5793.patch
After the HBASE-5747 modification, this one particular case hangs.
{code}
mvn test -PlocalTests -Dtest=TestHBaseFsck
{code}
It was hanging on a scan of a table that the test deleted. It expected a call
to thrown an exception after a timeout. HBASE-5747 changed the timeout to a
larger number of retries which caused mvn to fail the test.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4094) improve hbck tool to fix more hbase problem
[ https://issues.apache.org/jira/browse/HBASE-4094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-4094: -- Description: The hbck tool(org.apache.hadoop.hbase.util.HBaseFsck) can check and repair consistency problem. some error just be checked but not supply the way to repair, I plan to fix it by other tool(close_region...)or by new method. First, list it and discuss that is it right? Part A:check meta info 1.errors.reportError(ERROR_CODE.NULL_ROOT_REGION,Root Region or some of its attributes are null.); -- after delete the root table,execute hbck tool to check but the tool run error. how to reproduce this error? 2.errors.reportError(ERROR_CODE.NO_META_REGION, .META. is not found on any region.); --after delete the meta table,execute hbck tool to check but the tool run error. how to reproduce this error? 3.errors.reportError(ERROR_CODE.MULTI_META_REGION, .META. is found on more than one region.); -the logic:scan the root table to get META table regioninfo,if META table's regions is more than one,throw the error. HBase allow META table has more than one region,is it? Part B:check Consistency 4.ERROR_CODE.NOT_IN_META_HDFSclose it from regionserver. 5.ERROR_CODE.NOT_IN_META_OR_DEPLOYEDdo nothing,maybe it will be used to fix the chain hole in part C. 6.ERROR_CODE.NOT_IN_METAclose it from regionserver. 7.ERROR_CODE.NOT_IN_HDFS_OR_DEPLOYEDdelete it from META table,it will make a chain hole, when check chain integrity(in part C) to fix it. 8.ERROR_CODE.NOT_IN_HDFSdelete it from META table and close it from regionserver,when check chain integrity(in part C) to fix it. 9.ERROR_CODE.NOT_DEPLOYEDassign it. 10.ERROR_CODE.SHOULD_NOT_BE_DEPLOYEDdelete if from META table and close it from regionserver. 11.ERROR_CODE.MULTI_DEPLOYED---close all from regionservers,and reassign it. 12.ERROR_CODE.SERVER_DOES_NOT_MATCH_METAclose all from regionservers,and reassign it. Part C:check chain Integrity 13.ERROR_CODE.FIRST_REGION_STARTKEY_NOT_EMPTY---treat it as a hole problem(ERROR_CODE.HOLE_IN_REGION_CHAIN). 14.ERROR_CODE.LAST_REGION_ENDKEY_NOT_EMPTY(new add)---treat it as a hole problem(ERROR_CODE.HOLE_IN_REGION_CHAIN). 15.ERROR_CODE.REGION_CYCLEshut down cluster and merge two region by merge tool(org.apache.hadoop.hbase.util.Merge) 16.ERROR_CODE.DUPE_STARTKEYS---shut down cluster and merge two region by merge tool(org.apache.hadoop.hbase.util.Merge) 17.ERROR_CODE.OVERLAP_IN_REGION_CHAIN---shut down cluster and merge two region by merge tool(org.apache.hadoop.hbase.util.Merge) 18.ERROR_CODE.HOLE_IN_REGION_CHAIN---write a new method to fix it,the logic is:for recover the data,collect the regionfo from regionserver and hdfs.if a region's key range is overlaping with the hole range,put it in META table and assign it,maybe it will create overlapping problem,we can fix it by merge tool.if no region be collected,create a new region by the hole key range to fix it. improve hbck tool to fix more hbase problem --- Key: HBASE-4094 URL: https://issues.apache.org/jira/browse/HBASE-4094 Project: HBase Issue Type: New Feature Components: master Affects Versions: 0.90.3 Reporter: feng xu Fix For: 0.90.7 Attachments: HbaseFsck_TableChain.patch Original Estimate: 12h Remaining Estimate: 12h The hbck tool(org.apache.hadoop.hbase.util.HBaseFsck) can check and repair consistency problem. some error just be checked but not supply the way to repair, I plan to fix it by other tool(close_region...)or by new method. First, list it and discuss that is it right? Part A:check meta info 1.errors.reportError(ERROR_CODE.NULL_ROOT_REGION,Root Region or some of its attributes are null.); -- after delete the root table,execute hbck tool to check but the tool run error. how to reproduce this error? 2.errors.reportError(ERROR_CODE.NO_META_REGION, .META. is not found on any region.); --after delete the meta table,execute hbck tool to check but the tool run error. how to reproduce this error? 3.errors.reportError(ERROR_CODE.MULTI_META_REGION, .META. is found on more than one region.); -the logic:scan the root table to get META table regioninfo,if META table's regions is more than one,throw the error. HBase allow META table has more than one region,is it? Part B:check Consistency 4.ERROR_CODE.NOT_IN_META_HDFSclose it from regionserver. 5.ERROR_CODE.NOT_IN_META_OR_DEPLOYEDdo nothing,maybe it will be used to fix the chain hole in part C. 6.ERROR_CODE.NOT_IN_METAclose it from
[jira] [Updated] (HBASE-4094) improve hbck tool to fix more hbase problem
[ https://issues.apache.org/jira/browse/HBASE-4094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-4094: -- Fix Version/s: (was: 0.90.7) Cleaned up jira to follow convention. Marked as duplicate of HBASE-5128 improve hbck tool to fix more hbase problem --- Key: HBASE-4094 URL: https://issues.apache.org/jira/browse/HBASE-4094 Project: HBase Issue Type: New Feature Components: master Affects Versions: 0.90.3 Reporter: feng xu Attachments: HbaseFsck_TableChain.patch Original Estimate: 12h Remaining Estimate: 12h The hbck tool(org.apache.hadoop.hbase.util.HBaseFsck) can check and repair consistency problem. some error just be checked but not supply the way to repair, I plan to fix it by other tool(close_region...)or by new method. First, list it and discuss that is it right? Part A:check meta info 1.errors.reportError(ERROR_CODE.NULL_ROOT_REGION,Root Region or some of its attributes are null.); -- after delete the root table,execute hbck tool to check but the tool run error. how to reproduce this error? 2.errors.reportError(ERROR_CODE.NO_META_REGION, .META. is not found on any region.); --after delete the meta table,execute hbck tool to check but the tool run error. how to reproduce this error? 3.errors.reportError(ERROR_CODE.MULTI_META_REGION, .META. is found on more than one region.); -the logic:scan the root table to get META table regioninfo,if META table's regions is more than one,throw the error. HBase allow META table has more than one region,is it? Part B:check Consistency 4.ERROR_CODE.NOT_IN_META_HDFSclose it from regionserver. 5.ERROR_CODE.NOT_IN_META_OR_DEPLOYEDdo nothing,maybe it will be used to fix the chain hole in part C. 6.ERROR_CODE.NOT_IN_METAclose it from regionserver. 7.ERROR_CODE.NOT_IN_HDFS_OR_DEPLOYEDdelete it from META table,it will make a chain hole, when check chain integrity(in part C) to fix it. 8.ERROR_CODE.NOT_IN_HDFSdelete it from META table and close it from regionserver,when check chain integrity(in part C) to fix it. 9.ERROR_CODE.NOT_DEPLOYEDassign it. 10.ERROR_CODE.SHOULD_NOT_BE_DEPLOYEDdelete if from META table and close it from regionserver. 11.ERROR_CODE.MULTI_DEPLOYED---close all from regionservers,and reassign it. 12.ERROR_CODE.SERVER_DOES_NOT_MATCH_METAclose all from regionservers,and reassign it. Part C:check chain Integrity 13.ERROR_CODE.FIRST_REGION_STARTKEY_NOT_EMPTY---treat it as a hole problem(ERROR_CODE.HOLE_IN_REGION_CHAIN). 14.ERROR_CODE.LAST_REGION_ENDKEY_NOT_EMPTY(new add)---treat it as a hole problem(ERROR_CODE.HOLE_IN_REGION_CHAIN). 15.ERROR_CODE.REGION_CYCLEshut down cluster and merge two region by merge tool(org.apache.hadoop.hbase.util.Merge) 16.ERROR_CODE.DUPE_STARTKEYS---shut down cluster and merge two region by merge tool(org.apache.hadoop.hbase.util.Merge) 17.ERROR_CODE.OVERLAP_IN_REGION_CHAIN---shut down cluster and merge two region by merge tool(org.apache.hadoop.hbase.util.Merge) 18.ERROR_CODE.HOLE_IN_REGION_CHAIN---write a new method to fix it,the logic is:for recover the data,collect the regionfo from regionserver and hdfs.if a region's key range is overlaping with the hole range,put it in META table and assign it,maybe it will create overlapping problem,we can fix it by merge tool.if no region be collected,create a new region by the hole key range to fix it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5781) Zookeeper session got closed while trying to assign the region to RS using hbck -fix
[
https://issues.apache.org/jira/browse/HBASE-5781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hsieh updated HBASE-5781:
--
Priority: Critical (was: Major)
Affects Version/s: 0.96.0
Fix Version/s: 0.94.0
Zookeeper session got closed while trying to assign the region to RS using
hbck -fix
Key: HBASE-5781
URL: https://issues.apache.org/jira/browse/HBASE-5781
Project: HBase
Issue Type: Bug
Components: hbck
Affects Versions: 0.94.0, 0.96.0
Reporter: Kristam Subba Swathi
Assignee: Jonathan Hsieh
Priority: Critical
Fix For: 0.94.0
After running the hbck in the cluster ,it is found that one region is not
assigned
So the hbck -fix is used to fix this
But the assignment didnt happen since the zookeeper session is closed
Please find the attached trace for more details
-
Trying to fix unassigned region...
12/04/03 11:02:57 INFO util.HBaseFsckRepair: Region still in transition,
waiting for it to become assigned: {NAME =
'ufdr,002300,179123498.00871fbd7583512e12c4eb38e900be8d.', STARTKEY =
'002300', ENDKEY = '002311', ENCODED = 00871fbd7583512e12c4eb38e900be8d,}
12/04/03 11:02:58 INFO client.HConnectionManager$HConnectionImplementation:
Closed zookeeper sessionid=0x236738a263a
12/04/03 11:02:58 INFO zookeeper.ZooKeeper: Session: 0x236738a263a closed
ERROR: Region { meta =
ufdr,010444,179123857.01594219211d0035b9586f98954462e1., hdfs =
hdfs://10.18.40.25:9000/hbase/ufdr/01594219211d0035b9586f98954462e1, deployed
= } not deployed on any region server.
Trying to fix unassigned region...
12/04/03 11:02:58 INFO zookeeper.ClientCnxn: EventThread shut down
12/04/03 11:02:58 WARN zookeeper.ZKUtil: hconnection-0x236738a263a Unable
to set watcher on znode (/hbase)
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode
= Session expired for /hbase
at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1021)
at
org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:150)
at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:263)
at
org.apache.hadoop.hbase.zookeeper.ZooKeeperNodeTracker.checkIfBaseNodeAvailable(ZooKeeperNodeTracker.java:208)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.checkIfBaseNodeAvailable(HConnectionManager.java:695)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getMaster(HConnectionManager.java:626)
at org.apache.hadoop.hbase.client.HBaseAdmin.getMaster(HBaseAdmin.java:211)
at org.apache.hadoop.hbase.client.HBaseAdmin.assign(HBaseAdmin.java:1325)
at
org.apache.hadoop.hbase.util.HBaseFsckRepair.forceOfflineInZK(HBaseFsckRepair.java:109)
at
org.apache.hadoop.hbase.util.HBaseFsckRepair.fixUnassigned(HBaseFsckRepair.java:92)
at
org.apache.hadoop.hbase.util.HBaseFsck.tryAssignmentRepair(HBaseFsck.java:1235)
at
org.apache.hadoop.hbase.util.HBaseFsck.checkRegionConsistency(HBaseFsck.java:1351)
at
org.apache.hadoop.hbase.util.HBaseFsck.checkAndFixConsistency(HBaseFsck.java:1114)
at
org.apache.hadoop.hbase.util.HBaseFsck.onlineConsistencyRepair(HBaseFsck.java:356)
at org.apache.hadoop.hbase.util.HBaseFsck.onlineHbck(HBaseFsck.java:375)
at org.apache.hadoop.hbase.util.HBaseFsck.main(HBaseFsck.java:2894)
12/04/03 11:02:58 ERROR zookeeper.ZooKeeperWatcher:
hconnection-0x236738a263a Received unexpected KeeperException,
re-throwing exception
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode
= Session expired for /hbase
at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1021)
at
org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:150)
at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:263)
at
org.apache.hadoop.hbase.zookeeper.ZooKeeperNodeTracker.checkIfBaseNodeAvailable(ZooKeeperNodeTracker.java:208)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.checkIfBaseNodeAvailable(HConnectionManager.java:695)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getMaster(HConnectionManager.java:626)
at org.apache.hadoop.hbase.client.HBaseAdmin.getMaster(HBaseAdmin.java:211)
at org.apache.hadoop.hbase.client.HBaseAdmin.assign(HBaseAdmin.java:1325)
at
[jira] [Updated] (HBASE-5793) TestHBaseFsck#TestNoHdfsTable test hangs after HBASE-5747
[ https://issues.apache.org/jira/browse/HBASE-5793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5793: -- Issue Type: Sub-task (was: Bug) Parent: HBASE-5747 TestHBaseFsck#TestNoHdfsTable test hangs after HBASE-5747 - Key: HBASE-5793 URL: https://issues.apache.org/jira/browse/HBASE-5793 Project: HBase Issue Type: Sub-task Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh After the HBSAE-5747 modification, this one particular case hangs. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5793) TestHBaseFsck#TestNoHdfsTable test hangs after HBASE-5747
[
https://issues.apache.org/jira/browse/HBASE-5793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hsieh updated HBASE-5793:
--
Description:
After the HBASE-5747 modification, this one particular case hangs.
{code}
mvn test -PlocalTests -Dtest=TestHBaseFsck
{code}
It was hanging on a scan of a table that the test deleted. It expected a call
to thrown an exception after a timeout. HBASE-5747 changed the timeout to a
larger number of retries which caused mvn to fail the test.
was:After the HBSAE-5747 modification, this one particular case hangs.
TestHBaseFsck#TestNoHdfsTable test hangs after HBASE-5747
-
Key: HBASE-5793
URL: https://issues.apache.org/jira/browse/HBASE-5793
Project: HBase
Issue Type: Sub-task
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
After the HBASE-5747 modification, this one particular case hangs.
{code}
mvn test -PlocalTests -Dtest=TestHBaseFsck
{code}
It was hanging on a scan of a table that the test deleted. It expected a call
to thrown an exception after a timeout. HBASE-5747 changed the timeout to a
larger number of retries which caused mvn to fail the test.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5793) TestHBaseFsck#TestNoHdfsTable test hangs after HBASE-5747
[
https://issues.apache.org/jira/browse/HBASE-5793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hsieh updated HBASE-5793:
--
Attachment: hbase-5793.patch
TestHBaseFsck#TestNoHdfsTable test hangs after HBASE-5747
-
Key: HBASE-5793
URL: https://issues.apache.org/jira/browse/HBASE-5793
Project: HBase
Issue Type: Sub-task
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Attachments: hbase-5793.patch
After the HBASE-5747 modification, this one particular case hangs.
{code}
mvn test -PlocalTests -Dtest=TestHBaseFsck
{code}
It was hanging on a scan of a table that the test deleted. It expected a call
to thrown an exception after a timeout. HBASE-5747 changed the timeout to a
larger number of retries which caused mvn to fail the test.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5781) Zookeeper session got closed while trying to assign the region to RS using hbck -fix
[
https://issues.apache.org/jira/browse/HBASE-5781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hsieh updated HBASE-5781:
--
Affects Version/s: 0.90.7
0.92.1
Zookeeper session got closed while trying to assign the region to RS using
hbck -fix
Key: HBASE-5781
URL: https://issues.apache.org/jira/browse/HBASE-5781
Project: HBase
Issue Type: Bug
Components: hbck
Affects Versions: 0.90.7, 0.92.1, 0.94.0, 0.96.0
Reporter: Kristam Subba Swathi
Assignee: Jonathan Hsieh
Priority: Critical
Fix For: 0.94.0
After running the hbck in the cluster ,it is found that one region is not
assigned
So the hbck -fix is used to fix this
But the assignment didnt happen since the zookeeper session is closed
Please find the attached trace for more details
-
Trying to fix unassigned region...
12/04/03 11:02:57 INFO util.HBaseFsckRepair: Region still in transition,
waiting for it to become assigned: {NAME =
'ufdr,002300,179123498.00871fbd7583512e12c4eb38e900be8d.', STARTKEY =
'002300', ENDKEY = '002311', ENCODED = 00871fbd7583512e12c4eb38e900be8d,}
12/04/03 11:02:58 INFO client.HConnectionManager$HConnectionImplementation:
Closed zookeeper sessionid=0x236738a263a
12/04/03 11:02:58 INFO zookeeper.ZooKeeper: Session: 0x236738a263a closed
ERROR: Region { meta =
ufdr,010444,179123857.01594219211d0035b9586f98954462e1., hdfs =
hdfs://10.18.40.25:9000/hbase/ufdr/01594219211d0035b9586f98954462e1, deployed
= } not deployed on any region server.
Trying to fix unassigned region...
12/04/03 11:02:58 INFO zookeeper.ClientCnxn: EventThread shut down
12/04/03 11:02:58 WARN zookeeper.ZKUtil: hconnection-0x236738a263a Unable
to set watcher on znode (/hbase)
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode
= Session expired for /hbase
at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1021)
at
org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:150)
at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:263)
at
org.apache.hadoop.hbase.zookeeper.ZooKeeperNodeTracker.checkIfBaseNodeAvailable(ZooKeeperNodeTracker.java:208)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.checkIfBaseNodeAvailable(HConnectionManager.java:695)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getMaster(HConnectionManager.java:626)
at org.apache.hadoop.hbase.client.HBaseAdmin.getMaster(HBaseAdmin.java:211)
at org.apache.hadoop.hbase.client.HBaseAdmin.assign(HBaseAdmin.java:1325)
at
org.apache.hadoop.hbase.util.HBaseFsckRepair.forceOfflineInZK(HBaseFsckRepair.java:109)
at
org.apache.hadoop.hbase.util.HBaseFsckRepair.fixUnassigned(HBaseFsckRepair.java:92)
at
org.apache.hadoop.hbase.util.HBaseFsck.tryAssignmentRepair(HBaseFsck.java:1235)
at
org.apache.hadoop.hbase.util.HBaseFsck.checkRegionConsistency(HBaseFsck.java:1351)
at
org.apache.hadoop.hbase.util.HBaseFsck.checkAndFixConsistency(HBaseFsck.java:1114)
at
org.apache.hadoop.hbase.util.HBaseFsck.onlineConsistencyRepair(HBaseFsck.java:356)
at org.apache.hadoop.hbase.util.HBaseFsck.onlineHbck(HBaseFsck.java:375)
at org.apache.hadoop.hbase.util.HBaseFsck.main(HBaseFsck.java:2894)
12/04/03 11:02:58 ERROR zookeeper.ZooKeeperWatcher:
hconnection-0x236738a263a Received unexpected KeeperException,
re-throwing exception
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode
= Session expired for /hbase
at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1021)
at
org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:150)
at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:263)
at
org.apache.hadoop.hbase.zookeeper.ZooKeeperNodeTracker.checkIfBaseNodeAvailable(ZooKeeperNodeTracker.java:208)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.checkIfBaseNodeAvailable(HConnectionManager.java:695)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getMaster(HConnectionManager.java:626)
at org.apache.hadoop.hbase.client.HBaseAdmin.getMaster(HBaseAdmin.java:211)
at org.apache.hadoop.hbase.client.HBaseAdmin.assign(HBaseAdmin.java:1325)
at
[jira] [Updated] (HBASE-5781) Zookeeper session got closed while trying to assign the region to RS using hbck -fix
[
https://issues.apache.org/jira/browse/HBASE-5781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hsieh updated HBASE-5781:
--
Attachment: hbase-5781.patch
Implemented fix suggested in conversation. Applied to 0.92.x based hbase, and
confirmed that hbck's assignment operations worked.
* Fixed a borked test cluster
* On ok cluster, use hbase shell to closed a region, ran updated hbck to verify
detected, ran 'hbck -fix' to fix assignment and problem was repaired.
Note for this to pass on trunk, HBASE-5993 is needed as well.
Zookeeper session got closed while trying to assign the region to RS using
hbck -fix
Key: HBASE-5781
URL: https://issues.apache.org/jira/browse/HBASE-5781
Project: HBase
Issue Type: Bug
Components: hbck
Affects Versions: 0.90.7, 0.92.1, 0.94.0, 0.96.0
Reporter: Kristam Subba Swathi
Assignee: Jonathan Hsieh
Priority: Critical
Fix For: 0.94.0
Attachments: hbase-5781.patch
After running the hbck in the cluster ,it is found that one region is not
assigned
So the hbck -fix is used to fix this
But the assignment didnt happen since the zookeeper session is closed
Please find the attached trace for more details
-
Trying to fix unassigned region...
12/04/03 11:02:57 INFO util.HBaseFsckRepair: Region still in transition,
waiting for it to become assigned: {NAME =
'ufdr,002300,179123498.00871fbd7583512e12c4eb38e900be8d.', STARTKEY =
'002300', ENDKEY = '002311', ENCODED = 00871fbd7583512e12c4eb38e900be8d,}
12/04/03 11:02:58 INFO client.HConnectionManager$HConnectionImplementation:
Closed zookeeper sessionid=0x236738a263a
12/04/03 11:02:58 INFO zookeeper.ZooKeeper: Session: 0x236738a263a closed
ERROR: Region { meta =
ufdr,010444,179123857.01594219211d0035b9586f98954462e1., hdfs =
hdfs://10.18.40.25:9000/hbase/ufdr/01594219211d0035b9586f98954462e1, deployed
= } not deployed on any region server.
Trying to fix unassigned region...
12/04/03 11:02:58 INFO zookeeper.ClientCnxn: EventThread shut down
12/04/03 11:02:58 WARN zookeeper.ZKUtil: hconnection-0x236738a263a Unable
to set watcher on znode (/hbase)
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode
= Session expired for /hbase
at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1021)
at
org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:150)
at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:263)
at
org.apache.hadoop.hbase.zookeeper.ZooKeeperNodeTracker.checkIfBaseNodeAvailable(ZooKeeperNodeTracker.java:208)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.checkIfBaseNodeAvailable(HConnectionManager.java:695)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getMaster(HConnectionManager.java:626)
at org.apache.hadoop.hbase.client.HBaseAdmin.getMaster(HBaseAdmin.java:211)
at org.apache.hadoop.hbase.client.HBaseAdmin.assign(HBaseAdmin.java:1325)
at
org.apache.hadoop.hbase.util.HBaseFsckRepair.forceOfflineInZK(HBaseFsckRepair.java:109)
at
org.apache.hadoop.hbase.util.HBaseFsckRepair.fixUnassigned(HBaseFsckRepair.java:92)
at
org.apache.hadoop.hbase.util.HBaseFsck.tryAssignmentRepair(HBaseFsck.java:1235)
at
org.apache.hadoop.hbase.util.HBaseFsck.checkRegionConsistency(HBaseFsck.java:1351)
at
org.apache.hadoop.hbase.util.HBaseFsck.checkAndFixConsistency(HBaseFsck.java:1114)
at
org.apache.hadoop.hbase.util.HBaseFsck.onlineConsistencyRepair(HBaseFsck.java:356)
at org.apache.hadoop.hbase.util.HBaseFsck.onlineHbck(HBaseFsck.java:375)
at org.apache.hadoop.hbase.util.HBaseFsck.main(HBaseFsck.java:2894)
12/04/03 11:02:58 ERROR zookeeper.ZooKeeperWatcher:
hconnection-0x236738a263a Received unexpected KeeperException,
re-throwing exception
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode
= Session expired for /hbase
at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1021)
at
org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:150)
at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:263)
at
org.apache.hadoop.hbase.zookeeper.ZooKeeperNodeTracker.checkIfBaseNodeAvailable(ZooKeeperNodeTracker.java:208)
at
[jira] [Updated] (HBASE-5793) TestHBaseFsck#TestNoHdfsTable test hangs after HBASE-5747
[
https://issues.apache.org/jira/browse/HBASE-5793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hsieh updated HBASE-5793:
--
Affects Version/s: 0.96.0
Status: Patch Available (was: Open)
TestHBaseFsck#TestNoHdfsTable test hangs after HBASE-5747
-
Key: HBASE-5793
URL: https://issues.apache.org/jira/browse/HBASE-5793
Project: HBase
Issue Type: Sub-task
Affects Versions: 0.96.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Attachments: hbase-5793.patch
After the HBASE-5747 modification, this one particular case hangs.
{code}
mvn test -PlocalTests -Dtest=TestHBaseFsck
{code}
It was hanging on a scan of a table that the test deleted. It expected a call
to thrown an exception after a timeout. HBASE-5747 changed the timeout to a
larger number of retries which caused mvn to fail the test.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5781) Zookeeper session got closed while trying to assign the region to RS using hbck -fix
[
https://issues.apache.org/jira/browse/HBASE-5781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hsieh updated HBASE-5781:
--
Fix Version/s: 0.96.0
0.92.2
0.90.7
Status: Patch Available (was: Open)
Zookeeper session got closed while trying to assign the region to RS using
hbck -fix
Key: HBASE-5781
URL: https://issues.apache.org/jira/browse/HBASE-5781
Project: HBase
Issue Type: Bug
Components: hbck
Affects Versions: 0.92.1, 0.90.7, 0.94.0, 0.96.0
Reporter: Kristam Subba Swathi
Assignee: Jonathan Hsieh
Priority: Critical
Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0
Attachments: hbase-5781.patch
After running the hbck in the cluster ,it is found that one region is not
assigned
So the hbck -fix is used to fix this
But the assignment didnt happen since the zookeeper session is closed
Please find the attached trace for more details
-
Trying to fix unassigned region...
12/04/03 11:02:57 INFO util.HBaseFsckRepair: Region still in transition,
waiting for it to become assigned: {NAME =
'ufdr,002300,179123498.00871fbd7583512e12c4eb38e900be8d.', STARTKEY =
'002300', ENDKEY = '002311', ENCODED = 00871fbd7583512e12c4eb38e900be8d,}
12/04/03 11:02:58 INFO client.HConnectionManager$HConnectionImplementation:
Closed zookeeper sessionid=0x236738a263a
12/04/03 11:02:58 INFO zookeeper.ZooKeeper: Session: 0x236738a263a closed
ERROR: Region { meta =
ufdr,010444,179123857.01594219211d0035b9586f98954462e1., hdfs =
hdfs://10.18.40.25:9000/hbase/ufdr/01594219211d0035b9586f98954462e1, deployed
= } not deployed on any region server.
Trying to fix unassigned region...
12/04/03 11:02:58 INFO zookeeper.ClientCnxn: EventThread shut down
12/04/03 11:02:58 WARN zookeeper.ZKUtil: hconnection-0x236738a263a Unable
to set watcher on znode (/hbase)
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode
= Session expired for /hbase
at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1021)
at
org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:150)
at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:263)
at
org.apache.hadoop.hbase.zookeeper.ZooKeeperNodeTracker.checkIfBaseNodeAvailable(ZooKeeperNodeTracker.java:208)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.checkIfBaseNodeAvailable(HConnectionManager.java:695)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getMaster(HConnectionManager.java:626)
at org.apache.hadoop.hbase.client.HBaseAdmin.getMaster(HBaseAdmin.java:211)
at org.apache.hadoop.hbase.client.HBaseAdmin.assign(HBaseAdmin.java:1325)
at
org.apache.hadoop.hbase.util.HBaseFsckRepair.forceOfflineInZK(HBaseFsckRepair.java:109)
at
org.apache.hadoop.hbase.util.HBaseFsckRepair.fixUnassigned(HBaseFsckRepair.java:92)
at
org.apache.hadoop.hbase.util.HBaseFsck.tryAssignmentRepair(HBaseFsck.java:1235)
at
org.apache.hadoop.hbase.util.HBaseFsck.checkRegionConsistency(HBaseFsck.java:1351)
at
org.apache.hadoop.hbase.util.HBaseFsck.checkAndFixConsistency(HBaseFsck.java:1114)
at
org.apache.hadoop.hbase.util.HBaseFsck.onlineConsistencyRepair(HBaseFsck.java:356)
at org.apache.hadoop.hbase.util.HBaseFsck.onlineHbck(HBaseFsck.java:375)
at org.apache.hadoop.hbase.util.HBaseFsck.main(HBaseFsck.java:2894)
12/04/03 11:02:58 ERROR zookeeper.ZooKeeperWatcher:
hconnection-0x236738a263a Received unexpected KeeperException,
re-throwing exception
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode
= Session expired for /hbase
at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1021)
at
org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:150)
at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:263)
at
org.apache.hadoop.hbase.zookeeper.ZooKeeperNodeTracker.checkIfBaseNodeAvailable(ZooKeeperNodeTracker.java:208)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.checkIfBaseNodeAvailable(HConnectionManager.java:695)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getMaster(HConnectionManager.java:626)
at org.apache.hadoop.hbase.client.HBaseAdmin.getMaster(HBaseAdmin.java:211)
at
[jira] [Updated] (HBASE-5488) Fixed OfflineMetaRepair bug
[ https://issues.apache.org/jira/browse/HBASE-5488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5488: -- Attachment: hbase-5488-v2.patch Updated patch applies to trunk/0.94/0.92 Fixed OfflineMetaRepair bug Key: HBASE-5488 URL: https://issues.apache.org/jira/browse/HBASE-5488 Project: HBase Issue Type: Bug Affects Versions: 0.90.6 Reporter: gaojinchao Assignee: gaojinchao Priority: Minor Fix For: 0.92.2 Attachments: HBASE-5488-branch92.patch, HBASE-5488-trunk.patch, HBASE-5488_branch90.txt, hbase-5488-v2.patch I want to use OfflineMetaRepair tools and found onbody fix this bugs. I will make a patch. 12/01/05 23:23:30 ERROR util.HBaseFsck: Bailed out due to: java.lang.IllegalArgumentException: Wrong FS: hdfs:// us01-ciqps1-name01.carrieriq.com:9000/hbase/M2M-INTEGRATION-MM_TION-13 25190318714/0003d2ede27668737e192d8430dbe5d0/.regioninfo, expected: file:/// at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:352) at org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:47) at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:368) at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:251) at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.init(ChecksumFileSystem.java:126) at org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:284) at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:398) at org.apache.hadoop.hbase.util.HBaseFsck.loadMetaEntry(HBaseFsck.java:256) at org.apache.hadoop.hbase.util.HBaseFsck.loadTableInfo(HBaseFsck.java:284) at org.apache.hadoop.hbase.util.HBaseFsck.rebuildMeta(HBaseFsck.java:402) at org.apache.hadoop.hbase.util.hbck.OfflineMetaRepair.main(OfflineMetaRe -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5488) OfflineMetaRepair doesn't support hadoop 0.20's fs.default.name property
[ https://issues.apache.org/jira/browse/HBASE-5488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5488: -- Summary: OfflineMetaRepair doesn't support hadoop 0.20's fs.default.name property (was: Fixed OfflineMetaRepair bug ) Changed jira title to be more descriptive OfflineMetaRepair doesn't support hadoop 0.20's fs.default.name property Key: HBASE-5488 URL: https://issues.apache.org/jira/browse/HBASE-5488 Project: HBase Issue Type: Bug Affects Versions: 0.90.6 Reporter: gaojinchao Assignee: gaojinchao Priority: Minor Fix For: 0.92.2 Attachments: HBASE-5488-branch92.patch, HBASE-5488-trunk.patch, HBASE-5488_branch90.txt, hbase-5488-v2.patch I want to use OfflineMetaRepair tools and found onbody fix this bugs. I will make a patch. 12/01/05 23:23:30 ERROR util.HBaseFsck: Bailed out due to: java.lang.IllegalArgumentException: Wrong FS: hdfs:// us01-ciqps1-name01.carrieriq.com:9000/hbase/M2M-INTEGRATION-MM_TION-13 25190318714/0003d2ede27668737e192d8430dbe5d0/.regioninfo, expected: file:/// at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:352) at org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:47) at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:368) at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:251) at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.init(ChecksumFileSystem.java:126) at org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:284) at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:398) at org.apache.hadoop.hbase.util.HBaseFsck.loadMetaEntry(HBaseFsck.java:256) at org.apache.hadoop.hbase.util.HBaseFsck.loadTableInfo(HBaseFsck.java:284) at org.apache.hadoop.hbase.util.HBaseFsck.rebuildMeta(HBaseFsck.java:402) at org.apache.hadoop.hbase.util.hbck.OfflineMetaRepair.main(OfflineMetaRe -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5488) OfflineMetaRepair doesn't support hadoop 0.20's fs.default.name property
[ https://issues.apache.org/jira/browse/HBASE-5488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5488: -- Resolution: Fixed Fix Version/s: 0.96.0 0.94.0 0.90.7 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) @Anoop -- Thanks for checking up on these (I missed this when it came in) @gaojinchao - Thanks for the useful patch! I've tested the failures and the work for me locally. Committed to 0.90/0.92/0.94/0.96-trunk OfflineMetaRepair doesn't support hadoop 0.20's fs.default.name property Key: HBASE-5488 URL: https://issues.apache.org/jira/browse/HBASE-5488 Project: HBase Issue Type: Bug Affects Versions: 0.90.6 Reporter: gaojinchao Assignee: gaojinchao Priority: Minor Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0 Attachments: HBASE-5488-branch92.patch, HBASE-5488-trunk.patch, HBASE-5488_branch90.txt, hbase-5488-v2.patch I want to use OfflineMetaRepair tools and found onbody fix this bugs. I will make a patch. 12/01/05 23:23:30 ERROR util.HBaseFsck: Bailed out due to: java.lang.IllegalArgumentException: Wrong FS: hdfs:// us01-ciqps1-name01.carrieriq.com:9000/hbase/M2M-INTEGRATION-MM_TION-13 25190318714/0003d2ede27668737e192d8430dbe5d0/.regioninfo, expected: file:/// at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:352) at org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:47) at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:368) at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:251) at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.init(ChecksumFileSystem.java:126) at org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:284) at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:398) at org.apache.hadoop.hbase.util.HBaseFsck.loadMetaEntry(HBaseFsck.java:256) at org.apache.hadoop.hbase.util.HBaseFsck.loadTableInfo(HBaseFsck.java:284) at org.apache.hadoop.hbase.util.HBaseFsck.rebuildMeta(HBaseFsck.java:402) at org.apache.hadoop.hbase.util.hbck.OfflineMetaRepair.main(OfflineMetaRe -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5719) Enhance hbck to sideline overlapped mega regions
[ https://issues.apache.org/jira/browse/HBASE-5719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5719: -- Resolution: Fixed Fix Version/s: 0.94.0 0.92.2 0.90.7 Status: Resolved (was: Patch Available) Thanks for the patch and the backports Jimmy! Commited to 0.96/0.94/0.92/0.90. Enhance hbck to sideline overlapped mega regions Key: HBASE-5719 URL: https://issues.apache.org/jira/browse/HBASE-5719 Project: HBase Issue Type: New Feature Components: hbck Affects Versions: 0.94.0, 0.96.0 Reporter: Jimmy Xiang Assignee: Jimmy Xiang Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0 Attachments: 5719.patch, 5719_0.90.patch, 5719_0.92.patch, 5719_0.94.patch, hbase-5719.patch, hbase-5719_0.90.patch, hbase-5719_0.92.patch, hbase-5719_0.94.patch, hbase-5719_v3-new.patch, hbase-5719_v3.patch If there are too many regions in one overlapped group (by default, more than 10), hbck currently doesn't merge them since it takes time. In this case, we can sideline some regions in the group and break the overlapping to fix the inconsistency. Later on, sidelined regions can be bulk loaded manually. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5599) [hbck] handle NO_VERSION_FILE and SHOULD_NOT_BE_DEPLOYED inconsistencies
[ https://issues.apache.org/jira/browse/HBASE-5599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5599: -- Comment: was deleted (was: -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12522208/license.png against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1472//console This message is automatically generated.) [hbck] handle NO_VERSION_FILE and SHOULD_NOT_BE_DEPLOYED inconsistencies Key: HBASE-5599 URL: https://issues.apache.org/jira/browse/HBASE-5599 Project: HBase Issue Type: New Feature Components: hbck Affects Versions: 0.90.6 Reporter: fulin wang Assignee: fulin wang Attachments: 0.90-surefire-report-hbck.html, hbase-5599-0.90.patch, hbase-5599-0.90_v2.patch, hbase-5599-0.90_v3.patch, hbase-5599-0.90_v5.patch, hbase-5599-0.90_v6.patch, hbase-5599-0.90_v7.patch, hbase-5599-0.90_v8, hbase-5599-0.92_v5.patch, hbase-5599-0.94_v5.patch, hbase-5599-trunk_v5.patch, hbase-5599-trunk_v7.patch, hbase-5599-trunk_v8.patch, license.png The hbck tool can not fix the six scenarios. 1. Version file does not exist in root dir. Fix: I try to create a version file by 'FSUtils.setVersion' method. 2. [REGIONNAME][KEY] on HDFS, but not listed in META or deployed on any region server. Fix: I get region info form the hdfs file, this region info write to '.META.' table. 3. [REGIONNAME][KEY] not in META, but deployed on [SERVERNAME] Fix: I get region info form the hdfs file, this region info write to '.META.' table. 4. [REGIONNAME] should not be deployed according to META, but is deployed on [SERVERNAME] Fix: Close this region. 5. First region should start with an empty key. You need to create a new region and regioninfo in HDFS to plug the hole. Fix: The region info is not in hdfs and .META., so it create a empty region for this error. 6. There is a hole in the region chain between [KEY] and [KEY]. You need to create a new regioninfo and region dir in hdfs to plug the hole. Fix: The region info is not in hdfs and .META., so it create a empty region for this hole. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5599) [hbck] handle NO_VERSION_FILE and SHOULD_NOT_BE_DEPLOYED inconsistencies
[ https://issues.apache.org/jira/browse/HBASE-5599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5599: -- Resolution: Fixed Fix Version/s: 0.96.0 0.94.0 0.92.2 0.90.7 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Fulin, thanks for the patch! Committed to 0.90/0.92/0.94/0.96-trunk [hbck] handle NO_VERSION_FILE and SHOULD_NOT_BE_DEPLOYED inconsistencies Key: HBASE-5599 URL: https://issues.apache.org/jira/browse/HBASE-5599 Project: HBase Issue Type: New Feature Components: hbck Affects Versions: 0.90.6 Reporter: fulin wang Assignee: fulin wang Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0 Attachments: 0.90-surefire-report-hbck.html, hbase-5599-0.90.patch, hbase-5599-0.90_v2.patch, hbase-5599-0.90_v3.patch, hbase-5599-0.90_v5.patch, hbase-5599-0.90_v6.patch, hbase-5599-0.90_v7.patch, hbase-5599-0.90_v8, hbase-5599-0.92_v5.patch, hbase-5599-0.94_v5.patch, hbase-5599-92-v8.patch, hbase-5599-trunk_v5.patch, hbase-5599-trunk_v7.patch, hbase-5599-trunk_v8.patch, license.png The hbck tool can not fix the six scenarios. 1. Version file does not exist in root dir. Fix: I try to create a version file by 'FSUtils.setVersion' method. 2. [REGIONNAME][KEY] on HDFS, but not listed in META or deployed on any region server. Fix: I get region info form the hdfs file, this region info write to '.META.' table. 3. [REGIONNAME][KEY] not in META, but deployed on [SERVERNAME] Fix: I get region info form the hdfs file, this region info write to '.META.' table. 4. [REGIONNAME] should not be deployed according to META, but is deployed on [SERVERNAME] Fix: Close this region. 5. First region should start with an empty key. You need to create a new region and regioninfo in HDFS to plug the hole. Fix: The region info is not in hdfs and .META., so it create a empty region for this error. 6. There is a hole in the region chain between [KEY] and [KEY]. You need to create a new regioninfo and region dir in hdfs to plug the hole. Fix: The region info is not in hdfs and .META., so it create a empty region for this hole. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5599) [hbck] handle NO_VERSION_FILE and SHOULD_NOT_BE_DEPLOYED inconsistencies
[ https://issues.apache.org/jira/browse/HBASE-5599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5599: -- Attachment: hbase-5599-92-v8.patch Attached tweaked patch applied to 0.92 [hbck] handle NO_VERSION_FILE and SHOULD_NOT_BE_DEPLOYED inconsistencies Key: HBASE-5599 URL: https://issues.apache.org/jira/browse/HBASE-5599 Project: HBase Issue Type: New Feature Components: hbck Affects Versions: 0.90.6 Reporter: fulin wang Assignee: fulin wang Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0 Attachments: 0.90-surefire-report-hbck.html, hbase-5599-0.90.patch, hbase-5599-0.90_v2.patch, hbase-5599-0.90_v3.patch, hbase-5599-0.90_v5.patch, hbase-5599-0.90_v6.patch, hbase-5599-0.90_v7.patch, hbase-5599-0.90_v8, hbase-5599-0.92_v5.patch, hbase-5599-0.94_v5.patch, hbase-5599-92-v8.patch, hbase-5599-trunk_v5.patch, hbase-5599-trunk_v7.patch, hbase-5599-trunk_v8.patch, license.png The hbck tool can not fix the six scenarios. 1. Version file does not exist in root dir. Fix: I try to create a version file by 'FSUtils.setVersion' method. 2. [REGIONNAME][KEY] on HDFS, but not listed in META or deployed on any region server. Fix: I get region info form the hdfs file, this region info write to '.META.' table. 3. [REGIONNAME][KEY] not in META, but deployed on [SERVERNAME] Fix: I get region info form the hdfs file, this region info write to '.META.' table. 4. [REGIONNAME] should not be deployed according to META, but is deployed on [SERVERNAME] Fix: Close this region. 5. First region should start with an empty key. You need to create a new region and regioninfo in HDFS to plug the hole. Fix: The region info is not in hdfs and .META., so it create a empty region for this error. 6. There is a hole in the region chain between [KEY] and [KEY]. You need to create a new regioninfo and region dir in hdfs to plug the hole. Fix: The region info is not in hdfs and .META., so it create a empty region for this hole. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5645) [findbugs] Fix correctness warnings
[ https://issues.apache.org/jira/browse/HBASE-5645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5645: -- Resolution: Fixed Fix Version/s: 0.96.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Lgtm. Thanks David and thanks Uma and Stack for taking a look. I've committed to trunk. [findbugs] Fix correctness warnings --- Key: HBASE-5645 URL: https://issues.apache.org/jira/browse/HBASE-5645 Project: HBase Issue Type: Sub-task Components: scripts Reporter: Jonathan Hsieh Assignee: David S. Wang Fix For: 0.96.0 Attachments: HBASE-5645-2.patch, HBASE-5645-3.patch, HBASE-5645.patch, HBASE-5645.patch, hbase-5645-2-tweak.patch See https://builds.apache.org/job/PreCommit-HBASE-Build/1313//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Fix the warnings in the correctness section. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5599) [hbck] handle NO_VERSION_FILE and SHOULD_NOT_BE_DEPLOYED inconsistencies
[ https://issues.apache.org/jira/browse/HBASE-5599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5599: -- Fix Version/s: (was: 0.90.6) Summary: [hbck] handle NO_VERSION_FILE and SHOULD_NOT_BE_DEPLOYED inconsistencies (was: The hbck tool can not fix the six scenarios, it is NO_VERSION_FILE, NOT_IN_META_OR_DEPLOYED, NOT_IN_META, SHOULD_NOT_BE_DEPLOYED, FIRST_REGION_STARTKEY_NOT_EMPTY, HOLE_IN_REGION_CHAIN.) Renaming due to new scope of patch. [hbck] handle NO_VERSION_FILE and SHOULD_NOT_BE_DEPLOYED inconsistencies Key: HBASE-5599 URL: https://issues.apache.org/jira/browse/HBASE-5599 Project: HBase Issue Type: New Feature Components: hbck Affects Versions: 0.90.6 Reporter: fulin wang Assignee: fulin wang Attachments: 0.90-surefire-report-hbck.html, hbase-5599-0.90.patch, hbase-5599-0.90_v2.patch, hbase-5599-0.90_v3.patch, hbase-5599-0.90_v5.patch, hbase-5599-0.90_v6.patch, hbase-5599-0.90_v7.patch, hbase-5599-0.92_v5.patch, hbase-5599-0.94_v5.patch, hbase-5599-trunk_v5.patch, hbase-5599-trunk_v7.patch The hbck tool can not fix the six scenarios. 1. Version file does not exist in root dir. Fix: I try to create a version file by 'FSUtils.setVersion' method. 2. [REGIONNAME][KEY] on HDFS, but not listed in META or deployed on any region server. Fix: I get region info form the hdfs file, this region info write to '.META.' table. 3. [REGIONNAME][KEY] not in META, but deployed on [SERVERNAME] Fix: I get region info form the hdfs file, this region info write to '.META.' table. 4. [REGIONNAME] should not be deployed according to META, but is deployed on [SERVERNAME] Fix: Close this region. 5. First region should start with an empty key. You need to create a new region and regioninfo in HDFS to plug the hole. Fix: The region info is not in hdfs and .META., so it create a empty region for this error. 6. There is a hole in the region chain between [KEY] and [KEY]. You need to create a new regioninfo and region dir in hdfs to plug the hole. Fix: The region info is not in hdfs and .META., so it create a empty region for this hole. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5653) [findbugs] fix perf warnings
[ https://issues.apache.org/jira/browse/HBASE-5653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5653: -- Resolution: Fixed Fix Version/s: 0.96.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Thanks for the patch! I've committed to trunk. [findbugs] fix perf warnings Key: HBASE-5653 URL: https://issues.apache.org/jira/browse/HBASE-5653 Project: HBase Issue Type: Sub-task Components: scripts Reporter: Jonathan Hsieh Assignee: Uma Maheswara Rao G Fix For: 0.96.0 Attachments: HBASE-5653.patch, HBASE-5653.patch See https://builds.apache.org/job/PreCommit-HBASE-Build/1313//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html#Warnings_PERFORMANCE -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5599) [hbck] handle NO_VERSION_FILE and SHOULD_NOT_BE_DEPLOYED inconsistencies
[ https://issues.apache.org/jira/browse/HBASE-5599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5599: -- Attachment: license.png Attached a screen shot that shows ASF grant being selected. [hbck] handle NO_VERSION_FILE and SHOULD_NOT_BE_DEPLOYED inconsistencies Key: HBASE-5599 URL: https://issues.apache.org/jira/browse/HBASE-5599 Project: HBase Issue Type: New Feature Components: hbck Affects Versions: 0.90.6 Reporter: fulin wang Assignee: fulin wang Attachments: 0.90-surefire-report-hbck.html, hbase-5599-0.90.patch, hbase-5599-0.90_v2.patch, hbase-5599-0.90_v3.patch, hbase-5599-0.90_v5.patch, hbase-5599-0.90_v6.patch, hbase-5599-0.90_v7.patch, hbase-5599-0.92_v5.patch, hbase-5599-0.94_v5.patch, hbase-5599-trunk_v5.patch, hbase-5599-trunk_v7.patch, license.png The hbck tool can not fix the six scenarios. 1. Version file does not exist in root dir. Fix: I try to create a version file by 'FSUtils.setVersion' method. 2. [REGIONNAME][KEY] on HDFS, but not listed in META or deployed on any region server. Fix: I get region info form the hdfs file, this region info write to '.META.' table. 3. [REGIONNAME][KEY] not in META, but deployed on [SERVERNAME] Fix: I get region info form the hdfs file, this region info write to '.META.' table. 4. [REGIONNAME] should not be deployed according to META, but is deployed on [SERVERNAME] Fix: Close this region. 5. First region should start with an empty key. You need to create a new region and regioninfo in HDFS to plug the hole. Fix: The region info is not in hdfs and .META., so it create a empty region for this error. 6. There is a hole in the region chain between [KEY] and [KEY]. You need to create a new regioninfo and region dir in hdfs to plug the hole. Fix: The region info is not in hdfs and .META., so it create a empty region for this hole. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5645) [findbugs] Fix correctness warnings
[ https://issues.apache.org/jira/browse/HBASE-5645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5645: -- Comment: was deleted (was: lgtm. I just checked in HBASE_5653 that reduced the find bugs number, which likely breaks this patch. I'm going to submit patch to find out the new number and the tweak v2 to reflect this new number.) [findbugs] Fix correctness warnings --- Key: HBASE-5645 URL: https://issues.apache.org/jira/browse/HBASE-5645 Project: HBase Issue Type: Sub-task Components: scripts Reporter: Jonathan Hsieh Assignee: David S. Wang Attachments: HBASE-5645-2.patch, HBASE-5645.patch, HBASE-5645.patch See https://builds.apache.org/job/PreCommit-HBASE-Build/1313//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Fix the warnings in the correctness section. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5645) [findbugs] Fix correctness warnings
[ https://issues.apache.org/jira/browse/HBASE-5645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5645: -- Attachment: hbase-5645-2-tweak.patch Attached a tweaked version that excludes an HLog change already included by HBASE-5653 and removed count update so we can get new count. Letting bot run to get new findbugs count. [findbugs] Fix correctness warnings --- Key: HBASE-5645 URL: https://issues.apache.org/jira/browse/HBASE-5645 Project: HBase Issue Type: Sub-task Components: scripts Reporter: Jonathan Hsieh Assignee: David S. Wang Attachments: HBASE-5645-2.patch, HBASE-5645.patch, HBASE-5645.patch, hbase-5645-2-tweak.patch See https://builds.apache.org/job/PreCommit-HBASE-Build/1313//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Fix the warnings in the correctness section. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5680) Hbase94 and Hbase 92.2 is not compatible with the Hadoop 23.1
[
https://issues.apache.org/jira/browse/HBASE-5680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hsieh updated HBASE-5680:
--
Comment: was deleted
(was: How to test:
* Setup hadoop hdfs 0.23.x cluster.
* Compile against 1.0.x hadoop. {{mvn clean package -DskipTests}}
* Make HBASE_HOME ./target/hbase-version/hbase-version
* Replace hadoop jars in ./target/hbase-version/hbase-version/lib/ with hadoop
23 jars.
* start hbase.
* Look at master log.
)
Hbase94 and Hbase 92.2 is not compatible with the Hadoop 23.1
--
Key: HBASE-5680
URL: https://issues.apache.org/jira/browse/HBASE-5680
Project: HBase
Issue Type: Bug
Components: master
Reporter: Kristam Subba Swathi
Assignee: Jonathan Hsieh
Attachments: hbase-5680-94.patch
Hmaster is not able to start because of the following error
Please find the following error
2012-03-30 11:12:19,487 FATAL org.apache.hadoop.hbase.master.HMaster:
Unhandled exception. Starting shutdown.
java.lang.NoClassDefFoundError:
org/apache/hadoop/hdfs/protocol/FSConstants$SafeModeAction
at org.apache.hadoop.hbase.util.FSUtils.waitOnSafeMode(FSUtils.java:524)
at
org.apache.hadoop.hbase.master.MasterFileSystem.checkRootDir(MasterFileSystem.java:324)
at
org.apache.hadoop.hbase.master.MasterFileSystem.createInitialFileSystemLayout(MasterFileSystem.java:127)
at
org.apache.hadoop.hbase.master.MasterFileSystem.init(MasterFileSystem.java:112)
at
org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:496)
at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:363)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.ClassNotFoundException:
org.apache.hadoop.hdfs.protocol.FSConstants$SafeModeAction
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
... 7 more
There is a change in the FSConstants
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5680) Hbase94 and Hbase 92.2 is not compatible with the Hadoop 23.1
[ https://issues.apache.org/jira/browse/HBASE-5680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5680: -- Affects Version/s: 0.96.0 0.94.0 0.90.6 0.92.1 Status: Patch Available (was: Open) Hbase94 and Hbase 92.2 is not compatible with the Hadoop 23.1 -- Key: HBASE-5680 URL: https://issues.apache.org/jira/browse/HBASE-5680 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.92.1, 0.90.6, 0.94.0, 0.96.0 Reporter: Kristam Subba Swathi Assignee: Jonathan Hsieh Attachments: hbase-5680-94.patch Hmaster is not able to start because of the following error Please find the following error 2012-03-30 11:12:19,487 FATAL org.apache.hadoop.hbase.master.HMaster: Unhandled exception. Starting shutdown. java.lang.NoClassDefFoundError: org/apache/hadoop/hdfs/protocol/FSConstants$SafeModeAction at org.apache.hadoop.hbase.util.FSUtils.waitOnSafeMode(FSUtils.java:524) at org.apache.hadoop.hbase.master.MasterFileSystem.checkRootDir(MasterFileSystem.java:324) at org.apache.hadoop.hbase.master.MasterFileSystem.createInitialFileSystemLayout(MasterFileSystem.java:127) at org.apache.hadoop.hbase.master.MasterFileSystem.init(MasterFileSystem.java:112) at org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:496) at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:363) at java.lang.Thread.run(Thread.java:662) Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hdfs.protocol.FSConstants$SafeModeAction at java.net.URLClassLoader$1.run(URLClassLoader.java:202) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:190) at java.lang.ClassLoader.loadClass(ClassLoader.java:306) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) at java.lang.ClassLoader.loadClass(ClassLoader.java:247) ... 7 more There is a change in the FSConstants -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5680) Improve compatibility warning about HBase with Hadoop 0.23.x
[ https://issues.apache.org/jira/browse/HBASE-5680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5680: -- Resolution: Fixed Fix Version/s: 0.96.0 0.94.0 0.92.2 0.90.7 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Improve compatibility warning about HBase with Hadoop 0.23.x Key: HBASE-5680 URL: https://issues.apache.org/jira/browse/HBASE-5680 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.90.6, 0.92.1, 0.94.0, 0.96.0 Reporter: Kristam Subba Swathi Assignee: Jonathan Hsieh Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0 Attachments: hbase-5680-94.patch Hmaster is not able to start because of the following error Please find the following error 2012-03-30 11:12:19,487 FATAL org.apache.hadoop.hbase.master.HMaster: Unhandled exception. Starting shutdown. java.lang.NoClassDefFoundError: org/apache/hadoop/hdfs/protocol/FSConstants$SafeModeAction at org.apache.hadoop.hbase.util.FSUtils.waitOnSafeMode(FSUtils.java:524) at org.apache.hadoop.hbase.master.MasterFileSystem.checkRootDir(MasterFileSystem.java:324) at org.apache.hadoop.hbase.master.MasterFileSystem.createInitialFileSystemLayout(MasterFileSystem.java:127) at org.apache.hadoop.hbase.master.MasterFileSystem.init(MasterFileSystem.java:112) at org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:496) at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:363) at java.lang.Thread.run(Thread.java:662) Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hdfs.protocol.FSConstants$SafeModeAction at java.net.URLClassLoader$1.run(URLClassLoader.java:202) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:190) at java.lang.ClassLoader.loadClass(ClassLoader.java:306) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) at java.lang.ClassLoader.loadClass(ClassLoader.java:247) ... 7 more There is a change in the FSConstants -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5734) Change hbck sideline root
[ https://issues.apache.org/jira/browse/HBASE-5734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5734: -- Attachment: hbase-5734-90.patch Attched version that applies to 0.90. Change hbck sideline root - Key: HBASE-5734 URL: https://issues.apache.org/jira/browse/HBASE-5734 Project: HBase Issue Type: Improvement Components: hbck Affects Versions: 0.94.0, 0.96.0 Reporter: Jimmy Xiang Assignee: Jimmy Xiang Priority: Trivial Fix For: 0.96.0 Attachments: hbase-5734-90.patch, hbase-5734.patch Currently hbck sideline root is the root which can run into permission issue. We can change it to /hbck -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5734) Change hbck sideline root
[ https://issues.apache.org/jira/browse/HBASE-5734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5734: -- Resolution: Fixed Fix Version/s: 0.94.0 0.92.2 0.90.7 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Change hbck sideline root - Key: HBASE-5734 URL: https://issues.apache.org/jira/browse/HBASE-5734 Project: HBase Issue Type: Improvement Components: hbck Affects Versions: 0.94.0, 0.96.0 Reporter: Jimmy Xiang Assignee: Jimmy Xiang Priority: Trivial Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0 Attachments: hbase-5734-90.patch, hbase-5734.patch Currently hbck sideline root is the root which can run into permission issue. We can change it to /hbck -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5644) [findbugs] Fix null pointer warnings.
[ https://issues.apache.org/jira/browse/HBASE-5644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5644: -- Resolution: Fixed Fix Version/s: 0.96.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) @Uma. Nice work. I've committed a slightly modified version of your patch (fixed some spacing nits). [findbugs] Fix null pointer warnings. - Key: HBASE-5644 URL: https://issues.apache.org/jira/browse/HBASE-5644 Project: HBase Issue Type: Sub-task Components: scripts Reporter: Jonathan Hsieh Assignee: Uma Maheswara Rao G Fix For: 0.96.0 Attachments: HBASE-5644.patch, HBASE-5644.patch, NullPointerFindBugs_Analysis.xlsx See https://builds.apache.org/job/PreCommit-HBASE-Build/1313//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Fix the NP category -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4348) Add metrics for regions in transition
[ https://issues.apache.org/jira/browse/HBASE-4348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-4348: -- Status: Patch Available (was: Open) Add metrics for regions in transition - Key: HBASE-4348 URL: https://issues.apache.org/jira/browse/HBASE-4348 Project: HBase Issue Type: Improvement Components: metrics Affects Versions: 0.92.0 Reporter: Todd Lipcon Assignee: Himanshu Vashishtha Priority: Minor Labels: noob Fix For: 0.96.0 Attachments: 4348-metrics-v3.patch, 4348-v1.patch, 4348-v2.patch, RITs.png, RegionInTransitions2.png, hbase-4348.patch, metrics-v2.patch The following metrics would be useful for monitoring the master: - the number of regions in transition - the number of regions in transition that have been in transition for more than a minute - how many seconds has the oldest region-in-transition been in transition -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4348) Add metrics for regions in transition
[ https://issues.apache.org/jira/browse/HBASE-4348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-4348: -- Attachment: hbase-4348-v4.patch minor fix to deal with conflict Add metrics for regions in transition - Key: HBASE-4348 URL: https://issues.apache.org/jira/browse/HBASE-4348 Project: HBase Issue Type: Improvement Components: metrics Affects Versions: 0.92.0 Reporter: Todd Lipcon Assignee: Himanshu Vashishtha Priority: Minor Labels: noob Fix For: 0.96.0 Attachments: 4348-metrics-v3.patch, 4348-v1.patch, 4348-v2.patch, RITs.png, RegionInTransitions2.png, hbase-4348-v4.patch, hbase-4348.patch, metrics-v2.patch The following metrics would be useful for monitoring the master: - the number of regions in transition - the number of regions in transition that have been in transition for more than a minute - how many seconds has the oldest region-in-transition been in transition -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4348) Add metrics for regions in transition
[ https://issues.apache.org/jira/browse/HBASE-4348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-4348: -- Resolution: Fixed Status: Resolved (was: Patch Available) Add metrics for regions in transition - Key: HBASE-4348 URL: https://issues.apache.org/jira/browse/HBASE-4348 Project: HBase Issue Type: Improvement Components: metrics Affects Versions: 0.92.0 Reporter: Todd Lipcon Assignee: Himanshu Vashishtha Priority: Minor Labels: noob Fix For: 0.96.0 Attachments: 4348-metrics-v3.patch, 4348-v1.patch, 4348-v2.patch, RITs.png, RegionInTransitions2.png, hbase-4348-v4.patch, hbase-4348.patch, metrics-v2.patch The following metrics would be useful for monitoring the master: - the number of regions in transition - the number of regions in transition that have been in transition for more than a minute - how many seconds has the oldest region-in-transition been in transition -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5680) Hbase94 and Hbase 92.2 is not compatible with the Hadoop 23.1
[ https://issues.apache.org/jira/browse/HBASE-5680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5680: -- Attachment: hbase-5680-94.patch Attached version tested on 0.94. Hbase94 and Hbase 92.2 is not compatible with the Hadoop 23.1 -- Key: HBASE-5680 URL: https://issues.apache.org/jira/browse/HBASE-5680 Project: HBase Issue Type: Bug Components: master Reporter: Kristam Subba Swathi Assignee: Jonathan Hsieh Attachments: hbase-5680-94.patch Hmaster is not able to start because of the following error Please find the following error 2012-03-30 11:12:19,487 FATAL org.apache.hadoop.hbase.master.HMaster: Unhandled exception. Starting shutdown. java.lang.NoClassDefFoundError: org/apache/hadoop/hdfs/protocol/FSConstants$SafeModeAction at org.apache.hadoop.hbase.util.FSUtils.waitOnSafeMode(FSUtils.java:524) at org.apache.hadoop.hbase.master.MasterFileSystem.checkRootDir(MasterFileSystem.java:324) at org.apache.hadoop.hbase.master.MasterFileSystem.createInitialFileSystemLayout(MasterFileSystem.java:127) at org.apache.hadoop.hbase.master.MasterFileSystem.init(MasterFileSystem.java:112) at org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:496) at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:363) at java.lang.Thread.run(Thread.java:662) Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hdfs.protocol.FSConstants$SafeModeAction at java.net.URLClassLoader$1.run(URLClassLoader.java:202) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:190) at java.lang.ClassLoader.loadClass(ClassLoader.java:306) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) at java.lang.ClassLoader.loadClass(ClassLoader.java:247) ... 7 more There is a change in the FSConstants -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5628) Improve performance of uberhbck
[ https://issues.apache.org/jira/browse/HBASE-5628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5628: -- Comment: was deleted (was: - parallelize the reads of .regioninfo files in the diagnostic section of repairs.) Improve performance of uberhbck --- Key: HBASE-5628 URL: https://issues.apache.org/jira/browse/HBASE-5628 Project: HBase Issue Type: Improvement Components: hbck Affects Versions: 0.90.7, 0.92.2, 0.94.0, 0.96.0 Reporter: Jonathan Hsieh During reviews of HBASE-5128 there are several opportunities investigate for improving the performance of the tool. - Change regionInfoMap and tablesInfo from TreeMap to HashMap. - Change some full region set reloads to be incremental to require fewer passes. - Cache meta for subsequent calls of closeRegionSileneglyAndWait -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5714) Add write permissions check before any hbck run that modifies hdfs.
[ https://issues.apache.org/jira/browse/HBASE-5714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5714: -- Description: We encoutered a situation where hbck was run by an under-privileged user that was unable to write/modify/merge regions due to hdfs perms. Unfortunately, this user was alerted of this after several minutes of read-only operations. hbck should fail early by having a write perm check and providing actionable advice to the hbase admin. Maybe something like: Current user yy does not have write perms to hbase home. Please run hbck as hdfs user xxx was: We encoutered a situation where hbase was run by a different user and was unable to write/modify/merge regions due to hdfs perms. Unfortunately, this happened after several minutes of read-only operations. hbck should fail early by having a write perm check and providing actionable advice to the hbase admin. Maybe something like: Current user yy does not have write perms to hbase home. Please run hbck as hdfs user xxx Add write permissions check before any hbck run that modifies hdfs. --- Key: HBASE-5714 URL: https://issues.apache.org/jira/browse/HBASE-5714 Project: HBase Issue Type: Sub-task Components: hbck Affects Versions: 0.90.6, 0.92.2, 0.94.0, 0.96.0 Reporter: Jonathan Hsieh We encoutered a situation where hbck was run by an under-privileged user that was unable to write/modify/merge regions due to hdfs perms. Unfortunately, this user was alerted of this after several minutes of read-only operations. hbck should fail early by having a write perm check and providing actionable advice to the hbase admin. Maybe something like: Current user yy does not have write perms to hbase home. Please run hbck as hdfs user xxx -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5599) The hbck tool can not fix the six scenarios, it is NO_VERSION_FILE, NOT_IN_META_OR_DEPLOYED, NOT_IN_META, SHOULD_NOT_BE_DEPLOYED, FIRST_REGION_STARTKEY_NOT_EMPTY, HOLE_IN
[ https://issues.apache.org/jira/browse/HBASE-5599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5599: -- Assignee: fulin wang Status: Patch Available (was: Open) The hbck tool can not fix the six scenarios, it is NO_VERSION_FILE, NOT_IN_META_OR_DEPLOYED, NOT_IN_META, SHOULD_NOT_BE_DEPLOYED, FIRST_REGION_STARTKEY_NOT_EMPTY, HOLE_IN_REGION_CHAIN. Key: HBASE-5599 URL: https://issues.apache.org/jira/browse/HBASE-5599 Project: HBase Issue Type: New Feature Components: hbck Affects Versions: 0.90.6 Reporter: fulin wang Assignee: fulin wang Fix For: 0.90.6 Attachments: hbase-5599-0.90.patch, hbase-5599-0.90_v2.patch, hbase-5599-0.90_v3.patch, hbase-5599-0.90_v5.patch, hbase-5599-0.90_v6.patch, hbase-5599-0.92_v5.patch, hbase-5599-0.94_v5.patch, hbase-5599-trunk_v5.patch The hbck tool can not fix the six scenarios. 1. Version file does not exist in root dir. Fix: I try to create a version file by 'FSUtils.setVersion' method. 2. [REGIONNAME][KEY] on HDFS, but not listed in META or deployed on any region server. Fix: I get region info form the hdfs file, this region info write to '.META.' table. 3. [REGIONNAME][KEY] not in META, but deployed on [SERVERNAME] Fix: I get region info form the hdfs file, this region info write to '.META.' table. 4. [REGIONNAME] should not be deployed according to META, but is deployed on [SERVERNAME] Fix: Close this region. 5. First region should start with an empty key. You need to create a new region and regioninfo in HDFS to plug the hole. Fix: The region info is not in hdfs and .META., so it create a empty region for this error. 6. There is a hole in the region chain between [KEY] and [KEY]. You need to create a new regioninfo and region dir in hdfs to plug the hole. Fix: The region info is not in hdfs and .META., so it create a empty region for this hole. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5213) hbase master stop does not bring down backup masters
[ https://issues.apache.org/jira/browse/HBASE-5213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5213: -- Fix Version/s: 0.92.2 0.90.7 hbase master stop does not bring down backup masters -- Key: HBASE-5213 URL: https://issues.apache.org/jira/browse/HBASE-5213 Project: HBase Issue Type: Bug Affects Versions: 0.90.5, 0.92.0, 0.94.0, 0.96.0 Reporter: Gregory Chanan Assignee: Gregory Chanan Priority: Minor Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0 Attachments: HBASE-5213-v0-trunk.patch, HBASE-5213-v1-trunk.patch, HBASE-5213-v2-90.patch, HBASE-5213-v2-92.patch, HBASE-5213-v2-trunk.patch Typing hbase master stop produces the following message: stop Start cluster shutdown; Master signals RegionServer shutdown It seems like backup masters should be considered part of the cluster, but they are not brought down by hbase master stop. stop-hbase.sh does correctly bring down the backup masters. The same behavior is observed when a client app makes use of the client API HBaseAdmin.shutdown() http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/HBaseAdmin.html#shutdown() -- this isn't too surprising since I think hbase master stop just calls this API. It seems like HBASE-1448 address this; perhaps there was a regression? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5213) hbase master stop does not bring down backup masters
[ https://issues.apache.org/jira/browse/HBASE-5213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5213: -- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) hbase master stop does not bring down backup masters -- Key: HBASE-5213 URL: https://issues.apache.org/jira/browse/HBASE-5213 Project: HBase Issue Type: Bug Affects Versions: 0.90.5, 0.92.0, 0.94.0, 0.96.0 Reporter: Gregory Chanan Assignee: Gregory Chanan Priority: Minor Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0 Attachments: HBASE-5213-v0-trunk.patch, HBASE-5213-v1-trunk.patch, HBASE-5213-v2-90.patch, HBASE-5213-v2-92.patch, HBASE-5213-v2-trunk.patch Typing hbase master stop produces the following message: stop Start cluster shutdown; Master signals RegionServer shutdown It seems like backup masters should be considered part of the cluster, but they are not brought down by hbase master stop. stop-hbase.sh does correctly bring down the backup masters. The same behavior is observed when a client app makes use of the client API HBaseAdmin.shutdown() http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/HBaseAdmin.html#shutdown() -- this isn't too surprising since I think hbase master stop just calls this API. It seems like HBASE-1448 address this; perhaps there was a regression? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5685) [findbugs] Exclude Protobuf warnings from wire compat patches.
[ https://issues.apache.org/jira/browse/HBASE-5685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5685: -- Attachment: hbase-5598.patch [findbugs] Exclude Protobuf warnings from wire compat patches. -- Key: HBASE-5685 URL: https://issues.apache.org/jira/browse/HBASE-5685 Project: HBase Issue Type: Sub-task Components: scripts Affects Versions: 0.96.0 Reporter: Jonathan Hsieh Attachments: hbase-5598.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5685) [findbugs] Exclude Protobuf warnings from wire compat patches.
[ https://issues.apache.org/jira/browse/HBASE-5685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5685: -- Assignee: Jonathan Hsieh Affects Version/s: 0.96.0 Status: Patch Available (was: Open) [findbugs] Exclude Protobuf warnings from wire compat patches. -- Key: HBASE-5685 URL: https://issues.apache.org/jira/browse/HBASE-5685 Project: HBase Issue Type: Sub-task Components: scripts Affects Versions: 0.96.0 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Attachments: hbase-5598.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5213) hbase master stop does not bring down backup masters
[ https://issues.apache.org/jira/browse/HBASE-5213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5213: -- Fix Version/s: (was: 0.92.2) 0.96.0 0.94.0 Noticed that this was committed to 0.94 and 0.96/trunk but not to 0.92 or 0.90. running some tests on thos verions and if clean will commit. hbase master stop does not bring down backup masters -- Key: HBASE-5213 URL: https://issues.apache.org/jira/browse/HBASE-5213 Project: HBase Issue Type: Bug Affects Versions: 0.90.5, 0.92.0, 0.94.0, 0.96.0 Reporter: Gregory Chanan Assignee: Gregory Chanan Priority: Minor Fix For: 0.94.0, 0.96.0 Attachments: HBASE-5213-v0-trunk.patch, HBASE-5213-v1-trunk.patch, HBASE-5213-v2-90.patch, HBASE-5213-v2-92.patch, HBASE-5213-v2-trunk.patch Typing hbase master stop produces the following message: stop Start cluster shutdown; Master signals RegionServer shutdown It seems like backup masters should be considered part of the cluster, but they are not brought down by hbase master stop. stop-hbase.sh does correctly bring down the backup masters. The same behavior is observed when a client app makes use of the client API HBaseAdmin.shutdown() http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/HBaseAdmin.html#shutdown() -- this isn't too surprising since I think hbase master stop just calls this API. It seems like HBASE-1448 address this; perhaps there was a regression? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5685) [findbugs] Exclude Protobuf warnings from wire compat patches.
[ https://issues.apache.org/jira/browse/HBASE-5685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5685: -- Resolution: Fixed Fix Version/s: 0.96.0 Status: Resolved (was: Patch Available) [findbugs] Exclude Protobuf warnings from wire compat patches. -- Key: HBASE-5685 URL: https://issues.apache.org/jira/browse/HBASE-5685 Project: HBase Issue Type: Sub-task Components: scripts Affects Versions: 0.96.0 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Fix For: 0.96.0 Attachments: hbase-5598.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5638) Backport to 0.90 and 0.92 - NPE reading ZK config in HBase
[ https://issues.apache.org/jira/browse/HBASE-5638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5638: -- Comment: was deleted (was: -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12520414/HBASE-5638-0.92-v1.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1341//console This message is automatically generated.) Backport to 0.90 and 0.92 - NPE reading ZK config in HBase -- Key: HBASE-5638 URL: https://issues.apache.org/jira/browse/HBASE-5638 Project: HBase Issue Type: Sub-task Components: zookeeper Affects Versions: 0.90.6, 0.92.1 Reporter: Matteo Bertozzi Priority: Minor Fix For: 0.90.7, 0.92.2 Attachments: HBASE-5633-0.90.patch, HBASE-5633-0.92.patch, HBASE-5638-0.90-v1.patch, HBASE-5638-0.92-v1.patch, HBASE-5638-trunk-v1.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5638) Backport to 0.90 and 0.92 - NPE reading ZK config in HBase
[ https://issues.apache.org/jira/browse/HBASE-5638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5638: -- Comment: was deleted (was: -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12520414/HBASE-5638-0.92-v1.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1340//console This message is automatically generated.) Backport to 0.90 and 0.92 - NPE reading ZK config in HBase -- Key: HBASE-5638 URL: https://issues.apache.org/jira/browse/HBASE-5638 Project: HBase Issue Type: Sub-task Components: zookeeper Affects Versions: 0.90.6, 0.92.1 Reporter: Matteo Bertozzi Priority: Minor Fix For: 0.90.7, 0.92.2 Attachments: HBASE-5633-0.90.patch, HBASE-5633-0.92.patch, HBASE-5638-0.90-v1.patch, HBASE-5638-0.92-v1.patch, HBASE-5638-trunk-v1.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5638) Backport to 0.90 and 0.92 - NPE reading ZK config in HBase
[ https://issues.apache.org/jira/browse/HBASE-5638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5638: -- Assignee: Matteo Bertozzi Backport to 0.90 and 0.92 - NPE reading ZK config in HBase -- Key: HBASE-5638 URL: https://issues.apache.org/jira/browse/HBASE-5638 Project: HBase Issue Type: Sub-task Components: zookeeper Affects Versions: 0.90.6, 0.92.1 Reporter: Matteo Bertozzi Assignee: Matteo Bertozzi Priority: Minor Fix For: 0.90.7, 0.92.2 Attachments: HBASE-5633-0.90.patch, HBASE-5633-0.92.patch, HBASE-5638-0.90-v1.patch, HBASE-5638-0.92-v1.patch, HBASE-5638-trunk-v1.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5638) Backport to 0.90 and 0.92 - NPE reading ZK config in HBase
[ https://issues.apache.org/jira/browse/HBASE-5638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5638: -- Resolution: Fixed Fix Version/s: 0.94.1 0.94.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Thanks Matteo, and thanks for review stack. Backport to 0.90 and 0.92 - NPE reading ZK config in HBase -- Key: HBASE-5638 URL: https://issues.apache.org/jira/browse/HBASE-5638 Project: HBase Issue Type: Sub-task Components: zookeeper Affects Versions: 0.90.6, 0.92.1 Reporter: Matteo Bertozzi Assignee: Matteo Bertozzi Priority: Minor Fix For: 0.90.7, 0.92.2, 0.94.0, 0.94.1 Attachments: HBASE-5633-0.90.patch, HBASE-5633-0.92.patch, HBASE-5638-0.90-v1.patch, HBASE-5638-0.90-v2.patch, HBASE-5638-0.92-v1.patch, HBASE-5638-0.92-v2.patch, HBASE-5638-trunk-v1.patch, HBASE-5638-trunk-v2.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5675) Create table fails if we keep refreshing master's UI for task monitor status
[
https://issues.apache.org/jira/browse/HBASE-5675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hsieh updated HBASE-5675:
--
Affects Version/s: (was: 0.94.0)
Create table fails if we keep refreshing master's UI for task monitor status
Key: HBASE-5675
URL: https://issues.apache.org/jira/browse/HBASE-5675
Project: HBase
Issue Type: Bug
Components: master
Affects Versions: 0.90.4, 0.92.0
Reporter: Mubarak Seyed
Assignee: Mubarak Seyed
Labels: noob
I tried to create a table with 2K pre-split regions, region assignment was in
middle and i was keep refreshing master's web UI to find the status of the
task using task monitor, table creation was failed and {{META}} was showing
2K regions with server location value is {{null}} and regions weren't
deployed onto region-servers.
{code}
table_ACreating table table_A
java.io.IOException: java.io.IOException:
java.util.ConcurrentModificationException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at
org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:95)
at
org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:79)
at
org.apache.hadoop.hbase.client.HBaseAdmin.createTableAsync(HBaseAdmin.java:384)
at
org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:294)
at com.test.tools.hbase.schema.createIfNotExists(schema.java:520)
at com.test.tools.hbase.schema.main(schema.java:627)
Caused by: org.apache.hadoop.ipc.RemoteException: java.io.IOException:
java.util.ConcurrentModificationException
at java.util.SubList.checkForComodification(AbstractList.java:752)
at java.util.SubList.add(AbstractList.java:632)
at java.util.SubList.add(AbstractList.java:633)
at java.util.SubList.add(AbstractList.java:633)
..
..
at java.util.SubList.add(AbstractList.java:633)
at java.util.AbstractList.add(AbstractList.java:91)
at
org.apache.hadoop.hbase.monitoring.TaskMonitor.createStatus(TaskMonitor.java:76)
at org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:510)
at org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:490)
at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:853)
at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:813)
at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:780)
at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:570)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1039)
at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:771)
at
org.apache.hadoop.hbase.ipc.HBaseRPC$Invoker.invoke(HBaseRPC.java:257)
at $Proxy5.createTable(Unknown Source)
at
org.apache.hadoop.hbase.client.HBaseAdmin.createTableAsync(HBaseAdmin.java:382)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5642) [findbugs] Exclude Thrift and Protobuf warnings
[ https://issues.apache.org/jira/browse/HBASE-5642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5642: -- Resolution: Fixed Fix Version/s: 0.96.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed to trunk [findbugs] Exclude Thrift and Protobuf warnings --- Key: HBASE-5642 URL: https://issues.apache.org/jira/browse/HBASE-5642 Project: HBase Issue Type: Sub-task Components: build Affects Versions: 0.96.0 Reporter: Jonathan Hsieh Assignee: Uma Maheswara Rao G Fix For: 0.96.0 Attachments: HBASE-5642.patch, HBASE-5642.patch Exclude thrift and protobuf warnings since these are machine generated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5128) [uber hbck] Online automated repair of table integrity and region consistency problems
[
https://issues.apache.org/jira/browse/HBASE-5128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hsieh updated HBASE-5128:
--
Release Note:
HBaseFsck (hbck) has been updated with new repair capabilities. hbck is a tool
for checking the region consistency and the table integrity invariants of a
running HBase cluster. Checking region consistency verifies that .META.,
region deployment on region servers and the state of data in HDFS (.regioninfo
files) all are in accordance. Table integrity checks verify that all possible
row keys resolve to exactly one region of a table -- e.g. there are no
individual degenerate or backwards regions; no holes between regions; and no
overlapping regions. Previously hbck had the ability to diagnose
inconsistencies but only had the ability to repair deployment region
consistency problems. The updated version now has been augmented with the
ability repair region consistency problems in .META. (by patching holes),
repair overlapping regions (via merging), patch region holes (by fabricating
new regions), and detecting and adopting orphaned regions (by fabricating new
.regioninfo file if it is missing in a region's dir).
Caveats:
* The new hbck selects repairs assuming that HDFS as ground truth, the previous
version treated .META. as ground truth.
* The hbck '-fix' option is present but deprecated and replaced with
'-fixAssignments' option.
* This tool adds APIs in 0.90.7, 0.92.2 and 0.94.0 for clean repairs. The 0.90
version of the tool is compatible with HBase 0.90+, but may require restarting
the master or individual individual regionserver for table
enable/disable/delete commands to work properly.
was:
HBaseFsck (hbck) has been updated with new repair capabilities. hbck is a tool
for checking the region consistency and the table integrity invariants of a
running HBase cluster. Checking region consistency verifies that .META.,
region deployment on region servers and the state of data in HDFS (.regioninfo
files) all are in accordance. Table integrity checks verify that all possible
row keys resolve to exactly one region of a table -- e.g. there are no
individual degenerate or backwards regions; no holes between regions; and no
overlapping regions. Previously hbck had the ability to diagnose
inconsistencies but only had the ability to repair deployment region
consistency problems. The updated version now has been augmented with the
ability repair region consistency problems in .META. (by patching holes),
repair overlapping regions (via merging), patch region holes (by fabricating
new regions), and detecting and adopting orphaned regions (by fabricating new
.regioninfo file if it is missing in a region's dir).
Caveats:
* The new hbck selects repairs assuming that HDFS as ground truth, the previous
version treated .META. as ground truth.
* The hbck '-fix' option is present but deprecated and replaced with
-fixAssignments option.
* This tool adds APIs in 0.90.7, 0.92.2 and 0.94.0 for clean repairs. The 0.90
version of he tool is compatible with HBase 0.90+, but may require restarting
the master or individual individual regionserver for table
enable/disable/delete commands to work properly.
[uber hbck] Online automated repair of table integrity and region consistency
problems
--
Key: HBASE-5128
URL: https://issues.apache.org/jira/browse/HBASE-5128
Project: HBase
Issue Type: New Feature
Components: hbck
Affects Versions: 0.90.5, 0.92.0, 0.94.0, 0.96.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0
Attachments: 5128-trunk.addendum, hbase-5128-0.90-v2.patch,
hbase-5128-0.90-v2b.patch, hbase-5128-0.90-v4.patch,
hbase-5128-0.92-v2.patch, hbase-5128-0.92-v4.patch, hbase-5128-0.94-v2.patch,
hbase-5128-0.94-v4.patch, hbase-5128-trunk-v2.patch, hbase-5128-trunk.patch,
hbase-5128-v3.patch, hbase-5128-v4.patch
The current (0.90.5, 0.92.0rc2) versions of hbck detects most of region
consistency and table integrity invariant violations. However with '-fix' it
can only automatically repair region consistency cases having to do with
deployment problems. This updated version should be able to handle all cases
(including a new orphan regiondir case). When complete will likely deprecate
the OfflineMetaRepair tool and subsume several open META-hole related issue.
Here's the approach (from the comment of at the top of the new version of the
file).
{code}
/**
* HBaseFsck (hbck) is a tool for checking and repairing region consistency
and
* table integrity.
*
* Region consistency checks verify that META, region deployment on
* region servers and the state of data in HDFS (.regioninfo
[jira] [Updated] (HBASE-5651) [findbugs] Address wait/notify synchronization/inconsistency in sync
[ https://issues.apache.org/jira/browse/HBASE-5651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5651: -- Summary: [findbugs] Address wait/notify synchronization/inconsistency in sync (was: [findbugs] Address wiat/notify synchronization/inconsistency in sync) [findbugs] Address wait/notify synchronization/inconsistency in sync Key: HBASE-5651 URL: https://issues.apache.org/jira/browse/HBASE-5651 Project: HBase Issue Type: Sub-task Components: scripts Reporter: Jonathan Hsieh See https://builds.apache.org/job/PreCommit-HBASE-Build/1313//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html#Warnings_MT_CORRECTNESS fix classes IS,LI,MWM, NN, SWL, UG, UW -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5652) [findbugs] Fix lock release on all paths
[ https://issues.apache.org/jira/browse/HBASE-5652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5652: -- Description: See https://builds.apache.org/job/PreCommit-HBASE-Build/1313//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html#Warnings_MT_CORRECTNESS Category UL Summary: [findbugs] Fix lock release on all paths (was: [find) [findbugs] Fix lock release on all paths - Key: HBASE-5652 URL: https://issues.apache.org/jira/browse/HBASE-5652 Project: HBase Issue Type: Sub-task Components: scripts Reporter: Jonathan Hsieh See https://builds.apache.org/job/PreCommit-HBASE-Build/1313//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html#Warnings_MT_CORRECTNESS Category UL -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5128) [uber hbck] Online automated repair of table integrity and region consistency problems
[
https://issues.apache.org/jira/browse/HBASE-5128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hsieh updated HBASE-5128:
--
Release Note:
HBaseFsck (hbck) has been updated with new repair capabilities. hbck is a tool
for checking the region consistency and the table integrity invariants of a
running HBase cluster. Checking region consistency verifies that .META.,
region deployment on region servers and the state of data in HDFS (.regioninfo
files) all are in accordance. Table integrity checks verify that all possible
row keys resolve to exactly one region of a table -- e.g. there are no
individual degenerate or backwards regions; no holes between regions; and no
overlapping regions. Previously hbck had the ability to diagnose
inconsistencies but only had the ability to repair deployment region
consistency problems. The updated version now has been augmented with the
ability repair region consistency problems in .META. (by patching holes),
repair overlapping regions (via merging), patch region holes (by fabricating
new regions), and detecting and adopting orphaned regions (by fabricating new
.regioninfo file if it is missing in a region's dir).
Caveats:
* The new hbck selects repairs assuming that HDFS as ground truth, the previous
version treated .META. as ground truth.
* The hbck '-fix' option is present but deprecated and replaced with
-fixAssignments option.
* This tool adds APIs in 0.90.7, 0.92.2 and 0.94.0 for clean repairs. The 0.90
version of he tool is compatible with HBase 0.90+, but may require restarting
the master or individual individual regionserver for table
enable/disable/delete commands to work properly.
Updated release notes.
[uber hbck] Online automated repair of table integrity and region consistency
problems
--
Key: HBASE-5128
URL: https://issues.apache.org/jira/browse/HBASE-5128
Project: HBase
Issue Type: New Feature
Components: hbck
Affects Versions: 0.90.5, 0.92.0, 0.94.0, 0.96.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0
Attachments: 5128-trunk.addendum, hbase-5128-0.90-v2.patch,
hbase-5128-0.90-v2b.patch, hbase-5128-0.90-v4.patch,
hbase-5128-0.92-v2.patch, hbase-5128-0.92-v4.patch, hbase-5128-0.94-v2.patch,
hbase-5128-0.94-v4.patch, hbase-5128-trunk-v2.patch, hbase-5128-trunk.patch,
hbase-5128-v3.patch, hbase-5128-v4.patch
The current (0.90.5, 0.92.0rc2) versions of hbck detects most of region
consistency and table integrity invariant violations. However with '-fix' it
can only automatically repair region consistency cases having to do with
deployment problems. This updated version should be able to handle all cases
(including a new orphan regiondir case). When complete will likely deprecate
the OfflineMetaRepair tool and subsume several open META-hole related issue.
Here's the approach (from the comment of at the top of the new version of the
file).
{code}
/**
* HBaseFsck (hbck) is a tool for checking and repairing region consistency
and
* table integrity.
*
* Region consistency checks verify that META, region deployment on
* region servers and the state of data in HDFS (.regioninfo files) all are in
* accordance.
*
* Table integrity checks verify that that all possible row keys can resolve
to
* exactly one region of a table. This means there are no individual
degenerate
* or backwards regions; no holes between regions; and that there no
overlapping
* regions.
*
* The general repair strategy works in these steps.
* 1) Repair Table Integrity on HDFS. (merge or fabricate regions)
* 2) Repair Region Consistency with META and assignments
*
* For table integrity repairs, the tables their region directories are
scanned
* for .regioninfo files. Each table's integrity is then verified. If there
* are any orphan regions (regions with no .regioninfo files), or holes, new
* regions are fabricated. Backwards regions are sidelined as well as empty
* degenerate (endkey==startkey) regions. If there are any overlapping
regions,
* a new region is created and all data is merged into the new region.
*
* Table integrity repairs deal solely with HDFS and can be done offline --
the
* hbase region servers or master do not need to be running. These phase can
be
* use to completely reconstruct the META table in an offline fashion.
*
* Region consistency requires three conditions -- 1) valid .regioninfo file
* present in an hdfs region dir, 2) valid row with .regioninfo data in META,
* and 3) a region is deployed only at the regionserver that is was assigned
to.
*
* Region consistency requires hbck
[jira] [Updated] (HBASE-5637) Fix failing 0.90 TestHMsg testcase introduced by HBASE-5563.
[ https://issues.apache.org/jira/browse/HBASE-5637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5637: -- Attachment: hbase-5637.patch Fix failing 0.90 TestHMsg testcase introduced by HBASE-5563. Key: HBASE-5637 URL: https://issues.apache.org/jira/browse/HBASE-5637 Project: HBase Issue Type: Bug Affects Versions: 0.90.7 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Attachments: hbase-5637.patch After committing HBASE-5128 and HBASE-5563 to the 0.90 branch, Ted noticed that TestHMsg#getList started to fail consistently. This updates the test to deal with the updated equality semantics in HBASE-5563. This fixes that. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5637) Fix failing 0.90 TestHMsg testcase introduced by HBASE-5563.
[ https://issues.apache.org/jira/browse/HBASE-5637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5637: -- Status: Patch Available (was: Open) Fix failing 0.90 TestHMsg testcase introduced by HBASE-5563. Key: HBASE-5637 URL: https://issues.apache.org/jira/browse/HBASE-5637 Project: HBase Issue Type: Bug Affects Versions: 0.90.7 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Attachments: hbase-5637.patch After committing HBASE-5128 and HBASE-5563 to the 0.90 branch, Ted noticed that TestHMsg#getList started to fail consistently. This updates the test to deal with the updated equality semantics in HBASE-5563. This fixes that. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5596) Few minor bugs from HBASE-5209
[ https://issues.apache.org/jira/browse/HBASE-5596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5596: -- Attachment: hbase-5596-0.94.patch one-line change to apply to 0.94/0.92 Few minor bugs from HBASE-5209 -- Key: HBASE-5596 URL: https://issues.apache.org/jira/browse/HBASE-5596 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.92.1, 0.94.0 Reporter: David S. Wang Assignee: David S. Wang Priority: Minor Attachments: HBASE-5596.patch, hbase-5596-0.94.patch A few leftover bugs from HBASE-5209. Comments are documented here: https://reviews.apache.org/r/3892/ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5637) Fix failing 0.90 TestHMsg testcase introduced by HBASE-5563.
[ https://issues.apache.org/jira/browse/HBASE-5637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5637: -- Comment: was deleted (was: -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12519976/hbase-5637.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1306//console This message is automatically generated.) Fix failing 0.90 TestHMsg testcase introduced by HBASE-5563. Key: HBASE-5637 URL: https://issues.apache.org/jira/browse/HBASE-5637 Project: HBase Issue Type: Bug Affects Versions: 0.90.7 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Attachments: hbase-5637.patch After committing HBASE-5128 and HBASE-5563 to the 0.90 branch, Ted noticed that TestHMsg#getList started to fail consistently. This updates the test to deal with the updated equality semantics in HBASE-5563. This fixes that. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5637) Fix failing 0.90 TestHMsg testcase introduced by HBASE-5563.
[ https://issues.apache.org/jira/browse/HBASE-5637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5637: -- Resolution: Fixed Fix Version/s: 0.90.7 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Fix failing 0.90 TestHMsg testcase introduced by HBASE-5563. Key: HBASE-5637 URL: https://issues.apache.org/jira/browse/HBASE-5637 Project: HBase Issue Type: Bug Affects Versions: 0.90.7 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Fix For: 0.90.7 Attachments: hbase-5637.patch After committing HBASE-5128 and HBASE-5563 to the 0.90 branch, Ted noticed that TestHMsg#getList started to fail consistently. This updates the test to deal with the updated equality semantics in HBASE-5563. This fixes that. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5596) Few minor bugs from HBASE-5209
[ https://issues.apache.org/jira/browse/HBASE-5596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5596: -- Resolution: Fixed Fix Version/s: 0.96.0 0.94.0 0.92.2 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Few minor bugs from HBASE-5209 -- Key: HBASE-5596 URL: https://issues.apache.org/jira/browse/HBASE-5596 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.92.1, 0.94.0 Reporter: David S. Wang Assignee: David S. Wang Priority: Minor Fix For: 0.92.2, 0.94.0, 0.96.0 Attachments: HBASE-5596.patch, hbase-5596-0.94.patch A few leftover bugs from HBASE-5209. Comments are documented here: https://reviews.apache.org/r/3892/ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5633) NPE reading ZK config in HBase
[
https://issues.apache.org/jira/browse/HBASE-5633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hsieh updated HBASE-5633:
--
Assignee: Matteo Bertozzi
NPE reading ZK config in HBase
--
Key: HBASE-5633
URL: https://issues.apache.org/jira/browse/HBASE-5633
Project: HBase
Issue Type: Bug
Components: zookeeper
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Priority: Minor
Fix For: 0.94.0
Attachments: HBASE-5633-0.90.patch, HBASE-5633-0.92.patch,
HBASE-5633-v1.patch, HBASE-5633-v2.patch
If zoo.cfg contains server.* (server.0=server0:2888:3888\n) and
cluster.distributed property (in hbase-site.xml) is empty we get an NPE in
parseZooCfg().
The easy way to reproduce the bug is running
org.apache.hbase.zookeeper.TestHQuorumPeer with hbase-site.xml containing:
{code}
property
namehbase.cluster.distributed/name
value/value
/property
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5563) HRegionInfo#compareTo should compare regionId as well
[ https://issues.apache.org/jira/browse/HBASE-5563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5563: -- Fix Version/s: 0.90.7 HRegionInfo#compareTo should compare regionId as well - Key: HBASE-5563 URL: https://issues.apache.org/jira/browse/HBASE-5563 Project: HBase Issue Type: Bug Affects Versions: 0.90.5, 0.92.0, 0.94.0, 0.96.0 Reporter: chunhui shen Assignee: chunhui shen Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0 Attachments: HBASE-5563.patch, HBASE-5563v2.patch, HBASE-5563v2.patch, hbase-5563-0.90.patch, hbase-5563-v3-0.92.patch, hbase-5563-v3.patch In the one region multi assigned case, we could find that two regions have the same table name, same startKey, same endKey, and different regionId, so these two regions are same in TreeMap but different in HashMap. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5618) SplitLogManager - prevent unnecessary attempts to resubmits
[ https://issues.apache.org/jira/browse/HBASE-5618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5618: -- Component/s: zookeeper wal SplitLogManager - prevent unnecessary attempts to resubmits --- Key: HBASE-5618 URL: https://issues.apache.org/jira/browse/HBASE-5618 Project: HBase Issue Type: Improvement Components: wal, zookeeper Reporter: Prakash Khemani Currently once a watch fires that the task node has been updated (hearbeated) by the worker, the splitlogmanager still quite some time before it updates the last heard from time. This is because the manager currently schedules another getDataSetWatch() and only after that finishes will it update the task's last heard from time. This leads to a large number of zk-BadVersion warnings when resubmission is continuously attempted and it fails. Two changes should be made (1) On a resubmission failure because of BadVersion the task's lastUpdate time should get upped. (2) The task's lastUpdate time should get upped as soon as the nodeDataChanged() watch fires and without waiting for getDataSetWatch() to complete. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4910) thrift scannerstopwithfilter not honoring stop row
[ https://issues.apache.org/jira/browse/HBASE-4910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-4910: -- Component/s: thrift thrift scannerstopwithfilter not honoring stop row -- Key: HBASE-4910 URL: https://issues.apache.org/jira/browse/HBASE-4910 Project: HBase Issue Type: Sub-task Components: client, regionserver, thrift Reporter: Nicolas Spiegelberg Fix For: 0.96.0 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5599) The hbck tool can not fix the six scenarios, it is NO_VERSION_FILE, NOT_IN_META_OR_DEPLOYED, NOT_IN_META, SHOULD_NOT_BE_DEPLOYED, FIRST_REGION_STARTKEY_NOT_EMPTY, HOLE_IN
[ https://issues.apache.org/jira/browse/HBASE-5599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5599: -- Summary: The hbck tool can not fix the six scenarios, it is NO_VERSION_FILE, NOT_IN_META_OR_DEPLOYED, NOT_IN_META, SHOULD_NOT_BE_DEPLOYED, FIRST_REGION_STARTKEY_NOT_EMPTY, HOLE_IN_REGION_CHAIN. (was: The hbkc tool can not fix the six scenarios, it is NO_VERSION_FILE, NOT_IN_META_OR_DEPLOYED, NOT_IN_META, SHOULD_NOT_BE_DEPLOYED, FIRST_REGION_STARTKEY_NOT_EMPTY, HOLE_IN_REGION_CHAIN.) The hbck tool can not fix the six scenarios, it is NO_VERSION_FILE, NOT_IN_META_OR_DEPLOYED, NOT_IN_META, SHOULD_NOT_BE_DEPLOYED, FIRST_REGION_STARTKEY_NOT_EMPTY, HOLE_IN_REGION_CHAIN. Key: HBASE-5599 URL: https://issues.apache.org/jira/browse/HBASE-5599 Project: HBase Issue Type: New Feature Components: hbck Affects Versions: 0.90.6 Reporter: fulin wang Fix For: 0.90.6 Attachments: hbase-5599-0.90.patch, hbase-5599-0.90_v2.patch, hbase-5599-0.90_v3.patch The hbck tool can not fix the six scenarios. 1. Version file does not exist in root dir. Fix: I try to create a version file by 'FSUtils.setVersion' method. 2. [REGIONNAME][KEY] on HDFS, but not listed in META or deployed on any region server. Fix: I get region info form the hdfs file, this region info write to '.META.' table. 3. [REGIONNAME][KEY] not in META, but deployed on [SERVERNAME] Fix: I get region info form the hdfs file, this region info write to '.META.' table. 4. [REGIONNAME] should not be deployed according to META, but is deployed on [SERVERNAME] Fix: Close this region. 5. First region should start with an empty key. You need to create a new region and regioninfo in HDFS to plug the hole. Fix: The region info is not in hdfs and .META., so it create a empty region for this error. 6. There is a hole in the region chain between [KEY] and [KEY]. You need to create a new regioninfo and region dir in hdfs to plug the hole. Fix: The region info is not in hdfs and .META., so it create a empty region for this hole. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3852) ThriftServer leaks scanners
[ https://issues.apache.org/jira/browse/HBASE-3852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-3852: -- Component/s: thrift ThriftServer leaks scanners --- Key: HBASE-3852 URL: https://issues.apache.org/jira/browse/HBASE-3852 Project: HBase Issue Type: Bug Components: thrift Affects Versions: 0.90.2 Reporter: Jean-Daniel Cryans Priority: Critical Fix For: 0.94.1 Attachments: 3852.txt, thrift2-scanner.patch The scannerMap in ThriftServer relies on the user to clean it by closing the scanner. If that doesn't happen, the ResultScanner will stay in the thrift server's memory and if any pre-fetching was done, it will also start accumulating Results (with all their data). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5466) Opening a table also opens the metatable and never closes it.
[ https://issues.apache.org/jira/browse/HBASE-5466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5466: -- Fix Version/s: 0.94.0 This happend pre-0.94 branch, added fixver that this is in 0.94. Opening a table also opens the metatable and never closes it. - Key: HBASE-5466 URL: https://issues.apache.org/jira/browse/HBASE-5466 Project: HBase Issue Type: Bug Components: client Affects Versions: 0.90.5, 0.92.0 Reporter: Ashley Taylor Assignee: Ashley Taylor Fix For: 0.90.7, 0.92.1, 0.94.0 Attachments: MetaScanner_HBASE_5466(2).patch, MetaScanner_HBASE_5466(3).patch, MetaScanner_HBASE_5466.patch Having upgraded to CDH3U3 version of hbase we found we had a zookeeper connection leak, tracking it down we found that closing the connection will only close the zookeeper connection if all calls to get the connection have been closed, there is incCount and decCount in the HConnection class, When a table is opened it makes a call to the metascanner class which opens a connection to the meta table, this table never gets closed. This caused the count in the HConnection class to never return to zero meaning that the zookeeper connection will not close when we close all the tables or calling HConnectionManager.deleteConnection(config, true); -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5586) [replication] NPE in ReplicationSource when creating a stream to an inexistent cluster
[
https://issues.apache.org/jira/browse/HBASE-5586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hsieh updated HBASE-5586:
--
Component/s: replication
[replication] NPE in ReplicationSource when creating a stream to an
inexistent cluster
--
Key: HBASE-5586
URL: https://issues.apache.org/jira/browse/HBASE-5586
Project: HBase
Issue Type: Bug
Components: replication
Affects Versions: 0.90.5
Reporter: Jean-Daniel Cryans
Assignee: Jean-Daniel Cryans
Fix For: 0.92.2, 0.94.0, 0.96.0
Attachments: 5586-v2.txt, HBASE-5586-trunk.patch, HBASE-5586.java,
HBASE-5586.java
This is from 0.92.1-ish:
{noformat}
2012-03-15 09:52:16,589 ERROR
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource:
Unexpected exception in ReplicationSource, currentPath=null
java.lang.NullPointerException
at
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.chooseSinks(ReplicationSource.java:223)
at
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.connectToPeers(ReplicationSource.java:442)
at
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.run(ReplicationSource.java:246)
{noformat}
I wanted to add a replication stream to a cluster that wasn't existing yet so
that the logs would be buffered until then. This should just be treated as if
there was no region servers.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5128) [uber hbck] Enable hbck to automatically repair table integrity problems as well as region consistency problems while online.
[
https://issues.apache.org/jira/browse/HBASE-5128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hsieh updated HBASE-5128:
--
Attachment: hbase-5128-v4.patch
hbase-5128-0.94-v4.patch
hbase-5128-0.92-v4.patch
Updated to address ted's last concern, arcanist fixes, and a handful of findbug
fixes.
[uber hbck] Enable hbck to automatically repair table integrity problems as
well as region consistency problems while online.
-
Key: HBASE-5128
URL: https://issues.apache.org/jira/browse/HBASE-5128
Project: HBase
Issue Type: New Feature
Components: hbck
Affects Versions: 0.90.5, 0.92.0, 0.94.0, 0.96.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0
Attachments: hbase-5128-0.90-v2.patch, hbase-5128-0.90-v2b.patch,
hbase-5128-0.92-v2.patch, hbase-5128-0.92-v4.patch, hbase-5128-0.94-v2.patch,
hbase-5128-0.94-v4.patch, hbase-5128-trunk-v2.patch, hbase-5128-trunk.patch,
hbase-5128-v3.patch, hbase-5128-v4.patch
The current (0.90.5, 0.92.0rc2) versions of hbck detects most of region
consistency and table integrity invariant violations. However with '-fix' it
can only automatically repair region consistency cases having to do with
deployment problems. This updated version should be able to handle all cases
(including a new orphan regiondir case). When complete will likely deprecate
the OfflineMetaRepair tool and subsume several open META-hole related issue.
Here's the approach (from the comment of at the top of the new version of the
file).
{code}
/**
* HBaseFsck (hbck) is a tool for checking and repairing region consistency
and
* table integrity.
*
* Region consistency checks verify that META, region deployment on
* region servers and the state of data in HDFS (.regioninfo files) all are in
* accordance.
*
* Table integrity checks verify that that all possible row keys can resolve
to
* exactly one region of a table. This means there are no individual
degenerate
* or backwards regions; no holes between regions; and that there no
overlapping
* regions.
*
* The general repair strategy works in these steps.
* 1) Repair Table Integrity on HDFS. (merge or fabricate regions)
* 2) Repair Region Consistency with META and assignments
*
* For table integrity repairs, the tables their region directories are
scanned
* for .regioninfo files. Each table's integrity is then verified. If there
* are any orphan regions (regions with no .regioninfo files), or holes, new
* regions are fabricated. Backwards regions are sidelined as well as empty
* degenerate (endkey==startkey) regions. If there are any overlapping
regions,
* a new region is created and all data is merged into the new region.
*
* Table integrity repairs deal solely with HDFS and can be done offline --
the
* hbase region servers or master do not need to be running. These phase can
be
* use to completely reconstruct the META table in an offline fashion.
*
* Region consistency requires three conditions -- 1) valid .regioninfo file
* present in an hdfs region dir, 2) valid row with .regioninfo data in META,
* and 3) a region is deployed only at the regionserver that is was assigned
to.
*
* Region consistency requires hbck to contact the HBase master and region
* servers, so the connect() must first be called successfully. Much of the
* region consistency information is transient and less risky to repair.
*/
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5563) HRegionInfo#compareTo should compare regionId as well
[ https://issues.apache.org/jira/browse/HBASE-5563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5563: -- Attachment: hbase-5563-0.90.patch The 0.90 version of this patch is required for 0.90 HBASE-5128's unit tests to pass consistently. HRegionInfo#compareTo should compare regionId as well - Key: HBASE-5563 URL: https://issues.apache.org/jira/browse/HBASE-5563 Project: HBase Issue Type: Bug Affects Versions: 0.90.5, 0.92.0, 0.94.0, 0.96.0 Reporter: chunhui shen Assignee: chunhui shen Fix For: 0.92.2, 0.94.0, 0.96.0 Attachments: HBASE-5563.patch, HBASE-5563v2.patch, HBASE-5563v2.patch, hbase-5563-0.90.patch, hbase-5563-v3-0.92.patch, hbase-5563-v3.patch In the one region multi assigned case, we could find that two regions have the same table name, same startKey, same endKey, and different regionId, so these two regions are same in TreeMap but different in HashMap. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5128) [uber hbck] Enable hbck to automatically repair table integrity problems as well as region consistency problems while online.
[
https://issues.apache.org/jira/browse/HBASE-5128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hsieh updated HBASE-5128:
--
Attachment: hbase-5128-0.90-v4.patch
Cleaned up 0.90 version. Requires HBASE-5563 to pass consistently.
[uber hbck] Enable hbck to automatically repair table integrity problems as
well as region consistency problems while online.
-
Key: HBASE-5128
URL: https://issues.apache.org/jira/browse/HBASE-5128
Project: HBase
Issue Type: New Feature
Components: hbck
Affects Versions: 0.90.5, 0.92.0, 0.94.0, 0.96.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0
Attachments: hbase-5128-0.90-v2.patch, hbase-5128-0.90-v2b.patch,
hbase-5128-0.90-v4.patch, hbase-5128-0.92-v2.patch, hbase-5128-0.92-v4.patch,
hbase-5128-0.94-v2.patch, hbase-5128-0.94-v4.patch,
hbase-5128-trunk-v2.patch, hbase-5128-trunk.patch, hbase-5128-v3.patch,
hbase-5128-v4.patch
The current (0.90.5, 0.92.0rc2) versions of hbck detects most of region
consistency and table integrity invariant violations. However with '-fix' it
can only automatically repair region consistency cases having to do with
deployment problems. This updated version should be able to handle all cases
(including a new orphan regiondir case). When complete will likely deprecate
the OfflineMetaRepair tool and subsume several open META-hole related issue.
Here's the approach (from the comment of at the top of the new version of the
file).
{code}
/**
* HBaseFsck (hbck) is a tool for checking and repairing region consistency
and
* table integrity.
*
* Region consistency checks verify that META, region deployment on
* region servers and the state of data in HDFS (.regioninfo files) all are in
* accordance.
*
* Table integrity checks verify that that all possible row keys can resolve
to
* exactly one region of a table. This means there are no individual
degenerate
* or backwards regions; no holes between regions; and that there no
overlapping
* regions.
*
* The general repair strategy works in these steps.
* 1) Repair Table Integrity on HDFS. (merge or fabricate regions)
* 2) Repair Region Consistency with META and assignments
*
* For table integrity repairs, the tables their region directories are
scanned
* for .regioninfo files. Each table's integrity is then verified. If there
* are any orphan regions (regions with no .regioninfo files), or holes, new
* regions are fabricated. Backwards regions are sidelined as well as empty
* degenerate (endkey==startkey) regions. If there are any overlapping
regions,
* a new region is created and all data is merged into the new region.
*
* Table integrity repairs deal solely with HDFS and can be done offline --
the
* hbase region servers or master do not need to be running. These phase can
be
* use to completely reconstruct the META table in an offline fashion.
*
* Region consistency requires three conditions -- 1) valid .regioninfo file
* present in an hdfs region dir, 2) valid row with .regioninfo data in META,
* and 3) a region is deployed only at the regionserver that is was assigned
to.
*
* Region consistency requires hbck to contact the HBase master and region
* servers, so the connect() must first be called successfully. Much of the
* region consistency information is transient and less risky to repair.
*/
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5128) [uber hbck] Online automated repair of table integrity and region consistency problems
[
https://issues.apache.org/jira/browse/HBASE-5128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hsieh updated HBASE-5128:
--
Summary: [uber hbck] Online automated repair of table integrity and region
consistency problems (was: [uber hbck] Enable hbck to automatically repair
table integrity problems as well as region consistency problems while online.)
Issue rename to be more concise.
[uber hbck] Online automated repair of table integrity and region consistency
problems
--
Key: HBASE-5128
URL: https://issues.apache.org/jira/browse/HBASE-5128
Project: HBase
Issue Type: New Feature
Components: hbck
Affects Versions: 0.90.5, 0.92.0, 0.94.0, 0.96.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0
Attachments: hbase-5128-0.90-v2.patch, hbase-5128-0.90-v2b.patch,
hbase-5128-0.90-v4.patch, hbase-5128-0.92-v2.patch, hbase-5128-0.92-v4.patch,
hbase-5128-0.94-v2.patch, hbase-5128-0.94-v4.patch,
hbase-5128-trunk-v2.patch, hbase-5128-trunk.patch, hbase-5128-v3.patch,
hbase-5128-v4.patch
The current (0.90.5, 0.92.0rc2) versions of hbck detects most of region
consistency and table integrity invariant violations. However with '-fix' it
can only automatically repair region consistency cases having to do with
deployment problems. This updated version should be able to handle all cases
(including a new orphan regiondir case). When complete will likely deprecate
the OfflineMetaRepair tool and subsume several open META-hole related issue.
Here's the approach (from the comment of at the top of the new version of the
file).
{code}
/**
* HBaseFsck (hbck) is a tool for checking and repairing region consistency
and
* table integrity.
*
* Region consistency checks verify that META, region deployment on
* region servers and the state of data in HDFS (.regioninfo files) all are in
* accordance.
*
* Table integrity checks verify that that all possible row keys can resolve
to
* exactly one region of a table. This means there are no individual
degenerate
* or backwards regions; no holes between regions; and that there no
overlapping
* regions.
*
* The general repair strategy works in these steps.
* 1) Repair Table Integrity on HDFS. (merge or fabricate regions)
* 2) Repair Region Consistency with META and assignments
*
* For table integrity repairs, the tables their region directories are
scanned
* for .regioninfo files. Each table's integrity is then verified. If there
* are any orphan regions (regions with no .regioninfo files), or holes, new
* regions are fabricated. Backwards regions are sidelined as well as empty
* degenerate (endkey==startkey) regions. If there are any overlapping
regions,
* a new region is created and all data is merged into the new region.
*
* Table integrity repairs deal solely with HDFS and can be done offline --
the
* hbase region servers or master do not need to be running. These phase can
be
* use to completely reconstruct the META table in an offline fashion.
*
* Region consistency requires three conditions -- 1) valid .regioninfo file
* present in an hdfs region dir, 2) valid row with .regioninfo data in META,
* and 3) a region is deployed only at the regionserver that is was assigned
to.
*
* Region consistency requires hbck to contact the HBase master and region
* servers, so the connect() must first be called successfully. Much of the
* region consistency information is transient and less risky to repair.
*/
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5128) [uber hbck] Online automated repair of table integrity and region consistency problems
[
https://issues.apache.org/jira/browse/HBASE-5128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hsieh updated HBASE-5128:
--
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
[uber hbck] Online automated repair of table integrity and region consistency
problems
--
Key: HBASE-5128
URL: https://issues.apache.org/jira/browse/HBASE-5128
Project: HBase
Issue Type: New Feature
Components: hbck
Affects Versions: 0.90.5, 0.92.0, 0.94.0, 0.96.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0
Attachments: hbase-5128-0.90-v2.patch, hbase-5128-0.90-v2b.patch,
hbase-5128-0.90-v4.patch, hbase-5128-0.92-v2.patch, hbase-5128-0.92-v4.patch,
hbase-5128-0.94-v2.patch, hbase-5128-0.94-v4.patch,
hbase-5128-trunk-v2.patch, hbase-5128-trunk.patch, hbase-5128-v3.patch,
hbase-5128-v4.patch
The current (0.90.5, 0.92.0rc2) versions of hbck detects most of region
consistency and table integrity invariant violations. However with '-fix' it
can only automatically repair region consistency cases having to do with
deployment problems. This updated version should be able to handle all cases
(including a new orphan regiondir case). When complete will likely deprecate
the OfflineMetaRepair tool and subsume several open META-hole related issue.
Here's the approach (from the comment of at the top of the new version of the
file).
{code}
/**
* HBaseFsck (hbck) is a tool for checking and repairing region consistency
and
* table integrity.
*
* Region consistency checks verify that META, region deployment on
* region servers and the state of data in HDFS (.regioninfo files) all are in
* accordance.
*
* Table integrity checks verify that that all possible row keys can resolve
to
* exactly one region of a table. This means there are no individual
degenerate
* or backwards regions; no holes between regions; and that there no
overlapping
* regions.
*
* The general repair strategy works in these steps.
* 1) Repair Table Integrity on HDFS. (merge or fabricate regions)
* 2) Repair Region Consistency with META and assignments
*
* For table integrity repairs, the tables their region directories are
scanned
* for .regioninfo files. Each table's integrity is then verified. If there
* are any orphan regions (regions with no .regioninfo files), or holes, new
* regions are fabricated. Backwards regions are sidelined as well as empty
* degenerate (endkey==startkey) regions. If there are any overlapping
regions,
* a new region is created and all data is merged into the new region.
*
* Table integrity repairs deal solely with HDFS and can be done offline --
the
* hbase region servers or master do not need to be running. These phase can
be
* use to completely reconstruct the META table in an offline fashion.
*
* Region consistency requires three conditions -- 1) valid .regioninfo file
* present in an hdfs region dir, 2) valid row with .regioninfo data in META,
* and 3) a region is deployed only at the regionserver that is was assigned
to.
*
* Region consistency requires hbck to contact the HBase master and region
* servers, so the connect() must first be called successfully. Much of the
* region consistency information is transient and less risky to repair.
*/
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5628) Improve performance of uberhbck
[ https://issues.apache.org/jira/browse/HBASE-5628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5628: -- Description: During reviews of HBASE-5128 there are several opportunities investigate for improving the performance of the tool. - Change regionInfoMap and tablesInfo from TreeMap to HashMap. - Change some full region set reloads to be incremental to require fewer passes. - Cache meta for subsequent calls of closeRegionSileneglyAndWait was: During reviews of HBASE-5128 there are several opportunities investigate for improving the performance of the tool. - Change regionInfoMap and tablesInfo from TreeMap to HashMap. - Change some full region set reloads to be incremental to require fewer passes. Improve performance of uberhbck --- Key: HBASE-5628 URL: https://issues.apache.org/jira/browse/HBASE-5628 Project: HBase Issue Type: Improvement Components: hbck Affects Versions: 0.90.7, 0.92.2, 0.94.0, 0.96.0 Reporter: Jonathan Hsieh During reviews of HBASE-5128 there are several opportunities investigate for improving the performance of the tool. - Change regionInfoMap and tablesInfo from TreeMap to HashMap. - Change some full region set reloads to be incremental to require fewer passes. - Cache meta for subsequent calls of closeRegionSileneglyAndWait -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5632) hbck should deal with partial failures better.
[ https://issues.apache.org/jira/browse/HBASE-5632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5632: -- Affects Version/s: 0.96.0 0.94.0 0.92.2 0.90.7 hbck should deal with partial failures better. -- Key: HBASE-5632 URL: https://issues.apache.org/jira/browse/HBASE-5632 Project: HBase Issue Type: Improvement Components: hbck Affects Versions: 0.90.7, 0.92.2, 0.94.0, 0.96.0 Reporter: Jonathan Hsieh The first cut of HBASE-5128 failed fast and tried to avoid cases handling partial recovery failures. We can improve this. - collecting faults and recovering from them - Handling RejectedExectionException exceptions when using executor.execute. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5575) Configure Arcanist lint engine for HBase
[ https://issues.apache.org/jira/browse/HBASE-5575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5575: -- Component/s: build Fix Version/s: 0.89-fb 0.96.0 Configure Arcanist lint engine for HBase Key: HBASE-5575 URL: https://issues.apache.org/jira/browse/HBASE-5575 Project: HBase Issue Type: Improvement Components: build Reporter: Mikhail Bautin Assignee: Mikhail Bautin Fix For: 0.96.0, 0.89-fb Attachments: Enabling-lint-2012-03-16_13_40_37.patch We need to enable Arcanist lint engine in HBase, so that a commit could be checked by running arc lint. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5128) [uber hbck] Enable hbck to automatically repair table integrity problems as well as region consistency problems while online.
[
https://issues.apache.org/jira/browse/HBASE-5128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hsieh updated HBASE-5128:
--
Attachment: hbase-5128-v3.patch
[uber hbck] Enable hbck to automatically repair table integrity problems as
well as region consistency problems while online.
-
Key: HBASE-5128
URL: https://issues.apache.org/jira/browse/HBASE-5128
Project: HBase
Issue Type: New Feature
Components: hbck
Affects Versions: 0.90.5, 0.92.0, 0.94.0, 0.96.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0
Attachments: hbase-5128-0.90-v2.patch, hbase-5128-0.90-v2b.patch,
hbase-5128-0.92-v2.patch, hbase-5128-0.94-v2.patch,
hbase-5128-trunk-v2.patch, hbase-5128-trunk.patch, hbase-5128-v3.patch
The current (0.90.5, 0.92.0rc2) versions of hbck detects most of region
consistency and table integrity invariant violations. However with '-fix' it
can only automatically repair region consistency cases having to do with
deployment problems. This updated version should be able to handle all cases
(including a new orphan regiondir case). When complete will likely deprecate
the OfflineMetaRepair tool and subsume several open META-hole related issue.
Here's the approach (from the comment of at the top of the new version of the
file).
{code}
/**
* HBaseFsck (hbck) is a tool for checking and repairing region consistency
and
* table integrity.
*
* Region consistency checks verify that META, region deployment on
* region servers and the state of data in HDFS (.regioninfo files) all are in
* accordance.
*
* Table integrity checks verify that that all possible row keys can resolve
to
* exactly one region of a table. This means there are no individual
degenerate
* or backwards regions; no holes between regions; and that there no
overlapping
* regions.
*
* The general repair strategy works in these steps.
* 1) Repair Table Integrity on HDFS. (merge or fabricate regions)
* 2) Repair Region Consistency with META and assignments
*
* For table integrity repairs, the tables their region directories are
scanned
* for .regioninfo files. Each table's integrity is then verified. If there
* are any orphan regions (regions with no .regioninfo files), or holes, new
* regions are fabricated. Backwards regions are sidelined as well as empty
* degenerate (endkey==startkey) regions. If there are any overlapping
regions,
* a new region is created and all data is merged into the new region.
*
* Table integrity repairs deal solely with HDFS and can be done offline --
the
* hbase region servers or master do not need to be running. These phase can
be
* use to completely reconstruct the META table in an offline fashion.
*
* Region consistency requires three conditions -- 1) valid .regioninfo file
* present in an hdfs region dir, 2) valid row with .regioninfo data in META,
* and 3) a region is deployed only at the regionserver that is was assigned
to.
*
* Region consistency requires hbck to contact the HBase master and region
* servers, so the connect() must first be called successfully. Much of the
* region consistency information is transient and less risky to repair.
*/
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5589) Add of the offline call to the Master Interface
[ https://issues.apache.org/jira/browse/HBASE-5589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5589: -- Attachment: hbase-5589.patch hbase-5589-0.92.patch hbase-5589-0.90.patch hbase-5589.patch bump HMasterInterface RPC version and apply to 0.94 and trunk The 0.90 and 0.92 versions do not bump rpc to maintain compatibility but notes in comments that has been done. Add of the offline call to the Master Interface --- Key: HBASE-5589 URL: https://issues.apache.org/jira/browse/HBASE-5589 Project: HBase Issue Type: Improvement Components: hbck Affects Versions: 0.90.6, 0.92.0, 0.94.0, 0.96.0 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Attachments: hbase-5589-0.90.patch, hbase-5589-0.92.patch, hbase-5589.patch Hbck from HBASE-5128 requires an offline method on the master to properly cleanup state during certain assignment repair operations. This will this method will be added to recent and older versions of HBase. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5589) Add of the offline call to the Master Interface
[ https://issues.apache.org/jira/browse/HBASE-5589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5589: -- Status: Patch Available (was: Open) Add of the offline call to the Master Interface --- Key: HBASE-5589 URL: https://issues.apache.org/jira/browse/HBASE-5589 Project: HBase Issue Type: Improvement Components: hbck Affects Versions: 0.92.0, 0.90.6, 0.94.0, 0.96.0 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Attachments: hbase-5589-0.90.patch, hbase-5589-0.92.patch, hbase-5589.patch Hbck from HBASE-5128 requires an offline method on the master to properly cleanup state during certain assignment repair operations. This will this method will be added to recent and older versions of HBase. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5589) Add of the offline call to the Master Interface
[ https://issues.apache.org/jira/browse/HBASE-5589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5589: -- Attachment: hbase-5588-0.94.patch @Stack. Ok. Attached 0.94 version without the bump and with a slightly updated comment. Add of the offline call to the Master Interface --- Key: HBASE-5589 URL: https://issues.apache.org/jira/browse/HBASE-5589 Project: HBase Issue Type: Improvement Components: hbck Affects Versions: 0.90.6, 0.92.0, 0.94.0, 0.96.0 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Attachments: hbase-5588-0.94.patch, hbase-5589-0.90.patch, hbase-5589-0.92.patch, hbase-5589.patch Hbck from HBASE-5128 requires an offline method on the master to properly cleanup state during certain assignment repair operations. This will this method will be added to recent and older versions of HBase. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5589) Add of the offline call to the Master Interface
[ https://issues.apache.org/jira/browse/HBASE-5589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5589: -- Resolution: Fixed Fix Version/s: 0.96.0 0.94.0 0.92.2 0.90.7 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Add of the offline call to the Master Interface --- Key: HBASE-5589 URL: https://issues.apache.org/jira/browse/HBASE-5589 Project: HBase Issue Type: Improvement Components: hbck Affects Versions: 0.90.6, 0.92.0, 0.94.0, 0.96.0 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0 Attachments: hbase-5588-0.94.patch, hbase-5589-0.90.patch, hbase-5589-0.92.patch, hbase-5589.patch Hbck from HBASE-5128 requires an offline method on the master to properly cleanup state during certain assignment repair operations. This will this method will be added to recent and older versions of HBase. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5128) [uber hbck] Enable hbck to automatically repair table integrity problems as well as region consistency problems while online.
[
https://issues.apache.org/jira/browse/HBASE-5128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hsieh updated HBASE-5128:
--
Attachment: hbase-5128-trunk-v2.patch
[uber hbck] Enable hbck to automatically repair table integrity problems as
well as region consistency problems while online.
-
Key: HBASE-5128
URL: https://issues.apache.org/jira/browse/HBASE-5128
Project: HBase
Issue Type: New Feature
Components: hbck
Affects Versions: 0.90.5, 0.92.0, 0.94.0, 0.96.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Attachments: hbase-5128-trunk-v2.patch, hbase-5128-trunk.patch
The current (0.90.5, 0.92.0rc2) versions of hbck detects most of region
consistency and table integrity invariant violations. However with '-fix' it
can only automatically repair region consistency cases having to do with
deployment problems. This updated version should be able to handle all cases
(including a new orphan regiondir case). When complete will likely deprecate
the OfflineMetaRepair tool and subsume several open META-hole related issue.
Here's the approach (from the comment of at the top of the new version of the
file).
{code}
/**
* HBaseFsck (hbck) is a tool for checking and repairing region consistency
and
* table integrity.
*
* Region consistency checks verify that META, region deployment on
* region servers and the state of data in HDFS (.regioninfo files) all are in
* accordance.
*
* Table integrity checks verify that that all possible row keys can resolve
to
* exactly one region of a table. This means there are no individual
degenerate
* or backwards regions; no holes between regions; and that there no
overlapping
* regions.
*
* The general repair strategy works in these steps.
* 1) Repair Table Integrity on HDFS. (merge or fabricate regions)
* 2) Repair Region Consistency with META and assignments
*
* For table integrity repairs, the tables their region directories are
scanned
* for .regioninfo files. Each table's integrity is then verified. If there
* are any orphan regions (regions with no .regioninfo files), or holes, new
* regions are fabricated. Backwards regions are sidelined as well as empty
* degenerate (endkey==startkey) regions. If there are any overlapping
regions,
* a new region is created and all data is merged into the new region.
*
* Table integrity repairs deal solely with HDFS and can be done offline --
the
* hbase region servers or master do not need to be running. These phase can
be
* use to completely reconstruct the META table in an offline fashion.
*
* Region consistency requires three conditions -- 1) valid .regioninfo file
* present in an hdfs region dir, 2) valid row with .regioninfo data in META,
* and 3) a region is deployed only at the regionserver that is was assigned
to.
*
* Region consistency requires hbck to contact the HBase master and region
* servers, so the connect() must first be called successfully. Much of the
* region consistency information is transient and less risky to repair.
*/
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5128) [uber hbck] Enable hbck to automatically repair table integrity problems as well as region consistency problems while online.
[
https://issues.apache.org/jira/browse/HBASE-5128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hsieh updated HBASE-5128:
--
Attachment: hbase-5128-0.92-v2.patch
hbase-5128-0.94-v2.patch
0.94 and 0.92 versions have minor tweaks from trunk version and in both cases
TestHBaseFsck passes.
[uber hbck] Enable hbck to automatically repair table integrity problems as
well as region consistency problems while online.
-
Key: HBASE-5128
URL: https://issues.apache.org/jira/browse/HBASE-5128
Project: HBase
Issue Type: New Feature
Components: hbck
Affects Versions: 0.90.5, 0.92.0, 0.94.0, 0.96.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Attachments: hbase-5128-0.92-v2.patch, hbase-5128-0.94-v2.patch,
hbase-5128-trunk-v2.patch, hbase-5128-trunk.patch
The current (0.90.5, 0.92.0rc2) versions of hbck detects most of region
consistency and table integrity invariant violations. However with '-fix' it
can only automatically repair region consistency cases having to do with
deployment problems. This updated version should be able to handle all cases
(including a new orphan regiondir case). When complete will likely deprecate
the OfflineMetaRepair tool and subsume several open META-hole related issue.
Here's the approach (from the comment of at the top of the new version of the
file).
{code}
/**
* HBaseFsck (hbck) is a tool for checking and repairing region consistency
and
* table integrity.
*
* Region consistency checks verify that META, region deployment on
* region servers and the state of data in HDFS (.regioninfo files) all are in
* accordance.
*
* Table integrity checks verify that that all possible row keys can resolve
to
* exactly one region of a table. This means there are no individual
degenerate
* or backwards regions; no holes between regions; and that there no
overlapping
* regions.
*
* The general repair strategy works in these steps.
* 1) Repair Table Integrity on HDFS. (merge or fabricate regions)
* 2) Repair Region Consistency with META and assignments
*
* For table integrity repairs, the tables their region directories are
scanned
* for .regioninfo files. Each table's integrity is then verified. If there
* are any orphan regions (regions with no .regioninfo files), or holes, new
* regions are fabricated. Backwards regions are sidelined as well as empty
* degenerate (endkey==startkey) regions. If there are any overlapping
regions,
* a new region is created and all data is merged into the new region.
*
* Table integrity repairs deal solely with HDFS and can be done offline --
the
* hbase region servers or master do not need to be running. These phase can
be
* use to completely reconstruct the META table in an offline fashion.
*
* Region consistency requires three conditions -- 1) valid .regioninfo file
* present in an hdfs region dir, 2) valid row with .regioninfo data in META,
* and 3) a region is deployed only at the regionserver that is was assigned
to.
*
* Region consistency requires hbck to contact the HBase master and region
* servers, so the connect() must first be called successfully. Much of the
* region consistency information is transient and less risky to repair.
*/
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5128) [uber hbck] Enable hbck to automatically repair table integrity problems as well as region consistency problems while online.
[
https://issues.apache.org/jira/browse/HBASE-5128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hsieh updated HBASE-5128:
--
Attachment: hbase-5128-0.90-v2.patch
For the 0.90v2 version, TestHBaseFsck passes consistently. This is very close
to the version we've used to repair production clusters.
0.90 version has a different HBaseFsckRepair.forceOfflineInZK() which is
somehow responsible for that version not needed HBASE-5563. I haven't
investigated enough to determine why the equivalent method for the
0.92/0.94/trunk versions fail unit tests consistently.
[uber hbck] Enable hbck to automatically repair table integrity problems as
well as region consistency problems while online.
-
Key: HBASE-5128
URL: https://issues.apache.org/jira/browse/HBASE-5128
Project: HBase
Issue Type: New Feature
Components: hbck
Affects Versions: 0.90.5, 0.92.0, 0.94.0, 0.96.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Attachments: hbase-5128-0.90-v2.patch, hbase-5128-0.92-v2.patch,
hbase-5128-0.94-v2.patch, hbase-5128-trunk-v2.patch, hbase-5128-trunk.patch
The current (0.90.5, 0.92.0rc2) versions of hbck detects most of region
consistency and table integrity invariant violations. However with '-fix' it
can only automatically repair region consistency cases having to do with
deployment problems. This updated version should be able to handle all cases
(including a new orphan regiondir case). When complete will likely deprecate
the OfflineMetaRepair tool and subsume several open META-hole related issue.
Here's the approach (from the comment of at the top of the new version of the
file).
{code}
/**
* HBaseFsck (hbck) is a tool for checking and repairing region consistency
and
* table integrity.
*
* Region consistency checks verify that META, region deployment on
* region servers and the state of data in HDFS (.regioninfo files) all are in
* accordance.
*
* Table integrity checks verify that that all possible row keys can resolve
to
* exactly one region of a table. This means there are no individual
degenerate
* or backwards regions; no holes between regions; and that there no
overlapping
* regions.
*
* The general repair strategy works in these steps.
* 1) Repair Table Integrity on HDFS. (merge or fabricate regions)
* 2) Repair Region Consistency with META and assignments
*
* For table integrity repairs, the tables their region directories are
scanned
* for .regioninfo files. Each table's integrity is then verified. If there
* are any orphan regions (regions with no .regioninfo files), or holes, new
* regions are fabricated. Backwards regions are sidelined as well as empty
* degenerate (endkey==startkey) regions. If there are any overlapping
regions,
* a new region is created and all data is merged into the new region.
*
* Table integrity repairs deal solely with HDFS and can be done offline --
the
* hbase region servers or master do not need to be running. These phase can
be
* use to completely reconstruct the META table in an offline fashion.
*
* Region consistency requires three conditions -- 1) valid .regioninfo file
* present in an hdfs region dir, 2) valid row with .regioninfo data in META,
* and 3) a region is deployed only at the regionserver that is was assigned
to.
*
* Region consistency requires hbck to contact the HBase master and region
* servers, so the connect() must first be called successfully. Much of the
* region consistency information is transient and less risky to repair.
*/
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5128) [uber hbck] Enable hbck to automatically repair table integrity problems as well as region consistency problems while online.
[
https://issues.apache.org/jira/browse/HBASE-5128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hsieh updated HBASE-5128:
--
Fix Version/s: 0.96.0
0.94.0
0.92.2
0.90.7
[uber hbck] Enable hbck to automatically repair table integrity problems as
well as region consistency problems while online.
-
Key: HBASE-5128
URL: https://issues.apache.org/jira/browse/HBASE-5128
Project: HBase
Issue Type: New Feature
Components: hbck
Affects Versions: 0.90.5, 0.92.0, 0.94.0, 0.96.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0
Attachments: hbase-5128-0.90-v2.patch, hbase-5128-0.92-v2.patch,
hbase-5128-0.94-v2.patch, hbase-5128-trunk-v2.patch, hbase-5128-trunk.patch
The current (0.90.5, 0.92.0rc2) versions of hbck detects most of region
consistency and table integrity invariant violations. However with '-fix' it
can only automatically repair region consistency cases having to do with
deployment problems. This updated version should be able to handle all cases
(including a new orphan regiondir case). When complete will likely deprecate
the OfflineMetaRepair tool and subsume several open META-hole related issue.
Here's the approach (from the comment of at the top of the new version of the
file).
{code}
/**
* HBaseFsck (hbck) is a tool for checking and repairing region consistency
and
* table integrity.
*
* Region consistency checks verify that META, region deployment on
* region servers and the state of data in HDFS (.regioninfo files) all are in
* accordance.
*
* Table integrity checks verify that that all possible row keys can resolve
to
* exactly one region of a table. This means there are no individual
degenerate
* or backwards regions; no holes between regions; and that there no
overlapping
* regions.
*
* The general repair strategy works in these steps.
* 1) Repair Table Integrity on HDFS. (merge or fabricate regions)
* 2) Repair Region Consistency with META and assignments
*
* For table integrity repairs, the tables their region directories are
scanned
* for .regioninfo files. Each table's integrity is then verified. If there
* are any orphan regions (regions with no .regioninfo files), or holes, new
* regions are fabricated. Backwards regions are sidelined as well as empty
* degenerate (endkey==startkey) regions. If there are any overlapping
regions,
* a new region is created and all data is merged into the new region.
*
* Table integrity repairs deal solely with HDFS and can be done offline --
the
* hbase region servers or master do not need to be running. These phase can
be
* use to completely reconstruct the META table in an offline fashion.
*
* Region consistency requires three conditions -- 1) valid .regioninfo file
* present in an hdfs region dir, 2) valid row with .regioninfo data in META,
* and 3) a region is deployed only at the regionserver that is was assigned
to.
*
* Region consistency requires hbck to contact the HBase master and region
* servers, so the connect() must first be called successfully. Much of the
* region consistency information is transient and less risky to repair.
*/
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5128) [uber hbck] Enable hbck to automatically repair table integrity problems as well as region consistency problems while online.
[
https://issues.apache.org/jira/browse/HBASE-5128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hsieh updated HBASE-5128:
--
Attachment: hbase-5128-0.90-v2b.patch
Previous version accidentally included two dev tools that are not part of this
patch (ScrubMeta and DumpMeta).
[uber hbck] Enable hbck to automatically repair table integrity problems as
well as region consistency problems while online.
-
Key: HBASE-5128
URL: https://issues.apache.org/jira/browse/HBASE-5128
Project: HBase
Issue Type: New Feature
Components: hbck
Affects Versions: 0.90.5, 0.92.0, 0.94.0, 0.96.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0
Attachments: hbase-5128-0.90-v2.patch, hbase-5128-0.90-v2b.patch,
hbase-5128-0.92-v2.patch, hbase-5128-0.94-v2.patch,
hbase-5128-trunk-v2.patch, hbase-5128-trunk.patch
The current (0.90.5, 0.92.0rc2) versions of hbck detects most of region
consistency and table integrity invariant violations. However with '-fix' it
can only automatically repair region consistency cases having to do with
deployment problems. This updated version should be able to handle all cases
(including a new orphan regiondir case). When complete will likely deprecate
the OfflineMetaRepair tool and subsume several open META-hole related issue.
Here's the approach (from the comment of at the top of the new version of the
file).
{code}
/**
* HBaseFsck (hbck) is a tool for checking and repairing region consistency
and
* table integrity.
*
* Region consistency checks verify that META, region deployment on
* region servers and the state of data in HDFS (.regioninfo files) all are in
* accordance.
*
* Table integrity checks verify that that all possible row keys can resolve
to
* exactly one region of a table. This means there are no individual
degenerate
* or backwards regions; no holes between regions; and that there no
overlapping
* regions.
*
* The general repair strategy works in these steps.
* 1) Repair Table Integrity on HDFS. (merge or fabricate regions)
* 2) Repair Region Consistency with META and assignments
*
* For table integrity repairs, the tables their region directories are
scanned
* for .regioninfo files. Each table's integrity is then verified. If there
* are any orphan regions (regions with no .regioninfo files), or holes, new
* regions are fabricated. Backwards regions are sidelined as well as empty
* degenerate (endkey==startkey) regions. If there are any overlapping
regions,
* a new region is created and all data is merged into the new region.
*
* Table integrity repairs deal solely with HDFS and can be done offline --
the
* hbase region servers or master do not need to be running. These phase can
be
* use to completely reconstruct the META table in an offline fashion.
*
* Region consistency requires three conditions -- 1) valid .regioninfo file
* present in an hdfs region dir, 2) valid row with .regioninfo data in META,
* and 3) a region is deployed only at the regionserver that is was assigned
to.
*
* Region consistency requires hbck to contact the HBase master and region
* servers, so the connect() must first be called successfully. Much of the
* region consistency information is transient and less risky to repair.
*/
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5588) Deprecate/remove AssignmentManager#clearRegionFromTransition
[ https://issues.apache.org/jira/browse/HBASE-5588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5588: -- Resolution: Fixed Fix Version/s: 0.96.0 0.94.0 0.92.2 0.90.7 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Deprecate/remove AssignmentManager#clearRegionFromTransition Key: HBASE-5588 URL: https://issues.apache.org/jira/browse/HBASE-5588 Project: HBase Issue Type: Sub-task Components: hbck Affects Versions: 0.90.5, 0.92.0, 0.94.0, 0.96.0 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0 Attachments: hbase-5588-0.90.patch, hbase-5588-0.94.patch, hbase-5588.patch This method is essentially a dupe of Assignment#regionOffline. As suggested in early review of HBASE-5128 - deprecate up to 0.94 and remove from 0.96/trunk. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5588) Deprecate/remove AssignmentManager#clearRegionFromTransition
[ https://issues.apache.org/jira/browse/HBASE-5588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5588: -- Issue Type: Improvement (was: Sub-task) Parent: (was: HBASE-5128) Deprecate/remove AssignmentManager#clearRegionFromTransition Key: HBASE-5588 URL: https://issues.apache.org/jira/browse/HBASE-5588 Project: HBase Issue Type: Improvement Components: hbck Affects Versions: 0.90.5, 0.92.0, 0.94.0, 0.96.0 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0 Attachments: hbase-5588-0.90.patch, hbase-5588-0.94.patch, hbase-5588.patch This method is essentially a dupe of Assignment#regionOffline. As suggested in early review of HBASE-5128 - deprecate up to 0.94 and remove from 0.96/trunk. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5589) Add of the offline call to the Master Interface
[ https://issues.apache.org/jira/browse/HBASE-5589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5589: -- Issue Type: Improvement (was: Sub-task) Parent: (was: HBASE-5128) Add of the offline call to the Master Interface --- Key: HBASE-5589 URL: https://issues.apache.org/jira/browse/HBASE-5589 Project: HBase Issue Type: Improvement Components: hbck Affects Versions: 0.90.6, 0.92.0, 0.94.0, 0.96.0 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Hbck from HBASE-5128 requires an offline method on the master to properly cleanup state during certain assignment repair operations. This will this method will be added to recent and older versions of HBase. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5588) Deprecate/remove AssignmentManager#clearRegionFromTransition
[ https://issues.apache.org/jira/browse/HBASE-5588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5588: -- Status: Patch Available (was: Open) Suite against the four versions is running. Deprecate/remove AssignmentManager#clearRegionFromTransition Key: HBASE-5588 URL: https://issues.apache.org/jira/browse/HBASE-5588 Project: HBase Issue Type: Sub-task Components: hbck Affects Versions: 0.92.0, 0.90.5, 0.94.0, 0.96.0 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Attachments: hbase-5588-0.90.patch, hbase-5588-0.94.patch, hbase-5588.patch This method is essentially a dupe of Assignment#regionOffline. As suggested in early review of HBASE-5128 - deprecate up to 0.94 and remove from 0.96/trunk. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5588) Deprecate/remove AssignmentManager#clearRegionFromTransition
[ https://issues.apache.org/jira/browse/HBASE-5588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5588: -- Attachment: hbase-5588-0.90.patch hbase-5588-0.94.patch hbase-5588.patch hbase-5588.patch removes clearRegionFromTransition hbase-5588-0.94.patch deprecates and is also applicable to 0.92 hbase-5588-0.90.patch deprecates. Deprecate/remove AssignmentManager#clearRegionFromTransition Key: HBASE-5588 URL: https://issues.apache.org/jira/browse/HBASE-5588 Project: HBase Issue Type: Sub-task Components: hbck Affects Versions: 0.90.5, 0.92.0, 0.94.0, 0.96.0 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Attachments: hbase-5588-0.90.patch, hbase-5588-0.94.patch, hbase-5588.patch This method is essentially a dupe of Assignment#regionOffline. As suggested in early review of HBASE-5128 - deprecate up to 0.94 and remove from 0.96/trunk. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5563) HRegionInfo#compareTo add the comparison of regionId
[ https://issues.apache.org/jira/browse/HBASE-5563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5563: -- Resolution: Fixed Fix Version/s: 0.96.0 0.94.0 0.92.2 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) HRegionInfo#compareTo add the comparison of regionId Key: HBASE-5563 URL: https://issues.apache.org/jira/browse/HBASE-5563 Project: HBase Issue Type: Bug Affects Versions: 0.90.5, 0.92.0, 0.94.0, 0.96.0 Reporter: chunhui shen Assignee: chunhui shen Fix For: 0.92.2, 0.94.0, 0.96.0 Attachments: HBASE-5563.patch, HBASE-5563v2.patch, HBASE-5563v2.patch, hbase-5563-v3-0.92.patch, hbase-5563-v3.patch In the one region multi assigned case, we could find that two regions have the same table name, same startKey, same endKey, and different regionId, so these two regions are same in TreeMap but different in HashMap. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5358) HBaseObjectWritable should be able to serialize/deserialize generic arrays
[ https://issues.apache.org/jira/browse/HBASE-5358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5358: -- Fix Version/s: 0.92.1 HBaseObjectWritable should be able to serialize/deserialize generic arrays -- Key: HBASE-5358 URL: https://issues.apache.org/jira/browse/HBASE-5358 Project: HBase Issue Type: Improvement Components: coprocessors, io Affects Versions: 0.94.0 Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: 0.92.1, 0.94.0 Attachments: 5358-92.txt, HBASE-5358_v3.patch HBaseObjectWritable can encode Writable[]'s but, but cannot encode A[] where A extends Writable. This becomes an issue for example when adding a coprocessor method which takes A[] (see HBASE-5352). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5563) HRegionInfo#compareTo add the comparison of regionId
[ https://issues.apache.org/jira/browse/HBASE-5563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5563: -- Status: Patch Available (was: Open) Updated Patch to add test case and to fix broken test cases. HRegionInfo#compareTo add the comparison of regionId Key: HBASE-5563 URL: https://issues.apache.org/jira/browse/HBASE-5563 Project: HBase Issue Type: Bug Reporter: chunhui shen Assignee: chunhui shen Attachments: HBASE-5563.patch, HBASE-5563v2.patch, HBASE-5563v2.patch, hbase-5563-v3.patch In the one region multi assigned case, we could find that two regions have the same table name, same startKey, same endKey, and different regionId, so these two regions are same in TreeMap but different in HashMap. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5563) HRegionInfo#compareTo add the comparison of regionId
[ https://issues.apache.org/jira/browse/HBASE-5563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5563: -- Attachment: hbase-5563-v3.patch HRegionInfo#compareTo add the comparison of regionId Key: HBASE-5563 URL: https://issues.apache.org/jira/browse/HBASE-5563 Project: HBase Issue Type: Bug Reporter: chunhui shen Assignee: chunhui shen Attachments: HBASE-5563.patch, HBASE-5563v2.patch, HBASE-5563v2.patch, hbase-5563-v3.patch In the one region multi assigned case, we could find that two regions have the same table name, same startKey, same endKey, and different regionId, so these two regions are same in TreeMap but different in HashMap. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
