[jira] [Commented] (HBASE-5583) Master restart on create table with splitkeys does not recreate table with all the splitkey regions
[
https://issues.apache.org/jira/browse/HBASE-5583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14126664#comment-14126664
]
Hadoop QA commented on HBASE-5583:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12576723/HBASE-5583_new_1_review.patch
against trunk revision .
ATTACHMENT ID: 12576723
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 6 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10778//console
This message is automatically generated.
Master restart on create table with splitkeys does not recreate table with
all the splitkey regions
---
Key: HBASE-5583
URL: https://issues.apache.org/jira/browse/HBASE-5583
Project: HBase
Issue Type: Bug
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Fix For: 0.99.1
Attachments: HBASE-5583_new_1.patch, HBASE-5583_new_1_review.patch,
HBASE-5583_new_2.patch, HBASE-5583_new_4_WIP.patch,
HBASE-5583_new_5_WIP_using_tableznode.patch
- Create table using splitkeys
- MAster goes down before all regions are added to meta
- On master restart the table is again enabled but with less number of
regions than specified in splitkeys
Anyway client will get an exception if i had called sync create table. But
table exists or not check will say table exists.
Is this scenario to be handled by client only or can we have some mechanism
on the master side for this? Pls suggest.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-5583) Master restart on create table with splitkeys does not recreate table with all the splitkey regions
[
https://issues.apache.org/jira/browse/HBASE-5583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13849698#comment-13849698
]
Hadoop QA commented on HBASE-5583:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12576723/HBASE-5583_new_1_review.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 6 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/8186//console
This message is automatically generated.
Master restart on create table with splitkeys does not recreate table with
all the splitkey regions
---
Key: HBASE-5583
URL: https://issues.apache.org/jira/browse/HBASE-5583
Project: HBase
Issue Type: Bug
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Fix For: 0.99.0
Attachments: HBASE-5583_new_1.patch, HBASE-5583_new_1_review.patch,
HBASE-5583_new_2.patch, HBASE-5583_new_4_WIP.patch,
HBASE-5583_new_5_WIP_using_tableznode.patch
- Create table using splitkeys
- MAster goes down before all regions are added to meta
- On master restart the table is again enabled but with less number of
regions than specified in splitkeys
Anyway client will get an exception if i had called sync create table. But
table exists or not check will say table exists.
Is this scenario to be handled by client only or can we have some mechanism
on the master side for this? Pls suggest.
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Commented] (HBASE-5583) Master restart on create table with splitkeys does not recreate table with all the splitkey regions
[
https://issues.apache.org/jira/browse/HBASE-5583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13815420#comment-13815420
]
Hadoop QA commented on HBASE-5583:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12576723/HBASE-5583_new_1_review.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 6 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/7763//console
This message is automatically generated.
Master restart on create table with splitkeys does not recreate table with
all the splitkey regions
---
Key: HBASE-5583
URL: https://issues.apache.org/jira/browse/HBASE-5583
Project: HBase
Issue Type: Bug
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Fix For: 0.96.1
Attachments: HBASE-5583_new_1.patch, HBASE-5583_new_1_review.patch,
HBASE-5583_new_2.patch, HBASE-5583_new_4_WIP.patch,
HBASE-5583_new_5_WIP_using_tableznode.patch
- Create table using splitkeys
- MAster goes down before all regions are added to meta
- On master restart the table is again enabled but with less number of
regions than specified in splitkeys
Anyway client will get an exception if i had called sync create table. But
table exists or not check will say table exists.
Is this scenario to be handled by client only or can we have some mechanism
on the master side for this? Pls suggest.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HBASE-5583) Master restart on create table with splitkeys does not recreate table with all the splitkey regions
[ https://issues.apache.org/jira/browse/HBASE-5583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13662145#comment-13662145 ] ramkrishna.s.vasudevan commented on HBASE-5583: --- Its more than one month, is it ok if we can revive now? Ted agreed to it a month ago itself. What other feel? Master restart on create table with splitkeys does not recreate table with all the splitkey regions --- Key: HBASE-5583 URL: https://issues.apache.org/jira/browse/HBASE-5583 Project: HBase Issue Type: Bug Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.95.1 Attachments: HBASE-5583_new_1.patch, HBASE-5583_new_1_review.patch, HBASE-5583_new_2.patch, HBASE-5583_new_4_WIP.patch, HBASE-5583_new_5_WIP_using_tableznode.patch - Create table using splitkeys - MAster goes down before all regions are added to meta - On master restart the table is again enabled but with less number of regions than specified in splitkeys Anyway client will get an exception if i had called sync create table. But table exists or not check will say table exists. Is this scenario to be handled by client only or can we have some mechanism on the master side for this? Pls suggest. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5583) Master restart on create table with splitkeys does not recreate table with all the splitkey regions
[ https://issues.apache.org/jira/browse/HBASE-5583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13635346#comment-13635346 ] ramkrishna.s.vasudevan commented on HBASE-5583: --- If HBASE-5487 takes some more time can we check this ? Master restart on create table with splitkeys does not recreate table with all the splitkey regions --- Key: HBASE-5583 URL: https://issues.apache.org/jira/browse/HBASE-5583 Project: HBase Issue Type: Bug Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.95.1 Attachments: HBASE-5583_new_1.patch, HBASE-5583_new_1_review.patch, HBASE-5583_new_2.patch, HBASE-5583_new_4_WIP.patch, HBASE-5583_new_5_WIP_using_tableznode.patch - Create table using splitkeys - MAster goes down before all regions are added to meta - On master restart the table is again enabled but with less number of regions than specified in splitkeys Anyway client will get an exception if i had called sync create table. But table exists or not check will say table exists. Is this scenario to be handled by client only or can we have some mechanism on the master side for this? Pls suggest. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5583) Master restart on create table with splitkeys does not recreate table with all the splitkey regions
[ https://issues.apache.org/jira/browse/HBASE-5583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13635349#comment-13635349 ] Ted Yu commented on HBASE-5583: --- HBASE-5487 has no assignee and no Fix Version. I think we can revive discussion for this JIRA. Master restart on create table with splitkeys does not recreate table with all the splitkey regions --- Key: HBASE-5583 URL: https://issues.apache.org/jira/browse/HBASE-5583 Project: HBase Issue Type: Bug Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.95.1 Attachments: HBASE-5583_new_1.patch, HBASE-5583_new_1_review.patch, HBASE-5583_new_2.patch, HBASE-5583_new_4_WIP.patch, HBASE-5583_new_5_WIP_using_tableznode.patch - Create table using splitkeys - MAster goes down before all regions are added to meta - On master restart the table is again enabled but with less number of regions than specified in splitkeys Anyway client will get an exception if i had called sync create table. But table exists or not check will say table exists. Is this scenario to be handled by client only or can we have some mechanism on the master side for this? Pls suggest. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5583) Master restart on create table with splitkeys does not recreate table with all the splitkey regions
[ https://issues.apache.org/jira/browse/HBASE-5583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13620654#comment-13620654 ] Enis Soztutar commented on HBASE-5583: -- Thanks Ram. Yes, the design in HBASE-5487 is not finalized yet unfortunately. We wanted to bring FATE, which will only handle the client initiated operations, not stuff like region assignment. Then the other WAL approaches came, and now Sergey's design. In any case, the issue probably won't cut it for 0.94. I would not object to getting the patch in this issue in, given that the design is still in discussion. My only concern would be that the table state is checked in a lot of places, so we should be careful. If you think that solving this issue only for create table with splits is important we can go with your approach. Otherwise, I would wait for a resolution on HBASE-5487. Master restart on create table with splitkeys does not recreate table with all the splitkey regions --- Key: HBASE-5583 URL: https://issues.apache.org/jira/browse/HBASE-5583 Project: HBase Issue Type: Bug Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.95.0 Attachments: HBASE-5583_new_1.patch, HBASE-5583_new_1_review.patch, HBASE-5583_new_2.patch, HBASE-5583_new_4_WIP.patch, HBASE-5583_new_5_WIP_using_tableznode.patch - Create table using splitkeys - MAster goes down before all regions are added to meta - On master restart the table is again enabled but with less number of regions than specified in splitkeys Anyway client will get an exception if i had called sync create table. But table exists or not check will say table exists. Is this scenario to be handled by client only or can we have some mechanism on the master side for this? Pls suggest. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5583) Master restart on create table with splitkeys does not recreate table with all the splitkey regions
[
https://issues.apache.org/jira/browse/HBASE-5583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13620663#comment-13620663
]
Hadoop QA commented on HBASE-5583:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12576723/HBASE-5583_new_1_review.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 6 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces lines longer than
100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.master.TestAssignmentManager
org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/5112//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5112//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5112//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5112//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5112//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5112//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5112//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5112//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5112//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/5112//console
This message is automatically generated.
Master restart on create table with splitkeys does not recreate table with
all the splitkey regions
---
Key: HBASE-5583
URL: https://issues.apache.org/jira/browse/HBASE-5583
Project: HBase
Issue Type: Bug
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Fix For: 0.95.0
Attachments: HBASE-5583_new_1.patch, HBASE-5583_new_1_review.patch,
HBASE-5583_new_2.patch, HBASE-5583_new_4_WIP.patch,
HBASE-5583_new_5_WIP_using_tableznode.patch
- Create table using splitkeys
- MAster goes down before all regions are added to meta
- On master restart the table is again enabled but with less number of
regions than specified in splitkeys
Anyway client will get an exception if i had called sync create table. But
table exists or not check will say table exists.
Is this scenario to be handled by client only or can we have some mechanism
on the master side for this? Pls suggest.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5583) Master restart on create table with splitkeys does not recreate table with all the splitkey regions
[ https://issues.apache.org/jira/browse/HBASE-5583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13620779#comment-13620779 ] ramkrishna.s.vasudevan commented on HBASE-5583: --- bq.I would not object to getting the patch in this issue in, given that the design is still in discussion. My only concern would be that the table state is checked in a lot of places, so we should be careful. If you think that solving this issue only for create table with splits is important we can go with your approach. Otherwise, I would wait for a resolution on HBASE-5487. Enis concern here is valid. What other folks think on this, i can work based on that. [~saint@gmail.com],[~te...@apache.org],[~jxiang],[~rajesh23],[~mbertozzi] - Your thoughts? Master restart on create table with splitkeys does not recreate table with all the splitkey regions --- Key: HBASE-5583 URL: https://issues.apache.org/jira/browse/HBASE-5583 Project: HBase Issue Type: Bug Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.95.0 Attachments: HBASE-5583_new_1.patch, HBASE-5583_new_1_review.patch, HBASE-5583_new_2.patch, HBASE-5583_new_4_WIP.patch, HBASE-5583_new_5_WIP_using_tableznode.patch - Create table using splitkeys - MAster goes down before all regions are added to meta - On master restart the table is again enabled but with less number of regions than specified in splitkeys Anyway client will get an exception if i had called sync create table. But table exists or not check will say table exists. Is this scenario to be handled by client only or can we have some mechanism on the master side for this? Pls suggest. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5583) Master restart on create table with splitkeys does not recreate table with all the splitkey regions
[ https://issues.apache.org/jira/browse/HBASE-5583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13621011#comment-13621011 ] Jimmy Xiang commented on HBASE-5583: To me, I think we can wait for HBASE-5487, if this is not so urgent. Master restart on create table with splitkeys does not recreate table with all the splitkey regions --- Key: HBASE-5583 URL: https://issues.apache.org/jira/browse/HBASE-5583 Project: HBase Issue Type: Bug Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.95.0 Attachments: HBASE-5583_new_1.patch, HBASE-5583_new_1_review.patch, HBASE-5583_new_2.patch, HBASE-5583_new_4_WIP.patch, HBASE-5583_new_5_WIP_using_tableznode.patch - Create table using splitkeys - MAster goes down before all regions are added to meta - On master restart the table is again enabled but with less number of regions than specified in splitkeys Anyway client will get an exception if i had called sync create table. But table exists or not check will say table exists. Is this scenario to be handled by client only or can we have some mechanism on the master side for this? Pls suggest. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5583) Master restart on create table with splitkeys does not recreate table with all the splitkey regions
[ https://issues.apache.org/jira/browse/HBASE-5583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13620616#comment-13620616 ] ramkrishna.s.vasudevan commented on HBASE-5583: --- Added review request in RB - https://reviews.apache.org/r/10237/ Master restart on create table with splitkeys does not recreate table with all the splitkey regions --- Key: HBASE-5583 URL: https://issues.apache.org/jira/browse/HBASE-5583 Project: HBase Issue Type: Bug Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.95.0 Attachments: HBASE-5583_new_1.patch, HBASE-5583_new_1_review.patch, HBASE-5583_new_2.patch, HBASE-5583_new_4_WIP.patch, HBASE-5583_new_5_WIP_using_tableznode.patch - Create table using splitkeys - MAster goes down before all regions are added to meta - On master restart the table is again enabled but with less number of regions than specified in splitkeys Anyway client will get an exception if i had called sync create table. But table exists or not check will say table exists. Is this scenario to be handled by client only or can we have some mechanism on the master side for this? Pls suggest. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5583) Master restart on create table with splitkeys does not recreate table with all the splitkey regions
[ https://issues.apache.org/jira/browse/HBASE-5583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13620640#comment-13620640 ] Enis Soztutar commented on HBASE-5583: -- Sorry for coming in this late to this jira. Ram, have you looked at the generic master coordinated actions jira, HBASE-5487? What you are doing here, and what we need in general for operations(create table, disable table, etc) running in master is: - Client initiates the Master operation, gets a trackId in an async response. - Operations are written in a sequence of adempotent steps, where each step can be undone. Each step can work on top of partial results from a previous execution. - Master executes the steps of operation in sequence. Master either keeps the state of the whole execution stack, or uses a WAL for keeping the execution state. - Client query back the execution state by asking the master with the obtained trackId. This patch only solves the create table problem, for some cases, but cannot be generalized to other operations. Instead of doing it case-by-case, we should really solve this in HBASE-5487 I think. As I outlined there, there are 3 ways, that we can approach this: - Keeping the master state in zookeeper. Your patch does this for create table. However, we should go with the FATE approach if we decide on this. - Keeping the state in an WAL from master. This log will use hlog, but will be owned by the active master. recovery is done using this. - Keeping the state in a system table. Since HBase tables in turn use WAL's, this is just indirectly using a WAL. Master restart on create table with splitkeys does not recreate table with all the splitkey regions --- Key: HBASE-5583 URL: https://issues.apache.org/jira/browse/HBASE-5583 Project: HBase Issue Type: Bug Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.95.0 Attachments: HBASE-5583_new_1.patch, HBASE-5583_new_1_review.patch, HBASE-5583_new_2.patch, HBASE-5583_new_4_WIP.patch, HBASE-5583_new_5_WIP_using_tableznode.patch - Create table using splitkeys - MAster goes down before all regions are added to meta - On master restart the table is again enabled but with less number of regions than specified in splitkeys Anyway client will get an exception if i had called sync create table. But table exists or not check will say table exists. Is this scenario to be handled by client only or can we have some mechanism on the master side for this? Pls suggest. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5583) Master restart on create table with splitkeys does not recreate table with all the splitkey regions
[ https://issues.apache.org/jira/browse/HBASE-5583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13620650#comment-13620650 ] ramkrishna.s.vasudevan commented on HBASE-5583: --- @Enis Off late did not watch this JIRA closely. I agree all the master originated table management operations should be generic. I will watch the JIRA and participate in discussions over there. So if we don't need this feature then i can create a patch for 0.94 if it makes sense to have it there. Till FATE is in place can we go ahead with this? Just asking your opinion. Master restart on create table with splitkeys does not recreate table with all the splitkey regions --- Key: HBASE-5583 URL: https://issues.apache.org/jira/browse/HBASE-5583 Project: HBase Issue Type: Bug Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.95.0 Attachments: HBASE-5583_new_1.patch, HBASE-5583_new_1_review.patch, HBASE-5583_new_2.patch, HBASE-5583_new_4_WIP.patch, HBASE-5583_new_5_WIP_using_tableznode.patch - Create table using splitkeys - MAster goes down before all regions are added to meta - On master restart the table is again enabled but with less number of regions than specified in splitkeys Anyway client will get an exception if i had called sync create table. But table exists or not check will say table exists. Is this scenario to be handled by client only or can we have some mechanism on the master side for this? Pls suggest. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5583) Master restart on create table with splitkeys does not recreate table with all the splitkey regions
[ https://issues.apache.org/jira/browse/HBASE-5583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619002#comment-13619002 ] ramkrishna.s.vasudevan commented on HBASE-5583: --- I am facing some problems in making the testcases run. Or else would have uploaded a patch. I am still figuring out the root cause. Will keep updated. Master restart on create table with splitkeys does not recreate table with all the splitkey regions --- Key: HBASE-5583 URL: https://issues.apache.org/jira/browse/HBASE-5583 Project: HBase Issue Type: Bug Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.95.0 Attachments: HBASE-5583_new_1.patch, HBASE-5583_new_2.patch, HBASE-5583_new_4_WIP.patch, HBASE-5583_new_5_WIP_using_tableznode.patch - Create table using splitkeys - MAster goes down before all regions are added to meta - On master restart the table is again enabled but with less number of regions than specified in splitkeys Anyway client will get an exception if i had called sync create table. But table exists or not check will say table exists. Is this scenario to be handled by client only or can we have some mechanism on the master side for this? Pls suggest. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5583) Master restart on create table with splitkeys does not recreate table with all the splitkey regions
[ https://issues.apache.org/jira/browse/HBASE-5583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13618002#comment-13618002 ] ramkrishna.s.vasudevan commented on HBASE-5583: --- @Jimmy, When i was with my previous employer we actually wanted this feature. The reason was tables was dropped and created on a daily basis. So because of this problem, the developers had written a client code which creates the table by adding directly into META and then tries to enable those regions. I think that was in 0.90 code. The later version of hbase made the creation more reliable but often in our integration tests and Functional testing we end up with the tables getting half created and we end up in problems. That is why it prompted us to work on this. bq.s it simpler to disable the table and let the user create the table again in such a case Yes this is possible. But involves the following changes that i can think of immediately - Even if the table znode is in ENABLING we should still disable the table. - If we mark the table disabled, user should be very clear that the table he created is partial and that is what is DISABLED now. I think the better way is to use a new state- say UNUSABLE (or something like this) which will make the user to take a clear course of action. I still feel the above steps are simpler. But table creation is the basic step and handling the failure scenarios in this should be more of automated rather than user interference is what i felt. Hence decided to go on with this way. @Ted Thanks for going thro the patch. I will make the necessary changes and upload it. Just waiting for more ideas and views on this. Master restart on create table with splitkeys does not recreate table with all the splitkey regions --- Key: HBASE-5583 URL: https://issues.apache.org/jira/browse/HBASE-5583 Project: HBase Issue Type: Bug Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.95.0 Attachments: HBASE-5583_new_1.patch, HBASE-5583_new_2.patch, HBASE-5583_new_4_WIP.patch, HBASE-5583_new_5_WIP_using_tableznode.patch - Create table using splitkeys - MAster goes down before all regions are added to meta - On master restart the table is again enabled but with less number of regions than specified in splitkeys Anyway client will get an exception if i had called sync create table. But table exists or not check will say table exists. Is this scenario to be handled by client only or can we have some mechanism on the master side for this? Pls suggest. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5583) Master restart on create table with splitkeys does not recreate table with all the splitkey regions
[ https://issues.apache.org/jira/browse/HBASE-5583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13618106#comment-13618106 ] Jimmy Xiang commented on HBASE-5583: @Ram, I see. Thanks. When you are ready, can we put it on the review board? Master restart on create table with splitkeys does not recreate table with all the splitkey regions --- Key: HBASE-5583 URL: https://issues.apache.org/jira/browse/HBASE-5583 Project: HBase Issue Type: Bug Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.95.0 Attachments: HBASE-5583_new_1.patch, HBASE-5583_new_2.patch, HBASE-5583_new_4_WIP.patch, HBASE-5583_new_5_WIP_using_tableznode.patch - Create table using splitkeys - MAster goes down before all regions are added to meta - On master restart the table is again enabled but with less number of regions than specified in splitkeys Anyway client will get an exception if i had called sync create table. But table exists or not check will say table exists. Is this scenario to be handled by client only or can we have some mechanism on the master side for this? Pls suggest. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5583) Master restart on create table with splitkeys does not recreate table with all the splitkey regions
[
https://issues.apache.org/jira/browse/HBASE-5583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13617755#comment-13617755
]
Ted Yu commented on HBASE-5583:
---
{code}
+ public static SetString getCreatingTables(ZooKeeperWatcher zkw) throws
KeeperException {
{code}
I think getTablesInCreation would be better method name.
{code}
+ public void removeCreateTableStates(final String tableName) {
{code}
Suggest renaming removeCreateTableStates() as
removeTableStateForTableInCreation().
{code}
+ private final static Table.State createTableStates[] = {
Table.State.CREATING, Table.State.CREATING_TD,
{code}
Why do you use an array instead of a Set ? That way you don't need to iterate
through the states in ZKTableReadOnly.isTableState()
{code}
+ ListString listChildrenNoWatch =
ZKUtil.listChildrenNoWatch(this.watcher, this.watcher.tableZNode+/+child);
+ for (String string : listChildrenNoWatch) {
+ System.out.println(Status node should be prenet +string);
+ }
{code}
I guess you would remove the above in the next patch.
For createTableCreationStatusNode():
{code}
+LOG.info(Created children node +numberOfChildren);
{code}
If you keep the above log, please add tableName.
{code}
+ public boolean checkAndSetCreatingTableStates(final String tableName) throws
KeeperException {
{code}
Remove the trailing 's' in method name.
Master restart on create table with splitkeys does not recreate table with
all the splitkey regions
---
Key: HBASE-5583
URL: https://issues.apache.org/jira/browse/HBASE-5583
Project: HBase
Issue Type: Bug
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Fix For: 0.95.0
Attachments: HBASE-5583_new_1.patch, HBASE-5583_new_2.patch,
HBASE-5583_new_4_WIP.patch, HBASE-5583_new_5_WIP_using_tableznode.patch
- Create table using splitkeys
- MAster goes down before all regions are added to meta
- On master restart the table is again enabled but with less number of
regions than specified in splitkeys
Anyway client will get an exception if i had called sync create table. But
table exists or not check will say table exists.
Is this scenario to be handled by client only or can we have some mechanism
on the master side for this? Pls suggest.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5583) Master restart on create table with splitkeys does not recreate table with all the splitkey regions
[ https://issues.apache.org/jira/browse/HBASE-5583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13617762#comment-13617762 ] Jimmy Xiang commented on HBASE-5583: Please ignore if you have already considered about this: is it simpler to disable the table and let the user create the table again in such a case? I assume this won't happen many times. Or show the user a big warning in this case and let them decide how to move ahead? Users should use caution if the cluster fails in the middle when a table is created. Master restart on create table with splitkeys does not recreate table with all the splitkey regions --- Key: HBASE-5583 URL: https://issues.apache.org/jira/browse/HBASE-5583 Project: HBase Issue Type: Bug Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.95.0 Attachments: HBASE-5583_new_1.patch, HBASE-5583_new_2.patch, HBASE-5583_new_4_WIP.patch, HBASE-5583_new_5_WIP_using_tableznode.patch - Create table using splitkeys - MAster goes down before all regions are added to meta - On master restart the table is again enabled but with less number of regions than specified in splitkeys Anyway client will get an exception if i had called sync create table. But table exists or not check will say table exists. Is this scenario to be handled by client only or can we have some mechanism on the master side for this? Pls suggest. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5583) Master restart on create table with splitkeys does not recreate table with all the splitkey regions
[ https://issues.apache.org/jira/browse/HBASE-5583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13616178#comment-13616178 ] ramkrishna.s.vasudevan commented on HBASE-5583: --- Thinking on the problems of having two znodes is that - Table is ENABLED but before the status znode is cleared the Master goes down. Now the new master may see that the znode is ENABLED but the status znode will remain there. We now have to check for every table if the status znode is present and if the table is in ENABLED state we need to delete the znode. - The other case is - while the table creation was going on theere was a failure in table creation and so we go and try to delete the status and znode and also the table znode. Now assume the master goes down just after the status znode deletion. Now the new master knows it is a partially created table but it will not know what action to take if the status znode is not available. According to me this is the only scenario where having 2 status znodes will be problematic. - The advantage of having only one znode is, our entire recovery just depends on the state present in the table znode. If the table is in any of the state other than ENABLED/ENABLING, DISABLING,DISABLED we go ahead with the existing code. If the znode is any of the the new states that is introduced by this patch then master would be clear in knowing what action to take. But introducing new state to the new existing table znode - am not sure if others will accept to it. So some feedback on this part will help me to proceed with the patch. Master restart on create table with splitkeys does not recreate table with all the splitkey regions --- Key: HBASE-5583 URL: https://issues.apache.org/jira/browse/HBASE-5583 Project: HBase Issue Type: Bug Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.95.0 Attachments: HBASE-5583_new_1.patch, HBASE-5583_new_2.patch, HBASE-5583_new_4_WIP.patch - Create table using splitkeys - MAster goes down before all regions are added to meta - On master restart the table is again enabled but with less number of regions than specified in splitkeys Anyway client will get an exception if i had called sync create table. But table exists or not check will say table exists. Is this scenario to be handled by client only or can we have some mechanism on the master side for this? Pls suggest. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5583) Master restart on create table with splitkeys does not recreate table with all the splitkey regions
[ https://issues.apache.org/jira/browse/HBASE-5583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13609212#comment-13609212 ] ramkrishna.s.vasudevan commented on HBASE-5583: --- I am attaching a patch here with the current state of work. This is not the final one. Still Exception handling and testing is pending. I would like to get myself clarified whether am going in the right direction. My first question would be - In order to maintain different states at which we are in while creating table should we go for a special node under the table znode or can we add new states to the existing table states like (DISABLING, ENABLING, DISABLED etc).? BAsed on this i can rebase the patch. Also pls note that the ZooKeeperProto.java is showing as a complete change with out which this patch should be comparitively smaller. - This patch basically tries to handle table creation on master failure and restart. - Adds a new state CREATINGTABLE instead of ENABLING so that we know that this table is partially created. - Adds a status node under /table to maintain various states of the table creation process (is this seperate node needed or not is my doubt). - Table lock related things needs to be checked. Even after we do all this HBaseAdmin.isTableAvailable(tableName, splitKeys) should be used by the client to check if all the regions are created. Feed back is appreciated. I can do more testing and handle ZK exceptions in a better way in my further patches. Thanks. Master restart on create table with splitkeys does not recreate table with all the splitkey regions --- Key: HBASE-5583 URL: https://issues.apache.org/jira/browse/HBASE-5583 Project: HBase Issue Type: Bug Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.95.0 Attachments: HBASE-5583_new_1.patch - Create table using splitkeys - MAster goes down before all regions are added to meta - On master restart the table is again enabled but with less number of regions than specified in splitkeys Anyway client will get an exception if i had called sync create table. But table exists or not check will say table exists. Is this scenario to be handled by client only or can we have some mechanism on the master side for this? Pls suggest. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5583) Master restart on create table with splitkeys does not recreate table with all the splitkey regions
[
https://issues.apache.org/jira/browse/HBASE-5583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13609237#comment-13609237
]
Anoop Sam John commented on HBASE-5583:
---
Ram
Started going through the patch. It looks so big change but is it
really? There generated file is fully rewritten!
ZooKeeper.proto
{code}
ENABLING = 3;
+CREATINGTABLE = 4;
{code}
Can we make CREATINGTABLE - CREATING?
{code}
+ enum State {
+CREATED_TD = 0;
+CREATED_REGIONINFO = 1;
+MOVED_TO_ORIG_LOCATION = 2;
+ADD_TO_META = 3;
+ASSIGN_USER_REGIONS = 4;
+DONE = 4;
+ }
{code}
Repeated ordinal. Typo?
Will continue..
Master restart on create table with splitkeys does not recreate table with
all the splitkey regions
---
Key: HBASE-5583
URL: https://issues.apache.org/jira/browse/HBASE-5583
Project: HBase
Issue Type: Bug
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Fix For: 0.95.0
Attachments: HBASE-5583_new_1.patch
- Create table using splitkeys
- MAster goes down before all regions are added to meta
- On master restart the table is again enabled but with less number of
regions than specified in splitkeys
Anyway client will get an exception if i had called sync create table. But
table exists or not check will say table exists.
Is this scenario to be handled by client only or can we have some mechanism
on the master side for this? Pls suggest.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5583) Master restart on create table with splitkeys does not recreate table with all the splitkey regions
[ https://issues.apache.org/jira/browse/HBASE-5583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13609245#comment-13609245 ] Anoop Sam John commented on HBASE-5583: --- bq.In order to maintain different states at which we are in while creating table should we go for a special node under the table znode or can we add new states to the existing table states like (DISABLING, ENABLING, DISABLED etc) IMO go with separate node. Master restart on create table with splitkeys does not recreate table with all the splitkey regions --- Key: HBASE-5583 URL: https://issues.apache.org/jira/browse/HBASE-5583 Project: HBase Issue Type: Bug Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.95.0 Attachments: HBASE-5583_new_1.patch - Create table using splitkeys - MAster goes down before all regions are added to meta - On master restart the table is again enabled but with less number of regions than specified in splitkeys Anyway client will get an exception if i had called sync create table. But table exists or not check will say table exists. Is this scenario to be handled by client only or can we have some mechanism on the master side for this? Pls suggest. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5583) Master restart on create table with splitkeys does not recreate table with all the splitkey regions
[ https://issues.apache.org/jira/browse/HBASE-5583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13609926#comment-13609926 ] ramkrishna.s.vasudevan commented on HBASE-5583: --- But going with seperate node we have to do the bookkeeping work for that new node. Also need to take care of exception that come out on setting data on that znode. Advantage that we get by updating the same table znode is that things are easy to maintain and we can do the updates automically like the deletion/updation of the table node alone would suffice to indicate some operation is done. But when we have another znode we need to update that znode also and things will not be automic. Particularly on restart scenarios we may face some problems. Anyway will list out the probs that i faced with one and 2 znodes. Master restart on create table with splitkeys does not recreate table with all the splitkey regions --- Key: HBASE-5583 URL: https://issues.apache.org/jira/browse/HBASE-5583 Project: HBase Issue Type: Bug Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.95.0 Attachments: HBASE-5583_new_1.patch - Create table using splitkeys - MAster goes down before all regions are added to meta - On master restart the table is again enabled but with less number of regions than specified in splitkeys Anyway client will get an exception if i had called sync create table. But table exists or not check will say table exists. Is this scenario to be handled by client only or can we have some mechanism on the master side for this? Pls suggest. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5583) Master restart on create table with splitkeys does not recreate table with all the splitkey regions
[
https://issues.apache.org/jira/browse/HBASE-5583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13605268#comment-13605268
]
Matteo Bertozzi commented on HBASE-5583:
{quote}
Could we consider make creating table as a transaction, e.g. create files in
tmp directory, atomic put regions to .META.?
{quote}
We have already that, the problem is that META and the filesystem are two
separate operations... so you may fail in the middle and have META but not the
fs dir moved...
The idea of the file-tracking table (mentioned in others jira, e.g. HBASE-7806)
was exactly to solve the problem and having a single source of truth... and
have the table creation/deletion as transaction
Master restart on create table with splitkeys does not recreate table with
all the splitkey regions
---
Key: HBASE-5583
URL: https://issues.apache.org/jira/browse/HBASE-5583
Project: HBase
Issue Type: Bug
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Fix For: 0.95.0
- Create table using splitkeys
- MAster goes down before all regions are added to meta
- On master restart the table is again enabled but with less number of
regions than specified in splitkeys
Anyway client will get an exception if i had called sync create table. But
table exists or not check will say table exists.
Is this scenario to be handled by client only or can we have some mechanism
on the master side for this? Pls suggest.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5583) Master restart on create table with splitkeys does not recreate table with all the splitkey regions
[ https://issues.apache.org/jira/browse/HBASE-5583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13605323#comment-13605323 ] ramkrishna.s.vasudevan commented on HBASE-5583: --- @Matteo My plan was to maintain the state of the operation in ZK or HDFS. Can i refresh the patch that i have and submit for review? Or you feel that will not be needed at all. Master restart on create table with splitkeys does not recreate table with all the splitkey regions --- Key: HBASE-5583 URL: https://issues.apache.org/jira/browse/HBASE-5583 Project: HBase Issue Type: Bug Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.95.0 - Create table using splitkeys - MAster goes down before all regions are added to meta - On master restart the table is again enabled but with less number of regions than specified in splitkeys Anyway client will get an exception if i had called sync create table. But table exists or not check will say table exists. Is this scenario to be handled by client only or can we have some mechanism on the master side for this? Pls suggest. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5583) Master restart on create table with splitkeys does not recreate table with all the splitkey regions
[
https://issues.apache.org/jira/browse/HBASE-5583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13605336#comment-13605336
]
Matteo Bertozzi commented on HBASE-5583:
{quote}
My plan was to maintain the state of the operation in ZK or HDFS. Can i refresh
the patch that i have and submit for review?
Or you feel that will not be needed at all.
{quote}
There's a long way to the file-tracking table,
so I'm +1 to have an alternative now based on a ZK or HDFS state
Master restart on create table with splitkeys does not recreate table with
all the splitkey regions
---
Key: HBASE-5583
URL: https://issues.apache.org/jira/browse/HBASE-5583
Project: HBase
Issue Type: Bug
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Fix For: 0.95.0
- Create table using splitkeys
- MAster goes down before all regions are added to meta
- On master restart the table is again enabled but with less number of
regions than specified in splitkeys
Anyway client will get an exception if i had called sync create table. But
table exists or not check will say table exists.
Is this scenario to be handled by client only or can we have some mechanism
on the master side for this? Pls suggest.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5583) Master restart on create table with splitkeys does not recreate table with all the splitkey regions
[ https://issues.apache.org/jira/browse/HBASE-5583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13604834#comment-13604834 ] chunhui shen commented on HBASE-5583: - Could we consider make creating table as a transaction, e.g. create files in tmp directory, atomic put regions to .META. ? Master restart on create table with splitkeys does not recreate table with all the splitkey regions --- Key: HBASE-5583 URL: https://issues.apache.org/jira/browse/HBASE-5583 Project: HBase Issue Type: Bug Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.95.0 - Create table using splitkeys - MAster goes down before all regions are added to meta - On master restart the table is again enabled but with less number of regions than specified in splitkeys Anyway client will get an exception if i had called sync create table. But table exists or not check will say table exists. Is this scenario to be handled by client only or can we have some mechanism on the master side for this? Pls suggest. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5583) Master restart on create table with splitkeys does not recreate table with all the splitkey regions
[ https://issues.apache.org/jira/browse/HBASE-5583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13604846#comment-13604846 ] ramkrishna.s.vasudevan commented on HBASE-5583: --- Ya that was my idea. I was working on that..had some rough working patch. But that also had some set of problems. Will revive that patch and get back on the issues/problems that we may face. Master restart on create table with splitkeys does not recreate table with all the splitkey regions --- Key: HBASE-5583 URL: https://issues.apache.org/jira/browse/HBASE-5583 Project: HBase Issue Type: Bug Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.95.0 - Create table using splitkeys - MAster goes down before all regions are added to meta - On master restart the table is again enabled but with less number of regions than specified in splitkeys Anyway client will get an exception if i had called sync create table. But table exists or not check will say table exists. Is this scenario to be handled by client only or can we have some mechanism on the master side for this? Pls suggest. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5583) Master restart on create table with splitkeys does not recreate table with all the splitkey regions
[ https://issues.apache.org/jira/browse/HBASE-5583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13246311#comment-13246311 ] Zhihong Yu commented on HBASE-5583: --- Expanding table creation state coverage makes sense. Master restart on create table with splitkeys does not recreate table with all the splitkey regions --- Key: HBASE-5583 URL: https://issues.apache.org/jira/browse/HBASE-5583 Project: HBase Issue Type: Bug Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.96.0 - Create table using splitkeys - MAster goes down before all regions are added to meta - On master restart the table is again enabled but with less number of regions than specified in splitkeys Anyway client will get an exception if i had called sync create table. But table exists or not check will say table exists. Is this scenario to be handled by client only or can we have some mechanism on the master side for this? Pls suggest. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5583) Master restart on create table with splitkeys does not recreate table with all the splitkey regions
[ https://issues.apache.org/jira/browse/HBASE-5583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13245371#comment-13245371 ] ramkrishna.s.vasudevan commented on HBASE-5583: --- First step update the status in the below to CREATETABLE /hbase/table/tableName /hbase/table/tableName/currentstep - Create nodes with hregioninfo in the constructor of create table handler. Create those many nodes as in the splitkeys. - /hbase/table/tableName/currentStep CREATED_SPLIT_KEYS (after the creation of all splitkey nodes) CREATING_TD ADD_TO_META ASSIGN_USER_REGIONS if all are done 'ENABLED' will be the state of the table. - add regioninfo to META and on success delete the node created node one by one - IF master fail over happens See in what step the node is If failed in ASSIGN_USER_REGIONS Just reassign all the regions If failed in ADD_TO_META Check what regions are not yet added to meta and then do the assignment for all the regions We can ensure that the nodes that are available in the zk are the ones for which the meta entry is not updated If failed in CREATING_TD If tabledescriptor not found create it again. if not go to the next step If failed in CREATED_SPLIT_KEYS Delete the node so that atleast the next time creation is successful. Just log it as we cannot throw error back to the client. === We need to create one more api in client called 'isTableAvailable()' accepting splitKeys also. so that the user can use them to check if really the table is created with all the splitkeys. Currently the isTableAvailable() just checks if atleast one region is assigned to any RS. The current changes does not guarentee that the user table will be created in a synchronous way to the client createTable api. but will ensure that while creating any table if the master failover happens the table doesnot get created with less number of regions but still the user thinks table creation is sucessful. We now distinguish the states where a table was in create flow or in the enable flow. Previously the ENABLING state was in master fail over scenario. Pls provide your inputs. Master restart on create table with splitkeys does not recreate table with all the splitkey regions --- Key: HBASE-5583 URL: https://issues.apache.org/jira/browse/HBASE-5583 Project: HBase Issue Type: Bug Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.96.0 - Create table using splitkeys - MAster goes down before all regions are added to meta - On master restart the table is again enabled but with less number of regions than specified in splitkeys Anyway client will get an exception if i had called sync create table. But table exists or not check will say table exists. Is this scenario to be handled by client only or can we have some mechanism on the master side for this? Pls suggest. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5583) Master restart on create table with splitkeys does not recreate table with all the splitkey regions
[ https://issues.apache.org/jira/browse/HBASE-5583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13233252#comment-13233252 ] ramkrishna.s.vasudevan commented on HBASE-5583: --- @Jon Yes we should go with some form of persisting the splitkeys. ZK should be the best place for that. Master restart on create table with splitkeys does not recreate table with all the splitkey regions --- Key: HBASE-5583 URL: https://issues.apache.org/jira/browse/HBASE-5583 Project: HBase Issue Type: Bug Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.96.0 - Create table using splitkeys - MAster goes down before all regions are added to meta - On master restart the table is again enabled but with less number of regions than specified in splitkeys Anyway client will get an exception if i had called sync create table. But table exists or not check will say table exists. Is this scenario to be handled by client only or can we have some mechanism on the master side for this? Pls suggest. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5583) Master restart on create table with splitkeys does not recreate table with all the splitkey regions
[ https://issues.apache.org/jira/browse/HBASE-5583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13233456#comment-13233456 ] ramkrishna.s.vasudevan commented on HBASE-5583: --- One more issue is there If the master goes down just after writing the table descriptors and before adding anything to the META, on master restart we move the enabling state tables to enabled only if they have entry in META. So this is another area we need to address. So ZK should ideally used in a better way to track the table creation steps. Master restart on create table with splitkeys does not recreate table with all the splitkey regions --- Key: HBASE-5583 URL: https://issues.apache.org/jira/browse/HBASE-5583 Project: HBase Issue Type: Bug Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.96.0 - Create table using splitkeys - MAster goes down before all regions are added to meta - On master restart the table is again enabled but with less number of regions than specified in splitkeys Anyway client will get an exception if i had called sync create table. But table exists or not check will say table exists. Is this scenario to be handled by client only or can we have some mechanism on the master side for this? Pls suggest. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5583) Master restart on create table with splitkeys does not recreate table with all the splitkey regions
[ https://issues.apache.org/jira/browse/HBASE-5583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13233150#comment-13233150 ] Jonathan Hsieh commented on HBASE-5583: --- Maybe we should put plans for master actions into ZK, and then remove each step as they are completed. This would allow a backup master or the restarted master to pick up and continue where the previous master had died. So this we may have a zk structure act like a queue with region infos and then let the masters drain from there to create .META. rows. I was thinking about adding a flag to info:regioninfo or extra column (info:unfinshedPresplitCreate) but this would seem to suffer from a similar problem if the master failed as these were being removed after all regions created. Master restart on create table with splitkeys does not recreate table with all the splitkey regions --- Key: HBASE-5583 URL: https://issues.apache.org/jira/browse/HBASE-5583 Project: HBase Issue Type: Bug Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.96.0 - Create table using splitkeys - MAster goes down before all regions are added to meta - On master restart the table is again enabled but with less number of regions than specified in splitkeys Anyway client will get an exception if i had called sync create table. But table exists or not check will say table exists. Is this scenario to be handled by client only or can we have some mechanism on the master side for this? Pls suggest. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
