[jira] [Commented] (HBASE-3744) createTable blocks until all regions are out of transition
[ https://issues.apache.org/jira/browse/HBASE-3744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13030114#comment-13030114 ] stack commented on HBASE-3744: -- I applied the addendum w/ some massaging, no refactoring. Thanks for the patch Ted. Lets see if it cures the failing TestAdmin in 0.90. createTable blocks until all regions are out of transition -- Key: HBASE-3744 URL: https://issues.apache.org/jira/browse/HBASE-3744 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.90.1 Reporter: Todd Lipcon Assignee: Ted Yu Priority: Critical Fix For: 0.90.3 Attachments: 3744-addendum-for-TestAdmin.patch, 3744-addendum.txt, 3744-v2.txt, 3744-v3.txt, 3744.txt, create_big_tables.rb, create_big_tables.rb, create_big_tables.rb In HBASE-3305, the behavior of createTable was changed and introduced this bug: createTable now blocks until all regions have been assigned, since it uses BulkStartupAssigner. BulkStartupAssigner.waitUntilDone calls assignmentManager.waitUntilNoRegionsInTransition, which waits across all regions, not just the regions of the table that has just been created. We saw an issue where one table had a region which was unable to be opened, so it was stuck in RegionsInTransition permanently (every open was failing). Since this was the case, waitUntilDone would always block indefinitely even though the newly created table had been assigned. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3744) createTable blocks until all regions are out of transition
[ https://issues.apache.org/jira/browse/HBASE-3744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13025896#comment-13025896 ] Ted Yu commented on HBASE-3744: --- This JIRA changed the behavior of table creation to not wait for its regions to be fully assigned. This change caused TestAdmin failure. The addendum adds waiting in verifyRoundRobinDistribution(). createTable blocks until all regions are out of transition -- Key: HBASE-3744 URL: https://issues.apache.org/jira/browse/HBASE-3744 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.90.1 Reporter: Todd Lipcon Assignee: Ted Yu Priority: Critical Fix For: 0.90.3 Attachments: 3744-addendum-for-TestAdmin.patch, 3744-addendum.txt, 3744-v2.txt, 3744-v3.txt, 3744.txt, create_big_tables.rb, create_big_tables.rb, create_big_tables.rb In HBASE-3305, the behavior of createTable was changed and introduced this bug: createTable now blocks until all regions have been assigned, since it uses BulkStartupAssigner. BulkStartupAssigner.waitUntilDone calls assignmentManager.waitUntilNoRegionsInTransition, which waits across all regions, not just the regions of the table that has just been created. We saw an issue where one table had a region which was unable to be opened, so it was stuck in RegionsInTransition permanently (every open was failing). Since this was the case, waitUntilDone would always block indefinitely even though the newly created table had been assigned. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3744) createTable blocks until all regions are out of transition
[ https://issues.apache.org/jira/browse/HBASE-3744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13025906#comment-13025906 ] Jean-Daniel Cryans commented on HBASE-3744: --- Ah I see (reading the thread now on the ML), in the future it might be better to reference such discussions when attaching a patch. Regarding the code itself, would it be better to add a new method to the master and call it from the test (and pass the HRIs) instead of copying the code? createTable blocks until all regions are out of transition -- Key: HBASE-3744 URL: https://issues.apache.org/jira/browse/HBASE-3744 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.90.1 Reporter: Todd Lipcon Assignee: Ted Yu Priority: Critical Fix For: 0.90.3 Attachments: 3744-addendum-for-TestAdmin.patch, 3744-addendum.txt, 3744-v2.txt, 3744-v3.txt, 3744.txt, create_big_tables.rb, create_big_tables.rb, create_big_tables.rb In HBASE-3305, the behavior of createTable was changed and introduced this bug: createTable now blocks until all regions have been assigned, since it uses BulkStartupAssigner. BulkStartupAssigner.waitUntilDone calls assignmentManager.waitUntilNoRegionsInTransition, which waits across all regions, not just the regions of the table that has just been created. We saw an issue where one table had a region which was unable to be opened, so it was stuck in RegionsInTransition permanently (every open was failing). Since this was the case, waitUntilDone would always block indefinitely even though the newly created table had been assigned. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3744) createTable blocks until all regions are out of transition
[
https://issues.apache.org/jira/browse/HBASE-3744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13025916#comment-13025916
]
Ted Yu commented on HBASE-3744:
---
Separating step 6 of CreateTable() into a new method is good idea.
Shall we expose the new method through MasterServices ?
{code}
MasterServices services = TEST_UTIL.getMiniHBaseCluster().getMaster();
{code}
createTable blocks until all regions are out of transition
--
Key: HBASE-3744
URL: https://issues.apache.org/jira/browse/HBASE-3744
Project: HBase
Issue Type: Bug
Components: master
Affects Versions: 0.90.1
Reporter: Todd Lipcon
Assignee: Ted Yu
Priority: Critical
Fix For: 0.90.3
Attachments: 3744-addendum-for-TestAdmin.patch, 3744-addendum.txt,
3744-v2.txt, 3744-v3.txt, 3744.txt, create_big_tables.rb,
create_big_tables.rb, create_big_tables.rb
In HBASE-3305, the behavior of createTable was changed and introduced this
bug: createTable now blocks until all regions have been assigned, since it
uses BulkStartupAssigner. BulkStartupAssigner.waitUntilDone calls
assignmentManager.waitUntilNoRegionsInTransition, which waits across all
regions, not just the regions of the table that has just been created.
We saw an issue where one table had a region which was unable to be opened,
so it was stuck in RegionsInTransition permanently (every open was failing).
Since this was the case, waitUntilDone would always block indefinitely even
though the newly created table had been assigned.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3744) createTable blocks until all regions are out of transition
[ https://issues.apache.org/jira/browse/HBASE-3744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13025947#comment-13025947 ] Jean-Daniel Cryans commented on HBASE-3744: --- It seems it could be useful. createTable blocks until all regions are out of transition -- Key: HBASE-3744 URL: https://issues.apache.org/jira/browse/HBASE-3744 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.90.1 Reporter: Todd Lipcon Assignee: Ted Yu Priority: Critical Fix For: 0.90.3 Attachments: 3744-addendum-for-TestAdmin.patch, 3744-addendum.txt, 3744-v2.txt, 3744-v3.txt, 3744.txt, create_big_tables.rb, create_big_tables.rb, create_big_tables.rb In HBASE-3305, the behavior of createTable was changed and introduced this bug: createTable now blocks until all regions have been assigned, since it uses BulkStartupAssigner. BulkStartupAssigner.waitUntilDone calls assignmentManager.waitUntilNoRegionsInTransition, which waits across all regions, not just the regions of the table that has just been created. We saw an issue where one table had a region which was unable to be opened, so it was stuck in RegionsInTransition permanently (every open was failing). Since this was the case, waitUntilDone would always block indefinitely even though the newly created table had been assigned. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3744) createTable blocks until all regions are out of transition
[ https://issues.apache.org/jira/browse/HBASE-3744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13018772#comment-13018772 ] Hudson commented on HBASE-3744: --- Integrated in HBase-TRUNK #1846 (See [https://hudson.apache.org/hudson/job/HBase-TRUNK/1846/]) createTable blocks until all regions are out of transition -- Key: HBASE-3744 URL: https://issues.apache.org/jira/browse/HBASE-3744 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.90.1 Reporter: Todd Lipcon Assignee: Ted Yu Priority: Critical Fix For: 0.92.0 Attachments: 3744-addendum.txt, 3744-v2.txt, 3744-v3.txt, 3744.txt, create_big_tables.rb, create_big_tables.rb, create_big_tables.rb In HBASE-3305, the behavior of createTable was changed and introduced this bug: createTable now blocks until all regions have been assigned, since it uses BulkStartupAssigner. BulkStartupAssigner.waitUntilDone calls assignmentManager.waitUntilNoRegionsInTransition, which waits across all regions, not just the regions of the table that has just been created. We saw an issue where one table had a region which was unable to be opened, so it was stuck in RegionsInTransition permanently (every open was failing). Since this was the case, waitUntilDone would always block indefinitely even though the newly created table had been assigned. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3744) createTable blocks until all regions are out of transition
[ https://issues.apache.org/jira/browse/HBASE-3744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13017681#comment-13017681 ] Ted Yu commented on HBASE-3744: --- TestAdmin and TestLoadIncrementalHFiles passed. Version 3 is good. createTable blocks until all regions are out of transition -- Key: HBASE-3744 URL: https://issues.apache.org/jira/browse/HBASE-3744 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.90.1 Reporter: Todd Lipcon Assignee: Ted Yu Priority: Critical Fix For: 0.92.0 Attachments: 3744-v2.txt, 3744-v3.txt, 3744.txt In HBASE-3305, the behavior of createTable was changed and introduced this bug: createTable now blocks until all regions have been assigned, since it uses BulkStartupAssigner. BulkStartupAssigner.waitUntilDone calls assignmentManager.waitUntilNoRegionsInTransition, which waits across all regions, not just the regions of the table that has just been created. We saw an issue where one table had a region which was unable to be opened, so it was stuck in RegionsInTransition permanently (every open was failing). Since this was the case, waitUntilDone would always block indefinitely even though the newly created table had been assigned. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3744) createTable blocks until all regions are out of transition
[ https://issues.apache.org/jira/browse/HBASE-3744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13017770#comment-13017770 ] stack commented on HBASE-3744: -- Ok. I did some testing up on cluster creating tables and then bulk starting and killing a server during startup. I looked at writing tests but its awkward killing a server just after table creation and during assign. I'm going to commit. createTable blocks until all regions are out of transition -- Key: HBASE-3744 URL: https://issues.apache.org/jira/browse/HBASE-3744 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.90.1 Reporter: Todd Lipcon Assignee: Ted Yu Priority: Critical Fix For: 0.92.0 Attachments: 3744-v2.txt, 3744-v3.txt, 3744.txt In HBASE-3305, the behavior of createTable was changed and introduced this bug: createTable now blocks until all regions have been assigned, since it uses BulkStartupAssigner. BulkStartupAssigner.waitUntilDone calls assignmentManager.waitUntilNoRegionsInTransition, which waits across all regions, not just the regions of the table that has just been created. We saw an issue where one table had a region which was unable to be opened, so it was stuck in RegionsInTransition permanently (every open was failing). Since this was the case, waitUntilDone would always block indefinitely even though the newly created table had been assigned. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3744) createTable blocks until all regions are out of transition
[ https://issues.apache.org/jira/browse/HBASE-3744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13017800#comment-13017800 ] stack commented on HBASE-3744: -- Committed on TRUNK and Branch. Thanks for patch Ted. createTable blocks until all regions are out of transition -- Key: HBASE-3744 URL: https://issues.apache.org/jira/browse/HBASE-3744 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.90.1 Reporter: Todd Lipcon Assignee: Ted Yu Priority: Critical Fix For: 0.92.0 Attachments: 3744-v2.txt, 3744-v3.txt, 3744.txt In HBASE-3305, the behavior of createTable was changed and introduced this bug: createTable now blocks until all regions have been assigned, since it uses BulkStartupAssigner. BulkStartupAssigner.waitUntilDone calls assignmentManager.waitUntilNoRegionsInTransition, which waits across all regions, not just the regions of the table that has just been created. We saw an issue where one table had a region which was unable to be opened, so it was stuck in RegionsInTransition permanently (every open was failing). Since this was the case, waitUntilDone would always block indefinitely even though the newly created table had been assigned. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3744) createTable blocks until all regions are out of transition
[ https://issues.apache.org/jira/browse/HBASE-3744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13017191#comment-13017191 ] stack commented on HBASE-3744: -- So, I just ran into a problem running rolling restart where users were creating tables against the cluster. BulkStartupAssigner as originally written was rigged to fail out if anything went wrong during startup. Since create table uses this now, if anything goes wrong during table create, then the fail will kill master. Need to fix this. createTable blocks until all regions are out of transition -- Key: HBASE-3744 URL: https://issues.apache.org/jira/browse/HBASE-3744 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.90.1 Reporter: Todd Lipcon Assignee: Ted Yu Fix For: 0.92.0 Attachments: 3744.txt In HBASE-3305, the behavior of createTable was changed and introduced this bug: createTable now blocks until all regions have been assigned, since it uses BulkStartupAssigner. BulkStartupAssigner.waitUntilDone calls assignmentManager.waitUntilNoRegionsInTransition, which waits across all regions, not just the regions of the table that has just been created. We saw an issue where one table had a region which was unable to be opened, so it was stuck in RegionsInTransition permanently (every open was failing). Since this was the case, waitUntilDone would always block indefinitely even though the newly created table had been assigned. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3744) createTable blocks until all regions are out of transition
[
https://issues.apache.org/jira/browse/HBASE-3744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13017212#comment-13017212
]
stack commented on HBASE-3744:
--
Here is the exception I ran into:
{code}
2011-04-07 14:48:26,788 INFO
org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogWriter: Using syncFs --
HDFS-200
2011-04-07 14:48:26,789 INFO org.apache.hadoop.hbase.regionserver.wal.HLog: New
hlog
/hbase/jeichorn_test/710a5516e0de5da60ed7ce9466ac3b6f/.logs/hlog.1302212906763
2011-04-07 14:48:26,789 INFO org.apache.hadoop.hbase.regionserver.wal.HLog:
Using getNumCurrentReplicas--HDFS-826
2011-04-07 14:48:26,789 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
Instantiated jeichorn_test,,1302212906755.710a5516e0de5da60ed7ce9466ac3b6f.
2011-04-07 14:48:26,805 INFO org.apache.hadoop.hbase.regionserver.HRegion:
Onlined jeichorn_test,,1302212906755.710a5516e0de5da60ed7ce9466ac3b6f.; next
sequenceid=1
2011-04-07 14:48:26,807 INFO org.apache.hadoop.hbase.catalog.MetaEditor: Added
region jeichorn_test,,1302212906755.710a5516e0de5da60ed7ce9466ac3b6f. to META
2011-04-07 14:48:26,807 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
Closing jeichorn_test,,1302212906755.710a5516e0de5da60ed7ce9466ac3b6f.:
disabling compactions flushes
2011-04-07 14:48:26,807 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
Updates disabled for region
jeichorn_test,,1302212906755.710a5516e0de5da60ed7ce9466ac3b6f.
2011-04-07 14:48:26,807 DEBUG org.apache.hadoop.hbase.regionserver.Store:
closed default
2011-04-07 14:48:26,807 INFO org.apache.hadoop.hbase.regionserver.HRegion:
Closed jeichorn_test,,1302212906755.710a5516e0de5da60ed7ce9466ac3b6f.
2011-04-07 14:48:26,807 DEBUG org.apache.hadoop.hbase.regionserver.wal.HLog:
IPC Server handler 5 on 6.logSyncer interrupted while waiting for sync
requests
2011-04-07 14:48:26,807 INFO org.apache.hadoop.hbase.regionserver.wal.HLog: IPC
Server handler 5 on 6.logSyncer exiting
2011-04-07 14:48:26,808 DEBUG org.apache.hadoop.hbase.regionserver.wal.HLog:
closing hlog writer in
hdfs://sv2borg169:9000/hbase/jeichorn_test/710a5516e0de5da60ed7ce9466ac3b6f/.logs
2011-04-07 14:48:26,816 DEBUG org.apache.hadoop.hbase.regionserver.wal.HLog:
Moved 1 log files to
/hbase/jeichorn_test/710a5516e0de5da60ed7ce9466ac3b6f/.oldlogs
2011-04-07 14:48:26,817 INFO org.apache.hadoop.hbase.master.AssignmentManager:
Bulk assigning 1 region(s) round-robin across 9 server(s)
2011-04-07 14:48:26,817 DEBUG org.apache.hadoop.hbase.master.AssignmentManager:
Bulk assigning 0 region(s) to sv2borg160,60020,1302129205464
2011-04-07 14:48:26,818 DEBUG org.apache.hadoop.hbase.master.AssignmentManager:
Bulk assigning 0 region(s) to sv2borg161,60020,1302129205463
2011-04-07 14:48:26,818 DEBUG org.apache.hadoop.hbase.master.AssignmentManager:
Bulk assigning done for sv2borg160,60020,1302129205464
2011-04-07 14:48:26,818 DEBUG org.apache.hadoop.hbase.master.AssignmentManager:
Bulk assigning 0 region(s) to sv2borg162,60020,1302129205472
2011-04-07 14:48:26,819 DEBUG org.apache.hadoop.hbase.master.AssignmentManager:
Bulk assigning 0 region(s) to sv2borg163,60020,1302129205458
2011-04-07 14:48:26,820 DEBUG org.apache.hadoop.hbase.master.AssignmentManager:
Bulk assigning done for sv2borg161,60020,1302129205463
2011-04-07 14:48:26,820 DEBUG org.apache.hadoop.hbase.master.AssignmentManager:
Bulk assigning done for sv2borg162,60020,1302129205472
2011-04-07 14:48:26,821 DEBUG org.apache.hadoop.hbase.master.AssignmentManager:
Bulk assigning 0 region(s) to sv2borg164,60020,1302129205783
2011-04-07 14:48:26,822 DEBUG org.apache.hadoop.hbase.master.AssignmentManager:
Bulk assigning done for sv2borg163,60020,1302129205458
2011-04-07 14:48:26,822 DEBUG org.apache.hadoop.hbase.master.AssignmentManager:
Bulk assigning 1 region(s) to sv2borg165,60020,1302129205494
2011-04-07 14:48:26,822 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
master:6-0x2ec9c7920100ba Async create of unassigned node for
710a5516e0de5da60ed7ce9466ac3b6f with OFFLINE state
2011-04-07 14:48:26,823 DEBUG org.apache.hadoop.hbase.master.AssignmentManager:
Bulk assigning done for sv2borg164,60020,1302129205783
2011-04-07 14:48:26,823 DEBUG org.apache.hadoop.hbase.master.AssignmentManager:
Bulk assigning 0 region(s) to sv2borg166,60020,1302212905151
2011-04-07 14:48:26,823 DEBUG org.apache.hadoop.hbase.master.ServerManager: New
connection to sv2borg166,60020,1302212905151
2011-04-07 14:48:26,826 DEBUG org.apache.hadoop.hbase.master.AssignmentManager:
Bulk assigning 0 region(s) to sv2borg167,60020,1302129205469
2011-04-07 14:48:26,826 DEBUG
org.apache.hadoop.hbase.master.AssignmentManager$CreateUnassignedAsyncCallback:
rs=jeichorn_test,,1302212906755.710a5516e0de5da60ed7ce9466ac3b6f.
state=OFFLINE, ts=1302212906822, server=sv2borg165,60020,1302129205494
2011-04-07 14:48:26,826 FATAL org.apache.hadoop.hbase.master.HMaster: Failed
assignment of
[jira] [Commented] (HBASE-3744) createTable blocks until all regions are out of transition
[ https://issues.apache.org/jira/browse/HBASE-3744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13017217#comment-13017217 ] Ted Yu commented on HBASE-3744: --- Looks like SingleServerBulkAssigner.run() should be wrapped in try/catch block. createTable blocks until all regions are out of transition -- Key: HBASE-3744 URL: https://issues.apache.org/jira/browse/HBASE-3744 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.90.1 Reporter: Todd Lipcon Assignee: Ted Yu Priority: Critical Fix For: 0.92.0 Attachments: 3744.txt In HBASE-3305, the behavior of createTable was changed and introduced this bug: createTable now blocks until all regions have been assigned, since it uses BulkStartupAssigner. BulkStartupAssigner.waitUntilDone calls assignmentManager.waitUntilNoRegionsInTransition, which waits across all regions, not just the regions of the table that has just been created. We saw an issue where one table had a region which was unable to be opened, so it was stuck in RegionsInTransition permanently (every open was failing). Since this was the case, waitUntilDone would always block indefinitely even though the newly created table had been assigned. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3744) createTable blocks until all regions are out of transition
[ https://issues.apache.org/jira/browse/HBASE-3744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13017248#comment-13017248 ] Ted Yu commented on HBASE-3744: --- TestLoadIncrementalHFiles and TestAdmin pass. createTable blocks until all regions are out of transition -- Key: HBASE-3744 URL: https://issues.apache.org/jira/browse/HBASE-3744 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.90.1 Reporter: Todd Lipcon Assignee: Ted Yu Priority: Critical Fix For: 0.92.0 Attachments: 3744.txt, 3744.txt In HBASE-3305, the behavior of createTable was changed and introduced this bug: createTable now blocks until all regions have been assigned, since it uses BulkStartupAssigner. BulkStartupAssigner.waitUntilDone calls assignmentManager.waitUntilNoRegionsInTransition, which waits across all regions, not just the regions of the table that has just been created. We saw an issue where one table had a region which was unable to be opened, so it was stuck in RegionsInTransition permanently (every open was failing). Since this was the case, waitUntilDone would always block indefinitely even though the newly created table had been assigned. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3744) createTable blocks until all regions are out of transition
[ https://issues.apache.org/jira/browse/HBASE-3744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13017262#comment-13017262 ] Ted Yu commented on HBASE-3744: --- Thanks for detailed review Stack. Shall we call the new class BulkRegionAssigner ? Then we can use a boolean parameter, startup, for BulkRegionAssigner ctor to represent BulkStartupAssigner ? This parameter would determine the behavior of getUncaughtExceptionHandler(). createTable blocks until all regions are out of transition -- Key: HBASE-3744 URL: https://issues.apache.org/jira/browse/HBASE-3744 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.90.1 Reporter: Todd Lipcon Assignee: Ted Yu Priority: Critical Fix For: 0.92.0 Attachments: 3744.txt, 3744.txt In HBASE-3305, the behavior of createTable was changed and introduced this bug: createTable now blocks until all regions have been assigned, since it uses BulkStartupAssigner. BulkStartupAssigner.waitUntilDone calls assignmentManager.waitUntilNoRegionsInTransition, which waits across all regions, not just the regions of the table that has just been created. We saw an issue where one table had a region which was unable to be opened, so it was stuck in RegionsInTransition permanently (every open was failing). Since this was the case, waitUntilDone would always block indefinitely even though the newly created table had been assigned. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3744) createTable blocks until all regions are out of transition
[ https://issues.apache.org/jira/browse/HBASE-3744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13017269#comment-13017269 ] stack commented on HBASE-3744: -- I think BulkRegionAssigner is not such a good name. BulkAssigner does Regions. Thats what it does so adding Region to the name doesn't really help. And i don't think a boolean in the ctor the way to go either. Leave BulkStartupAssigner as it is. Then make a new class to use in your round robin table create. I think assignUserRegions should be changed to be assignAllUserRegions and you should not use this in your createTable method. Instead you should make a assignTableUserRegions or some such method and in there, not use BulkStartupAssigner, but your BulkTableAssigner instead. Something like that. createTable blocks until all regions are out of transition -- Key: HBASE-3744 URL: https://issues.apache.org/jira/browse/HBASE-3744 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.90.1 Reporter: Todd Lipcon Assignee: Ted Yu Priority: Critical Fix For: 0.92.0 Attachments: 3744.txt, 3744.txt In HBASE-3305, the behavior of createTable was changed and introduced this bug: createTable now blocks until all regions have been assigned, since it uses BulkStartupAssigner. BulkStartupAssigner.waitUntilDone calls assignmentManager.waitUntilNoRegionsInTransition, which waits across all regions, not just the regions of the table that has just been created. We saw an issue where one table had a region which was unable to be opened, so it was stuck in RegionsInTransition permanently (every open was failing). Since this was the case, waitUntilDone would always block indefinitely even though the newly created table had been assigned. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3744) createTable blocks until all regions are out of transition
[
https://issues.apache.org/jira/browse/HBASE-3744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13017270#comment-13017270
]
Ted Yu commented on HBASE-3744:
---
Looking closer at the exception Stack posted, I think the call chain was as
follows:
{code}
assign()
AssignmentManager.SingleServerBulkAssigner.run()
{code}
Therefore master abortion wasn't from UncaughtExceptionHandler.
createTable blocks until all regions are out of transition
--
Key: HBASE-3744
URL: https://issues.apache.org/jira/browse/HBASE-3744
Project: HBase
Issue Type: Bug
Components: master
Affects Versions: 0.90.1
Reporter: Todd Lipcon
Assignee: Ted Yu
Priority: Critical
Fix For: 0.92.0
Attachments: 3744.txt, 3744.txt
In HBASE-3305, the behavior of createTable was changed and introduced this
bug: createTable now blocks until all regions have been assigned, since it
uses BulkStartupAssigner. BulkStartupAssigner.waitUntilDone calls
assignmentManager.waitUntilNoRegionsInTransition, which waits across all
regions, not just the regions of the table that has just been created.
We saw an issue where one table had a region which was unable to be opened,
so it was stuck in RegionsInTransition permanently (every open was failing).
Since this was the case, waitUntilDone would always block indefinitely even
though the newly created table had been assigned.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3744) createTable blocks until all regions are out of transition
[ https://issues.apache.org/jira/browse/HBASE-3744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13017275#comment-13017275 ] stack commented on HBASE-3744: -- Bulk assign on table create is needed because of HBASE-3503 createTable blocks until all regions are out of transition -- Key: HBASE-3744 URL: https://issues.apache.org/jira/browse/HBASE-3744 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.90.1 Reporter: Todd Lipcon Assignee: Ted Yu Priority: Critical Fix For: 0.92.0 Attachments: 3744.txt, 3744.txt In HBASE-3305, the behavior of createTable was changed and introduced this bug: createTable now blocks until all regions have been assigned, since it uses BulkStartupAssigner. BulkStartupAssigner.waitUntilDone calls assignmentManager.waitUntilNoRegionsInTransition, which waits across all regions, not just the regions of the table that has just been created. We saw an issue where one table had a region which was unable to be opened, so it was stuck in RegionsInTransition permanently (every open was failing). Since this was the case, waitUntilDone would always block indefinitely even though the newly created table had been assigned. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3744) createTable blocks until all regions are out of transition
[
https://issues.apache.org/jira/browse/HBASE-3744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13017279#comment-13017279
]
Ted Yu commented on HBASE-3744:
---
Since AssignmentManager.assign() is outside BulkStartupAssigner and
SingleServerBulkAssigner. It needs a startup parameter so that decision of
whether to abort master can be made.
So BulkStartupAssigner and BulkTableAssigner would pass startup parameter to
SingleServerBulkAssigner.
The only difference between BulkStartupAssigner and BulkTableAssigner is in
their getUncaughtExceptionHandler() method. Meaning, we can write this:
{code}
static class BulkTableAssigner extends BulkStartupAssigner {
@Override
protected UncaughtExceptionHandler getUncaughtExceptionHandler() {
return new UncaughtExceptionHandler() {
@Override
public void uncaughtException(Thread t, Throwable e) {
LOG.error(Error assigning regions in + t.getName(), e);
}
};
}
}
{code}
createTable blocks until all regions are out of transition
--
Key: HBASE-3744
URL: https://issues.apache.org/jira/browse/HBASE-3744
Project: HBase
Issue Type: Bug
Components: master
Affects Versions: 0.90.1
Reporter: Todd Lipcon
Assignee: Ted Yu
Priority: Critical
Fix For: 0.92.0
Attachments: 3744.txt, 3744.txt
In HBASE-3305, the behavior of createTable was changed and introduced this
bug: createTable now blocks until all regions have been assigned, since it
uses BulkStartupAssigner. BulkStartupAssigner.waitUntilDone calls
assignmentManager.waitUntilNoRegionsInTransition, which waits across all
regions, not just the regions of the table that has just been created.
We saw an issue where one table had a region which was unable to be opened,
so it was stuck in RegionsInTransition permanently (every open was failing).
Since this was the case, waitUntilDone would always block indefinitely even
though the newly created table had been assigned.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3744) createTable blocks until all regions are out of transition
[ https://issues.apache.org/jira/browse/HBASE-3744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13017283#comment-13017283 ] Ted Yu commented on HBASE-3744: --- Currently assignAllUserRegions() calls assignUserRegions() for round-robin assignment. createTable blocks until all regions are out of transition -- Key: HBASE-3744 URL: https://issues.apache.org/jira/browse/HBASE-3744 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.90.1 Reporter: Todd Lipcon Assignee: Ted Yu Priority: Critical Fix For: 0.92.0 Attachments: 3744.txt, 3744.txt In HBASE-3305, the behavior of createTable was changed and introduced this bug: createTable now blocks until all regions have been assigned, since it uses BulkStartupAssigner. BulkStartupAssigner.waitUntilDone calls assignmentManager.waitUntilNoRegionsInTransition, which waits across all regions, not just the regions of the table that has just been created. We saw an issue where one table had a region which was unable to be opened, so it was stuck in RegionsInTransition permanently (every open was failing). Since this was the case, waitUntilDone would always block indefinitely even though the newly created table had been assigned. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3744) createTable blocks until all regions are out of transition
[ https://issues.apache.org/jira/browse/HBASE-3744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13016577#comment-13016577 ] Ted Yu commented on HBASE-3744: --- From IRC: tlipcon_: it seems wrong that we use something called BukStartupAssigner in the first place [2:54pm] tlipcon_: since this is not startup ... tlipcon_: i think BulkStartupAssigner should be renamed [3:11pm] tlipcon_: and then either a parameter or a different subclass that doesn't wait createTable blocks until all regions are out of transition -- Key: HBASE-3744 URL: https://issues.apache.org/jira/browse/HBASE-3744 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.90.1 Reporter: Todd Lipcon Assignee: Ted Yu Fix For: 0.92.0 Attachments: 3744.txt In HBASE-3305, the behavior of createTable was changed and introduced this bug: createTable now blocks until all regions have been assigned, since it uses BulkStartupAssigner. BulkStartupAssigner.waitUntilDone calls assignmentManager.waitUntilNoRegionsInTransition, which waits across all regions, not just the regions of the table that has just been created. We saw an issue where one table had a region which was unable to be opened, so it was stuck in RegionsInTransition permanently (every open was failing). Since this was the case, waitUntilDone would always block indefinitely even though the newly created table had been assigned. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3744) createTable blocks until all regions are out of transition
[
https://issues.apache.org/jira/browse/HBASE-3744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13016583#comment-13016583
]
Ted Yu commented on HBASE-3744:
---
Currently we have:
{code}
static class BulkStartupAssigner extends BulkAssigner {
{code}
We also have:
{code}
class BulkDisabler extends BulkAssigner {
{code}
If we rename BulkAssigner as AbstractBulkAssigner, we can rename
BulkStartupAssigner as BulkAssigner
createTable blocks until all regions are out of transition
--
Key: HBASE-3744
URL: https://issues.apache.org/jira/browse/HBASE-3744
Project: HBase
Issue Type: Bug
Components: master
Affects Versions: 0.90.1
Reporter: Todd Lipcon
Assignee: Ted Yu
Fix For: 0.92.0
Attachments: 3744.txt
In HBASE-3305, the behavior of createTable was changed and introduced this
bug: createTable now blocks until all regions have been assigned, since it
uses BulkStartupAssigner. BulkStartupAssigner.waitUntilDone calls
assignmentManager.waitUntilNoRegionsInTransition, which waits across all
regions, not just the regions of the table that has just been created.
We saw an issue where one table had a region which was unable to be opened,
so it was stuck in RegionsInTransition permanently (every open was failing).
Since this was the case, waitUntilDone would always block indefinitely even
though the newly created table had been assigned.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3744) createTable blocks until all regions are out of transition
[ https://issues.apache.org/jira/browse/HBASE-3744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13016692#comment-13016692 ] stack commented on HBASE-3744: -- BulkStartupAssigner was for bulk assigning on startup. Seems like its been pulled around. Could BulkStartupAssigner be renamed BulkOpener? And BulkDisabler named BulkCloser? I see need of a BulkOpen on regionserver crash (if its not being used already). createTable blocks until all regions are out of transition -- Key: HBASE-3744 URL: https://issues.apache.org/jira/browse/HBASE-3744 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.90.1 Reporter: Todd Lipcon Assignee: Ted Yu Fix For: 0.92.0 Attachments: 3744.txt In HBASE-3305, the behavior of createTable was changed and introduced this bug: createTable now blocks until all regions have been assigned, since it uses BulkStartupAssigner. BulkStartupAssigner.waitUntilDone calls assignmentManager.waitUntilNoRegionsInTransition, which waits across all regions, not just the regions of the table that has just been created. We saw an issue where one table had a region which was unable to be opened, so it was stuck in RegionsInTransition permanently (every open was failing). Since this was the case, waitUntilDone would always block indefinitely even though the newly created table had been assigned. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3744) createTable blocks until all regions are out of transition
[
https://issues.apache.org/jira/browse/HBASE-3744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13016698#comment-13016698
]
Ted Yu commented on HBASE-3744:
---
BulkOpen isn't used.
We also have:
{code}
class BulkEnabler extends BulkAssigner {
{code}
createTable blocks until all regions are out of transition
--
Key: HBASE-3744
URL: https://issues.apache.org/jira/browse/HBASE-3744
Project: HBase
Issue Type: Bug
Components: master
Affects Versions: 0.90.1
Reporter: Todd Lipcon
Assignee: Ted Yu
Fix For: 0.92.0
Attachments: 3744.txt
In HBASE-3305, the behavior of createTable was changed and introduced this
bug: createTable now blocks until all regions have been assigned, since it
uses BulkStartupAssigner. BulkStartupAssigner.waitUntilDone calls
assignmentManager.waitUntilNoRegionsInTransition, which waits across all
regions, not just the regions of the table that has just been created.
We saw an issue where one table had a region which was unable to be opened,
so it was stuck in RegionsInTransition permanently (every open was failing).
Since this was the case, waitUntilDone would always block indefinitely even
though the newly created table had been assigned.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
