[jira] [Commented] (HBASE-1730) Near-instantaneous online schema and table state updates
[
https://issues.apache.org/jira/browse/HBASE-1730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13092169#comment-13092169
]
nileema shingte commented on HBASE-1730:
Subbu,
I am not working on this. You can go ahead with this! :
> Near-instantaneous online schema and table state updates
>
>
> Key: HBASE-1730
> URL: https://issues.apache.org/jira/browse/HBASE-1730
> Project: HBase
> Issue Type: Improvement
>Reporter: Andrew Purtell
>Assignee: stack
>Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 1730-v2.patch, 1730-v3.patch, 1730.patch,
> HBASE-1730.patch
>
>
> We should not need to take a table offline to update HCD or HTD.
> One option for that is putting HTDs and HCDs up into ZK, with mirror on disk
> catalog tables to be used only for cold init scenarios, as discussed on IRC.
> In this scheme, regionservers hosting regions of a table would watch
> permanent nodes in ZK associated with that table for schema updates and take
> appropriate actions out of the watcher. In effect, schema updates become
> another item in the ToDo list.
> {{/hbase/tables//schema}}
> Must be associated with a write locking scheme also handled with ZK
> primitives to avoid situations where one concurrent update clobbers another.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4227) Modify the webUI so that default values of column families are not shown
[ https://issues.apache.org/jira/browse/HBASE-4227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13090309#comment-13090309 ] nileema shingte commented on HBASE-4227: I will create a patch for this with a describe 'table_name' and describe_details 'table_name' (former will print only non-default values and latter will print all) > Modify the webUI so that default values of column families are not shown > > > Key: HBASE-4227 > URL: https://issues.apache.org/jira/browse/HBASE-4227 > Project: HBase > Issue Type: Improvement >Reporter: Nicolas Spiegelberg >Assignee: nileema shingte >Priority: Minor > Fix For: 0.92.0 > > Attachments: HBASE-4227.patch, HBASE-4227.patch > > > With the introduction of online schema changes, it will become more > advantageous to put configuration knobs at the column family level vs global > configuration settings. This will create a nasty web UI experience for > showing table properties unless we default to showing the custom values > instead of all values. It's on the table if we want to modify the shell's > 'describe' method as well. scan '.META.' should definitely return the full > properties however. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4227) Modify the webUI so that default values of column families are not shown
[ https://issues.apache.org/jira/browse/HBASE-4227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nileema shingte updated HBASE-4227: --- Attachment: HBASE-4227.patch Addressed Stack's comments. > Modify the webUI so that default values of column families are not shown > > > Key: HBASE-4227 > URL: https://issues.apache.org/jira/browse/HBASE-4227 > Project: HBase > Issue Type: Improvement >Reporter: Nicolas Spiegelberg >Assignee: nileema shingte >Priority: Minor > Fix For: 0.92.0 > > Attachments: HBASE-4227.patch, HBASE-4227.patch > > > With the introduction of online schema changes, it will become more > advantageous to put configuration knobs at the column family level vs global > configuration settings. This will create a nasty web UI experience for > showing table properties unless we default to showing the custom values > instead of all values. It's on the table if we want to modify the shell's > 'describe' method as well. scan '.META.' should definitely return the full > properties however. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4227) Modify the webUI so that default values of column families are not shown
[ https://issues.apache.org/jira/browse/HBASE-4227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13087506#comment-13087506 ] nileema shingte commented on HBASE-4227: Yes, it should be a static map. I will send across a new patch for this. Thanks for the prompt review, Stack! > Modify the webUI so that default values of column families are not shown > > > Key: HBASE-4227 > URL: https://issues.apache.org/jira/browse/HBASE-4227 > Project: HBase > Issue Type: Improvement >Reporter: Nicolas Spiegelberg >Assignee: nileema shingte >Priority: Minor > Fix For: 0.92.0 > > Attachments: HBASE-4227.patch > > > With the introduction of online schema changes, it will become more > advantageous to put configuration knobs at the column family level vs global > configuration settings. This will create a nasty web UI experience for > showing table properties unless we default to showing the custom values > instead of all values. It's on the table if we want to modify the shell's > 'describe' method as well. scan '.META.' should definitely return the full > properties however. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4227) Modify the webUI so that default values of column families are not shown
[ https://issues.apache.org/jira/browse/HBASE-4227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nileema shingte updated HBASE-4227: --- Release Note: This patch addresses the following: 1. On the master web UI, the table description lists out only column families with non-default values. 2. A details link through which the complete table description is available. Status: Patch Available (was: Open) > Modify the webUI so that default values of column families are not shown > > > Key: HBASE-4227 > URL: https://issues.apache.org/jira/browse/HBASE-4227 > Project: HBase > Issue Type: Improvement >Reporter: Nicolas Spiegelberg >Assignee: nileema shingte >Priority: Minor > Fix For: 0.92.0 > > Attachments: HBASE-4227.patch > > > With the introduction of online schema changes, it will become more > advantageous to put configuration knobs at the column family level vs global > configuration settings. This will create a nasty web UI experience for > showing table properties unless we default to showing the custom values > instead of all values. It's on the table if we want to modify the shell's > 'describe' method as well. scan '.META.' should definitely return the full > properties however. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4227) Modify the webUI so that default values of column families are not shown
[ https://issues.apache.org/jira/browse/HBASE-4227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nileema shingte updated HBASE-4227: --- Attachment: HBASE-4227.patch > Modify the webUI so that default values of column families are not shown > > > Key: HBASE-4227 > URL: https://issues.apache.org/jira/browse/HBASE-4227 > Project: HBase > Issue Type: Improvement >Reporter: Nicolas Spiegelberg >Assignee: nileema shingte >Priority: Minor > Fix For: 0.92.0 > > Attachments: HBASE-4227.patch > > > With the introduction of online schema changes, it will become more > advantageous to put configuration knobs at the column family level vs global > configuration settings. This will create a nasty web UI experience for > showing table properties unless we default to showing the custom values > instead of all values. It's on the table if we want to modify the shell's > 'describe' method as well. scan '.META.' should definitely return the full > properties however. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-1730) Near-instantaneous online schema and table state updates
[
https://issues.apache.org/jira/browse/HBASE-1730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13083966#comment-13083966
]
nileema shingte commented on HBASE-1730:
Hi Ted,
Thank you for reviewing the code!
I have incorporated them and posted it on the review board here:
https://reviews.apache.org/r/1479/
> Near-instantaneous online schema and table state updates
>
>
> Key: HBASE-1730
> URL: https://issues.apache.org/jira/browse/HBASE-1730
> Project: HBase
> Issue Type: Improvement
>Reporter: Andrew Purtell
>Assignee: stack
>Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 1730-v2.patch, 1730-v3.patch, 1730.patch,
> HBASE-1730.patch
>
>
> We should not need to take a table offline to update HCD or HTD.
> One option for that is putting HTDs and HCDs up into ZK, with mirror on disk
> catalog tables to be used only for cold init scenarios, as discussed on IRC.
> In this scheme, regionservers hosting regions of a table would watch
> permanent nodes in ZK associated with that table for schema updates and take
> appropriate actions out of the watcher. In effect, schema updates become
> another item in the ToDo list.
> {{/hbase/tables//schema}}
> Must be associated with a write locking scheme also handled with ZK
> primitives to avoid situations where one concurrent update clobbers another.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-1730) Near-instantaneous online schema and table state updates
[
https://issues.apache.org/jira/browse/HBASE-1730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13083886#comment-13083886
]
nileema shingte commented on HBASE-1730:
The attached patch (HBASE-1730.patch) does the following:
When the master receives an alter table call (addColumn, modifyColumn,
deleteColumn, modifyTable), it updates the .tableinfo and then closes all the
regions of that table. The patch includes:
1. Changes to reopen the regions when any of the above operations are
performed.
2. Best effort is made to preserve the locality of regions by assigning it a
region plan before closing it.
3. Throttling logic that ensures that only a configurable number of regions are
closed per region server at a time.
4. alter command in the hbase shell will block until all the regions are
updated, providing a status "x/y regions updated" every second.
5. alter_async command that works exactly like alter, except that it does not
block for completion or provide the status.
6. alter_status which is a sync call and blocks to provide the
"x/y regions updated" status per second until all regions are updated.
7. modification in the unit test for enabling alter without disabling the
table.
Number of regions that are closed at a time (per region server) is
configurable.
> Near-instantaneous online schema and table state updates
>
>
> Key: HBASE-1730
> URL: https://issues.apache.org/jira/browse/HBASE-1730
> Project: HBase
> Issue Type: Improvement
>Reporter: Andrew Purtell
>Assignee: stack
>Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 1730-v2.patch, 1730-v3.patch, 1730.patch,
> HBASE-1730.patch
>
>
> We should not need to take a table offline to update HCD or HTD.
> One option for that is putting HTDs and HCDs up into ZK, with mirror on disk
> catalog tables to be used only for cold init scenarios, as discussed on IRC.
> In this scheme, regionservers hosting regions of a table would watch
> permanent nodes in ZK associated with that table for schema updates and take
> appropriate actions out of the watcher. In effect, schema updates become
> another item in the ToDo list.
> {{/hbase/tables//schema}}
> Must be associated with a write locking scheme also handled with ZK
> primitives to avoid situations where one concurrent update clobbers another.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-1730) Near-instantaneous online schema and table state updates
[
https://issues.apache.org/jira/browse/HBASE-1730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nileema shingte updated HBASE-1730:
---
Attachment: HBASE-1730.patch
Here is a patch for Dhruba's proposal.
> Near-instantaneous online schema and table state updates
>
>
> Key: HBASE-1730
> URL: https://issues.apache.org/jira/browse/HBASE-1730
> Project: HBase
> Issue Type: Improvement
>Reporter: Andrew Purtell
>Assignee: stack
>Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 1730-v2.patch, 1730-v3.patch, 1730.patch,
> HBASE-1730.patch
>
>
> We should not need to take a table offline to update HCD or HTD.
> One option for that is putting HTDs and HCDs up into ZK, with mirror on disk
> catalog tables to be used only for cold init scenarios, as discussed on IRC.
> In this scheme, regionservers hosting regions of a table would watch
> permanent nodes in ZK associated with that table for schema updates and take
> appropriate actions out of the watcher. In effect, schema updates become
> another item in the ToDo list.
> {{/hbase/tables//schema}}
> Must be associated with a write locking scheme also handled with ZK
> primitives to avoid situations where one concurrent update clobbers another.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-1730) Near-instantaneous online schema and table state updates
[
https://issues.apache.org/jira/browse/HBASE-1730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nileema shingte updated HBASE-1730:
---
Status: Patch Available (was: Open)
> Near-instantaneous online schema and table state updates
>
>
> Key: HBASE-1730
> URL: https://issues.apache.org/jira/browse/HBASE-1730
> Project: HBase
> Issue Type: Improvement
>Reporter: Andrew Purtell
>Assignee: stack
>Priority: Critical
> Fix For: 0.92.0
>
> Attachments: 1730-v2.patch, 1730-v3.patch, 1730.patch
>
>
> We should not need to take a table offline to update HCD or HTD.
> One option for that is putting HTDs and HCDs up into ZK, with mirror on disk
> catalog tables to be used only for cold init scenarios, as discussed on IRC.
> In this scheme, regionservers hosting regions of a table would watch
> permanent nodes in ZK associated with that table for schema updates and take
> appropriate actions out of the watcher. In effect, schema updates become
> another item in the ToDo list.
> {{/hbase/tables//schema}}
> Must be associated with a write locking scheme also handled with ZK
> primitives to avoid situations where one concurrent update clobbers another.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4046) expose more statistics from HBase Master node
[ https://issues.apache.org/jira/browse/HBASE-4046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nileema shingte updated HBASE-4046: --- Attachment: HBASE-4046.patch > expose more statistics from HBase Master node > - > > Key: HBASE-4046 > URL: https://issues.apache.org/jira/browse/HBASE-4046 > Project: HBase > Issue Type: Improvement >Reporter: nileema shingte >Assignee: nileema shingte >Priority: Minor > Attachments: HBASE-4046.patch > > > We can add the following statistics to the master. Some stats that can be > added are: > 1. number of logs split > 2. size of logs split > 3. number of region servers online > 4. number of region servers opened > 5. number of region servers expired -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4046) expose more statistics from HBase Master node
[ https://issues.apache.org/jira/browse/HBASE-4046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nileema shingte updated HBASE-4046: --- Status: Patch Available (was: In Progress) > expose more statistics from HBase Master node > - > > Key: HBASE-4046 > URL: https://issues.apache.org/jira/browse/HBASE-4046 > Project: HBase > Issue Type: Improvement >Reporter: nileema shingte >Assignee: nileema shingte >Priority: Minor > Attachments: HBASE-4046.patch > > > We can add the following statistics to the master. Some stats that can be > added are: > 1. number of logs split > 2. size of logs split > 3. number of region servers online > 4. number of region servers opened > 5. number of region servers expired -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Work started] (HBASE-4046) expose more statistics from HBase Master node
[ https://issues.apache.org/jira/browse/HBASE-4046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-4046 started by nileema shingte. > expose more statistics from HBase Master node > - > > Key: HBASE-4046 > URL: https://issues.apache.org/jira/browse/HBASE-4046 > Project: HBase > Issue Type: Improvement >Reporter: nileema shingte >Assignee: nileema shingte >Priority: Minor > Attachments: HBASE-4046.patch > > > We can add the following statistics to the master. Some stats that can be > added are: > 1. number of logs split > 2. size of logs split > 3. number of region servers online > 4. number of region servers opened > 5. number of region servers expired -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4046) expose more statistics from HBase Master node
[ https://issues.apache.org/jira/browse/HBASE-4046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nileema shingte updated HBASE-4046: --- Status: Open (was: Patch Available) > expose more statistics from HBase Master node > - > > Key: HBASE-4046 > URL: https://issues.apache.org/jira/browse/HBASE-4046 > Project: HBase > Issue Type: Improvement >Reporter: nileema shingte >Assignee: nileema shingte >Priority: Minor > > We can add the following statistics to the master. Some stats that can be > added are: > 1. number of logs split > 2. size of logs split > 3. number of region servers online > 4. number of region servers opened > 5. number of region servers expired -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4046) expose more statistics from HBase Master node
[ https://issues.apache.org/jira/browse/HBASE-4046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nileema shingte updated HBASE-4046: --- Status: Patch Available (was: Open) > expose more statistics from HBase Master node > - > > Key: HBASE-4046 > URL: https://issues.apache.org/jira/browse/HBASE-4046 > Project: HBase > Issue Type: Improvement >Reporter: nileema shingte >Assignee: nileema shingte >Priority: Minor > > We can add the following statistics to the master. Some stats that can be > added are: > 1. number of logs split > 2. size of logs split > 3. number of region servers online > 4. number of region servers opened > 5. number of region servers expired -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-4046) expose more statistics from HBase Master node
expose more statistics from HBase Master node - Key: HBASE-4046 URL: https://issues.apache.org/jira/browse/HBASE-4046 Project: HBase Issue Type: Improvement Reporter: nileema shingte Assignee: nileema shingte Priority: Minor We can add the following statistics to the master. Some stats that can be added are: 1. number of logs split 2. size of logs split 3. number of region servers online 4. number of region servers opened 5. number of region servers expired -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
