[jira] [Updated] (HBASE-24528) Improve balancer decision observability

2021-10-18 Thread Viraj Jasani (Jira)


 [ 
https://issues.apache.org/jira/browse/HBASE-24528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Viraj Jasani updated HBASE-24528:
-
Attachment: (was: NamedQueue_Framework_Design.pdf)

> Improve balancer decision observability
> ---
>
> Key: HBASE-24528
> URL: https://issues.apache.org/jira/browse/HBASE-24528
> Project: HBase
>  Issue Type: New Feature
>  Components: Admin, Balancer, Operability, shell, UI
>Reporter: Andrew Kyle Purtell
>Assignee: Viraj Jasani
>Priority: Major
> Fix For: 3.0.0-alpha-1, 1.7.0, 2.4.0
>
> Attachments: 
> NamedQueue_Framework_Design_HBASE-24528_HBASE-22978_HBASE-24718.pdf, 
> Screenshot 2020-08-12 at 11.50.43 PM.png, Screenshot 2020-08-15 at 8.08.57 
> PM.png
>
>
> We provide detailed INFO and DEBUG level logging of balancer decision 
> factors, outcome, and reassignment planning, as well as similarly detailed 
> logging of the resulting assignment manager activity. However, an operator 
> may need to perform online and interactive observation, debugging, or 
> performance analysis of current balancer activity. Scraping and correlating 
> the many log lines resulting from a balancer execution is labor intensive and 
> has a lot of latency (order of ~minutes to acquire and index, order of 
> ~minutes to correlate). 
> The balancer should maintain a rolling window of history, e.g. the last 100 
> region move plans, or last 1000 region move plans submitted to the assignment 
> manager. This history should include decision factor details and weights and 
> costs. The rsgroups balancer may be able to provide fairly simple decision 
> factors, like for example "this table was reassigned to that regionserver 
> group". The underlying or vanilla stochastic balancer on the other hand, 
> after a walk over random assignment plans, will have considered a number of 
> cost functions with various inputs (locality, load, etc.) and multipliers, 
> including custom cost functions. We can devise an extensible class structure 
> that represents explanations for balancer decisions, and for each region move 
> plan that is actually submitted to the assignment manager, we can keep the 
> explanations of all relevant decision factors alongside the other details of 
> the assignment plan like the region name, and the source and destination 
> regionservers. 
> This history should be available via API for use by new shell commands and 
> admin UI widgets.
> The new shell commands and UI widgets can unpack the representation of 
> balancer decision components into human readable output. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HBASE-24528) Improve balancer decision observability

2021-10-18 Thread Viraj Jasani (Jira)


 [ 
https://issues.apache.org/jira/browse/HBASE-24528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Viraj Jasani updated HBASE-24528:
-
Attachment: 
NamedQueue_Framework_Design_HBASE-24528_HBASE-22978_HBASE-24718.pdf

> Improve balancer decision observability
> ---
>
> Key: HBASE-24528
> URL: https://issues.apache.org/jira/browse/HBASE-24528
> Project: HBase
>  Issue Type: New Feature
>  Components: Admin, Balancer, Operability, shell, UI
>Reporter: Andrew Kyle Purtell
>Assignee: Viraj Jasani
>Priority: Major
> Fix For: 3.0.0-alpha-1, 1.7.0, 2.4.0
>
> Attachments: 
> NamedQueue_Framework_Design_HBASE-24528_HBASE-22978_HBASE-24718.pdf, 
> Screenshot 2020-08-12 at 11.50.43 PM.png, Screenshot 2020-08-15 at 8.08.57 
> PM.png
>
>
> We provide detailed INFO and DEBUG level logging of balancer decision 
> factors, outcome, and reassignment planning, as well as similarly detailed 
> logging of the resulting assignment manager activity. However, an operator 
> may need to perform online and interactive observation, debugging, or 
> performance analysis of current balancer activity. Scraping and correlating 
> the many log lines resulting from a balancer execution is labor intensive and 
> has a lot of latency (order of ~minutes to acquire and index, order of 
> ~minutes to correlate). 
> The balancer should maintain a rolling window of history, e.g. the last 100 
> region move plans, or last 1000 region move plans submitted to the assignment 
> manager. This history should include decision factor details and weights and 
> costs. The rsgroups balancer may be able to provide fairly simple decision 
> factors, like for example "this table was reassigned to that regionserver 
> group". The underlying or vanilla stochastic balancer on the other hand, 
> after a walk over random assignment plans, will have considered a number of 
> cost functions with various inputs (locality, load, etc.) and multipliers, 
> including custom cost functions. We can devise an extensible class structure 
> that represents explanations for balancer decisions, and for each region move 
> plan that is actually submitted to the assignment manager, we can keep the 
> explanations of all relevant decision factors alongside the other details of 
> the assignment plan like the region name, and the source and destination 
> regionservers. 
> This history should be available via API for use by new shell commands and 
> admin UI widgets.
> The new shell commands and UI widgets can unpack the representation of 
> balancer decision components into human readable output. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HBASE-24528) Improve balancer decision observability

2021-10-12 Thread Viraj Jasani (Jira)


 [ 
https://issues.apache.org/jira/browse/HBASE-24528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Viraj Jasani updated HBASE-24528:
-
Attachment: NamedQueue_Framework_Design.pdf

> Improve balancer decision observability
> ---
>
> Key: HBASE-24528
> URL: https://issues.apache.org/jira/browse/HBASE-24528
> Project: HBase
>  Issue Type: New Feature
>  Components: Admin, Balancer, Operability, shell, UI
>Reporter: Andrew Kyle Purtell
>Assignee: Viraj Jasani
>Priority: Major
> Fix For: 3.0.0-alpha-1, 1.7.0, 2.4.0
>
> Attachments: NamedQueue_Framework_Design.pdf, Screenshot 2020-08-12 
> at 11.50.43 PM.png, Screenshot 2020-08-15 at 8.08.57 PM.png
>
>
> We provide detailed INFO and DEBUG level logging of balancer decision 
> factors, outcome, and reassignment planning, as well as similarly detailed 
> logging of the resulting assignment manager activity. However, an operator 
> may need to perform online and interactive observation, debugging, or 
> performance analysis of current balancer activity. Scraping and correlating 
> the many log lines resulting from a balancer execution is labor intensive and 
> has a lot of latency (order of ~minutes to acquire and index, order of 
> ~minutes to correlate). 
> The balancer should maintain a rolling window of history, e.g. the last 100 
> region move plans, or last 1000 region move plans submitted to the assignment 
> manager. This history should include decision factor details and weights and 
> costs. The rsgroups balancer may be able to provide fairly simple decision 
> factors, like for example "this table was reassigned to that regionserver 
> group". The underlying or vanilla stochastic balancer on the other hand, 
> after a walk over random assignment plans, will have considered a number of 
> cost functions with various inputs (locality, load, etc.) and multipliers, 
> including custom cost functions. We can devise an extensible class structure 
> that represents explanations for balancer decisions, and for each region move 
> plan that is actually submitted to the assignment manager, we can keep the 
> explanations of all relevant decision factors alongside the other details of 
> the assignment plan like the region name, and the source and destination 
> regionservers. 
> This history should be available via API for use by new shell commands and 
> admin UI widgets.
> The new shell commands and UI widgets can unpack the representation of 
> balancer decision components into human readable output. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HBASE-24528) Improve balancer decision observability

2021-03-16 Thread Viraj Jasani (Jira)


 [ 
https://issues.apache.org/jira/browse/HBASE-24528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Viraj Jasani updated HBASE-24528:
-
Release Note: 
Retrieve latest balancer decisions made by LoadBalancers.

Examples:
  hbase> get_balancer_decisions   
Retrieve recent balancer decisions with region plans

  hbase> get_balancer_decisions LIMIT => 10
Retrieve 10 most recent balancer decisions with region plans


Config change:

hbase.master.balancer.decision.buffer.enabled:

  Indicates whether active HMaster has ring buffer running for storing
  balancer decisions in FIFO manner with limited entries. The size of
  the ring buffer is indicated by config:
  hbase.master.balancer.decision.queue.size



  was:
Retrieve latest balancer decisions made by LoadBalancers.

Examples:
  hbase> get_balancer_decisions   => Retrieve recent 
balancer decisions with
 region plans
  hbase> get_balancer_decisions LIMIT => 10   => Retrieve 10 most 
recent balancer decisions
 with region plans

Config change:

hbase.master.balancer.decision.buffer.enabled:

  Indicates whether active HMaster has ring buffer running for storing
  balancer decisions in FIFO manner with limited entries. The size of
  the ring buffer is indicated by config:
  hbase.master.balancer.decision.queue.size




> Improve balancer decision observability
> ---
>
> Key: HBASE-24528
> URL: https://issues.apache.org/jira/browse/HBASE-24528
> Project: HBase
>  Issue Type: New Feature
>  Components: Admin, Balancer, Operability, shell, UI
>Reporter: Andrew Kyle Purtell
>Assignee: Viraj Jasani
>Priority: Major
> Fix For: 3.0.0-alpha-1, 1.7.0, 2.4.0
>
> Attachments: Screenshot 2020-08-12 at 11.50.43 PM.png, Screenshot 
> 2020-08-15 at 8.08.57 PM.png
>
>
> We provide detailed INFO and DEBUG level logging of balancer decision 
> factors, outcome, and reassignment planning, as well as similarly detailed 
> logging of the resulting assignment manager activity. However, an operator 
> may need to perform online and interactive observation, debugging, or 
> performance analysis of current balancer activity. Scraping and correlating 
> the many log lines resulting from a balancer execution is labor intensive and 
> has a lot of latency (order of ~minutes to acquire and index, order of 
> ~minutes to correlate). 
> The balancer should maintain a rolling window of history, e.g. the last 100 
> region move plans, or last 1000 region move plans submitted to the assignment 
> manager. This history should include decision factor details and weights and 
> costs. The rsgroups balancer may be able to provide fairly simple decision 
> factors, like for example "this table was reassigned to that regionserver 
> group". The underlying or vanilla stochastic balancer on the other hand, 
> after a walk over random assignment plans, will have considered a number of 
> cost functions with various inputs (locality, load, etc.) and multipliers, 
> including custom cost functions. We can devise an extensible class structure 
> that represents explanations for balancer decisions, and for each region move 
> plan that is actually submitted to the assignment manager, we can keep the 
> explanations of all relevant decision factors alongside the other details of 
> the assignment plan like the region name, and the source and destination 
> regionservers. 
> This history should be available via API for use by new shell commands and 
> admin UI widgets.
> The new shell commands and UI widgets can unpack the representation of 
> balancer decision components into human readable output. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HBASE-24528) Improve balancer decision observability

2021-03-16 Thread Viraj Jasani (Jira)


 [ 
https://issues.apache.org/jira/browse/HBASE-24528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Viraj Jasani updated HBASE-24528:
-
Release Note: 
Retrieve latest balancer decisions made by LoadBalancers.

Examples:
  hbase> get_balancer_decisions   => Retrieve recent 
balancer decisions with
 region plans
  hbase> get_balancer_decisions LIMIT => 10   => Retrieve 10 most 
recent balancer decisions
 with region plans

Config change:

hbase.master.balancer.decision.buffer.enabled:

  Indicates whether active HMaster has ring buffer running for storing
  balancer decisions in FIFO manner with limited entries. The size of
  the ring buffer is indicated by config:
  hbase.master.balancer.decision.queue.size



> Improve balancer decision observability
> ---
>
> Key: HBASE-24528
> URL: https://issues.apache.org/jira/browse/HBASE-24528
> Project: HBase
>  Issue Type: New Feature
>  Components: Admin, Balancer, Operability, shell, UI
>Reporter: Andrew Kyle Purtell
>Assignee: Viraj Jasani
>Priority: Major
> Fix For: 3.0.0-alpha-1, 1.7.0, 2.4.0
>
> Attachments: Screenshot 2020-08-12 at 11.50.43 PM.png, Screenshot 
> 2020-08-15 at 8.08.57 PM.png
>
>
> We provide detailed INFO and DEBUG level logging of balancer decision 
> factors, outcome, and reassignment planning, as well as similarly detailed 
> logging of the resulting assignment manager activity. However, an operator 
> may need to perform online and interactive observation, debugging, or 
> performance analysis of current balancer activity. Scraping and correlating 
> the many log lines resulting from a balancer execution is labor intensive and 
> has a lot of latency (order of ~minutes to acquire and index, order of 
> ~minutes to correlate). 
> The balancer should maintain a rolling window of history, e.g. the last 100 
> region move plans, or last 1000 region move plans submitted to the assignment 
> manager. This history should include decision factor details and weights and 
> costs. The rsgroups balancer may be able to provide fairly simple decision 
> factors, like for example "this table was reassigned to that regionserver 
> group". The underlying or vanilla stochastic balancer on the other hand, 
> after a walk over random assignment plans, will have considered a number of 
> cost functions with various inputs (locality, load, etc.) and multipliers, 
> including custom cost functions. We can devise an extensible class structure 
> that represents explanations for balancer decisions, and for each region move 
> plan that is actually submitted to the assignment manager, we can keep the 
> explanations of all relevant decision factors alongside the other details of 
> the assignment plan like the region name, and the source and destination 
> regionservers. 
> This history should be available via API for use by new shell commands and 
> admin UI widgets.
> The new shell commands and UI widgets can unpack the representation of 
> balancer decision components into human readable output. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HBASE-24528) Improve balancer decision observability

2020-10-16 Thread Viraj Jasani (Jira)


 [ 
https://issues.apache.org/jira/browse/HBASE-24528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Viraj Jasani updated HBASE-24528:
-
Hadoop Flags: Reviewed
  Resolution: Fixed
  Status: Resolved  (was: Patch Available)

> Improve balancer decision observability
> ---
>
> Key: HBASE-24528
> URL: https://issues.apache.org/jira/browse/HBASE-24528
> Project: HBase
>  Issue Type: New Feature
>  Components: Admin, Balancer, Operability, shell, UI
>Reporter: Andrew Kyle Purtell
>Assignee: Viraj Jasani
>Priority: Major
> Fix For: 3.0.0-alpha-1, 1.7.0, 2.4.0
>
> Attachments: Screenshot 2020-08-12 at 11.50.43 PM.png, Screenshot 
> 2020-08-15 at 8.08.57 PM.png
>
>
> We provide detailed INFO and DEBUG level logging of balancer decision 
> factors, outcome, and reassignment planning, as well as similarly detailed 
> logging of the resulting assignment manager activity. However, an operator 
> may need to perform online and interactive observation, debugging, or 
> performance analysis of current balancer activity. Scraping and correlating 
> the many log lines resulting from a balancer execution is labor intensive and 
> has a lot of latency (order of ~minutes to acquire and index, order of 
> ~minutes to correlate). 
> The balancer should maintain a rolling window of history, e.g. the last 100 
> region move plans, or last 1000 region move plans submitted to the assignment 
> manager. This history should include decision factor details and weights and 
> costs. The rsgroups balancer may be able to provide fairly simple decision 
> factors, like for example "this table was reassigned to that regionserver 
> group". The underlying or vanilla stochastic balancer on the other hand, 
> after a walk over random assignment plans, will have considered a number of 
> cost functions with various inputs (locality, load, etc.) and multipliers, 
> including custom cost functions. We can devise an extensible class structure 
> that represents explanations for balancer decisions, and for each region move 
> plan that is actually submitted to the assignment manager, we can keep the 
> explanations of all relevant decision factors alongside the other details of 
> the assignment plan like the region name, and the source and destination 
> regionservers. 
> This history should be available via API for use by new shell commands and 
> admin UI widgets.
> The new shell commands and UI widgets can unpack the representation of 
> balancer decision components into human readable output. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HBASE-24528) Improve balancer decision observability

2020-08-27 Thread Viraj Jasani (Jira)


 [ 
https://issues.apache.org/jira/browse/HBASE-24528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Viraj Jasani updated HBASE-24528:
-
Fix Version/s: 2.4.0
   1.7.0
   3.0.0-alpha-1

> Improve balancer decision observability
> ---
>
> Key: HBASE-24528
> URL: https://issues.apache.org/jira/browse/HBASE-24528
> Project: HBase
>  Issue Type: New Feature
>  Components: Admin, Balancer, Operability, shell, UI
>Reporter: Andrew Kyle Purtell
>Assignee: Viraj Jasani
>Priority: Major
> Fix For: 3.0.0-alpha-1, 1.7.0, 2.4.0
>
> Attachments: Screenshot 2020-08-12 at 11.50.43 PM.png, Screenshot 
> 2020-08-15 at 8.08.57 PM.png
>
>
> We provide detailed INFO and DEBUG level logging of balancer decision 
> factors, outcome, and reassignment planning, as well as similarly detailed 
> logging of the resulting assignment manager activity. However, an operator 
> may need to perform online and interactive observation, debugging, or 
> performance analysis of current balancer activity. Scraping and correlating 
> the many log lines resulting from a balancer execution is labor intensive and 
> has a lot of latency (order of ~minutes to acquire and index, order of 
> ~minutes to correlate). 
> The balancer should maintain a rolling window of history, e.g. the last 100 
> region move plans, or last 1000 region move plans submitted to the assignment 
> manager. This history should include decision factor details and weights and 
> costs. The rsgroups balancer may be able to provide fairly simple decision 
> factors, like for example "this table was reassigned to that regionserver 
> group". The underlying or vanilla stochastic balancer on the other hand, 
> after a walk over random assignment plans, will have considered a number of 
> cost functions with various inputs (locality, load, etc.) and multipliers, 
> including custom cost functions. We can devise an extensible class structure 
> that represents explanations for balancer decisions, and for each region move 
> plan that is actually submitted to the assignment manager, we can keep the 
> explanations of all relevant decision factors alongside the other details of 
> the assignment plan like the region name, and the source and destination 
> regionservers. 
> This history should be available via API for use by new shell commands and 
> admin UI widgets.
> The new shell commands and UI widgets can unpack the representation of 
> balancer decision components into human readable output. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HBASE-24528) Improve balancer decision observability

2020-08-15 Thread Viraj Jasani (Jira)


 [ 
https://issues.apache.org/jira/browse/HBASE-24528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Viraj Jasani updated HBASE-24528:
-
Status: Patch Available  (was: In Progress)

> Improve balancer decision observability
> ---
>
> Key: HBASE-24528
> URL: https://issues.apache.org/jira/browse/HBASE-24528
> Project: HBase
>  Issue Type: New Feature
>  Components: Admin, Balancer, Operability, shell, UI
>Reporter: Andrew Kyle Purtell
>Assignee: Viraj Jasani
>Priority: Major
> Attachments: Screenshot 2020-08-12 at 11.50.43 PM.png, Screenshot 
> 2020-08-15 at 8.08.57 PM.png
>
>
> We provide detailed INFO and DEBUG level logging of balancer decision 
> factors, outcome, and reassignment planning, as well as similarly detailed 
> logging of the resulting assignment manager activity. However, an operator 
> may need to perform online and interactive observation, debugging, or 
> performance analysis of current balancer activity. Scraping and correlating 
> the many log lines resulting from a balancer execution is labor intensive and 
> has a lot of latency (order of ~minutes to acquire and index, order of 
> ~minutes to correlate). 
> The balancer should maintain a rolling window of history, e.g. the last 100 
> region move plans, or last 1000 region move plans submitted to the assignment 
> manager. This history should include decision factor details and weights and 
> costs. The rsgroups balancer may be able to provide fairly simple decision 
> factors, like for example "this table was reassigned to that regionserver 
> group". The underlying or vanilla stochastic balancer on the other hand, 
> after a walk over random assignment plans, will have considered a number of 
> cost functions with various inputs (locality, load, etc.) and multipliers, 
> including custom cost functions. We can devise an extensible class structure 
> that represents explanations for balancer decisions, and for each region move 
> plan that is actually submitted to the assignment manager, we can keep the 
> explanations of all relevant decision factors alongside the other details of 
> the assignment plan like the region name, and the source and destination 
> regionservers. 
> This history should be available via API for use by new shell commands and 
> admin UI widgets.
> The new shell commands and UI widgets can unpack the representation of 
> balancer decision components into human readable output. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HBASE-24528) Improve balancer decision observability

2020-08-15 Thread Viraj Jasani (Jira)


 [ 
https://issues.apache.org/jira/browse/HBASE-24528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Viraj Jasani updated HBASE-24528:
-
Attachment: Screenshot 2020-08-15 at 8.08.57 PM.png

> Improve balancer decision observability
> ---
>
> Key: HBASE-24528
> URL: https://issues.apache.org/jira/browse/HBASE-24528
> Project: HBase
>  Issue Type: New Feature
>  Components: Admin, Balancer, Operability, shell, UI
>Reporter: Andrew Kyle Purtell
>Assignee: Viraj Jasani
>Priority: Major
> Attachments: Screenshot 2020-08-12 at 11.50.43 PM.png, Screenshot 
> 2020-08-15 at 8.08.57 PM.png
>
>
> We provide detailed INFO and DEBUG level logging of balancer decision 
> factors, outcome, and reassignment planning, as well as similarly detailed 
> logging of the resulting assignment manager activity. However, an operator 
> may need to perform online and interactive observation, debugging, or 
> performance analysis of current balancer activity. Scraping and correlating 
> the many log lines resulting from a balancer execution is labor intensive and 
> has a lot of latency (order of ~minutes to acquire and index, order of 
> ~minutes to correlate). 
> The balancer should maintain a rolling window of history, e.g. the last 100 
> region move plans, or last 1000 region move plans submitted to the assignment 
> manager. This history should include decision factor details and weights and 
> costs. The rsgroups balancer may be able to provide fairly simple decision 
> factors, like for example "this table was reassigned to that regionserver 
> group". The underlying or vanilla stochastic balancer on the other hand, 
> after a walk over random assignment plans, will have considered a number of 
> cost functions with various inputs (locality, load, etc.) and multipliers, 
> including custom cost functions. We can devise an extensible class structure 
> that represents explanations for balancer decisions, and for each region move 
> plan that is actually submitted to the assignment manager, we can keep the 
> explanations of all relevant decision factors alongside the other details of 
> the assignment plan like the region name, and the source and destination 
> regionservers. 
> This history should be available via API for use by new shell commands and 
> admin UI widgets.
> The new shell commands and UI widgets can unpack the representation of 
> balancer decision components into human readable output. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HBASE-24528) Improve balancer decision observability

2020-08-12 Thread Viraj Jasani (Jira)


 [ 
https://issues.apache.org/jira/browse/HBASE-24528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Viraj Jasani updated HBASE-24528:
-
Attachment: Screenshot 2020-08-12 at 11.50.43 PM.png

> Improve balancer decision observability
> ---
>
> Key: HBASE-24528
> URL: https://issues.apache.org/jira/browse/HBASE-24528
> Project: HBase
>  Issue Type: New Feature
>  Components: Admin, Balancer, Operability, shell, UI
>Reporter: Andrew Kyle Purtell
>Assignee: Viraj Jasani
>Priority: Major
> Attachments: Screenshot 2020-08-12 at 11.50.43 PM.png
>
>
> We provide detailed INFO and DEBUG level logging of balancer decision 
> factors, outcome, and reassignment planning, as well as similarly detailed 
> logging of the resulting assignment manager activity. However, an operator 
> may need to perform online and interactive observation, debugging, or 
> performance analysis of current balancer activity. Scraping and correlating 
> the many log lines resulting from a balancer execution is labor intensive and 
> has a lot of latency (order of ~minutes to acquire and index, order of 
> ~minutes to correlate). 
> The balancer should maintain a rolling window of history, e.g. the last 100 
> region move plans, or last 1000 region move plans submitted to the assignment 
> manager. This history should include decision factor details and weights and 
> costs. The rsgroups balancer may be able to provide fairly simple decision 
> factors, like for example "this table was reassigned to that regionserver 
> group". The underlying or vanilla stochastic balancer on the other hand, 
> after a walk over random assignment plans, will have considered a number of 
> cost functions with various inputs (locality, load, etc.) and multipliers, 
> including custom cost functions. We can devise an extensible class structure 
> that represents explanations for balancer decisions, and for each region move 
> plan that is actually submitted to the assignment manager, we can keep the 
> explanations of all relevant decision factors alongside the other details of 
> the assignment plan like the region name, and the source and destination 
> regionservers. 
> This history should be available via API for use by new shell commands and 
> admin UI widgets.
> The new shell commands and UI widgets can unpack the representation of 
> balancer decision components into human readable output. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HBASE-24528) Improve balancer decision observability

2020-06-10 Thread Sean Busbey (Jira)


 [ 
https://issues.apache.org/jira/browse/HBASE-24528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey updated HBASE-24528:

Component/s: Operability

> Improve balancer decision observability
> ---
>
> Key: HBASE-24528
> URL: https://issues.apache.org/jira/browse/HBASE-24528
> Project: HBase
>  Issue Type: New Feature
>  Components: Admin, Balancer, Operability, shell, UI
>Reporter: Andrew Kyle Purtell
>Priority: Major
>
> We provide detailed INFO and DEBUG level logging of balancer decision 
> factors, outcome, and reassignment planning, as well as similarly detailed 
> logging of the resulting assignment manager activity. However, an operator 
> may need to perform online and interactive observation, debugging, or 
> performance analysis of current balancer activity. Scraping and correlating 
> the many log lines resulting from a balancer execution is labor intensive and 
> has a lot of latency (order of ~minutes to acquire and index, order of 
> ~minutes to correlate). 
> The balancer should maintain a rolling window of history, e.g. the last 100 
> region move plans, or last 1000 region move plans submitted to the assignment 
> manager. This history should include decision factor details and weights and 
> costs. The rsgroups balancer may be able to provide fairly simple decision 
> factors, like for example "this table was reassigned to that regionserver 
> group". The underlying or vanilla stochastic balancer on the other hand, 
> after a walk over random assignment plans, will have considered a number of 
> cost functions with various inputs (locality, load, etc.) and multipliers, 
> including custom cost functions. We can devise an extensible class structure 
> that represents explanations for balancer decisions, and for each region move 
> plan that is actually submitted to the assignment manager, we can keep the 
> explanations of all relevant decision factors alongside the other details of 
> the assignment plan like the region name, and the source and destination 
> regionservers. 
> This history should be available via API for use by new shell commands and 
> admin UI widgets.
> The new shell commands and UI widgets can unpack the representation of 
> balancer decision components into human readable output. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)