[
https://issues.apache.org/jira/browse/HBASE-20914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-20914:
--------------------------
Description:
While working on the parent issue, looking at a heap from a Master tha was
running ~650 servers and > 300k regions, I tripped over some silly items in the
heap:
1. Balancer has a regions x server matrix which takes up 18% of the Master heap
according to jxray and 40% according to eclipse. Looks like the matrix should
be regions x racks which would be much smaller (Issue came in with HBASE-18164
Fast locality computation in balancer -Added new LocalityCostFunction and
LocalityCandidateGenerator ..). See !Screen Shot 2018-07-19 at 1.20.23 PM.png!
!Screen Shot 2018-07-19 at 1.38.56 PM.png!
2. Duplicate Strings make up ~5% of the Master heap. Of these, ServerName seems
to be the font. Interesting is report that there 54k instances of ServerName in
this heap though there are only 650 Servers. See !Screen Shot 2018-07-19 at
1.20.23 PM.png!
3. ArrayDequeue initializes its internal elements array with 16 elements. We
use this in a few places. In Procedures, of which there are many in this heap,
we near never make use of this array. See !Screen Shot 2018-07-19 at 1.20.23
PM.png!
was:
While working on the parent issue, looking at a heap from a Master tha was
running ~650 servers and > 300k regions, I tripped over some silly items in the
heap:
1. Balancer has a regions x server matrix which takes up 18% of the Master heap
according to jxray and 40% according to eclipse. Looks like the matrix should
be regions x racks which would be much smaller (Issue came in with HBASE-18164
Fast locality computation in balancer -Added new LocalityCostFunction and
LocalityCandidateGenerator ..). See !Screen Shot 2018-07-19 at 2.22.42 PM.png!
and !Screen Shot 2018-07-19 at 2.43.42 PM.png!
2. Duplicate Strings make up ~5% of the Master heap. Of these, ServerName seems
to be the font. Interesting is report that there 54k instances of ServerName in
this heap though there are only 650 Servers. See !Screen Shot 2018-07-19 at
1.38.56 PM.png!
3. ArrayDequeue initializes its internal elements array with 16 elements. We
use this in a few places. In Procedures, of which there are many in this heap,
we near never make use of this array. See !Screen Shot 2018-07-19 at 1.20.23
PM.png!
> Trim Master memory usage
> ------------------------
>
> Key: HBASE-20914
> URL: https://issues.apache.org/jira/browse/HBASE-20914
> Project: HBase
> Issue Type: Sub-task
> Components: master
> Reporter: stack
> Assignee: stack
> Priority: Major
> Fix For: 2.0.2
>
> Attachments: Screen Shot 2018-07-19 at 1.20.23 PM.png, Screen Shot
> 2018-07-19 at 1.20.23 PM.png, Screen Shot 2018-07-19 at 1.38.56 PM.png,
> Screen Shot 2018-07-19 at 1.38.56 PM.png, Screen Shot 2018-07-19 at 2.22.42
> PM.png, Screen Shot 2018-07-19 at 2.22.42 PM.png, Screen Shot 2018-07-19 at
> 2.43.42 PM.png
>
>
> While working on the parent issue, looking at a heap from a Master tha was
> running ~650 servers and > 300k regions, I tripped over some silly items in
> the heap:
> 1. Balancer has a regions x server matrix which takes up 18% of the Master
> heap according to jxray and 40% according to eclipse. Looks like the matrix
> should be regions x racks which would be much smaller (Issue came in with
> HBASE-18164 Fast locality computation in balancer -Added new
> LocalityCostFunction and LocalityCandidateGenerator ..). See !Screen Shot
> 2018-07-19 at 1.20.23 PM.png! !Screen Shot 2018-07-19 at 1.38.56 PM.png!
> 2. Duplicate Strings make up ~5% of the Master heap. Of these, ServerName
> seems to be the font. Interesting is report that there 54k instances of
> ServerName in this heap though there are only 650 Servers. See !Screen Shot
> 2018-07-19 at 1.20.23 PM.png!
> 3. ArrayDequeue initializes its internal elements array with 16 elements. We
> use this in a few places. In Procedures, of which there are many in this
> heap, we near never make use of this array. See !Screen Shot 2018-07-19 at
> 1.20.23 PM.png!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
