[jira] [Commented] (HBASE-19959) How much RAM space is to be really consumed by the memstore?
[ https://issues.apache.org/jira/browse/HBASE-19959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16359868#comment-16359868 ] Chance Li commented on HBASE-19959: --- To flush all the data to make the memstore size is 0 . > How much RAM space is to be really consumed by the memstore? > > > Key: HBASE-19959 > URL: https://issues.apache.org/jira/browse/HBASE-19959 > Project: HBase > Issue Type: Brainstorming > Components: regionserver >Reporter: Chance Li >Priority: Minor > > Let's consider this scenario where memstoreLAB and ChunkPool is enable and > max memstore size is 10G, and after some time all pooled chunk have been > created, then flush all data, now memstore size is 0 but RAM actually have > consumed 10G, then continue writing big cell which will not use the chunk > pool but jvm heap, then memstore size will be increased to 10G(maybe more > because overhead). now we can see RAM actually consumed 20G (10G of pooled > chunk + 10G java objects), but the max memstore size is only 10G. > what I say is the max memstore size not only take care about the cell "size" > but also RAM really used. This will be a strict memory management: the max > memstore size limit the RAM space which the memstore or related module can be > used. > This really rarely occured. It's just for a robust memory managemant > semantically. > What do you think? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19959) How much RAM space is to be really consumed by the memstore?
[
https://issues.apache.org/jira/browse/HBASE-19959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16359851#comment-16359851
]
Anastasia Braginsky commented on HBASE-19959:
-
OK, I may be understood it wrongly what you write. But then:
{quote}Let's consider this scenario where memstoreLAB and ChunkPool is enable
and max memstore size is 10G, and after some time all pooled chunk have been
created, then flush all data, now memstore size is 0
{quote}
Why do you say memstore size is 0? If you consume pool chunks and write there
data, memstore must have value...
> How much RAM space is to be really consumed by the memstore?
>
>
> Key: HBASE-19959
> URL: https://issues.apache.org/jira/browse/HBASE-19959
> Project: HBase
> Issue Type: Brainstorming
> Components: regionserver
>Reporter: Chance Li
>Priority: Minor
>
> Let's consider this scenario where memstoreLAB and ChunkPool is enable and
> max memstore size is 10G, and after some time all pooled chunk have been
> created, then flush all data, now memstore size is 0 but RAM actually have
> consumed 10G, then continue writing big cell which will not use the chunk
> pool but jvm heap, then memstore size will be increased to 10G(maybe more
> because overhead). now we can see RAM actually consumed 20G (10G of pooled
> chunk + 10G java objects), but the max memstore size is only 10G.
> what I say is the max memstore size not only take care about the cell "size"
> but also RAM really used. This will be a strict memory management: the max
> memstore size limit the RAM space which the memstore or related module can be
> used.
> This really rarely occured. It's just for a robust memory managemant
> semantically.
> What do you think?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19959) How much RAM space is to be really consumed by the memstore?
[ https://issues.apache.org/jira/browse/HBASE-19959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358014#comment-16358014 ] Chance Li commented on HBASE-19959: --- In this scenario, every chunk in pool was used at least once , and the number of pooled chunk has reached the upper limit. That means chunk pool have take the physical RAM. > How much RAM space is to be really consumed by the memstore? > > > Key: HBASE-19959 > URL: https://issues.apache.org/jira/browse/HBASE-19959 > Project: HBase > Issue Type: Brainstorming > Components: regionserver >Reporter: Chance Li >Priority: Minor > > Let's consider this scenario where memstoreLAB and ChunkPool is enable and > max memstore size is 10G, and after some time all pooled chunk have been > created, then flush all data, now memstore size is 0 but RAM actually have > consumed 10G, then continue writing big cell which will not use the chunk > pool but jvm heap, then memstore size will be increased to 10G(maybe more > because overhead). now we can see RAM actually consumed 20G (10G of pooled > chunk + 10G java objects), but the max memstore size is only 10G. > what I say is the max memstore size not only take care about the cell "size" > but also RAM really used. This will be a strict memory management: the max > memstore size limit the RAM space which the memstore or related module can be > used. > This really rarely occured. It's just for a robust memory managemant > semantically. > What do you think? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-19959) How much RAM space is to be really consumed by the memstore?
[ https://issues.apache.org/jira/browse/HBASE-19959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16356980#comment-16356980 ] Anastasia Braginsky commented on HBASE-19959: - Pay attention that when something is allocated (using new or malloc) like chunks in pool, but never written (as you explain), than it only takes part of virtual memory (virtual address space). No physical RAM is used/taken. > How much RAM space is to be really consumed by the memstore? > > > Key: HBASE-19959 > URL: https://issues.apache.org/jira/browse/HBASE-19959 > Project: HBase > Issue Type: Brainstorming > Components: regionserver >Reporter: Chance Li >Priority: Minor > > Let's consider this scenario where memstoreLAB and ChunkPool is enable and > max memstore size is 10G, and after some time all pooled chunk have been > created, then flush all data, now memstore size is 0 but RAM actually have > consumed 10G, then continue writing big cell which will not use the chunk > pool but jvm heap, then memstore size will be increased to 10G(maybe more > because overhead). now we can see RAM actually consumed 20G (10G of pooled > chunk + 10G java objects), but the max memstore size is only 10G. > what I say is the max memstore size not only take care about the cell "size" > but also RAM really used. This will be a strict memory management: the max > memstore size limit the RAM space which the memstore or related module can be > used. > This really rarely occured. It's just for a robust memory managemant > semantically. > What do you think? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
