[jira] [Commented] (HBASE-20188) [TESTING] Performance
[ https://issues.apache.org/jira/browse/HBASE-20188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16425146#comment-16425146 ] Eshcar Hillel commented on HBASE-20188: --- bq. can you please share how can we switch on short-circuit reads in our experiments? OK I see how to do it in HBASE-20377. Do I need to set all 5 parameters that are mentioned there or just the first? > [TESTING] Performance > - > > Key: HBASE-20188 > URL: https://issues.apache.org/jira/browse/HBASE-20188 > Project: HBase > Issue Type: Umbrella > Components: Performance >Reporter: stack >Assignee: stack >Priority: Blocker > Fix For: 2.0.0 > > Attachments: CAM-CONFIG-V01.patch, HBASE-20188.sh, HBase 2.0 > performance evaluation - Basic vs None_ system settings.pdf, > ITBLL2.5B_1.2.7vs2.0.0_cpu.png, ITBLL2.5B_1.2.7vs2.0.0_gctime.png, > ITBLL2.5B_1.2.7vs2.0.0_iops.png, ITBLL2.5B_1.2.7vs2.0.0_load.png, > ITBLL2.5B_1.2.7vs2.0.0_memheap.png, ITBLL2.5B_1.2.7vs2.0.0_memstore.png, > ITBLL2.5B_1.2.7vs2.0.0_ops.png, > ITBLL2.5B_1.2.7vs2.0.0_ops_NOT_summing_regions.png, YCSB_CPU.png, > YCSB_GC_TIME.png, YCSB_IN_MEMORY_COMPACTION=NONE.ops.png, YCSB_MEMSTORE.png, > YCSB_OPs.png, YCSB_in-memory-compaction=NONE.ops.png, YCSB_load.png, > flamegraph-1072.1.svg, flamegraph-1072.2.svg, > lock.127.workloadc.20180402T200918Z.svg, > lock.2.memsize2.c.20180403T160257Z.svg, tree.txt > > > How does 2.0.0 compare to old versions? Is it faster, slower? There is rumor > that it is much slower, that the problem is the asyncwal writing. Does > in-memory compaction slow us down or speed us up? What happens when you > enable offheaping? > Keep notes here in this umbrella issue. Need to be able to say something > about perf when 2.0.0 ships. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20188) [TESTING] Performance
[
https://issues.apache.org/jira/browse/HBASE-20188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16426130#comment-16426130
]
Eshcar Hillel commented on HBASE-20188:
---
I am trying to write a new workload that applies client side buffering by using
{{clientbuffering=true}}
However when running the workload I get the following exception in line
{{Preconditions.checkNotNull(bufferedMutator);}}
{code}
java.lang.NoClassDefFoundError: com/google/common/base/Preconditions
at com.yahoo.ycsb.db.HBaseClient10.update(HBaseClient10.java:441)
at com.yahoo.ycsb.DBWrapper.update(DBWrapper.java:198)
at
com.yahoo.ycsb.workloads.CoreWorkload.doTransactionUpdate(CoreWorkload.java:775)
at
com.yahoo.ycsb.workloads.CoreWorkload.doTransaction(CoreWorkload.java:608)
at com.yahoo.ycsb.ClientThread.run(Client.java:454)
at java.lang.Thread.run(Thread.java:745)
{code}
I was able to use this ycsb property in the past.
Anyone aware of changes to client implementation that result in a null
{{bufferedMutator}}?
> [TESTING] Performance
> -
>
> Key: HBASE-20188
> URL: https://issues.apache.org/jira/browse/HBASE-20188
> Project: HBase
> Issue Type: Umbrella
> Components: Performance
>Reporter: stack
>Assignee: stack
>Priority: Blocker
> Fix For: 2.0.0
>
> Attachments: CAM-CONFIG-V01.patch, HBASE-20188.sh, HBase 2.0
> performance evaluation - Basic vs None_ system settings.pdf,
> ITBLL2.5B_1.2.7vs2.0.0_cpu.png, ITBLL2.5B_1.2.7vs2.0.0_gctime.png,
> ITBLL2.5B_1.2.7vs2.0.0_iops.png, ITBLL2.5B_1.2.7vs2.0.0_load.png,
> ITBLL2.5B_1.2.7vs2.0.0_memheap.png, ITBLL2.5B_1.2.7vs2.0.0_memstore.png,
> ITBLL2.5B_1.2.7vs2.0.0_ops.png,
> ITBLL2.5B_1.2.7vs2.0.0_ops_NOT_summing_regions.png, YCSB_CPU.png,
> YCSB_GC_TIME.png, YCSB_IN_MEMORY_COMPACTION=NONE.ops.png, YCSB_MEMSTORE.png,
> YCSB_OPs.png, YCSB_in-memory-compaction=NONE.ops.png, YCSB_load.png,
> flamegraph-1072.1.svg, flamegraph-1072.2.svg, hbase-env.sh, hbase-site.xml,
> lock.127.workloadc.20180402T200918Z.svg,
> lock.2.memsize2.c.20180403T160257Z.svg, run_ycsb.sh, tree.txt
>
>
> How does 2.0.0 compare to old versions? Is it faster, slower? There is rumor
> that it is much slower, that the problem is the asyncwal writing. Does
> in-memory compaction slow us down or speed us up? What happens when you
> enable offheaping?
> Keep notes here in this umbrella issue. Need to be able to say something
> about perf when 2.0.0 ships.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20188) [TESTING] Performance
[
https://issues.apache.org/jira/browse/HBASE-20188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16426134#comment-16426134
]
Eshcar Hillel commented on HBASE-20188:
---
This is the code that initiates {{bufferedMutator}} in ycsb:
{code:java}
final TableName tName = TableName.valueOf(table);
this.currentTable = connection.getTable(tName);
if (clientSideBuffering) {
final BufferedMutatorParams p = new BufferedMutatorParams(tName);
p.writeBufferSize(writeBufferSize);
this.bufferedMutator = connection.getBufferedMutator(p);
}
{code}
so need to understand why {{connection.getBufferedMutator(p)}} returns null
> [TESTING] Performance
> -
>
> Key: HBASE-20188
> URL: https://issues.apache.org/jira/browse/HBASE-20188
> Project: HBase
> Issue Type: Umbrella
> Components: Performance
>Reporter: stack
>Assignee: stack
>Priority: Blocker
> Fix For: 2.0.0
>
> Attachments: CAM-CONFIG-V01.patch, HBASE-20188.sh, HBase 2.0
> performance evaluation - Basic vs None_ system settings.pdf,
> ITBLL2.5B_1.2.7vs2.0.0_cpu.png, ITBLL2.5B_1.2.7vs2.0.0_gctime.png,
> ITBLL2.5B_1.2.7vs2.0.0_iops.png, ITBLL2.5B_1.2.7vs2.0.0_load.png,
> ITBLL2.5B_1.2.7vs2.0.0_memheap.png, ITBLL2.5B_1.2.7vs2.0.0_memstore.png,
> ITBLL2.5B_1.2.7vs2.0.0_ops.png,
> ITBLL2.5B_1.2.7vs2.0.0_ops_NOT_summing_regions.png, YCSB_CPU.png,
> YCSB_GC_TIME.png, YCSB_IN_MEMORY_COMPACTION=NONE.ops.png, YCSB_MEMSTORE.png,
> YCSB_OPs.png, YCSB_in-memory-compaction=NONE.ops.png, YCSB_load.png,
> flamegraph-1072.1.svg, flamegraph-1072.2.svg, hbase-env.sh, hbase-site.xml,
> lock.127.workloadc.20180402T200918Z.svg,
> lock.2.memsize2.c.20180403T160257Z.svg, run_ycsb.sh, tree.txt
>
>
> How does 2.0.0 compare to old versions? Is it faster, slower? There is rumor
> that it is much slower, that the problem is the asyncwal writing. Does
> in-memory compaction slow us down or speed us up? What happens when you
> enable offheaping?
> Keep notes here in this umbrella issue. Need to be able to say something
> about perf when 2.0.0 ships.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20188) [TESTING] Performance
[
https://issues.apache.org/jira/browse/HBASE-20188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16426168#comment-16426168
]
Eshcar Hillel commented on HBASE-20188:
---
I am running hbase12 client
This is the code of the client
{code}
package com.yahoo.ycsb.db.hbase12;
/**
* HBase 1.2 client for YCSB framework.
*
* A modified version of HBaseClient (which targets HBase v1.2) utilizing the
* shaded client.
*
* It should run equivalent to following the hbase098 binding README.
*
*/
public class HBaseClient12 extends com.yahoo.ycsb.db.HBaseClient10 {
}
{code}
The difference from hbase10 is just in the pom.xml file I believe, which
includes shaded-client instead of hbase-client - could this be to blame?
{code}
com.yahoo.ycsb
hbase10-binding
${project.version}
org.apache.hbase
hbase-client
...
org.apache.hbase
hbase-shaded-client
${hbase12.version}
{code}
> [TESTING] Performance
> -
>
> Key: HBASE-20188
> URL: https://issues.apache.org/jira/browse/HBASE-20188
> Project: HBase
> Issue Type: Umbrella
> Components: Performance
>Reporter: stack
>Assignee: stack
>Priority: Blocker
> Fix For: 2.0.0
>
> Attachments: CAM-CONFIG-V01.patch, HBASE-20188.sh, HBase 2.0
> performance evaluation - Basic vs None_ system settings.pdf,
> ITBLL2.5B_1.2.7vs2.0.0_cpu.png, ITBLL2.5B_1.2.7vs2.0.0_gctime.png,
> ITBLL2.5B_1.2.7vs2.0.0_iops.png, ITBLL2.5B_1.2.7vs2.0.0_load.png,
> ITBLL2.5B_1.2.7vs2.0.0_memheap.png, ITBLL2.5B_1.2.7vs2.0.0_memstore.png,
> ITBLL2.5B_1.2.7vs2.0.0_ops.png,
> ITBLL2.5B_1.2.7vs2.0.0_ops_NOT_summing_regions.png, YCSB_CPU.png,
> YCSB_GC_TIME.png, YCSB_IN_MEMORY_COMPACTION=NONE.ops.png, YCSB_MEMSTORE.png,
> YCSB_OPs.png, YCSB_in-memory-compaction=NONE.ops.png, YCSB_load.png,
> flamegraph-1072.1.svg, flamegraph-1072.2.svg, hbase-env.sh, hbase-site.xml,
> lock.127.workloadc.20180402T200918Z.svg,
> lock.2.memsize2.c.20180403T160257Z.svg, run_ycsb.sh, tree.txt
>
>
> How does 2.0.0 compare to old versions? Is it faster, slower? There is rumor
> that it is much slower, that the problem is the asyncwal writing. Does
> in-memory compaction slow us down or speed us up? What happens when you
> enable offheaping?
> Keep notes here in this umbrella issue. Need to be able to say something
> about perf when 2.0.0 ships.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20188) [TESTING] Performance
[ https://issues.apache.org/jira/browse/HBASE-20188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-20188: -- Attachment: HBase 2.0 performance evaluation - 8GB.pdf > [TESTING] Performance > - > > Key: HBASE-20188 > URL: https://issues.apache.org/jira/browse/HBASE-20188 > Project: HBase > Issue Type: Umbrella > Components: Performance >Reporter: stack >Assignee: stack >Priority: Blocker > Fix For: 2.0.0 > > Attachments: CAM-CONFIG-V01.patch, HBASE-20188.sh, HBase 2.0 > performance evaluation - 8GB.pdf, HBase 2.0 performance evaluation - Basic vs > None_ system settings.pdf, ITBLL2.5B_1.2.7vs2.0.0_cpu.png, > ITBLL2.5B_1.2.7vs2.0.0_gctime.png, ITBLL2.5B_1.2.7vs2.0.0_iops.png, > ITBLL2.5B_1.2.7vs2.0.0_load.png, ITBLL2.5B_1.2.7vs2.0.0_memheap.png, > ITBLL2.5B_1.2.7vs2.0.0_memstore.png, ITBLL2.5B_1.2.7vs2.0.0_ops.png, > ITBLL2.5B_1.2.7vs2.0.0_ops_NOT_summing_regions.png, YCSB_CPU.png, > YCSB_GC_TIME.png, YCSB_IN_MEMORY_COMPACTION=NONE.ops.png, YCSB_MEMSTORE.png, > YCSB_OPs.png, YCSB_in-memory-compaction=NONE.ops.png, YCSB_load.png, > flamegraph-1072.1.svg, flamegraph-1072.2.svg, hbase-env.sh, hbase-site.xml, > lock.127.workloadc.20180402T200918Z.svg, > lock.2.memsize2.c.20180403T160257Z.svg, run_ycsb.sh, tree.txt > > > How does 2.0.0 compare to old versions? Is it faster, slower? There is rumor > that it is much slower, that the problem is the asyncwal writing. Does > in-memory compaction slow us down or speed us up? What happens when you > enable offheaping? > Keep notes here in this umbrella issue. Need to be able to say something > about perf when 2.0.0 ships. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20188) [TESTING] Performance
[ https://issues.apache.org/jira/browse/HBASE-20188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-20188: -- Attachment: hbase-site.xml workloadx HBASE-20188-xac.sh > [TESTING] Performance > - > > Key: HBASE-20188 > URL: https://issues.apache.org/jira/browse/HBASE-20188 > Project: HBase > Issue Type: Umbrella > Components: Performance >Reporter: stack >Assignee: stack >Priority: Blocker > Fix For: 2.0.0 > > Attachments: CAM-CONFIG-V01.patch, HBASE-20188-xac.sh, > HBASE-20188.sh, HBase 2.0 performance evaluation - 8GB.pdf, HBase 2.0 > performance evaluation - Basic vs None_ system settings.pdf, > ITBLL2.5B_1.2.7vs2.0.0_cpu.png, ITBLL2.5B_1.2.7vs2.0.0_gctime.png, > ITBLL2.5B_1.2.7vs2.0.0_iops.png, ITBLL2.5B_1.2.7vs2.0.0_load.png, > ITBLL2.5B_1.2.7vs2.0.0_memheap.png, ITBLL2.5B_1.2.7vs2.0.0_memstore.png, > ITBLL2.5B_1.2.7vs2.0.0_ops.png, > ITBLL2.5B_1.2.7vs2.0.0_ops_NOT_summing_regions.png, YCSB_CPU.png, > YCSB_GC_TIME.png, YCSB_IN_MEMORY_COMPACTION=NONE.ops.png, YCSB_MEMSTORE.png, > YCSB_OPs.png, YCSB_in-memory-compaction=NONE.ops.png, YCSB_load.png, > flamegraph-1072.1.svg, flamegraph-1072.2.svg, hbase-env.sh, hbase-site.xml, > hbase-site.xml, lock.127.workloadc.20180402T200918Z.svg, > lock.2.memsize2.c.20180403T160257Z.svg, run_ycsb.sh, tree.txt, workloadx > > > How does 2.0.0 compare to old versions? Is it faster, slower? There is rumor > that it is much slower, that the problem is the asyncwal writing. Does > in-memory compaction slow us down or speed us up? What happens when you > enable offheaping? > Keep notes here in this umbrella issue. Need to be able to say something > about perf when 2.0.0 ships. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20188) [TESTING] Performance
[ https://issues.apache.org/jira/browse/HBASE-20188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16431148#comment-16431148 ] Eshcar Hillel commented on HBASE-20188: --- I attached the results of some additional experiments we ran with 8GB heap. We created a new workload [^workloadx]. It is a write-only skewed distribution, with client side batching. Results show that IMC with 2% active and 2 pipeline segments has an advantage over none in write-only workloads. Script to run the experiments is here [^HBASE-20188-xac.sh], and relevant hbase-site configuration here [^hbase-site.xml] we would still like to investigate the read latency with IMC we have some direction we plan to explore; will come back with results. > [TESTING] Performance > - > > Key: HBASE-20188 > URL: https://issues.apache.org/jira/browse/HBASE-20188 > Project: HBase > Issue Type: Umbrella > Components: Performance >Reporter: stack >Assignee: stack >Priority: Blocker > Fix For: 2.0.0 > > Attachments: CAM-CONFIG-V01.patch, HBASE-20188-xac.sh, > HBASE-20188.sh, HBase 2.0 performance evaluation - 8GB.pdf, HBase 2.0 > performance evaluation - Basic vs None_ system settings.pdf, > ITBLL2.5B_1.2.7vs2.0.0_cpu.png, ITBLL2.5B_1.2.7vs2.0.0_gctime.png, > ITBLL2.5B_1.2.7vs2.0.0_iops.png, ITBLL2.5B_1.2.7vs2.0.0_load.png, > ITBLL2.5B_1.2.7vs2.0.0_memheap.png, ITBLL2.5B_1.2.7vs2.0.0_memstore.png, > ITBLL2.5B_1.2.7vs2.0.0_ops.png, > ITBLL2.5B_1.2.7vs2.0.0_ops_NOT_summing_regions.png, YCSB_CPU.png, > YCSB_GC_TIME.png, YCSB_IN_MEMORY_COMPACTION=NONE.ops.png, YCSB_MEMSTORE.png, > YCSB_OPs.png, YCSB_in-memory-compaction=NONE.ops.png, YCSB_load.png, > flamegraph-1072.1.svg, flamegraph-1072.2.svg, hbase-env.sh, hbase-site.xml, > hbase-site.xml, lock.127.workloadc.20180402T200918Z.svg, > lock.2.memsize2.c.20180403T160257Z.svg, run_ycsb.sh, tree.txt, workloadx > > > How does 2.0.0 compare to old versions? Is it faster, slower? There is rumor > that it is much slower, that the problem is the asyncwal writing. Does > in-memory compaction slow us down or speed us up? What happens when you > enable offheaping? > Keep notes here in this umbrella issue. Need to be able to say something > about perf when 2.0.0 ships. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20188) [TESTING] Performance
[ https://issues.apache.org/jira/browse/HBASE-20188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-20188: -- Attachment: HBase 2.0 performance evaluation - 8GB(1).pdf > [TESTING] Performance > - > > Key: HBASE-20188 > URL: https://issues.apache.org/jira/browse/HBASE-20188 > Project: HBase > Issue Type: Umbrella > Components: Performance >Reporter: stack >Assignee: stack >Priority: Blocker > Fix For: 2.0.0 > > Attachments: CAM-CONFIG-V01.patch, HBASE-20188-xac.sh, > HBASE-20188.sh, HBase 2.0 performance evaluation - 8GB(1).pdf, HBase 2.0 > performance evaluation - 8GB.pdf, HBase 2.0 performance evaluation - Basic vs > None_ system settings.pdf, ITBLL2.5B_1.2.7vs2.0.0_cpu.png, > ITBLL2.5B_1.2.7vs2.0.0_gctime.png, ITBLL2.5B_1.2.7vs2.0.0_iops.png, > ITBLL2.5B_1.2.7vs2.0.0_load.png, ITBLL2.5B_1.2.7vs2.0.0_memheap.png, > ITBLL2.5B_1.2.7vs2.0.0_memstore.png, ITBLL2.5B_1.2.7vs2.0.0_ops.png, > ITBLL2.5B_1.2.7vs2.0.0_ops_NOT_summing_regions.png, YCSB_CPU.png, > YCSB_GC_TIME.png, YCSB_IN_MEMORY_COMPACTION=NONE.ops.png, YCSB_MEMSTORE.png, > YCSB_OPs.png, YCSB_in-memory-compaction=NONE.ops.png, YCSB_load.png, > flamegraph-1072.1.svg, flamegraph-1072.2.svg, hbase-env.sh, hbase-site.xml, > hbase-site.xml, lock.127.workloadc.20180402T200918Z.svg, > lock.2.memsize2.c.20180403T160257Z.svg, run_ycsb.sh, tree.txt, workloadx > > > How does 2.0.0 compare to old versions? Is it faster, slower? There is rumor > that it is much slower, that the problem is the asyncwal writing. Does > in-memory compaction slow us down or speed us up? What happens when you > enable offheaping? > Keep notes here in this umbrella issue. Need to be able to say something > about perf when 2.0.0 ships. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20188) [TESTING] Performance
[ https://issues.apache.org/jira/browse/HBASE-20188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16432783#comment-16432783 ] Eshcar Hillel commented on HBASE-20188: --- Attaching additional benchmark results where we update and read single column rows [^HBase 2.0 performance evaluation - 8GB(1).pdf] In workloadx (write-only) with a single (wider) column -- Adaptive outperforms None by 15%. In workloads a and c with a single wide column – Adaptive and None are comparable. > [TESTING] Performance > - > > Key: HBASE-20188 > URL: https://issues.apache.org/jira/browse/HBASE-20188 > Project: HBase > Issue Type: Umbrella > Components: Performance >Reporter: stack >Assignee: stack >Priority: Blocker > Fix For: 2.0.0 > > Attachments: CAM-CONFIG-V01.patch, HBASE-20188-xac.sh, > HBASE-20188.sh, HBase 2.0 performance evaluation - 8GB(1).pdf, HBase 2.0 > performance evaluation - 8GB.pdf, HBase 2.0 performance evaluation - Basic vs > None_ system settings.pdf, ITBLL2.5B_1.2.7vs2.0.0_cpu.png, > ITBLL2.5B_1.2.7vs2.0.0_gctime.png, ITBLL2.5B_1.2.7vs2.0.0_iops.png, > ITBLL2.5B_1.2.7vs2.0.0_load.png, ITBLL2.5B_1.2.7vs2.0.0_memheap.png, > ITBLL2.5B_1.2.7vs2.0.0_memstore.png, ITBLL2.5B_1.2.7vs2.0.0_ops.png, > ITBLL2.5B_1.2.7vs2.0.0_ops_NOT_summing_regions.png, YCSB_CPU.png, > YCSB_GC_TIME.png, YCSB_IN_MEMORY_COMPACTION=NONE.ops.png, YCSB_MEMSTORE.png, > YCSB_OPs.png, YCSB_in-memory-compaction=NONE.ops.png, YCSB_load.png, > flamegraph-1072.1.svg, flamegraph-1072.2.svg, hbase-env.sh, hbase-site.xml, > hbase-site.xml, lock.127.workloadc.20180402T200918Z.svg, > lock.2.memsize2.c.20180403T160257Z.svg, run_ycsb.sh, tree.txt, workloadx > > > How does 2.0.0 compare to old versions? Is it faster, slower? There is rumor > that it is much slower, that the problem is the asyncwal writing. Does > in-memory compaction slow us down or speed us up? What happens when you > enable offheaping? > Keep notes here in this umbrella issue. Need to be able to say something > about perf when 2.0.0 ships. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20188) [TESTING] Performance
[ https://issues.apache.org/jira/browse/HBASE-20188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-20188: -- Attachment: workloadx > [TESTING] Performance > - > > Key: HBASE-20188 > URL: https://issues.apache.org/jira/browse/HBASE-20188 > Project: HBase > Issue Type: Umbrella > Components: Performance >Reporter: stack >Assignee: stack >Priority: Blocker > Fix For: 2.0.0 > > Attachments: CAM-CONFIG-V01.patch, HBASE-20188-xac.sh, > HBASE-20188.sh, HBase 2.0 performance evaluation - 8GB(1).pdf, HBase 2.0 > performance evaluation - 8GB.pdf, HBase 2.0 performance evaluation - Basic vs > None_ system settings.pdf, ITBLL2.5B_1.2.7vs2.0.0_cpu.png, > ITBLL2.5B_1.2.7vs2.0.0_gctime.png, ITBLL2.5B_1.2.7vs2.0.0_iops.png, > ITBLL2.5B_1.2.7vs2.0.0_load.png, ITBLL2.5B_1.2.7vs2.0.0_memheap.png, > ITBLL2.5B_1.2.7vs2.0.0_memstore.png, ITBLL2.5B_1.2.7vs2.0.0_ops.png, > ITBLL2.5B_1.2.7vs2.0.0_ops_NOT_summing_regions.png, YCSB_CPU.png, > YCSB_GC_TIME.png, YCSB_IN_MEMORY_COMPACTION=NONE.ops.png, YCSB_MEMSTORE.png, > YCSB_OPs.png, YCSB_in-memory-compaction=NONE.ops.png, YCSB_load.png, > flamegraph-1072.1.svg, flamegraph-1072.2.svg, hbase-env.sh, hbase-site.xml, > hbase-site.xml, lock.127.workloadc.20180402T200918Z.svg, > lock.2.memsize2.c.20180403T160257Z.svg, run_ycsb.sh, tree.txt, workloadx, > workloadx > > > How does 2.0.0 compare to old versions? Is it faster, slower? There is rumor > that it is much slower, that the problem is the asyncwal writing. Does > in-memory compaction slow us down or speed us up? What happens when you > enable offheaping? > Keep notes here in this umbrella issue. Need to be able to say something > about perf when 2.0.0 ships. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20188) [TESTING] Performance
[ https://issues.apache.org/jira/browse/HBASE-20188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16433700#comment-16433700 ] Eshcar Hillel commented on HBASE-20188: --- Just to summarize the results again -- We see that in write workloads IMC improves performance; it delays flush to disk and hence reduces number of disk compaction. When values are small IMC reduces memory occupancy by reducing metadata size (regardless of workload distribution), when the distribution is skewed IMC reduces memory occupancy by eliminating data duplication (regardless of value size), when the values are big and workload is uniform IMC doesn't help. For reads IMC is either comparable or slightly worse than None (no in-memory compaction). In addition measures we did in past experiments show that IMC reduces write amplification, again due to reducing number of disk compaction. I am opening a new Jira to change the default to the parameters that showed best performance in the recent benchmarks. Namely, IMC policy = ADAPTIVE, active segment porition = 0.02, limit on number of segments in pipeline = 2. We are continuing with our experiments to see if any additional changes can help improve the performance. > [TESTING] Performance > - > > Key: HBASE-20188 > URL: https://issues.apache.org/jira/browse/HBASE-20188 > Project: HBase > Issue Type: Umbrella > Components: Performance >Reporter: stack >Assignee: stack >Priority: Blocker > Fix For: 2.0.0 > > Attachments: CAM-CONFIG-V01.patch, HBASE-20188-xac.sh, > HBASE-20188.sh, HBase 2.0 performance evaluation - 8GB(1).pdf, HBase 2.0 > performance evaluation - 8GB.pdf, HBase 2.0 performance evaluation - Basic vs > None_ system settings.pdf, ITBLL2.5B_1.2.7vs2.0.0_cpu.png, > ITBLL2.5B_1.2.7vs2.0.0_gctime.png, ITBLL2.5B_1.2.7vs2.0.0_iops.png, > ITBLL2.5B_1.2.7vs2.0.0_load.png, ITBLL2.5B_1.2.7vs2.0.0_memheap.png, > ITBLL2.5B_1.2.7vs2.0.0_memstore.png, ITBLL2.5B_1.2.7vs2.0.0_ops.png, > ITBLL2.5B_1.2.7vs2.0.0_ops_NOT_summing_regions.png, YCSB_CPU.png, > YCSB_GC_TIME.png, YCSB_IN_MEMORY_COMPACTION=NONE.ops.png, YCSB_MEMSTORE.png, > YCSB_OPs.png, YCSB_in-memory-compaction=NONE.ops.png, YCSB_load.png, > flamegraph-1072.1.svg, flamegraph-1072.2.svg, hbase-env.sh, hbase-site.xml, > hbase-site.xml, lock.127.workloadc.20180402T200918Z.svg, > lock.2.memsize2.c.20180403T160257Z.svg, run_ycsb.sh, tree.txt, workloadx, > workloadx > > > How does 2.0.0 compare to old versions? Is it faster, slower? There is rumor > that it is much slower, that the problem is the asyncwal writing. Does > in-memory compaction slow us down or speed us up? What happens when you > enable offheaping? > Keep notes here in this umbrella issue. Need to be able to say something > about perf when 2.0.0 ships. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20188) [TESTING] Performance
[ https://issues.apache.org/jira/browse/HBASE-20188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16433704#comment-16433704 ] Eshcar Hillel commented on HBASE-20188: --- re-attaching [^workloadx] from the most recent experiment > [TESTING] Performance > - > > Key: HBASE-20188 > URL: https://issues.apache.org/jira/browse/HBASE-20188 > Project: HBase > Issue Type: Umbrella > Components: Performance >Reporter: stack >Assignee: stack >Priority: Blocker > Fix For: 2.0.0 > > Attachments: CAM-CONFIG-V01.patch, HBASE-20188-xac.sh, > HBASE-20188.sh, HBase 2.0 performance evaluation - 8GB(1).pdf, HBase 2.0 > performance evaluation - 8GB.pdf, HBase 2.0 performance evaluation - Basic vs > None_ system settings.pdf, ITBLL2.5B_1.2.7vs2.0.0_cpu.png, > ITBLL2.5B_1.2.7vs2.0.0_gctime.png, ITBLL2.5B_1.2.7vs2.0.0_iops.png, > ITBLL2.5B_1.2.7vs2.0.0_load.png, ITBLL2.5B_1.2.7vs2.0.0_memheap.png, > ITBLL2.5B_1.2.7vs2.0.0_memstore.png, ITBLL2.5B_1.2.7vs2.0.0_ops.png, > ITBLL2.5B_1.2.7vs2.0.0_ops_NOT_summing_regions.png, YCSB_CPU.png, > YCSB_GC_TIME.png, YCSB_IN_MEMORY_COMPACTION=NONE.ops.png, YCSB_MEMSTORE.png, > YCSB_OPs.png, YCSB_in-memory-compaction=NONE.ops.png, YCSB_load.png, > flamegraph-1072.1.svg, flamegraph-1072.2.svg, hbase-env.sh, hbase-site.xml, > hbase-site.xml, lock.127.workloadc.20180402T200918Z.svg, > lock.2.memsize2.c.20180403T160257Z.svg, run_ycsb.sh, tree.txt, workloadx, > workloadx > > > How does 2.0.0 compare to old versions? Is it faster, slower? There is rumor > that it is much slower, that the problem is the asyncwal writing. Does > in-memory compaction slow us down or speed us up? What happens when you > enable offheaping? > Keep notes here in this umbrella issue. Need to be able to say something > about perf when 2.0.0 ships. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20188) [TESTING] Performance
[ https://issues.apache.org/jira/browse/HBASE-20188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16433710#comment-16433710 ] Eshcar Hillel commented on HBASE-20188: --- And another thing, might be just a distraction but we think it is worth mentioning – Backward scans should run faster with IMC since the flat segments are simply arrays and it is very simple to go to prev in O(1), while in skip list prev is implemented by seeking the prev key which is done in O(log n). We don't have an easy way to verify this since YCSB does not support testing this kind of operation, but we are looking into it and will report if something interesting comes up. > [TESTING] Performance > - > > Key: HBASE-20188 > URL: https://issues.apache.org/jira/browse/HBASE-20188 > Project: HBase > Issue Type: Umbrella > Components: Performance >Reporter: stack >Assignee: stack >Priority: Blocker > Fix For: 2.0.0 > > Attachments: CAM-CONFIG-V01.patch, HBASE-20188-xac.sh, > HBASE-20188.sh, HBase 2.0 performance evaluation - 8GB(1).pdf, HBase 2.0 > performance evaluation - 8GB.pdf, HBase 2.0 performance evaluation - Basic vs > None_ system settings.pdf, ITBLL2.5B_1.2.7vs2.0.0_cpu.png, > ITBLL2.5B_1.2.7vs2.0.0_gctime.png, ITBLL2.5B_1.2.7vs2.0.0_iops.png, > ITBLL2.5B_1.2.7vs2.0.0_load.png, ITBLL2.5B_1.2.7vs2.0.0_memheap.png, > ITBLL2.5B_1.2.7vs2.0.0_memstore.png, ITBLL2.5B_1.2.7vs2.0.0_ops.png, > ITBLL2.5B_1.2.7vs2.0.0_ops_NOT_summing_regions.png, YCSB_CPU.png, > YCSB_GC_TIME.png, YCSB_IN_MEMORY_COMPACTION=NONE.ops.png, YCSB_MEMSTORE.png, > YCSB_OPs.png, YCSB_in-memory-compaction=NONE.ops.png, YCSB_load.png, > flamegraph-1072.1.svg, flamegraph-1072.2.svg, hbase-env.sh, hbase-site.xml, > hbase-site.xml, lock.127.workloadc.20180402T200918Z.svg, > lock.2.memsize2.c.20180403T160257Z.svg, run_ycsb.sh, tree.txt, workloadx, > workloadx > > > How does 2.0.0 compare to old versions? Is it faster, slower? There is rumor > that it is much slower, that the problem is the asyncwal writing. Does > in-memory compaction slow us down or speed us up? What happens when you > enable offheaping? > Keep notes here in this umbrella issue. Need to be able to say something > about perf when 2.0.0 ships. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20188) [TESTING] Performance
[ https://issues.apache.org/jira/browse/HBASE-20188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16434384#comment-16434384 ] Eshcar Hillel commented on HBASE-20188: --- vq. I should add workloadx to the suite of tests I'm trying? Yes please do. In my script [^HBASE-20188-xac.sh] I run it after (pre)splitting into 10 regions. Opening a sub-jira to update defaults. > [TESTING] Performance > - > > Key: HBASE-20188 > URL: https://issues.apache.org/jira/browse/HBASE-20188 > Project: HBase > Issue Type: Umbrella > Components: Performance >Reporter: stack >Assignee: stack >Priority: Blocker > Fix For: 2.0.0 > > Attachments: CAM-CONFIG-V01.patch, HBASE-20188-xac.sh, > HBASE-20188.sh, HBase 2.0 performance evaluation - 8GB(1).pdf, HBase 2.0 > performance evaluation - 8GB.pdf, HBase 2.0 performance evaluation - Basic vs > None_ system settings.pdf, ITBLL2.5B_1.2.7vs2.0.0_cpu.png, > ITBLL2.5B_1.2.7vs2.0.0_gctime.png, ITBLL2.5B_1.2.7vs2.0.0_iops.png, > ITBLL2.5B_1.2.7vs2.0.0_load.png, ITBLL2.5B_1.2.7vs2.0.0_memheap.png, > ITBLL2.5B_1.2.7vs2.0.0_memstore.png, ITBLL2.5B_1.2.7vs2.0.0_ops.png, > ITBLL2.5B_1.2.7vs2.0.0_ops_NOT_summing_regions.png, YCSB_CPU.png, > YCSB_GC_TIME.png, YCSB_IN_MEMORY_COMPACTION=NONE.ops.png, YCSB_MEMSTORE.png, > YCSB_OPs.png, YCSB_in-memory-compaction=NONE.ops.png, YCSB_load.png, > flamegraph-1072.1.svg, flamegraph-1072.2.svg, hbase-env.sh, hbase-site.xml, > hbase-site.xml, lock.127.workloadc.20180402T200918Z.svg, > lock.2.memsize2.c.20180403T160257Z.svg, run_ycsb.sh, tree.txt, workloadx, > workloadx > > > How does 2.0.0 compare to old versions? Is it faster, slower? There is rumor > that it is much slower, that the problem is the asyncwal writing. Does > in-memory compaction slow us down or speed us up? What happens when you > enable offheaping? > Keep notes here in this umbrella issue. Need to be able to say something > about perf when 2.0.0 ships. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-20390) IMC Default Parameters for 2.0.0
Eshcar Hillel created HBASE-20390: - Summary: IMC Default Parameters for 2.0.0 Key: HBASE-20390 URL: https://issues.apache.org/jira/browse/HBASE-20390 Project: HBase Issue Type: Task Reporter: Eshcar Hillel Setting new default parameters for in-memory compaction based on performance tests done in HBASE-20188 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20390) IMC Default Parameters for 2.0.0
[ https://issues.apache.org/jira/browse/HBASE-20390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-20390: -- Issue Type: Sub-task (was: Task) Parent: HBASE-20188 > IMC Default Parameters for 2.0.0 > > > Key: HBASE-20390 > URL: https://issues.apache.org/jira/browse/HBASE-20390 > Project: HBase > Issue Type: Sub-task >Reporter: Eshcar Hillel >Priority: Major > > Setting new default parameters for in-memory compaction based on performance > tests done in HBASE-20188 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HBASE-20390) IMC Default Parameters for 2.0.0
[ https://issues.apache.org/jira/browse/HBASE-20390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel reassigned HBASE-20390: - Assignee: Eshcar Hillel > IMC Default Parameters for 2.0.0 > > > Key: HBASE-20390 > URL: https://issues.apache.org/jira/browse/HBASE-20390 > Project: HBase > Issue Type: Sub-task >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel >Priority: Major > > Setting new default parameters for in-memory compaction based on performance > tests done in HBASE-20188 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-20188) [TESTING] Performance
[ https://issues.apache.org/jira/browse/HBASE-20188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16434384#comment-16434384 ] Eshcar Hillel edited comment on HBASE-20188 at 4/12/18 9:25 AM: bq. I should add workloadx to the suite of tests I'm trying? Yes please do. In my script [^HBASE-20188-xac.sh] I run it after (pre)splitting into 10 regions. Opening a sub-jira to update defaults. was (Author: eshcar): vq. I should add workloadx to the suite of tests I'm trying? Yes please do. In my script [^HBASE-20188-xac.sh] I run it after (pre)splitting into 10 regions. Opening a sub-jira to update defaults. > [TESTING] Performance > - > > Key: HBASE-20188 > URL: https://issues.apache.org/jira/browse/HBASE-20188 > Project: HBase > Issue Type: Umbrella > Components: Performance >Reporter: stack >Assignee: stack >Priority: Blocker > Fix For: 2.0.0 > > Attachments: CAM-CONFIG-V01.patch, HBASE-20188-xac.sh, > HBASE-20188.sh, HBase 2.0 performance evaluation - 8GB(1).pdf, HBase 2.0 > performance evaluation - 8GB.pdf, HBase 2.0 performance evaluation - Basic vs > None_ system settings.pdf, ITBLL2.5B_1.2.7vs2.0.0_cpu.png, > ITBLL2.5B_1.2.7vs2.0.0_gctime.png, ITBLL2.5B_1.2.7vs2.0.0_iops.png, > ITBLL2.5B_1.2.7vs2.0.0_load.png, ITBLL2.5B_1.2.7vs2.0.0_memheap.png, > ITBLL2.5B_1.2.7vs2.0.0_memstore.png, ITBLL2.5B_1.2.7vs2.0.0_ops.png, > ITBLL2.5B_1.2.7vs2.0.0_ops_NOT_summing_regions.png, YCSB_CPU.png, > YCSB_GC_TIME.png, YCSB_IN_MEMORY_COMPACTION=NONE.ops.png, YCSB_MEMSTORE.png, > YCSB_OPs.png, YCSB_in-memory-compaction=NONE.ops.png, YCSB_load.png, > flamegraph-1072.1.svg, flamegraph-1072.2.svg, hbase-env.sh, hbase-site.xml, > hbase-site.xml, lock.127.workloadc.20180402T200918Z.svg, > lock.2.memsize2.c.20180403T160257Z.svg, run_ycsb.sh, tree.txt, workloadx, > workloadx > > > How does 2.0.0 compare to old versions? Is it faster, slower? There is rumor > that it is much slower, that the problem is the asyncwal writing. Does > in-memory compaction slow us down or speed us up? What happens when you > enable offheaping? > Keep notes here in this umbrella issue. Need to be able to say something > about perf when 2.0.0 ships. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20188) [TESTING] Performance
[ https://issues.apache.org/jira/browse/HBASE-20188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16435236#comment-16435236 ] Eshcar Hillel commented on HBASE-20188: --- We also ran the same experiments in HDD machines and indeed we see that in HDD the lift of IMC over None is smaller. We now repeat all experiments again to eliminate noisy results. We will post a summary of both SSD and HDD results by EoD tomorrow. > [TESTING] Performance > - > > Key: HBASE-20188 > URL: https://issues.apache.org/jira/browse/HBASE-20188 > Project: HBase > Issue Type: Umbrella > Components: Performance >Reporter: stack >Assignee: stack >Priority: Blocker > Fix For: 2.0.0 > > Attachments: CAM-CONFIG-V01.patch, HBASE-20188-xac.sh, > HBASE-20188.sh, HBase 2.0 performance evaluation - 8GB(1).pdf, HBase 2.0 > performance evaluation - 8GB.pdf, HBase 2.0 performance evaluation - Basic vs > None_ system settings.pdf, ITBLL2.5B_1.2.7vs2.0.0_cpu.png, > ITBLL2.5B_1.2.7vs2.0.0_gctime.png, ITBLL2.5B_1.2.7vs2.0.0_iops.png, > ITBLL2.5B_1.2.7vs2.0.0_load.png, ITBLL2.5B_1.2.7vs2.0.0_memheap.png, > ITBLL2.5B_1.2.7vs2.0.0_memstore.png, ITBLL2.5B_1.2.7vs2.0.0_ops.png, > ITBLL2.5B_1.2.7vs2.0.0_ops_NOT_summing_regions.png, YCSB_CPU.png, > YCSB_GC_TIME.png, YCSB_IN_MEMORY_COMPACTION=NONE.ops.png, YCSB_MEMSTORE.png, > YCSB_OPs.png, YCSB_in-memory-compaction=NONE.ops.png, YCSB_load.png, > flamegraph-1072.1.svg, flamegraph-1072.2.svg, hbase-env.sh, hbase-site.xml, > hbase-site.xml, lock.127.workloadc.20180402T200918Z.svg, > lock.2.memsize2.c.20180403T160257Z.svg, run_ycsb.sh, tree.txt, workloadx, > workloadx > > > How does 2.0.0 compare to old versions? Is it faster, slower? There is rumor > that it is much slower, that the problem is the asyncwal writing. Does > in-memory compaction slow us down or speed us up? What happens when you > enable offheaping? > Keep notes here in this umbrella issue. Need to be able to say something > about perf when 2.0.0 ships. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20188) [TESTING] Performance
[ https://issues.apache.org/jira/browse/HBASE-20188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16437794#comment-16437794 ] Eshcar Hillel commented on HBASE-20188: --- [~anoop.hbase] can you post the command line or script + settings you are using so we can re-produce these results? > [TESTING] Performance > - > > Key: HBASE-20188 > URL: https://issues.apache.org/jira/browse/HBASE-20188 > Project: HBase > Issue Type: Umbrella > Components: Performance >Reporter: stack >Assignee: stack >Priority: Blocker > Fix For: 2.0.0 > > Attachments: CAM-CONFIG-V01.patch, HBASE-20188-xac.sh, > HBASE-20188.sh, HBase 2.0 performance evaluation - 8GB(1).pdf, HBase 2.0 > performance evaluation - 8GB.pdf, HBase 2.0 performance evaluation - Basic vs > None_ system settings.pdf, ITBLL2.5B_1.2.7vs2.0.0_cpu.png, > ITBLL2.5B_1.2.7vs2.0.0_gctime.png, ITBLL2.5B_1.2.7vs2.0.0_iops.png, > ITBLL2.5B_1.2.7vs2.0.0_load.png, ITBLL2.5B_1.2.7vs2.0.0_memheap.png, > ITBLL2.5B_1.2.7vs2.0.0_memstore.png, ITBLL2.5B_1.2.7vs2.0.0_ops.png, > ITBLL2.5B_1.2.7vs2.0.0_ops_NOT_summing_regions.png, YCSB_CPU.png, > YCSB_GC_TIME.png, YCSB_IN_MEMORY_COMPACTION=NONE.ops.png, YCSB_MEMSTORE.png, > YCSB_OPs.png, YCSB_in-memory-compaction=NONE.ops.png, YCSB_load.png, > flamegraph-1072.1.svg, flamegraph-1072.2.svg, hbase-env.sh, hbase-site.xml, > hbase-site.xml, lock.127.workloadc.20180402T200918Z.svg, > lock.2.memsize2.c.20180403T160257Z.svg, run_ycsb.sh, tree.txt, workloadx, > workloadx > > > How does 2.0.0 compare to old versions? Is it faster, slower? There is rumor > that it is much slower, that the problem is the asyncwal writing. Does > in-memory compaction slow us down or speed us up? What happens when you > enable offheaping? > Keep notes here in this umbrella issue. Need to be able to say something > about perf when 2.0.0 ships. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20390) IMC Default Parameters for 2.0.0
[ https://issues.apache.org/jira/browse/HBASE-20390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-20390: -- Attachment: HBASE-20390-branch-2.0-01.patch > IMC Default Parameters for 2.0.0 > > > Key: HBASE-20390 > URL: https://issues.apache.org/jira/browse/HBASE-20390 > Project: HBase > Issue Type: Sub-task >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel >Priority: Major > Attachments: HBASE-20390-branch-2.0-01.patch > > > Setting new default parameters for in-memory compaction based on performance > tests done in HBASE-20188 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20390) IMC Default Parameters for 2.0.0
[ https://issues.apache.org/jira/browse/HBASE-20390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-20390: -- Release Note: Changing in-memory compaction (IMC) defaults: (1) active segment is 2% of entire memstore (hbase.memstore.inmemoryflush.threshold.factor), (2) number of segments in compaction pipeline is 2 (hbase.hregion.compacting.pipeline.segments.limit) Status: Patch Available (was: Open) > IMC Default Parameters for 2.0.0 > > > Key: HBASE-20390 > URL: https://issues.apache.org/jira/browse/HBASE-20390 > Project: HBase > Issue Type: Sub-task >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel >Priority: Major > Attachments: HBASE-20390-branch-2.0-01.patch > > > Setting new default parameters for in-memory compaction based on performance > tests done in HBASE-20188 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20411) Ameliorate MutableSegment synchronize
[ https://issues.apache.org/jira/browse/HBASE-20411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16439140#comment-16439140 ] Eshcar Hillel commented on HBASE-20411: --- I briefly went over the code in RB. Did you had a chance to check if this patch boosts performance? > Ameliorate MutableSegment synchronize > - > > Key: HBASE-20411 > URL: https://issues.apache.org/jira/browse/HBASE-20411 > Project: HBase > Issue Type: Bug >Reporter: stack >Priority: Major > Attachments: 2.load.patched.17704.lock.svg, > 2.load.patched.2.17704.lock.svg, 41901.lock.svg, > HBASE-20411.branch-2.0.001.patch, HBASE-20411.branch-2.0.002.patch, > HBASE-20411.branch-2.0.003.patch > > > This item is migrated from HBASE-20236 so it gets dedicated issue. > Let me upload evidence that has this synchronize as a stake in our write-time > perf. I'll migrate the patch I posted with updates that come of comments > posted by [~mdrob] on the HBASE-20236 issue. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20390) IMC Default Parameters for 2.0.0
[ https://issues.apache.org/jira/browse/HBASE-20390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-20390: -- Attachment: HBASE-20390.branch-2.0.002.patch > IMC Default Parameters for 2.0.0 > > > Key: HBASE-20390 > URL: https://issues.apache.org/jira/browse/HBASE-20390 > Project: HBase > Issue Type: Sub-task >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel >Priority: Major > Attachments: HBASE-20390-branch-2.0-01.patch, > HBASE-20390-branch-2.0-01.patch, HBASE-20390.branch-2.0.002.patch > > > Setting new default parameters for in-memory compaction based on performance > tests done in HBASE-20188 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20188) [TESTING] Performance
[ https://issues.apache.org/jira/browse/HBASE-20188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16440582#comment-16440582 ] Eshcar Hillel commented on HBASE-20188: --- [~anoop.hbase] can you share the exact command line you are running. Can you report the throughput numbers for 1.4.2 and for 2.0.0. Thanks. > [TESTING] Performance > - > > Key: HBASE-20188 > URL: https://issues.apache.org/jira/browse/HBASE-20188 > Project: HBase > Issue Type: Umbrella > Components: Performance >Reporter: stack >Assignee: stack >Priority: Blocker > Fix For: 2.0.0 > > Attachments: CAM-CONFIG-V01.patch, HBASE-20188-xac.sh, > HBASE-20188.sh, HBase 2.0 performance evaluation - 8GB(1).pdf, HBase 2.0 > performance evaluation - 8GB.pdf, HBase 2.0 performance evaluation - Basic vs > None_ system settings.pdf, ITBLL2.5B_1.2.7vs2.0.0_cpu.png, > ITBLL2.5B_1.2.7vs2.0.0_gctime.png, ITBLL2.5B_1.2.7vs2.0.0_iops.png, > ITBLL2.5B_1.2.7vs2.0.0_load.png, ITBLL2.5B_1.2.7vs2.0.0_memheap.png, > ITBLL2.5B_1.2.7vs2.0.0_memstore.png, ITBLL2.5B_1.2.7vs2.0.0_ops.png, > ITBLL2.5B_1.2.7vs2.0.0_ops_NOT_summing_regions.png, YCSB_CPU.png, > YCSB_GC_TIME.png, YCSB_IN_MEMORY_COMPACTION=NONE.ops.png, YCSB_MEMSTORE.png, > YCSB_OPs.png, YCSB_in-memory-compaction=NONE.ops.png, YCSB_load.png, > flamegraph-1072.1.svg, flamegraph-1072.2.svg, hbase-env.sh, hbase-site.xml, > hbase-site.xml, hits.png, lock.127.workloadc.20180402T200918Z.svg, > lock.2.memsize2.c.20180403T160257Z.svg, perregion.png, run_ycsb.sh, > total.png, tree.txt, workloadx, workloadx > > > How does 2.0.0 compare to old versions? Is it faster, slower? There is rumor > that it is much slower, that the problem is the asyncwal writing. Does > in-memory compaction slow us down or speed us up? What happens when you > enable offheaping? > Keep notes here in this umbrella issue. Need to be able to say something > about perf when 2.0.0 ships. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20455) [IHC] Workloadx
[
https://issues.apache.org/jira/browse/HBASE-20455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16445547#comment-16445547
]
Eshcar Hillel commented on HBASE-20455:
---
In your experiment with workloadx you use buffer size=2MB while the workload
was set to use buffer size=10KB
The values are also smaller so you need to run 100M ops and have 100M records
to write 25GB of data.
You can try using the shell script I posted HBASE-20188-xac.sh -- it pre-splits
the table which makes the experiment less "noisy".
If you want to have a bigger buffer size and run longer experiment you can just
run 500M ops.
> [IHC] Workloadx
> ---
>
> Key: HBASE-20455
> URL: https://issues.apache.org/jira/browse/HBASE-20455
> Project: HBase
> Issue Type: Sub-task
>Reporter: stack
>Priority: Major
> Fix For: 2.0.0
>
>
> I tried workloadx from parent issue, the ycsb workload that is meant to make
> IHC shine. I'm doing something wrong. I tried just plugging it in and
> doing this:
> {code}
> ycsb run hbase12 -P /home/stack/ycsb/workloads/workloadx -p table=ycsb
> -threads 48 -cp /home/stack/conf_hbase -p columnfamily=family -p
> clientbuffering=true -p writebuffersize=2097152 -s -p maxexecutiontime=1200
> -jvm-args=-Xmx8192
> m -Djava.security.egd=file:/dev/./urandom -p recordcount=2500 -p
> operationcount=2500 -p
> exportfile=logs/ycsb-workloadx-measurements-ve0524-20180419T02:58:14Z.json -p
> exporter=com.yahoo.ycsb.measurements.exporter.JSONArrayMeasurementsExporter
> {code}
> It completes in a minute and a half. Claims stuff like this for throughput:
> 236825 ops/second. In this case I have in-memory-compaction set to NONE. I
> was going to compare before and after.
> [~eshcar]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20188) [TESTING] Performance
[ https://issues.apache.org/jira/browse/HBASE-20188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-20188: -- Attachment: HBase 2.0 performance evaluation - throughput SSD_HDD.pdf > [TESTING] Performance > - > > Key: HBASE-20188 > URL: https://issues.apache.org/jira/browse/HBASE-20188 > Project: HBase > Issue Type: Umbrella > Components: Performance >Reporter: stack >Assignee: stack >Priority: Blocker > Fix For: 3.0.0, 2.1.0 > > Attachments: CAM-CONFIG-V01.patch, HBASE-20188-xac.sh, > HBASE-20188.sh, HBase 2.0 performance evaluation - 8GB(1).pdf, HBase 2.0 > performance evaluation - 8GB.pdf, HBase 2.0 performance evaluation - Basic vs > None_ system settings.pdf, HBase 2.0 performance evaluation - throughput > SSD_HDD.pdf, ITBLL2.5B_1.2.7vs2.0.0_cpu.png, > ITBLL2.5B_1.2.7vs2.0.0_gctime.png, ITBLL2.5B_1.2.7vs2.0.0_iops.png, > ITBLL2.5B_1.2.7vs2.0.0_load.png, ITBLL2.5B_1.2.7vs2.0.0_memheap.png, > ITBLL2.5B_1.2.7vs2.0.0_memstore.png, ITBLL2.5B_1.2.7vs2.0.0_ops.png, > ITBLL2.5B_1.2.7vs2.0.0_ops_NOT_summing_regions.png, YCSB_CPU.png, > YCSB_GC_TIME.png, YCSB_IN_MEMORY_COMPACTION=NONE.ops.png, YCSB_MEMSTORE.png, > YCSB_OPs.png, YCSB_in-memory-compaction=NONE.ops.png, YCSB_load.png, > flamegraph-1072.1.svg, flamegraph-1072.2.svg, hbase-env.sh, hbase-site.xml, > hbase-site.xml, hits.png, hits_with_fp_scheduler.png, > lock.127.workloadc.20180402T200918Z.svg, > lock.2.memsize2.c.20180403T160257Z.svg, perregion.png, run_ycsb.sh, > total.png, tree.txt, workloadx, workloadx > > > How does 2.0.0 compare to old versions? Is it faster, slower? There is rumor > that it is much slower, that the problem is the asyncwal writing. Does > in-memory compaction slow us down or speed us up? What happens when you > enable offheaping? > Keep notes here in this umbrella issue. Need to be able to say something > about perf when 2.0.0 ships. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20188) [TESTING] Performance
[ https://issues.apache.org/jira/browse/HBASE-20188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16467229#comment-16467229 ] Eshcar Hillel commented on HBASE-20188: --- Hi, wanted to share some interesting insights and benchmark results. We tried to understand why the benefit of in-memory compaction decreases when MSLABs are used. We finally realized this is due to internal fragmentation that causes under utilization of the memory. For example, setting the active segment threshold to A=0.02 means it stores 0.02*128MB=2.56MB. Each such 2.5MB segment utilizes 2 chunks (spanning *4MB*) which are carried in the compaction pipeline until the data is flushed to disk. Each 2.5MB data taking 4MB space means IMC heap utilization is roughly at 65%. Not ideal. We therefore experimented with A=0.014, namely active segment of size 1.8MB, which fits into a single chunk (leaving some space for overflow etc). Running workloadx+workloada+workloadc show improvement in performance in all these workloads wrt the default parameters of IMC (results are attached [^HBase 2.0 performance evaluation - throughput SSD_HDD.pdf] ). While the new default improves performance we believe there are still cases where an overflow may cause using 2 chunks instead of one. We have and idea how to circumvent this problem. Suggesting to move in two phases: (1) in Jira HBASE-20390 set IMC default parameters to best utilize memory also when using MSLABs. (2) in a new Jira present and implement a solution that avoids the chunk overflow problem. In addition, we are also considering more optimization in HBASE-20480 that potentially reduces overhead of temporary cell objects in while searching in a CCM segment. > [TESTING] Performance > - > > Key: HBASE-20188 > URL: https://issues.apache.org/jira/browse/HBASE-20188 > Project: HBase > Issue Type: Umbrella > Components: Performance >Reporter: stack >Assignee: stack >Priority: Blocker > Fix For: 3.0.0, 2.1.0 > > Attachments: CAM-CONFIG-V01.patch, HBASE-20188-xac.sh, > HBASE-20188.sh, HBase 2.0 performance evaluation - 8GB(1).pdf, HBase 2.0 > performance evaluation - 8GB.pdf, HBase 2.0 performance evaluation - Basic vs > None_ system settings.pdf, HBase 2.0 performance evaluation - throughput > SSD_HDD.pdf, ITBLL2.5B_1.2.7vs2.0.0_cpu.png, > ITBLL2.5B_1.2.7vs2.0.0_gctime.png, ITBLL2.5B_1.2.7vs2.0.0_iops.png, > ITBLL2.5B_1.2.7vs2.0.0_load.png, ITBLL2.5B_1.2.7vs2.0.0_memheap.png, > ITBLL2.5B_1.2.7vs2.0.0_memstore.png, ITBLL2.5B_1.2.7vs2.0.0_ops.png, > ITBLL2.5B_1.2.7vs2.0.0_ops_NOT_summing_regions.png, YCSB_CPU.png, > YCSB_GC_TIME.png, YCSB_IN_MEMORY_COMPACTION=NONE.ops.png, YCSB_MEMSTORE.png, > YCSB_OPs.png, YCSB_in-memory-compaction=NONE.ops.png, YCSB_load.png, > flamegraph-1072.1.svg, flamegraph-1072.2.svg, hbase-env.sh, hbase-site.xml, > hbase-site.xml, hits.png, hits_with_fp_scheduler.png, > lock.127.workloadc.20180402T200918Z.svg, > lock.2.memsize2.c.20180403T160257Z.svg, perregion.png, run_ycsb.sh, > total.png, tree.txt, workloadx, workloadx > > > How does 2.0.0 compare to old versions? Is it faster, slower? There is rumor > that it is much slower, that the problem is the asyncwal writing. Does > in-memory compaction slow us down or speed us up? What happens when you > enable offheaping? > Keep notes here in this umbrella issue. Need to be able to say something > about perf when 2.0.0 ships. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-20542) Better heap utilization for IMC with MSLABs
Eshcar Hillel created HBASE-20542: - Summary: Better heap utilization for IMC with MSLABs Key: HBASE-20542 URL: https://issues.apache.org/jira/browse/HBASE-20542 Project: HBase Issue Type: Task Reporter: Eshcar Hillel Following HBASE-20188 we realized in-memory compaction combined with MSLABs may suffer from heap under-utilization due to internal fragmentation. This jira presents a solution to circumvent this problem. The main idea is to have each update operation check if it will cause overflow in the active segment *before* it is writing the new value (instead of checking the size after the write is completed), and if it is then the active segment is atomically swapped with a new empty segment, and is pushed (full-yet-not-overflowed) to the compaction pipeline. Later on the IMC deamon will run its compaction operation (flatten index/merge indices/data compaction) in the background. Some subtle concurrency issues should be handled with care. We next elaborate on them. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20542) Better heap utilization for IMC with MSLABs
[ https://issues.apache.org/jira/browse/HBASE-20542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-20542: -- Issue Type: Sub-task (was: Task) Parent: HBASE-20188 > Better heap utilization for IMC with MSLABs > --- > > Key: HBASE-20542 > URL: https://issues.apache.org/jira/browse/HBASE-20542 > Project: HBase > Issue Type: Sub-task >Reporter: Eshcar Hillel >Priority: Major > > Following HBASE-20188 we realized in-memory compaction combined with MSLABs > may suffer from heap under-utilization due to internal fragmentation. This > jira presents a solution to circumvent this problem. The main idea is to have > each update operation check if it will cause overflow in the active segment > *before* it is writing the new value (instead of checking the size after the > write is completed), and if it is then the active segment is atomically > swapped with a new empty segment, and is pushed (full-yet-not-overflowed) to > the compaction pipeline. Later on the IMC deamon will run its compaction > operation (flatten index/merge indices/data compaction) in the background. > Some subtle concurrency issues should be handled with care. We next elaborate > on them. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20390) IMC Default Parameters for 2.0.0
[ https://issues.apache.org/jira/browse/HBASE-20390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16467243#comment-16467243 ] Eshcar Hillel commented on HBASE-20390: --- Following HBASE-20188 we realized in-memory compaction combined with MSLABs may suffer from heap under-utilization due to internal fragmentation. For example, setting the active segment threshold to A=0.02 means it stores 0.02*128MB=2.56MB. Each such 2.5MB segment utilizes 2 chunks (spanning *4MB*) which are carried in the compaction pipeline until the data is flushed to disk. Each 2.5MB data taking 4MB space means IMC heap utilization is roughly at 65%. Not ideal. We therefore experimented with A=0.014, namely active segment of size 1.8MB, which fits into a single chunk (leaving some space for overflow etc). Running workloadx+workloada+workloadc show improvement in performance in all these workloads wrt the default parameters of IMC (results are attached). We therefore suggest to set the IMC defaults to A=0.014, S=2. I will make a new patch. This number is obliviously working well with the current default chunk size. In parallel we are working on a new solution in HBASE-20542 to circumvent this problem regardless of the size of chunks, workload distribution or size of data written. > IMC Default Parameters for 2.0.0 > > > Key: HBASE-20390 > URL: https://issues.apache.org/jira/browse/HBASE-20390 > Project: HBase > Issue Type: Sub-task >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel >Priority: Major > Attachments: HBASE-20390-branch-2.0-01.patch, > HBASE-20390-branch-2.0-01.patch, HBASE-20390.branch-2.0.002.patch, HBase 2.0 > performance evaluation - throughput SSD_HDD.pdf, hits.ihc.png > > > Setting new default parameters for in-memory compaction based on performance > tests done in HBASE-20188 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20390) IMC Default Parameters for 2.0.0
[ https://issues.apache.org/jira/browse/HBASE-20390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-20390: -- Attachment: HBase 2.0 performance evaluation - throughput SSD_HDD.pdf > IMC Default Parameters for 2.0.0 > > > Key: HBASE-20390 > URL: https://issues.apache.org/jira/browse/HBASE-20390 > Project: HBase > Issue Type: Sub-task >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel >Priority: Major > Attachments: HBASE-20390-branch-2.0-01.patch, > HBASE-20390-branch-2.0-01.patch, HBASE-20390.branch-2.0.002.patch, HBase 2.0 > performance evaluation - throughput SSD_HDD.pdf, hits.ihc.png > > > Setting new default parameters for in-memory compaction based on performance > tests done in HBASE-20188 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20188) [TESTING] Performance
[ https://issues.apache.org/jira/browse/HBASE-20188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16467272#comment-16467272 ] Eshcar Hillel commented on HBASE-20188: --- Thanks Anoop (: Meanwhile I have opened HBASE-20542 to elaborate on the new solution. We can continue the discussion there. > [TESTING] Performance > - > > Key: HBASE-20188 > URL: https://issues.apache.org/jira/browse/HBASE-20188 > Project: HBase > Issue Type: Umbrella > Components: Performance >Reporter: stack >Assignee: stack >Priority: Blocker > Fix For: 3.0.0, 2.1.0 > > Attachments: CAM-CONFIG-V01.patch, HBASE-20188-xac.sh, > HBASE-20188.sh, HBase 2.0 performance evaluation - 8GB(1).pdf, HBase 2.0 > performance evaluation - 8GB.pdf, HBase 2.0 performance evaluation - Basic vs > None_ system settings.pdf, HBase 2.0 performance evaluation - throughput > SSD_HDD.pdf, ITBLL2.5B_1.2.7vs2.0.0_cpu.png, > ITBLL2.5B_1.2.7vs2.0.0_gctime.png, ITBLL2.5B_1.2.7vs2.0.0_iops.png, > ITBLL2.5B_1.2.7vs2.0.0_load.png, ITBLL2.5B_1.2.7vs2.0.0_memheap.png, > ITBLL2.5B_1.2.7vs2.0.0_memstore.png, ITBLL2.5B_1.2.7vs2.0.0_ops.png, > ITBLL2.5B_1.2.7vs2.0.0_ops_NOT_summing_regions.png, YCSB_CPU.png, > YCSB_GC_TIME.png, YCSB_IN_MEMORY_COMPACTION=NONE.ops.png, YCSB_MEMSTORE.png, > YCSB_OPs.png, YCSB_in-memory-compaction=NONE.ops.png, YCSB_load.png, > flamegraph-1072.1.svg, flamegraph-1072.2.svg, hbase-env.sh, hbase-site.xml, > hbase-site.xml, hits.png, hits_with_fp_scheduler.png, > lock.127.workloadc.20180402T200918Z.svg, > lock.2.memsize2.c.20180403T160257Z.svg, perregion.png, run_ycsb.sh, > total.png, tree.txt, workloadx, workloadx > > > How does 2.0.0 compare to old versions? Is it faster, slower? There is rumor > that it is much slower, that the problem is the asyncwal writing. Does > in-memory compaction slow us down or speed us up? What happens when you > enable offheaping? > Keep notes here in this umbrella issue. Need to be able to say something > about perf when 2.0.0 ships. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20390) IMC Default Parameters for 2.0.0
[ https://issues.apache.org/jira/browse/HBASE-20390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-20390: -- Release Note: Changing in-memory compaction (IMC) defaults: (1) active segment is 1.4% of entire memstore (hbase.memstore.inmemoryflush.threshold.factor), (2) number of segments in compaction pipeline is 2 (hbase.hregion.compacting.pipeline.segments.limit) was: Changing in-memory compaction (IMC) defaults: (1) active segment is 2% of entire memstore (hbase.memstore.inmemoryflush.threshold.factor), (2) number of segments in compaction pipeline is 2 (hbase.hregion.compacting.pipeline.segments.limit) > IMC Default Parameters for 2.0.0 > > > Key: HBASE-20390 > URL: https://issues.apache.org/jira/browse/HBASE-20390 > Project: HBase > Issue Type: Sub-task >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel >Priority: Major > Attachments: HBASE-20390-branch-2.0-01.patch, > HBASE-20390-branch-2.0-01.patch, HBASE-20390.branch-2.0.002.patch, HBase 2.0 > performance evaluation - throughput SSD_HDD.pdf, hits.ihc.png > > > Setting new default parameters for in-memory compaction based on performance > tests done in HBASE-20188 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-20390) IMC Default Parameters for 2.0.0
[ https://issues.apache.org/jira/browse/HBASE-20390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-20390: -- Attachment: HBASE-20390.branch-2.0.003.patch > IMC Default Parameters for 2.0.0 > > > Key: HBASE-20390 > URL: https://issues.apache.org/jira/browse/HBASE-20390 > Project: HBase > Issue Type: Sub-task >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel >Priority: Major > Attachments: HBASE-20390-branch-2.0-01.patch, > HBASE-20390-branch-2.0-01.patch, HBASE-20390.branch-2.0.002.patch, > HBASE-20390.branch-2.0.003.patch, HBase 2.0 performance evaluation - > throughput SSD_HDD.pdf, hits.ihc.png > > > Setting new default parameters for in-memory compaction based on performance > tests done in HBASE-20188 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HBASE-20188) [TESTING] Performance
[ https://issues.apache.org/jira/browse/HBASE-20188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel reassigned HBASE-20188: - Assignee: Eshcar Hillel (was: stack) > [TESTING] Performance > - > > Key: HBASE-20188 > URL: https://issues.apache.org/jira/browse/HBASE-20188 > Project: HBase > Issue Type: Umbrella > Components: Performance >Reporter: stack >Assignee: Eshcar Hillel >Priority: Blocker > Fix For: 3.0.0, 2.1.0 > > Attachments: CAM-CONFIG-V01.patch, HBASE-20188-xac.sh, > HBASE-20188.sh, HBase 2.0 performance evaluation - 8GB(1).pdf, HBase 2.0 > performance evaluation - 8GB.pdf, HBase 2.0 performance evaluation - Basic vs > None_ system settings.pdf, HBase 2.0 performance evaluation - throughput > SSD_HDD.pdf, ITBLL2.5B_1.2.7vs2.0.0_cpu.png, > ITBLL2.5B_1.2.7vs2.0.0_gctime.png, ITBLL2.5B_1.2.7vs2.0.0_iops.png, > ITBLL2.5B_1.2.7vs2.0.0_load.png, ITBLL2.5B_1.2.7vs2.0.0_memheap.png, > ITBLL2.5B_1.2.7vs2.0.0_memstore.png, ITBLL2.5B_1.2.7vs2.0.0_ops.png, > ITBLL2.5B_1.2.7vs2.0.0_ops_NOT_summing_regions.png, YCSB_CPU.png, > YCSB_GC_TIME.png, YCSB_IN_MEMORY_COMPACTION=NONE.ops.png, YCSB_MEMSTORE.png, > YCSB_OPs.png, YCSB_in-memory-compaction=NONE.ops.png, YCSB_load.png, > flamegraph-1072.1.svg, flamegraph-1072.2.svg, hbase-env.sh, hbase-site.xml, > hbase-site.xml, hits.png, hits_with_fp_scheduler.png, > lock.127.workloadc.20180402T200918Z.svg, > lock.2.memsize2.c.20180403T160257Z.svg, perregion.png, run_ycsb.sh, > total.png, tree.txt, workloadx, workloadx > > > How does 2.0.0 compare to old versions? Is it faster, slower? There is rumor > that it is much slower, that the problem is the asyncwal writing. Does > in-memory compaction slow us down or speed us up? What happens when you > enable offheaping? > Keep notes here in this umbrella issue. Need to be able to say something > about perf when 2.0.0 ships. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HBASE-20188) [TESTING] Performance
[ https://issues.apache.org/jira/browse/HBASE-20188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel reassigned HBASE-20188: - Assignee: (was: Eshcar Hillel) > [TESTING] Performance > - > > Key: HBASE-20188 > URL: https://issues.apache.org/jira/browse/HBASE-20188 > Project: HBase > Issue Type: Umbrella > Components: Performance >Reporter: stack >Priority: Blocker > Fix For: 3.0.0, 2.1.0 > > Attachments: CAM-CONFIG-V01.patch, HBASE-20188-xac.sh, > HBASE-20188.sh, HBase 2.0 performance evaluation - 8GB(1).pdf, HBase 2.0 > performance evaluation - 8GB.pdf, HBase 2.0 performance evaluation - Basic vs > None_ system settings.pdf, HBase 2.0 performance evaluation - throughput > SSD_HDD.pdf, ITBLL2.5B_1.2.7vs2.0.0_cpu.png, > ITBLL2.5B_1.2.7vs2.0.0_gctime.png, ITBLL2.5B_1.2.7vs2.0.0_iops.png, > ITBLL2.5B_1.2.7vs2.0.0_load.png, ITBLL2.5B_1.2.7vs2.0.0_memheap.png, > ITBLL2.5B_1.2.7vs2.0.0_memstore.png, ITBLL2.5B_1.2.7vs2.0.0_ops.png, > ITBLL2.5B_1.2.7vs2.0.0_ops_NOT_summing_regions.png, YCSB_CPU.png, > YCSB_GC_TIME.png, YCSB_IN_MEMORY_COMPACTION=NONE.ops.png, YCSB_MEMSTORE.png, > YCSB_OPs.png, YCSB_in-memory-compaction=NONE.ops.png, YCSB_load.png, > flamegraph-1072.1.svg, flamegraph-1072.2.svg, hbase-env.sh, hbase-site.xml, > hbase-site.xml, hits.png, hits_with_fp_scheduler.png, > lock.127.workloadc.20180402T200918Z.svg, > lock.2.memsize2.c.20180403T160257Z.svg, perregion.png, run_ycsb.sh, > total.png, tree.txt, workloadx, workloadx > > > How does 2.0.0 compare to old versions? Is it faster, slower? There is rumor > that it is much slower, that the problem is the asyncwal writing. Does > in-memory compaction slow us down or speed us up? What happens when you > enable offheaping? > Keep notes here in this umbrella issue. Need to be able to say something > about perf when 2.0.0 ships. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18294) Reduce global heap pressure: flush based on heap occupancy
[ https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-18294: -- Attachment: HBASE-18294.07.patch > Reduce global heap pressure: flush based on heap occupancy > -- > > Key: HBASE-18294 > URL: https://issues.apache.org/jira/browse/HBASE-18294 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel > Attachments: HBASE-18294.01.patch, HBASE-18294.02.patch, > HBASE-18294.03.patch, HBASE-18294.04.patch, HBASE-18294.05.patch, > HBASE-18294.06.patch, HBASE-18294.07.patch > > > A region is flushed if its memory component exceed a threshold (default size > is 128MB). > A flush policy decides whether to flush a store by comparing the size of the > store to another threshold (that can be configured with > hbase.hregion.percolumnfamilyflush.size.lower.bound). > Currently the implementation (in both cases) compares the data size > (key-value only) to the threshold where it should compare the heap size > (which includes index size, and metadata). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18294) Reduce global heap pressure: flush based on heap occupancy
[ https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-18294: -- Status: Open (was: Patch Available) > Reduce global heap pressure: flush based on heap occupancy > -- > > Key: HBASE-18294 > URL: https://issues.apache.org/jira/browse/HBASE-18294 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel > Attachments: HBASE-18294.01.patch, HBASE-18294.02.patch, > HBASE-18294.03.patch, HBASE-18294.04.patch, HBASE-18294.05.patch, > HBASE-18294.06.patch, HBASE-18294.07.patch > > > A region is flushed if its memory component exceed a threshold (default size > is 128MB). > A flush policy decides whether to flush a store by comparing the size of the > store to another threshold (that can be configured with > hbase.hregion.percolumnfamilyflush.size.lower.bound). > Currently the implementation (in both cases) compares the data size > (key-value only) to the threshold where it should compare the heap size > (which includes index size, and metadata). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18294) Reduce global heap pressure: flush based on heap occupancy
[ https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-18294: -- Status: Patch Available (was: Open) > Reduce global heap pressure: flush based on heap occupancy > -- > > Key: HBASE-18294 > URL: https://issues.apache.org/jira/browse/HBASE-18294 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel > Attachments: HBASE-18294.01.patch, HBASE-18294.02.patch, > HBASE-18294.03.patch, HBASE-18294.04.patch, HBASE-18294.05.patch, > HBASE-18294.06.patch, HBASE-18294.07.patch > > > A region is flushed if its memory component exceed a threshold (default size > is 128MB). > A flush policy decides whether to flush a store by comparing the size of the > store to another threshold (that can be configured with > hbase.hregion.percolumnfamilyflush.size.lower.bound). > Currently the implementation (in both cases) compares the data size > (key-value only) to the threshold where it should compare the heap size > (which includes index size, and metadata). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18294) Reduce global heap pressure: flush based on heap occupancy
[ https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-18294: -- Status: Open (was: Patch Available) > Reduce global heap pressure: flush based on heap occupancy > -- > > Key: HBASE-18294 > URL: https://issues.apache.org/jira/browse/HBASE-18294 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel > Attachments: HBASE-18294.01.patch, HBASE-18294.02.patch, > HBASE-18294.03.patch, HBASE-18294.04.patch, HBASE-18294.05.patch, > HBASE-18294.06.patch, HBASE-18294.07.patch, HBASE-18294.07.patch > > > A region is flushed if its memory component exceed a threshold (default size > is 128MB). > A flush policy decides whether to flush a store by comparing the size of the > store to another threshold (that can be configured with > hbase.hregion.percolumnfamilyflush.size.lower.bound). > Currently the implementation (in both cases) compares the data size > (key-value only) to the threshold where it should compare the heap size > (which includes index size, and metadata). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18294) Reduce global heap pressure: flush based on heap occupancy
[ https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-18294: -- Attachment: HBASE-18294.07.patch > Reduce global heap pressure: flush based on heap occupancy > -- > > Key: HBASE-18294 > URL: https://issues.apache.org/jira/browse/HBASE-18294 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel > Attachments: HBASE-18294.01.patch, HBASE-18294.02.patch, > HBASE-18294.03.patch, HBASE-18294.04.patch, HBASE-18294.05.patch, > HBASE-18294.06.patch, HBASE-18294.07.patch, HBASE-18294.07.patch > > > A region is flushed if its memory component exceed a threshold (default size > is 128MB). > A flush policy decides whether to flush a store by comparing the size of the > store to another threshold (that can be configured with > hbase.hregion.percolumnfamilyflush.size.lower.bound). > Currently the implementation (in both cases) compares the data size > (key-value only) to the threshold where it should compare the heap size > (which includes index size, and metadata). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18294) Reduce global heap pressure: flush based on heap occupancy
[ https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-18294: -- Status: Patch Available (was: Open) > Reduce global heap pressure: flush based on heap occupancy > -- > > Key: HBASE-18294 > URL: https://issues.apache.org/jira/browse/HBASE-18294 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel > Attachments: HBASE-18294.01.patch, HBASE-18294.02.patch, > HBASE-18294.03.patch, HBASE-18294.04.patch, HBASE-18294.05.patch, > HBASE-18294.06.patch, HBASE-18294.07.patch, HBASE-18294.07.patch > > > A region is flushed if its memory component exceed a threshold (default size > is 128MB). > A flush policy decides whether to flush a store by comparing the size of the > store to another threshold (that can be configured with > hbase.hregion.percolumnfamilyflush.size.lower.bound). > Currently the implementation (in both cases) compares the data size > (key-value only) to the threshold where it should compare the heap size > (which includes index size, and metadata). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18294) Reduce global heap pressure: flush based on heap occupancy
[ https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16299712#comment-16299712 ] Eshcar Hillel commented on HBASE-18294: --- I attached a new patch, much thinner. MemStoreSizing only has two entries, and no threshold is added. The main change is that in store level and region level we check heap size and not data size to make flush decisions. I think this is something we can all agree on. Patch is in RB, please review. > Reduce global heap pressure: flush based on heap occupancy > -- > > Key: HBASE-18294 > URL: https://issues.apache.org/jira/browse/HBASE-18294 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel > Attachments: HBASE-18294.01.patch, HBASE-18294.02.patch, > HBASE-18294.03.patch, HBASE-18294.04.patch, HBASE-18294.05.patch, > HBASE-18294.06.patch, HBASE-18294.07.patch, HBASE-18294.07.patch > > > A region is flushed if its memory component exceed a threshold (default size > is 128MB). > A flush policy decides whether to flush a store by comparing the size of the > store to another threshold (that can be configured with > hbase.hregion.percolumnfamilyflush.size.lower.bound). > Currently the implementation (in both cases) compares the data size > (key-value only) to the threshold where it should compare the heap size > (which includes index size, and metadata). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18294) Reduce global heap pressure: flush based on heap occupancy
[ https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16300507#comment-16300507 ] Eshcar Hillel commented on HBASE-18294: --- Hey [~jmspaggi], We are definitely going to use the 128MB limit, only that we will compare it to the total heap size occupancy of the store/region and not just to the size of the data. For example, consider a store in which the data of each cell is 100B. We have noticed in our experiments that the (average) heap size of each such cell is *150B*. Which means -- if we have 1M records in memory the data size is only 100MB, while the heap size is 150MB. If we compare the data size against the threshold we do not flush yet, while if we compare the heap size this store was flushed to disk when it exceeded 128MB. > Reduce global heap pressure: flush based on heap occupancy > -- > > Key: HBASE-18294 > URL: https://issues.apache.org/jira/browse/HBASE-18294 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel > Attachments: HBASE-18294.01.patch, HBASE-18294.02.patch, > HBASE-18294.03.patch, HBASE-18294.04.patch, HBASE-18294.05.patch, > HBASE-18294.06.patch, HBASE-18294.07.patch, HBASE-18294.07.patch > > > A region is flushed if its memory component exceed a threshold (default size > is 128MB). > A flush policy decides whether to flush a store by comparing the size of the > store to another threshold (that can be configured with > hbase.hregion.percolumnfamilyflush.size.lower.bound). > Currently the implementation (in both cases) compares the data size > (key-value only) to the threshold where it should compare the heap size > (which includes index size, and metadata). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18294) Reduce global heap pressure: flush based on heap occupancy
[ https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-18294: -- Attachment: HBASE-18294.08.patch > Reduce global heap pressure: flush based on heap occupancy > -- > > Key: HBASE-18294 > URL: https://issues.apache.org/jira/browse/HBASE-18294 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel > Attachments: HBASE-18294.01.patch, HBASE-18294.02.patch, > HBASE-18294.03.patch, HBASE-18294.04.patch, HBASE-18294.05.patch, > HBASE-18294.06.patch, HBASE-18294.07.patch, HBASE-18294.07.patch, > HBASE-18294.08.patch > > > A region is flushed if its memory component exceed a threshold (default size > is 128MB). > A flush policy decides whether to flush a store by comparing the size of the > store to another threshold (that can be configured with > hbase.hregion.percolumnfamilyflush.size.lower.bound). > Currently the implementation (in both cases) compares the data size > (key-value only) to the threshold where it should compare the heap size > (which includes index size, and metadata). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18294) Reduce global heap pressure: flush based on heap occupancy
[ https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-18294: -- Attachment: HBASE-18294.09.patch > Reduce global heap pressure: flush based on heap occupancy > -- > > Key: HBASE-18294 > URL: https://issues.apache.org/jira/browse/HBASE-18294 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel > Attachments: HBASE-18294.01.patch, HBASE-18294.02.patch, > HBASE-18294.03.patch, HBASE-18294.04.patch, HBASE-18294.05.patch, > HBASE-18294.06.patch, HBASE-18294.07.patch, HBASE-18294.07.patch, > HBASE-18294.08.patch, HBASE-18294.09.patch > > > A region is flushed if its memory component exceed a threshold (default size > is 128MB). > A flush policy decides whether to flush a store by comparing the size of the > store to another threshold (that can be configured with > hbase.hregion.percolumnfamilyflush.size.lower.bound). > Currently the implementation (in both cases) compares the data size > (key-value only) to the threshold where it should compare the heap size > (which includes index size, and metadata). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19282) CellChunkMap Benchmarking and User Interface
[ https://issues.apache.org/jira/browse/HBASE-19282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16302893#comment-16302893 ] Eshcar Hillel commented on HBASE-19282: --- Left some comments on RB. Once fixed +1 by me. > CellChunkMap Benchmarking and User Interface > > > Key: HBASE-19282 > URL: https://issues.apache.org/jira/browse/HBASE-19282 > Project: HBase > Issue Type: Sub-task >Reporter: Anastasia Braginsky > Attachments: CCM Benchmarking.pdf, HBASE-19282-V03.patch, > HBASE-19282-V05.patch, HBASE-19282-V06.patch, HBASE-19282-V06.patch, > HBASE-19282.patch > > > We have made some experiments how working with CellChunkMap (CCM) influences > the performance when running on-heap and off-heap. Based on those results it > is suggested to tie the MSLAB usage (off-heap or on-heap) with CCM index > usage. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18294) Reduce global heap pressure: flush based on heap occupancy
[ https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-18294: -- Attachment: HBASE-18294.10.patch > Reduce global heap pressure: flush based on heap occupancy > -- > > Key: HBASE-18294 > URL: https://issues.apache.org/jira/browse/HBASE-18294 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel > Attachments: HBASE-18294.01.patch, HBASE-18294.02.patch, > HBASE-18294.03.patch, HBASE-18294.04.patch, HBASE-18294.05.patch, > HBASE-18294.06.patch, HBASE-18294.07.patch, HBASE-18294.07.patch, > HBASE-18294.08.patch, HBASE-18294.09.patch, HBASE-18294.10.patch > > > A region is flushed if its memory component exceed a threshold (default size > is 128MB). > A flush policy decides whether to flush a store by comparing the size of the > store to another threshold (that can be configured with > hbase.hregion.percolumnfamilyflush.size.lower.bound). > Currently the implementation (in both cases) compares the data size > (key-value only) to the threshold where it should compare the heap size > (which includes index size, and metadata). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18294) Reduce global heap pressure: flush based on heap occupancy
[ https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16303287#comment-16303287 ] Eshcar Hillel commented on HBASE-18294: --- Sorry [~ram_krish] I missed you comment. bq. Probably you should go with || case - check both data + heap or only data. Because for offheap memstore heap won't reach this value. Note that with the new patch heap size is always larger than data size because it includes both data and metadata regardless of where it is allocated heap or off heap. > Reduce global heap pressure: flush based on heap occupancy > -- > > Key: HBASE-18294 > URL: https://issues.apache.org/jira/browse/HBASE-18294 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel > Attachments: HBASE-18294.01.patch, HBASE-18294.02.patch, > HBASE-18294.03.patch, HBASE-18294.04.patch, HBASE-18294.05.patch, > HBASE-18294.06.patch, HBASE-18294.07.patch, HBASE-18294.07.patch, > HBASE-18294.08.patch, HBASE-18294.09.patch, HBASE-18294.10.patch > > > A region is flushed if its memory component exceed a threshold (default size > is 128MB). > A flush policy decides whether to flush a store by comparing the size of the > store to another threshold (that can be configured with > hbase.hregion.percolumnfamilyflush.size.lower.bound). > Currently the implementation (in both cases) compares the data size > (key-value only) to the threshold where it should compare the heap size > (which includes index size, and metadata). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18294) Reduce global heap pressure: flush based on heap occupancy
[ https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-18294: -- Status: Open (was: Patch Available) > Reduce global heap pressure: flush based on heap occupancy > -- > > Key: HBASE-18294 > URL: https://issues.apache.org/jira/browse/HBASE-18294 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel > Attachments: HBASE-18294.01.patch, HBASE-18294.02.patch, > HBASE-18294.03.patch, HBASE-18294.04.patch, HBASE-18294.05.patch, > HBASE-18294.06.patch, HBASE-18294.07.patch, HBASE-18294.07.patch, > HBASE-18294.08.patch, HBASE-18294.09.patch, HBASE-18294.10.patch > > > A region is flushed if its memory component exceed a threshold (default size > is 128MB). > A flush policy decides whether to flush a store by comparing the size of the > store to another threshold (that can be configured with > hbase.hregion.percolumnfamilyflush.size.lower.bound). > Currently the implementation (in both cases) compares the data size > (key-value only) to the threshold where it should compare the heap size > (which includes index size, and metadata). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18294) Reduce global heap pressure: flush based on heap occupancy
[ https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-18294: -- Status: Patch Available (was: Open) > Reduce global heap pressure: flush based on heap occupancy > -- > > Key: HBASE-18294 > URL: https://issues.apache.org/jira/browse/HBASE-18294 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel > Attachments: HBASE-18294.01.patch, HBASE-18294.02.patch, > HBASE-18294.03.patch, HBASE-18294.04.patch, HBASE-18294.05.patch, > HBASE-18294.06.patch, HBASE-18294.07.patch, HBASE-18294.07.patch, > HBASE-18294.08.patch, HBASE-18294.09.patch, HBASE-18294.10.patch > > > A region is flushed if its memory component exceed a threshold (default size > is 128MB). > A flush policy decides whether to flush a store by comparing the size of the > store to another threshold (that can be configured with > hbase.hregion.percolumnfamilyflush.size.lower.bound). > Currently the implementation (in both cases) compares the data size > (key-value only) to the threshold where it should compare the heap size > (which includes index size, and metadata). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18294) Reduce global heap pressure: flush based on heap occupancy
[ https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-18294: -- Attachment: HBASE-18294.11.patch > Reduce global heap pressure: flush based on heap occupancy > -- > > Key: HBASE-18294 > URL: https://issues.apache.org/jira/browse/HBASE-18294 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel > Attachments: HBASE-18294.01.patch, HBASE-18294.02.patch, > HBASE-18294.03.patch, HBASE-18294.04.patch, HBASE-18294.05.patch, > HBASE-18294.06.patch, HBASE-18294.07.patch, HBASE-18294.07.patch, > HBASE-18294.08.patch, HBASE-18294.09.patch, HBASE-18294.10.patch, > HBASE-18294.11.patch > > > A region is flushed if its memory component exceed a threshold (default size > is 128MB). > A flush policy decides whether to flush a store by comparing the size of the > store to another threshold (that can be configured with > hbase.hregion.percolumnfamilyflush.size.lower.bound). > Currently the implementation (in both cases) compares the data size > (key-value only) to the threshold where it should compare the heap size > (which includes index size, and metadata). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18294) Reduce global heap pressure: flush based on heap occupancy
[ https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16303629#comment-16303629 ] Eshcar Hillel commented on HBASE-18294: --- I did not change the global level - see current patch in RB > Reduce global heap pressure: flush based on heap occupancy > -- > > Key: HBASE-18294 > URL: https://issues.apache.org/jira/browse/HBASE-18294 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel > Attachments: HBASE-18294.01.patch, HBASE-18294.02.patch, > HBASE-18294.03.patch, HBASE-18294.04.patch, HBASE-18294.05.patch, > HBASE-18294.06.patch, HBASE-18294.07.patch, HBASE-18294.07.patch, > HBASE-18294.08.patch, HBASE-18294.09.patch, HBASE-18294.10.patch, > HBASE-18294.11.patch > > > A region is flushed if its memory component exceed a threshold (default size > is 128MB). > A flush policy decides whether to flush a store by comparing the size of the > store to another threshold (that can be configured with > hbase.hregion.percolumnfamilyflush.size.lower.bound). > Currently the implementation (in both cases) compares the data size > (key-value only) to the threshold where it should compare the heap size > (which includes index size, and metadata). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18294) Reduce global heap pressure: flush based on heap occupancy
[ https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-18294: -- Attachment: HBASE-18294.11.patch Test passes locally, trying to re-submit > Reduce global heap pressure: flush based on heap occupancy > -- > > Key: HBASE-18294 > URL: https://issues.apache.org/jira/browse/HBASE-18294 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel > Attachments: HBASE-18294.01.patch, HBASE-18294.02.patch, > HBASE-18294.03.patch, HBASE-18294.04.patch, HBASE-18294.05.patch, > HBASE-18294.06.patch, HBASE-18294.07.patch, HBASE-18294.07.patch, > HBASE-18294.08.patch, HBASE-18294.09.patch, HBASE-18294.10.patch, > HBASE-18294.11.patch, HBASE-18294.11.patch > > > A region is flushed if its memory component exceed a threshold (default size > is 128MB). > A flush policy decides whether to flush a store by comparing the size of the > store to another threshold (that can be configured with > hbase.hregion.percolumnfamilyflush.size.lower.bound). > Currently the implementation (in both cases) compares the data size > (key-value only) to the threshold where it should compare the heap size > (which includes index size, and metadata). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18294) Reduce global heap pressure: flush based on heap occupancy
[ https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16303858#comment-16303858 ] Eshcar Hillel commented on HBASE-18294: --- QA passed I would like to commit this in the next few days. Please let me know if there are issues that were not addressed. > Reduce global heap pressure: flush based on heap occupancy > -- > > Key: HBASE-18294 > URL: https://issues.apache.org/jira/browse/HBASE-18294 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel > Attachments: HBASE-18294.01.patch, HBASE-18294.02.patch, > HBASE-18294.03.patch, HBASE-18294.04.patch, HBASE-18294.05.patch, > HBASE-18294.06.patch, HBASE-18294.07.patch, HBASE-18294.07.patch, > HBASE-18294.08.patch, HBASE-18294.09.patch, HBASE-18294.10.patch, > HBASE-18294.11.patch, HBASE-18294.11.patch > > > A region is flushed if its memory component exceed a threshold (default size > is 128MB). > A flush policy decides whether to flush a store by comparing the size of the > store to another threshold (that can be configured with > hbase.hregion.percolumnfamilyflush.size.lower.bound). > Currently the implementation (in both cases) compares the data size > (key-value only) to the threshold where it should compare the heap size > (which includes index size, and metadata). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18294) Reduce global heap pressure: flush based on heap occupancy
[ https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-18294: -- Attachment: HBASE-18294.12.patch > Reduce global heap pressure: flush based on heap occupancy > -- > > Key: HBASE-18294 > URL: https://issues.apache.org/jira/browse/HBASE-18294 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel > Attachments: HBASE-18294.01.patch, HBASE-18294.02.patch, > HBASE-18294.03.patch, HBASE-18294.04.patch, HBASE-18294.05.patch, > HBASE-18294.06.patch, HBASE-18294.07.patch, HBASE-18294.07.patch, > HBASE-18294.08.patch, HBASE-18294.09.patch, HBASE-18294.10.patch, > HBASE-18294.11.patch, HBASE-18294.11.patch, HBASE-18294.12.patch > > > A region is flushed if its memory component exceed a threshold (default size > is 128MB). > A flush policy decides whether to flush a store by comparing the size of the > store to another threshold (that can be configured with > hbase.hregion.percolumnfamilyflush.size.lower.bound). > Currently the implementation (in both cases) compares the data size > (key-value only) to the threshold where it should compare the heap size > (which includes index size, and metadata). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18294) Reduce global heap pressure: flush based on heap occupancy
[ https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-18294: -- Attachment: HBASE-18294.13.patch > Reduce global heap pressure: flush based on heap occupancy > -- > > Key: HBASE-18294 > URL: https://issues.apache.org/jira/browse/HBASE-18294 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel > Attachments: HBASE-18294.01.patch, HBASE-18294.02.patch, > HBASE-18294.03.patch, HBASE-18294.04.patch, HBASE-18294.05.patch, > HBASE-18294.06.patch, HBASE-18294.07.patch, HBASE-18294.07.patch, > HBASE-18294.08.patch, HBASE-18294.09.patch, HBASE-18294.10.patch, > HBASE-18294.11.patch, HBASE-18294.11.patch, HBASE-18294.12.patch, > HBASE-18294.13.patch > > > A region is flushed if its memory component exceed a threshold (default size > is 128MB). > A flush policy decides whether to flush a store by comparing the size of the > store to another threshold (that can be configured with > hbase.hregion.percolumnfamilyflush.size.lower.bound). > Currently the implementation (in both cases) compares the data size > (key-value only) to the threshold where it should compare the heap size > (which includes index size, and metadata). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18294) Reduce global heap pressure: flush based on heap occupancy
[ https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16304443#comment-16304443 ] Eshcar Hillel commented on HBASE-18294: --- Thanks all for the reviews, attached a new patch with the final changes. You got it all correct [~stack]: - At the region level we make flush decisions based on heap-occupancy instead of data size; same care for on- and off-heap cases. - At the RS level all stays as before with special care for heap and off heap cases. I looked at HBASE-19639 seems like updates are at a rate that the flush cannot handle so eventually you exceed the blocking threshold. Can you add some more information from the logs, how many flushes succeed before the test is stuck? what's preventing the flush from completing? > Reduce global heap pressure: flush based on heap occupancy > -- > > Key: HBASE-18294 > URL: https://issues.apache.org/jira/browse/HBASE-18294 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel > Attachments: HBASE-18294.01.patch, HBASE-18294.02.patch, > HBASE-18294.03.patch, HBASE-18294.04.patch, HBASE-18294.05.patch, > HBASE-18294.06.patch, HBASE-18294.07.patch, HBASE-18294.07.patch, > HBASE-18294.08.patch, HBASE-18294.09.patch, HBASE-18294.10.patch, > HBASE-18294.11.patch, HBASE-18294.11.patch, HBASE-18294.12.patch, > HBASE-18294.13.patch > > > A region is flushed if its memory component exceed a threshold (default size > is 128MB). > A flush policy decides whether to flush a store by comparing the size of the > store to another threshold (that can be configured with > hbase.hregion.percolumnfamilyflush.size.lower.bound). > Currently the implementation (in both cases) compares the data size > (key-value only) to the threshold where it should compare the heap size > (which includes index size, and metadata). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18294) Reduce global heap pressure: flush based on heap occupancy
[
https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16304514#comment-16304514
]
Eshcar Hillel commented on HBASE-18294:
---
My previous solution had three different counters in MemStoreSize: (1) one for
data size (2) one for on-heap size (3) one for off-heap size.
The main reason I rolled back this change is the understanding that in the near
future HBase is going to support only clusters that are either entirely on-heap
or entirely off-heap and so there is no need to measure on- and off-heap
occupancy separately in any level (cluster, RS, region or memstore).
This means that when we say heap: heap size, heap occupancy, heap space, etc.
we mean all bytes allocated on the heap including JVM and OS heap.
If an implementation of the {{Cell}} class the method {{heapSize()}} does not
return the number of bytes allocated to store data that is related to this cell
(key, value, metadata, etc.) then I would say the implementation does not
satisfy the specification of the method and needs to be changed.
Is there any reason the method was implemented this way??
How about the following change adding {{KeyValueUtil.length(this)}} to the
computation in the implementation of {{ByteBufferKV}}
I think it will fix the problem. Do you see any reason it shouldn't work?
{code}
public long heapSize() {
if (this.buf.hasArray()) {
return ClassSize.align(FIXED_OVERHEAD + length);
}
return ClassSize.align(FIXED_OVERHEAD) + KeyValueUtil.length(this);
}
{code}
> Reduce global heap pressure: flush based on heap occupancy
> --
>
> Key: HBASE-18294
> URL: https://issues.apache.org/jira/browse/HBASE-18294
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0
>Reporter: Eshcar Hillel
>Assignee: Eshcar Hillel
> Attachments: HBASE-18294.01.patch, HBASE-18294.02.patch,
> HBASE-18294.03.patch, HBASE-18294.04.patch, HBASE-18294.05.patch,
> HBASE-18294.06.patch, HBASE-18294.07.patch, HBASE-18294.07.patch,
> HBASE-18294.08.patch, HBASE-18294.09.patch, HBASE-18294.10.patch,
> HBASE-18294.11.patch, HBASE-18294.11.patch, HBASE-18294.12.patch,
> HBASE-18294.13.patch
>
>
> A region is flushed if its memory component exceed a threshold (default size
> is 128MB).
> A flush policy decides whether to flush a store by comparing the size of the
> store to another threshold (that can be configured with
> hbase.hregion.percolumnfamilyflush.size.lower.bound).
> Currently the implementation (in both cases) compares the data size
> (key-value only) to the threshold where it should compare the heap size
> (which includes index size, and metadata).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18294) Reduce global heap pressure: flush based on heap occupancy
[
https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16305325#comment-16305325
]
Eshcar Hillel commented on HBASE-18294:
---
Thanks [~stack] for the document, I think its a good summary of the issue.
I think it is important to have symmetric behavior for on-heap and off-heap
clusters. Even if users do special tuning for off-heap clusters it is not
reasonable to expect them understand that on-heap flushes when total occupancy
exceeds 128MB and off-heap flushes when key-values are exceeding 128MB.
I would like to avoid checking heapsize >128MB OR datasize > 128MB since this
is exactly defining unsymmetrical behavior for on-heap and off-heap clusters.
I understand also that [~anoop.hbase] and [~ram_krish] are objecting to change
the implementation of {{heapSize()}} so it accounts also for off-heap size,
either because they feel it is confusing or because they think it will cause
some real problems in the off-heap implementation
So here is a suggestion:
(1) I will change the name of the counter in {{MemStoreSize}} from {{heapSize}}
to something like {{totalAllocationSize}} or {{totalOccupancy}}
(2) I will add methods {{estimatedTotalOccupancyOf(final Cell cell)}} and
{{totalOccupancy()}} to compute total occupancy of data+metadata no matter were
they are allocated java-heap or off-heap (the way I suggested above) and will
use them to update the counters in {{MemStoreSize}}
(3) only changing region level flush decision, not changing RS global
accounting and decisions
If this is acceptable, I can make a new patch in a day or two.
> Reduce global heap pressure: flush based on heap occupancy
> --
>
> Key: HBASE-18294
> URL: https://issues.apache.org/jira/browse/HBASE-18294
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0
>Reporter: Eshcar Hillel
>Assignee: Eshcar Hillel
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-18294.01.patch, HBASE-18294.02.patch,
> HBASE-18294.03.patch, HBASE-18294.04.patch, HBASE-18294.05.patch,
> HBASE-18294.06.patch, HBASE-18294.07.patch, HBASE-18294.07.patch,
> HBASE-18294.08.patch, HBASE-18294.09.patch, HBASE-18294.10.patch,
> HBASE-18294.11.patch, HBASE-18294.11.patch, HBASE-18294.12.patch,
> HBASE-18294.13.patch
>
>
> A region is flushed if its memory component exceed a threshold (default size
> is 128MB).
> A flush policy decides whether to flush a store by comparing the size of the
> store to another threshold (that can be configured with
> hbase.hregion.percolumnfamilyflush.size.lower.bound).
> Currently the implementation (in both cases) compares the data size
> (key-value only) to the threshold where it should compare the heap size
> (which includes index size, and metadata).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18294) Reduce global heap pressure: flush based on heap occupancy
[ https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16305509#comment-16305509 ] Eshcar Hillel commented on HBASE-18294: --- Thank you [~anoop.hbase] for explaining. I think I understand now what is going on, however, I must say things are not at all simple :) In different places we give different semantic to the same name. For example, data size means in some places key-value size and in some places means off-heap size; heap size sometimes means heap size and sometimes means metadata size. This is very hard to follow and maintain. My understanding is that for off heap clusters part is allocated on-heap and part is allocated off-heap and there is no way around it. As this is the state of the things, I strongly feel that we should go back to the patch that introduced three counter in MemStoreSize data size, heap size, and off-heap size. At the region level we'll have MemStoreSize accounting (with 3 counters), and have flush decisions based on comparing to (existing) on-heap and (new) off-heap thresholds; while at the RS level we'll have only off-heap accounting and on-heap accounting to be compared against the two existing thresholds. This may seem like a lot to change, however, (1) the patch is practically ready (might need some small modifications and adjustments due to recent code reviews) (2) at the end of the day each counter has its own semantic that is highly correlated with its name and is consistent across levels. I think #2 is super important to avoid future bugs and facilitate maintenance of the code for further adjustments. Do you see any reasons to object to the change beside your view that it is more complicated than the current trunk code? > Reduce global heap pressure: flush based on heap occupancy > -- > > Key: HBASE-18294 > URL: https://issues.apache.org/jira/browse/HBASE-18294 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-18294.01.patch, HBASE-18294.02.patch, > HBASE-18294.03.patch, HBASE-18294.04.patch, HBASE-18294.05.patch, > HBASE-18294.06.patch, HBASE-18294.07.patch, HBASE-18294.07.patch, > HBASE-18294.08.patch, HBASE-18294.09.patch, HBASE-18294.10.patch, > HBASE-18294.11.patch, HBASE-18294.11.patch, HBASE-18294.12.patch, > HBASE-18294.13.patch > > > A region is flushed if its memory component exceed a threshold (default size > is 128MB). > A flush policy decides whether to flush a store by comparing the size of the > store to another threshold (that can be configured with > hbase.hregion.percolumnfamilyflush.size.lower.bound). > Currently the implementation (in both cases) compares the data size > (key-value only) to the threshold where it should compare the heap size > (which includes index size, and metadata). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18294) Reduce global heap pressure: flush based on heap occupancy
[ https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16305641#comment-16305641 ] Eshcar Hillel commented on HBASE-18294: --- OK let me prepare the patch. > Reduce global heap pressure: flush based on heap occupancy > -- > > Key: HBASE-18294 > URL: https://issues.apache.org/jira/browse/HBASE-18294 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-18294.01.patch, HBASE-18294.02.patch, > HBASE-18294.03.patch, HBASE-18294.04.patch, HBASE-18294.05.patch, > HBASE-18294.06.patch, HBASE-18294.07.patch, HBASE-18294.07.patch, > HBASE-18294.08.patch, HBASE-18294.09.patch, HBASE-18294.10.patch, > HBASE-18294.11.patch, HBASE-18294.11.patch, HBASE-18294.12.patch, > HBASE-18294.13.patch > > > A region is flushed if its memory component exceed a threshold (default size > is 128MB). > A flush policy decides whether to flush a store by comparing the size of the > store to another threshold (that can be configured with > hbase.hregion.percolumnfamilyflush.size.lower.bound). > Currently the implementation (in both cases) compares the data size > (key-value only) to the threshold where it should compare the heap size > (which includes index size, and metadata). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18294) Reduce global heap pressure: flush based on heap occupancy
[ https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-18294: -- Attachment: HBASE-18294.15.patch > Reduce global heap pressure: flush based on heap occupancy > -- > > Key: HBASE-18294 > URL: https://issues.apache.org/jira/browse/HBASE-18294 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-18294.01.patch, HBASE-18294.02.patch, > HBASE-18294.03.patch, HBASE-18294.04.patch, HBASE-18294.05.patch, > HBASE-18294.06.patch, HBASE-18294.07.patch, HBASE-18294.07.patch, > HBASE-18294.08.patch, HBASE-18294.09.patch, HBASE-18294.10.patch, > HBASE-18294.11.patch, HBASE-18294.11.patch, HBASE-18294.12.patch, > HBASE-18294.13.patch, HBASE-18294.15.patch > > > A region is flushed if its memory component exceed a threshold (default size > is 128MB). > A flush policy decides whether to flush a store by comparing the size of the > store to another threshold (that can be configured with > hbase.hregion.percolumnfamilyflush.size.lower.bound). > Currently the implementation (in both cases) compares the data size > (key-value only) to the threshold where it should compare the heap size > (which includes index size, and metadata). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18294) Reduce global heap pressure: flush based on heap occupancy
[ https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-18294: -- Status: Open (was: Patch Available) > Reduce global heap pressure: flush based on heap occupancy > -- > > Key: HBASE-18294 > URL: https://issues.apache.org/jira/browse/HBASE-18294 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-18294.01.patch, HBASE-18294.02.patch, > HBASE-18294.03.patch, HBASE-18294.04.patch, HBASE-18294.05.patch, > HBASE-18294.06.patch, HBASE-18294.07.patch, HBASE-18294.07.patch, > HBASE-18294.08.patch, HBASE-18294.09.patch, HBASE-18294.10.patch, > HBASE-18294.11.patch, HBASE-18294.11.patch, HBASE-18294.12.patch, > HBASE-18294.13.patch, HBASE-18294.15.patch > > > A region is flushed if its memory component exceed a threshold (default size > is 128MB). > A flush policy decides whether to flush a store by comparing the size of the > store to another threshold (that can be configured with > hbase.hregion.percolumnfamilyflush.size.lower.bound). > Currently the implementation (in both cases) compares the data size > (key-value only) to the threshold where it should compare the heap size > (which includes index size, and metadata). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18294) Reduce global heap pressure: flush based on heap occupancy
[ https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-18294: -- Status: Patch Available (was: Open) > Reduce global heap pressure: flush based on heap occupancy > -- > > Key: HBASE-18294 > URL: https://issues.apache.org/jira/browse/HBASE-18294 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-18294.01.patch, HBASE-18294.02.patch, > HBASE-18294.03.patch, HBASE-18294.04.patch, HBASE-18294.05.patch, > HBASE-18294.06.patch, HBASE-18294.07.patch, HBASE-18294.07.patch, > HBASE-18294.08.patch, HBASE-18294.09.patch, HBASE-18294.10.patch, > HBASE-18294.11.patch, HBASE-18294.11.patch, HBASE-18294.12.patch, > HBASE-18294.13.patch, HBASE-18294.15.patch > > > A region is flushed if its memory component exceed a threshold (default size > is 128MB). > A flush policy decides whether to flush a store by comparing the size of the > store to another threshold (that can be configured with > hbase.hregion.percolumnfamilyflush.size.lower.bound). > Currently the implementation (in both cases) compares the data size > (key-value only) to the threshold where it should compare the heap size > (which includes index size, and metadata). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-18294) Reduce global heap pressure: flush based on heap occupancy
[ https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16333525#comment-16333525 ] Eshcar Hillel commented on HBASE-18294: --- We are back to a ``fat'' patch, so here is a summary of major changes w.r.t. master: - MemStoreSize and MemStoreSizing now manage three counters (1) data size (2) heap size (3) off heap size, - FlushLargeStorePolicy::shouldFlush(), HRegion::checkResources(), HRegion::isFlushSize() and HRegion::requestFlushIfNeeded() check both heap size and off heap size against their (heap and off-heap) bounds, - RegionServerAccounting manages 3 counters (off-heap size is added), RegionServerAccounting::isAboveHighWaterMark(), RegionServerAccounting::isAboveLowWaterMark(), and RegionServerAccounting::getFlushPressure() compare on-heap and off-heap sizes to their bounds, - ByteBufferKeyValue::heapSize() always considers key-value length. Patch is also in RB. > Reduce global heap pressure: flush based on heap occupancy > -- > > Key: HBASE-18294 > URL: https://issues.apache.org/jira/browse/HBASE-18294 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-18294.01.patch, HBASE-18294.02.patch, > HBASE-18294.03.patch, HBASE-18294.04.patch, HBASE-18294.05.patch, > HBASE-18294.06.patch, HBASE-18294.07.patch, HBASE-18294.07.patch, > HBASE-18294.08.patch, HBASE-18294.09.patch, HBASE-18294.10.patch, > HBASE-18294.11.patch, HBASE-18294.11.patch, HBASE-18294.12.patch, > HBASE-18294.13.patch, HBASE-18294.15.patch > > > A region is flushed if its memory component exceed a threshold (default size > is 128MB). > A flush policy decides whether to flush a store by comparing the size of the > store to another threshold (that can be configured with > hbase.hregion.percolumnfamilyflush.size.lower.bound). > Currently the implementation (in both cases) compares the data size > (key-value only) to the threshold where it should compare the heap size > (which includes index size, and metadata). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-18294) Reduce global heap pressure: flush based on heap occupancy
[ https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16333527#comment-16333527 ] Eshcar Hillel commented on HBASE-18294: --- Thanks Ted. I will try it if QA doesn't succeed this time as well. > Reduce global heap pressure: flush based on heap occupancy > -- > > Key: HBASE-18294 > URL: https://issues.apache.org/jira/browse/HBASE-18294 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-18294.01.patch, HBASE-18294.02.patch, > HBASE-18294.03.patch, HBASE-18294.04.patch, HBASE-18294.05.patch, > HBASE-18294.06.patch, HBASE-18294.07.patch, HBASE-18294.07.patch, > HBASE-18294.08.patch, HBASE-18294.09.patch, HBASE-18294.10.patch, > HBASE-18294.11.patch, HBASE-18294.11.patch, HBASE-18294.12.patch, > HBASE-18294.13.patch, HBASE-18294.15.patch > > > A region is flushed if its memory component exceed a threshold (default size > is 128MB). > A flush policy decides whether to flush a store by comparing the size of the > store to another threshold (that can be configured with > hbase.hregion.percolumnfamilyflush.size.lower.bound). > Currently the implementation (in both cases) compares the data size > (key-value only) to the threshold where it should compare the heap size > (which includes index size, and metadata). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18294) Reduce global heap pressure: flush based on heap occupancy
[ https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-18294: -- Status: Open (was: Patch Available) > Reduce global heap pressure: flush based on heap occupancy > -- > > Key: HBASE-18294 > URL: https://issues.apache.org/jira/browse/HBASE-18294 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-18294.01.patch, HBASE-18294.02.patch, > HBASE-18294.03.patch, HBASE-18294.04.patch, HBASE-18294.05.patch, > HBASE-18294.06.patch, HBASE-18294.07.patch, HBASE-18294.07.patch, > HBASE-18294.08.patch, HBASE-18294.09.patch, HBASE-18294.10.patch, > HBASE-18294.11.patch, HBASE-18294.11.patch, HBASE-18294.12.patch, > HBASE-18294.13.patch, HBASE-18294.15.patch > > > A region is flushed if its memory component exceed a threshold (default size > is 128MB). > A flush policy decides whether to flush a store by comparing the size of the > store to another threshold (that can be configured with > hbase.hregion.percolumnfamilyflush.size.lower.bound). > Currently the implementation (in both cases) compares the data size > (key-value only) to the threshold where it should compare the heap size > (which includes index size, and metadata). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18294) Reduce global heap pressure: flush based on heap occupancy
[ https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-18294: -- Attachment: HBASE-18294.01.patch > Reduce global heap pressure: flush based on heap occupancy > -- > > Key: HBASE-18294 > URL: https://issues.apache.org/jira/browse/HBASE-18294 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-18294.01.patch, HBASE-18294.02.patch, > HBASE-18294.03.patch, HBASE-18294.04.patch, HBASE-18294.05.patch, > HBASE-18294.06.patch, HBASE-18294.07.patch, HBASE-18294.07.patch, > HBASE-18294.08.patch, HBASE-18294.09.patch, HBASE-18294.10.patch, > HBASE-18294.11.patch, HBASE-18294.11.patch, HBASE-18294.12.patch, > HBASE-18294.13.patch, HBASE-18294.15.patch > > > A region is flushed if its memory component exceed a threshold (default size > is 128MB). > A flush policy decides whether to flush a store by comparing the size of the > store to another threshold (that can be configured with > hbase.hregion.percolumnfamilyflush.size.lower.bound). > Currently the implementation (in both cases) compares the data size > (key-value only) to the threshold where it should compare the heap size > (which includes index size, and metadata). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18294) Reduce global heap pressure: flush based on heap occupancy
[ https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-18294: -- Attachment: (was: HBASE-18294.01.patch) > Reduce global heap pressure: flush based on heap occupancy > -- > > Key: HBASE-18294 > URL: https://issues.apache.org/jira/browse/HBASE-18294 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-18294.01.patch, HBASE-18294.02.patch, > HBASE-18294.03.patch, HBASE-18294.04.patch, HBASE-18294.05.patch, > HBASE-18294.06.patch, HBASE-18294.07.patch, HBASE-18294.07.patch, > HBASE-18294.08.patch, HBASE-18294.09.patch, HBASE-18294.10.patch, > HBASE-18294.11.patch, HBASE-18294.11.patch, HBASE-18294.12.patch, > HBASE-18294.13.patch, HBASE-18294.15.patch > > > A region is flushed if its memory component exceed a threshold (default size > is 128MB). > A flush policy decides whether to flush a store by comparing the size of the > store to another threshold (that can be configured with > hbase.hregion.percolumnfamilyflush.size.lower.bound). > Currently the implementation (in both cases) compares the data size > (key-value only) to the threshold where it should compare the heap size > (which includes index size, and metadata). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18294) Reduce global heap pressure: flush based on heap occupancy
[ https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-18294: -- Status: Patch Available (was: Open) > Reduce global heap pressure: flush based on heap occupancy > -- > > Key: HBASE-18294 > URL: https://issues.apache.org/jira/browse/HBASE-18294 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-18294.01.patch, HBASE-18294.02.patch, > HBASE-18294.03.patch, HBASE-18294.04.patch, HBASE-18294.05.patch, > HBASE-18294.06.patch, HBASE-18294.07.patch, HBASE-18294.07.patch, > HBASE-18294.08.patch, HBASE-18294.09.patch, HBASE-18294.10.patch, > HBASE-18294.11.patch, HBASE-18294.11.patch, HBASE-18294.12.patch, > HBASE-18294.13.patch, HBASE-18294.15.patch > > > A region is flushed if its memory component exceed a threshold (default size > is 128MB). > A flush policy decides whether to flush a store by comparing the size of the > store to another threshold (that can be configured with > hbase.hregion.percolumnfamilyflush.size.lower.bound). > Currently the implementation (in both cases) compares the data size > (key-value only) to the threshold where it should compare the heap size > (which includes index size, and metadata). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18294) Reduce global heap pressure: flush based on heap occupancy
[ https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-18294: -- Status: Open (was: Patch Available) > Reduce global heap pressure: flush based on heap occupancy > -- > > Key: HBASE-18294 > URL: https://issues.apache.org/jira/browse/HBASE-18294 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-18294.01.patch, HBASE-18294.02.patch, > HBASE-18294.03.patch, HBASE-18294.04.patch, HBASE-18294.05.patch, > HBASE-18294.06.patch, HBASE-18294.07.patch, HBASE-18294.07.patch, > HBASE-18294.08.patch, HBASE-18294.09.patch, HBASE-18294.10.patch, > HBASE-18294.11.patch, HBASE-18294.11.patch, HBASE-18294.12.patch, > HBASE-18294.13.patch, HBASE-18294.15.patch > > > A region is flushed if its memory component exceed a threshold (default size > is 128MB). > A flush policy decides whether to flush a store by comparing the size of the > store to another threshold (that can be configured with > hbase.hregion.percolumnfamilyflush.size.lower.bound). > Currently the implementation (in both cases) compares the data size > (key-value only) to the threshold where it should compare the heap size > (which includes index size, and metadata). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18294) Reduce global heap pressure: flush based on heap occupancy
[ https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-18294: -- Attachment: HBASE-18294.16.patch > Reduce global heap pressure: flush based on heap occupancy > -- > > Key: HBASE-18294 > URL: https://issues.apache.org/jira/browse/HBASE-18294 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-18294.01.patch, HBASE-18294.02.patch, > HBASE-18294.03.patch, HBASE-18294.04.patch, HBASE-18294.05.patch, > HBASE-18294.06.patch, HBASE-18294.07.patch, HBASE-18294.07.patch, > HBASE-18294.08.patch, HBASE-18294.09.patch, HBASE-18294.10.patch, > HBASE-18294.11.patch, HBASE-18294.11.patch, HBASE-18294.12.patch, > HBASE-18294.13.patch, HBASE-18294.15.patch, HBASE-18294.16.patch > > > A region is flushed if its memory component exceed a threshold (default size > is 128MB). > A flush policy decides whether to flush a store by comparing the size of the > store to another threshold (that can be configured with > hbase.hregion.percolumnfamilyflush.size.lower.bound). > Currently the implementation (in both cases) compares the data size > (key-value only) to the threshold where it should compare the heap size > (which includes index size, and metadata). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18294) Reduce global heap pressure: flush based on heap occupancy
[ https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-18294: -- Status: Patch Available (was: Open) > Reduce global heap pressure: flush based on heap occupancy > -- > > Key: HBASE-18294 > URL: https://issues.apache.org/jira/browse/HBASE-18294 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-18294.01.patch, HBASE-18294.02.patch, > HBASE-18294.03.patch, HBASE-18294.04.patch, HBASE-18294.05.patch, > HBASE-18294.06.patch, HBASE-18294.07.patch, HBASE-18294.07.patch, > HBASE-18294.08.patch, HBASE-18294.09.patch, HBASE-18294.10.patch, > HBASE-18294.11.patch, HBASE-18294.11.patch, HBASE-18294.12.patch, > HBASE-18294.13.patch, HBASE-18294.15.patch, HBASE-18294.16.patch > > > A region is flushed if its memory component exceed a threshold (default size > is 128MB). > A flush policy decides whether to flush a store by comparing the size of the > store to another threshold (that can be configured with > hbase.hregion.percolumnfamilyflush.size.lower.bound). > Currently the implementation (in both cases) compares the data size > (key-value only) to the threshold where it should compare the heap size > (which includes index size, and metadata). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18294) Reduce global heap pressure: flush based on heap occupancy
[ https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-18294: -- Attachment: HBASE-18294.01.patch > Reduce global heap pressure: flush based on heap occupancy > -- > > Key: HBASE-18294 > URL: https://issues.apache.org/jira/browse/HBASE-18294 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-18294.01.patch, HBASE-18294.01.patch, > HBASE-18294.02.patch, HBASE-18294.03.patch, HBASE-18294.04.patch, > HBASE-18294.05.patch, HBASE-18294.06.patch, HBASE-18294.07.patch, > HBASE-18294.07.patch, HBASE-18294.08.patch, HBASE-18294.09.patch, > HBASE-18294.10.patch, HBASE-18294.11.patch, HBASE-18294.11.patch, > HBASE-18294.12.patch, HBASE-18294.13.patch, HBASE-18294.15.patch, > HBASE-18294.16.patch > > > A region is flushed if its memory component exceed a threshold (default size > is 128MB). > A flush policy decides whether to flush a store by comparing the size of the > store to another threshold (that can be configured with > hbase.hregion.percolumnfamilyflush.size.lower.bound). > Currently the implementation (in both cases) compares the data size > (key-value only) to the threshold where it should compare the heap size > (which includes index size, and metadata). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18294) Reduce global heap pressure: flush based on heap occupancy
[ https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-18294: -- Attachment: HBASE-18294.master.01.patch > Reduce global heap pressure: flush based on heap occupancy > -- > > Key: HBASE-18294 > URL: https://issues.apache.org/jira/browse/HBASE-18294 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-18294.01.patch, HBASE-18294.01.patch, > HBASE-18294.02.patch, HBASE-18294.03.patch, HBASE-18294.04.patch, > HBASE-18294.05.patch, HBASE-18294.06.patch, HBASE-18294.07.patch, > HBASE-18294.07.patch, HBASE-18294.08.patch, HBASE-18294.09.patch, > HBASE-18294.10.patch, HBASE-18294.11.patch, HBASE-18294.11.patch, > HBASE-18294.12.patch, HBASE-18294.13.patch, HBASE-18294.15.patch, > HBASE-18294.16.patch, HBASE-18294.master.01.patch > > > A region is flushed if its memory component exceed a threshold (default size > is 128MB). > A flush policy decides whether to flush a store by comparing the size of the > store to another threshold (that can be configured with > hbase.hregion.percolumnfamilyflush.size.lower.bound). > Currently the implementation (in both cases) compares the data size > (key-value only) to the threshold where it should compare the heap size > (which includes index size, and metadata). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18294) Reduce global heap pressure: flush based on heap occupancy
[ https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-18294: -- Attachment: HBASE-18294.01.patch > Reduce global heap pressure: flush based on heap occupancy > -- > > Key: HBASE-18294 > URL: https://issues.apache.org/jira/browse/HBASE-18294 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-18294.01.patch, HBASE-18294.01.patch, > HBASE-18294.01.patch, HBASE-18294.02.patch, HBASE-18294.03.patch, > HBASE-18294.04.patch, HBASE-18294.05.patch, HBASE-18294.06.patch, > HBASE-18294.07.patch, HBASE-18294.07.patch, HBASE-18294.08.patch, > HBASE-18294.09.patch, HBASE-18294.10.patch, HBASE-18294.11.patch, > HBASE-18294.11.patch, HBASE-18294.12.patch, HBASE-18294.13.patch, > HBASE-18294.15.patch, HBASE-18294.16.patch, HBASE-18294.master.01.patch > > > A region is flushed if its memory component exceed a threshold (default size > is 128MB). > A flush policy decides whether to flush a store by comparing the size of the > store to another threshold (that can be configured with > hbase.hregion.percolumnfamilyflush.size.lower.bound). > Currently the implementation (in both cases) compares the data size > (key-value only) to the threshold where it should compare the heap size > (which includes index size, and metadata). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-18294) Reduce global heap pressure: flush based on heap occupancy
[ https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16337712#comment-16337712 ] Eshcar Hillel commented on HBASE-18294: --- The failed test passes on my machine. So indeed any additional feedback would be valuable before we commit this. Thanks. > Reduce global heap pressure: flush based on heap occupancy > -- > > Key: HBASE-18294 > URL: https://issues.apache.org/jira/browse/HBASE-18294 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-18294.01.patch, HBASE-18294.01.patch, > HBASE-18294.01.patch, HBASE-18294.02.patch, HBASE-18294.03.patch, > HBASE-18294.04.patch, HBASE-18294.05.patch, HBASE-18294.06.patch, > HBASE-18294.07.patch, HBASE-18294.07.patch, HBASE-18294.08.patch, > HBASE-18294.09.patch, HBASE-18294.10.patch, HBASE-18294.11.patch, > HBASE-18294.11.patch, HBASE-18294.12.patch, HBASE-18294.13.patch, > HBASE-18294.15.patch, HBASE-18294.16.patch, HBASE-18294.master.01.patch > > > A region is flushed if its memory component exceed a threshold (default size > is 128MB). > A flush policy decides whether to flush a store by comparing the size of the > store to another threshold (that can be configured with > hbase.hregion.percolumnfamilyflush.size.lower.bound). > Currently the implementation (in both cases) compares the data size > (key-value only) to the threshold where it should compare the heap size > (which includes index size, and metadata). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HBASE-14918) In-Memory MemStore Flush and Compaction
[ https://issues.apache.org/jira/browse/HBASE-14918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel resolved HBASE-14918. --- Resolution: Done > In-Memory MemStore Flush and Compaction > --- > > Key: HBASE-14918 > URL: https://issues.apache.org/jira/browse/HBASE-14918 > Project: HBase > Issue Type: Umbrella >Affects Versions: 2.0.0 >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel >Priority: Major > Attachments: CellBlocksSegmentDesign.pdf, > HBASE-16417-benchmarkresults.pdf, MSLABMove.patch > > > A memstore serves as the in-memory component of a store unit, absorbing all > updates to the store. From time to time these updates are flushed to a file > on disk, where they are compacted (by eliminating redundancies) and > compressed (i.e., written in a compressed format to reduce their storage > size). > We aim to speed up data access, and therefore suggest to apply in-memory > memstore flush. That is to flush the active in-memory segment into an > intermediate buffer where it can be accessed by the application. Data in the > buffer is subject to compaction and can be stored in any format that allows > it to take up smaller space in RAM. The less space the buffer consumes the > longer it can reside in memory before data is flushed to disk, resulting in > better performance. > Specifically, the optimization is beneficial for workloads with > medium-to-high key churn which incur many redundant cells, like persistent > messaging. > We suggest to structure the solution as 4 subtasks (respectively, patches). > (1) Infrastructure - refactoring of the MemStore hierarchy, introducing > segment (StoreSegment) as first-class citizen, and decoupling memstore > scanner from the memstore implementation; > (2) Adding StoreServices facility at the region level to allow memstores > update region counters and access region level synchronization mechanism; > (3) Implementation of a new memstore (CompactingMemstore) with non-optimized > immutable segment representation, and > (4) Memory optimization including compressed format representation and off > heap allocations. > This Jira continues the discussion in HBASE-13408. > Design documents, evaluation results and previous patches can be found in > HBASE-13408. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-17339) Scan-Memory-First Optimization for Get Operations
[ https://issues.apache.org/jira/browse/HBASE-17339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-17339: -- > Scan-Memory-First Optimization for Get Operations > - > > Key: HBASE-17339 > URL: https://issues.apache.org/jira/browse/HBASE-17339 > Project: HBase > Issue Type: Improvement >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel >Priority: Major > Attachments: HBASE-17339-V01.patch, HBASE-17339-V02.patch, > HBASE-17339-V03.patch, HBASE-17339-V03.patch, HBASE-17339-V04.patch, > HBASE-17339-V05.patch, HBASE-17339-V06.patch, read-latency-mixed-workload.jpg > > > The current implementation of a get operation (to retrieve values for a > specific key) scans through all relevant stores of the region; for each store > both memory components (memstores segments) and disk components (hfiles) are > scanned in parallel. > We suggest to apply an optimization that speculatively scans memory-only > components first and only if the result is incomplete scans both memory and > disk. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18294) Reduce global heap pressure: flush based on heap occupancy
[ https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-18294: -- Status: Open (was: Patch Available) > Reduce global heap pressure: flush based on heap occupancy > -- > > Key: HBASE-18294 > URL: https://issues.apache.org/jira/browse/HBASE-18294 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-18294.01.patch, HBASE-18294.01.patch, > HBASE-18294.01.patch, HBASE-18294.01.patch, HBASE-18294.02.patch, > HBASE-18294.03.patch, HBASE-18294.04.patch, HBASE-18294.05.patch, > HBASE-18294.06.patch, HBASE-18294.07.patch, HBASE-18294.07.patch, > HBASE-18294.08.patch, HBASE-18294.09.patch, HBASE-18294.10.patch, > HBASE-18294.11.patch, HBASE-18294.11.patch, HBASE-18294.12.patch, > HBASE-18294.13.patch, HBASE-18294.15.patch, HBASE-18294.16.patch, > HBASE-18294.master.01.patch > > > A region is flushed if its memory component exceed a threshold (default size > is 128MB). > A flush policy decides whether to flush a store by comparing the size of the > store to another threshold (that can be configured with > hbase.hregion.percolumnfamilyflush.size.lower.bound). > Currently the implementation (in both cases) compares the data size > (key-value only) to the threshold where it should compare the heap size > (which includes index size, and metadata). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-18294) Reduce global heap pressure: flush based on heap occupancy
[ https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16351674#comment-16351674 ] Eshcar Hillel commented on HBASE-18294: --- Here is my response from RB regarding the change in HCD In the new implementation we compare on-heap size against on-heap threshold and off-heap size against off-heap threshold. Exceeding one of these two thresholds triggers a flush. If we only consider one unified threshold for on- and off-heap data we can get OOME. For example, consider an off-heap cluster with 96GB memory machines, and assume in each machine 90GB are allocated to off-heap data and 6GB are allocated to on-heap objects. Assume we manage 700 regions per RS, and the flush size is 128MB. It only requires 46 regions to get to 128MB on-heap or for all regions to exceed 8.5MB(!) on-heap to get an OOME. Therefore I think giving the user an optional(!) control on the flush size both for on-heap and off-heap sizes is essential. And they both fall back on default value so they are both optional and may not be an unnecessary burden on the user. I also see I need to re-base my patch. Waiting for review comments to create my next patch. > Reduce global heap pressure: flush based on heap occupancy > -- > > Key: HBASE-18294 > URL: https://issues.apache.org/jira/browse/HBASE-18294 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-18294.01.patch, HBASE-18294.01.patch, > HBASE-18294.01.patch, HBASE-18294.01.patch, HBASE-18294.02.patch, > HBASE-18294.03.patch, HBASE-18294.04.patch, HBASE-18294.05.patch, > HBASE-18294.06.patch, HBASE-18294.07.patch, HBASE-18294.07.patch, > HBASE-18294.08.patch, HBASE-18294.09.patch, HBASE-18294.10.patch, > HBASE-18294.11.patch, HBASE-18294.11.patch, HBASE-18294.12.patch, > HBASE-18294.13.patch, HBASE-18294.15.patch, HBASE-18294.16.patch, > HBASE-18294.master.01.patch > > > A region is flushed if its memory component exceed a threshold (default size > is 128MB). > A flush policy decides whether to flush a store by comparing the size of the > store to another threshold (that can be configured with > hbase.hregion.percolumnfamilyflush.size.lower.bound). > Currently the implementation (in both cases) compares the data size > (key-value only) to the threshold where it should compare the heap size > (which includes index size, and metadata). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-18294) Reduce global heap pressure: flush based on heap occupancy
[ https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16352186#comment-16352186 ] Eshcar Hillel commented on HBASE-18294: --- I would generally agree with your approach [~chancelq] - simple abstraction are usually the right way to go. However, the user controls the sizes of memory allocated on-heap and off-heap and if we are not care to respect both limits we may end up crossing one of them and getting OOME even if the "global" memory is not exhausted. Specifically, you can review the discussion from December 28 above. > Reduce global heap pressure: flush based on heap occupancy > -- > > Key: HBASE-18294 > URL: https://issues.apache.org/jira/browse/HBASE-18294 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-18294.01.patch, HBASE-18294.01.patch, > HBASE-18294.01.patch, HBASE-18294.01.patch, HBASE-18294.02.patch, > HBASE-18294.03.patch, HBASE-18294.04.patch, HBASE-18294.05.patch, > HBASE-18294.06.patch, HBASE-18294.07.patch, HBASE-18294.07.patch, > HBASE-18294.08.patch, HBASE-18294.09.patch, HBASE-18294.10.patch, > HBASE-18294.11.patch, HBASE-18294.11.patch, HBASE-18294.12.patch, > HBASE-18294.13.patch, HBASE-18294.15.patch, HBASE-18294.16.patch, > HBASE-18294.master.01.patch > > > A region is flushed if its memory component exceed a threshold (default size > is 128MB). > A flush policy decides whether to flush a store by comparing the size of the > store to another threshold (that can be configured with > hbase.hregion.percolumnfamilyflush.size.lower.bound). > Currently the implementation (in both cases) compares the data size > (key-value only) to the threshold where it should compare the heap size > (which includes index size, and metadata). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-18294) Flush policy checks data size instead of heap size
Eshcar Hillel created HBASE-18294: - Summary: Flush policy checks data size instead of heap size Key: HBASE-18294 URL: https://issues.apache.org/jira/browse/HBASE-18294 Project: HBase Issue Type: Bug Reporter: Eshcar Hillel Assignee: Eshcar Hillel A flush policy decides whether to flush a store by comparing the size of the store to a threshold (that can be configured with hbase.hregion.percolumnfamilyflush.size.lower.bound). Currently the implementation compares the data size (key-value only) to the threshold where it should compare the heap size (which includes index size, and metadata). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18294) Flush policy checks data size instead of heap size
[ https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16068102#comment-16068102 ] Eshcar Hillel commented on HBASE-18294: --- While benchmarking, we noticed the number of flushes in stores with basic compaction is similar to the number of flushes in stores with no compaction. We expected the number of flushes with basic compaction to be smaller as the index is smaller and therefore the heap size is smaller and the memstore can absorb more data before it is required to flush to disk. This is a very small change, I will make a patch later today. However, it might have significant affect on performance, and therefore should be at least considered also for 2.0.0. > Flush policy checks data size instead of heap size > -- > > Key: HBASE-18294 > URL: https://issues.apache.org/jira/browse/HBASE-18294 > Project: HBase > Issue Type: Bug >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel > > A flush policy decides whether to flush a store by comparing the size of the > store to a threshold (that can be configured with > hbase.hregion.percolumnfamilyflush.size.lower.bound). > Currently the implementation compares the data size (key-value only) to the > threshold where it should compare the heap size (which includes index size, > and metadata). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18294) Flush policy checks data size instead of heap size
[
https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16068379#comment-16068379
]
Eshcar Hillel commented on HBASE-18294:
---
Currently the total memstore size in a region also measures only the data size
and not overall heap size:
{code}
long size = this.memstoreDataSize.addAndGet(memstoreSize.getDataSize());
{code}
I think this is wrong.
When the data is flushed to disk
(1) metadata changes
(2) data is compacted
(3) data is compressed
All these can cause the file to be smaller than 128MB so the user cannot expect
to get files of certain sizes.
The purpose of a flush is to release memory when the memory exceeds the
threshold. It does not guarantee creating files of minimal sizes.
If indeed there are cases where the data only takes half of the memory or even
less and the rest is used for index and metadata, and we only check if data
exceeds 128MB than we may use double the size of the memory.
As a result
(*) the system can exceed the blocking threshold (where updates are blocked)
more frequently than we would like, and
(*) we can even get out-of-memory-exceptoion in certain scenarios.
I think a better practice would be to respect the thresholds while considering
the total heap size -- both in region level when triggering a flush, and in
store level when selecting stores to be flushed.
> Flush policy checks data size instead of heap size
> --
>
> Key: HBASE-18294
> URL: https://issues.apache.org/jira/browse/HBASE-18294
> Project: HBase
> Issue Type: Bug
>Reporter: Eshcar Hillel
>Assignee: Eshcar Hillel
>
> A flush policy decides whether to flush a store by comparing the size of the
> store to a threshold (that can be configured with
> hbase.hregion.percolumnfamilyflush.size.lower.bound).
> Currently the implementation compares the data size (key-value only) to the
> threshold where it should compare the heap size (which includes index size,
> and metadata).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18294) Flush is based on data size instead of heap size
[ https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-18294: -- Summary: Flush is based on data size instead of heap size (was: Flush policy checks data size instead of heap size) > Flush is based on data size instead of heap size > > > Key: HBASE-18294 > URL: https://issues.apache.org/jira/browse/HBASE-18294 > Project: HBase > Issue Type: Bug >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel > > A flush policy decides whether to flush a store by comparing the size of the > store to a threshold (that can be configured with > hbase.hregion.percolumnfamilyflush.size.lower.bound). > Currently the implementation compares the data size (key-value only) to the > threshold where it should compare the heap size (which includes index size, > and metadata). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18294) Flush is based on data size instead of heap size
[ https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-18294: -- Description: A region is flushed if its memory component exceed a threshold (default size is 128MB). A flush policy decides whether to flush a store by comparing the size of the store to another threshold (that can be configured with hbase.hregion.percolumnfamilyflush.size.lower.bound). Currently the implementation (in both cases) compares the data size (key-value only) to the threshold where it should compare the heap size (which includes index size, and metadata). was: A flush policy decides whether to flush a store by comparing the size of the store to a threshold (that can be configured with hbase.hregion.percolumnfamilyflush.size.lower.bound). Currently the implementation compares the data size (key-value only) to the threshold where it should compare the heap size (which includes index size, and metadata). > Flush is based on data size instead of heap size > > > Key: HBASE-18294 > URL: https://issues.apache.org/jira/browse/HBASE-18294 > Project: HBase > Issue Type: Bug >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel > > A region is flushed if its memory component exceed a threshold (default size > is 128MB). > A flush policy decides whether to flush a store by comparing the size of the > store to another threshold (that can be configured with > hbase.hregion.percolumnfamilyflush.size.lower.bound). > Currently the implementation (in both cases) compares the data size > (key-value only) to the threshold where it should compare the heap size > (which includes index size, and metadata). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18294) Flush is based on data size instead of heap size
[ https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16071594#comment-16071594 ] Eshcar Hillel commented on HBASE-18294: --- Considering heap size for triggering global heap pressure reduction is fine, however, we expect the system to get to this point rarely. As you mention in your document disregarding heap size can cause blocking, excessive GC and even OOME. Following HBASE-16747 there are 3 cases where considering data size and not heap size (1) when choosing biggest region to flush following global heap pressure (2) when choosing large stores to flush within the region (3) when deciding to trigger region level flush (exceeding the 128MB threshold). Lets consider the on-heap scenario first: For (1)+(2) it is already decided that there is a flush and now we are looking to choose biggest memory consumers (biggest region/largest store). Therefore we should compare heap size. Otherwise, we may choose a smaller region/store simply because its data is bigger but it may actually take less heap space. For (3) the configurable threshold gives the admin a way to compute how many regions the RS can handle with a given memory. That was documented in blogs etc. Now with the change we actually have no way to predict what is the maximum memory size of a region. This may come as a surprise to some admins. bq. For the off heap based memstores also the data size alone based region flush is better. bq. Offheap memstores need this change so that we are able to decide the region flush based on data size alone as the whole data is offheap. Then lets make a different policy for off-heap cases. Enforcing the same policy for on/off-heap doesn't seem to be optimal for both. I suggest to make different policies for on-heap data (to consider heap size in all the cases above), and off-heap that considers heap/data size any way you think is best for off-heap. > Flush is based on data size instead of heap size > > > Key: HBASE-18294 > URL: https://issues.apache.org/jira/browse/HBASE-18294 > Project: HBase > Issue Type: Bug >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel > > A region is flushed if its memory component exceed a threshold (default size > is 128MB). > A flush policy decides whether to flush a store by comparing the size of the > store to another threshold (that can be configured with > hbase.hregion.percolumnfamilyflush.size.lower.bound). > Currently the implementation (in both cases) compares the data size > (key-value only) to the threshold where it should compare the heap size > (which includes index size, and metadata). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18294) Flush is based on data size instead of heap size
[ https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16072467#comment-16072467 ] Eshcar Hillel commented on HBASE-18294: --- The per-cell metadata overhead with basic compaction is about 60B. This means that with small values metadata can easily double the heap occupancy of the memstore. bq. When the data size of the memstore is high (choosing one to flush), the heap occupancy of it also will be on higher side no? Not necessarily. It depends on the size of the values. one memstore can have 50MB data and 100MB total heap size while another memstore has 70MB data and only 80MB total heap size. bq. considering data size only for the per region flush decision is more inline with a normal user thinking. Not necessarily. Here are 2 blogs which consider 128MB to be the total memstore occupancy: https://www.quora.com/HBase-Region-Server-guidelines-give-a-size-range-of-about-1TB-whereas-data-nodes-are-configured-20-times-bigger-Why by [~larsgeorge] _"we can address 4GB of heap for writes, allowing us up to have 32 regions that are written to while flushing at 128MB"_ http://hadoop-hbase.blogspot.co.uk/2013/01/hbase-region-server-memory-sizing.html by [~lhofhansl] _"... you'd need ~338 regions. @128MB that's about 43GB"_ Users who came to understand HBase through this kind of blogs expect a region (memstore) not to exceed the 128MB threshold. With the current implementation all places including regular and blocking thresholds at the region level and both selection policies at the RS-region and region-store level consider data size and not heap size. > Flush is based on data size instead of heap size > > > Key: HBASE-18294 > URL: https://issues.apache.org/jira/browse/HBASE-18294 > Project: HBase > Issue Type: Bug >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel > > A region is flushed if its memory component exceed a threshold (default size > is 128MB). > A flush policy decides whether to flush a store by comparing the size of the > store to another threshold (that can be configured with > hbase.hregion.percolumnfamilyflush.size.lower.bound). > Currently the implementation (in both cases) compares the data size > (key-value only) to the threshold where it should compare the heap size > (which includes index size, and metadata). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18294) Flush is based on data size instead of heap size
[ https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16073265#comment-16073265 ] Eshcar Hillel commented on HBASE-18294: --- bq. We should change the 128 MB flush size default value then? This is an orthogonal discussion. I don't have a strong opinion on whether it should be higher or lower. Plus the user has the ability to control this parameter. The discussion here is what this parameter is limiting data size or heap size. bq. When to be flushed, flushing the one with max data size in it make sense no? For me flushing the one with max heap space makes more sense. bq. Any way after the compacting memstore, we are reducing the heap overhead in btw by in memory flushes. For compacting memstore the per-cell metadata overhead is about 60B, which can be 50% of the heap size for 60B cells and 10% of the heap for 600B cells. So there are cases where the metadata is not negligible. > Flush is based on data size instead of heap size > > > Key: HBASE-18294 > URL: https://issues.apache.org/jira/browse/HBASE-18294 > Project: HBase > Issue Type: Bug >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel > > A region is flushed if its memory component exceed a threshold (default size > is 128MB). > A flush policy decides whether to flush a store by comparing the size of the > store to another threshold (that can be configured with > hbase.hregion.percolumnfamilyflush.size.lower.bound). > Currently the implementation (in both cases) compares the data size > (key-value only) to the threshold where it should compare the heap size > (which includes index size, and metadata). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18010) Connect CellChunkMap to be used for flattening in CompactingMemStore
[ https://issues.apache.org/jira/browse/HBASE-18010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16074450#comment-16074450 ] Eshcar Hillel commented on HBASE-18010: --- lgtm +1 > Connect CellChunkMap to be used for flattening in CompactingMemStore > > > Key: HBASE-18010 > URL: https://issues.apache.org/jira/browse/HBASE-18010 > Project: HBase > Issue Type: Sub-task >Reporter: Anastasia Braginsky >Assignee: Anastasia Braginsky > Attachments: HBASE-18010-V04.patch, HBASE-18010-V06.patch, > HBASE-18010-V07.patch, HBASE-18010-V08.patch, HBASE-18010-V09.patch, > HBASE-18010-V10.patch, HBASE-18010-V11.patch > > > The CellChunkMap helps to create a new type of ImmutableSegment, where the > index (CellSet's delegatee) is going to be CellChunkMap. No big cells or > upserted cells are going to be supported here. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18375) The pool chunks from ChunkCreator are deallocated while in pool because there is no reference to them
[ https://issues.apache.org/jira/browse/HBASE-18375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16121672#comment-16121672 ] Eshcar Hillel commented on HBASE-18375: --- lgtm left some comments in review board +1 > The pool chunks from ChunkCreator are deallocated while in pool because there > is no reference to them > - > > Key: HBASE-18375 > URL: https://issues.apache.org/jira/browse/HBASE-18375 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0-alpha-1 >Reporter: Anastasia Braginsky >Assignee: Anastasia Braginsky >Priority: Critical > Fix For: 2.0.0, 3.0.0, 2.0.0-alpha-2 > > Attachments: HBASE-18375-V01.patch, HBASE-18375-V02.patch, > HBASE-18375-V03.patch, HBASE-18375-V04.patch, HBASE-18375-V05.patch > > > Because MSLAB list of chunks was changed to list of chunk IDs, the chunks > returned back to pool can be deallocated by JVM because there is no reference > to them. The solution is to protect pool chunks from GC by the strong map of > ChunkCreator introduced by HBASE-18010. Will prepare the patch today. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18294) Reduce global heap pressure: flush based on heap occupancy
[
https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16356649#comment-16356649
]
Eshcar Hillel commented on HBASE-18294:
---
[~anoop.hbase]
{quote} the general agreement we made is for on heap cases, we must continue to
check for 128 MB limit against the memstore heap size. Not just data size. Also
we have agreed that for off heap also, we will consider the off heap size +
heap overhead.
{quote}
>From the beginning I aimed to have as symmetric behavior as possible of
>on-heap and off-heap cases, so I don't believe I agreed on having two
>different computations. One way to make it symmetric is to compare the two
>counters against two thresholds. Another way to unify it is to always consider
>the sum of off-heap and on-heap sizes at the region level. We still need to
>manage two separate counters since the global bounds are different.
bq. Ideally checking the data size alone here would have been the best way. I
mean for any decision per region level.
You keep saying that but it seems to be based more on intuition rather than on
experiments. While considering both data and heap overhead for region level
flush have shown to improve the performance significantly.
bq.When the size breach is because of off heap size, we have to select regions
having maximum data size and when breach because of on heap size limit, select
the regions with more heap overhead.
Again, Why? you say we should have different decision making but you don't
explain why, and don't have numbers to support your claims.
I argue that unless shown there is a great performance benefit in making
different rules, on-heap and off heap should follow the same set of rules,
embedding them with their respective bounds.
So, I will make a new patch, leave only one flush size configuration property
(remove off-heap flush size), flush size at the region level will always
consider on-heap+off-heap size. The rest will be similar to the current patch.
Patch will be ready in a few days.
> Reduce global heap pressure: flush based on heap occupancy
> --
>
> Key: HBASE-18294
> URL: https://issues.apache.org/jira/browse/HBASE-18294
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0
>Reporter: Eshcar Hillel
>Assignee: Eshcar Hillel
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-18294.01.patch, HBASE-18294.01.patch,
> HBASE-18294.01.patch, HBASE-18294.01.patch, HBASE-18294.02.patch,
> HBASE-18294.03.patch, HBASE-18294.04.patch, HBASE-18294.05.patch,
> HBASE-18294.06.patch, HBASE-18294.07.patch, HBASE-18294.07.patch,
> HBASE-18294.08.patch, HBASE-18294.09.patch, HBASE-18294.10.patch,
> HBASE-18294.11.patch, HBASE-18294.11.patch, HBASE-18294.12.patch,
> HBASE-18294.13.patch, HBASE-18294.15.patch, HBASE-18294.16.patch,
> HBASE-18294.master.01.patch
>
>
> A region is flushed if its memory component exceed a threshold (default size
> is 128MB).
> A flush policy decides whether to flush a store by comparing the size of the
> store to another threshold (that can be configured with
> hbase.hregion.percolumnfamilyflush.size.lower.bound).
> Currently the implementation (in both cases) compares the data size
> (key-value only) to the threshold where it should compare the heap size
> (which includes index size, and metadata).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-18294) Reduce global heap pressure: flush based on heap occupancy
[ https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-18294: -- Attachment: HBASE-18294.master.01.patch > Reduce global heap pressure: flush based on heap occupancy > -- > > Key: HBASE-18294 > URL: https://issues.apache.org/jira/browse/HBASE-18294 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-18294.01.patch, HBASE-18294.01.patch, > HBASE-18294.01.patch, HBASE-18294.01.patch, HBASE-18294.02.patch, > HBASE-18294.03.patch, HBASE-18294.04.patch, HBASE-18294.05.patch, > HBASE-18294.06.patch, HBASE-18294.07.patch, HBASE-18294.07.patch, > HBASE-18294.08.patch, HBASE-18294.09.patch, HBASE-18294.10.patch, > HBASE-18294.11.patch, HBASE-18294.11.patch, HBASE-18294.12.patch, > HBASE-18294.13.patch, HBASE-18294.15.patch, HBASE-18294.16.patch, > HBASE-18294.master.01.patch, HBASE-18294.master.01.patch > > > A region is flushed if its memory component exceed a threshold (default size > is 128MB). > A flush policy decides whether to flush a store by comparing the size of the > store to another threshold (that can be configured with > hbase.hregion.percolumnfamilyflush.size.lower.bound). > Currently the implementation (in both cases) compares the data size > (key-value only) to the threshold where it should compare the heap size > (which includes index size, and metadata). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18294) Reduce global heap pressure: flush based on heap occupancy
[ https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-18294: -- Status: Patch Available (was: Open) > Reduce global heap pressure: flush based on heap occupancy > -- > > Key: HBASE-18294 > URL: https://issues.apache.org/jira/browse/HBASE-18294 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-18294.01.patch, HBASE-18294.01.patch, > HBASE-18294.01.patch, HBASE-18294.01.patch, HBASE-18294.01.patch, > HBASE-18294.02.patch, HBASE-18294.03.patch, HBASE-18294.04.patch, > HBASE-18294.05.patch, HBASE-18294.06.patch, HBASE-18294.07.patch, > HBASE-18294.07.patch, HBASE-18294.08.patch, HBASE-18294.09.patch, > HBASE-18294.10.patch, HBASE-18294.11.patch, HBASE-18294.11.patch, > HBASE-18294.12.patch, HBASE-18294.13.patch, HBASE-18294.15.patch, > HBASE-18294.16.patch, HBASE-18294.master.01.patch, HBASE-18294.master.01.patch > > > A region is flushed if its memory component exceed a threshold (default size > is 128MB). > A flush policy decides whether to flush a store by comparing the size of the > store to another threshold (that can be configured with > hbase.hregion.percolumnfamilyflush.size.lower.bound). > Currently the implementation (in both cases) compares the data size > (key-value only) to the threshold where it should compare the heap size > (which includes index size, and metadata). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18294) Reduce global heap pressure: flush based on heap occupancy
[ https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-18294: -- Attachment: HBASE-18294.01.patch > Reduce global heap pressure: flush based on heap occupancy > -- > > Key: HBASE-18294 > URL: https://issues.apache.org/jira/browse/HBASE-18294 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-18294.01.patch, HBASE-18294.01.patch, > HBASE-18294.01.patch, HBASE-18294.01.patch, HBASE-18294.01.patch, > HBASE-18294.02.patch, HBASE-18294.03.patch, HBASE-18294.04.patch, > HBASE-18294.05.patch, HBASE-18294.06.patch, HBASE-18294.07.patch, > HBASE-18294.07.patch, HBASE-18294.08.patch, HBASE-18294.09.patch, > HBASE-18294.10.patch, HBASE-18294.11.patch, HBASE-18294.11.patch, > HBASE-18294.12.patch, HBASE-18294.13.patch, HBASE-18294.15.patch, > HBASE-18294.16.patch, HBASE-18294.master.01.patch, HBASE-18294.master.01.patch > > > A region is flushed if its memory component exceed a threshold (default size > is 128MB). > A flush policy decides whether to flush a store by comparing the size of the > store to another threshold (that can be configured with > hbase.hregion.percolumnfamilyflush.size.lower.bound). > Currently the implementation (in both cases) compares the data size > (key-value only) to the threshold where it should compare the heap size > (which includes index size, and metadata). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-18294) Reduce global heap pressure: flush based on heap occupancy
[ https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eshcar Hillel updated HBASE-18294: -- Attachment: HBASE-18294.01.patch > Reduce global heap pressure: flush based on heap occupancy > -- > > Key: HBASE-18294 > URL: https://issues.apache.org/jira/browse/HBASE-18294 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: Eshcar Hillel >Assignee: Eshcar Hillel >Priority: Major > Fix For: 2.0.0-beta-2 > > Attachments: HBASE-18294.01.patch, HBASE-18294.01.patch, > HBASE-18294.01.patch, HBASE-18294.01.patch, HBASE-18294.01.patch, > HBASE-18294.01.patch, HBASE-18294.02.patch, HBASE-18294.03.patch, > HBASE-18294.04.patch, HBASE-18294.05.patch, HBASE-18294.06.patch, > HBASE-18294.07.patch, HBASE-18294.07.patch, HBASE-18294.08.patch, > HBASE-18294.09.patch, HBASE-18294.10.patch, HBASE-18294.11.patch, > HBASE-18294.11.patch, HBASE-18294.12.patch, HBASE-18294.13.patch, > HBASE-18294.15.patch, HBASE-18294.16.patch, HBASE-18294.master.01.patch, > HBASE-18294.master.01.patch > > > A region is flushed if its memory component exceed a threshold (default size > is 128MB). > A flush policy decides whether to flush a store by comparing the size of the > store to another threshold (that can be configured with > hbase.hregion.percolumnfamilyflush.size.lower.bound). > Currently the implementation (in both cases) compares the data size > (key-value only) to the threshold where it should compare the heap size > (which includes index size, and metadata). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-18294) Reduce global heap pressure: flush based on heap occupancy
[
https://issues.apache.org/jira/browse/HBASE-18294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16360087#comment-16360087
]
Eshcar Hillel commented on HBASE-18294:
---
TestClientPushback is a flaky test, sometimes passes sometimes fails; I am not
sure if its failure is related to this patch.
In the test the load of the region is calculated by asking the region for its
heap size, and there is a comment above the calculation saying
{color:#808080}"get the current load on RS. Hopefully memstore isn't flushed
since we wrote the the data"{color} {color:#33}(Hopefully ??!?), then this
load is compared against the stats returned from the server and if they are not
equal the test fails. The comment says "{color}{color:#808080}check to see we
found some load on the memstore"{color}{color:#33} I guess it was meant to
be "same load" but in some cases a flush causes the
difference.{color}{color:#808080}
{color}
{color:#33}It could be that due to this patch the memstore is flushed
earlier than before and so the test fails more frequently, but it seems to be
unreasonable to *hope* for no flush during the test.{color}
{color:#33}My solution was simply to double the flush size for this test.
Any other more elegant suggestions for handling this?{color}
> Reduce global heap pressure: flush based on heap occupancy
> --
>
> Key: HBASE-18294
> URL: https://issues.apache.org/jira/browse/HBASE-18294
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0
>Reporter: Eshcar Hillel
>Assignee: Eshcar Hillel
>Priority: Major
> Fix For: 2.0.0-beta-2
>
> Attachments: HBASE-18294.01.patch, HBASE-18294.01.patch,
> HBASE-18294.01.patch, HBASE-18294.01.patch, HBASE-18294.01.patch,
> HBASE-18294.01.patch, HBASE-18294.02.patch, HBASE-18294.03.patch,
> HBASE-18294.04.patch, HBASE-18294.05.patch, HBASE-18294.06.patch,
> HBASE-18294.07.patch, HBASE-18294.07.patch, HBASE-18294.08.patch,
> HBASE-18294.09.patch, HBASE-18294.10.patch, HBASE-18294.11.patch,
> HBASE-18294.11.patch, HBASE-18294.12.patch, HBASE-18294.13.patch,
> HBASE-18294.15.patch, HBASE-18294.16.patch, HBASE-18294.master.01.patch,
> HBASE-18294.master.01.patch
>
>
> A region is flushed if its memory component exceed a threshold (default size
> is 128MB).
> A flush policy decides whether to flush a store by comparing the size of the
> store to another threshold (that can be configured with
> hbase.hregion.percolumnfamilyflush.size.lower.bound).
> Currently the implementation (in both cases) compares the data size
> (key-value only) to the threshold where it should compare the heap size
> (which includes index size, and metadata).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
