[jira] [Commented] (CASSANDRA-12358) Slow PostFlush execution due to 2i flushing can cause near OOM to OOM
[ https://issues.apache.org/jira/browse/CASSANDRA-12358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15413774#comment-15413774 ] Aleksey Yeschenko commented on CASSANDRA-12358: --- Committed as [4878852fe4aae3516c21fdeafac5c5746a93c31f|https://github.com/apache/cassandra/commit/4878852fe4aae3516c21fdeafac5c5746a93c31f] to trunk, thanks. > Slow PostFlush execution due to 2i flushing can cause near OOM to OOM > - > > Key: CASSANDRA-12358 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12358 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Ariel Weisberg >Assignee: Ariel Weisberg > Fix For: 3.10 > > > 2i can be slow to flush for a variety of reasons. Potentially slower than the > rate at which Memtables can ingest and flush data. If this occurs the heap > fills up with Memtables that are waiting for PostFlush to run. > This occurs because reclaiming the memory is done before PostFlush runs. > I will post a branch that has the reclaim memory task run after PostFlush has > completed. As far as I can tell this is safe and correct since the memory is > committed up until that point. > It's not clear to me if PostFlush has to bind the Memtables or not. I suspect > it does, but I'm not sure if that is a route I should go down. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12358) Slow PostFlush execution due to 2i flushing can cause near OOM to OOM
[ https://issues.apache.org/jira/browse/CASSANDRA-12358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15413756#comment-15413756 ] Branimir Lambov commented on CASSANDRA-12358: - The dtest didn't actually run correctly (just a fraction of the tests ran). The next run is fine, though (one fail, same as trunk). LGTM > Slow PostFlush execution due to 2i flushing can cause near OOM to OOM > - > > Key: CASSANDRA-12358 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12358 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Ariel Weisberg >Assignee: Ariel Weisberg > Fix For: 3.10 > > > 2i can be slow to flush for a variety of reasons. Potentially slower than the > rate at which Memtables can ingest and flush data. If this occurs the heap > fills up with Memtables that are waiting for PostFlush to run. > This occurs because reclaiming the memory is done before PostFlush runs. > I will post a branch that has the reclaim memory task run after PostFlush has > completed. As far as I can tell this is safe and correct since the memory is > committed up until that point. > It's not clear to me if PostFlush has to bind the Memtables or not. I suspect > it does, but I'm not sure if that is a route I should go down. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12358) Slow PostFlush execution due to 2i flushing can cause near OOM to OOM
[ https://issues.apache.org/jira/browse/CASSANDRA-12358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15412576#comment-15412576 ] Ariel Weisberg commented on CASSANDRA-12358: Double blue! Hats off to the C* team that is awesome. > Slow PostFlush execution due to 2i flushing can cause near OOM to OOM > - > > Key: CASSANDRA-12358 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12358 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Ariel Weisberg >Assignee: Ariel Weisberg > Fix For: 3.10 > > > 2i can be slow to flush for a variety of reasons. Potentially slower than the > rate at which Memtables can ingest and flush data. If this occurs the heap > fills up with Memtables that are waiting for PostFlush to run. > This occurs because reclaiming the memory is done before PostFlush runs. > I will post a branch that has the reclaim memory task run after PostFlush has > completed. As far as I can tell this is safe and correct since the memory is > committed up until that point. > It's not clear to me if PostFlush has to bind the Memtables or not. I suspect > it does, but I'm not sure if that is a route I should go down. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12358) Slow PostFlush execution due to 2i flushing can cause near OOM to OOM
[ https://issues.apache.org/jira/browse/CASSANDRA-12358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15411441#comment-15411441 ] Branimir Lambov commented on CASSANDRA-12358: - Looks good. > Slow PostFlush execution due to 2i flushing can cause near OOM to OOM > - > > Key: CASSANDRA-12358 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12358 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Ariel Weisberg >Assignee: Ariel Weisberg > Fix For: 3.10 > > > 2i can be slow to flush for a variety of reasons. Potentially slower than the > rate at which Memtables can ingest and flush data. If this occurs the heap > fills up with Memtables that are waiting for PostFlush to run. > This occurs because reclaiming the memory is done before PostFlush runs. > I will post a branch that has the reclaim memory task run after PostFlush has > completed. As far as I can tell this is safe and correct since the memory is > committed up until that point. > It's not clear to me if PostFlush has to bind the Memtables or not. I suspect > it does, but I'm not sure if that is a route I should go down. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12358) Slow PostFlush execution due to 2i flushing can cause near OOM to OOM
[ https://issues.apache.org/jira/browse/CASSANDRA-12358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15403128#comment-15403128 ] Ariel Weisberg commented on CASSANDRA-12358: |[trunk|https://github.com/apache/cassandra/compare/trunk...aweisberg:CASSANDRA-12358-trunk?expand=1]|[utests|https://cassci.datastax.com/view/Dev/view/aweisberg/job/aweisberg-CASSANDRA-12358-trunk-testall/1/]|[dtests|https://cassci.datastax.com/view/Dev/view/aweisberg/job/aweisberg-CASSANDRA-12358-trunk-dtest/1/]| > Slow PostFlush execution due to 2i flushing can cause near OOM to OOM > - > > Key: CASSANDRA-12358 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12358 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Ariel Weisberg >Assignee: Ariel Weisberg > Fix For: 3.10 > > > 2i can be slow to flush for a variety of reasons. Potentially slower than the > rate at which Memtables can ingest and flush data. If this occurs the heap > fills up with Memtables that are waiting for PostFlush to run. > This occurs because reclaiming the memory is done before PostFlush runs. > I will post a branch that has the reclaim memory task run after PostFlush has > completed. As far as I can tell this is safe and correct since the memory is > committed up until that point. > It's not clear to me if PostFlush has to bind the Memtables or not. I suspect > it does, but I'm not sure if that is a route I should go down. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
