[
https://issues.apache.org/jira/browse/CASSANDRA-7546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14141858#comment-14141858
]
Benedict edited comment on CASSANDRA-7546 at 9/20/14 7:54 AM:
--------------------------------------------------------------
In general the idea for the auto mode is to get a general overview of the
various conditions, _especially_ when run in target uncertainty (err<) mode,
which is the default mode. I've just committed a minor change, that was
previously talked about, that supports running all thread counts in the range
unconditionally, however it will log a warning if you run this with target
uncertainty mode, as the workloads will be different.
Really we should be tearing down and rebuilding the cluster between runs.
However it looks like the results are pretty much a wash for all modes except
those where high contention on a single partition is to be expected. It's a bit
strange that .999%ile is higher with the patch for the highest thread counts
but lower contention, but that may be noise. Certainly the heap reduction looks
promising.
It would be great to see runs where the workload is consistently defined (i.e.
n=, instead of err<), and to get the raw output
was (Author: benedict):
In general the idea for the auto mode is to get a general overview of the
various conditions, _especially_ when run in target uncertainty (err<) mode,
which is the default mode. I've just committed a minor change, that was
previously talked about, that supports running all thread counts in the range
unconditionally, however it will log a warning if you run this with target
uncertainty mode, as the workloads will be different.
Really we should be tearing down and rebuilding the cluster between runs.
However it looks like the results are pretty much a wash for all modes except
those where high contention on a single partition is to be expected. It's a bit
strange that .999%ile is higher with the patch for the highest thread counts
but lower contention, but that may be noise. Certainly the heap reduction looks
promising.
> AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory
> -----------------------------------------------------------------------------
>
> Key: CASSANDRA-7546
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7546
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Reporter: graham sanderson
> Assignee: graham sanderson
> Fix For: 2.1.1
>
> Attachments: 7546.20.txt, 7546.20_2.txt, 7546.20_3.txt,
> 7546.20_4.txt, 7546.20_5.txt, 7546.20_6.txt, 7546.20_7.txt, 7546.20_7b.txt,
> 7546.20_alt.txt, 7546.20_async.txt, 7546.21_v1.txt, graphs1.png,
> hint_spikes.png, suggestion1.txt, suggestion1_21.txt, young_gen_gc.png
>
>
> In order to preserve atomicity, this code attempts to read, clone/update,
> then CAS the state of the partition.
> Under heavy contention for updating a single partition this can cause some
> fairly staggering memory growth (the more cores on your machine the worst it
> gets).
> Whilst many usage patterns don't do highly concurrent updates to the same
> partition, hinting today, does, and in this case wild (order(s) of magnitude
> more than expected) memory allocation rates can be seen (especially when the
> updates being hinted are small updates to different partitions which can
> happen very fast on their own) - see CASSANDRA-7545
> It would be best to eliminate/reduce/limit the spinning memory allocation
> whilst not slowing down the very common un-contended case.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
