[
https://issues.apache.org/jira/browse/CASSANDRA-15388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16993136#comment-16993136
]
Benedict Elliott Smith commented on CASSANDRA-15388:
----------------------------------------------------
With absolutely no preconceived answer to this question on my part: have you
considered and rejected JMH's GC profiler?
I'm unsure how the approaches compare with respect to sophistication (I haven't
looked closely at how either underlying tool is implemented), but having played
with it myself recently, using jmh 1.22, it seems to produce exactly the
numbers I would calculate manually for small functions performing easily
calculated allocations.
I only ask because it would hook into our (minimal) pre-existing benchmark
workflows, and produce throughput/latency data alongside. The latest version
of jmh can even neatly hook into perf counters to produce very sophisticated
ancillary data to consider at the same time, to weigh the pros/cons of a change
(although it's definitely not as easy to explore as a flamegraph).
> Add compaction allocation measurement test to support compaction gc
> optimization.
> ----------------------------------------------------------------------------------
>
> Key: CASSANDRA-15388
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15388
> Project: Cassandra
> Issue Type: Sub-task
> Components: Local/Compaction
> Reporter: Blake Eggleston
> Assignee: Blake Eggleston
> Priority: Normal
> Fix For: 4.0
>
>
> This adds a test that is able to quickly and accurately measure the effect of
> potential gc optimizations against a wide range of (synthetic) compaction
> workloads. This test accurately measures allocation rates from 16 workloads
> in less that 2 minutes.
> This test uses google’s {{java-allocation-instrumenter}} agent to measure the
> workloads. Measurements using this agent are very accurate and pretty
> repeatable from run to run, with most variance being negligible (1-2 bytes
> per partition), although workloads with larger but fewer partitions vary a
> bit more (still less that 0.03%).
> The thinking behind this patch is that with compaction, we’re generally
> interested in the memory allocated per partition, since garbage scales more
> or less linearly with the number of partitions compacted. So measuring
> allocation from a small number of partitions that otherwise represent real
> world use cases is a good enough approximation.
> In addition to helping with compaction optimizations, this test could be used
> as a template for future optimization work. This pattern could also be used
> to set allocation limits on workloads/operations and fail CI if the
> allocation behavior changes past some threshold.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
