[ https://issues.apache.org/jira/browse/CASSANDRA-16201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17225523#comment-17225523 ]
Michael Semb Wever commented on CASSANDRA-16201: ------------------------------------------------ Re-doing the screenshots as done in CASSANDRA-15430 for comparison. h4. 2.1.18 Skipping these results as its based on the same C* code as in 15430, and I've confirmed the allocation counts and object sizes proportionally match. h4. 3.0.20 Allocations * BatchMessage.execute - 1926411 ** BatchStatement.getMutations => 1008170 == 52% ( previously 60%) ** BatchStatement.executeWithoutConditions => 692322 == 36% ( previously 30%) !16201_jfr_3023_alloc.png! Sizes by object under {{BatchStatement.getMutations}} !16201_jfr_3023_obj.png! With the invocation count only a 1% decrease, the {{Object[]}} size has gone from 29.8GB down to 18.7GB, or 37% reduction. h4. 3.11.8 Allocations * BatchMessage.execute - 1210873 ** BatchStatement.getMutations => 396645 == 33% ( previously 62%) ** BatchStatement.executeWithoutConditions => 664650 == 55% ( previously 30%) !16201_jfr_3118_alloc.png! Sizes by object under {{BatchStatement.getMutations}} !16201_jfr_3118_obj.png! The {{Object[]}} size has gone from 116GB down to 8.14GB. With the invocation count 40% that from CASSANDRA-15430, proportionally this is a 82% reduction. h4. 4.0-beta2 Allocations * BatchMessage.execute - 969189 ** BatchStatement.getMutations => 410739 == 42% ( previously 70%) ** BatchStatement.executeWithoutConditions => 425337 == 44% ( previously 23%) !16201_jfr_40b3_alloc.png! Sizes by object under {{BatchStatement.getMutations}} !16201_jfr_40b3_obj.png! The {{Object[]}} size has gone from 129GB down to 8.52GB. With the invocation count 30% that from CASSANDRA-15430, proportionally this is a 78% reduction. > Reduce amount of allocations during batch statement execution > ------------------------------------------------------------- > > Key: CASSANDRA-16201 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16201 > Project: Cassandra > Issue Type: Bug > Components: Local/Other > Reporter: Thomas Steinmaurer > Assignee: Marcus Eriksson > Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0-beta > > Attachments: 16201_jfr_3023_alloc.png, 16201_jfr_3023_obj.png, > 16201_jfr_3118_alloc.png, 16201_jfr_3118_obj.png, 16201_jfr_40b3_alloc.png, > 16201_jfr_40b3_obj.png, screenshot-1.png, screenshot-2.png, screenshot-3.png > > > In a Cas 2.1 / 3.0 / 3.11 / 4.0b2 comparison test with the same load profile, > we see 4.0b2 going OOM from time to time. According to a heap dump, we have > multiple NTR threads in a 3-digit MB range. > This is likely related to object array pre-allocations at the size of > {{BatchUpdatesCollector.updatedRows}} per {{BTree}} although there is always > only 1 {{BTreeRow}} in the {{BTree}}. > !screenshot-1.png|width=100%! > So it seems we have many, many 20K elemnts pre-allocated object arrays > resulting in a shallow heap of 80K each, although there is only one element > in the array. > This sort of pre-allocation is causing a lot of memory pressure. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org