[jira] [Updated] (CASSANDRA-16201) Reduce amount of allocations during batch statement execution
[ https://issues.apache.org/jira/browse/CASSANDRA-16201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-16201: --- Fix Version/s: 4.0 > Reduce amount of allocations during batch statement execution > - > > Key: CASSANDRA-16201 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16201 > Project: Cassandra > Issue Type: Bug > Components: Local/Other >Reporter: Thomas Steinmaurer >Assignee: Marcus Eriksson >Priority: Normal > Fix For: 3.11.10, 4.0-beta5, 4.0 > > Attachments: 16201_jfr_3023_alloc.png, 16201_jfr_3023_obj.png, > 16201_jfr_3118_alloc.png, 16201_jfr_3118_obj.png, 16201_jfr_40b3_alloc.png, > 16201_jfr_40b3_obj.png, screenshot-1.png, screenshot-2.png, screenshot-3.png, > screenshot-4.png > > > In a Cas 2.1 / 3.0 / 3.11 / 4.0b2 comparison test with the same load profile, > we see 4.0b2 going OOM from time to time. According to a heap dump, we have > multiple NTR threads in a 3-digit MB range. > This is likely related to object array pre-allocations at the size of > {{BatchUpdatesCollector.updatedRows}} per {{BTree}} although there is always > only 1 {{BTreeRow}} in the {{BTree}}. > !screenshot-1.png|width=100%! > So it seems we have many, many 20K elemnts pre-allocated object arrays > resulting in a shallow heap of 80K each, although there is only one element > in the array. > This sort of pre-allocation is causing a lot of memory pressure. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16201) Reduce amount of allocations during batch statement execution
[ https://issues.apache.org/jira/browse/CASSANDRA-16201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-16201: Fix Version/s: (was: 4.0-beta) (was: 3.11.x) (was: 3.0.x) 4.0-beta5 3.11.10 Source Control Link: https://github.com/apache/cassandra/commit/8333d0b0890f6299fac1fd219f58aee0c62cbf9c Resolution: Fixed Status: Resolved (was: Ready to Commit) And committed, thanks everyone Decided to not merge the 3.0 patch as the improvement there was quite small - if anyone has a benchmark showing big 3.0 improvements I'd be happy to commit it as well. > Reduce amount of allocations during batch statement execution > - > > Key: CASSANDRA-16201 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16201 > Project: Cassandra > Issue Type: Bug > Components: Local/Other >Reporter: Thomas Steinmaurer >Assignee: Marcus Eriksson >Priority: Normal > Fix For: 3.11.10, 4.0-beta5 > > Attachments: 16201_jfr_3023_alloc.png, 16201_jfr_3023_obj.png, > 16201_jfr_3118_alloc.png, 16201_jfr_3118_obj.png, 16201_jfr_40b3_alloc.png, > 16201_jfr_40b3_obj.png, screenshot-1.png, screenshot-2.png, screenshot-3.png, > screenshot-4.png > > > In a Cas 2.1 / 3.0 / 3.11 / 4.0b2 comparison test with the same load profile, > we see 4.0b2 going OOM from time to time. According to a heap dump, we have > multiple NTR threads in a 3-digit MB range. > This is likely related to object array pre-allocations at the size of > {{BatchUpdatesCollector.updatedRows}} per {{BTree}} although there is always > only 1 {{BTreeRow}} in the {{BTree}}. > !screenshot-1.png|width=100%! > So it seems we have many, many 20K elemnts pre-allocated object arrays > resulting in a shallow heap of 80K each, although there is only one element > in the array. > This sort of pre-allocation is causing a lot of memory pressure. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16201) Reduce amount of allocations during batch statement execution
[ https://issues.apache.org/jira/browse/CASSANDRA-16201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-16201: Status: Ready to Commit (was: Review In Progress) > Reduce amount of allocations during batch statement execution > - > > Key: CASSANDRA-16201 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16201 > Project: Cassandra > Issue Type: Bug > Components: Local/Other >Reporter: Thomas Steinmaurer >Assignee: Marcus Eriksson >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0-beta > > Attachments: 16201_jfr_3023_alloc.png, 16201_jfr_3023_obj.png, > 16201_jfr_3118_alloc.png, 16201_jfr_3118_obj.png, 16201_jfr_40b3_alloc.png, > 16201_jfr_40b3_obj.png, screenshot-1.png, screenshot-2.png, screenshot-3.png, > screenshot-4.png > > > In a Cas 2.1 / 3.0 / 3.11 / 4.0b2 comparison test with the same load profile, > we see 4.0b2 going OOM from time to time. According to a heap dump, we have > multiple NTR threads in a 3-digit MB range. > This is likely related to object array pre-allocations at the size of > {{BatchUpdatesCollector.updatedRows}} per {{BTree}} although there is always > only 1 {{BTreeRow}} in the {{BTree}}. > !screenshot-1.png|width=100%! > So it seems we have many, many 20K elemnts pre-allocated object arrays > resulting in a shallow heap of 80K each, although there is only one element > in the array. > This sort of pre-allocation is causing a lot of memory pressure. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16201) Reduce amount of allocations during batch statement execution
[ https://issues.apache.org/jira/browse/CASSANDRA-16201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-16201: Reviewers: Benjamin Lerer, Michael Semb Wever, Yifan Cai (was: Michael Semb Wever, Yifan Cai) > Reduce amount of allocations during batch statement execution > - > > Key: CASSANDRA-16201 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16201 > Project: Cassandra > Issue Type: Bug > Components: Local/Other >Reporter: Thomas Steinmaurer >Assignee: Marcus Eriksson >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0-beta > > Attachments: 16201_jfr_3023_alloc.png, 16201_jfr_3023_obj.png, > 16201_jfr_3118_alloc.png, 16201_jfr_3118_obj.png, 16201_jfr_40b3_alloc.png, > 16201_jfr_40b3_obj.png, screenshot-1.png, screenshot-2.png, screenshot-3.png, > screenshot-4.png > > > In a Cas 2.1 / 3.0 / 3.11 / 4.0b2 comparison test with the same load profile, > we see 4.0b2 going OOM from time to time. According to a heap dump, we have > multiple NTR threads in a 3-digit MB range. > This is likely related to object array pre-allocations at the size of > {{BatchUpdatesCollector.updatedRows}} per {{BTree}} although there is always > only 1 {{BTreeRow}} in the {{BTree}}. > !screenshot-1.png|width=100%! > So it seems we have many, many 20K elemnts pre-allocated object arrays > resulting in a shallow heap of 80K each, although there is only one element > in the array. > This sort of pre-allocation is causing a lot of memory pressure. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16201) Reduce amount of allocations during batch statement execution
[ https://issues.apache.org/jira/browse/CASSANDRA-16201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Steinmaurer updated CASSANDRA-16201: --- Attachment: screenshot-4.png > Reduce amount of allocations during batch statement execution > - > > Key: CASSANDRA-16201 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16201 > Project: Cassandra > Issue Type: Bug > Components: Local/Other >Reporter: Thomas Steinmaurer >Assignee: Marcus Eriksson >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0-beta > > Attachments: 16201_jfr_3023_alloc.png, 16201_jfr_3023_obj.png, > 16201_jfr_3118_alloc.png, 16201_jfr_3118_obj.png, 16201_jfr_40b3_alloc.png, > 16201_jfr_40b3_obj.png, screenshot-1.png, screenshot-2.png, screenshot-3.png, > screenshot-4.png > > > In a Cas 2.1 / 3.0 / 3.11 / 4.0b2 comparison test with the same load profile, > we see 4.0b2 going OOM from time to time. According to a heap dump, we have > multiple NTR threads in a 3-digit MB range. > This is likely related to object array pre-allocations at the size of > {{BatchUpdatesCollector.updatedRows}} per {{BTree}} although there is always > only 1 {{BTreeRow}} in the {{BTree}}. > !screenshot-1.png|width=100%! > So it seems we have many, many 20K elemnts pre-allocated object arrays > resulting in a shallow heap of 80K each, although there is only one element > in the array. > This sort of pre-allocation is causing a lot of memory pressure. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16201) Reduce amount of allocations during batch statement execution
[ https://issues.apache.org/jira/browse/CASSANDRA-16201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-16201: --- Attachment: 16201_jfr_3023_obj.png > Reduce amount of allocations during batch statement execution > - > > Key: CASSANDRA-16201 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16201 > Project: Cassandra > Issue Type: Bug > Components: Local/Other >Reporter: Thomas Steinmaurer >Assignee: Marcus Eriksson >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0-beta > > Attachments: 16201_jfr_3023_alloc.png, 16201_jfr_3023_obj.png, > 16201_jfr_3118_alloc.png, 16201_jfr_3118_obj.png, 16201_jfr_40b3_alloc.png, > 16201_jfr_40b3_obj.png, screenshot-1.png, screenshot-2.png, screenshot-3.png > > > In a Cas 2.1 / 3.0 / 3.11 / 4.0b2 comparison test with the same load profile, > we see 4.0b2 going OOM from time to time. According to a heap dump, we have > multiple NTR threads in a 3-digit MB range. > This is likely related to object array pre-allocations at the size of > {{BatchUpdatesCollector.updatedRows}} per {{BTree}} although there is always > only 1 {{BTreeRow}} in the {{BTree}}. > !screenshot-1.png|width=100%! > So it seems we have many, many 20K elemnts pre-allocated object arrays > resulting in a shallow heap of 80K each, although there is only one element > in the array. > This sort of pre-allocation is causing a lot of memory pressure. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16201) Reduce amount of allocations during batch statement execution
[ https://issues.apache.org/jira/browse/CASSANDRA-16201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-16201: --- Attachment: 16201_jfr_3023_alloc.png > Reduce amount of allocations during batch statement execution > - > > Key: CASSANDRA-16201 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16201 > Project: Cassandra > Issue Type: Bug > Components: Local/Other >Reporter: Thomas Steinmaurer >Assignee: Marcus Eriksson >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0-beta > > Attachments: 16201_jfr_3023_alloc.png, 16201_jfr_3118_alloc.png, > 16201_jfr_3118_obj.png, 16201_jfr_40b3_alloc.png, 16201_jfr_40b3_obj.png, > screenshot-1.png, screenshot-2.png, screenshot-3.png > > > In a Cas 2.1 / 3.0 / 3.11 / 4.0b2 comparison test with the same load profile, > we see 4.0b2 going OOM from time to time. According to a heap dump, we have > multiple NTR threads in a 3-digit MB range. > This is likely related to object array pre-allocations at the size of > {{BatchUpdatesCollector.updatedRows}} per {{BTree}} although there is always > only 1 {{BTreeRow}} in the {{BTree}}. > !screenshot-1.png|width=100%! > So it seems we have many, many 20K elemnts pre-allocated object arrays > resulting in a shallow heap of 80K each, although there is only one element > in the array. > This sort of pre-allocation is causing a lot of memory pressure. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16201) Reduce amount of allocations during batch statement execution
[ https://issues.apache.org/jira/browse/CASSANDRA-16201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-16201: --- Attachment: 16201_jfr_3118_obj.png > Reduce amount of allocations during batch statement execution > - > > Key: CASSANDRA-16201 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16201 > Project: Cassandra > Issue Type: Bug > Components: Local/Other >Reporter: Thomas Steinmaurer >Assignee: Marcus Eriksson >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0-beta > > Attachments: 16201_jfr_3118_alloc.png, 16201_jfr_3118_obj.png, > 16201_jfr_40b3_alloc.png, 16201_jfr_40b3_obj.png, screenshot-1.png, > screenshot-2.png, screenshot-3.png > > > In a Cas 2.1 / 3.0 / 3.11 / 4.0b2 comparison test with the same load profile, > we see 4.0b2 going OOM from time to time. According to a heap dump, we have > multiple NTR threads in a 3-digit MB range. > This is likely related to object array pre-allocations at the size of > {{BatchUpdatesCollector.updatedRows}} per {{BTree}} although there is always > only 1 {{BTreeRow}} in the {{BTree}}. > !screenshot-1.png|width=100%! > So it seems we have many, many 20K elemnts pre-allocated object arrays > resulting in a shallow heap of 80K each, although there is only one element > in the array. > This sort of pre-allocation is causing a lot of memory pressure. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16201) Reduce amount of allocations during batch statement execution
[ https://issues.apache.org/jira/browse/CASSANDRA-16201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-16201: --- Attachment: 16201_jfr_3118_alloc.png > Reduce amount of allocations during batch statement execution > - > > Key: CASSANDRA-16201 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16201 > Project: Cassandra > Issue Type: Bug > Components: Local/Other >Reporter: Thomas Steinmaurer >Assignee: Marcus Eriksson >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0-beta > > Attachments: 16201_jfr_3118_alloc.png, 16201_jfr_40b3_alloc.png, > 16201_jfr_40b3_obj.png, screenshot-1.png, screenshot-2.png, screenshot-3.png > > > In a Cas 2.1 / 3.0 / 3.11 / 4.0b2 comparison test with the same load profile, > we see 4.0b2 going OOM from time to time. According to a heap dump, we have > multiple NTR threads in a 3-digit MB range. > This is likely related to object array pre-allocations at the size of > {{BatchUpdatesCollector.updatedRows}} per {{BTree}} although there is always > only 1 {{BTreeRow}} in the {{BTree}}. > !screenshot-1.png|width=100%! > So it seems we have many, many 20K elemnts pre-allocated object arrays > resulting in a shallow heap of 80K each, although there is only one element > in the array. > This sort of pre-allocation is causing a lot of memory pressure. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16201) Reduce amount of allocations during batch statement execution
[ https://issues.apache.org/jira/browse/CASSANDRA-16201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-16201: --- Attachment: 16201_jfr_40b3_obj.png > Reduce amount of allocations during batch statement execution > - > > Key: CASSANDRA-16201 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16201 > Project: Cassandra > Issue Type: Bug > Components: Local/Other >Reporter: Thomas Steinmaurer >Assignee: Marcus Eriksson >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0-beta > > Attachments: 16201_jfr_40b3_alloc.png, 16201_jfr_40b3_obj.png, > screenshot-1.png, screenshot-2.png, screenshot-3.png > > > In a Cas 2.1 / 3.0 / 3.11 / 4.0b2 comparison test with the same load profile, > we see 4.0b2 going OOM from time to time. According to a heap dump, we have > multiple NTR threads in a 3-digit MB range. > This is likely related to object array pre-allocations at the size of > {{BatchUpdatesCollector.updatedRows}} per {{BTree}} although there is always > only 1 {{BTreeRow}} in the {{BTree}}. > !screenshot-1.png|width=100%! > So it seems we have many, many 20K elemnts pre-allocated object arrays > resulting in a shallow heap of 80K each, although there is only one element > in the array. > This sort of pre-allocation is causing a lot of memory pressure. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16201) Reduce amount of allocations during batch statement execution
[ https://issues.apache.org/jira/browse/CASSANDRA-16201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-16201: --- Attachment: 16201_jfr_40b3_alloc.png > Reduce amount of allocations during batch statement execution > - > > Key: CASSANDRA-16201 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16201 > Project: Cassandra > Issue Type: Bug > Components: Local/Other >Reporter: Thomas Steinmaurer >Assignee: Marcus Eriksson >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0-beta > > Attachments: 16201_jfr_40b3_alloc.png, screenshot-1.png, > screenshot-2.png, screenshot-3.png > > > In a Cas 2.1 / 3.0 / 3.11 / 4.0b2 comparison test with the same load profile, > we see 4.0b2 going OOM from time to time. According to a heap dump, we have > multiple NTR threads in a 3-digit MB range. > This is likely related to object array pre-allocations at the size of > {{BatchUpdatesCollector.updatedRows}} per {{BTree}} although there is always > only 1 {{BTreeRow}} in the {{BTree}}. > !screenshot-1.png|width=100%! > So it seems we have many, many 20K elemnts pre-allocated object arrays > resulting in a shallow heap of 80K each, although there is only one element > in the array. > This sort of pre-allocation is causing a lot of memory pressure. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16201) Reduce amount of allocations during batch statement execution
[ https://issues.apache.org/jira/browse/CASSANDRA-16201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-16201: --- Reviewers: Michael Semb Wever, Yifan Cai (was: Michael Semb Wever) > Reduce amount of allocations during batch statement execution > - > > Key: CASSANDRA-16201 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16201 > Project: Cassandra > Issue Type: Bug > Components: Local/Other >Reporter: Thomas Steinmaurer >Assignee: Marcus Eriksson >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0-beta > > Attachments: screenshot-1.png, screenshot-2.png, screenshot-3.png > > > In a Cas 2.1 / 3.0 / 3.11 / 4.0b2 comparison test with the same load profile, > we see 4.0b2 going OOM from time to time. According to a heap dump, we have > multiple NTR threads in a 3-digit MB range. > This is likely related to object array pre-allocations at the size of > {{BatchUpdatesCollector.updatedRows}} per {{BTree}} although there is always > only 1 {{BTreeRow}} in the {{BTree}}. > !screenshot-1.png|width=100%! > So it seems we have many, many 20K elemnts pre-allocated object arrays > resulting in a shallow heap of 80K each, although there is only one element > in the array. > This sort of pre-allocation is causing a lot of memory pressure. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16201) Reduce amount of allocations during batch statement execution
[ https://issues.apache.org/jira/browse/CASSANDRA-16201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-16201: --- Reviewers: Michael Semb Wever, Michael Semb Wever (was: Michael Semb Wever) Michael Semb Wever, Michael Semb Wever (was: Michael Semb Wever) Status: Review In Progress (was: Patch Available) > Reduce amount of allocations during batch statement execution > - > > Key: CASSANDRA-16201 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16201 > Project: Cassandra > Issue Type: Bug > Components: Local/Other >Reporter: Thomas Steinmaurer >Assignee: Marcus Eriksson >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0-beta > > Attachments: screenshot-1.png, screenshot-2.png, screenshot-3.png > > > In a Cas 2.1 / 3.0 / 3.11 / 4.0b2 comparison test with the same load profile, > we see 4.0b2 going OOM from time to time. According to a heap dump, we have > multiple NTR threads in a 3-digit MB range. > This is likely related to object array pre-allocations at the size of > {{BatchUpdatesCollector.updatedRows}} per {{BTree}} although there is always > only 1 {{BTreeRow}} in the {{BTree}}. > !screenshot-1.png|width=100%! > So it seems we have many, many 20K elemnts pre-allocated object arrays > resulting in a shallow heap of 80K each, although there is only one element > in the array. > This sort of pre-allocation is causing a lot of memory pressure. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16201) Reduce amount of allocations during batch statement execution
[ https://issues.apache.org/jira/browse/CASSANDRA-16201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Steinmaurer updated CASSANDRA-16201: --- Attachment: screenshot-3.png > Reduce amount of allocations during batch statement execution > - > > Key: CASSANDRA-16201 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16201 > Project: Cassandra > Issue Type: Bug > Components: Local/Other >Reporter: Thomas Steinmaurer >Assignee: Marcus Eriksson >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0-beta > > Attachments: screenshot-1.png, screenshot-2.png, screenshot-3.png > > > In a Cas 2.1 / 3.0 / 3.11 / 4.0b2 comparison test with the same load profile, > we see 4.0b2 going OOM from time to time. According to a heap dump, we have > multiple NTR threads in a 3-digit MB range. > This is likely related to object array pre-allocations at the size of > {{BatchUpdatesCollector.updatedRows}} per {{BTree}} although there is always > only 1 {{BTreeRow}} in the {{BTree}}. > !screenshot-1.png|width=100%! > So it seems we have many, many 20K elemnts pre-allocated object arrays > resulting in a shallow heap of 80K each, although there is only one element > in the array. > This sort of pre-allocation is causing a lot of memory pressure. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16201) Reduce amount of allocations during batch statement execution
[ https://issues.apache.org/jira/browse/CASSANDRA-16201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-16201: Test and Documentation Plan: cci runs Status: Patch Available (was: Open) 4.0: [patch|https://github.com/krummas/cassandra/commits/marcuse/16201-4.0-new], [cci|https://app.circleci.com/pipelines/github/krummas/cassandra?branch=marcuse%2F16201-4.0-new] 3.11: [patch|https://github.com/krummas/cassandra/commits/marcuse/16201-3.11-new], [cci|https://app.circleci.com/pipelines/github/krummas/cassandra?branch=marcuse%2F16201-3.11-new] 3.0: [patch|https://github.com/krummas/cassandra/commits/marcuse/16201-3.0-new], [cci|https://app.circleci.com/pipelines/github/krummas/cassandra?branch=marcuse%2F16201-3.0-new] this focuses only on getMutations as that is where the OOM happened - there is much more to do, but I think these fixes are fairly safe and make performance slightly more acceptable > Reduce amount of allocations during batch statement execution > - > > Key: CASSANDRA-16201 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16201 > Project: Cassandra > Issue Type: Bug > Components: Local/Other >Reporter: Thomas Steinmaurer >Assignee: Marcus Eriksson >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0-beta > > Attachments: screenshot-1.png, screenshot-2.png > > > In a Cas 2.1 / 3.0 / 3.11 / 4.0b2 comparison test with the same load profile, > we see 4.0b2 going OOM from time to time. According to a heap dump, we have > multiple NTR threads in a 3-digit MB range. > This is likely related to object array pre-allocations at the size of > {{BatchUpdatesCollector.updatedRows}} per {{BTree}} although there is always > only 1 {{BTreeRow}} in the {{BTree}}. > !screenshot-1.png|width=100%! > So it seems we have many, many 20K elemnts pre-allocated object arrays > resulting in a shallow heap of 80K each, although there is only one element > in the array. > This sort of pre-allocation is causing a lot of memory pressure. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16201) Reduce amount of allocations during batch statement execution
[ https://issues.apache.org/jira/browse/CASSANDRA-16201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-16201: --- Reviewers: Michael Semb Wever > Reduce amount of allocations during batch statement execution > - > > Key: CASSANDRA-16201 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16201 > Project: Cassandra > Issue Type: Bug > Components: Local/Other >Reporter: Thomas Steinmaurer >Assignee: Marcus Eriksson >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0-beta > > Attachments: screenshot-1.png, screenshot-2.png > > > In a Cas 2.1 / 3.0 / 3.11 / 4.0b2 comparison test with the same load profile, > we see 4.0b2 going OOM from time to time. According to a heap dump, we have > multiple NTR threads in a 3-digit MB range. > This is likely related to object array pre-allocations at the size of > {{BatchUpdatesCollector.updatedRows}} per {{BTree}} although there is always > only 1 {{BTreeRow}} in the {{BTree}}. > !screenshot-1.png|width=100%! > So it seems we have many, many 20K elemnts pre-allocated object arrays > resulting in a shallow heap of 80K each, although there is only one element > in the array. > This sort of pre-allocation is causing a lot of memory pressure. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16201) Reduce amount of allocations during batch statement execution
[ https://issues.apache.org/jira/browse/CASSANDRA-16201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-16201: --- Since Version: 3.0 alpha 1 > Reduce amount of allocations during batch statement execution > - > > Key: CASSANDRA-16201 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16201 > Project: Cassandra > Issue Type: Bug > Components: Local/Other >Reporter: Thomas Steinmaurer >Assignee: Marcus Eriksson >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0-beta > > Attachments: screenshot-1.png, screenshot-2.png > > > In a Cas 2.1 / 3.0 / 3.11 / 4.0b2 comparison test with the same load profile, > we see 4.0b2 going OOM from time to time. According to a heap dump, we have > multiple NTR threads in a 3-digit MB range. > This is likely related to object array pre-allocations at the size of > {{BatchUpdatesCollector.updatedRows}} per {{BTree}} although there is always > only 1 {{BTreeRow}} in the {{BTree}}. > !screenshot-1.png|width=100%! > So it seems we have many, many 20K elemnts pre-allocated object arrays > resulting in a shallow heap of 80K each, although there is only one element > in the array. > This sort of pre-allocation is causing a lot of memory pressure. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16201) Reduce amount of allocations during batch statement execution
[ https://issues.apache.org/jira/browse/CASSANDRA-16201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-16201: --- Fix Version/s: 3.0.x > Reduce amount of allocations during batch statement execution > - > > Key: CASSANDRA-16201 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16201 > Project: Cassandra > Issue Type: Bug > Components: Local/Other >Reporter: Thomas Steinmaurer >Assignee: Marcus Eriksson >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0-beta > > Attachments: screenshot-1.png, screenshot-2.png > > > In a Cas 2.1 / 3.0 / 3.11 / 4.0b2 comparison test with the same load profile, > we see 4.0b2 going OOM from time to time. According to a heap dump, we have > multiple NTR threads in a 3-digit MB range. > This is likely related to object array pre-allocations at the size of > {{BatchUpdatesCollector.updatedRows}} per {{BTree}} although there is always > only 1 {{BTreeRow}} in the {{BTree}}. > !screenshot-1.png|width=100%! > So it seems we have many, many 20K elemnts pre-allocated object arrays > resulting in a shallow heap of 80K each, although there is only one element > in the array. > This sort of pre-allocation is causing a lot of memory pressure. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16201) Reduce amount of allocations during batch statement execution
[ https://issues.apache.org/jira/browse/CASSANDRA-16201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-16201: Summary: Reduce amount of allocations during batch statement execution (was: Reduce amount of allocations during batch statement executions) > Reduce amount of allocations during batch statement execution > - > > Key: CASSANDRA-16201 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16201 > Project: Cassandra > Issue Type: Bug > Components: Local/Other >Reporter: Thomas Steinmaurer >Assignee: Marcus Eriksson >Priority: Normal > Fix For: 3.11.x, 4.0-beta > > Attachments: screenshot-1.png, screenshot-2.png > > > In a Cas 2.1 / 3.0 / 3.11 / 4.0b2 comparison test with the same load profile, > we see 4.0b2 going OOM from time to time. According to a heap dump, we have > multiple NTR threads in a 3-digit MB range. > This is likely related to object array pre-allocations at the size of > {{BatchUpdatesCollector.updatedRows}} per {{BTree}} although there is always > only 1 {{BTreeRow}} in the {{BTree}}. > !screenshot-1.png|width=100%! > So it seems we have many, many 20K elemnts pre-allocated object arrays > resulting in a shallow heap of 80K each, although there is only one element > in the array. > This sort of pre-allocation is causing a lot of memory pressure. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org