[
https://issues.apache.org/jira/browse/CASSANDRA-13929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16286044#comment-16286044
]
Michael Shuler commented on CASSANDRA-13929:
--------------------------------------------
You may have some better success getting eyes on this JIRA by attaching a
proper patch for the 3.11 branch (and a separate trunk patch, if merge up isn't
clean). You can do this with a simple text file attachment of the diff, or
linking to a github branch.
Bonus points for adding a test that reproduces the problem/fix, as well as
getting test run through circleci.
This JIRA can be assigned to yourself as the author and you can set the status
to "Patch Available" when there's an actual patch/branch to review. Then you
can possibly seek out a reviewer on the dev@ mailing list, if it's urgent.
It looks like there are currently 115 Patch Available tickets, so it may still
take some time, but getting this set up for someone else to review would be a
great step in getting the process rolling further than just a comment.
https://issues.apache.org/jira/issues/?jql=project%20%3D%20CASSANDRA%20AND%20status%20%3D%20%22Patch%20Available%22
> BTree$Builder / io.netty.util.Recycler$Stack leaking memory
> -----------------------------------------------------------
>
> Key: CASSANDRA-13929
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13929
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Reporter: Thomas Steinmaurer
> Attachments: cassandra_3.11.0_min_memory_utilization.jpg,
> cassandra_3.11.1_NORECYCLE_memory_utilization.jpg,
> cassandra_3.11.1_mat_dominator_classes.png,
> cassandra_3.11.1_mat_dominator_classes_FIXED.png,
> cassandra_3.11.1_snapshot_heaputilization.png
>
>
> Different to CASSANDRA-13754, there seems to be another memory leak in
> 3.11.0+ in BTree$Builder / io.netty.util.Recycler$Stack.
> * heap utilization increase after upgrading to 3.11.0 =>
> cassandra_3.11.0_min_memory_utilization.jpg
> * No difference after upgrading to 3.11.1 (snapshot build) =>
> cassandra_3.11.1_snapshot_heaputilization.png; thus most likely after fixing
> CASSANDRA-13754, more visible now
> * MAT shows io.netty.util.Recycler$Stack as top contributing class =>
> cassandra_3.11.1_mat_dominator_classes.png
> * With -Xmx8G (CMS) and our load pattern, we have to do a rolling restart
> after ~ 72 hours
> Verified the following fix, namely explicitly unreferencing the
> _recycleHandle_ member (making it non-final). In
> _org.apache.cassandra.utils.btree.BTree.Builder.recycle()_
> {code}
> public void recycle()
> {
> if (recycleHandle != null)
> {
> this.cleanup();
> builderRecycler.recycle(this, recycleHandle);
> recycleHandle = null; // ADDED
> }
> }
> {code}
> Patched a single node in our loadtest cluster with this change and after ~ 10
> hours uptime, no sign of the previously offending class in MAT anymore =>
> cassandra_3.11.1_mat_dominator_classes_FIXED.png
> Can' say if this has any other side effects etc., but I doubt.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
