Hi Pankaj, Le 16/10/2018 à 16:56, Pankaj Rathod a écrit : > Hi Emmanuel, > > Thanks for analyzing the heapdumps. > > You have mentioned two possible scenarios for this accumulation of > NioSocketSession objects. > > 1. - the clients start a connection, and never close it. > After finishing the testrun, I had checked number of connections to port > using netstat command. There were 0 connections. So I don't think this can > be the cause. > I checked below workitem is mentioned in apacheDS jira. Is this going to be > included in any upoming apacheDS release? > http://mail-archives.apache.org/mod_mbox/directory-dev/201808.mbox/%3cjira.13144434.1520885847000.120166.1534807140...@atlassian.jira%3E
I don't think so. The probem with this approach is that it would eat a lot of CPU checking sessions every second, for a very little gain: - most of the sessions will always be properly closed: they will be opened, do some operation, and then closed. If there is some delay between the opening and the closing, we will check every second for their idle status, for nothing if the client is properly written - if you have tens of thousands sessions, that will litterally kill the server as every second, it will check each session for its status. I'd rather depend on the underlying OS to properly handle TCP connections. > > 2. - the clients start a connection, close it, but the server never ditch > it. In your test, can you tell me if your client is closing the connection, > and how ? > Our testrun client executes new threads repetitively where every thread > performs below operations before terminating the thread. > execute bind > perform search > execute unbind Sounds fine. > We are observing memory growth if we enable or disable the "execute > unbind" task for the threads before termination. Sounds like a bug. > > We did some investigation in MINA for our test run and observed that the > session objects present in memory for longer time have status "Closing". We > are expecting this could be because of below issue > https://issues.apache.org/jira/browse/DIRMINA-1021 Unlikely. This JIRA has been fixed in MINA 2.0.14, ApacheDS M24 uses MINA 2.0.16 > > The fix for this issue seems included in MINA version which is bundled in > apacheDS M24. Going to apacheDS M24/M25 will be time consuming process for > us. Although, we will plan to upgrade in future. Switching to ApacheDS M25 should not take that much time: you just have to reload the database from scratch, as the format has changed. > > Emmanuel, could you comment on below queries related to MINA issues for > apacheDS? Your response will help us deciding to test the fix for > DIRMINA-1021. If you suggest then I will forward this mailthread on MINA > forum instead. > 1. Can heapmemory get full because of this issue? Or something in > MINA/apacheDS will remove these accumulated CLOSING state NioSocketSession > objects? Will the heapdumps show the exact same picture (top consumer > hierarchy) if memory crashes because of this issue? We noticed in our > heapdumps that top consumer hierarchy is almost similar to the hprof I > shared with you. We observed large number of NioSocketSession objects were > occupying more than 95% of memory. I don't think MINA is the culprit here. There is clearly some investigation to be done to see if we are correctly handling closing session in ApacheDS. > 2. The mailthread for DIRMINA-1021 mentions code changes as > "writebuffer method should call removeNow method when IOException occurs". > Could you please comment if this single line code change needed for this > fix? (If you suggest, then I will forward this question on MINA forum > instead.) I don't reming from the top of my head what was the exact fix was, it was 3 years ago, more or less :-) Question : would it be possible for you to share your load test, so that I can reproduc it on my server to see what's going on ? Thanks ! -- Emmanuel Lecharny Symas.com directory.apache.org
pEpkey.asc
Description: application/pgp-keys
