Re: Potential dbi memory leak.
On Thu, May 28, 2015 at 03:59:51PM +1200, Duncan McEwan wrote: > On Tue, 26 May 2015 14:13:05 +0100 Tim Bunce wrote: > > > I've added this as a note: > > > > Note that the ChildHandles array holds weak references and that 'from > > time to time' the old slots get freed up. This isn't a leak, it just > > appears to be if you're not familiar with the caching that DBI does > > internally. You can rest assured that if the DBI did have a real leak > > a) a great many people would be affected and b) it would get fixed very > > quickly. > > > > I think 'from time to time' is every 120 or so newly created child handles. > But seeing this response from Tim about the fact that the DBI can cache up to > 120 or so handles made me wonder if this is true for database handles as well > as statement handles? Is it possible that our "problem" was simply the > correctly working DBI caching misbehaving due to our application running > persistently in multiple fcgid processes. Short answer: no. When a new handle is created the DBI simply pushes a weak reference to the handle onto the end of the ChildHandles array. Because it's a weak reference it doesn't interfere with the handle getting destroyed when the last (non weak) reference is removed. When the handle is destroyed, the corresponding element in the ChildHandles array is set to undef by the weak reference mechanism deep in the perl internals. The DBI isn't involved in that. The apparent 'leak' is because the element in the array still exists, albeit as an undef. The DBI clears out the undef elements occasionally to prevent the array growing indefinitely. Tim.
Re: Potential dbi memory leak.
I haven't tried it but you could probably modify the code I posted to test it. With the statement handle after 120(?) loops of apparent memory leak +1, there is a correction of -119. Good luck. Steve. On 28 May 2015 05:00, "Duncan McEwan" wrote: > Apologies for butting in on this thread, but I saw the following response > from Tim recently and it made me wonder ... > > On Tue, 26 May 2015 14:13:05 +0100 Tim Bunce wrote: > > > I've added this as a note: > > > > Note that the ChildHandles array holds weak references and that 'from > > time to time' the old slots get freed up. This isn't a leak, it just > > appears to be if you're not familiar with the caching that DBI does > > internally. You can rest assured that if the DBI did have a real leak > > a) a great many people would be affected and b) it would get fixed > very > > quickly. > > > > I think 'from time to time' is every 120 or so newly created child > handles. > > A while ago we had a mysterious problem using DBI in an application that > was > written as a plugin for the foswiki platform. Since our foswiki instance > was > running persistently under fcgid it was long-running and over time we'd > see a > gradual increase in the open connections it held to our mysql database > server. Eventually our server would reach it's maximum connection count > and > reject new connections. > > The most recent time I tried to debug this was over a year ago (March 2014) > and there was a brief exchange of emails on this list with the subject "DBI > Mysql Driver Handle Mysteriously Changes". > > Since then we've given up (!) and changed the way our application ran so it > is no longer a foswiki plugin. That seems to have "fixed" the connection > leakage and so we are unlikely to ever go back to find out exactly what was > going on here. > > But seeing this response from Tim about the fact that the DBI can cache up > to > 120 or so handles made me wonder if this is true for database handles as > well > as statement handles? Is it possible that our "problem" was simply the > correctly working DBI caching misbehaving due to our application running > persistently in multiple fcgid processes. > > I'm not looking to re-open investigating this issue - our environement has > now changed sufficiently that recreating the set up with the connection > leak > to do more debugging would be quite difficult. But I was just curious > about > whether the above could be the case. If the answer is "no" I'll be happy > to > just leave it at that! > > Thanks. > > Duncan >
