> Jeff; > It seems that you are noting the same peculiarities as I did. > Basically; do effectively nothing in the script and watch the > memory disappear.
I think that's the loading of modules and if you have a lot of objects, I believe that may contribute (but others on the list will surely correct me where I'm wrong :) I noticed a hit when connecting the first time, but after that, it's much stabler. > > Have you tried explicitly 'killing off' the supposedly local > variables, as I did? I found it made a difference. > Maybe even Perl itself should be suspected for the moment? Are > you using Activestate Buld 633? I'm using 631. My script calls a lot of functions which clean up the handles when the functions return, so yes, it's killing them off. > > I think your test app is a better test program than I could have > given you. > My application is several thousand lines of code, and I doubt > that a distilled version would be representative. It would be nice to know if you are loading a lot of modules...and instantiating objects based upon them. That may be a big part of it. What would concern me greatly is that each iteration of the loop the memory grows and all you are doing is database access. I've had quite a few programs written using DBI and DBD::Oracle in the past which simply loop on the database and create/reset the password/inactivate NT user accounts based upon database entries...and that just ran in a loop on a production machine. No-one saw a memory leak. If there was one, it was hardly noticable...even over weeks. Regards, Jeff > >>> "Jeff Urlwin" <[EMAIL PROTECTED]> 07/30/2002 7:03:04 PM >>> > > Well -- just doing a simple test connect and disconnect eat up > nearly 1,000 items (multiple connects and disconnects have no > effect on the qty of items eaten, at this point -- so, I'm > "guessing" that some of this is due to autoloading and just > loading the modules to be used), but selecting no rows only eats > up 6. Inserting a number into a simple table doesn't eat any > SVs. That's if I'm using it right. > > Jeff > > -----Original Message----- > From: Gary Gauthier [mailto:[EMAIL PROTECTED]] > Sent: Tuesday, July 30, 2002 1:41 PM > To: [EMAIL PROTECTED] > Subject: RE: Apparent memory leak in DBI v1.30 > > > I'll see what I can do. For you to run it, I'll need to get you a > copy of the database and some data. > I have to switch to something else here for balance of today, so > I'll see about doing it first thing tomorrow. > > Did note, however, that I dropped by a count of 22 (reported by > Leak) when I explicity > set the recordset pointer to 'undef', even though it was defined > as a 'my' variable. Will see > if i can figure out why. Sounds like a scoping issue. > > Regards; > Gary > > > >>> "Jeff Urlwin" <[EMAIL PROTECTED]> 07/30/2002 12:34:35 PM >>> > > If you can wrap the small test up for me, I may be able to get to > it, I just don't want to promise it :) > > Jeff > -----Original Message----- > From: Gary Gauthier [mailto:[EMAIL PROTECTED]] > Sent: Tuesday, July 30, 2002 12:30 PM > To: [EMAIL PROTECTED] > Subject: RE: Apparent memory leak in DBI v1.30 > > > Jeff; > I am also on vacation for a week beginning Thursday evening, so > it wouldn't matter anyway. > Thanks for letting me know. > > By the way; I constructed a small program to do some testing, and > it is still showing leakage. > I'll see what I can find out, and I'll wait to hear from you when > you get back. > > Have a good vacation. > > Regards; > Gary > > > >>> "Jeff Urlwin" <[EMAIL PROTECTED]> 07/30/2002 12:00:40 PM >>> > > Gary -- I don't know if anyone has DEBUGGING turned on. That's > an interesting question. I haven't gotten around to the memory > leak testing yet. I may just have to have a debugging version of > perl, which is a pain and large and slow. I don't blame you for > not doing it! I'm not sure I will be able to look into memory > leaks further until I get back from Vacation next week. I leave > on Thursday with lots to do between now and then. > > Regards, > > Jeff > -----Original Message----- > From: Gary Gauthier [mailto:[EMAIL PROTECTED]] > Sent: Tuesday, July 30, 2002 10:57 AM > To: [EMAIL PROTECTED] > Subject: RE: Apparent memory leak in DBI v1.30 > > > Jeff; > > You may not believe it, but I had a feeling such a thing might > happen with a Microsoft product, > and I designed the modules to allow me to specify a handle or a > connection string for each level in > the process. I changed it over to separate connections and ran > it. While the program worked, the > Devel::Leak showed a count of 52046 before and 53333 after a > short test of 11 queries in total. > > Do you know of anyone who has a copy of the Ativestate Perl with > the Debugging turned on? > I understand that the Leak detector gives more diagnostic info, > if that is enabled. The reason I > don't build it myself is that I'm trying to use 'tried and true' > versions where possible, to cut the > number of degreees of freedom in the problem-space. > > Regards; > Gary > > > >>> "Jeff Urlwin" <[EMAIL PROTECTED]> 07/30/2002 10:21:31 AM >>> > > > > > DBD::ODBC::st execute failed: [Microsoft][ODBC SQL Server > > Driver]Connection is b > > usy with results for another hstmt (SQL-S1000)(DBD: > > st_execute/SQLExecute err=-1 > > ) at TestStep_Record.pm line 410, <STDIN> line 1. > > > > Am I to understand from this that ODBC connections do not permit nested > > queries to use the same connection handle? I'm sure ADO does. The > > code queries > > the database, then uses the result of each record to launch > > another query on a second table. > > Gary -- > > That's VERY database dependant and, no, technically, SQL Server doesn't > support it. Oracle does, for example. There is a way, though, > to get it to > work for SQL Server. See the perldoc DBD::ODBC for the full > description in > the section on DBD::ODBC 0.29. However: I remember an e-mail I received > from someone with *STRONG* knowledge on this and I believe the e-mail > indicated that you can actually put your SQL Server in a "spin loop" where > you have to restart the machine. Don't quote me on that, but it sounded > like there could be a problem. I can't find it, but I remember reading > something about it. Use it at your own risk...:) > > Regards, > > Jeff > > > > Regards; > > Gary > > > > > > >>> Tim Bunce <[EMAIL PROTECTED]> 07/30/2002 8:51:25 AM >>> > > On Tue, Jul 30, 2002 at 08:21:39AM -0400, Gary Gauthier wrote: > > > Jeff and Tim; > > > > > > The program only uses non-parameterized SELECT statements, > > > accesses an SQL2000 database using DBD:ODBC, > > > and data is always read using bound variables. Unfortunately; > > > it is a VERY large application, all object-oriented, and doesn't > > > lend itself to being made smaller to test for leaks. > > > > You could try going in the other direction... write a small script > > that does the same kind of thing and see if it leaks. > > > > If it doesn't then we're likely to point the finger at your application. > > > > Probably worth trying it with an older version of the DBI as a check. > > And/or with an older DBD::ODBC version. > > > > > How does one use the Devel::Leak module to the best advantage? > > > > Read the docs. It's not too complicated > > http://search-beta.cpan.org/author/NI-S/Devel-Leak/Leak.pm > > > > Tim. > > > > > It may be the easiest way to check it out. > > > > > > Regards; > > > Gary > > > > > > > > > >>> Tim Bunce <[EMAIL PROTECTED]> 07/29/2002 4:38:39 PM >>> > > > On Mon, Jul 29, 2002 at 09:52:33AM -0400, Jeff Urlwin wrote: > > > > Gary -- it's more likely in DBD::ODBC than DBI. > > > > > > But not by much :) > > > > > > Probably worth trying it with an older version of the DBI as a check. > > > > > > > If you can create a > > > > self-contained sample, it would be greatly appreciated and > > looked at faster > > > > :) > > > > > > Yes, a _small_ simple self-contained example script is a big help. > > > Copy the problem script and keep cutting away at it till the leak > > > goes away, or it gets so small it's obvious where the problem is. > > > > > > > You didn't indicate if the loop is a set of inserts, queries, > > etc. Does it > > > > have longs? What's the LongReadLen value? > > > > > > > > If you can, include a drop table and create table statement > > at the top, > > > > insert test data, then run through something similar to your > > loop that's > > > > leaking. > > > > > > > > Tim -- if you have suggestions as to how to approach this > > with Perl, I'd be > > > > very appreciative! > > > > > > The "keep making the sample code smaller" approach usually works well. > > > > > > The Devel::Leak module may be useful: > > > http://search-beta.cpan.org/author/NI-S/Devel-Leak-0.02/Leak.pm > > > > > > Another approach is to let the process grow huge (many MB) then > > > core dump it and look through the binary (using less or other viewer > > > that can read binary) and look for clues/patterns in the > leaked memory. > > > > > > Tim. > > > > > > > Regards, > > > > > > > > Jeff > > > > > > > > > > Tim; > > > > > > > > > > I'm using DBI v1.30 and DBD-ODBC v0.43. My database(SQL2000) app > > > > > is invoked about 40,000 times over the course of about 2 days to > > > > > generate test scripts for some automated test equipment. I've > > > > > been noticing the memory usage creeping up over the course of the > > > > > runs (up to Meg per loop at times). I have seen a report on the > > > > > developers mail list regarding this problem in recent versions. > > > > > Have you done any tests to check for leaks, or have you had any > > > > > reports of leaks? Perhaps there is some peculiarity of SQL2000, > > > > > that requires explicit releasing of handles or buffer space? > > > > > > > > > > Any help would be appreciated. > > > > > > > > > > Regards; > > > > > Gary > > > > > > > > > > > > > > > >
