Re: [Zope-dev] Memory 2.1.4-2.1.6
Bill Anderson wrote: > > Eric Sattler wrote: > > > > Bill, > > > > Jason and I seem to have gotten mixed up here. > > Yeah, mix-ups tend to happen on lists ;-) > > > Here is my > > configuration and situation, the 2.1.4-2.1.6 means that it occurs > > under both versions of Zope. It also occurs with the Neil Schemenauer > > gc python patches. Here is my original posting: > > > > I also am having severe problems with memory creep. Our problem > > exhibits itself by quickly using gobs of memory and requiring a > > zope restart after about a day and a half at about 100M resident. > > > > First our configuration: > > > > Core Components: > > > > FreeBSD 4.0/FreeBSD 3.4 - on separate machines of course > > Postgres 6.5.3/Postgres 7.0(fixes some memory leaks) > > Python 1.5.2 > > Zope 2.1.4/2.1.6 have tried both > > pcgi with ZServer > > Apache w/mod_ssl 1.3.9 and 1.3.12 > > > > Products: > > > > ZPySQLDA > > SQLSession > > GenericUserFolder > > I don't know if you saw it or not, but so far GUF has been determined to > be a memory leak, and further was dermined to be causing further leakage > in SQLSession (as well as on or two other products, IIRC). A patch/new > version was posted, and may be worth a trial upgrade to see if that > fixes it. IIRC, it was related to things not being removed from cache, > imagine that! :) > Did see that, sounds like some promising progress has been made. Is the latest version of SQLSession up on the Zope site? I think Anthony Baxter referred to releasing a 0-2-9 version? Can't seem to find anything greater than 0-2-3, or else there is a patch directory I am missing. > To do this without compromising your production site, you can basically > copy your setup to the other machine (remember that the paths should be > kept the same if possible), carry out the experiments, and note the > results. This is how I test many things. Got several machines with 3 different versions of zope on 'em ready once I install the patches for GUF and SQLSession. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Memory 2.1.4-2.1.6 a.k.a. how to get objects out of the cache
Shane Hathaway wrote: > > Stuart 'Zen' Bishop wrote: > > /me hops on the 'real garbage collection for Python' bandwagon > > Have you looked at this? Yes, I actually built, installed and enabled it. It doesn't seem to have a major impact on my problems at least...neither does Zen's latest set of fixes. I enabled Neil Schemenauer's fixes, rebuilt python, and then tried tracking through chunks of the CPickleCache code. There did not seem to be any hanging python references. I believe the problem to be circular zope references on a higher level. I could be wrong though. The gc code for cPickleCache.c is not trivial, and I believe there is something wrong there, but have not proved it yet. This is the code that determines when things are removed from the zope cache. The wierd thing is that in my case, I am not really writing anything to the Zope db. Everything is stored externally in a postgres database. I am using GUF, and SQLSession though. Could it be that connection objects are not being cleaned up properly? As far as I understand it, connection objects are containers for zope objects for the duration of that request. At least in the case of postgres 6.5.3 there was some connection memory leaks, so I did try postgres 7.0, it solved some of postgres's problem but not zope's. sorry for rambling, just trying to get all of the tidbits that I know out there, maybe someone will see something. eric. > > http://www.enme.ucalgary.ca/~nascheme/python/gc.html > > I'd like to know how successful this project is. I wonder whether it > would have an issue with ExtensionClass instances (one of the > foundations of Zope). CPickleCache > > Shane > > ___ > Zope-Dev maillist - [EMAIL PROTECTED] > http://lists.zope.org/mailman/listinfo/zope-dev > ** No cross posts or HTML encoding! ** > (Related lists - > http://lists.zope.org/mailman/listinfo/zope-announce > http://lists.zope.org/mailman/listinfo/zope ) ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Memory 2.1.4-2.1.6 a.k.a. how to get objects out of the cache
All, I also am having severe problems with memory creep. Our problem exhibits itself by quickly using gobs of memory and requiring a zope restart after about a day and a half at about 100M resident. First our configuration: Core Components: FreeBSD 4.0/FreeBSD 3.4 - on separate machines of course Postgres 6.5.3/Postgres 7.0(fixes some memory leaks) Python 1.5.2 Zope 2.1.4/2.1.6 have tried both pcgi with ZServer Apache w/mod_ssl 1.3.9 and 1.3.12 Products: ZPySQLDA SQLSession GenericUserFolder I have looked at a lot of different things. This includes: Installing the gc(garbage collection) python package - Neil Schemenauer's patches Installing Postgres 7.0(fixes some connection memory leaks) Tracking the cache cleanup inside zope Tracking thread locking inside DB.py, Transaction.py Results: gc python didn't help alleviate the problem Postgres 7.0 instead of 6.5.3 didn't help the problem I do see GenericUserFolder and SQLSession objects with the Control_Panel_Debug screen, and they do not seem to go away. I wrote a simple python script to do nothing more than authenticate (log in) using the GenericUserFolder method docLogin. The memory usage quickly grows out of control. After waiting 15 minutes(my cookie timeout), no decrease in memory usage. The objects are still in the cache also. Restarting zope clears everything up, and starts out nice and clean again. We have a cron job doing this right now. Questions: 1. How does a user's connection resources etc. get cleaned up by Zope after a cookie timeout? Do I need to do this myself? 2. How does a genericuserfolder's set of objects used by an authenticated user get cleaned up. I can't seem to make those objects get reclaimed by the system, even when the user logs off? 3. I am not explicitly removing SQLSession objects. Will my usage counts for these objects remain > 1, thereby never allowing them to get cleaned up, and thereby keeping my usage counts for genericuserfolder's > 1? 4. Since GenericUserFolder inherits from Folder objects, is it possible that the reason my memory grows so fast is that Zope does not release the resources used by a user properly, during an abrupt disconnect? I will assist/help trying different things to try to solve this problem! thanks. eric. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
