Re: [Zope-dev] Zope reference counts
On Tue, Jan 21, 2003 at 04:27:05PM +, Toby Dickenson wrote: On Tuesday 21 January 2003 4:14 pm, Ivo van der Wijk wrote: Does this mean each thread keeps it's own cache pool of objects? yes Do these references point to the same object instances, or are these copied as well? They are independant copies I have the impression the memoryconsumption grows with the number of threads - is this true? yes Cheers I hope this helps It clarifies alot. Why exactly is this? concurrency issues? Is there a way to optimize this? I assume it's an issue on the ZODB level? Cheers Ivo -- Drs. I.R. van der Wijk -=- Brouwersgracht 132 Amaze Internet Services V.O.F. 1013 HA Amsterdam, NL -=- T +31-20-4688336 F +31-20-4688337Linux/Web/Zope/SQL/MMBase W http://www.amaze.nlE [EMAIL PROTECTED] Network Solutions W http://vanderwijk.info E [EMAIL PROTECTED] Consultancy PGP http://vanderwijk.info/pgp -=- ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: AdaptableStorage
seb bacon wrote: I'd prefer just to have a method somewhere that, as Shane suggested, could be hit by URL, etc. A whole seperate server seems like overkill... But what about, for example, databases which don't have an efficient way to do callbacks to external applications? You may have to do a SELECT id FROM tblObjects WHERE timestamp some_time or a similar kludge from a polling server. You may want this server to reside at the same location as the RDBMS, rather than as a thread in Zope. Well, if by server you could mean script that gets run by cron every 1 minute and hits a URL in Zope if something has changed, then I might be in agreement ;-) I'm worrying that if we are not to be restricted to Oracle or bleeding edge kernels, the notification part of the cache invalidation scheme may be (a) kludgy, (b) inefficient, and (c) utterly different in design between different storages. I think this is as such is life problem. Provided AdaptableStorage provides some way (exposed url?) for an external process to say that things have changed, I think that's the best form of flexibility we can provide. cheers, Chris ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: AdaptableStorage
On Wed, 22 Jan 2003, Chris Withers wrote: I think this is as such is life problem. Provided AdaptableStorage provides some way (exposed url?) for an external process to say that things have changed, I think that's the best form of flexibility we can provide. I think I'll provide such an URL, then. Thanks for going over this. Shane ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: AdaptableStorage
Chris Withers wrote: seb bacon wrote: I'd prefer just to have a method somewhere that, as Shane suggested, could be hit by URL, etc. A whole seperate server seems like overkill... Well, if by server you could mean script that gets run by cron every 1 minute and hits a URL in Zope if something has changed, then I might be in agreement ;-) Put it like this, I don't mean managed blade cluster running custom distributed architecture :-) ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope reference counts
Toby Dickenson wrote: On Wednesday 22 January 2003 11:43 am, Ivo van der Wijk wrote: Do these references point to the same object instances, or are these copied as well? They are independant copies Is there a way to optimize this? 1. Obviously, use fewer threads. Do you have a component that otherwise benefits from many threads? Also note that it is almost always a better use of memory to increase the cache size than the number of threads (this is the size of each cache, not the total cache size). Are you aware of the activity and cache parameters tab in the database management part of the control panel? If not, you may not be using a 2.6x Zope - there are some significant improvements in cache usage (especially under load) in these versions. Look at the Help on the cache parameters page for more info. seb ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] [PATCH] Forking daemon in debug mode, and new start scripts
On Tuesday 21 January 2003 06:45 pm, Adrian van den Dries wrote: [Sorry about the broken thread] On January 20, Guido van Rossum wrote: You seem to forget that in debugging mode there are other things that happen (or not) besides logging. Ah, the infamous Zope side-effect, aka other cool stuff. :-) Thanks for the heads-up. And you can log to stderr (that's what I use 99% of the time). But this means I can't really write a start script that works (or makes sense) with debug mode both on and off. Redirecting stderr to EVENT_LOG_FILE doesn't really make sense if we're detaching, and is ugly, etc.. I could write some shell polava to figure it out, but: Maybe the debug flag is misnamed? All it really means is don't detach. Yes. I would much prefer an explicit flag for each, because I run a number of dev Zopes on a number of boxes to which I only have ssh access, and I don't want to have to remember to play games with my stdio, or restart Zope if I restart screen, etc. Speaking of the (side-)effects of debug mode, what exactly are they? Is it just the automatic re-loading of DTMLFiles and PageTemplates from disk? There's a lot of code that gets enabled when the debug (also known as development) mode gets turned on. Most of it involves auto-refreshing things from the file system, but there are many other things. Zope generally runs *much* slower in debug mode then in production mode also. -Casey ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope reference counts
On Wed, Jan 22, 2003 at 12:14:27PM +, seb bacon wrote: Toby Dickenson wrote: On Wednesday 22 January 2003 11:43 am, Ivo van der Wijk wrote: Do these references point to the same object instances, or are these copied as well? They are independant copies Is there a way to optimize this? 1. Obviously, use fewer threads. Do you have a component that otherwise benefits from many threads? Also note that it is almost always a better use of memory to increase the cache size than the number of threads (this is the size of each cache, not the total cache size). Are you aware of the activity and cache parameters tab in the database management part of the control panel? If not, you may not be using a 2.6x Zope - there are some significant improvements in cache usage (especially under load) in these versions. I'm working on a 2.3.x instance which I have already upgraded to 2.4.4. It soudns asif a further upgrde is reasonable. Also, the 'activity' tab is new to me (looks interesting ;), and caching settins seem to be ignored mostly until 2.6, right? Cheers Ivo -- Drs. I.R. van der Wijk -=- Brouwersgracht 132 Amaze Internet Services V.O.F. 1013 HA Amsterdam, NL -=- T +31-20-4688336 F +31-20-4688337Linux/Web/Zope/SQL/MMBase W http://www.amaze.nlE [EMAIL PROTECTED] Network Solutions W http://vanderwijk.info E [EMAIL PROTECTED] Consultancy PGP http://vanderwijk.info/pgp -=- ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope reference counts
On Wed, Jan 22, 2003 at 11:57:20AM +, Toby Dickenson wrote: Is there a way to optimize this? 1. Obviously, use fewer threads. Do you have a component that otherwise benefits from many threads? No, except that the server is pretty heavy loaded sometimes, and multiple threads should improve responsiveness (right?) 2. If you have large immutable state that does not hold references to persistent objects, use application-specific code to share this state between threads. Not really. The bloat is caused by a large squishdot site (20.000 comments) Anyway, I'll give 2.6 a try. Thanks! Ivo -- Drs. I.R. van der Wijk -=- Brouwersgracht 132 Amaze Internet Services V.O.F. 1013 HA Amsterdam, NL -=- T +31-20-4688336 F +31-20-4688337Linux/Web/Zope/SQL/MMBase W http://www.amaze.nlE [EMAIL PROTECTED] Network Solutions W http://vanderwijk.info E [EMAIL PROTECTED] Consultancy PGP http://vanderwijk.info/pgp -=- ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope reference counts
On Wednesday 22 January 2003 3:01 pm, Ivo van der Wijk wrote: It soudns asif a further upgrde is reasonable. I think you will have much more joy with 2.6 Also, the 'activity' tab is new to me (looks interesting ;), and caching settins seem to be ignored mostly until 2.6, right? it worked in a way that lead to strange behavior. 2.6 is more predictable, and understandable. -- Toby Dickenson http://www.geminidataloggers.com/people/tdickenson ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
