Re: [Zope-dev] How to make Zope fail nicely under high load?

[Erik A . Dahl]

using the _v_ variables won't work for this since different threads 
won't agree on who was latest.  Its definitely a bad idea to do queries 
for these three data points.  What about putting them in the 
temp_folder then you can still use ZEO with a bunch of clients that 
will all share the same data but you don't need to write the values to 
disk every time they change.  Even putting them in a normal ZODB 
storage would be a _lot_ faster than doing a query.

-EAD

On Feb 12, 2004, at 8:19 AM, Bjorn Stabell wrote:

Toby wrote:
One of the optimization we're thinking of is storing results of
ZCatalog searches (e.g., number of replies to postings) in volatile
variables so we don't have to run the catalog search at all.  We'd
like to use memory space shared between threads for this.
Using ZEO would require us to store this in the ZODB.
Storing that in ZODB would be a bad idea. Theres no reason to
think that this cache would be faster than ZCatalog.
I dont see why you would be *required* to store in ZODB...
just dont share your cache between publisher threads on different
Zope instances.
(my apologies if this is obvious)
Okay, for example, since it's a BBS we need to have quick access to:

        the latest poster
        the latest posting
        the latest registered member
All of these are complex queries (in the case of the latest registered
member it's not even a query but a brute-force search).  What I think
most ASP/PHP boards do is that they store these values in the database
instead of querying for them all the time.
Since many of these can be found through queries, they don't really 
need
persistent storage, but updates need to be seen by all threads; thus 
the
requirement for ZODB if we use ZEO, but the possibility to use (even
faster) shared memory if we don't use ZEO.

In effect we want to do something like this:

def getLatestPoster(self):
        if not hasattr(self, '_v_latestPoster'):
                self._v_latestPoster = whatever_query()
        return self._v_latestPoster
                return self._v_latestPoster
def setLatestPoster(self, poster):
        self._v_latestPoster = poster
But instead of having thread-local variables we wanted to use Dieter
Maurer's SharedResource in order to share the cache between threads:
http://www.dieter.handshake.de/pyprojects/zope/SharedResource.html

Regards,
--
Bjorn
_______________________________________________
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists -
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )


_______________________________________________
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )