Suresh V wrote at 2006-12-22 03:02 +0530:
...
We hiked it up to 60K per thread because we have lots of memory and
wanted to make Zope perform better. Is there a disadvantage to this?
Not, if you have enough RAM.
Note that each connection has its own cache and that you control
the cache size only by the the number of objects it may contain
and not by the size of these objects.
In a standard Zope setup, you cache capacity of 60 K means
that up to 240 k objects can be cached.
If you are using lots of File objects, then the File objects
are split in 64 kB chunks. If your cache happens to be filled
with such chunks, then you need about 15 GB just for ZODB caching
purposes.
When the ZODB reported that it has read some number of objects,
it is very likely that it did indeed.
But even under very little site activity, it seems to be loading a lots
of objects. Is this normal?
It is quite easy to get the persistency design wrong.
To analyse such bad design, I have instrumented our Zope copy to log
information about the communication between a ZEO client (e.g. Zope)
and the ZEO server).
I attach the instrumented module. It is likely that you will not
be able to replace the module in your Zope implementation but you
can get a feeling how to approach such an instrumentation. Watch out
for _doLOG.
##
#
# Copyright (c) 2001, 2002, 2003 Zope Corporation and Contributors.
# All Rights Reserved.
#
# This software is subject to the provisions of the Zope Public License,
# Version 2.1 (ZPL). A copy of the ZPL should accompany this distribution.
# THIS SOFTWARE IS PROVIDED AS IS AND ANY AND ALL EXPRESS OR IMPLIED
# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
# FOR A PARTICULAR PURPOSE
#
##
RPC stubs for interface exported by StorageServer.
# DM 2004-12-10 -- see below
ZEO_MAX_RESPONSE_TIME = 60 # s, maybe controllable via an environment variable?
from time import time
##
# ZEO storage server.
# p
# Remote method calls can be synchronous or asynchronous. If the call
# is synchronous, the client thread blocks until the call returns. A
# single client can only have one synchronous request outstanding. If
# several threads share a single client, threads other than the caller
# will block only if the attempt to make another synchronous call.
# An asynchronous call does not cause the client thread to block. An
# exception raised by an asynchronous method is logged on the server,
# but is not returned to the client.
class StorageServer:
An RPC stub class for the interface exported by ClientStorage.
This is the interface presented by the StorageServer to the
ClientStorage; i.e. the ClientStorage calls these methods and they
are executed in the StorageServer.
See the StorageServer module for documentation on these methods,
with the exception of _update(), which is documented here.
def __init__(self, rpc):
Constructor.
The argument is a connection: an instance of the
zrpc.connection.Connection class.
self.rpc = rpc
# Wait until we know what version the other side is using.
# DM 2004-12-10: this free running loop occasionally did
# not finish (reasons still unknown) and consumed an excessive
# amount of CPU time. We call pending with a timeout of
# 30s (this is effective only for synchronous mode!)
# and in addition check that we finish within ZEO_MAX_RESPONSE_TIME.
#while rpc.peer_protocol_version is None:
#rpc.pending()
st = time()
while rpc.peer_protocol_version is None:
if time() - st ZEO_MAX_RESPONSE_TIME:
# the server took too long to report its version
rpc.close()
raise EnvironmentError(ZEO server's initial response exceeded %ss % ZEO_MAX_RESPONSE_TIME)
rpc.pending(30)
if rpc.peer_protocol_version == 'Z200':
self.lastTransaction = lambda: None
self.getInvalidations = lambda tid: None
self.getAuthProtocol = lambda: None
def extensionMethod(self, name):
return ExtensionMethodWrapper(self.rpc, name).call
##
# Register current connection with a storage and a mode.
# In effect, it is like an open call.
# @param storage_name a string naming the storage. This argument
#is primarily for backwards compatibility with servers
#that supported multiple storages.
# @param read_only boolean
# @exception ValueError unknown storage_name or already registered
# @exception ReadOnlyError storage is read-only and a read-write
#connectio was requested
def register(self, storage_name, read_only):
