Tim Peters wrote at 2005-6-27 15:38 -0400:
> ...
[Dieter]
>> Probably, "ZEOClientStorage" (and the ZEO server) should use
>> "SO_KEEPALIVE" to enable TCP keepalive messages. However, the default
>> TCP timeouts are probably too high (2 hours) for many ZODB applications
>> (like ours).
[Tim]
>Section 4.2.3.6 of RFC 1122 doesn't read like it thinks keep-alives are
>either portable or appropriate for this. In particular,
>
> It is extremely important to remember that ACK segments that
> contain no data are not reliably transmitted by TCP.
> Consequently, if a keep-alive mechanism is implemented it
> MUST NOT interpret failure to respond to any specific probe
> as a dead connection.
[Dieter]
Then hopefully, TCP knows that and observes it in the
implementation of its "keep-alive" mechanism.
At least, it is quite careful (with Linux default parameters):
Only after 9 keep alive messages are lost, the connection is
considered broken.
>...
[Dieter]
>> I will implement an application specific keep alive mechanism.
[Tim]
>How will you do that?
I attach the script (wrapped up as a Zope "product").
> Would it be appropriate for ZEO to do something
>similar itself?
Yes. It would be very nice.
> If ZEO did, would you still need to bother?
No.
And I would be happy, because I am using internal
undocumented implementation details that already once
changed between ZEO releases.
--
Dieter
# $Id: __init__.py,v 1.1.1.1 2005/06/27 08:37:21 dieter Exp $
'''ZEOKeepAlive.
Our cluster checks made us aware that 'ZEOClientStorage' depends
on the operating system to inform about connection breaks
and that this cannot do this reliably during inactivity periods.
This product starts a thread that periodically sends a keep alive
message to all ZEO connections. Sending will fail when the
connection is no longer intact triggering 'ZEOClientStorage's
automatic reconnect.
The thread is controlled by the environment variable
'ZEO_KEEPALIVE_PERIOD'. If not specified, the keep alive thread is
not started.
For use by non Zope ZEO clients, it defines the function
'startThread'. It has the optional argument 'period' specifying
the period (default: use value given by environment variable
'ZEO_KEEPALIVE_PERIOD').
As in 'ZEOClientStorage', we assume that no communication will hang, not
even under abnormal conditions. The solution would be much more
complex had we not made this assumption.
Logs
We log as 'INFO' whether or not we start the keep alive thread.
We log as 'WARNING' unready storages
We log as 'BLATHER' send and response events for keep alive messages
'''
from os import environ
from time import sleep
from thread import start_new_thread
from zLOG import LOG, INFO, WARNING, BLATHER, ERROR, PANIC
from App.config import getConfiguration
from ZEO.ClientStorage import ClientStorage
_period = None
def startThread(period=None):
global _period
if _period:
raise ValueError('KeepAlive thread already started')
if period is None:
period = environ.get('ZEO_KEEPALIVE_PERIOD')
if period is not None: period = int(period)
period = _period = period and period > 0 and period
_log(INFO,
period and 'Thread started with period %s sec' % period
or 'Thread not started'
)
if period: start_new_thread(_keepAlive, (period,))
def initialize(context):
startThread()
def _keepAlive(period):
try:
while True:
sleep(period)
dbs = getConfiguration().dbtab
for db in dbs.opened.values():
S = db._storage
if isinstance(S, ClientStorage):
name = S.getName()
if S._ready.isSet():
_log(BLATHER, "Sending keep alive for %s" % name)
S._load_lock.acquire()
try:
try:
S._server.get_info()
_log(BLATHER, "Keep alive for %s successful" % name)
except:
_log(ERROR, "Keep alive for %s failed" % name, error=True)
finally:
S._load_lock.release()
else: _log(WARNING, "Connection for %s is down" % name)
except:
_log(PANIC, "Exception in keep alive thread", error=True)
def _log(*args, **kw):
LOG('ZEO KeepAlive', *args, **kw)
_______________________________________________
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/
ZODB-Dev mailing list - ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev
