[Zope] Zeo Client Hanging Unresponsive
I have posted this several times, but have not until now been able to
get DeadlockDebugger installed.
zope 2.9.5 + zeo
pythonm2.4.3
Red Hat RHEL 4
Plone 2.5.1
Our zeo clients hang intermittently. We have no way of reproducing the
problem, but it occurs daily. The client hangs and a restart seems to
fix the
problem.
In the event log with tracing on we get
Trace zeo.zrpc.Connection(C) wait(16697) {server:8100} pending, async=0
There are hundreds to thousands of these until the server is restarted.
In the zeo log we get
Error caught in asyncor asyncore.py
error:(110,'Connection timed out')
We have been trying to track this down and have had no luck. Does
anyone have
any suggestions? Below is our deadlock debugger output
Threads traceback dump at 2007-02-23 15:26:50
Thread -1269564496 (GET
/VirtualHostBase/https/soawds:443/VirtualHostRoot/Content///training):
File
/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZServer/PubC
ore/ZServerPublisher.py, line 23, in __init__
File
/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZPublisher/P
ublish.py, line 395, in publish_module
File
/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZPublisher/P
ublish.py, line 196, in publish_module_standard
File
/apps1/zope2.9.5/navo_instance/Products/PlacelessTranslationService/Pat
chStringIO.py, line 34, in new_publish
x = Publish.old_publish(request, module_name, after_list, debug)
File
/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZPublisher/P
ublish.py, line 115, in publish
File
/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZPublisher/m
apply.py, line 88, in mapply
File
/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZPublisher/P
ublish.py, line 41, in call_object
File
/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/Shared/DC/Sc
ripts/Bindings.py, line 311, in __call__
File
/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/Shared/DC/Sc
ripts/Bindings.py, line 348, in _bindAndExec
File
/apps1/zope2.9.5/navo_instance/Products/CMFCore/FSPageTemplate.py,
line 195, in _exec
result = self.pt_render(extra_context=bound_names)
File
/apps1/zope2.9.5/navo_instance/Products/CacheSetup/patch_cmf.py, line
38, in FSPT_pt_render
result = FSPageTemplate.inheritedAttribute('pt_render')(
File
/apps1/zope2.9.5/navo_instance/Products/CacheSetup/patch_cmf.py, line
92, in PT_pt_render
tal=not source, strictinsert=0)()
File
/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/TAL/TALInter
preter.py, line 238, in __call__
File
/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/TAL/TALInter
preter.py, line 281, in interpret
File
/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/TAL/TALInter
preter.py, line 749, in do_useMacro
File
/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/TAL/TALInter
preter.py, line 281, in interpret
File
/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/TAL/TALInter
preter.py, line 457, in do_optTag_tal
File
/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/TAL/TALInter
preter.py, line 442, in do_optTag
File
/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/TAL/TALInter
preter.py, line 437, in no_tag
File
/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/TAL/TALInter
preter.py, line 281, in interpret
File
/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/TAL/TALInter
preter.py, line 749, in do_useMacro
File
/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/TAL/TALInter
preter.py, line 281, in interpret
File
/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/TAL/TALInter
preter.py, line 507, in do_setLocal_tal
File
/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/Products/Pag
eTemplates/TALES.py, line 221, in evaluate
File
/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/Products/Pag
eTemplates/Expressions.py, line 185, in __call__
File
/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/Products/Pag
eTemplates/Expressions.py, line 180, in _eval
File
/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/Products/Pag
eTemplates/Expressions.py, line 85, in render
File
/apps1/zope2.9.5/navo_instance/Products/CMFPlone/browser/plone.py,
line 66, in globalize
self._initializeData(options=options)
File
/apps1/zope2.9.5/navo_instance/Products/CMFPlone/browser/plone.py,
line 147, in _initializeData
self._data['language'] = self.request.get('language', None) or \
File /apps1/zope2.9.5/navo_instance/Products/Archetypes/ClassGen.py,
line 58, in generatedAccessor
return schema[name].get(self, **kw)
File /apps1/zope2.9.5/navo_instance/Products/Archetypes/Field.py,
line 802, in get
value = ObjectField.get(self, instance, **kwargs)
File /apps1/zope2.9.5/navo_instance/Products/Archetypes/Field.py,
line 671, in get
return
Re: [Zope] Zeo Client hanging
On Wed, Oct 18, 2006 at 07:32:38AM +0200, Maciej Wisniowski wrote:
zope 2.8.5 + zeo
python 2.3.4 (red hat distribution)
Red Hat RHEL 4
Plone 2.1.2
Our zeo clients hang intermittently. We have no way of reproducing the
problem, but it occurs daily. The client hangs and a restart seems to
fix the
problem.
Install DeadlockDebugger and you'll be able to see what is
the state of your threads when Zope becomes unresponsive.
In the event log with tracing on we get
Trace zeo.zrpc.Connection(C) wait(16697) {server:8100} pending, async=0
There are hundreds to thousands of these until the server is restarted.
In the zeo log we get
Error caught in asyncore asyncore.py
error:(110,'Connection timed out')
I'm not sure what these errors are. Maybe firewall between zeo
server and zeo client closes the connections or something like that?
That was my guess too.
See this thread:
http://aspn.activestate.com/ASPN/Mail/Message/zope-list/2916870
--
Paul Winkler
http://www.slinkp.com
___
Zope maillist - Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Zeo Client hanging
I see references to asyncore.py near the end of the thread listing when
using DeadLockDebugger and seeing a stuck thread process. The tread
eventually clears out but sometimes an impatient user reloads, reloads,
reloads...and then all 4 threads are stuck on exact same thing. Then you
can see the same URL jump to the other Zope/ZEO client and same thing
happens there. Another few reloads and one user has effectively crashed
the siteuntil the threads clear themselves out. Usually is a catalog
search for a news article or something spanning our 6 years of online
news articles searching on the word 'bob' or something like that.
+1 for DeadlockDebugger... love it.
Allen
Paul Winkler wrote:
On Wed, Oct 18, 2006 at 07:32:38AM +0200, Maciej Wisniowski wrote:
zope 2.8.5 + zeo
python 2.3.4 (red hat distribution)
Red Hat RHEL 4
Plone 2.1.2
Our zeo clients hang intermittently. We have no way of reproducing the
problem, but it occurs daily. The client hangs and a restart seems to
fix the
problem.
Install DeadlockDebugger and you'll be able to see what is
the state of your threads when Zope becomes unresponsive.
In the event log with tracing on we get
Trace zeo.zrpc.Connection(C) wait(16697) {server:8100} pending, async=0
There are hundreds to thousands of these until the server is restarted.
In the zeo log we get
Error caught in asyncore asyncore.py
error:(110,'Connection timed out')
I'm not sure what these errors are. Maybe firewall between zeo
server and zeo client closes the connections or something like that?
That was my guess too.
See this thread:
http://aspn.activestate.com/ASPN/Mail/Message/zope-list/2916870
___
Zope maillist - Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] Zeo Client hanging
Hello,We have been having problem with our production servers. We currently have:zope 2.8.5 + zeopython 2.3.4 (red hat distribution)Red Hat RHEL 4Plone 2.1.2Our zeo clients hang intermittently. We have no way of reproducing the problem, but it occurs daily. The client hangs and a restart seems to fix the problem.In the event log with tracing on we getTrace zeo.zrpc.Connection(C) wait(16697) {server:8100} pending, async=0There are hundreds to thousands of these until the server is restarted.In the zeo log we getError caught in asyncore asyncore.pyerror:(110,'Connection timed out')We have been trying to track this down and have had no luck. Does anyone have any suggestions?Thanks,Paul
___
Zope maillist - Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Zeo Client hanging
zope 2.8.5 + zeo
python 2.3.4 (red hat distribution)
Red Hat RHEL 4
Plone 2.1.2
Our zeo clients hang intermittently. We have no way of reproducing the
problem, but it occurs daily. The client hangs and a restart seems to
fix the
problem.
Install DeadlockDebugger and you'll be able to see what is
the state of your threads when Zope becomes unresponsive.
In the event log with tracing on we get
Trace zeo.zrpc.Connection(C) wait(16697) {server:8100} pending, async=0
There are hundreds to thousands of these until the server is restarted.
In the zeo log we get
Error caught in asyncore asyncore.py
error:(110,'Connection timed out')
I'm not sure what these errors are. Maybe firewall between zeo
server and zeo client closes the connections or something like that?
--
Maciej Wisniowski
___
Zope maillist - Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )
