[Zope] Re: zope unresponsive

Tue, 27 Feb 2007 05:14:39 -0800

Ok, here is what we have. I did a netstat on both machines, client and server. The client sees and established connection and the server does not. In the server log there is a disconnect. As far as hardware between them, there is a switch (dell powerconnect 6024). Web Server Directors might get hold of it but there are no hops on traceroute. Traceroute only shows the client machine and the server machine.
So the client is just continuously polling the connection but getting nothing back. 


What we are thinking about doing is changing the code in 
zrpc/connection.py to close the connection in wait (line 638 zope 
version 2.9.5) if the wait time gets too large or the poll has happened 
too many times.



We are great at plone development, but have very little backend zope 
development.  Would someone please advise me as to whether this is going 
to cause more problems?


Thanks,
Paul Williams

Paul Williams wrote:

I have posted this several times, but have not until now been able to 
get DeadlockDebugger installed.  I see several people have had this 
problem, but no-one has posted a solution.



 


zope 2.9.5 + zeo

pythonm2.4.3

Red Hat RHEL 4

Plone 2.5.1


 

Our zeo clients hang intermittently.  We have no way of reproducing the 

problem, but it occurs daily.  The client hangs and a restart seems to fix the 


problem.


 


In the event log with tracing on we get


 


Trace zeo.zrpc.Connection(C) wait(16697) {server:8100} pending, async=0


 


There are hundreds to thousands of these until the server is restarted.


 


In the zeo log we get


 


Error caught in asyncor asyncore.py


 


error:(110,'Connection timed out')


 

We have been trying to track this down and have had no luck.  Does anyone have 


any suggestions?  Below is our deadlock debugger output


 

 


Threads traceback dump at 2007-02-23 15:26:50


 

Thread -1269564496 (GET 
/VirtualHostBase/https/soawds:443/VirtualHostRoot/Content///training):



  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZServer/PubCore/ZServerPublisher.py", 
line 23, in __init__



  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZPublisher/Publish.py", 
line 395, in publish_module



  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZPublisher/Publish.py", 
line 196, in publish_module_standard



  File 
"/apps1/zope2.9.5/navo_instance/Products/PlacelessTranslationService/PatchStringIO.py", 
line 34, in new_publish


    x = Publish.old_publish(request, module_name, after_list, debug)


  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZPublisher/Publish.py", 
line 115, in publish



  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZPublisher/mapply.py", 
line 88, in mapply



  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZPublisher/Publish.py", 
line 41, in call_object



  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/Shared/DC/Scripts/Bindings.py", 
line 311, in __call__



  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/Shared/DC/Scripts/Bindings.py", 
line 348, in _bindAndExec



  File 
"/apps1/zope2.9.5/navo_instance/Products/CMFCore/FSPageTemplate.py", 
line 195, in _exec


    result = self.pt_render(extra_context=bound_names)


  File 
"/apps1/zope2.9.5/navo_instance/Products/CacheSetup/patch_cmf.py", line 
38, in FSPT_pt_render


    result = FSPageTemplate.inheritedAttribute('pt_render')(


  File 
"/apps1/zope2.9.5/navo_instance/Products/CacheSetup/patch_cmf.py", line 
92, in PT_pt_render


    tal=not source, strictinsert=0)()


  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/TAL/TALInterpreter.py", 
line 238, in __call__



  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/TAL/TALInterpreter.py", 
line 281, in interpret



  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/TAL/TALInterpreter.py", 
line 749, in do_useMacro



  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/TAL/TALInterpreter.py", 
line 281, in interpret



  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/TAL/TALInterpreter.py", 
line 457, in do_optTag_tal



  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/TAL/TALInterpreter.py", 
line 442, in do_optTag



  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/TAL/TALInterpreter.py", 
line 437, in no_tag



  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/TAL/TALInterpreter.py", 
line 281, in interpret



  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/TAL/TALInterpreter.py", 
line 749, in do_useMacro



  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/TAL/TALInterpreter.py", 
line 281, in interpret



  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/TAL/TALInterpreter.py", 
line 507, in do_setLocal_tal



  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/Products/PageTemplates/TALES.py", 
line 221, in evaluate



  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/Products/PageTemplates/Expressions.py", 
line 185, in __call__



  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/Products/PageTemplates/Expressions.py", 
line 180, in _eval



  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/Products/PageTemplates/Expressions.py", 
line 85, in render



  File 
"/apps1/zope2.9.5/navo_instance/Products/CMFPlone/browser/plone.py", 
line 66, in globalize


    self._initializeData(options=options)


  File 
"/apps1/zope2.9.5/navo_instance/Products/CMFPlone/browser/plone.py", 
line 147, in _initializeData


    self._data['language'] =  self.request.get('language', None) or \


  File "/apps1/zope2.9.5/navo_instance/Products/Archetypes/ClassGen.py", 
line 58, in generatedAccessor


    return schema[name].get(self, **kw)


  File "/apps1/zope2.9.5/navo_instance/Products/Archetypes/Field.py", 
line 802, in get


    value = ObjectField.get(self, instance, **kwargs)


  File "/apps1/zope2.9.5/navo_instance/Products/Archetypes/Field.py", 
line 671, in get


    return self.getStorage(instance).get(self.getName(), instance, **kwargs)


  File 
"/apps1/zope2.9.5/navo_instance/Products/Archetypes/Storage/__init__.py", 
line 175, in get


    value = base._md[name]


  File 
"/var/tmp/python2.4-2.4.3-root/apps1/python/lib/python2.4/UserDict.py", 
line 17, in __getitem__


    def __getitem__(self, key): return self.data[key]


  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZODB/Connection.py", 
line 732, in setstate



  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZODB/Connection.py", 
line 786, in _setstate



  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZODB/serialize.py", 
line 604, in setGhostState



  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZODB/serialize.py", 
line 597, in getState



  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZODB/serialize.py", 
line 471, in _persistent_load



  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZODB/serialize.py", 
line 537, in load_oid



  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZODB/Connection.py", 
line 201, in get



  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZEO/ClientStorage.py", 
line 746, in load



  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZEO/ClientStorage.py", 
line 760, in loadEx



 

Thread -1290544208 (GET 
/VirtualHostBase/https/soawds:443/VirtualHostRoot/Content//nav):



  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZServer/PubCore/ZServerPublisher.py", 
line 23, in __init__



  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZPublisher/Publish.py", 
line 395, in publish_module



  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZPublisher/Publish.py", 
line 196, in publish_module_standard



  File 
"/apps1/zope2.9.5/navo_instance/Products/PlacelessTranslationService/PatchStringIO.py", 
line 34, in new_publish


    x = Publish.old_publish(request, module_name, after_list, debug)


  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZPublisher/Publish.py", 
line 106, in publish



  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZPublisher/BaseRequest.py", 
line 366, in traverse



  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZODB/Connection.py", 
line 732, in setstate



  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZODB/Connection.py", 
line 786, in _setstate



  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZODB/serialize.py", 
line 604, in setGhostState



  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZODB/serialize.py", 
line 597, in getState



  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZODB/serialize.py", 
line 471, in _persistent_load



  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZODB/serialize.py", 
line 537, in load_oid



  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZODB/Connection.py", 
line 201, in get



  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZEO/ClientStorage.py", 
line 746, in load



  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZEO/ClientStorage.py", 
line 760, in loadEx



 

Thread -1246884944 (GET 
/VirtualHostBase/https/soawds:443/VirtualHostRoot/Content/carrier.jpg):



  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZServer/PubCore/ZServerPublisher.py", 
line 23, in __init__



  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZPublisher/Publish.py", 
line 395, in publish_module



  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZPublisher/Publish.py", 
line 196, in publish_module_standard



  File 
"/apps1/zope2.9.5/navo_instance/Products/PlacelessTranslationService/PatchStringIO.py", 
line 34, in new_publish


    x = Publish.old_publish(request, module_name, after_list, debug)


  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZPublisher/Publish.py", 
line 115, in publish



  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZPublisher/mapply.py", 
line 88, in mapply



  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZPublisher/Publish.py", 
line 41, in call_object



  File 
"/apps1/zope2.9.5/navo_instance/Products/ATContentTypes/content/base.py", 
line 414, in index_html


    if data:


  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZODB/Connection.py", 
line 732, in setstate



  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZODB/Connection.py", 
line 786, in _setstate



  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZODB/serialize.py", 
line 604, in setGhostState



  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZODB/serialize.py", 
line 597, in getState



  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZODB/serialize.py", 
line 471, in _persistent_load



  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZODB/serialize.py", 
line 537, in load_oid



  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZODB/Connection.py", 
line 201, in get



  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZEO/ClientStorage.py", 
line 746, in load



  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZEO/ClientStorage.py", 
line 769, in loadEx



  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZEO/ServerStub.py", 
line 192, in loadEx



  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZEO/zrpc/connection.py", 
line 531, in call



  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZEO/zrpc/connection.py", 
line 638, in wait



  File 
"/var/tmp/python2.4-2.4.3-root/apps1/python/lib/python2.4/asyncore.py", 
line 122, in poll


    r, w, e = select.select(r, w, e, timeout)


 

Thread -1280054352 (GET 
/VirtualHostBase/https/soawds:443/VirtualHostRoot/Content/):



  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZServer/PubCore/ZServerPublisher.py", 
line 23, in __init__



  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZPublisher/Publish.py", 
line 395, in publish_module



  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZPublisher/Publish.py", 
line 196, in publish_module_standard



  File 
"/apps1/zope2.9.5/navo_instance/Products/PlacelessTranslationService/PatchStringIO.py", 
line 34, in new_publish


    x = Publish.old_publish(request, module_name, after_list, debug)


  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZPublisher/Publish.py", 
line 106, in publish



  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZPublisher/BaseRequest.py", 
line 366, in traverse



  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZODB/Connection.py", 
line 732, in setstate



  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZODB/Connection.py", 
line 786, in _setstate



  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZODB/serialize.py", 
line 604, in setGhostState



  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZODB/serialize.py", 
line 597, in getState



  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZODB/serialize.py", 
line 471, in _persistent_load



  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZODB/serialize.py", 
line 537, in load_oid



  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZODB/Connection.py", 
line 201, in get



  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZEO/ClientStorage.py", 
line 746, in load



  File 
"/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZEO/ClientStorage.py", 
line 760, in loadEx



 


End of dump


 


Thank you,

Paul Williams


 



------------------------------------------------------------------------

_______________________________________________
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **

(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce

 http://mail.zope.org/mailman/listinfo/zope-dev )


_______________________________________________
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **

(Related lists - 
http://mail.zope.org/mailman/listinfo/zope-announce

http://mail.zope.org/mailman/listinfo/zope-dev )






















					Reply via email to