Robert P. Thille created THRIFT-2883: ----------------------------------------
Summary: TTwisted.py, during ConnectionLost processing: exceptions.RuntimeError: dictionary changed size during iteration Key: THRIFT-2883 URL: https://issues.apache.org/jira/browse/THRIFT-2883 Project: Thrift Issue Type: Bug Affects Versions: 0.9.2 Reporter: Robert P. Thille If any of the errbacks called by the connectionLost routine attempt to make new requests over thrift (retries), the requests dictionary will change size and Python with throw the RuntimeError exception and the clients will not be properly notified of the connectionLost state change. 2014-12-06 02:22:16,808 ERROR twisted Unhandled Error Traceback (most recent call last): File "/usr/lib/pymodules/python2.7/companyutils/service/__init__.py", line 292, in main self.reactor.run() File "/usr/lib/python2.7/dist-packages/twisted/internet/base.py", line 1169, in run self.mainLoop() File "/usr/lib/python2.7/dist-packages/twisted/internet/base.py", line 1181, in mainLoop self.doIteration(t) File "/usr/lib/python2.7/dist-packages/twisted/internet/epollreactor.py", line 379, in doPoll log.callWithLogger(selectable, _drdw, selectable, fd, event) --- <exception caught here> --- File "/usr/lib/python2.7/dist-packages/twisted/python/log.py", line 84, in callWithLogger return callWithContext({"system": lp}, func, *args, **kw) File "/usr/lib/python2.7/dist-packages/twisted/python/log.py", line 69, in callWithContext return context.call({ILogContext: newCtx}, func, *args, **kw) File "/usr/lib/python2.7/dist-packages/twisted/python/context.py", line 118, in callWithContext return self.currentContext().callWithContext(ctx, func, *args, **kw) File "/usr/lib/python2.7/dist-packages/twisted/python/context.py", line 81, in callWithContext return func(*args,**kw) File "/usr/lib/python2.7/dist-packages/twisted/internet/posixbase.py", line 627, in _doReadOrWrite self._disconnectSelectable(selectable, why, inRead) File "/usr/lib/python2.7/dist-packages/twisted/internet/posixbase.py", line 258, in _disconnectSelectable selectable.readConnectionLost(f) File "/usr/lib/python2.7/dist-packages/twisted/internet/tcp.py", line 267, in readConnectionLost self.connectionLost(reason) File "/usr/lib/python2.7/dist-packages/twisted/internet/tcp.py", line 473, in connectionLost self._commonConnection.connectionLost(self, reason) File "/usr/lib/python2.7/dist-packages/twisted/internet/tcp.py", line 287, in connectionLost protocol.connectionLost(reason) File "/usr/lib/pymodules/python2.7/telephus/protocol.py", line 63, in connectionLost self._parent_protocol.connectionLost(self, reason) File "/usr/lib/pymodules/python2.7/thrift/transport/TTwisted.py", line 84, in connectionLost for k, v in self.client._reqs.iteritems(): exceptions.RuntimeError: dictionary changed size during iteration -- This message was sent by Atlassian JIRA (v6.3.4#6332)