Hi Glyph, Here is the minimal version
class FooResource(resource.Resource): def render_GET(request): future = asyncio.ensure_future(self.fetch_response(request)) // some async await functions d = Deferred.fromFuture(future) d.addCallback(print_json_response, request) // this is actually where the error is triggered. d.addErrback(lambda failure: failure.trap(defer.CancelledError)) finished_errback = request.notifyFinish() finished_errback.addErrback(self.handle_cancel, d) // simple logs and cancels d by d.cancel() return NOT_DONE_YET The traceback is like 'Traceback (most recent call last):\n File "/usr/local/Cellar/python/3.7.1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/events.py", line 88, in _run\n self._context.run(self._callback, *self._args)\n File "/usr/local/lib/python3.7/site-packages/twisted/internet/defer.py", line 827, in adapt\n adapt.actual.callback(extracted)\n File "/usr/local/lib/python3.7/site-packages/twisted/internet/defer.py", line 460, in callback\n self._startRunCallbacks(result)\n File "/usr/local/lib/python3.7/site-packages/twisted/internet/defer.py", line 568, in _startRunCallbacks\n self._runCallbacks()\n--- <exception caught here> ---\n File "/usr/local/lib/python3.7/site-packages/twisted/internet/defer.py", line 654, in _runCallbacks\n current.result = callback(current.result, *args, **kw)\n File "/Users/wqKhan/twisted_eg/parse_response.py", line 3, in print_json_response\n *print_json_response*(response, request)\n File "/Users/wqKhan/twisted_eg/parse_response.py", line 21, in *print_json_response*\n request.finish()\n File "/usr/local/lib/python3.7/site-packages/twisted/web/server.py", line 268, in finish\n return http.Request.finish(self)\n File "/usr/local/lib/python3.7/site-packages/twisted/web/http.py", line 1071, in finish\n "Request.finish called on a request after its connection was lost; "\nbuiltins.RuntimeError: Request.finish called on a request after its connection was lost; use Request.notifyFinish to keep track of this.\n' On Sun, Aug 4, 2019 at 10:36 PM Glyph <gl...@twistedmatrix.com> wrote: > > > > On Aug 4, 2019, at 5:57 PM, Waqar Khan <wk80...@gmail.com> wrote: > > > > Hi Glyph, > > Thanks for the suggestion. > > I tried the suggestion.. While it fixes the self.channel NoneType > issue.. It creates another issue. > > > > Traceback (most recent call last): > > File > "/usr/local/Cellar/python/3.7.1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/events.py", > line 88, in _run > > self._context.run(self._callback, *self._args) > > File > "/usr/local/lib/python3.7/site-packages/twisted/internet/defer.py", line > 827, in adapt > > adapt.actual.callback(extracted) > > File > "/usr/local/lib/python3.7/site-packages/twisted/internet/defer.py", line > 460, in callback > > self._startRunCallbacks(result) > > File > "/usr/local/lib/python3.7/site-packages/twisted/internet/defer.py", line > 568, in _startRunCallbacks > > self._runCallbacks() > > --- <exception caught here> --- > > File > "/usr/local/lib/python3.7/site-packages/twisted/internet/defer.py", line > 654, in _runCallbacks > > current.result = callback(current.result, *args, **kw) > > ...... > > request.finish() > > File "/usr/local/lib/python3.7/site-packages/twisted/web/server.py", > line 268, in finish > > return http.Request.finish(self) > > File "/usr/local/lib/python3.7/site-packages/twisted/web/http.py", > line 1071, in finish > > "Request.finish called on a request after its connection was lost; " > > builtins.RuntimeError: Request.finish called on a request after its > connection was lost; use Request.notifyFinish to keep track of this. > > Oh, hrm. The idea here is that any code that returns NOT_DONE_YET (i.e. > does asynchronous work to generate a response) should also be tracking > whether it needs to call `.finish()` by watching a `.notifyFinish()` > Deferred. I bet that there are plenty of resources in Twisted which don't > follow this rule, and we should fix those; but possibly we should also make > this error message less stern. Do you have a minimal example? > > > I have got some handle on the issue. I was wondering if you have any > advise on how to debug the issue. > > Somewhere in the codebase are async/await code (from asyncio world). > Hence, when the connection is closed down by client, seems like those > pending coroutines are lingering on. > > This, I have no idea about. What tasks are you starting, when are you > expecting them to get cleaned up, what are they blocking on, how do these > interact with Twisted? There are a couple dozen questions I'd have to know > the answer to in order to even begin debugging this. If you cancel all the > outstanding tasks and look at their tracebacks when exiting it might give > you more of a sense of where they're stuck... > > -g > > _______________________________________________ > Twisted-Python mailing list > Twisted-Python@twistedmatrix.com > https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python >
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python