[issue26259] Memleak when repeated calls to asyncio.queue.Queue.get is performed, without push to queue.

Andrew Svetlov added the comment: Duplicate of https://bugs.python.org/issue31620 -- nosy: +asvetlov resolution: -> duplicate stage: -> resolved status: open -> closed superseder: -> asyncio.Queue leaks memory if the queue is empty and consumers poll it

[issue26259] Memleak when repeated calls to asyncio.queue.Queue.get is performed, without push to queue.

Suren Nihalani added the comment: @cjrh brought up that this issue is the same as https://bugs.python.org/issue31620. I put up a PR there. Can people review that and then we can close this? -- nosy: +snihalani ___ Python

[issue26259] Memleak when repeated calls to asyncio.queue.Queue.get is performed, without push to queue.

Changes by Guido van Rossum : -- nosy: -gvanrossum ___ Python tracker ___ ___

[issue26259] Memleak when repeated calls to asyncio.queue.Queue.get is performed, without push to queue.

Changes by STINNER Victor : -- nosy: -haypo ___ Python tracker ___ ___

[issue26259] Memleak when repeated calls to asyncio.queue.Queue.get is performed, without push to queue.

Bartek Biernacki added the comment: I ran into similar problem with many getters timing out and staying in memory until some put is called. I think this may be solved by using a callback on Future which would clean it if it was cancelled. I am presenting the idea on attached poc. If you

[issue26259] Memleak when repeated calls to asyncio.queue.Queue.get is performed, without push to queue.

Jonas Brunsgaard added the comment: You are right that get_nowait() is the correct api for my use case, using get_nowait() nothing is pushed to the internal _getters deque. The reason for my us of get() is that job futures are created one place in the code and then thrown in a processing

[issue26259] Memleak when repeated calls to asyncio.queue.Queue.get is performed, without push to queue.

Jonas Brunsgaard added the comment: Okay I thoroughly read the code again. Can you describe the architectural changes to the code regarding a patch, I will do a proposal. But I have to know we are on the same page, so I do not waste my and your time :) --

[issue26259] Memleak when repeated calls to asyncio.queue.Queue.get is performed, without push to queue.

Guido van Rossum added the comment: First, you wrote "The easy solution was to check if the queue was empty and skip the read (get call) if there was nothing in the queue" -- which is exactly what get_nowait() does. As to a patch, I think you should just describe how you would change the code,

[issue26259] Memleak when repeated calls to asyncio.queue.Queue.get is performed, without push to queue.

Jonas Brunsgaard added the comment: In my particular case, I developed an application close to beanstalkd, but with redis as "engine". I did create a callbackback reader class for users to subclass, the callbackreader is checking every second, on every tube(queue.Object). If new data has

[issue26259] Memleak when repeated calls to asyncio.queue.Queue.get is performed, without push to queue.

Guido van Rossum added the comment: I think this is expected behavior when you do a lot of get() calls that are cancelled by a timeout. Each call leaves a cancelled Future in the deque self._getters; these are removed only when _wakeup_next() is called (by put or put_nowait). Why is your app

[issue26259] Memleak when repeated calls to asyncio.queue.Queue.get is performed, without push to queue.

Guido van Rossum added the comment: > The easy solution was to check if the queue was empty and skip the read (get > call) if there was nothing in the queue. There's an API for that -- get_nowait(). I'm happy to review a fix if you come up with one. But I currently don't have time to devise a

[issue26259] Memleak when repeated calls to asyncio.queue.Queue.get is performed, without push to queue.

New submission from Jonas Brunsgaard: When making repeated calls to queue.get, memory is building up and is not freed until queue.push is called. I wrote this little program to show my findings. The program will perform a lot of calls to queue.get and once every 60 seconds a queue.push is