[issue37334] Add a cancel method to asyncio Queues

Martin Teichmann added the comment: This is a years old issue, unfortunately it never got neither merged nor rejected. I just rebased it and hope somebody could finish the review. -- versions: +Python 3.11 -Python 3.8 ___ Python tracker

[issue37334] Add a cancel method to asyncio Queues

Martin Teichmann added the comment: I do not think that exposing the lists of futures does any good. I cannot come up with a semantics that could be implemented with that other than the one proposed. Also I think that close() or cancel() is something a reader intuitively understands, while

[issue37334] Add a cancel method to asyncio Queues

Andrew Svetlov added the comment: Lists of producers/consumers are just lists of future objects. I don't think that we need to expose futures in high-level public API. -- ___ Python tracker

[issue37334] Add a cancel method to asyncio Queues

Change by Emmanuel Arias : -- nosy: +eamanu ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue37334] Add a cancel method to asyncio Queues

Yury Selivanov added the comment: > 1. A CancelledError (or maybe`QueueCancelled`?) exception is raised in all > producers and consumers ) - this gives a producer a chance to handle the > error and do something with the waiting item that could not be `put()` > 2. Items currently on the

[issue37334] Add a cancel method to asyncio Queues

Andrew Svetlov added the comment: My point is that closing (or cancellation) should be one-way ticket. Is there a case where continuing after stopping? -- ___ Python tracker

[issue37334] Add a cancel method to asyncio Queues

Martin Teichmann added the comment: Yes, in the one-producer-many-consumers situation on can indeed to the trick with the None. But this is just a clumsy hack, cancelling the tasks is IMHO more in line with asyncio. In the many-producers-one-consumer scenario this does not work. The one

[issue37334] Add a cancel method to asyncio Queues

Caleb Hattingh added the comment: Ok, I see now. The improvement with only a single producer/consumer might be marginal, but the proposition that `queue.cancel()` might simplify the situation with multiple producers and/or consumers is more compelling. Usually, assuming M producers and N

[issue37334] Add a cancel method to asyncio Queues

Martin Teichmann added the comment: Hi Andrew, I still don't get your point. First, this is an extension to the asyncio library, so concurrency is not an issue. And sure, if you call random methods of an object without any reason the outcome won't be anything useful, why, in your example,

[issue37334] Add a cancel method to asyncio Queues

Andrew Svetlov added the comment: 1. Suppose we have 2 concurrent producers, a single queue and a consumer. 2. The first producer puts several items into the queue and then calls q.close() 3. The second producer also puts several items. It doesn't matter the second producer closes the queue

[issue37334] Add a cancel method to asyncio Queues

Yury Selivanov added the comment: This seems like a useful idea. I recommend to write a test implementation and play with it. Andrew: > I think the proposal makes the queues API more error-prone: concurrent put() > and close() produces unpredictable result on get() side. How? Can you

[issue37334] Add a cancel method to asyncio Queues

Martin Teichmann added the comment: Given the reactions I gather "close" is a better name for the method, so I changed it accordingly. In the current implementation, items that had been put on the queue but not processed yet still get processed after the close, and I think this is the

[issue37334] Add a cancel method to asyncio Queues

Andrew Svetlov added the comment: Agree with Caleb. The more I think the more I doubt about the proposal. Cancellation is for tasks, not for queues or locks. When should I cancel a queue but cannot cancel a task? -- ___ Python tracker

[issue37334] Add a cancel method to asyncio Queues

Caleb Hattingh added the comment: I'm interested in how this change would affect the pattern of shutting down a queue-processing task. How would one decide between whether to cancel the queue or the task? (I'm asking for real, this is not an objection to the PR). For example, looking at

[issue37334] Add a cancel method to asyncio Queues

Martin Teichmann added the comment: I also thought about `.close()` but then found `.cancel()` more intuitive. But intuition is not universal, so I am open to any wording. -- ___ Python tracker

[issue37334] Add a cancel method to asyncio Queues

Andrew Svetlov added the comment: Sounds like `.close()` is better name for described behavior. -- ___ Python tracker ___ ___

[issue37334] Add a cancel method to asyncio Queues

Change by Martin Teichmann : -- keywords: +patch pull_requests: +14061 stage: -> patch review pull_request: https://github.com/python/cpython/pull/14227 ___ Python tracker

[issue37334] Add a cancel method to asyncio Queues

New submission from Martin Teichmann : When working with queues, it is not uncommon that at some point the producer stops producing data for good, or the consumer stops consuming, for example because a network connection broke down or some user simply closed the session. In this situation it