On Tue, 28 Mar 2023 at 12:26, Travis Griggs <travisgri...@gmail.com> wrote: > > A while ago I chose to use a deque that is shared between two threads. I did > so because the docs say: > > "Deques support thread-safe, memory efficient appends and pops from either > side of the deque with approximately the same O(1) performance in either > direction.” > > (https://docs.python.org/3.11/library/collections.html?highlight=deque#collections.deque) > > Earlier today, looking through some server logs I noticed that from time to > I’m getting a > > RuntimeError: deque mutated during iteration > > I guess this surprised me. When I see “thread safe”, I don’t expect to get > errors. >
I'd like to see your code, but here's an example with no threads whatsoever that has the same error: >>> from collections import deque >>> q = deque([1, 2, 3, 4]) >>> for item in q: ... if item % 2: q.append(item * 2) ... print(item) ... 1 Traceback (most recent call last): File "<stdin>", line 1, in <module> RuntimeError: deque mutated during iteration This error comes from having an active iterator, then mutating the deque, then continuing to iterate. That MAY be a result of threading, but it isn't necessarily. For threaded usage, I would recommend restricting yourself to append/appendleft/pop/popleft (the standard mutators), with any operations on the entire queue being done on a copy instead (either q.copy() or list(q) depending on what you need). The act of taking a copy should itself be thread-safe, and obviously anything done on a separate copy will be independent of changes to the original. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
