[issue26194] Undefined behavior for deque.insert() when len(d) == maxlen

Changes by Raymond Hettinger : -- status: open -> closed ___ Python tracker ___

[issue26194] Undefined behavior for deque.insert() when len(d) == maxlen

Martin Panter added the comment: I thought the 3.6.0 NEWS file generally listed bugs fixed since (at least) the 3.5.0 release. But the NEWS in default has no mention of Issue 26194. Serhiy is right that there is nothing more to do for the merging problem. I was just pointing out that

[issue26194] Undefined behavior for deque.insert() when len(d) == maxlen

Raymond Hettinger added the comment: I can't say that I agree with putting an additional entry the 3.6 NEWS. In the past, I've not made news entries in unreleased versions regarding bug fixes in prior versions. That seems pointless to me and it would tend to clutter a document whose size

[issue26194] Undefined behavior for deque.insert() when len(d) == maxlen

Raymond Hettinger added the comment: [Martin] > the default branch never got any NEWS entry in > 58266f5101cc, so maybe it needs to be added? No need for a news entry in default. We haven't released 3.6 yet, so the bugfix from 3.5 is just a carry forward of the corrected behavior. That

[issue26194] Undefined behavior for deque.insert() when len(d) == maxlen

STINNER Victor added the comment: Ok no problem, it was just a suggestion. In asyncio, we try to document behaviour change but asyncio is different. Major functions are still modified, asyncio API is still provisional, but people rely on tehe exact API. --

[issue26194] Undefined behavior for deque.insert() when len(d) == maxlen

Martin Panter added the comment: FYI Raymond, your revision 0731f097157b didn’t actually merge the 3.5 branch (It only had one parent). Also, the default branch never got any NEWS entry in 58266f5101cc, so maybe it needs to be added? -- nosy: +martin.panter

[issue26194] Undefined behavior for deque.insert() when len(d) == maxlen

STINNER Victor added the comment: > FYI Raymond, your revision 0731f097157b didn’t actually merge the 3.5 branch > (It only had one parent). Also, the default branch never got any NEWS entry > in 58266f5101cc, so maybe it needs to be added? Maybe it's also worth to document the change in

[issue26194] Undefined behavior for deque.insert() when len(d) == maxlen

Serhiy Storchaka added the comment: Reopened. Otherwise Martin's comments can be lost in spam. > Maybe it's also worth to document the change in Python 3.5 and 3.6 > documentation with a ".. versionchanged::"? I don't think this is needed. This isn't new feature, this is just a bugfix, old

[issue26194] Undefined behavior for deque.insert() when len(d) == maxlen

Changes by Raymond Hettinger : -- resolution: -> fixed status: open -> closed ___ Python tracker ___

[issue26194] Undefined behavior for deque.insert() when len(d) == maxlen

Roundup Robot added the comment: New changeset 7dabb94bdf63 by Raymond Hettinger in branch '3.5': Issue #26194: Inserting into a full deque to raise an IndexError https://hg.python.org/cpython/rev/7dabb94bdf63 -- ___ Python tracker

[issue26194] Undefined behavior for deque.insert() when len(d) == maxlen

Serhiy Storchaka added the comment: *All* reasons look reasonable. May be discuss this on Python-Dev or ask BDFL? >From my point it looks that correct implementation of the insertion is too >complex and this feature should go only in development release (if any). --

[issue26194] Undefined behavior for deque.insert() when len(d) == maxlen

Raymond Hettinger added the comment: Since slicing is going to be added to deques, it is worth thinking about what the behavior should be there as well: d = deque('abcde', maxlen=7) d[3:4] = 'efghijklm' Side note: repetition behaves like extend() using maxlen to decide how much to

[issue26194] Undefined behavior for deque.insert() when len(d) == maxlen

Serhiy Storchaka added the comment: full_deque2.diff LGTM. But I have added minor suggestions on Rietveld. -- ___ Python tracker ___

[issue26194] Undefined behavior for deque.insert() when len(d) == maxlen

Raymond Hettinger added the comment: My only aversion to raising an exception is that it goes against the original intent of maxlen as giving an automatic-pop-to-make-room behavior. Rather that introduce a discontinuity, I like the "smoothness" of letting d.insert(len(d), obj) follow the

[issue26194] Undefined behavior for deque.insert() when len(d) == maxlen

Josh Rosenberg added the comment: I agree with Tim that arbitrarily deciding that insert should behave more like appendleft is surprising. This really feels like guessing at what the user wants, silently doing something that really isn't predictable. Any of the following seem nicer (not

[issue26194] Undefined behavior for deque.insert() when len(d) == maxlen

Tim Peters added the comment: My opinion doesn't change: I'd rather see an exception. I see no use case for inserting "into the middle" of a full bounded queue. If I had one, it would remain trivial to force the specific behavior I intended. --

[issue26194] Undefined behavior for deque.insert() when len(d) == maxlen

Changes by Raymond Hettinger : Added file: http://bugs.python.org/file41732/full_deque2.diff ___ Python tracker ___

[issue26194] Undefined behavior for deque.insert() when len(d) == maxlen

Raymond Hettinger added the comment: The expected behavior should be "insert normally as if the deque were unbounded and then pop-off the rightmost element to restore the maxlen invariant". The applied fix does that for non-negative indices but gives the wrong result for negative indicies:

[issue26194] Undefined behavior for deque.insert() when len(d) == maxlen

Tim Peters added the comment: I'd raise an exception when trying to insert into a bounded deque that's already full. There's simply no way to guess what was _intended_; it's dead easy for the user to implement what they _do_ intend (first make room by deleting the specific item they no

[issue26194] Undefined behavior for deque.insert() when len(d) == maxlen

Serhiy Storchaka added the comment: What if implement the bahavior described by Raymond for index -len(d) <= i < len(d), but raise an exception if the index is out of this range? The result of deque('abc', maxlen=3).insert(i, 'X') would be: -4: error -3: ['X', 'a', 'b'] -2: ['a', 'X', 'b']

[issue26194] Undefined behavior for deque.insert() when len(d) == maxlen

Serhiy Storchaka added the comment: But the postcondition d[i] == newitem is broken if i is negative. >>> d = deque('ABC', maxlen=3) >>> d.insert(-1, None) >>> d deque(['A', None, 'B'], maxlen=3) I would expected ['A', 'B', None]. -- resolution: fixed -> status: closed -> open

[issue26194] Undefined behavior for deque.insert() when len(d) == maxlen

STINNER Victor added the comment: > -1 is the position before the last element ('C'). After popping-off extra > element the result can be ['A', 'B', None] or ['B', None, 'C']. Oh ok :-) Now I'm confused, I don't know what is the expected behaviour :-) --

[issue26194] Undefined behavior for deque.insert() when len(d) == maxlen

STINNER Victor added the comment: > I would expected ['A', 'B', None]. Hum, it seems consistent with list behaviour, no? >>> x=list("abc") >>> x.insert(-1, "X") >>> x ['a', 'b', 'X', 'c'] -1 is the same than len(x)-1 (2 in this example). deque(['A', None, 'B'], maxlen=3) seems correct to me.

[issue26194] Undefined behavior for deque.insert() when len(d) == maxlen

Serhiy Storchaka added the comment: -1 is the position before the last element ('C'). After popping-off extra element the result can be ['A', 'B', None] or ['B', None, 'C']. -- ___ Python tracker

[issue26194] Undefined behavior for deque.insert() when len(d) == maxlen

Changes by Raymond Hettinger : -- resolution: -> fixed status: open -> closed ___ Python tracker ___

[issue26194] Undefined behavior for deque.insert() when len(d) == maxlen

Roundup Robot added the comment: New changeset ce3d47eaeb21 by Raymond Hettinger in branch '3.5': Issue #26194: Fix undefined behavior for deque.insert() when len(d) == maxlen https://hg.python.org/cpython/rev/ce3d47eaeb21 -- nosy: +python-dev ___

[issue26194] Undefined behavior for deque.insert() when len(d) == maxlen

Serhiy Storchaka added the comment: New behavior looks less surprising to me. Old behavior is even more weird for negative indices: >>> from collections import deque >>> d = deque(range(20), maxlen=10) >>> d deque([10, 11, 12, 13, 14, 15, 16, 17, 18, 19], maxlen=10) >>> d.insert(-3, 'New') >>>

[issue26194] Undefined behavior for deque.insert() when len(d) == maxlen

Raymond Hettinger added the comment: The plain-insert-followed-by-a-pop-on-the-right is giving reasonable results with nice properties: d.insert(0, i) --- 0 --> deque([0], maxlen=4) 1 --> deque([1, 0], maxlen=4) 2 --> deque([2, 1, 0], maxlen=4) 3 --> deque([3,

[issue26194] Undefined behavior for deque.insert() when len(d) == maxlen

Raymond Hettinger added the comment: Another take: Do a regular insertion with no special cases, followed by a pop from the right: >>> for i in range(len(d) + 1): s = list(d) s.insert(i, None) _ = s.pop() print(i, s) 0 [None, 'a', 'b'] 1 ['a',

[issue26194] Undefined behavior for deque.insert() when len(d) == maxlen

Changes by Raymond Hettinger : -- keywords: +patch Added file: http://bugs.python.org/file41717/full_deque.diff ___ Python tracker

[issue26194] Undefined behavior for deque.insert() when len(d) == maxlen

Changes by Raymond Hettinger : Added file: http://bugs.python.org/file41718/full_deque_alt.diff ___ Python tracker ___

[issue26194] Undefined behavior for deque.insert() when len(d) == maxlen

Raymond Hettinger added the comment: Mark, Serhiy and Timmy, do you have any thoughts on what is the right behavior? One option is to always pop the rightmost element to make room, but that results in a weird asymmetry between d.insert(len(d), item) and what d.append(item) would do. I can't