[issue29438] use after free in key sharing dict

Change by Serhiy Storchaka : -- pull_requests: -960 ___ Python tracker ___ ___

[issue29438] use after free in key sharing dict

Changes by Donald Stufft : -- pull_requests: +960 ___ Python tracker ___ ___

[issue29438] use after free in key sharing dict

INADA Naoki added the comment: New changeset 2294f3aee14a6074b17c67ef936c607430bb3c7a by INADA Naoki in branch 'master': bpo-29438: fixed use-after-free in key sharing dict (#17) https://github.com/python/cpython/commit/2294f3aee14a6074b17c67ef936c607430bb3c7a --

[issue29438] use after free in key sharing dict

INADA Naoki added the comment: New changeset 06a4fcb2458c5904968b5c8fe6b64940ba83a50d by INADA Naoki in branch '3.5': bpo-29438: Fixed use-after-free in key sharing dict (#40) https://github.com/python/cpython/commit/06a4fcb2458c5904968b5c8fe6b64940ba83a50d --

[issue29438] use after free in key sharing dict

INADA Naoki added the comment: New changeset 89ddffbe9dcb38b79f99563b0d4d594d1105a192 by INADA Naoki in branch '3.6': bpo-29438: fixed use-after-free in key sharing dict (#39) https://github.com/python/cpython/commit/89ddffbe9dcb38b79f99563b0d4d594d1105a192 --

[issue29438] use after free in key sharing dict

Changes by Larry Hastings : -- pull_requests: +621 ___ Python tracker ___ ___

[issue29438] use after free in key sharing dict

INADA Naoki added the comment: All pull requests are merged. https://github.com/python/cpython/pull/17 (master) https://github.com/python/cpython/pull/39 (3.6) https://github.com/python/cpython/pull/40 (3.5) -- resolution: -> fixed stage: patch review -> resolved status: open -> closed

[issue29438] use after free in key sharing dict

Changes by INADA Naoki : -- pull_requests: +30 ___ Python tracker ___ ___

[issue29438] use after free in key sharing dict

INADA Naoki added the comment: I'll send PR to github. Please continue there. -- ___ Python tracker ___ ___

[issue29438] use after free in key sharing dict

INADA Naoki added the comment: Since Python 3.5's key sharing dict allows deletion, py35-2.patch is slightly different from py36-2.patch. Since dictresize won't happen in normal (no weakref/__del__ callback) deletion, I removed `CACHED_KEYS(tp) = NULL` entirely. -- Added file:

[issue29438] use after free in key sharing dict

Changes by Serhiy Storchaka : -- nosy: +Mark.Shannon, benjamin.peterson, rhettinger, tim.peters versions: +Python 3.5 ___ Python tracker

[issue29438] use after free in key sharing dict

Serhiy Storchaka added the comment: I think same patch should be applied to Python 3.5 too. -- ___ Python tracker ___

[issue29438] use after free in key sharing dict

INADA Naoki added the comment: > Why res == 0 is added? If PyDict_SetItem() triggers recursive calling of > _PyObjectDict_SetItem() which calls PyDict_SetItem() it may be possible that > the first PyDict_SetItem() is failed while the dict is changed by the second > PyDict_SetItem() and

[issue29438] use after free in key sharing dict

Serhiy Storchaka added the comment: Okay, if there is no way to test this with certainty, tests may be omitted. Why res == 0 is added? If PyDict_SetItem() triggers recursive calling of _PyObjectDict_SetItem() which calls PyDict_SetItem() it may be possible that the first PyDict_SetItem() is

[issue29438] use after free in key sharing dict

INADA Naoki added the comment: to: Serhiy I can reproduce the issue by 29438-minimum.py, on Python 3.7 with pydebug. But since this issue is "use after free", it may and may not crash. It's up to how freed memory block is used from another part of Python. Deterministic test which doesn't tied

[issue29438] use after free in key sharing dict

Serhiy Storchaka added the comment: Could you please write tests Inada? It would be nice to test also the case that fails with 29438-sharedkey-useafterfree-py36.patch. Actually I don't know if this is easy to reproduce, it was just my guessing. --

[issue29438] use after free in key sharing dict

Xiang Zhang added the comment: I left one review about the comment on Rietvied last patch. :-) -- ___ Python tracker ___

[issue29438] use after free in key sharing dict

Changes by INADA Naoki : Added file: http://bugs.python.org/file46572/29438-sharedkey-useafterfree-py36-2.patch ___ Python tracker ___

[issue29438] use after free in key sharing dict

Xiang Zhang added the comment: > if (was_shared && (cached = CACHED_KEYS(tp)) != NULL && cached != > ((PyDictObject *)dict)->ma_keys) +1 on this and I think the deletion should also use if ((cached = CACHED_KEYS(tp) ！= NULL) -- ___ Python tracker

[issue29438] use after free in key sharing dict

Serhiy Storchaka added the comment: PyDict_SetItem() can trigger destructor which first call _PyObjectDict_SetItem() which change CACHED_KEYS(tp) and then call PyDict_SetItem() which call dictresize(). At the end it may be possible that cached != ((PyDictObject *)dict)->ma_keys and cached !=

[issue29438] use after free in key sharing dict

Changes by Serhiy Storchaka : -- components: +Interpreter Core stage: -> patch review ___ Python tracker ___

[issue29438] use after free in key sharing dict

Changes by INADA Naoki : Added file: http://bugs.python.org/file46556/29438-sharedkey-useafterfree-py36.patch ___ Python tracker ___

[issue29438] use after free in key sharing dict

Changes by INADA Naoki : -- keywords: +patch Added file: http://bugs.python.org/file46521/29438-sharedkey-useafterfree-py35.patch ___ Python tracker

[issue29438] use after free in key sharing dict

INADA Naoki added the comment: It's similar to issue27945, but different. I confirmed this issue is in 3.4 too. https://github.com/python/cpython/blob/3.4/Objects/dictobject.c#L3798 // _PyObjectDict_SetItem() if ((tp->tp_flags & Py_TPFLAGS_HEAPTYPE) && (cached = CACHED_KEYS(tp))) {

[issue29438] use after free in key sharing dict

Serhiy Storchaka added the comment: Is it related to issue27945? -- nosy: +serhiy.storchaka ___ Python tracker ___

[issue29438] use after free in key sharing dict

INADA Naoki added the comment: I can reproduce it on Python 3.5 with attached script. I think this bug is from Python 3.3, since key-sharing dict is implemented. "Trigger key sharing dict resize while callbacks (weakref or __del__) called from setitem" is step to reproduce. It's not easy to