[issue32421] Keeping an exception in cache can segfault the interpreter

[Yonatan Zunger]

Yonatan Zunger  added the comment:

Apologies -- I've been completely swamped and haven't had time to build a repro 
case. I don't think I have enough state on the way I found this to reconstruct 
it easily right now, so I'll close it and re-open if I find it again.

--
resolution:  -> not a bug
stage: test needed -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32421] Keeping an exception in cache can segfault the interpreter

[Terry J. Reedy]

Terry J. Reedy  added the comment:

I repeated Julian's test with installed 3.7.0b2 on Windows 10 and there is no 
crash when running either from a command line or from an IDLE editor.  So 
either the issue has been fixed since 3.6.2, or it is specific to some 
undisclosed system or class of systems, or the specific claim in the title is a 
misinterpretation of behavior.  In any case, there is currently no possibility 
of action on our part.

Yonatan, if you have the promised minimal reproducible test case, for current 
python, please post it, with version and OS.  If not, we should close this 
('not a bug' fits best) until there is one.

--
stage:  -> test needed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32421] Keeping an exception in cache can segfault the interpreter

[Julien Palard]

Julien Palard  added the comment:

Serhiy: Sorry the ∈ comes from emacs' pretty-mode, I should copy from outside 
emacs. Version tested are listed in my previous message.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32421] Keeping an exception in cache can segfault the interpreter

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

> for _ ∈ range(5):

What version of Python are you using?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32421] Keeping an exception in cache can segfault the interpreter

[Terry J. Reedy]

Terry J. Reedy  added the comment:

Yonathon, when replying by email, please delete the quoted message, except 
possibly for a specific line,  as it is redundant noise when posted on the web 
page.

--
nosy: +terry.reedy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32421] Keeping an exception in cache can segfault the interpreter

[Yonatan Zunger]

Yonatan Zunger  added the comment:

3.6.2. But Nick Coghlan pointed out elsewhere that the underlying issue is
more likely to be one of the other objects being pinned in-memory because
of the traceback, so the exception may actually be just an incidental
thing. I'm going to have to work on a better minimal repro case -- this
particular example had both a subprocess.Popen open at the time and a
message-channel object that holds some os.pipes, so there are all sorts of
fun things that may be causing the trouble.

I'll update the bug when I have a chance to do that and narrow it down a
bit more.

On Tue, Dec 26, 2017 at 3:26 AM, Julien Palard 
wrote:

>
> Julien Palard  added the comment:
>
> Tested with:
>
> @memoize()
> def foo(x):
> raise Exception("From foo")
>
>
> for _ ∈ range(5):
> try:
> foo(42)
> except Exception as err:
> print(err)
>
> using a python 3.5.4, 3.6.4, and master(13a6c09), and was unable to
> reproduce it, which version of Python are you using? How are you
> reproducing it?
>
> --
> nosy: +mdk
>
> ___
> Python tracker 
> 
> ___
>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32421] Keeping an exception in cache can segfault the interpreter

[Julien Palard]

Julien Palard  added the comment:

Tested with:

@memoize()
def foo(x):
raise Exception("From foo")


for _ ∈ range(5):
try:
foo(42)
except Exception as err:
print(err)

using a python 3.5.4, 3.6.4, and master(13a6c09), and was unable to reproduce 
it, which version of Python are you using? How are you reproducing it?

--
nosy: +mdk

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32421] Keeping an exception in cache can segfault the interpreter

[Yonatan Zunger]

Yonatan Zunger  added the comment:

It's just a clean version of reraise. Part of the six (Py2/Py3
compatibility) library, but pretty useful on its own anyway.

On Sun, Dec 24, 2017 at 8:37 PM, Nitish  wrote:

>
> Nitish  added the comment:
>
> What is six.reraise in except block?
>
> --
> nosy: +nitishch
>
> ___
> Python tracker 
> 
> ___
>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32421] Keeping an exception in cache can segfault the interpreter

[Nitish]

Nitish  added the comment:

What is six.reraise in except block?

--
nosy: +nitishch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32421] Keeping an exception in cache can segfault the interpreter

[Josh Holland]

Change by Josh Holland :


--
nosy: +anowlcalledjosh

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32421] Keeping an exception in cache can segfault the interpreter

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32421] Keeping an exception in cache can segfault the interpreter

[Yonatan Zunger]

New submission from Yonatan Zunger :

Using the following decorator to memoize a function causes the interpreter to 
segfault on exit if any exceptions were stored in the cache. Changing the bad 
line to instead store copy.copy(e) makes the problem go away.

Almost certainly some kind of problem with the GC of the traceback not 
happening at the moment the interpreter expects it, but I haven't had time to 
investigate any more deeply yet.

def memoize(keyFunc=lambda x: x, cacheExceptions: bool=True):
def wrapper(func):
return _Memo(func, keyFunc, cacheExceptions)
return wrapper

class _Memo(object):
def __init__(self, func, keyFunc, cacheExceptions: bool) -> None:
self.func = func
self.keyFunc = keyFunc
self.cacheExceptions = cacheExceptions
self.lock = threading.Lock()
self.cache = {}

def __call__(self, *args, **kwargs) -> Any:
key = self.keyFunc(*args, **kwargs)
assert isinstance(key, collections.Hashable)
with self.lock:
if key in self.cache:
value = self.cache[key]
if isinstance(value, BaseException):
raise value
return value

try:
result = self.func(*args, **kwargs)
except BaseException as e:
if self.cacheExceptions:
with self.lock:
self.cache[key] = e   # BAD LINE
six.reraise(*sys.exc_info())
else:
with self.lock:
self.cache[key] = result
return result

--
components: Interpreter Core
messages: 308978
nosy: zunger
priority: normal
severity: normal
status: open
title: Keeping an exception in cache can segfault the interpreter
type: crash
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com