[issue31285] a SystemError and an assertion failure in warnings.warn_explicit()

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
versions: +Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31285] a SystemError and an assertion failure in warnings.warn_explicit()

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:


New changeset 40d736bcf40c10aa5fc7d6cc305a6641afd3cd0e by Serhiy Storchaka 
(Oren Milman) in branch '2.7':
[2.7] bpo-31285: Don't raise a SystemError in warnings.warn_explicit() in case 
__loader__.get_source() has a bad splitlines() method. (GH-3219) (#3823)
https://github.com/python/cpython/commit/40d736bcf40c10aa5fc7d6cc305a6641afd3cd0e


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31285] a SystemError and an assertion failure in warnings.warn_explicit()

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:


New changeset 66c2b9f13ef2197a5212fd58372173124df76467 by Serhiy Storchaka 
(Miss Islington (bot)) in branch '3.6':
[3.6] bpo-31285: Remove splitlines identifier from Python/_warnings.c (GH-3803) 
(#3829)
https://github.com/python/cpython/commit/66c2b9f13ef2197a5212fd58372173124df76467


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31285] a SystemError and an assertion failure in warnings.warn_explicit()

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

I just missed PR 3803.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31285] a SystemError and an assertion failure in warnings.warn_explicit()

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:


New changeset 8b4ff53c440dfcde40fbeb02c5e666c85190528f by Serhiy Storchaka 
(Oren Milman) in branch 'master':
bpo-31285: Remove splitlines identifier from Python/_warnings.c (#3803)
https://github.com/python/cpython/commit/8b4ff53c440dfcde40fbeb02c5e666c85190528f


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31285] a SystemError and an assertion failure in warnings.warn_explicit()

[Roundup Robot]

Change by Roundup Robot :


--
pull_requests: +3812

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31285] a SystemError and an assertion failure in warnings.warn_explicit()

[Oren Milman]

Change by Oren Milman :


--
pull_requests: +3808

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31285] a SystemError and an assertion failure in warnings.warn_explicit()

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

I didn't check. I supposed that the other code can support unicode by 
implicitly converting it to str. But now I see that show_warning() uses 
PyString_AS_STRING() and therefore supports only str. I agree, using 
str.splitlines() would be correct solution.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31285] a SystemError and an assertion failure in warnings.warn_explicit()

[Oren Milman]

Oren Milman  added the comment:

But in case get_source() returned a unicode, is it likely that the splitlines() 
method
of this unicode would return a 8-bit string? Currently show_warning() doesn't 
handle this
scenario, as it assumes splitlines() returned an 8-bit string. Or do you think 
that
show_warning() should also accept unicode?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31285] a SystemError and an assertion failure in warnings.warn_explicit()

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

What if get_source() returned a unicode string? Usually it returns 8-bit 
string, but in many cases unicode is accepted if str is expected, so you need 
to check this option too.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31285] a SystemError and an assertion failure in warnings.warn_explicit()

[Oren Milman]

Oren Milman  added the comment:

oh, of course, checking that get_source() returned a string before passing it to
str.splitlines() is not needed.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31285] a SystemError and an assertion failure in warnings.warn_explicit()

[Oren Milman]

Oren Milman  added the comment:

Another thought - the existing code assumes that splitlines() returned a string.
So maybe we could just check that get_source() returned a string, and then call
the method str.splitlines() on it?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31285] a SystemError and an assertion failure in warnings.warn_explicit()

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

> In 2.7, PyUnicode_Splitlines() first does:
> string = PyUnicode_FromObject(string);

And it raises an exception if the string contains non-ASCII characters.

It is better to avoid str<->unicode convertion as long as possible. And when 
do it for the output a warning, use "replace" or "backslashreplace" error 
handlers or "latin1" decoder to avoid a failure.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31285] a SystemError and an assertion failure in warnings.warn_explicit()

[Oren Milman]

Oren Milman  added the comment:

In 2.7, PyUnicode_Splitlines() first does:
string = PyUnicode_FromObject(string);

So i thought that PyUnicode_Splitlines() would be fine with receiving a string.

But now i realize that even in case i was right there, PyUnicode_Splitlines()
returns a unicode, and not a string, so there should be problems later.
I wonder how the tests still passed..

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31285] a SystemError and an assertion failure in warnings.warn_explicit()

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

PyUnicode_Splitlines() cannot be used here in 2.7, because the source is likely 
a 8-bit string.

Actually I'm not sure this issue is worth to be fixed in 2.7. The fix would be 
different from 3.x and more complex.

--
versions: +Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31285] a SystemError and an assertion failure in warnings.warn_explicit()

[Oren Milman]

Change by Oren Milman :


--
pull_requests: +3791

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31285] a SystemError and an assertion failure in warnings.warn_explicit()

[Oren Milman]

Change by Oren Milman :


--
pull_requests: +3789

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31285] a SystemError and an assertion failure in warnings.warn_explicit()

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:


New changeset 90fe25a051bd8cf64d52be533c9b60cad9bdd7fb by Serhiy Storchaka in 
branch '3.6':
[3.6] bpo-31285: Fix an assertion failure and a SystemError in 
warnings.warn_explicit. (GH-3219) (#3775)
https://github.com/python/cpython/commit/90fe25a051bd8cf64d52be533c9b60cad9bdd7fb


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31285] a SystemError and an assertion failure in warnings.warn_explicit()

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
keywords: +patch
pull_requests: +3762
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31285] a SystemError and an assertion failure in warnings.warn_explicit()

[Serhiy Storchaka]

Serhiy Storchaka added the comment:


New changeset 91fb0afe181986b48abfc6092dcca912b39de51d by Serhiy Storchaka 
(Oren Milman) in branch 'master':
bpo-31285: Fix an assertion failure and a SystemError in 
warnings.warn_explicit. (#3219)
https://github.com/python/cpython/commit/91fb0afe181986b48abfc6092dcca912b39de51d


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31285] a SystemError and an assertion failure in warnings.warn_explicit()

[Oren Milman]

Oren Milman added the comment:

ISTM that your solution is better than mine, Serhiy, so I updated the PR.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31285] a SystemError and an assertion failure in warnings.warn_explicit()

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

An alternative solution is using str.splitlines(source) instead of 
source.splitlines(). It raises a TypeError if the argument is not a text string 
and always returns a list of strings.

According to the documentation get_source() must return a text string or None.

https://docs.python.org/3/library/importlib.html?highlight=get_source#importlib.abc.InspectLoader.get_source

--
nosy: +brett.cannon, eric.snow, ncoghlan, serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31285] a SystemError and an assertion failure in warnings.warn_explicit()

[Oren Milman]

Changes by Oren Milman :


--
pull_requests: +3259

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31285] a SystemError and an assertion failure in warnings.warn_explicit()

[Oren Milman]

Oren Milman added the comment:

on a second thought, BadSource could be a subclass of str, so maybe we
should just check whether
module_globals['__loader__'].get_source(module_globals['__name__']).splitlines()[lineno-1]
is a str,
and whether
module_globals['__loader__'].get_source(module_globals['__name__']).splitlines()
is a list.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31285] a SystemError and an assertion failure in warnings.warn_explicit()

[Oren Milman]

New submission from Oren Milman:

1.
the following causes an assertion failure in Python/_warnings.c in
show_warning():

import warnings

class BadLoader:
def get_source(self, fullname):
class BadSource:
def splitlines(self):
return [42]
return BadSource()

del warnings._showwarnmsg
warnings.warn_explicit(message='foo', category=ArithmeticError, filename='bar',
   lineno=1, module_globals={'__loader__': BadLoader(),
 '__name__': 'foobar'})

in short, the assertion failure would happen in warnings.warn_explicit() in case
module_globals['__loader__'].get_source(module_globals['__name__']).splitlines()[lineno-1]
is not a str.


2.
the following raises a SystemError:

import warnings

class BadLoader:
def get_source(self, fullname):
class BadSource:
def splitlines(self):
return 42
return BadSource()

warnings.warn_explicit(message='foo', category=UserWarning, filename='bar',
   lineno=42, module_globals={'__loader__': BadLoader(),
  '__name__': 'foobar'})

in short, warnings.warn_explicit() raises the SystemError in case
module_globals['__loader__'].get_source(module_globals['__name__']).splitlines()
is not a list.



ISTM that adding a check in warnings_warn_explicit() (in Python/_warnings.c),
to check whether
module_globals['__loader__'].get_source(module_globals['__name__'])
is a str (after existing code found out that it isn't None) would prevent both
the assertion failure and the SystemError.

What do you think? Is it OK to permit get_source() to return only None or a str?

--
components: Interpreter Core
messages: 300892
nosy: Oren Milman
priority: normal
severity: normal
status: open
title: a SystemError and an assertion failure in warnings.warn_explicit()
type: crash
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com