[issue31455] ElementTree.XMLParser() mishandles exceptions

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31455] ElementTree.XMLParser() mishandles exceptions

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

New changeset c498cd8bf81fc47acf2f1f702e8b3bc9bd4aed65 by Serhiy Storchaka 
(scoder) in branch '2.7':
bpo-31544: Fix a reference leak to 'self' after the previous target error 
handling fixes. (GH-6318)
https://github.com/python/cpython/commit/c498cd8bf81fc47acf2f1f702e8b3bc9bd4aed65

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31455] ElementTree.XMLParser() mishandles exceptions

[Stefan Behnel]

Stefan Behnel  added the comment:

PR 6318 fixes the reference leak for Py2.7.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31455] ElementTree.XMLParser() mishandles exceptions

[Stefan Behnel]

Change by Stefan Behnel :


--
pull_requests: +6034
stage: resolved -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31455] ElementTree.XMLParser() mishandles exceptions

[Stefan Behnel]

Stefan Behnel  added the comment:

Right, Zachary, thanks for noticing. Py2.7 is actually way more different than 
I thought, and I hadn't paid enough attention to that. Py3 does all of this in 
"__init__", whereas Py2 essentially implements "__new__" in C, which requires 
more cleanup.

BTW, the implementation in Py3 would also benefit from refactoring the error 
handling code and moving it all in one place. But it shouldn't suffer from the 
same kind of problem, at least.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31455] ElementTree.XMLParser() mishandles exceptions

[Zachary Ware]

Zachary Ware  added the comment:

This added a refleak on 2.7, see 
http://buildbot.python.org/all/#/builders/78/builds/122 and later builds.  
Also, the bug ID in the blurb is incorrect (31544 vs 31455).

--
nosy: +zach.ware
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31455] ElementTree.XMLParser() mishandles exceptions

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

New changeset 0694b6a651ba2a53f6323ffb3b23358f43885815 by Serhiy Storchaka 
(scoder) in branch '2.7':
bpo-31544: Avoid calling "PyObject_GetAttrString()" (and potentially executing 
user code) with a live exception set. (GH-3992)
https://github.com/python/cpython/commit/0694b6a651ba2a53f6323ffb3b23358f43885815

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31455] ElementTree.XMLParser() mishandles exceptions

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31455] ElementTree.XMLParser() mishandles exceptions

[Stefan Behnel]

Stefan Behnel  added the comment:

Backport PR for 2.7 added: 3992

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31455] ElementTree.XMLParser() mishandles exceptions

[Stefan Behnel]

Change by Stefan Behnel :


--
pull_requests: +3968

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31455] ElementTree.XMLParser() mishandles exceptions

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

It would be nice. But I see you already have opened issue31758 for reference 
leaks. I think that other problems can be solved in the same issue.

Do you mind to backport your patch to 2.7 Stefan? If this makes sense. 
Otherwise I'll just close this issue. Live exceptions don't cause crash in 2.7.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31455] ElementTree.XMLParser() mishandles exceptions

[Oren Milman]

Oren Milman  added the comment:

Serhiy, in addition to the problems you mentioned with not calling __init__(), 
it seems
that calling every method of an uninitialized XMLParser object would crash.

If you don't mind, i would be happy to open an issue to fix these crashes.

--
nosy: +Oren Milman

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31455] ElementTree.XMLParser() mishandles exceptions

[Serhiy Storchaka]

Serhiy Storchaka added the comment:


New changeset 49caab46f687eb201898fb6c2c40d47bdcb0e58b by Serhiy Storchaka 
(Miss Islington (bot)) in branch '3.6':
[3.6] bpo-31455: Fix an assertion failure in ElementTree.XMLParser(). (GH-3545) 
(#3585)
https://github.com/python/cpython/commit/49caab46f687eb201898fb6c2c40d47bdcb0e58b


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31455] ElementTree.XMLParser() mishandles exceptions

[Roundup Robot]

Changes by Roundup Robot :


--
pull_requests: +3576

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31455] ElementTree.XMLParser() mishandles exceptions

[Serhiy Storchaka]

Serhiy Storchaka added the comment:


New changeset c8d8e15bfc24abeeaaf3d8be9073276b0c011cdf by Serhiy Storchaka 
(scoder) in branch 'master':
bpo-31455: Fix an assertion failure in ElementTree.XMLParser(). (#3545)
https://github.com/python/cpython/commit/c8d8e15bfc24abeeaaf3d8be9073276b0c011cdf


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31455] ElementTree.XMLParser() mishandles exceptions

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

I agree. While these issues are not totally independent (they changes the same 
code), my issue needs uncommon use of __new__ and __init__, while your issue 
can be exposed in normal cases.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31455] ElementTree.XMLParser() mishandles exceptions

[Stefan Behnel]

Stefan Behnel added the comment:

Feel free to provide a separate pull request. These issues seem independent of 
the exception handling problem that I wrote a fix for.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31455] ElementTree.XMLParser() mishandles exceptions

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Oh, I see many other issues with C implementation of ElementTree.

1. If XMLParser.__init__ is called twice, it leaks references and the Expat 
parser.

Possible solution: use the Py_XSETREF() macro instead of simple assignment. The 
Expat parser needs special handling.

Other possible solution: drop __init__() and make all initialization in 
__new__(). But this is a solution only for 3.7 because can break compatibility.

2. If XMLParser.__init__ is not called or if it fails to initialize the Expat 
parser, self->entity and self->target are NULL. Later in xmlparser_getattro() 
they are increfed unconditionally.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31455] ElementTree.XMLParser() mishandles exceptions

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
nosy: +eli.bendersky, haypo, serhiy.storchaka
versions: +Python 2.7, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31455] ElementTree.XMLParser() mishandles exceptions

[Stefan Behnel]

Changes by Stefan Behnel :


--
keywords: +patch
pull_requests: +3542
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31455] ElementTree.XMLParser() mishandles exceptions

[Stefan Behnel]

New submission from Stefan Behnel:

The "XMLParser.__init__()" method in "_elementtree.c" contains this code:

self->handle_start = PyObject_GetAttrString(target, "start");
self->handle_data = PyObject_GetAttrString(target, "data");
self->handle_end = PyObject_GetAttrString(target, "end");
self->handle_comment = PyObject_GetAttrString(target, "comment");
self->handle_pi = PyObject_GetAttrString(target, "pi");
self->handle_close = PyObject_GetAttrString(target, "close");
self->handle_doctype = PyObject_GetAttrString(target, "doctype");
PyErr_Clear();

This ignores all exceptions, not only AttributeError.
It also passes live exceptions into the later lookup calls, which may execute 
arbitrary user code.

--
components: Extension Modules
messages: 302101
nosy: scoder
priority: normal
severity: normal
status: open
title: ElementTree.XMLParser() mishandles exceptions
type: behavior
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com