[issue25707] Add the close method for ElementTree.iterparse() object

[Jacob Walls]

Change by Jacob Walls :


--
keywords: +patch
nosy: +jacobtylerwalls
nosy_count: 4.0 -> 5.0
pull_requests: +29816
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/31696

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25707] Add the close method for ElementTree.iterparse() object

[Furkan Onder]

Furkan Onder  added the comment:

Python 3.8.2 (default, Apr  8 2020, 14:31:25) 
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import xml.etree.ElementTree as ET
>>> import gc
>>> ET.iterparse('/dev/null')
.IterParseIterator object at 
0x7fb96f679d00>
>>> gc.collect()
34

The warning(__main__:1: ResourceWarning: unclosed file <_io.BufferedReader 
name='/dev/null'>) is no longer available in python3.8.2

--
nosy: +furkanonder

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25707] Add the close method for ElementTree.iterparse() object

[Stefan Behnel]

Stefan Behnel  added the comment:

Ok, I think it's reasonable to make the resource management explicit for the 
specific case of letting iterparse() open the file. That suggests that there 
should also be context manager support, given that safe usages would often 
involve a try-finally.

Since it might not always be obvious for users when they need to close the 
iterator or not, I would also suggest to not let it raise an error on a 
double-close, i.e. if .close() was already called or the iterator was already 
exhausted (and the file closed automatically), calling .close() should just do 
nothing.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25707] Add the close method for ElementTree.iterparse() object

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Implicit closing an exhausted iterator helps only the iterator is iterated to 
the end. If the iteration has been stopped before the end, we get a leak of the 
file descriptor. Closing the file descriptor in the finalizer can be deferred 
to undefined term, especially in implementations without reference counting. 
Since file descriptors are limited resource, this can cause troubles in real 
programs.

Reasons for close() in iterparse objects are the same as for close in files and 
generators.

Maybe we will need to implement the full generator protocol (send() and 
throw()) in the iterparse objects, but currently I do not know use cases for 
this.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25707] Add the close method for ElementTree.iterparse() object

[Stefan Behnel]

Stefan Behnel  added the comment:

I don't think there is a need for a close() method. Instead, the iterator 
should close the file first thing when it's done with it, but only if it owns 
it. Therefore, the fix in issue 25688 seems correct.

Closing can also be done explicitly in a finaliser of the iterator, if implicit 
closing via decref is too lax.

--
nosy: +scoder
versions: +Python 3.8 -Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25707] Add the close method for ElementTree.iterparse() object

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
dependencies: +Verify the etree_parse and etree_iterparse benchmarks are 
working appropriately

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25707] Add the close method for ElementTree.iterparse() object

[Emanuel Barry]

Emanuel Barry added the comment:

Oh, my bad. Ignore my last message, behaviour is identical then. Thanks for 
clearing that up.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25707] Add the close method for ElementTree.iterparse() object

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

You have to enable deprecation warnings. Run the interpreter with the -Wa 
option.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25707] Add the close method for ElementTree.iterparse() object

[Emanuel Barry]

Emanuel Barry added the comment:

I am unable to reproduce the issue on Windows 7 with 3.5.0; I have tried 
opening a small (non-empty) text. Here's the result:


>>> import xml.etree.ElementTree as ET
>>> import gc
>>> ET.iterparse("E:/New.txt")

>>> gc.collect()
59

--
nosy: +ebarry

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25707] Add the close method for ElementTree.iterparse() object

[Serhiy Storchaka]

New submission from Serhiy Storchaka:

If ElementTree.iterparse() is called with file names, it opens a file. When 
resulting iterator is not exhausted, the file lefts not closed.

>>> import xml.etree.ElementTree as ET
>>> import gc
>>> ET.iterparse('/dev/null')

>>> gc.collect()
__main__:1: ResourceWarning: unclosed file <_io.BufferedReader name='/dev/null'>
34

Martin Panter proposed in issue25688 to add an explicit way to clean it up, 
like a generator.close() method.

--
assignee: serhiy.storchaka
components: Library (Lib)
messages: 255159
nosy: serhiy.storchaka
priority: normal
severity: normal
stage: needs patch
status: open
title: Add the close method for ElementTree.iterparse() object
type: resource usage
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com