[issue25476] close() behavior on non-blocking BufferedIO objects with sockets

[Martin Panter]

Change by Martin Panter :


--
stage:  -> resolved
status: pending -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25476] close() behavior on non-blocking BufferedIO objects with sockets

[Martin Panter]

Martin Panter added the comment:

I propose to reject this. Close() should always close the underlying file 
descriptor or socket, even if there is a blocking error or other exception.

--
resolution:  -> rejected
status: open -> pending

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25476] close() behavior on non-blocking BufferedIO objects with sockets

[Martin Panter]

Martin Panter added the comment:

David, what’s your use case for doing non-blocking buffered writing 
“correctly”? Would you be able to use the context manager functionality? I 
would have thought you would explicitly call flush() as many times as 
necessary, but only call close() once when you are done.

At least in blocking mode, close() is meant do as much as possible, despite any 
intermediate exceptions. It doesn’t seem wise to break this rule in 
non-blocking mode.

--
nosy: +martin.panter

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25476] close() behavior on non-blocking BufferedIO objects with sockets

[eryksun]

eryksun added the comment:

I meant that you need a check in buffered_close such as the following:

if (res == NULL) {
if (PyErr_ExceptionMatches(PyExc_BlockingIOError))
goto end;
PyErr_Fetch(, , );
} else
Py_DECREF(res);

For 2.7 you could create a function similar to _PyIO_trap_eintr, but trap the 
errors that Python 3 maps to BlockingIOError: EAGAIN/EWOULDBLOCK, EALREADY, and 
EINPROGRESS.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25476] close() behavior on non-blocking BufferedIO objects with sockets

[eryksun]

eryksun added the comment:

Per issue 16597, when an exception occurs in flush(), the file is closed 
anyway. You'd have to check the exception and only skip to the end for 
EWOULDBLOCK or EAGAIN. That way you're not introducing a regression for ENOSPC 
and other exceptions for which retrying will just repeatedly fail. 

I'm adding 2.7 to keep the io module consistent; however, socket.makefile in 
2.7 doesn't use the io module.

--
nosy: +eryksun
versions: +Python 2.7, Python 3.4, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25476] close() behavior on non-blocking BufferedIO objects with sockets

[David Beazley]

David Beazley added the comment:

Please don't make flush() close the file on a BlockingIOError.   That would be 
an unfortunate mistake and make it impossible to implement non-blocking I/O 
correctly with buffered I/O.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25476] close() behavior on non-blocking BufferedIO objects with sockets

[David Beazley]

New submission from David Beazley:

First comment: In the I/O library, there is documented behavior for how things 
work in the presence of non-blocking I/O.  For example, read/write methods 
returning None on raw file objects.  Methods on BufferedIO instances raise a 
BlockingIOError for operations that can't complete. 

However, the implementation of close() is currently broken.  If buffered I/O is 
being used and a file is closed, it's possible that the close will fail due to 
a BlockingIOError occurring as buffered data is flushed to output.  However, in 
this case, the file is closed anyways and there is no possibility to retry.  
Here is an example to illustrate:

>>> from socket import *
>>> s = socket(AF_INET, SOCK_STREAM)
>>> s.connect(('somehost', port))
>>> s.setblocking(False)
>>> f = s.makefile('wb', buffering=1000)   # Large buffer
>>> f.write(b'x'*100)
>>>

Now, watch carefully

>>> f
<_io.BufferedWriter name=4>
>>> f.closed
False
>>> f.close()
Traceback (most recent call last):
  File "", line 1, in 
BlockingIOError: [Errno 35] write could not complete without blocking
>>> f
<_io.BufferedWriter name=-1>
>>> f.closed
True
>>>

I believe this can be fixed by changing a single line in 
Modules/_io/bufferedio.c:

--- bufferedio_orig.c   2015-10-25 16:40:22.0 -0500
+++ bufferedio.c2015-10-25 16:40:35.0 -0500
@@ -530,10 +530,10 @@
 res = PyObject_CallMethodObjArgs((PyObject *)self, _PyIO_str_flush, NULL);
 if (!ENTER_BUFFERED(self))
 return NULL;
-if (res == NULL)
-PyErr_Fetch(, , );
-else
-Py_DECREF(res);
+if (res == NULL) 
+  goto end;
+else 
+  Py_DECREF(res);
 
 res = PyObject_CallMethodObjArgs(self->raw, _PyIO_str_close, NULL);

With this patch, the close() method can be retried as appropriate until all 
buffered data is successfully written.

--
components: IO
messages: 253438
nosy: dabeaz
priority: normal
severity: normal
status: open
title: close() behavior on non-blocking BufferedIO objects with sockets
type: behavior
versions: Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com