[issue36709] Asyncio SSL keep-alive connections raise errors after loop close.

[Andrew Svetlov]

Andrew Svetlov  added the comment:

Thanks!

--
resolution:  -> fixed
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36709] Asyncio SSL keep-alive connections raise errors after loop close.

[Kumar Aditya]

Kumar Aditya  added the comment:

The original issue is fixed on main branch with bpo-44011 GH-31275,
It now only raises warnings but no exceptions:

---
(env) @kumaraditya303 ➜ /workspaces/cpython (latin1 ✗) $ python main.py
/workspaces/cpython/main.py:12: DeprecationWarning: There is no current event 
loop
  loop = asyncio.get_event_loop()
/workspaces/cpython/env/lib/python3.11/site-packages/certifi/core.py:36: 
DeprecationWarning: path is deprecated. Use files() instead. Refer to 
https://importlib-resources.readthedocs.io/en/latest/using.html#migrating-from-legacy
 for migration advice.
  _CACERT_CTX = get_path("certifi", "cacert.pem")
sys:1: ResourceWarning: unclosed 
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/workspaces/cpython/Lib/asyncio/sslproto.py:116: ResourceWarning: unclosed 
transport 
  _warnings.warn(
/workspaces/cpython/Lib/asyncio/selector_events.py:710: ResourceWarning: 
unclosed transport <_SelectorSocketTransport fd=7>
  _warn(f"unclosed transport {self!r}", ResourceWarning, source=self)
ResourceWarning: Enable tracemalloc to get the object allocation traceback

---

@asvetlvo This can be closed now.

--
nosy: +kumaraditya303
type:  -> behavior
versions: +Python 3.11 -Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42206] Pegen: Raise errors occurred by the AST validator

[Batuhan Taskaya]

Change by Batuhan Taskaya :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42206] Pegen: Raise errors occurred by the AST validator

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:


New changeset 3af4b585527743e455145d294049c46b7c823ed9 by Batuhan Taskaya in 
branch 'master':
bpo-42206: Propagate and raise errors from PyAST_Validate in the parser 
(GH-23035)
https://github.com/python/cpython/commit/3af4b585527743e455145d294049c46b7c823ed9


--

___
Python tracker 
<https://bugs.python.org/issue42206>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42206] Pegen: Raise errors occurred by the AST validator

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

This will slow down the parser massively. The reason we went' with the assert 
is so we only do the check in debug mode. We could propagate if you want, but 
it needs to be surrounded by debug preprocessor checks.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42206] Pegen: Raise errors occurred by the AST validator

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

> This will slow down the parser massively. The reason we went' with the assert 
> is so we only do the check in debug mode. We could propagate if you want, but 
> it needs to be surrounded by debug preprocessor checks.

Oh, I just checked and it currently is surrounded. Nevermind

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42206] Pegen: Raise errors occurred by the AST validator

[Batuhan Taskaya]

Change by Batuhan Taskaya :


--
keywords: +patch
pull_requests: +21954
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/23035

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42206] Pegen: Raise errors occurred by the AST validator

[Batuhan Taskaya]

New submission from Batuhan Taskaya :

When working on the AST validator, it is unclear that which of the changes 
caused parser to fail on valid input. I propose to propagate the error and 
raise it.

Current:
$ ./python t.py
python: Parser/pegen.c:1146: _PyPegen_run_parser: Assertion 
`PyAST_Validate(res)' failed.

Proposed:
$ ./python t.py
ValueError: Test AST error raised by the Python/ast.c

--
components: Interpreter Core
messages: 379941
nosy: BTaskaya, gvanrossum, lys.nikolaou, pablogsal
priority: normal
severity: normal
status: open
title: Pegen: Raise errors occurred by the AST validator

___
Python tracker 
<https://bugs.python.org/issue42206>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36709] Asyncio SSL keep-alive connections raise errors after loop close.

[Christian Heimes]

Christian Heimes  added the comment:

This seems to be an asyncio problem.

--
assignee: christian.heimes -> 

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36709] Asyncio SSL keep-alive connections raise errors after loop close.

[Andrew Svetlov]

Andrew Svetlov  added the comment:

Sorry, I'm not comfortable with such change just before the feature freeze.
The idea is maybe good but let's discuss and implement it later.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36709] Asyncio SSL keep-alive connections raise errors after loop close.

[Yury Selivanov]

Yury Selivanov  added the comment:

> Not sure if we have to drop this warning, it enforces writing good code that 
> controls all created resources lifecycle.

Right, maybe we add "transport.terminate()" as well?  Synchronously & 
immediately closing a transport is a valid use case. 

TBH I don't see why we absolutely must wait the "connection_lost" callback 
call.  I mean it would be reasonable to allow users to terminate the connection 
(even if it means that in some cases, like SSL, it won't be correctly closed) 
and not care about what happens to it next.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36709] Asyncio SSL keep-alive connections raise errors after loop close.

[Andrew Svetlov]

Andrew Svetlov  added the comment:

It's not about streams only.
The stream protocol can have such flag, sure.

But transport emits a warning like "unclosed transport ..."

Not sure if we have to drop this warning, it enforces writing good code that 
controls all created resources lifecycle.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36709] Asyncio SSL keep-alive connections raise errors after loop close.

[Yury Selivanov]

Yury Selivanov  added the comment:

> Sorry, that's how asyncio is designed.

Andrew, couldn't we provide a "stream.terminate()" method (not a coroutine) 
that would do the following:

* close the transport
* set a flag in the protocol that the stream has been terminated.  When the 
flag is set, the protocol simply ignores all errors (i.e. they are never shown 
to the user or logged)

This way Tom could have a weakref to the stream object from his high-level 
wrapper, and whenever the wrapper object is dereferenced it could terminate its 
stream.

> Sorry, that's how asyncio is designed.

I think it's a real problem, let's try to find out if we can provide a solution.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36709] Asyncio SSL keep-alive connections raise errors after loop close.

[Andrew Svetlov]

Andrew Svetlov  added the comment:

The difference is that socket.close() is an instant call.
After socket.close() the socket is done.
But transport.close() doesn't close the transport instantly.
asyncio requires at least one loop iteration for calling 
protocol.connection_lost() and actual socket closing.

In case of SSL it may take much longer.

Sorry, that's how asyncio is designed.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36709] Asyncio SSL keep-alive connections raise errors after loop close.

[Tom Christie]

Tom Christie  added the comment:

Right, and `requests` *does* provide both those styles.

The point more being that *not* having closed the transport at the point of 
exit shouldn't end up raising a hard error. It doesn't raise errors in 
sync-land, and it shouldn't do so in async-land.

Similarly, we wouldn't expect an open file resource to cause errors to be 
raised at the point of exit.

--

___
Python tracker 
<https://bugs.python.org/issue36709>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36709] Asyncio SSL keep-alive connections raise errors after loop close.

[Andrew Svetlov]

Andrew Svetlov  added the comment:

I would say that if requests a designed from scratch more idiomatic way could be

with requests.Session() as session:
session.get('https://example.com/')

or

session = requests.Session()
session.get('https://example.com/')
session.close()

Like the recommended way to handle files.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36709] Asyncio SSL keep-alive connections raise errors after loop close.

[Tom Christie]

Tom Christie  added the comment:

> From my understanding, the correct code should close all transports and wait 
> for their connection_lost() callbacks before closing the loop.

Ideally, yes, although we should be able to expect that an SSL connection that 
hasn't been gracefully closed wouldn't loudly error on teardown like that.

In standard sync code, the equivelent would running something like this...

```python
session = requests.Session()
session.get('https://example.com/')
```

We wouldn't expect a traceback to be raised on exiting. (Even though the user 
*hasn't* explicitly closed the session, and even though a keep alive SSL 
connection will be open at the point of exit.)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36709] Asyncio SSL keep-alive connections raise errors after loop close.

[Andrew Svetlov]

Andrew Svetlov  added the comment:

>From my understanding, the correct code should close all transports and wait 
>for their connection_lost() callbacks before closing the loop.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36709] Asyncio SSL keep-alive connections raise errors after loop close.

[Tom Christie]

Tom Christie  added the comment:

This appears somewhat related: https://bugs.python.org/issue34506

As it *also* logs exceptions occuring during `_fatal_error` and `_force_close`.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36709] Asyncio SSL keep-alive connections raise errors after loop close.

[Tom Christie]

New submission from Tom Christie :

If an asyncio SSL connection is left open (eg. any kind of keep-alive 
connection) then after closing the event loop, an exception will be raised...

Python:

```
import asyncio
import ssl
import certifi


async def f():
ssl_context = ssl.create_default_context()
ssl_context.load_verify_locations(cafile=certifi.where())
await asyncio.open_connection('example.org', 443, ssl=ssl_context)


loop = asyncio.get_event_loop()
loop.run_until_complete(f())
loop.close()
```

Traceback:

```
$ python example.py 
Fatal write error on socket transport
protocol: 
transport: <_SelectorSocketTransport fd=8>
Traceback (most recent call last):
  File 
"/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/selector_events.py",
 line 868, in write
n = self._sock.send(data)
OSError: [Errno 9] Bad file descriptor
Fatal error on SSL transport
protocol: 
transport: <_SelectorSocketTransport closing fd=8>
Traceback (most recent call last):
  File 
"/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/selector_events.py",
 line 868, in write
n = self._sock.send(data)
OSError: [Errno 9] Bad file descriptor

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File 
"/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/sslproto.py",
 line 676, in _process_write_backlog
self._transport.write(chunk)
  File 
"/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/selector_events.py",
 line 872, in write
self._fatal_error(exc, 'Fatal write error on socket transport')
  File 
"/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/selector_events.py",
 line 681, in _fatal_error
self._force_close(exc)
  File 
"/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/selector_events.py",
 line 693, in _force_close
self._loop.call_soon(self._call_connection_lost, exc)
  File 
"/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/base_events.py",
 line 677, in call_soon
self._check_closed()
  File 
"/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/base_events.py",
 line 469, in _check_closed
raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed
```

It looks to me like the original "OSError: [Errno 9] Bad file descriptor" 
probably shouldn't be raised in any case - if when attempting to tear down the 
SSL connection, then we should probably pass silently in the case that the 
socket has already been closed uncleanly.

Bought to my attention via: https://github.com/encode/httpcore/issues/16

--
assignee: christian.heimes
components: SSL, asyncio
messages: 340764
nosy: asvetlov, christian.heimes, tomchristie, yselivanov
priority: normal
severity: normal
status: open
title: Asyncio SSL keep-alive connections raise errors after loop close.
versions: Python 3.7

___
Python tracker 
<https://bugs.python.org/issue36709>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26685] Raise errors from socket.close()

[STINNER Victor]

STINNER Victor added the comment:


New changeset 580cd5cd3603317d294a881628880a92ea221334 by Victor Stinner in 
branch '3.6':
bpo-30319: socket.close() now ignores ECONNRESET (#2565) (#2566)
https://github.com/python/cpython/commit/580cd5cd3603317d294a881628880a92ea221334


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26685] Raise errors from socket.close()

[STINNER Victor]

Changes by STINNER Victor :


--
pull_requests: +2636

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26685] Raise errors from socket.close()

[STINNER Victor]

STINNER Victor added the comment:


New changeset 67e1478dba6efe60b8e1890192014b8b06dd6bd9 by Victor Stinner in 
branch 'master':
bpo-30319: socket.close() now ignores ECONNRESET (#2565)
https://github.com/python/cpython/commit/67e1478dba6efe60b8e1890192014b8b06dd6bd9


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26685] Raise errors from socket.close()

[Yury Selivanov]

Yury Selivanov added the comment:

>> Would you mind to open an issue specific for asyncio?

> I'll open an issue on GH today to discuss with you/Guido/asyncio devs.

Guido, Victor, please take a look: https://github.com/python/asyncio/pull/449

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26685] Raise errors from socket.close()

[Yury Selivanov]

Yury Selivanov added the comment:

> Ah, you became reasonable :-D

Come on... :)

> Would you mind to open an issue specific for asyncio?

I'll open an issue on GH today to discuss with you/Guido/asyncio devs.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26685] Raise errors from socket.close()

[STINNER Victor]

STINNER Victor added the comment:

"After thinking about this problem for a while, I arrived to the conclusion 
that we need to fix asyncio."

Ah, you became reasonable :-D

"Essentially, when a user passes a socket object to the event loop API like 
'create_server', they give up the control of the socket to the loop.  The loop 
should detach the socket object and have a full control over it."

I don't know how asyncio should be modified exactly, but I agree that someone 
should change in asyncio. The question is more about how to reduce headache 
because of the backward compatibility and don't kill performances.

Would you mind to open an issue specific for asyncio?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26685] Raise errors from socket.close()

[Yury Selivanov]

Yury Selivanov added the comment:

After thinking about this problem for a while, I arrived to the conclusion that 
we need to fix asyncio.  Essentially, when a user passes a socket object to the 
event loop API like 'create_server', they give up the control of the socket to 
the loop.  The loop should detach the socket object and have a full control 
over it.

--
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26685] Raise errors from socket.close()

[Yury Selivanov]

Yury Selivanov added the comment:

Maybe we should fix asyncio.  Maybe if a user passes an existing socket object 
into loop.create_connection, it should duplicate the socket.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26685] Raise errors from socket.close()

[STINNER Victor]

STINNER Victor added the comment:

> My proposal: ignore EBADF in socket.close().  It means that the socket is 
> closed already.  It doesn't matter why.

As Martin explained, getting EBAD means that your code smells. Your
code may close completely unrelated file descriptor... Say hello to
race conditions :-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26685] Raise errors from socket.close()

[Yury Selivanov]

Yury Selivanov added the comment:

My proposal: ignore EBADF in socket.close().  It means that the socket is 
closed already.  It doesn't matter why.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26685] Raise errors from socket.close()

[Yury Selivanov]

Yury Selivanov added the comment:

Another example: some asyncio (run with uvloop) code:
 
conn, _ = lsock.accept()
f = loop.create_task(
loop.connect_accepted_socket(
proto_factory, conn))
# More code
loop.run_forever()
conn.close()

Suppose the above snippet of code is some real-world program.

Now, in Python 3.5 everything works as expected.  In Python 3.6, "conn.close()" 
will raise an exception.

Why: uvloop passes the FD of "conn" to libuv, which does its thing and closes 
the connection when it should be closed.

Now:

> 1. Your code should call sock0.detach() rather than fileno(), so that sock0 
> no longer “owns” the file descriptor, or

I can't modify "connect_accepted_socket" to call "detach" on "conn", as it 
would make "conn" unusable right after the call.  This option can't be 
implemented, as it would break the backwards compatibility.

> 2. libuv should not close file descriptors that it doesn’t own.

This is not just about uvloop/libuv.  It's about any Python program out there 
that will break in 3.6.  A lot of code extracts the FD out of socket objects 
and does something with it.  We can't just make socket.close() to raise in 3.6 
-- that's not how backwards compatibility promise works.

Guido, what do you think about this issue?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26685] Raise errors from socket.close()

[Martin Panter]

Martin Panter added the comment:

If libuv closes the FD (step 3), won’t you get the same sort of problem if the 
uvloop user tries to do something else with the Python socket object, e.g. call 
getpeername()?

I see the fileno=... parameter for sockets as a parallel to the os.fdopen() 
function, which does raise exceptions from FileIO.close().

Maybe one option is to only trigger a DeprecationWarning, and raise a proper 
OSError in a future version.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26685] Raise errors from socket.close()

[Martin Panter]

Martin Panter added the comment:

I think your code example is not very robust, because the “sock” object refers 
to a freed file descriptor, and could easily close an unrelated file:

$ python3.5 -q
>>> import socket
>>> sock0 = socket.socket()
>>> sock = socket.socket(fileno=sock0.fileno())
>>> sock0.close()
>>> f = open("/dev/null")  # Unrelated code/thread opens a file
>>> sock.close()  # Closes unrelated file descriptor!
>>> f.close()  # Error even in 3.5
Traceback (most recent call last):
  File "", line 1, in 
OSError: [Errno 9] Bad file descriptor

I am not familiar with your use case or library, but I would suggest that 
either:

1. Your code should call sock0.detach() rather than fileno(), so that sock0 no 
longer “owns” the file descriptor, or

2. libuv should not close file descriptors that it doesn’t own.

Calling socket.close() should be safe if the Python socket object is registered 
as closed, but IMO calling close(), or any other method, when the OS socket 
(file descriptor) has been released behind Python’s back is a programming error.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26685] Raise errors from socket.close()

[Yury Selivanov]

Yury Selivanov added the comment:

IOW, what is happening in uvloop:

1. A user has a Python socket object and passes it asyncio (run with uvloop) 
API.

2. uvloop extracts the FD of the socket, and passes it to low-level libuv.

3. At some point of time, libuv closes the FD.

4. When user tries to close the socket, it silently fails in 3.5, and raises an 
exception in 3.6.

One way to solve this would be to use os.dup() in uvloop (i.e. pass a 
duplicated FD to libuv).  But that would mean that programs that use uvloop 
will consume file descriptors (limited resource) faster than programs that use 
vanilla asyncio.

Currently, there's a documented way of creating a socket out of an existing FD 
(fourth argument to the socket's constructor).  This means that the FD might be 
controlled by someone.  At least in this case, sock.close() must not raise if 
it fails.  It's OK if the FD is already close, because the Python socket was 
only wrapping it in the first place.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26685] Raise errors from socket.close()

[Yury Selivanov]

Yury Selivanov added the comment:

I think this patch should be reverted.  It breaks backwards compatibility:

import socket

sock0 = socket.socket(socket.AF_INET, socket.SOCK_STREAM, 0)
sock = socket.socket(
socket.AF_INET, socket.SOCK_STREAM, 0, sock0.fileno())
sock0.close()

sock.close()


This code behaves differently on 3.5 vs 3.6.

In uvloop (https://github.com/magicstack/uvloop) I create Python sockets for 
libuv socket handles. Sometimes, those handles are closed by libuv, and I have 
no control over that. So right now, a lot of uvloop tests fail because 
socket.close() now can raise an error.

It also doesn't make any sense to raise it anyways.  socket.close() *must* be 
safe to call even when the socket is already closed.

--
nosy: +yselivanov
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26685] Raise errors from socket.close()

[Martin Panter]

Martin Panter added the comment:

I tweaked the test again for Windows, anticipating that it will raise ENOTSOCK 
rather than EBADF.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26685] Raise errors from socket.close()

[Roundup Robot]

Roundup Robot added the comment:

New changeset bd665613ed67 by Martin Panter in branch 'default':
Issue #26685: Raise OSError if closing a socket fails
https://hg.python.org/cpython/rev/bd665613ed67

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26685] Raise errors from socket.close()

[Martin Panter]

Martin Panter added the comment:

Here is socket.close.v2.patch which hopefully fixes the test for Windows (still 
untested by me though)

--
Added file: http://bugs.python.org/file42361/socket.close.v2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26685] Raise errors from socket.close()

[Martin Panter]

Martin Panter added the comment:

Victor suggested making these errors be reported by the finalize method 
(garbage collection). I started investigating this, but faced various test 
suite failures (test_io, test_curses), as well as many new unhandled exceptions 
printed out during the tests which do not actually trigger failures. Maybe this 
could become a worthwhile change, but it needs more investigation and wasn’t my 
original intention. Finalize-exception.patch is my patch so far.

There are many comments over the place that make me uneasy about this change, 
so I think it should be investigated separately. E.g. Modules/_io/iobase.c:

/* Silencing I/O errors is bad, but printing spurious tracebacks is
   equally as bad, and potentially more frequent (because of
   shutdown issues). */

Lib/_pyio.py:

# The try/except block is in case this is called at program
# exit time, when it's possible that globals have already been
# deleted, and then the close() call might fail.  Since
# there's nothing we can do about such failures and they annoy
# the end users, we suppress the traceback.

--
Added file: http://bugs.python.org/file42360/finalize-exception.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26685] Raise errors from socket.close()

[STINNER Victor]

STINNER Victor added the comment:

I like the idea :-)

--
nosy: +haypo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26685] Raise errors from socket.close()

[Martin Panter]

Changes by Martin Panter :


--
dependencies: +test_fileno of test_urllibnet intermittently fails

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26685] Raise errors from socket.close()

[Martin Panter]

New submission from Martin Panter:

While looking at a recent failure of test_fileno() in test_urllibnet, I 
discovered that socket.close() ignores the return value of the close() system 
call. It looks like it has always been this way: 
<https://docs.python.org/3/library/socket.html#socket.socket.fileno>.

On the other hand, both FileIO.close() and os.close() raise an exception if the 
underlying close() call fails. So I propose to make socket.close() also raise 
an exception for any failure indicated by the underlying close() call. The 
benefit is that a programming error causing EBADF would be more easily noticed.

--
components: Extension Modules
files: socket.close.patch
keywords: patch
messages: 262735
nosy: martin.panter
priority: normal
severity: normal
stage: patch review
status: open
title: Raise errors from socket.close()
type: enhancement
versions: Python 3.6
Added file: http://bugs.python.org/file42342/socket.close.patch

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue26685>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: raise errors

[Gabriel Genellina]

En Mon, 21 Sep 2009 06:17:20 -0300, daved170 daved...@gmail.com escribió:


I need help with exceptions raising.
My goal is to print at the outer functions all the errors including
the most inner one.


The default exception report contains a lot of information, you don't have  
to do anything special.



def foo1(self):
   try:
foo2()
   except ? :
 print outer Err at foo1 + ??

def foo2(self):
   try:
error occured
   except ? :
 raise inner Err at foo2


The bare bones structure is:

def foo1():
  foo2()

def foo2():
  undefinedname()

foo1()

and you get this error:

D:\temppython test638580.py
Traceback (most recent call last):
  File test638580.py, line 7, in module
foo1()
  File test638580.py, line 2, in foo1
foo2()
  File test638580.py, line 5, in foo2
undefinedname()
NameError: global name 'undefinedname' is not defined



I would like the print to be : outer Err at foo1 , inner Err at foo1


It already includes that info.

  File test638580.py, line 2, in foo1
foo2()

means that in line 2 of test638580.py, inside function foo1, there was a  
call to foo2.


  File test638580.py, line 5, in foo2
undefinedname()

and that means that foo2 tried to call undefinedname.

  NameError: global name 'undefinedname' is not defined

And that's the actual error.

Do you want it to be more nicely formatted? Do you want to include  
additional information? Or what?


--
Gabriel Genellina

--
http://mail.python.org/mailman/listinfo/python-list

raise errors

[daved170]

Hi everybody,
I need help with exceptions raising.
My goal is to print at the outer functions all the errors including
the most inner one.

For example:

def foo1(self):
   try:
foo2()
   except ? :
 print outer Err at foo1 + ??

def foo2(self):
   try:
error occured
   except ? :
 raise inner Err at foo2


the ? remarks that I have no idea what to use.

I would like the print to be : outer Err at foo1 , inner Err at foo1

thanks
daved
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: raise errors

[Ulrich Eckhardt]

daved170 wrote:
 I need help with exceptions raising.
 My goal is to print at the outer functions all the errors including
 the most inner one.
 
 For example:
 
 def foo1(self):
try:
 foo2()
except ? :
  print outer Err at foo1 + ??
 
 def foo2(self):
try:
 error occured
except ? :
  raise inner Err at foo2
 
 
 the ? remarks that I have no idea what to use.
 
 I would like the print to be : outer Err at foo1 , inner Err at foo1

Take a look at the builtin debugger facility, i.e. 'import pdb'. There, you
have a traceback module which might already give you what you want.

One more advise though: Do not catch exceptions you don't handle! Make sure
they can pass through your code without hurting anything and then catch and
log them in a place where you actually can handle them. In particular,
don't wrap each and every tiny morsel of code into try/except handlers as
you would (have to) do if you were using return values.

Uli

-- 
Sator Laser GmbH
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: raise errors

[Peter Otten]

daved170 wrote:

 I need help with exceptions raising.
 My goal is to print at the outer functions all the errors including
 the most inner one.
 
 For example:
 
 def foo1(self):
try:
 foo2()
except ? :
  print outer Err at foo1 + ??
 
 def foo2(self):
try:
 error occured
except ? :
  raise inner Err at foo2
 
 
 the ? remarks that I have no idea what to use.
 
 I would like the print to be : outer Err at foo1 , inner Err at foo1

Have a look at the traceback module. Example:

 def foo():
... try:
... oops
... except:
... traceback.print_exc()
... raise
...
 def bar():
... try:
... foo()
... except:
... traceback.print_exc()
...
 import traceback
 bar()
Traceback (most recent call last):
  File stdin, line 3, in foo
NameError: global name 'oops' is not defined
Traceback (most recent call last):
  File stdin, line 3, in bar
  File stdin, line 3, in foo
NameError: global name 'oops' is not defined

Peter

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: raise errors

[Benjamin Kaplan]

On Mon, Sep 21, 2009 at 5:17 AM, daved170 daved...@gmail.com wrote:
 Hi everybody,
 I need help with exceptions raising.
 My goal is to print at the outer functions all the errors including
 the most inner one.

 For example:

 def foo1(self):
   try:
        foo2()
   except ? :
         print outer Err at foo1 + ??

 def foo2(self):
   try:
        error occured
   except ? :
         raise inner Err at foo2


 the ? remarks that I have no idea what to use.

 I would like the print to be : outer Err at foo1 , inner Err at foo1

 thanks
 daved
 --

First of all, what version of Python are you using? String exceptions
are deprecated in Python 2.5 and removed in 2.6. You should switch to
using Exceptions. In particular, you'll probably want to subclass
Exception.

String exceptions use the syntax except 'exception string': which
means you need to know the string ahead of time. You'll instead want
to do this.

def foo1() :
   try :
 foo2()
   except Exception, e :
  print outer exception at foo1, %s % str(e)

def foo2() :
   raise Exception(inner exception at foo2)

In Python 3.x, the syntax changed to except Exception as e but that
syntax isn't available in versions prior to 2.6


 http://mail.python.org/mailman/listinfo/python-list

-- 
http://mail.python.org/mailman/listinfo/python-list