[issue46256] Objects __del__ called after module have been removed

[Daniele Varrazzo]

Daniele Varrazzo  added the comment:

Thank you @pablogsal. For me it was surprising as never seen before 3.10 
instance.

If this is the expected behaviour (and the stdlib expects it) we can try and 
take the same type of care in psycopg.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46256] Objects __del__ called after module have been removed

[Christopher Vickery]

Christopher Vickery  added the comment:

This it very clear and totally consistent with the observed behavior. I can 
think of a couple of workarounds for my app (move the db access from module 
initialization to a function that gets invoked "on demand", or redirect 
sys.stderr to /dev/null just before exiting).

Perhaps the Psycopg team can deal with it more directly.

Many thanks to all for the time and effort taken to make clear what's happening!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46256] Objects __del__ called after module have been removed

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

This doesn't look like a bug to me and is documented behavior:

https://docs.python.org/3/reference/datamodel.html#object.__del__

Check this paragraph:

Warning Due to the precarious circumstances under which __del__() methods are 
invoked, exceptions that occur during their execution are ignored, and a 
warning is printed to sys.stderr instead. In particular:
__del__() can be invoked when arbitrary code is being executed, including from 
any arbitrary thread. If __del__() needs to take a lock or invoke any other 
blocking resource, it may deadlock as the resource may already be taken by the 
code that gets interrupted to execute __del__().

__del__() can be executed during interpreter shutdown. As a consequence, the 
global variables it needs to access (including other modules) may already have 
been deleted or set to None. Python guarantees that globals whose name begins 
with a single underscore are deleted from their module before other globals are 
deleted; if no other references to such globals exist, this may help in 
assuring that imported modules are still available at the time when the 
__del__() method is called.



Here, the object __del__ should take s strong reference to whatever is going to 
use to ensure is available when needed. Indeed, the standard library does this 
regularly, for example:

https://github.com/python/cpython/blob/f4c03484da59049eb62a9bfb963e2267d187/Lib/asyncio/windows_utils.py#L110


I am closing it as "not a bug" but feel free to reopen if we think we missed 
something or I misinterpret the bug report.

--
nosy: +pablogsal
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46256] Objects __del__ called after module have been removed

[Christopher Vickery]

Christopher Vickery  added the comment:

The app has to open a db connection during module initialization to trigger the 
problem, so I can't provide a self-contained way to reproduce the problem.

The closest I can come is the attached file, format_rules_annotated.py.

With lines 66-69 in place, the problem occurs, even if there are no command 
line arguments, with Python 3.10, but not with Python 3.9. If I comment out 
lines 66-69 there is no problem with either 3.10 or 3.9, even if I add a 
command line argument that causes connections to be opened within functions 
defined within the module. (I added the explicit initialization of the 
institution_names dict so I could test those function calls without accessing 
the db during module initialization.)

FWIW, here is an execution under 3.10 with lines 66-69 commented out:

$ python -m format_rules_annotated -r QCC01:QNS01:PH:35
QCC01:QNS01:PH:35
C- or above in PH 101 at Queensborough, 4.0 credits, transfers to Queens as 
PHYS-11 and PHYS-14, 4.0 credits.
[No "Exception ignored" message appears here because db was not accessed during 
module initialization]
$

--
Added file: https://bugs.python.org/file50542/format_rules_annotated.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46256] Objects __del__ called after module have been removed

[Dennis Sweeney]

Dennis Sweeney  added the comment:

I'm getting this:

Traceback (most recent call last):
  File "/mnt/c/Users/sween/Source/Repos/cpython2/cpython/delbug.py", line 65, 
in 
conn = psycopg.connect('dbname=cuny_curriculum')
  File "/home/sween/.local/lib/python3.10/site-packages/psycopg/connection.py", 
line 572, in connect
raise ex.with_traceback(None)
psycopg.OperationalError: connection is bad: No such file or directory
Is the server running locally and accepting connections on that socket?

Is there a self-contained way to reproduce the issue without a DB up and 
running? Or to somehow shrink the reproducer down to its cpython-relevant bare 
essentials?

--
nosy: +Dennis Sweeney

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46256] Objects __del__ called after module have been removed

[Christopher Vickery]

Christopher Vickery  added the comment:

If the attached module is run with no command line arguments, it will reproduce 
the problem without doing anything else. When run using Python 3.9 it does not 
cause the problem. (This module comes from 
https://github.com/cvickery/transfer-app, with an unnecessary import commented 
out.)

--
nosy: +cvickery
Added file: https://bugs.python.org/file50540/format_rules.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46256] Objects __del__ called after module have been removed

[Daniele Varrazzo]

New submission from Daniele Varrazzo :

The following bug has been reported to Psycopg:

https://github.com/psycopg/psycopg/issues/198

At the end of the program, errors such as the following are dumped:

Exception ignored in: 
Traceback (most recent call last):
  File 
"/opt/homebrew/lib/python3.10/site-packages/psycopg/pq/pq_ctypes.py", line 91, 
in __del__
TypeError: 'NoneType' object is not callable

Where `None` is some module-level objects, such as `os.getpid`.

Unfortunately I don't have a full repro. The error seems only happening in 
Python 3.10.

--
components: Interpreter Core
messages: 409684
nosy: piro
priority: normal
severity: normal
status: open
title: Objects __del__ called after module have been removed
versions: Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com