New submission from Aleksandr Balezin <geschei...@gmail.com>:
There are a couple of bugs in sqlite bindings have been found related to
reference-counting.
Short info:
sqlite connection class uses a dict to keep references to callbacks. This
mechanism is not suitable for objects which equals but not the same.
con = sqlite3.connect()
con.set_trace_callback(logger.debug)
con.set_trace_callback(logger.debug) # logger.debug == logger.debug is True,
but logger.debug is logger.debug is False because logger.debug bound method is
creating in every call
leads to segmentation fault during calling of trace_callback.
My patch fixes this behavior by using a dedicated variable for keeping
references to each callback and using dict indexed by function name in case of
named callbacks(e.g. create_function()).
Also, due to keeping objects in a variable or in a dict value, it is possible
to use unhashable objects as callbacks. e.g. issue7478
Long version:
Sqlite under the hood use dict(called function_pinboard) to keep references to
callbacks like progress_handler.
It needs to decref callbacks objects after closing sqlite connection.
This mechanism works tolerably(see bug with leaks) with functions but if you
try to use bounded methods it causes a segmentation fault.
Let see how it works.
static PyObject *
Custom_set_callback(CustomObject *self, PyObject* args)
{
PyObject* display_str;
display_str = PyUnicode_FromFormat("set_callback called with cb=%R
id=%i ob_refcnt=%i\n", args, args, args->ob_refcnt);
PyObject_Print(display_str, stdout, Py_PRINT_RAW);
if (PyDict_SetItem(self->function_pinboard, args, Py_None) == -1)
return NULL;
//sqlite3_trace(self->db, _trace_callback, trace_callback);
self->callback_handler = args;
display_str = PyUnicode_FromFormat("set_callback done for cb=%R id=%i
ob_refcnt=%i\n", args, args, args->ob_refcnt);
PyObject_Print(display_str, stdout, Py_PRINT_RAW);
Py_RETURN_NONE;
}
static PyObject *
Custom_call_callback(CustomObject *self)
{
PyObject* display_str;
display_str = PyUnicode_FromFormat("call with id=%i ob_refcnt=%i\n",
self->callback_handler ,
self->callback_handler->ob_refcnt);
PyObject_Print(display_str, stdout, Py_PRINT_RAW);
Py_RETURN_NONE;
}
Python code:
>>>> class TEST:
def log(self, msg=""):
pass
>>>> t = TEST()
>>>> conn = Custom()
>>>> conn.set_trace_callback(t.log)
set_callback called with cb=<bound method TEST.log of <__main__.TEST object at
0x10bc60128>> id=196094408 ob_refcnt=1
set_callback done for cb=<bound method TEST.log of <__main__.TEST object at
0x10bc60128>> id=196094408 ob_refcnt=2
>>>> conn.set_trace_callback(t.log)
set_callback called with cb=<bound method TEST.log of <__main__.TEST object at
0x10bc60128>> id=196095112 ob_refcnt=1
set_callback done for cb=<bound method TEST.log of <__main__.TEST object at
0x10bc60128>> id=196095112 ob_refcnt=1
conn.call()
call with id=196095112 ob_refcnt=0
After second conn.set_trace_callback(t.log) call, object t.log reference-count
is not increased because 't.log in self->function_pinboard' returns True thus
self->function_pinboard[t.log] is not replaced and t.log is not increfed, but
it replaces old object in self->callback_handler.
In the end, self->callback_handler keeps a pointer to t.log with ob_refcnt = 0.
Also, there is no cleaning of self->function_pinboard. This leads to leaks
every object passed as callback(see test_leak() in bug.py).
----------
components: Extension Modules
messages: 346114
nosy: gescheit, ghaering
priority: normal
severity: normal
status: open
title: Reference-counting problem in sqlite
type: crash
versions: Python 2.7, Python 3.5, Python 3.6, Python 3.7, Python 3.8, Python 3.9
_______________________________________
Python tracker <rep...@bugs.python.org>
<https://bugs.python.org/issue37347>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
