Hi,
I would like to know if you could please share some input about this
ThreadConnectionPool class for libschevo/ZODB. :)
Thanks!
Etienne
Le 2018-04-20 à 04:49, Etienne Robillard a écrit :
Heads up people!
I've finally managed to make a working `ThreadedConnectionPool` class
for supporting multiple ZODB databases connections on top of libschevo
API! :)
Technically, I decided to use gevent.Greenlet under the hood to spawn
multiple cooperative ClientStorage client/server connections.
I'm also making all my testing and development under PyPy 5.9.
You can check out the code here:
https://bitbucket.org/tkadm30/libschevo/commits/37feb029615d76f3d81233990e509b6eb6ffb5d7
Comments are welcome! :)
Etienne
Le 2018-04-16 à 18:27, Etienne Robillard a écrit :
Hi Jason,
I just made some more changes to my ThreadedConnectionPool class
here:
https://bitbucket.org/tkadm30/django-hotsauce/commits/81d2e8f30019840d9c8bbdf7f82df6de2be024fc
In summary, the `ThreadedConnectionPool` class is now a subclass of
`threading.Thread` and is extending the `run` method to populate a
thread local dictionary with ClientStorage connections.
Kind regards,
Etienne
Le 2018-04-16 à 09:14, Jason Madden a écrit :
On Apr 15, 2018, at 8:17 PM, Etienne Robillard
<tkad...@yandex.com> wrote:
I would like to define a `ThreadedConnectionPoll` class to allow
multithreaded caching of ZODB databases into memory.
The ZODB.DB object is *already* thread safe for connection
management. You shouldn't need to do anything other than use
ZODB.DB().open() and conn.close() from all the threads you're using.
(That is, create exactly one ZODB.DB object for each database you
are using and share that object amongst your threads, using
db.open()/conn.close() as needed. For storages like RelStorage that
have caches at the ZODB.DB level---which are also thread safe---this
is important.)
If you are managing multiple databases and want to be able to make
references between them (a multiple-database) it is critical that
they share the same `databases` object. But that is an advanced usage.
Here's a updated version of my code so far using threading.local :
from threading import local
from notmm.dbapi.orm import ClientStorageProxy
class ThreadedConnectionPool(object):
def __init__(self, d, debug=True):
if debug:
assert isinstance(d, dict) == True
local.pool = d
for key,value in local.pool.iteritems():
Unless you left out part of the code, it appears that you're trying
to set (and read) a class attribute on the threading.local class.
This does not create thread-local state. It is also not portable and
doesn't work when local is an extension type:
py> from threading import local
py> import sys
py> sys.version_info # CPython 3.7; same result in 2.7
sys.version_info(major=3, minor=7, micro=0, releaselevel='beta',
serial=3)
py> local
<class '_thread._local'>
py> local.foo = 1
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: can't set attributes of built-in/extension type
'_thread._local'
pypy> sys.version_info # pypy 2
(major=2, minor=7, micro=13, releaselevel='final', serial=42)
pypy> from threading import local
pypy> local.foo = 1
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: can't set attributes on type object '_local'
It generally also shouldn't be necessary as the parameters to
__init__ are preserved and used again in each thread; you can use
this to share information if those parameters are mutable:
py> from threading import get_ident
py> from threading import Thread
py> class MyLocal(local):
... def __init__(self, l):
... print("Creating in thread", get_ident(), "param", l)
... l.append(get_ident())
...
py> l = []
py> mylocal = MyLocal(l)
Creating in thread 140736147411840 param []
py> l
[140736147411840]
py> def target():
... mylocal.foo = 1
...
py> t = Thread(target=target)
py> t.start()
Creating in thread 123145538318336 param [140736147411840]
None
py> l
[140736147411840, 123145538318336]
Jason
--
Etienne Robillard
tkad...@yandex.com
https://www.isotopesoftware.ca/
--
https://mail.python.org/mailman/listinfo/python-list
