[issue3783] dbm.sqlite proof of concept

[Runar Tenfjord]

Runar Tenfjord runar.tenfj...@gmail.com added the comment:

Multi threading:

According to http://www.sqlite.org/cvstrac/wiki?p=MultiThreading
we need to keep a connection for each thread to support multi threaded
access to the database. 

I came across this when deploying an application in a multi threaded
environment. Solution is to keep the connection in the thread local-data.

Also note that a memory database is not shared between threads and
a hairy workaround with a separate working thread with queues for access
is needed. A memory database could perhaps be disallowed as the dbm is file 
only?

import threading

class SQLhash(collections.MutableMapping):
def __init__(self, filename=':memory:', flags='r', mode=None):
self.__filename = filename
self.__local = threading.local()

MAKE_SHELF = 'CREATE TABLE IF NOT EXISTS shelf (key TEXT PRIMARY KEY, 
value TEXT NOT NULL)'
self.conn.execute(MAKE_SHELF)
self.conn.commit()

@property
def conn(self):
try:
conn = self.__local.conn
except AttributeError:
conn = self.__local.conn = sqlite3.connect(self.__filename)
self.conn.text_factory = bytes

return conn

--
type: feature request - behavior

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue3783
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3783] dbm.sqlite proof of concept

[Runar Tenfjord]

Runar Tenfjord runar.tenfj...@gmail.com added the comment:

By utilizing triggers on inserts and deletes it is possible to
keep track of the size and speed up __len__ by 10 x.

SQL:

CREATE TABLE IF NOT EXISTS info
   (key TEXT UNIQUE NOT NULL,
value INTEGER NOT NULL);

INSERT OR IGNORE INTO info (key,value) VALUES ('size',0);

CREATE TABLE IF NOT EXISTS shelf
(key TEXT UNIQUE NOT NULL,
 value TEXT NOT NULL);

CREATE TRIGGER IF NOT EXISTS insert_shelf
AFTER INSERT ON shelf
BEGIN
 UPDATE info SET value = value + 1 WHERE key = 'size';
END;

CREATE TRIGGER IF NOT EXISTS delete_shelf
AFTER DELETE ON shelf
BEGIN
 UPDATE info SET value = value - 1 WHERE key = 'size';
END;

On my laptop this increase the speed of 'len' about 10x

I have a slightly modified version of dbsqlite.py for
running on python 2.5 utilizing the triggers for 
keep track of the size:

http://dpaste.com/hold/122439/

--
nosy: +rute

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue3783
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com