Hi Paul,
[Paul Rubin] > Basically I wish there was a way to have persistent in-memory objects > in a Python app, maybe a multi-process one. So you could have a > persistent dictionary d, and if you say > d[x] = Frob(foo=9, bar=23) > that creates a Frob instance and stores it in d[x]. Then if you > exit the app and restart it later, there'd be a way to bring d back > into the process and have that Frob instance be there.
Have you looked at Ian Bicking's SQLObject?
http://sqlobject.org/
To define a class
class MyPersistentObj(SQLObject):
foo = IntCol()
bar = IntCol()To instantiate a new object
my_new_object = MyPersistentObj(foo=9, bar=23)
Once the new object has been created, it has already been persisted into a RDBMS table automatically. To reload it from the table/database, e.g. after a system restart, simply supply its id.
my_existing_object = MyPersistentObj.get(id=42)
Select a subset of your persistent objects using SQL-style queries
my_foo_9_objects = MyPersistentObj.select(MyPersistentObj.q.foo == 9)
for o in my_foo_nine_objects:
process(o)SQLObject also takes care of caching, in that objects are optionally cached, associated with a specific connection to the database. (this means that it is possible to have different versions of the same object cached with different connections, but that's easy to solve with good application architecture). So in your case, if your (web?) app is persistent/long-running, then you can simply have SQLObject cache all your objects, assuming you've got enough memory. (Hmm, I wonder if SQLObject could be made to work with weak-references?). Lastly, caching can be disabled.
I've found performance of SQLObject to be pretty good, but since you haven't specified particular requirements for performance, it's not possible to say if it meets your criteria. Although I feel comfortable in saying that SQLObject combined with an SQLite in-memory database should give pretty good performance, if you've got the memory to spare for the large databases you describe.
Other nice features include
1. RDBMS independent: currently supported are PostGres, FireBird, MySQL, SQLite, Oracle, Sybase, DBM. SQLServer support is in the pipepline. SQLObject code should be completely portable between such backend stores.
2. Full support for ACID transactional updates to data.
3. A nice facility for building SQL queries using python syntax.
4. Automated creation of tables and databases. Table structure modification supported on most databases.
5. Full support for one-to-one, one-to-many and many-to-many relationships between objects.
All in all, a great little package. I recommend that you take a close look.
Regards,
-- alan kennedy ------------------------------------------------------ email alan: http://xhaus.com/contact/alan -- http://mail.python.org/mailman/listinfo/python-list
