On Fri, 5 Jan 2007, Peter Fein wrote:
On Friday 05 January 2007 12:31, Andi Vajda wrote:
On Thu, 4 Jan 2007, Peter Fein wrote:
[0] Has anyone else had difficulties w/ bsddb (outside PyLucene)?
Despite it's excellent reputation, it's given us lots of trouble. We've
been using gdbm instead.
What kind of troubles ?
Hmm, let's see:
* len(db) doesn't work reliably - sometimes gives wrong/negative answers
* keys sometimes get 'lost' when iterating over largish (~200K) db's - the
data's not corrupted, the library just confuses itself. There's a python bug
for this, I can't find it ATM.
I see, bugs with the bsddb python API wrappers. I don't use that API it
anymore as it's usually out of date with Berkeley DB itself since it's not
maintained by Sleepycat (now Oracle) but is part of Python itself. Because
these two software products don't coordinate their release cycles, this
invariably leads to the APIs and their wrappers to be out of sync. I ended up
writing my own python wrappers for Berkeley DB giving me much finer control
over how data was accessed and better choice of API usage [1].
* DBNoMemoryError: (12, 'Cannot allocate memory -- Lock table is out of
available locker entries') - this was the last straw, never found out what
the problem was. Some sort of create-time config option, IIRC.
Yes, I saw this a long time ago and worked it around by adding more such
lockers with the set_lk_max_locks() and set_lk_max_objects() APIs.
The new Berkeley DB 4.5.20 release supports MVCC [2][3] and that more or less
does away with needing read locks. If your usage patterns fit the MVCC
constraints, it's a very neat feature.
Basically, we found it just required too much hand-holding / broke far too
easily to make it worthwhile.
Yes, it can be finnicky but I've found it quite good if handled correctly.
Handling locks and deadlock situations can be particularly troublesome but
again, understanding your access patterns helps a lot with untangling the
the locks.
Andi..
[1]
http://svn.osafoundation.org/chandler/trunk/internal/chandlerdb/chandlerdb/persistence/
[2]
http://www.oracle.com/technology/documentation/berkeley-db/db/api_c/frame.html
[3]
http://lists.osafoundation.org/pipermail/chandler-dev/2007-January/007481.html
_______________________________________________
pylucene-dev mailing list
[email protected]
http://lists.osafoundation.org/mailman/listinfo/pylucene-dev
