On Sat, May 17, 2008 at 7:29 AM, Matthew Toseland
<toad at amphibian.dyndns.org> wrote:
> Ian and I have eventually come to the conclusion that we should include db4o,
> and use it for our various persistence needs. I eventually reached the
> conclusion that while we can do most of what we need to do with simple
> flatfile databases, there are big chunks that will require a real database of
> some kind (even if it's only a persistent hash table). db4o has various
> advantages:
> - Robust in real-world use. See for example this testimonial from a company
> who used it on cell phones:
> http://www.db4o.com/about/customers/success/mandalait.aspx
> BDBJE has not met our expectations in this regard. It seems very sensitive to
> unusual situations - in particular, it will spontaneously corrupt and lose
> all data on running out of disk space.
> - True object database: no SQL, simple and powerful queries, etc.
> - Transparent or manual activation of objects from storage.
> - 800K jar, so not big enough to be a problem.
> - Mature and actively maintained.
> - Allows for future expansion (e.g. passive requests will need to store a fair
> amount of persistent data).
> - Much more flexible than the hand-coded solution I was thinking of. We can
> persistent the entire queue (not just the splitfiles), if it's useful to do
> that.
> - Transactions (although this requires some juggling of in-memory objects on
> rollback).
Look good to me (except it's website need registration)
> Tasks:
> - Add db4o to freenet-ext.jar.
> - Think about using it for the datastore. We don't want to have two databases!
> Sdiz's new datastore may be the One True Store, or it may not be. If it's
> not, we don't want to keep BDBJE: we could build a db4o-based store, with or
> without LRU replacement. It would have the advantage of filling up more
> quickly than sdiz's store. It should require reconstructing less frequently
> than BDBJE!
> - Migrate the client layer, including splitfiles, pendingKeys, and so on, to
> be persisted via db4o.
> Of course there will be latency here when objects are
> not cached, so we will need to cache a few request choices in advance for
> each RequestStarter. And we will need to devise some way to deal with
> requests that don't want to be persisted - presumably we'd keep them in RAM.
Please don't.
According to what I have read, db4o should be good enough to use directly:
- Active objects are kept with a WeakReference, so as long as it is not GC'ed,
you don't have to read the disk.... (there is an option to use
hard reference too)
- Data are prefetched and activated in batch.. if the query is
well-written, we have
the items in memory already.
- CachedIoAdapter provide a low level disk cache.
This maybe not as good as a custom cache scheme.. But I really hate
the idea of having yet-another-caching scheme for some marginal (or
even hypo-theoretical) performance benefit -- we have had too much of
them already.
Please, do that ONLY if it's supported by some benchmarks / cpu profiles.
--
