Hey ... thanks for your suggestions .. as a quick background, i'm one of the authors of omdb.org[2], Jens and I wrote the search backend for omdb, some parts of that backend is now part of AAF .. But omdb.org still has its own search logic.. basically because searching is a complex task. I don't see a plugin, that automagically resolves all the search related topic.. however some of your points are valid. i'll give my 2cent to your suggestions, even though, omdb.org does not use AAF :-)
> 1) Each model needs remote => true adding to it for remoting to work. > This is a breach of the DRY principle. > Solution: Get rid of this requirement, if the config is there then use > it, if its not then don't. thats true.. basically you just need :remote => false in testing.. at least thats how omdb is doing it .. > 3) Drb server is a central point of failure. > Solution 1: Lucence locks the file while it works which effectively > allows many threads to write but it serialises them. This should be > the > case with aaf as well, at least then basic parrallism will work > without > having to setup up Drb or any other config settings. Sure you *could* > get fancy with partial locking schemes but right now what I think I > care > about is it at least work. Drb should be the next performance step > from > Solution 1, improving performance at the cost of reliability. omdb uses a queue to serialize all indexing requests. We also have two indizes, one offline index (new index requests will be added to that), and on online index (used for searching). As soon as our queue is empty or we processed 256 indexing requests, we'll sync online and offline index. This eliminates all of our search/indexing problems and is running very stable. Afaik, AAF is using backgroundrb for searching, this is something i would change, as i don't see a problem having the mongrel processes using a R/O access to the index (like the Searcher class). > Solution 3: Allow the index to be stored in the database and use > transactions to manage the updates (Just as compass does to lucene) That's an interesting point, and Jens and I even talked about that some month ago. Basically it is possible to write a new StorageEngine for Ferret (it just supports Memory and FS storage right now), but never got the time to do it. I'm fine with the online/offline index concept in the filesystem. This can even be distributed to several servers via rsync. But a db-backend is an option. > > 4) AAF fails unexpectedly if you just use it without Drb. > Solution: Produce a warning if more than one thread/process seems > to be > updating the file at any one time. Although bare in mind Solution 1 > for > problem (3) solves this in a different way and is a better solution > overall. I agree with your point from earlier on, I would simply eliminiate the non-drb option :-) Just use non-drb for testing .. btw, the whole omdb/ferret integration is open source, so feel free to take a look[2]. Ben [1] http://www.omdb.org [2] http://bugs.omdb.org/browser/branches/2007.1/lib/omdb/ferret _______________________________________________ Ferret-talk mailing list [email protected] http://rubyforge.org/mailman/listinfo/ferret-talk
