On Fri, Sep 28, 2007 at 03:42:30PM -0400, Erik Morton wrote: > Thanks Jens. > > I'm seeing very strange behavior. The ferret/drb server is running on > the web server. I disable AAF in my environment.rb like so: > > MyModel.disable_ferret > > Then, every hour I run a script that grabs the records that have been > updated and I call > > MyModel.bulk_index array_of_changed_objects > > When there are, say, 100 or so objects that have changed the server > where DRB is will have approximately 50% of the CPU waiting on IO, > and will drop from 120MB free of memory to 10MB free, though the ruby/ > drb process doesn't seem to actually consume that memory -- or at > least it's not reported to by top.
Where do you get these numbers from? possibly it's just the os using unused ram for filesystem buffers? > While the batch update is happening it seems like my entire site is > locked up. Requests usually hang until the indexing completes. > Further, I can't even run script/ console from a different machine > until the indexing completes. I'm doing the bulk index on two > indexes. One is about 41K records, the other is 1 million records. In > both cases there has been at most 100 or so objects that needed to be > indexed in bulk. The fact that script/ console, when run from a > different server, doesn't load until the index stops makes me think > that either something is blocking in the ferret/drb server, or the > optimization of the 3GB index after the bulk_index of 100 records is > consuming all of the web server's resources. While the batch update is running, nobody else is able to update the index. So yes, every other request that wants to update the index will hang. However requests not using aaf at all, or searches, should do fine. If you feel like the DRb takes too much CPU, use renice or nice when you start it to lower it's priority. Another possibility that comes to mind is database locks, however I can't imagine where these should come from. With your index size, the optimizing might be the culprit - just comment out that portion and look how it goes without it (in ferret_extensions.rb). cheers, Jens -- Jens Krämer http://www.jkraemer.net/ - Blog http://www.omdb.org/ - The new free film database _______________________________________________ Ferret-talk mailing list [email protected] http://rubyforge.org/mailman/listinfo/ferret-talk
