Thanks for the response Jens.
Indeed I am sorting by something other than relevancy, so that would
explain it. Optimized, it's extremely fast and handles a good load,
but new records kill it until I optimize.
I haven't tried :merge_factor as I wasn't aware of it. I'm not sure
it will help given the above.
Regarding the re-index locking code, it's not 100% threadsafe or
necessarily the desired behavior (calling rebuild_index when the last
rebuild isn't finished won't result in a new index,) but it works for
me. I just added this to the beginning of rebuild_index:
if @rebuilding
@logger.warn "already rebuilding index - will lock until
rebuilding is complete"
while @rebuilding
sleep 1
end
return
end
@rebuilding = true
The big problem was in new_index_for, which always uses the same
path, and causes the multiple rebuilds to be stomping on each other.
It's not necessary to fix this after adding the above though.
Thanks again.
--
Alex Neth
Liivid Inc / cribQ
www.liivid.com / www.cribq.com
On Jan 28, 2008, at 3:45 PM, [EMAIL PROTECTED] wrote:
Date: Sat, 26 Jan 2008 10:53:22 +0100
From: Jens Kraemer <[EMAIL PROTECTED]>
Subject: Re: [Ferret-talk] ferret success stories?
To: [email protected]
Message-ID: <[EMAIL PROTECTED]>
Content-Type: text/plain; charset=iso-8859-1
Hi Alex,
sorry about not responding to that mail, been pretty busy with other
stuff at that time.
I just dugg it out of my mail archive now and yes, I'd like to have a
look at your locking code to prevent parallel index rebuilds :-)
Regarding your optimization troubles - I'm afraid I don't know what's
going wrong there with your index being slow when you don't
optimize it
(unless you're sorting your search results by something else than
relevancy - that is indeed known to be slow when the index isn't
optimized).
Did you try to tweak some of Ferret's more obscure indexing parameters
like :merge_factor (lowering it (the Ferret shortcut suggests a
value of
2 or 3 for better search performance) will let Ferret merge segments
more frequently so on average there are less files in the index which
should improve search performance)?
Cheers,
Jens
_______________________________________________
Ferret-talk mailing list
[email protected]
http://rubyforge.org/mailman/listinfo/ferret-talk
