Can you describe the queries in more detail? Can you narrow down
exactly which operations / types of queries are substantially slower?
Also, I'm assuming both of you are NOT on Windows? NIOFSDirectory has
poor performance on Windows due to this bug in Sun's JVM:
http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6265734
Mike
Eric Bowman wrote:
[EMAIL PROTECTED] wrote:
On an index of around 20 gigs I've been seeing a performance drop of
around 35% after upgrading to 2.4 (measured on ~10000 requests
identical requests, executed in parallel against a threaded lucene /
apache setup, after a roughly 10000 query warmup). The principal
changes I've made so far are just to switch to NIOFSDirectories and
use read-only index readers.
Our design is roughly as follows: we have some pre-query filters,
queries typically involving around 25 clauses, and some
post-processing of hits. We collect counts and filter post query
using
a hit collector, which uses the (now deprecated) bits() method of
Filters.
I looked at converting us to use the new DocIdSet infrastructure (to
gain the supposed 30% speed bump), but this seems to be somewhat
problematic as there is no guarantee for whether we will get back a
set we can do binary operations on (for example, if we get back a
SortedVIntList, we're pretty much out of luck - the cardinality of
the set
is large (as it's a sortedvintlist), so we can't coerce it into
another type, and it doesn't have the set operations we need to use
it
directly.
Has anyone else seen this? Is there anything else
we should be changing in the upgrade to 2.4?
We are seeing something very similar to this. We had an IndexReader
per
core, and performance was ok. Since upgrading to 2.4, performance has
gone down, in some cases quite dramatically. We switched to a single
IndexReader, and things got better.
We are now also looking at NIOFSDirectory after noticing a lot of
contention in IndexReader, but that didn't help as much as we had
hoped.
To be honest I'm considering trying to downgrade back to 2.3 for now,
this is looking like a step backwards for us.
Thanks for any suggestions,
Eric
--
Eric Bowman
Boboco Ltd
[EMAIL PROTECTED]
http://www.boboco.ie/ebowman/pubkey.pgp
+35318394189/+353872801532
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
