On Sun, 2008-01-20 at 05:44 -0500, Michael McCandless wrote: > These results are very interesting. With 3 threads on SSD your > searches run 87% faster if you use 3 IndexSearchers instead of > sharing a single one.
That is my observation, yes. Please note that this is with Lucene 2.1. I've tried with Lucene 2.3pr2 with the old index though. The relative results are the same, although overall performance is quite a bit lower. == Lucene 2.1 - Harddisks (2 * 15000 RPM in software RAID 1) == 1 thread, 1 searcher: 109 queries/sec 2 threads, 1 searcher: 118 queries/sec 2 threads, 2 searchers: 157 queries/sec 3 threads, 1 searcher: 111 queries/sec 3 threads, 3 searchers: 177 queries/sec 4 threads, 1 searcher: 108 queries/sec 4 threads, 4 searchers: 169 queries/sec == Lucene 2.1 - SSD (2 * 32 GB Samsung in software RAID 0) == 1 thread, 1 searcher: 193 queries/sec 2 threads, 1 searcher: 295 queries/sec 2 threads, 2 searchers: 357 queries/sec 3 threads, 1 searcher: 197 queries/sec 3 threads, 3 searchers: 369 queries/sec 4 threads, 1 searcher: 192 queries/sec 4 threads, 4 searchers: 302 queries/sec == Lucene 2.3pr2 - Harddisks (2 * 15000 RPM in software RAID 1) == 1 thread, 1 searcher: 93 queries/sec 2 threads, 1 searcher: 112 queries/sec 2 threads, 2 searchers: 135 queries/sec 3 threads, 1 searcher: 96 queries/sec 3 threads, 3 searchers: 153 queries/sec 4 threads, 1 searcher: 98 queries/sec 4 threads, 4 searchers: 150 queries/sec == Lucene 2.3pr2 - SSD (2 * 32 GB Samsung in software RAID 0) == 1 thread, 1 searcher: 171 queries/sec 2 threads, 1 searcher: 261 queries/sec 2 threads, 2 searchers: 325 queries/sec 3 threads, 1 searcher: 185 queries/sec 3 threads, 3 searchers: 336 queries/sec 4 threads, 1 searcher: 182 queries/sec 4 threads, 4 searchers: 280 queries/sec I don't know if the lower performance is because I'm using the old index instead of generating a new one with Lucene 2.3pr2? > Have you done any profiling to understand where the threads are > waiting when you share one IndexSearcher? Not at all. The whole threads/searchers-thing in Lucene is quite new to me - in our production system we use synchronized access to the searcher with only one search at a time. > EG YourKit can tell you where the threads are waiting... I'll be happy to investigate further. It'll have to wait until friday though, if it involves more than a trivial amount of time. I'll also put the (very simple) code online for review. Unfortunately I can't provide neither the index, nor the queries as I'm not allowed to do so. --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
