[
https://issues.apache.org/jira/browse/LUCENE-3653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13172637#comment-13172637
]
Uwe Schindler edited comment on LUCENE-3653 at 12/19/11 10:16 PM:
------------------------------------------------------------------
LUCENE-1669 seems to be relict and should already be solved since Lucene 2.9 in
newer Windows versions? And BTW, MMap is only useful for reading indexes,
written are they by NIO/SimpleFS.
bq. BTW, the statement on optimize does not regard a search feature but an
index one
This statement was just a comparison, to underline that both statements "RAMDir
is fast" and "Optimize is needed" are myths.
bq. I was responding to the statement that RAMDirectory going away
See my previous comment:
bq. We are working on a RAM-Dir like approach storing the files outside Java
heap using a large DirectByteBuffer (which is the same code as MMapDirctory).
The problem is writing to such a directory, but reading is as fast (or even
faster) than RAMDirectory without locks.
was (Author: thetaphi):
LUCENE-1669 is a relict and already solved since Lucene 2.9 (calling
DirectByteBuffer.cleaner().clean() to unmap).
bq. BTW, the statement on optimize does not regard a search feature but an
index one
This statement was just a comparison, to underline that both statements "RAMDir
is fast" and "Optimize is needed" are myths.
bq. I was responding to the statement that RAMDirectory going away
See my previous comment:
bq. We are working on a RAM-Dir like approach storing the files outside Java
heap using a large DirectByteBuffer (which is the same code as MMapDirctory).
The problem is writing to such a directory, but reading is as fast (or even
faster) than RAMDirectory without locks.
> Lucene Search not scalling
> --------------------------
>
> Key: LUCENE-3653
> URL: https://issues.apache.org/jira/browse/LUCENE-3653
> Project: Lucene - Java
> Issue Type: Improvement
> Reporter: Gerrit Jansen van Vuuren
> Attachments: App.java,
> LUCENE-3653-VirtualMethod+AttributeSource.patch,
> LUCENE-3653-VirtualMethod+AttributeSource.patch,
> LUCENE-3653-VirtualMethod+AttributeSource.patch, LUCENE-3653-no-sync.png,
> LUCENE-3653-sync-.png, LUCENE-3653.patch,
> LUCENE-3653.patch-BiasedLockingStartupDelay_1.png,
> LUCENE-3653.patch-BiasedLockingStartupDelay_2.png,
> LUCENE-3653.patch-BiasedLockingStartupDelay_3.png,
> Threads-LUCENE-3653.patch.png, lucene-unsync.diff, profile_1_a.png,
> profile_1_b.png, profile_1_c.png, profile_1_d.png, profile_2_a.png,
> profile_2_b.png, profile_2_c.png
>
>
> I've noticed that when doing thousands of searches in a single thread the
> average time is quite low i.e. a few milliseconds. When adding more
> concurrent searches doing exactly the same search the average time increases
> drastically.
> I've profiled the search classes and found that the whole of lucene blocks on
> org.apache.lucene.index.SegmentCoreReaders.getTermsReader
> org.apache.lucene.util.VirtualMethod
> public synchronized int getImplementationDistance
> org.apache.lucene.util.AttributeSourcew.getAttributeInterfaces
> These cause search times to increase from a few milliseconds to up to 2
> seconds when doing 500 concurrent searches on the same in memory index. Note:
> That the index is not being updates at all, so not refresh methods are called
> at any stage.
> Some questions:
> Why do we need synchronization here?
> There must be a non-lockable solution for these, they basically cause
> lucene to be ok for single thread applications but disastrous for any
> concurrent implementation.
> I'll do some experiments by removing the synchronization from the methods of
> these classes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
