[ https://issues.apache.org/jira/browse/LUCENE-1313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12703366#action_12703366 ]
Michael McCandless commented on LUCENE-1313: -------------------------------------------- {quote} Agreed that DW can write the segment to the RAMDir. I started coding along these lines however what do we do about the RAMDir merging? This is why I was thinking we'll need a separate IW? Otherwise the ram segments (if they are treated the same as disk segments) would quickly be merged to disk? Or we have two separate merging paths? {quote} Hmm, right. We could exclude RAMDir segments from consideration by MergePolicy? Alternatively, we could "expect" the MergePolicy to recognize this and be smart about choosing merges (ie don't mix merges)? EG we do in fact want some merging of the RAM segments if they get too numerous (since that will impact search performance). {quote} > we should make with NRT is to not close the doc store > (stored fields, term vector) files when flushing for an NRT > reader. Agreed, I think this feature is a must otherwise we're doing unnecessary in ram merging. {quote} OK, let's do this as a separate issue/optimization for NRT. There are two separate parts to it: * Ability to store doc stores in "real" directory (looks like you opened LUCENE-1618 for this part). * Ability to "share" IndexOutput & IndexInput {quote} I ran into problems with this before, I was trying to reuse Directory to write a transaction log. It seemed theoretically doable however it didn't work in practice. It could have been the seeking and replacing but I don't remember. FSIndexOutput uses a writeable RAF and FSIndexInput is read only why would there be an issue? {quote} Hmm... seems like we need to investigate further. We could either "ask" an IndexOutput for its IndexInput (sharing the underlying RAF), or try to separately open an IndexInput (which may not work on Windows). {quote} To implement this functionality in parallel (and perhaps make the overall patch cleaner), writing doc stores directly to a separate directory can be a different patch? There can be an option IW.setDocStoresDirectory(Directory) that the patch implements? Then some unit tests that are separate from the near realtime portion. {quote} Yes, separate issue (LUCENE-1618). > Realtime Search > --------------- > > Key: LUCENE-1313 > URL: https://issues.apache.org/jira/browse/LUCENE-1313 > Project: Lucene - Java > Issue Type: New Feature > Components: Index > Affects Versions: 2.4.1 > Reporter: Jason Rutherglen > Priority: Minor > Fix For: 2.9 > > Attachments: LUCENE-1313.jar, LUCENE-1313.patch, LUCENE-1313.patch, > LUCENE-1313.patch, LUCENE-1313.patch, lucene-1313.patch, lucene-1313.patch, > lucene-1313.patch, lucene-1313.patch > > > Realtime search with transactional semantics. > Possible future directions: > * Optimistic concurrency > * Replication > Encoding each transaction into a set of bytes by writing to a RAMDirectory > enables replication. It is difficult to replicate using other methods > because while the document may easily be serialized, the analyzer cannot. > I think this issue can hold realtime benchmarks which include indexing and > searching concurrently. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. --------------------------------------------------------------------- To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org For additional commands, e-mail: java-dev-h...@lucene.apache.org