Hi Alexander, IndexWriter by default will only keep the segment files of the latest commit and delete the old segments that are not referenced by IndexWriter anymore, so the situation you described do exists
To address the problem, there's several options: 1. You can use Lucene's NRT search feature, where the reader is directly opened from IndexWriter and the IndexWriter will be aware of opened reader and won't delete files that are still being used by readers/searchers. (See DirectoryReader#open(IndexWriter) method) 2. You can control the file deletion behavior of IndexWriter by calling `IndexWriterConfig#setIndexDeletionPolicy`, you can probably set that to NoDeletionPolicy to prevent indexWriter from delete anything and manage the directory yourself. 3. Or if you don't want to change the deletion policy, you can try to play with IndexWriter#incRefDeleter and IndexWriter#decRefDeleter those two APIs a bit to tell IndexWriter which commit your searcher is using. Those are all doable I believe but I guess 1 is probably the easiest? Patrick On Fri, May 17, 2024 at 11:45 AM 쿨해머 <ajtwlstmd...@gmail.com> wrote: > Hello. Let me share what I know. Indexing is always done and read in units > called segments. When the IndexWriter writes new indexes, it is creating > new segments. The important point here is that it does not touch the > existing segments. This ensures that the consistency for searchers using > the existing segments can be maintained. For reference, even when merging, > it is done by combining the segments in a separate segment file and then > removing the old segments. This is what I know. If there are any mistakes, > I would appreciate it if someone could correct me. > > Thank you. > > 2024년 5월 17일 (금) 오후 11:30, Alexander Panchenko <ird...@gmail.com>님이 작성: > >> Hello! >> >> I was examining the Lucene source code with the desire to figure out how >> an IndexSearcher could be safely used while IndexWriter is open on the same >> directory. My concern is that when an IndexWriter is open, it could remove >> existing files in the directory (for example: during the merge procedure), >> meanwhile these files are being used by IndexSearcher. I'm asking because I >> want to implement my own directory and I'm not sure if it's safe to keep an >> IndexWriter open while I'm using IndexSearch to execute search queries. I >> see that once an IndexSearcher is open, it opens several segments files and >> keeps them open. >> >> Is it safe to use IndexWriter and IdnexSearcher at the same time? And if >> so, how does IndexWriter not break the active IndexSearcher? >> >> Thank you. >> >
