Thanks Mark.
Ok, I got your point. So it happens like this :
a) If it is me, who is re-opening an IndxReader, at any time, but
"manually-programmatically". That is, I don't want
a-sort-of-automatic-reopening-of-IndexWriter, then I am fine.
b) If I do wish this automatic-reopening of index (using IndexAccessor),
then I am forced to rely on all the indexer threads releasing the reference
to IndexWriter, which by the way, as a developer, can never be sure of (that
is, I don't have any control, as to when exactly all the threads leave the
reference ).
Will be obliged if you could give a confirmation to my understanding.
Thanks
Ajay Garg
markrmiller wrote:
>
> You are right that if auto-commit=true and a user reopens an
> IndexReader, the docs will absolutely be visible as they are flushed. I
> think the part you are missing is that you need to be cooperating with
> the IndexAccessor: a user should not be reopening an IndexReader. The
> whole point of IndexAccessor is to coordinate these things...when a
> Writer is released, we know the index has changed, so that is when the
> IndexReaders are reopened for you. Because the IndexWriter is cached and
> shared by Threads, a thread might release the Writer while another is
> still using it...that is why things are not reopened and the Writer not
> closed until the last thread releases its reference to it. Essentially,
> IndexAccessor control visibility by controlling how current the view of
> the Readers is, by controlling their reopening -- a user should agree
> not to reopen -- just like he must agree not to use a ReadingWriter to
> delete.
>
> If you want to just set an IndexWriter to indexing for eternity and then
> have some Readers that you occasionally reopen, you don't need
> IndexAccessor. Its purpose is to coordinate ReaderReaders,
> WritingReaders, Searchers, and Writers for you. You are proposing to
> coordinate them yourself. IndexAccess reopens Readers for you after a
> Writer has been used, and enforces Lucene requirements, like a
> WritingReader cannot be used at the same time as a Writer...etc.
>
> Technically, IndexAccessor could reopen the readers every 2
> seconds...and then you would see your changes...instead it only tries to
> reopen them if a change has been made to the index...and it does not
> want to get greedy if a Writer is batch loading, so it waits for you to
> release the Writer. You can control how often the 'view' is updated by
> releasing the Writer more often -- say every 50 docs. Write 50 docs,
> release, get, write 50 docs.
>
> - Mark
>
> ajay_garg wrote:
>> @Mark.
>>
>> I am sorry, but I need a bit more of explanation. So you mean to say ::
>>
>> "If auto-commit is false, then of course, docs will not be visible in the
>> index, until all the threads release themselves out of a particular
>> IndexWriter instance, and close() the IndexWriter instance.
>> If auto-commit is true, even then the above holds true. In particular,
>> let's
>> say iI need an application
>> with the following requirements ::
>>
>> a) There are multiple indexer threads indexing on a SINGLE indexwriter
>> instance with auto-commit true
>> b) Each thread 'flushes' according to a pre-defined criteria at some
>> point
>> of time.
>> c) The index should be updated immediately, that is, if any user re-opens
>> the IndexSearcher, then the
>> documents added till-that-snapshot-of-index must be visible. Note
>> that
>> the IndexWriter instance hasn't
>> been closed as yet, the indexer threads will be indexing till
>> eternity,
>> so that IndexWriter instance will
>> never be closed.
>>
>> So, you presume that building an application with the above requirements
>> is
>> impossible, even with auto-commit set to true. "
>>
>> ( If I sound ambiguous at any point, kindly forgive me for my lack of
>> language skills. I will try to explain better, if need arises ).
>>
>> Looking forward to a reply
>> Ajay Garg
>>
>> markrmiller wrote:
>>
>>> You are correct that autocommit=false means that docs will be in the
>>> index before the last thread releases its concurrent hold on a Writer,
>>> *but because IndexAccessor controls* *when the IndexSearchers are
>>> reopened*, those docs will still not be visible until the last thread
>>> holding a Writer releases it...that is when the reopening of Searchers
>>> occurs as well as when the Writer is closed.
>>>
>>> - Mark
>>>
>>> ajay_garg wrote:
>>>
>>>> Hi. Sorry if I seem a stranger in this thread, but there is something
>>>> that I
>>>> can't resist clearing myself on.
>>>>
>>>> Mark, you say that the additional documents added to a index, won't
>>>> show
>>>> up
>>>> until the # of threads accessing the index hits 0; and subsequently the
>>>> indexwriter instance is closed.
>>>>
>>>> But I suppose that the autocommit=true, asserts that all flushed
>>>> (Added)
>>>> documents are immediately committed ( and hence visible ) in the index,
>>>> and
>>>> no explicit cclosing ( releasiing ) of the Indexwriter instance is
>>>> required.
>>>> ( Of course, re-opening an IndexSearcher instance is required ).
>>>>
>>>> Am I being dumb ?
>>>>
>>>> Looking eagerly for you to shed some light on my doubt.
>>>>
>>>> Thanks
>>>> Ajay Garg
>>>>
>>>>
>>>> codetester wrote:
>>>>
>>>>
>>>>> Hi All,
>>>>>
>>>>> A newbie out here.... I am using lucene 2.3.0. I need to use lucene to
>>>>> perform live searching and indexing. To achieve that, I tried the
>>>>> following
>>>>>
>>>>> FSDirectory directory = FSDirectory.getDirectory(location);
>>>>> IndexReader reader = IndexReader.open(directory );
>>>>> IndexWriter writer = new IndexWriter(directory , new SimpleAnalyzer(),
>>>>> true); // <- I want to recreate the index every time
>>>>> IndexSearcher searcher = new IndexSearcher( reader );
>>>>>
>>>>> For Searching, I have the following code
>>>>> QueryParser queryParser = new QueryParser("xyz", new
>>>>> StandardAnalyzer());
>>>>> Hits hits = searcher .search(queryParser.parse(displayName + "*"));
>>>>>
>>>>> And for adding records, I have the following code
>>>>> // Create doc object
>>>>> writer.addDocument(doc);
>>>>>
>>>>> IndexReader newIndexReader = reader.reopen() ;
>>>>> if ( newIndexReader != reader ) {
>>>>> reader.close() ;
>>>>> }
>>>>> reader = newIndexReader ;
>>>>> searcher.close() ;
>>>>> searcher = new IndexSearcher(reader );
>>>>>
>>>>> So the issues that I face are
>>>>>
>>>>> 1) The addition of new record is not reflected in the search ( even
>>>>> though
>>>>> I have reinited IndexSearcher )
>>>>>
>>>>> 2) Obviously, the add record code is not thread safe. I am trying to
>>>>> close
>>>>> and update the reference to IndexSearcher object. I could add a sync
>>>>> block, but the bigger question would be that what is the ideal way to
>>>>> achieve this case where I need to add and search record real-time ?
>>>>>
>>>>> Thanks !
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: [EMAIL PROTECTED]
>>> For additional commands, e-mail: [EMAIL PROTECTED]
>>>
>>>
>>>
>>>
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
>
>
>
--
View this message in context:
http://www.nabble.com/Concurrent-Indexing-%2B-Searching-tp15234463p15288452.html
Sent from the Lucene - Java Users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
