I noticed that if i do the get() before the maybeReopen then I get no
results. But otherwise I can change it further.
On Mon, Mar 2, 2009 at 11:46 AM, Michael McCandless <
luc...@mikemccandless.com> wrote:
>
> There is no such thing as final code -- code is alive and is always
> changing ;)
>
> It looks good to me.
>
> Though one trivial thing is: I would move the code in the try clause up to
> and including the multiSearcher=get() out above the try. I always attempt
> to "shrink wrap" what's inside a try clause to the minimum that needs to be
> there. Ie, your code that creates a query, finds the right sort & filter to
> use, etc, can all happen outside the try, because you have not yet acquired
> the multiSearcher.
>
> If you do that, you also don't need the null check in the finally clause,
> because multiSearcher must be non-null on entering the try.
>
> Mike
>
> Amin Mohammed-Coleman wrote:
>
> Hi there
>> Good morning! Here is the final search code:
>>
>> public Summary[] search(final SearchRequest searchRequest)
>> throwsSearchExecutionException {
>>
>> final String searchTerm = searchRequest.getSearchTerm();
>>
>> if (StringUtils.isBlank(searchTerm)) {
>>
>> throw new SearchExecutionException("Search string cannot be empty. There
>> will be too many results to process.");
>>
>> }
>>
>> List<Summary> summaryList = new ArrayList<Summary>();
>>
>> StopWatch stopWatch = new StopWatch("searchStopWatch");
>>
>> stopWatch.start();
>>
>> MultiSearcher multiSearcher = null;
>>
>> try {
>>
>> LOGGER.debug("Ensuring all index readers are up to date...");
>>
>> maybeReopen();
>>
>> Query query = queryParser.parse(searchTerm);
>>
>> LOGGER.debug("Search Term '" + searchTerm +"' ----> Lucene Query '" +
>> query.toString() +"'");
>>
>> Sort sort = null;
>>
>> sort = applySortIfApplicable(searchRequest);
>>
>> Filter[] filters =applyFiltersIfApplicable(searchRequest);
>>
>> ChainedFilter chainedFilter = null;
>>
>> if (filters != null) {
>>
>> chainedFilter = new ChainedFilter(filters, ChainedFilter.OR);
>>
>> }
>>
>> multiSearcher = get();
>>
>> TopDocs topDocs = multiSearcher.search(query,chainedFilter ,100,sort);
>>
>> ScoreDoc[] scoreDocs = topDocs.scoreDocs;
>>
>> LOGGER.debug("total number of hits for [" + query.toString() + " ] =
>> "+topDocs.
>> totalHits);
>>
>> for (ScoreDoc scoreDoc : scoreDocs) {
>>
>> final Document doc = multiSearcher.doc(scoreDoc.doc);
>>
>> float score = scoreDoc.score;
>>
>> final BaseDocument baseDocument = new BaseDocument(doc, score);
>>
>> Summary documentSummary = new DocumentSummaryImpl(baseDocument);
>>
>> summaryList.add(documentSummary);
>>
>> }
>>
>> } catch (Exception e) {
>>
>> throw new IllegalStateException(e);
>>
>> } finally {
>>
>> if (multiSearcher != null) {
>>
>> release(multiSearcher);
>>
>> }
>>
>> }
>>
>> stopWatch.stop();
>>
>> LOGGER.debug("total time taken for document seach: " +
>> stopWatch.getTotalTimeMillis() + " ms");
>>
>> return summaryList.toArray(new Summary[] {});
>>
>> }
>>
>>
>>
>> I hope this makes sense...thanks again!
>>
>>
>> Cheers
>>
>> Amin
>>
>>
>>
>> On Sun, Mar 1, 2009 at 8:09 PM, Michael McCandless <
>> luc...@mikemccandless.com> wrote:
>>
>>
>>> You're calling get() too many times. For every call to get() you must
>>> match with a call to release().
>>>
>>> So, once at the front of your search method you should:
>>>
>>> MultiSearcher searcher = get();
>>>
>>> then use that searcher to do searching, retrieve docs, etc.
>>>
>>> Then in the finally clause, pass that searcher to release.
>>>
>>> So, only one call to get() and one matching call to release().
>>>
>>> Mike
>>>
>>> Amin Mohammed-Coleman wrote:
>>>
>>> Hi
>>>
>>>> The searchers are injected into the class via Spring. So when a client
>>>> calls the class it is fully configured with a list of index searchers.
>>>> However I have removed this list and instead injecting a list of
>>>> directories which are passed to the DocumentSearchManager.
>>>> DocumentSearchManager is SearchManager (should've mentioned that
>>>> earlier).
>>>> So finally I have modified by release code to do the following:
>>>>
>>>> private void release(MultiSearcher multiSeacher) throws Exception {
>>>>
>>>> IndexSearcher[] indexSearchers = (IndexSearcher[])
>>>> multiSeacher.getSearchables();
>>>>
>>>> for(int i =0 ; i < indexSearchers.length;i++) {
>>>>
>>>> documentSearcherManagers[i].release(indexSearchers[i]);
>>>>
>>>> }
>>>>
>>>> }
>>>>
>>>>
>>>> and it's use looks like this:
>>>>
>>>>
>>>> public Summary[] search(final SearchRequest searchRequest)
>>>> throwsSearchExecutionException {
>>>>
>>>> final String searchTerm = searchRequest.getSearchTerm();
>>>>
>>>> if (StringUtils.isBlank(searchTerm)) {
>>>>
>>>> throw new SearchExecutionException("Search string cannot be empty. There
>>>> will be too many results to process.");
>>>>
>>>> }
>>>>
>>>> List<Summary> summaryList = new ArrayList<Summary>();
>>>>
>>>> StopWatch stopWatch = new StopWatch("searchStopWatch");
>>>>
>>>> stopWatch.start();
>>>>
>>>> List<IndexSearcher> indexSearchers = new ArrayList<IndexSearcher>();
>>>>
>>>> try {
>>>>
>>>> LOGGER.debug("Ensuring all index readers are up to date...");
>>>>
>>>> maybeReopen();
>>>>
>>>> LOGGER.debug("All Index Searchers are up to date. No of index searchers
>>>> '"
>>>> +
>>>> indexSearchers.size() +"'");
>>>>
>>>> Query query = queryParser.parse(searchTerm);
>>>>
>>>> LOGGER.debug("Search Term '" + searchTerm +"' ----> Lucene Query '" +
>>>> query.toString() +"'");
>>>>
>>>> Sort sort = null;
>>>>
>>>> sort = applySortIfApplicable(searchRequest);
>>>>
>>>> Filter[] filters =applyFiltersIfApplicable(searchRequest);
>>>>
>>>> ChainedFilter chainedFilter = null;
>>>>
>>>> if (filters != null) {
>>>>
>>>> chainedFilter = new ChainedFilter(filters, ChainedFilter.OR);
>>>>
>>>> }
>>>>
>>>> TopDocs topDocs = get().search(query,chainedFilter ,100,sort);
>>>>
>>>> ScoreDoc[] scoreDocs = topDocs.scoreDocs;
>>>>
>>>> LOGGER.debug("total number of hits for [" + query.toString() + " ] =
>>>> "+topDocs.
>>>> totalHits);
>>>>
>>>> for (ScoreDoc scoreDoc : scoreDocs) {
>>>>
>>>> final Document doc = get().doc(scoreDoc.doc);
>>>>
>>>> float score = scoreDoc.score;
>>>>
>>>> final BaseDocument baseDocument = new BaseDocument(doc, score);
>>>>
>>>> Summary documentSummary = new DocumentSummaryImpl(baseDocument);
>>>>
>>>> summaryList.add(documentSummary);
>>>>
>>>> }
>>>>
>>>> } catch (Exception e) {
>>>>
>>>> throw new IllegalStateException(e);
>>>>
>>>> } finally {
>>>>
>>>> release(get());
>>>>
>>>> }
>>>>
>>>> stopWatch.stop();
>>>>
>>>> LOGGER.debug("total time taken for document seach: " +
>>>> stopWatch.getTotalTimeMillis() + " ms");
>>>>
>>>> return summaryList.toArray(new Summary[] {});
>>>>
>>>> }
>>>>
>>>>
>>>> So the final post construct constructs the DocumentSearchMangers with
>>>> the
>>>> list of directories..looking like this
>>>>
>>>>
>>>> @PostConstruct
>>>>
>>>> public void initialiseDocumentSearcher() {
>>>>
>>>> PerFieldAnalyzerWrapper analyzerWrapper = new PerFieldAnalyzerWrapper(
>>>> analyzer);
>>>>
>>>> analyzerWrapper.addAnalyzer(FieldNameEnum.TYPE.getDescription(),
>>>> newKeywordAnalyzer());
>>>>
>>>> queryParser =
>>>> newMultiFieldQueryParser(FieldNameEnum.fieldNameDescriptions(),
>>>> analyzerWrapper);
>>>>
>>>> try {
>>>>
>>>> LOGGER.debug("Initialising multi searcher ....");
>>>>
>>>> documentSearcherManagers = new
>>>> DocumentSearcherManager[directories.size()];
>>>>
>>>> for (int i = 0; i < directories.size() ;i++) {
>>>>
>>>> Directory directory = directories.get(i);
>>>>
>>>> DocumentSearcherManager documentSearcherManager =
>>>> newDocumentSearcherManager(directory);
>>>>
>>>> documentSearcherManagers[i]=documentSearcherManager;
>>>>
>>>> }
>>>>
>>>> LOGGER.debug("multi searcher initialised");
>>>>
>>>> } catch (IOException e) {
>>>>
>>>> throw new IllegalStateException(e);
>>>>
>>>> }
>>>>
>>>> }
>>>>
>>>>
>>>>
>>>> Cheers
>>>>
>>>> Amin
>>>>
>>>>
>>>>
>>>> On Sun, Mar 1, 2009 at 6:15 PM, Michael McCandless <
>>>> luc...@mikemccandless.com> wrote:
>>>>
>>>>
>>>> I don't understand where searchers comes from, prior to
>>>>> initializeDocumentSearcher? You should, instead, simply create the
>>>>> SearcherManager (from your Directory instances). You don't need any
>>>>> searchers during initialize.
>>>>>
>>>>> Is DocumentSearcherManager the same as SearcherManager (just renamed)?
>>>>>
>>>>> The release method is wrong -- you're calling .get() and then
>>>>> immediately release. Instead, you should step through the searchers
>>>>> from your MultiSearcher and release them to each SearcherManager.
>>>>>
>>>>> You should call your release() in a finally clause.
>>>>>
>>>>> Mike
>>>>>
>>>>> Amin Mohammed-Coleman wrote:
>>>>>
>>>>> Sorry...i'm getting slightly confused.
>>>>>
>>>>> I have a PostConstruct which is where I should create an array of
>>>>>> SearchManagers (per indexSeacher). From there I initialise the
>>>>>> multisearcher using the get(). After which I need to call maybeReopen
>>>>>> for
>>>>>> each IndexSearcher. So I'll do the following:
>>>>>>
>>>>>> @PostConstruct
>>>>>>
>>>>>> public void initialiseDocumentSearcher() {
>>>>>>
>>>>>> PerFieldAnalyzerWrapper analyzerWrapper = new PerFieldAnalyzerWrapper(
>>>>>> analyzer);
>>>>>>
>>>>>> analyzerWrapper.addAnalyzer(FieldNameEnum.TYPE.getDescription(),
>>>>>> newKeywordAnalyzer());
>>>>>>
>>>>>> queryParser =
>>>>>> newMultiFieldQueryParser(FieldNameEnum.fieldNameDescriptions(),
>>>>>> analyzerWrapper);
>>>>>>
>>>>>> try {
>>>>>>
>>>>>> LOGGER.debug("Initialising multi searcher ....");
>>>>>>
>>>>>> documentSearcherManagers = new
>>>>>> DocumentSearcherManager[searchers.size()];
>>>>>>
>>>>>> for (int i = 0; i < searchers.size() ;i++) {
>>>>>>
>>>>>> IndexSearcher indexSearcher = searchers.get(i);
>>>>>>
>>>>>> Directory directory = indexSearcher.getIndexReader().directory();
>>>>>>
>>>>>> DocumentSearcherManager documentSearcherManager =
>>>>>> newDocumentSearcherManager(directory);
>>>>>>
>>>>>> documentSearcherManagers[i]=documentSearcherManager;
>>>>>>
>>>>>> }
>>>>>>
>>>>>> LOGGER.debug("multi searcher initialised");
>>>>>>
>>>>>> } catch (IOException e) {
>>>>>>
>>>>>> throw new IllegalStateException(e);
>>>>>>
>>>>>> }
>>>>>>
>>>>>> }
>>>>>>
>>>>>>
>>>>>> This initialises search managers. I then have methods:
>>>>>>
>>>>>>
>>>>>> private void maybeReopen() throws Exception {
>>>>>>
>>>>>> LOGGER.debug("Initiating reopening of index readers...");
>>>>>>
>>>>>> for (DocumentSearcherManager documentSearcherManager :
>>>>>> documentSearcherManagers) {
>>>>>>
>>>>>> documentSearcherManager.maybeReopen();
>>>>>>
>>>>>> }
>>>>>>
>>>>>> }
>>>>>>
>>>>>>
>>>>>>
>>>>>> private void release() throws Exception {
>>>>>>
>>>>>> for (DocumentSearcherManager documentSearcherManager :
>>>>>> documentSearcherManagers) {
>>>>>>
>>>>>> documentSearcherManager.release(documentSearcherManager.get());
>>>>>>
>>>>>> }
>>>>>>
>>>>>> }
>>>>>>
>>>>>>
>>>>>> private MultiSearcher get() {
>>>>>>
>>>>>> List<IndexSearcher> listOfIndexSeachers = new
>>>>>> ArrayList<IndexSearcher>();
>>>>>>
>>>>>> for (DocumentSearcherManager documentSearcherManager :
>>>>>> documentSearcherManagers) {
>>>>>>
>>>>>> listOfIndexSeachers.add(documentSearcherManager.get());
>>>>>>
>>>>>> }
>>>>>>
>>>>>> try {
>>>>>>
>>>>>> multiSearcher = new
>>>>>> MultiSearcher(listOfIndexSeachers.toArray(newIndexSearcher[] {}));
>>>>>>
>>>>>> } catch (IOException e) {
>>>>>>
>>>>>> throw new IllegalStateException(e);
>>>>>>
>>>>>> }
>>>>>>
>>>>>> return multiSearcher;
>>>>>>
>>>>>> }
>>>>>>
>>>>>>
>>>>>> These methods are used in the following manner in the search code:
>>>>>>
>>>>>>
>>>>>> public Summary[] search(final SearchRequest searchRequest)
>>>>>> throwsSearchExecutionException {
>>>>>>
>>>>>> final String searchTerm = searchRequest.getSearchTerm();
>>>>>>
>>>>>> if (StringUtils.isBlank(searchTerm)) {
>>>>>>
>>>>>> throw new SearchExecutionException("Search string cannot be empty.
>>>>>> There
>>>>>> will be too many results to process.");
>>>>>>
>>>>>> }
>>>>>>
>>>>>> List<Summary> summaryList = new ArrayList<Summary>();
>>>>>>
>>>>>> StopWatch stopWatch = new StopWatch("searchStopWatch");
>>>>>>
>>>>>> stopWatch.start();
>>>>>>
>>>>>> List<IndexSearcher> indexSearchers = new ArrayList<IndexSearcher>();
>>>>>>
>>>>>> try {
>>>>>>
>>>>>> LOGGER.debug("Ensuring all index readers are up to date...");
>>>>>>
>>>>>> maybeReopen();
>>>>>>
>>>>>> LOGGER.debug("All Index Searchers are up to date. No of index
>>>>>> searchers
>>>>>> '"
>>>>>> +
>>>>>> indexSearchers.size() +"'");
>>>>>>
>>>>>> Query query = queryParser.parse(searchTerm);
>>>>>>
>>>>>> LOGGER.debug("Search Term '" + searchTerm +"' ----> Lucene Query '" +
>>>>>> query.toString() +"'");
>>>>>>
>>>>>> Sort sort = null;
>>>>>>
>>>>>> sort = applySortIfApplicable(searchRequest);
>>>>>>
>>>>>> Filter[] filters =applyFiltersIfApplicable(searchRequest);
>>>>>>
>>>>>> ChainedFilter chainedFilter = null;
>>>>>>
>>>>>> if (filters != null) {
>>>>>>
>>>>>> chainedFilter = new ChainedFilter(filters, ChainedFilter.OR);
>>>>>>
>>>>>> }
>>>>>>
>>>>>> TopDocs topDocs = get().search(query,chainedFilter ,100,sort);
>>>>>>
>>>>>> ScoreDoc[] scoreDocs = topDocs.scoreDocs;
>>>>>>
>>>>>> LOGGER.debug("total number of hits for [" + query.toString() + " ] =
>>>>>> "+topDocs.
>>>>>> totalHits);
>>>>>>
>>>>>> for (ScoreDoc scoreDoc : scoreDocs) {
>>>>>>
>>>>>> final Document doc = get().doc(scoreDoc.doc);
>>>>>>
>>>>>> float score = scoreDoc.score;
>>>>>>
>>>>>> final BaseDocument baseDocument = new BaseDocument(doc, score);
>>>>>>
>>>>>> Summary documentSummary = new DocumentSummaryImpl(baseDocument);
>>>>>>
>>>>>> summaryList.add(documentSummary);
>>>>>>
>>>>>> }
>>>>>>
>>>>>> release();
>>>>>>
>>>>>> } catch (Exception e) {
>>>>>>
>>>>>> throw new IllegalStateException(e);
>>>>>>
>>>>>> }
>>>>>>
>>>>>> stopWatch.stop();
>>>>>>
>>>>>> LOGGER.debug("total time taken for document seach: " +
>>>>>> stopWatch.getTotalTimeMillis() + " ms");
>>>>>>
>>>>>> return summaryList.toArray(new Summary[] {});
>>>>>>
>>>>>> }
>>>>>>
>>>>>>
>>>>>> Does this look better? Again..I really really appreciate your help!
>>>>>>
>>>>>>
>>>>>> On Sun, Mar 1, 2009 at 4:18 PM, Michael McCandless <
>>>>>> luc...@mikemccandless.com> wrote:
>>>>>>
>>>>>>
>>>>>> This is not quite right -- you should only create SearcherManager once
>>>>>>
>>>>>>> (per Direcotry) at startup/app load, not with every search request.
>>>>>>>
>>>>>>> And I don't see release -- it must call SearcherManager.release of
>>>>>>> each of the IndexSearchers previously returned from get().
>>>>>>>
>>>>>>> Mike
>>>>>>>
>>>>>>> Amin Mohammed-Coleman wrote:
>>>>>>>
>>>>>>> Hi
>>>>>>>
>>>>>>> Thanks again for helping on a Sunday!
>>>>>>>
>>>>>>>>
>>>>>>>> I have now modified my maybeOpen() to do the following:
>>>>>>>>
>>>>>>>> private void maybeReopen() throws Exception {
>>>>>>>>
>>>>>>>> LOGGER.debug("Initiating reopening of index readers...");
>>>>>>>>
>>>>>>>> IndexSearcher[] indexSearchers = (IndexSearcher[]) multiSearcher
>>>>>>>> .getSearchables();
>>>>>>>>
>>>>>>>> for (IndexSearcher indexSearcher : indexSearchers) {
>>>>>>>>
>>>>>>>> IndexReader indexReader = indexSearcher.getIndexReader();
>>>>>>>>
>>>>>>>> SearcherManager documentSearcherManager = new
>>>>>>>> SearcherManager(indexReader.directory());
>>>>>>>>
>>>>>>>> documentSearcherManager.maybeReopen();
>>>>>>>>
>>>>>>>> }
>>>>>>>>
>>>>>>>> }
>>>>>>>>
>>>>>>>>
>>>>>>>> And get() to:
>>>>>>>>
>>>>>>>>
>>>>>>>> private synchronized MultiSearcher get() {
>>>>>>>>
>>>>>>>> IndexSearcher[] indexSearchers = (IndexSearcher[]) multiSearcher
>>>>>>>> .getSearchables();
>>>>>>>>
>>>>>>>> List<IndexSearcher> indexSearchersList = new
>>>>>>>> ArrayList<IndexSearcher>();
>>>>>>>>
>>>>>>>> for (IndexSearcher indexSearcher : indexSearchers) {
>>>>>>>>
>>>>>>>> IndexReader indexReader = indexSearcher.getIndexReader();
>>>>>>>>
>>>>>>>> SearcherManager documentSearcherManager = null;
>>>>>>>>
>>>>>>>> try {
>>>>>>>>
>>>>>>>> documentSearcherManager = new
>>>>>>>> SearcherManager(indexReader.directory());
>>>>>>>>
>>>>>>>> } catch (IOException e) {
>>>>>>>>
>>>>>>>> throw new IllegalStateException(e);
>>>>>>>>
>>>>>>>> }
>>>>>>>>
>>>>>>>> indexSearchersList.add(documentSearcherManager.get());
>>>>>>>>
>>>>>>>> }
>>>>>>>>
>>>>>>>> try {
>>>>>>>>
>>>>>>>> multiSearcher = new
>>>>>>>> MultiSearcher(indexSearchersList.toArray(newIndexSearcher[] {}));
>>>>>>>>
>>>>>>>> } catch (IOException e) {
>>>>>>>>
>>>>>>>> throw new IllegalStateException(e);
>>>>>>>>
>>>>>>>> }
>>>>>>>>
>>>>>>>> return multiSearcher;
>>>>>>>>
>>>>>>>> }
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> This makes all my test pass. I am using the SearchManager that you
>>>>>>>> recommended. Does this look ok?
>>>>>>>>
>>>>>>>>
>>>>>>>> On Sun, Mar 1, 2009 at 2:38 PM, Michael McCandless <
>>>>>>>> luc...@mikemccandless.com> wrote:
>>>>>>>>
>>>>>>>> Your maybeReopen has an excess incRef().
>>>>>>>>
>>>>>>>>
>>>>>>>> I'm not sure how you open the searchers in the first place? The
>>>>>>>>> list
>>>>>>>>> starts as empty, and nothing populates it?
>>>>>>>>>
>>>>>>>>> When you do the initial population, you need an incRef.
>>>>>>>>>
>>>>>>>>> I think you're hitting IllegalStateException because maybeReopen is
>>>>>>>>> closing a reader before get() can get it (since they synchronize on
>>>>>>>>> different objects).
>>>>>>>>>
>>>>>>>>> I'd recommend switching to the SearcherManager class. Instantiate
>>>>>>>>> one
>>>>>>>>> for each of your searchers. On each search request, go through
>>>>>>>>> them
>>>>>>>>> and call maybeReopen(), and then call get() and gather each
>>>>>>>>> IndexSearcher instance into a new array. Then, make a new
>>>>>>>>> MultiSearcher (opposite of what I said before): while that creates
>>>>>>>>> a
>>>>>>>>> small amount of garbage, it'll keep your code simpler (good
>>>>>>>>> tradeoff).
>>>>>>>>>
>>>>>>>>> Mike
>>>>>>>>>
>>>>>>>>> Amin Mohammed-Coleman wrote:
>>>>>>>>>
>>>>>>>>> sorrry I added
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> release(multiSearcher);
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> instead of multiSearcher.close();
>>>>>>>>>>
>>>>>>>>>> On Sun, Mar 1, 2009 at 2:17 PM, Amin Mohammed-Coleman <
>>>>>>>>>> ami...@gmail.com
>>>>>>>>>>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Hi
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I've now done the following:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> public Summary[] search(final SearchRequest searchRequest)
>>>>>>>>>>> throwsSearchExecutionException {
>>>>>>>>>>>
>>>>>>>>>>> final String searchTerm = searchRequest.getSearchTerm();
>>>>>>>>>>>
>>>>>>>>>>> if (StringUtils.isBlank(searchTerm)) {
>>>>>>>>>>>
>>>>>>>>>>> throw new SearchExecutionException("Search string cannot be
>>>>>>>>>>> empty.
>>>>>>>>>>> There
>>>>>>>>>>> will be too many results to process.");
>>>>>>>>>>>
>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>> List<Summary> summaryList = new ArrayList<Summary>();
>>>>>>>>>>>
>>>>>>>>>>> StopWatch stopWatch = new StopWatch("searchStopWatch");
>>>>>>>>>>>
>>>>>>>>>>> stopWatch.start();
>>>>>>>>>>>
>>>>>>>>>>> List<IndexSearcher> indexSearchers = new
>>>>>>>>>>> ArrayList<IndexSearcher>();
>>>>>>>>>>>
>>>>>>>>>>> try {
>>>>>>>>>>>
>>>>>>>>>>> LOGGER.debug("Ensuring all index readers are up to date...");
>>>>>>>>>>>
>>>>>>>>>>> maybeReopen();
>>>>>>>>>>>
>>>>>>>>>>> LOGGER.debug("All Index Searchers are up to date. No of index
>>>>>>>>>>> searchers
>>>>>>>>>>> '"+ indexSearchers.size() +
>>>>>>>>>>> "'");
>>>>>>>>>>>
>>>>>>>>>>> Query query = queryParser.parse(searchTerm);
>>>>>>>>>>>
>>>>>>>>>>> LOGGER.debug("Search Term '" + searchTerm +"' ----> Lucene Query
>>>>>>>>>>> '"
>>>>>>>>>>> +
>>>>>>>>>>> query.toString() +"'");
>>>>>>>>>>>
>>>>>>>>>>> Sort sort = null;
>>>>>>>>>>>
>>>>>>>>>>> sort = applySortIfApplicable(searchRequest);
>>>>>>>>>>>
>>>>>>>>>>> Filter[] filters =applyFiltersIfApplicable(searchRequest);
>>>>>>>>>>>
>>>>>>>>>>> ChainedFilter chainedFilter = null;
>>>>>>>>>>>
>>>>>>>>>>> if (filters != null) {
>>>>>>>>>>>
>>>>>>>>>>> chainedFilter = new ChainedFilter(filters, ChainedFilter.OR);
>>>>>>>>>>>
>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>> TopDocs topDocs = get().search(query,chainedFilter ,100,sort);
>>>>>>>>>>>
>>>>>>>>>>> ScoreDoc[] scoreDocs = topDocs.scoreDocs;
>>>>>>>>>>>
>>>>>>>>>>> LOGGER.debug("total number of hits for [" + query.toString() + "
>>>>>>>>>>> ]
>>>>>>>>>>> =
>>>>>>>>>>> "+topDocs.
>>>>>>>>>>> totalHits);
>>>>>>>>>>>
>>>>>>>>>>> for (ScoreDoc scoreDoc : scoreDocs) {
>>>>>>>>>>>
>>>>>>>>>>> final Document doc = multiSearcher.doc(scoreDoc.doc);
>>>>>>>>>>>
>>>>>>>>>>> float score = scoreDoc.score;
>>>>>>>>>>>
>>>>>>>>>>> final BaseDocument baseDocument = new BaseDocument(doc, score);
>>>>>>>>>>>
>>>>>>>>>>> Summary documentSummary = new DocumentSummaryImpl(baseDocument);
>>>>>>>>>>>
>>>>>>>>>>> summaryList.add(documentSummary);
>>>>>>>>>>>
>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>> multiSearcher.close();
>>>>>>>>>>>
>>>>>>>>>>> } catch (Exception e) {
>>>>>>>>>>>
>>>>>>>>>>> throw new IllegalStateException(e);
>>>>>>>>>>>
>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>> stopWatch.stop();
>>>>>>>>>>>
>>>>>>>>>>> LOGGER.debug("total time taken for document seach: " +
>>>>>>>>>>> stopWatch.getTotalTimeMillis() + " ms");
>>>>>>>>>>>
>>>>>>>>>>> return summaryList.toArray(new Summary[] {});
>>>>>>>>>>>
>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> And have the following methods:
>>>>>>>>>>>
>>>>>>>>>>> @PostConstruct
>>>>>>>>>>>
>>>>>>>>>>> public void initialiseQueryParser() {
>>>>>>>>>>>
>>>>>>>>>>> PerFieldAnalyzerWrapper analyzerWrapper = new
>>>>>>>>>>> PerFieldAnalyzerWrapper(
>>>>>>>>>>> analyzer);
>>>>>>>>>>>
>>>>>>>>>>> analyzerWrapper.addAnalyzer(FieldNameEnum.TYPE.getDescription(),
>>>>>>>>>>> newKeywordAnalyzer());
>>>>>>>>>>>
>>>>>>>>>>> queryParser =
>>>>>>>>>>> newMultiFieldQueryParser(FieldNameEnum.fieldNameDescriptions(),
>>>>>>>>>>>
>>>>>>>>>>> analyzerWrapper);
>>>>>>>>>>>
>>>>>>>>>>> try {
>>>>>>>>>>>
>>>>>>>>>>> LOGGER.debug("Initialising multi searcher ....");
>>>>>>>>>>>
>>>>>>>>>>> this.multiSearcher = new
>>>>>>>>>>> MultiSearcher(searchers.toArray(newIndexSearcher[] {}));
>>>>>>>>>>>
>>>>>>>>>>> LOGGER.debug("multi searcher initialised");
>>>>>>>>>>>
>>>>>>>>>>> } catch (IOException e) {
>>>>>>>>>>>
>>>>>>>>>>> throw new IllegalStateException(e);
>>>>>>>>>>>
>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Initialises mutltisearcher when this class is creared by spring.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> private synchronized void swapMultiSearcher(MultiSearcher
>>>>>>>>>>> newMultiSearcher) {
>>>>>>>>>>>
>>>>>>>>>>> try {
>>>>>>>>>>>
>>>>>>>>>>> release(multiSearcher);
>>>>>>>>>>>
>>>>>>>>>>> } catch (IOException e) {
>>>>>>>>>>>
>>>>>>>>>>> throw new IllegalStateException(e);
>>>>>>>>>>>
>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>> multiSearcher = newMultiSearcher;
>>>>>>>>>>>
>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>> public void maybeReopen() throws IOException {
>>>>>>>>>>>
>>>>>>>>>>> MultiSearcher newMultiSeacher = null;
>>>>>>>>>>>
>>>>>>>>>>> boolean refreshMultiSeacher = false;
>>>>>>>>>>>
>>>>>>>>>>> List<IndexSearcher> indexSearchers = new
>>>>>>>>>>> ArrayList<IndexSearcher>();
>>>>>>>>>>>
>>>>>>>>>>> synchronized (searchers) {
>>>>>>>>>>>
>>>>>>>>>>> for (IndexSearcher indexSearcher: searchers) {
>>>>>>>>>>>
>>>>>>>>>>> IndexReader reader = indexSearcher.getIndexReader();
>>>>>>>>>>>
>>>>>>>>>>> reader.incRef();
>>>>>>>>>>>
>>>>>>>>>>> Directory directory = reader.directory();
>>>>>>>>>>>
>>>>>>>>>>> long currentVersion = reader.getVersion();
>>>>>>>>>>>
>>>>>>>>>>> if (IndexReader.getCurrentVersion(directory) != currentVersion) {
>>>>>>>>>>>
>>>>>>>>>>> IndexReader newReader = indexSearcher.getIndexReader().reopen();
>>>>>>>>>>>
>>>>>>>>>>> if (newReader != reader) {
>>>>>>>>>>>
>>>>>>>>>>> reader.decRef();
>>>>>>>>>>>
>>>>>>>>>>> refreshMultiSeacher = true;
>>>>>>>>>>>
>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>> reader = newReader;
>>>>>>>>>>>
>>>>>>>>>>> IndexSearcher newSearcher = new IndexSearcher(newReader);
>>>>>>>>>>>
>>>>>>>>>>> indexSearchers.add(newSearcher);
>>>>>>>>>>>
>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> if (refreshMultiSeacher) {
>>>>>>>>>>>
>>>>>>>>>>> newMultiSeacher = new
>>>>>>>>>>> MultiSearcher(indexSearchers.toArray(newIndexSearcher[] {}));
>>>>>>>>>>>
>>>>>>>>>>> warm(newMultiSeacher);
>>>>>>>>>>>
>>>>>>>>>>> swapMultiSearcher(newMultiSeacher);
>>>>>>>>>>>
>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> private void warm(MultiSearcher newMultiSeacher) {
>>>>>>>>>>>
>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> private synchronized MultiSearcher get() {
>>>>>>>>>>>
>>>>>>>>>>> for (IndexSearcher indexSearcher: searchers) {
>>>>>>>>>>>
>>>>>>>>>>> indexSearcher.getIndexReader().incRef();
>>>>>>>>>>>
>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>> return multiSearcher;
>>>>>>>>>>>
>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>> private synchronized void release(MultiSearcher multiSearcher)
>>>>>>>>>>> throwsIOException {
>>>>>>>>>>>
>>>>>>>>>>> for (IndexSearcher indexSearcher: searchers) {
>>>>>>>>>>>
>>>>>>>>>>> indexSearcher.getIndexReader().decRef();
>>>>>>>>>>>
>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> However I am now getting
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> java.lang.IllegalStateException:
>>>>>>>>>>> org.apache.lucene.store.AlreadyClosedException: this IndexReader
>>>>>>>>>>> is
>>>>>>>>>>> closed
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> on the call:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> private synchronized MultiSearcher get() {
>>>>>>>>>>>
>>>>>>>>>>> for (IndexSearcher indexSearcher: searchers) {
>>>>>>>>>>>
>>>>>>>>>>> indexSearcher.getIndexReader().incRef();
>>>>>>>>>>>
>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>> return multiSearcher;
>>>>>>>>>>>
>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> I'm doing something wrong ..obviously..not sure where though..
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Cheers
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Sun, Mar 1, 2009 at 1:36 PM, Michael McCandless <
>>>>>>>>>>> luc...@mikemccandless.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> I was wondering the same thing ;)
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> It's best to call this method from a single BG "warming" thread,
>>>>>>>>>>>> in
>>>>>>>>>>>> which
>>>>>>>>>>>> case it would not need its own synchronization.
>>>>>>>>>>>>
>>>>>>>>>>>> But, to be safe, I'll add internal synchronization to it. You
>>>>>>>>>>>> can't
>>>>>>>>>>>> simply put synchronized in front of the method, since you don't
>>>>>>>>>>>> want
>>>>>>>>>>>> this to
>>>>>>>>>>>> block searching.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Mike
>>>>>>>>>>>>
>>>>>>>>>>>> Amin Mohammed-Coleman wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> just a quick point:
>>>>>>>>>>>>
>>>>>>>>>>>> public void maybeReopen() throws IOException {
>>>>>>>>>>>> //D
>>>>>>>>>>>>
>>>>>>>>>>>> long currentVersion =
>>>>>>>>>>>>
>>>>>>>>>>>>> currentSearcher.getIndexReader().getVersion();
>>>>>>>>>>>>> if (IndexReader.getCurrentVersion(dir) != currentVersion) {
>>>>>>>>>>>>> IndexReader newReader =
>>>>>>>>>>>>> currentSearcher.getIndexReader().reopen();
>>>>>>>>>>>>> assert newReader != currentSearcher.getIndexReader();
>>>>>>>>>>>>> IndexSearcher newSearcher = new IndexSearcher(newReader);
>>>>>>>>>>>>> warm(newSearcher);
>>>>>>>>>>>>> swapSearcher(newSearcher);
>>>>>>>>>>>>> }
>>>>>>>>>>>>> }
>>>>>>>>>>>>>
>>>>>>>>>>>>> should the above be synchronised?
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Sun, Mar 1, 2009 at 1:25 PM, Amin Mohammed-Coleman <
>>>>>>>>>>>>> ami...@gmail.com
>>>>>>>>>>>>>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>> thanks. i will rewrite..in between giving my baby her feed
>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> playing
>>>>>>>>>>>>>
>>>>>>>>>>>>> with the other child and my wife who wants me to do several
>>>>>>>>>>>>> other
>>>>>>>>>>>>>
>>>>>>>>>>>>> things!
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Sun, Mar 1, 2009 at 1:20 PM, Michael McCandless <
>>>>>>>>>>>>>> luc...@mikemccandless.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Amin Mohammed-Coleman wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks for your input. I would like to have a go at doing
>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> myself
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> first, Solr may be an option.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> * You are creating a new Analyzer & QueryParser every time,
>>>>>>>>>>>>>>>> also
>>>>>>>>>>>>>>>> creating unnecessary garbage; instead, they should be
>>>>>>>>>>>>>>>> created
>>>>>>>>>>>>>>>> once
>>>>>>>>>>>>>>>> & reused.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> -- I can moved the code out so that it is only created once
>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>> reused.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> * You always make a new IndexSearcher and a new
>>>>>>>>>>>>>>>> MultiSearcher
>>>>>>>>>>>>>>>> even
>>>>>>>>>>>>>>>> when nothing has changed. This just generates unnecessary
>>>>>>>>>>>>>>>> garbage
>>>>>>>>>>>>>>>> which GC then must sweep up.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> -- This was something I thought about. I could move it out
>>>>>>>>>>>>>>>> so
>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>> it's
>>>>>>>>>>>>>>>> created once. However I presume inside my code i need to
>>>>>>>>>>>>>>>> check
>>>>>>>>>>>>>>>> whether
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>> indexreaders are update to date. This needs to be
>>>>>>>>>>>>>>>> synchronized
>>>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>> well I
>>>>>>>>>>>>>>>> guess(?)
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Yes you should synchronize the check for whether the
>>>>>>>>>>>>>>>> IndexReader
>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> current.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> * I don't see any synchronization -- it looks like two search
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> requests are allowed into this method at the same time?
>>>>>>>>>>>>>>> Which
>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> dangerous... eg both (or, more) will wastefully reopen the
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> readers.
>>>>>>>>>>>>>>>> -- So i need to extract the logic for reopening and provide
>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>> synchronisation mechanism.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Yes.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Ok. So I have some work to do. I'll refactor the code and
>>>>>>>>>>>>>>> see
>>>>>>>>>>>>>>> if
>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> get
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> inline to your recommendations.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Sun, Mar 1, 2009 at 12:11 PM, Michael McCandless <
>>>>>>>>>>>>>>>> luc...@mikemccandless.com> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On a quick look, I think there are a few problems with the
>>>>>>>>>>>>>>>> code:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> * I don't see any synchronization -- it looks like two
>>>>>>>>>>>>>>>> search
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> requests are allowed into this method at the same time?
>>>>>>>>>>>>>>>>> Which
>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>> dangerous... eg both (or, more) will wastefully reopen the
>>>>>>>>>>>>>>>>> readers.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> * You are over-incRef'ing (the reader.incRef inside the
>>>>>>>>>>>>>>>>> loop)
>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>> don't see a corresponding decRef.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> * You reopen and warm your searchers "live" (vs with BG
>>>>>>>>>>>>>>>>> thread);
>>>>>>>>>>>>>>>>> meaning the unlucky search request that hits a reopen pays
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> cost. This might be OK if the index is small enough that
>>>>>>>>>>>>>>>>> reopening & warming takes very little time. But if index
>>>>>>>>>>>>>>>>> gets
>>>>>>>>>>>>>>>>> large, making a random search pay that warming cost is not
>>>>>>>>>>>>>>>>> nice
>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>> the end user. It erodes their trust in you.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> * You always make a new IndexSearcher and a new
>>>>>>>>>>>>>>>>> MultiSearcher
>>>>>>>>>>>>>>>>> even
>>>>>>>>>>>>>>>>> when nothing has changed. This just generates unnecessary
>>>>>>>>>>>>>>>>> garbage
>>>>>>>>>>>>>>>>> which GC then must sweep up.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> * You are creating a new Analyzer & QueryParser every time,
>>>>>>>>>>>>>>>>> also
>>>>>>>>>>>>>>>>> creating unnecessary garbage; instead, they should be
>>>>>>>>>>>>>>>>> created
>>>>>>>>>>>>>>>>> once
>>>>>>>>>>>>>>>>> & reused.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> You should consider simply using Solr -- it handles all
>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>> logic
>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>> you and has been well debugged with time...
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Mike
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Amin Mohammed-Coleman wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> The reason for the indexreader.reopen is because I have a
>>>>>>>>>>>>>>>>> webapp
>>>>>>>>>>>>>>>>> which
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> enables users to upload files and then search for the
>>>>>>>>>>>>>>>>> documents.
>>>>>>>>>>>>>>>>> If
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> don't
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> reopen i'm concerned that the facet hit counter won't be
>>>>>>>>>>>>>>>>>> updated.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Tue, Feb 24, 2009 at 8:32 PM, Amin Mohammed-Coleman <
>>>>>>>>>>>>>>>>>> ami...@gmail.com
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Hi
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I have been able to get the code working for my scenario,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> however
>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> question and I was wondering if I could get some help. I
>>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>> list
>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>> IndexSearchers which are used in a MultiSearcher class.
>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>> use
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> indexsearchers to get each indexreader and put them into
>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>> MultiIndexReader.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> IndexReader[] readers = new
>>>>>>>>>>>>>>>>>>> IndexReader[searchables.length];
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> for (int i =0 ; i < searchables.length;i++) {
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> IndexSearcher indexSearcher =
>>>>>>>>>>>>>>>>>>> (IndexSearcher)searchables[i];
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> readers[i] = indexSearcher.getIndexReader();
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> IndexReader newReader = readers[i].reopen();
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> if (newReader != readers[i]) {
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> readers[i].close();
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> readers[i] = newReader;
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> multiReader = new MultiReader(readers);
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> OpenBitSetFacetHitCounter facetHitCounter =
>>>>>>>>>>>>>>>>>>> newOpenBitSetFacetHitCounter();
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> IndexSearcher indexSearcher = new
>>>>>>>>>>>>>>>>>>> IndexSearcher(multiReader);
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I then use the indexseacher to do the facet stuff. I end
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> code
>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>> closing the multireader. This is causing problems in
>>>>>>>>>>>>>>>>>>> another
>>>>>>>>>>>>>>>>>>> method
>>>>>>>>>>>>>>>>>>> where I
>>>>>>>>>>>>>>>>>>> do some other search as the indexreaders are closed. Is
>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>> ok
>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>> close
>>>>>>>>>>>>>>>>>>> the multiindexreader or should I do some additional
>>>>>>>>>>>>>>>>>>> checks
>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> other
>>>>>>>>>>>>>>>>>>> method to see if the indexreader is closed?
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Cheers
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> P.S. Hope that made sense...!
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Mon, Feb 23, 2009 at 7:20 AM, Amin Mohammed-Coleman <
>>>>>>>>>>>>>>>>>>> ami...@gmail.com
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Hi
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Thanks just what I needed!
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Cheers
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Amin
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On 22 Feb 2009, at 16:11, Marcelo Ochoa <
>>>>>>>>>>>>>>>>>>>> marcelo.oc...@gmail.com>
>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Hi Amin:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Please take a look a this blog post:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> http://sujitpal.blogspot.com/2007/04/lucene-search-within-search-with.html
>>>>>>>>>>>>>>>>>>>>> Best regards, Marcelo.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> On Sun, Feb 22, 2009 at 1:18 PM, Amin Mohammed-Coleman
>>>>>>>>>>>>>>>>>>>>> <
>>>>>>>>>>>>>>>>>>>>> ami...@gmail.com>
>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Hi
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Sorry to re send this email but I was wondering if I
>>>>>>>>>>>>>>>>>>>>> could
>>>>>>>>>>>>>>>>>>>>> get
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> some
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> advice
>>>>>>>>>>>>>>>>>>>>>> on this.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Cheers
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Amin
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> On 16 Feb 2009, at 20:37, Amin Mohammed-Coleman <
>>>>>>>>>>>>>>>>>>>>>> ami...@gmail.com>
>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Hi
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> I am looking at building a faceted search using
>>>>>>>>>>>>>>>>>>>>>> Lucene.
>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>> know
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Solr
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> comes with this built in, however I would like to try
>>>>>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>>>>> myself
>>>>>>>>>>>>>>>>>>>>>>> (something to add to my CV!). I have been looking
>>>>>>>>>>>>>>>>>>>>>>> around
>>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>> found
>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>> you can use the IndexReader and use TermVectors.
>>>>>>>>>>>>>>>>>>>>>>> This
>>>>>>>>>>>>>>>>>>>>>>> looks
>>>>>>>>>>>>>>>>>>>>>>> ok
>>>>>>>>>>>>>>>>>>>>>>> but
>>>>>>>>>>>>>>>>>>>>>>> I'm
>>>>>>>>>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>>>>>>> sure how to filter the results so that a particular
>>>>>>>>>>>>>>>>>>>>>>> user
>>>>>>>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>>>>>> only
>>>>>>>>>>>>>>>>>>>>>>> see
>>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>>>> subset of results. The next option I was looking at
>>>>>>>>>>>>>>>>>>>>>>> was
>>>>>>>>>>>>>>>>>>>>>>> something
>>>>>>>>>>>>>>>>>>>>>>> like
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Term term1 = new Term("brand", "ford");
>>>>>>>>>>>>>>>>>>>>>>> Term term2 = new Term("brand", "vw");
>>>>>>>>>>>>>>>>>>>>>>> Term[] termsArray = new Term[] { term1, term2 };un
>>>>>>>>>>>>>>>>>>>>>>> int[] docFreqs = indexSearcher.docFreqs(termsArray);
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> The only problem here is that I have to provide the
>>>>>>>>>>>>>>>>>>>>>>> brand
>>>>>>>>>>>>>>>>>>>>>>> type
>>>>>>>>>>>>>>>>>>>>>>> each
>>>>>>>>>>>>>>>>>>>>>>> time a
>>>>>>>>>>>>>>>>>>>>>>> new brand is created. Again I'm not sure how I can
>>>>>>>>>>>>>>>>>>>>>>> filter
>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>> results
>>>>>>>>>>>>>>>>>>>>>>> here.
>>>>>>>>>>>>>>>>>>>>>>> It may be that I'm using the wrong api methods to do
>>>>>>>>>>>>>>>>>>>>>>> this.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> I would be grateful if I could get some advice on
>>>>>>>>>>>>>>>>>>>>>>> this.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Cheers
>>>>>>>>>>>>>>>>>>>>>>> Amin
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> P.S. I am basically trying to do something that
>>>>>>>>>>>>>>>>>>>>>>> displays
>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>> following
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Personal Contact (23) Business Contact (45) and so
>>>>>>>>>>>>>>>>>>>>>>> on..
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Marcelo F. Ochoa
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> http://marceloochoa.blogspot.com/
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> http://marcelo.ochoa.googlepages.com/home
>>>>>>>>>>>>>>>>>>>>> ______________
>>>>>>>>>>>>>>>>>>>>> Want to integrate Lucene and Oracle?
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> http://marceloochoa.blogspot.com/2007/09/running-lucene-inside-your-oracle-jvm.html
>>>>>>>>>>>>>>>>>>>>> Is Oracle 11g REST ready?
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> http://marceloochoa.blogspot.com/2008/02/is-oracle-11g-rest-ready.html
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>>>>>>>>>>>>>> To unsubscribe, e-mail:
>>>>>>>>>>>>>>>>>>>>> java-user-unsubscr...@lucene.apache.org
>>>>>>>>>>>>>>>>>>>>> For additional commands, e-mail:
>>>>>>>>>>>>>>>>>>>>> java-user-h...@lucene.apache.org
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> To unsubscribe, e-mail:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> java-user-unsubscr...@lucene.apache.org
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> For additional commands, e-mail:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> java-user-h...@lucene.apache.org
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> To unsubscribe, e-mail:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> java-user-unsubscr...@lucene.apache.org
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> For additional commands, e-mail:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> java-user-h...@lucene.apache.org
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> To unsubscribe, e-mail:
>>>>>>>>>>>>>>
>>>>>>>>>>>>> java-user-unsubscr...@lucene.apache.org
>>>>>>>>>>>>>
>>>>>>>>>>>>> For additional commands, e-mail:
>>>>>>>>>>>> java-user-h...@lucene.apache.org
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
>>>>>>>>>>
>>>>>>>>> For additional commands, e-mail: java-user-h...@lucene.apache.org
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>>
>>>>>>>> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
>>>>>>> For additional commands, e-mail: java-user-h...@lucene.apache.org
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
>>>>> For additional commands, e-mail: java-user-h...@lucene.apache.org
>>>>>
>>>>>
>>>>>
>>>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
>>> For additional commands, e-mail: java-user-h...@lucene.apache.org
>>>
>>>
>>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
> For additional commands, e-mail: java-user-h...@lucene.apache.org
>
>
