Hello,
One of our Lucene indexes has started misbehaving on indexWriter.close and I'm
searching for ideas about what may have happened and how to fix it.
Here's our scenario:
- We have seven Lucene indexes that contain different sets of data from a web
application are indexed for searching by
grade.
>
> The answers above is relevant for 3x, but most of them are also relevant for
> 2.9. If you have an older version of Lucene, then some of the solutions
> might still apply (such as close(false)).
>
> Hope this helps,
> Shai
>
> On Tue, Nov 2, 2010 at 12:55 AM, Mark
point and am thinking I may have to rebuild the index,
though I would definitely prefer to avoid doing that and would like to know why
this is happening.
Thanks for your help,
Mark
On Nov 2, 2010, at 9:26 AM, Mark Kristensson wrote:
>
> Wonderful information on what happens during indexW
t;
> On Wed, Nov 3, 2010 at 2:53 PM, Mark Kristensson
> wrote:
>> I've successfully reproduced the issue in our lab with a copy from
>> production and have broken the close() call into parts, as suggested, with
>> one addition.
>>
>> Previously, the call
unique fields names in an index or segment? Any
suggestions for potentially mitigating the problem?
Thanks,
Mark
On Nov 3, 2010, at 2:02 PM, Michael McCandless wrote:
> On Wed, Nov 3, 2010 at 4:27 PM, Mark Kristensson
> wrote:
>>
>> I've run checkIndex against the in
this'd prevent big segments from being produced.
> Disabling norms should also workaround this, though that will affect
> hit scores...
>
> Mike
>
> On Wed, Nov 3, 2010 at 7:37 PM, Mark Kristensson
> wrote:
>> Yes, we do have a large number of unique field names in
Reader? (Eg, run java with
> -agentlib:hprof=cpu=samples,depth=16,interval=1).
>
> Yes, both Index.NOT_ANALYZED_NO_NORMS and Index.NO will disable norms
> as long as no document in the index ever had norms on (yes it does
> "infect" heh).
>
> Mike
>
> On Fri,
uch a huge number of
>> unique field names is not an area of Lucene that's very well tested...
>> perhaps there's something silly somewhere. Maybe you can try
>> profiling just the init of your IndexReader? (Eg, run java with
>> -agentlib:hprof=cpu=samples,dept
ike
>>>>
>>>> On Wed, Nov 3, 2010 at 7:37 PM, Mark Kristensson
>>>> wrote:
>>>>> Yes, we do have a large number of unique field names in that index,
>>>>> because they are driven by user named fields in our application (with
lly somewhere. Maybe you can try
>> profiling just the init of your IndexReader? (Eg, run java with
>> -agentlib:hprof=cpu=samples,depth=16,interval=1).
>>
>> Yes, both Index.NOT_ANALYZED_NO_NORMS and Index.NO will disable norms
>> as long as no document in the inde
that be instead of monkeying with StringHelper or in addition to it?
Thanks,
Mark
On Nov 20, 2010, at 5:44 AM, Yonik Seeley wrote:
> On Fri, Nov 19, 2010 at 5:41 PM, Mark Kristensson
> wrote:
>> Here's the changes I made to org.apache.lucene.util.StringHelper:
>>
>> //
I've had to deal with exactly this same scenario and, from what I know,
there is no equivalent in Lucene to a SQL != kind of query. So, you have to
have some way to select "everything" (whatever that means in your scenario)
and then remove the stuff that you don't want (the NOT portion). For my
app
While implementing a solution for keeping warmed indexReaders around for our
various indexes (so users don't have to wait while we open an indexReader for
our one slow index), I've run into some serious problems trying to track down
all of the outstanding file handles to my indexes.
For a while
eld names might cause a problem opening an index reader. However,
I'm not sure why closing (committing changes to) an index writer would have
such a problem. Why is that?
Thank you!
Mark
On Tue, Nov 23, 2010 at 2:22 PM, Mark Kristensson <
mark.kristens...@smartsheet.com> wrote:
> I&
14 matches
Mail list logo
