Re: One complex wildcard query lead solr OOM

2016-01-24 Thread Jian Mou
Hi Jack,

Thanks! Do you know how to disable wildcards, What I want is if input is
wildcards, just treat it as a normal char. I other words,
I just want to disable wildcard search.

Thanks,
Jian

On Fri, Jan 22, 2016 at 1:55 PM, Jack Krupansky 
wrote:

> The Lucene WildcardQuery class does have an additional constructor that has
> a maxDeterminizedStates parameter to limit the size of the FSM generated by
> a wildcard queery, and the QueryParserBase class does have a method to set
> that parameter, setMaxDeterminizedStates, but there is no Solr support for
> invoking that method.
>
> It is probably worth a Jira to get such support. Even then, the question is
> how Solr should respond to the exception that gets thrown when that limit
> is reached.
>
> Even if Solr had an option to disable complex wildcards, the question is
> what you want to happen when a complex wildcard is used - should an
> exception be thrown, or... what?
>
> I suppose it might be simplest to have a Solr option to limit the number of
> wildcard characters used in a term, like to 4 or 8 or something like that.
> IOW, have Solr check the term before the WildcardQuery is generated.
>
> -- Jack Krupansky
>
> On Thu, Jan 21, 2016 at 8:18 PM, Jian Mou  wrote:
>
> > We are using Solr as our search engine, and recently notice some user
> > input wildcard query can lead to Solr dead loop in
> >
> > org.apache.lucene.util.automaton.Operations.determinize()
> >
> > , and it also eats memory and finally OOM.
> >
> > the wildcard query seems like **?-???o·???è??**。
> >
> > Although we can validate the input parameter, but I also wonder is there
> > any configuration which can disable complex wildcard query like this
> which
> > lead to serve performance problems.
> >
> >
> > Related statcktrace
> >
> >
> > [image: Inline image 1]
> >
> >
> >
> > Thanks,
> >
> > Jian
> >
>


One complex wildcard query lead solr OOM

2016-01-21 Thread Jian Mou
We are using Solr as our search engine, and recently notice some user input
wildcard query can lead to Solr dead loop in

org.apache.lucene.util.automaton.Operations.determinize()

, and it also eats memory and finally OOM.

the wildcard query seems like **?-???o·???è??**。

Although we can validate the input parameter, but I also wonder is there
any configuration which can disable complex wildcard query like this which
lead to serve performance problems.


Related statcktrace


[image: Inline image 1]



Thanks,

Jian