Re: field "title_ngram" was indexed without position data; cannot run PhraseQuery
If you consider what n-grams do this should make sense to you. Consider the following piece of data: White iPod If the field is fed through a bigram filter (n-gram with size of 2) the resulting token stream would appear as such: wh hi it te ip po od The usual use of n-grams is to match those partial tokens, essentially giving you a great deal of power in creating non-wildcard partial matches. How you use this is up to your imagination, but one easy use is in partial matches in autosuggest features. I can't speak for the intent behind the way it's coded, but it makes a great deal of sense to me that positional data would be seen as unnecessary since the intent of n-grams typically doesn't collide with phrase searches. If you need both behaviors it's far better to use copyField and have one field dedicated to standard tokenization and token filters, and another field for n-grams. I hope that's useful to you. On Oct 15, 2013, at 6:14 AM, MC wrote: > Hello, > > Could someone explain (or perhaps provide a documentation link) what does the > following error mean: > "field "title_ngram" was indexed without position data; cannot run > PhraseQuery" > > I'll do some more searching online, I was just wondering if anyone has > encountered this error before, and what the possible solution might be. I've > recently upgraded my version of solr from 3.6.0 to 4.5.0, I'm not sure if > this has any bearing or not. > Thanks, > > M >
Re: field "title_ngram" was indexed without position data; cannot run PhraseQuery
Show us the field and field type from your schema. Likely you are "omitting" position info for the field, and the field type has "autoGeneratePhraseQueries="true"" - the ngram analyzer generates a sequence of terms for a single source term and then the query parser generates a PhraseQuery for that sequence, but that requires position info in the index but you have omitted them. That's one theory. So, if that theory is correct, either retain position info by getting rid of the "omit", or remove the autoGeneratePhraseQueries. -- Jack Krupansky -Original Message- From: Jason Hellman Sent: Tuesday, October 15, 2013 11:19 AM To: solr-user@lucene.apache.org Subject: Re: field "title_ngram" was indexed without position data; cannot run PhraseQuery If you consider what n-grams do this should make sense to you. Consider the following piece of data: White iPod If the field is fed through a bigram filter (n-gram with size of 2) the resulting token stream would appear as such: wh hi it te ip po od The usual use of n-grams is to match those partial tokens, essentially giving you a great deal of power in creating non-wildcard partial matches. How you use this is up to your imagination, but one easy use is in partial matches in autosuggest features. I can't speak for the intent behind the way it's coded, but it makes a great deal of sense to me that positional data would be seen as unnecessary since the intent of n-grams typically doesn't collide with phrase searches. If you need both behaviors it's far better to use copyField and have one field dedicated to standard tokenization and token filters, and another field for n-grams. I hope that's useful to you. On Oct 15, 2013, at 6:14 AM, MC wrote: Hello, Could someone explain (or perhaps provide a documentation link) what does the following error mean: "field "title_ngram" was indexed without position data; cannot run PhraseQuery" I'll do some more searching online, I was just wondering if anyone has encountered this error before, and what the possible solution might be. I've recently upgraded my version of solr from 3.6.0 to 4.5.0, I'm not sure if this has any bearing or not. Thanks, M
Re: field "title_ngram" was indexed without position data; cannot run PhraseQuery
Hello, Thank you all for your help. There was indeed a property which was not set right in schema.xml: omitTermFreqAndPositions="true" After changing it to false phrase lookup started working OK. Thanks, M On 10/15/13 12:01 PM, Jack Krupansky wrote: Show us the field and field type from your schema. Likely you are "omitting" position info for the field, and the field type has "autoGeneratePhraseQueries="true"" - the ngram analyzer generates a sequence of terms for a single source term and then the query parser generates a PhraseQuery for that sequence, but that requires position info in the index but you have omitted them. That's one theory. So, if that theory is correct, either retain position info by getting rid of the "omit", or remove the autoGeneratePhraseQueries. -- Jack Krupansky -Original Message- From: Jason Hellman Sent: Tuesday, October 15, 2013 11:19 AM To: solr-user@lucene.apache.org Subject: Re: field "title_ngram" was indexed without position data; cannot run PhraseQuery If you consider what n-grams do this should make sense to you. Consider the following piece of data: White iPod If the field is fed through a bigram filter (n-gram with size of 2) the resulting token stream would appear as such: wh hi it te ip po od The usual use of n-grams is to match those partial tokens, essentially giving you a great deal of power in creating non-wildcard partial matches. How you use this is up to your imagination, but one easy use is in partial matches in autosuggest features. I can't speak for the intent behind the way it's coded, but it makes a great deal of sense to me that positional data would be seen as unnecessary since the intent of n-grams typically doesn't collide with phrase searches. If you need both behaviors it's far better to use copyField and have one field dedicated to standard tokenization and token filters, and another field for n-grams. I hope that's useful to you. On Oct 15, 2013, at 6:14 AM, MC wrote: Hello, Could someone explain (or perhaps provide a documentation link) what does the following error mean: "field "title_ngram" was indexed without position data; cannot run PhraseQuery" I'll do some more searching online, I was just wondering if anyone has encountered this error before, and what the possible solution might be. I've recently upgraded my version of solr from 3.6.0 to 4.5.0, I'm not sure if this has any bearing or not. Thanks, M
