Re: Dismax Question
It turns out that Solr 3.5.0 does not have the dismax issue, so, we have reverted. Hopefully, the bug will be fixed.
Re: Dismax Question
From: Joel Rosen >To: solr-user@lucene.apache.org; Steve Fatula >Cc: Ahmet Arslan ; Tom Burton-West >Sent: Monday, July 2, 2012 10:31 AM >Subject: Re: Dismax Question > >I and another user recently posted about this exact same issue. It sounds >like maybe this is a new bug introduced in 3.6: > >http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201206.mbox/%3CCAMKKMTx_ybPqsbgU5NtQ19t%2B0kWdAHtq-CZTZxfYxdu6rS1u1g%40mail.gmail.com%3E > >http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201206.mbox/%3CCAMySt%2BE6Hr6%3DgOkkDeZU9PCTpgJ4Mb1i8YrzfAndfqUzdot8xw%40mail.gmail.com%3E > >Does anyone happen to know if 3.5 (we went from 3.4 to 3.6) happens to have >the problem? If not, we'd probably revert since we can't deal with the >millions if extra search results that should not be there.
Re: Dismax Question
From: Joel Rosen To: solr-user@lucene.apache.org; Steve Fatula >Cc: Ahmet Arslan ; Tom Burton-West >Sent: Monday, July 2, 2012 10:31 AM >Subject: Re: Dismax Question > > >I and another user recently posted about this exact same issue. It sounds >like maybe this is a new bug introduced in 3.6: > > >http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201206.mbox/%3CCAMKKMTx_ybPqsbgU5NtQ19t%2B0kWdAHtq-CZTZxfYxdu6rS1u1g%40mail.gmail.com%3E > > >http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201206.mbox/%3CCAMySt%2BE6Hr6%3DgOkkDeZU9PCTpgJ4Mb1i8YrzfAndfqUzdot8xw%40mail.gmail.com%3E > >That sounds like the same thing, I've noticed searching for SEP-100 actually >does a SEP or 100, even though it's supposed to be AND. Has a bug report been filed? I've managed to figure out a fix that is working well enough for my own application right now. I set autoGeneratePhraseQueries to "true" on my field, and also set qs=2. The high query slop value simulates the AND behavior that I want since my documents are relatively short, but this is obviously not the correct solution, and I don't know if there are any performance issues with using really high query slop values. > >Ok, so, I need to figure out autoGeneratePhraseQueries I guess. The way I >understand it, if I search for WORD1 WORD2, it will only find "WORD1 WORD2", >is that correct? That would be bad if so since I'd really want WORD1 AND >WORD2, but not the phrase. Trying to find some good doc for this feature!
Re: Dismax Question
I and another user recently posted about this exact same issue. It sounds like maybe this is a new bug introduced in 3.6: http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201206.mbox/%3CCAMKKMTx_ybPqsbgU5NtQ19t%2B0kWdAHtq-CZTZxfYxdu6rS1u1g%40mail.gmail.com%3E http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201206.mbox/%3CCAMySt%2BE6Hr6%3DgOkkDeZU9PCTpgJ4Mb1i8YrzfAndfqUzdot8xw%40mail.gmail.com%3E I've managed to figure out a fix that is working well enough for my own application right now. I set autoGeneratePhraseQueries to "true" on my field, and also set qs=2. The high query slop value simulates the AND behavior that I want since my documents are relatively short, but this is obviously not the correct solution, and I don't know if there are any performance issues with using really high query slop values. On Mon, Jul 2, 2012 at 9:16 AM, Steve Fatula wrote: > > >From: Ahmet Arslan > >To: solr-user@lucene.apache.org; Steve Fatula > >Sent: Monday, July 2, 2012 6:22 AM > >Subject: Re: Dismax Question > > > >> So, my question is how do we get Solr search to work with > >> AND when it is splitting words? The splitting part is good, > >> the bad part is that it is searching for any one of those > >> split words. > > > >Setting autoGeneratePhraseQueries="true" and &mm=100% might help you. > > > > autoGeneratePhraseQueries="true"> > > > >I set mm to 100%, no effect at all. It works only for words typed in that > are separated already. Remember, the example here is: > > > > > >DualHead2Go finds all kinds of matches (it splits into dual head 2 go) > > > > > >Dial Head 2 Go finds the correct matches, indicating it is adding them > based on q/op, defautOperator, and mm.
Re: Dismax Question
>From: Ahmet Arslan >To: solr-user@lucene.apache.org; Steve Fatula >Sent: Monday, July 2, 2012 6:22 AM >Subject: Re: Dismax Question > >> So, my question is how do we get Solr search to work with >> AND when it is splitting words? The splitting part is good, >> the bad part is that it is searching for any one of those >> split words. > >Setting autoGeneratePhraseQueries="true" and &mm=100% might help you. > > > >I set mm to 100%, no effect at all. It works only for words typed in that are >separated already. Remember, the example here is: > > >DualHead2Go finds all kinds of matches (it splits into dual head 2 go) > > >Dial Head 2 Go finds the correct matches, indicating it is adding them based >on q/op, defautOperator, and mm.
Re: Dismax Question
>From: Vadim Kisselmann >To: solr-user@lucene.apache.org; Steve Fatula >Sent: Monday, July 2, 2012 4:31 AM >Subject: Re: Dismax Question > >in your schema.xml you can set the default query parser operator, in >your case , but it's >deprecated. > > >I do set the default query operator, as shown by using separate words in my >example, it correct ands them. The different is when using one words, SOLR >splits it into 3 words, and does not and. I don't understand why it does not >and them when solr splits the words, but does when solr does not split them. > > >I've specified mm as 100% as well with no impact. > >This used to work on Solr 3.4.
Re: Dismax Question
in your schema.xml you can set the default query parser operator, in your case , but it's deprecated. When you use the edismax, read this:http://drupal.org/node/1559394 . mm-param is here the answer. Best regards Vadim 2012/7/2 Steve Fatula : > Let's say a user types in: > > DualHead2Go > > > The way solr is working, it splits this into: > > Dual Head 2 Go > > And searches the index for various fields, and finds records where any ONE of > them matches. > > Now, if I simply type the search terms Dual Head 2 Go, it finds records where > ALL of them match. This is because we set q.op to AND. > > Recently, we went from Solr 3.4 to 3.6, and, 3.4 used to work ok, 3.6 seems o > behave differently, or, perhaps we mucked something up. > > So, my question is how do we get Solr search to work with AND when it is > splitting words? The splitting part is good, the bad part is that it is > searching for any one of those split words. > > Steve
Re: Dismax Question
> So, my question is how do we get Solr search to work with > AND when it is splitting words? The splitting part is good, > the bad part is that it is searching for any one of those > split words. Setting autoGeneratePhraseQueries="true" and &mm=100% might help you. http://wiki.apache.org/solr/DisMaxQParserPlugin#mm_.28Minimum_.27Should.27_Match.29
Dismax Question
Let's say a user types in: DualHead2Go The way solr is working, it splits this into: Dual Head 2 Go And searches the index for various fields, and finds records where any ONE of them matches. Now, if I simply type the search terms Dual Head 2 Go, it finds records where ALL of them match. This is because we set q.op to AND. Recently, we went from Solr 3.4 to 3.6, and, 3.4 used to work ok, 3.6 seems o behave differently, or, perhaps we mucked something up. So, my question is how do we get Solr search to work with AND when it is splitting words? The splitting part is good, the bad part is that it is searching for any one of those split words. Steve
Re: Date boosting with dismax question
Also, those boosts on your qf and pf are a red flag and may be causing you issues. Look at explains provided with debugQuery=true output to see how your field/phrase boosts are working in conjunction with your date boosting attempts. Erik On Oct 23, 2011, at 17:15 , Erick Erickson wrote: > Define "not working". Show what you're getting and what you > expect to find. Show your data. Note that the example given > boosts on quite coarse dates, it *tends* to make documents > published in a particular *year* score higher. > > You might review: > http://wiki.apache.org/solr/UsingMailingLists > > Best > Erick > > On Sun, Oct 23, 2011 at 11:08 PM, Craig Stadler > wrote: >> Yes I have and I cannot get it to work. Perhaps something is out of version >> for my setup? >> I tried for 3 hours to get ever example I could find to work. >> >> - Original Message - From: "Erick Erickson" >> >> To: >> Sent: Sunday, October 23, 2011 5:07 PM >> Subject: Re: Date boosting with dismax question >> >> >> Have you seen this? >> >> http://wiki.apache.org/solr/SolrRelevancyFAQ#How_can_I_boost_the_score_of_newer_documents >> >> Best >> Erick >> >> >> On Sat, Oct 22, 2011 at 3:26 AM, Craig Stadler >> wrote: >>> >>> Solr Specification Version: 1.4.0 >>> Solr Implementation Version: 1.4.0 833479 - grantingersoll - 2009-11-06 >>> 12:33:40 >>> Lucene Specification Version: 2.9.1 >>> Lucene Implementation Version: 2.9.1 832363 - 2009-11-03 04:37:25 >>> >>> >> precisionStep="6" positionIncrementGap="0"/> >>> >>> >> stored="false" omitNorms="true" required="false" >>> omitTermFreqAndPositions="true" /> >>> >>> I am using 'created' as the name of the date field. >>> >>> My dates are being populated as such : >>> 1980-01-01T00:00:00Z >>> >>> Search handler (solrconfig) : >>> >>> >>> >>> dismax >>> explicit >>> 0.1 >>> name0^2 other ^1 >>> name0^2 other ^1 >>> 3 >>> 3 >>> *:* >>> >>> >>> >>> -- >>> >>> Query : >>> >>> /solr/ftf/dismax/?q=libya >>> &debugQuery=off >>> &hl=true >>> &start= >>> &rows=10 >>> -- >>> >>> I am trying to factor in created to the SCORE. (boost) I have tried a >>> million ways to do this, no success. I know the dates are populating >>> correctly because I can sort by them. Can anyone help me implement date >>> boosting with dismax under this scenario??? >>> >>> -Craig >>> >> >>
Re: Date boosting with dismax question
Define "not working". Show what you're getting and what you expect to find. Show your data. Note that the example given boosts on quite coarse dates, it *tends* to make documents published in a particular *year* score higher. You might review: http://wiki.apache.org/solr/UsingMailingLists Best Erick On Sun, Oct 23, 2011 at 11:08 PM, Craig Stadler wrote: > Yes I have and I cannot get it to work. Perhaps something is out of version > for my setup? > I tried for 3 hours to get ever example I could find to work. > > - Original Message - From: "Erick Erickson" > > To: > Sent: Sunday, October 23, 2011 5:07 PM > Subject: Re: Date boosting with dismax question > > > Have you seen this? > > http://wiki.apache.org/solr/SolrRelevancyFAQ#How_can_I_boost_the_score_of_newer_documents > > Best > Erick > > > On Sat, Oct 22, 2011 at 3:26 AM, Craig Stadler > wrote: >> >> Solr Specification Version: 1.4.0 >> Solr Implementation Version: 1.4.0 833479 - grantingersoll - 2009-11-06 >> 12:33:40 >> Lucene Specification Version: 2.9.1 >> Lucene Implementation Version: 2.9.1 832363 - 2009-11-03 04:37:25 >> >> > precisionStep="6" positionIncrementGap="0"/> >> >> > stored="false" omitNorms="true" required="false" >> omitTermFreqAndPositions="true" /> >> >> I am using 'created' as the name of the date field. >> >> My dates are being populated as such : >> 1980-01-01T00:00:00Z >> >> Search handler (solrconfig) : >> >> >> >> dismax >> explicit >> 0.1 >> name0^2 other ^1 >> name0^2 other ^1 >> 3 >> 3 >> *:* >> >> >> >> -- >> >> Query : >> >> /solr/ftf/dismax/?q=libya >> &debugQuery=off >> &hl=true >> &start= >> &rows=10 >> -- >> >> I am trying to factor in created to the SCORE. (boost) I have tried a >> million ways to do this, no success. I know the dates are populating >> correctly because I can sort by them. Can anyone help me implement date >> boosting with dismax under this scenario??? >> >> -Craig >> > >
Re: Date boosting with dismax question
Yes I have and I cannot get it to work. Perhaps something is out of version for my setup? I tried for 3 hours to get ever example I could find to work. - Original Message - From: "Erick Erickson" To: Sent: Sunday, October 23, 2011 5:07 PM Subject: Re: Date boosting with dismax question Have you seen this? http://wiki.apache.org/solr/SolrRelevancyFAQ#How_can_I_boost_the_score_of_newer_documents Best Erick On Sat, Oct 22, 2011 at 3:26 AM, Craig Stadler wrote: Solr Specification Version: 1.4.0 Solr Implementation Version: 1.4.0 833479 - grantingersoll - 2009-11-06 12:33:40 Lucene Specification Version: 2.9.1 Lucene Implementation Version: 2.9.1 832363 - 2009-11-03 04:37:25 I am using 'created' as the name of the date field. My dates are being populated as such : 1980-01-01T00:00:00Z Search handler (solrconfig) : dismax explicit 0.1 name0^2 other ^1 name0^2 other ^1 3 3 *:* -- Query : /solr/ftf/dismax/?q=libya &debugQuery=off &hl=true &start= &rows=10 -- I am trying to factor in created to the SCORE. (boost) I have tried a million ways to do this, no success. I know the dates are populating correctly because I can sort by them. Can anyone help me implement date boosting with dismax under this scenario??? -Craig
Re: Date boosting with dismax question
Have you seen this? http://wiki.apache.org/solr/SolrRelevancyFAQ#How_can_I_boost_the_score_of_newer_documents Best Erick On Sat, Oct 22, 2011 at 3:26 AM, Craig Stadler wrote: > Solr Specification Version: 1.4.0 > Solr Implementation Version: 1.4.0 833479 - grantingersoll - 2009-11-06 > 12:33:40 > Lucene Specification Version: 2.9.1 > Lucene Implementation Version: 2.9.1 832363 - 2009-11-03 04:37:25 > > precisionStep="6" positionIncrementGap="0"/> > > stored="false" omitNorms="true" required="false" > omitTermFreqAndPositions="true" /> > > I am using 'created' as the name of the date field. > > My dates are being populated as such : > 1980-01-01T00:00:00Z > > Search handler (solrconfig) : > > > > dismax > explicit > 0.1 > name0^2 other ^1 > name0^2 other ^1 > 3 > 3 > *:* > > > > -- > > Query : > > /solr/ftf/dismax/?q=libya > &debugQuery=off > &hl=true > &start= > &rows=10 > -- > > I am trying to factor in created to the SCORE. (boost) I have tried a > million ways to do this, no success. I know the dates are populating > correctly because I can sort by them. Can anyone help me implement date > boosting with dismax under this scenario??? > > -Craig >
Date boosting with dismax question
Solr Specification Version: 1.4.0 Solr Implementation Version: 1.4.0 833479 - grantingersoll - 2009-11-06 12:33:40 Lucene Specification Version: 2.9.1 Lucene Implementation Version: 2.9.1 832363 - 2009-11-03 04:37:25 precisionStep="6" positionIncrementGap="0"/> stored="false" omitNorms="true" required="false" omitTermFreqAndPositions="true" /> I am using 'created' as the name of the date field. My dates are being populated as such : 1980-01-01T00:00:00Z Search handler (solrconfig) : dismax explicit 0.1 name0^2 other ^1 name0^2 other ^1 3 3 *:* -- Query : /solr/ftf/dismax/?q=libya &debugQuery=off &hl=true &start= &rows=10 -- I am trying to factor in created to the SCORE. (boost) I have tried a million ways to do this, no success. I know the dates are populating correctly because I can sort by them. Can anyone help me implement date boosting with dismax under this scenario??? -Craig
Re: Dismax question
: > Deja-Vu...
:
: >
http://www.lucidimagination.com/search/document/3551f130b6772799/excluding_docs_from_results_based_on_matched_field
:
: > -Hoss
:
: Thanks for the answer, the problem is that the query like this:
:
: q=foo&defType=dismax&qf=title&bq={!dismax qf='title desc' v=$q}
:
: causes exception in Solr - "infinite recursion detected".
Hmmm, yeah ... that makes sense given the way local params work (although
i'm amazed Solr actually detects it instead of just crashing -- the local
param parsing is smarter then i thought!)
Assuming this is what i think it is, it's because the "inner" dismax query
is looking for a "bq" param, which is leading to infinite recursion
because there is no bq local param -- if you specify bf='' inside the
{!dismax} that error should go away (but you may still not get the results
you expect if you also have other top level dismax params like pf, etc...
http://www.lucidimagination.com/search/document/a60e420f5c3d8365/searching_for_terms_on_specific_fields#a60e420f5c3d8365
You should be able to bypass the kind of thing entirely by making *both*
of the dismax queries nested queries of some other top level query (so
they can't possibly inherit params from eachother) instead of using bq ...
qq = foo
q = +_query_:"$main" _query_:"$extra"
main = {!dismax ... v=$qq}
extra = {!dismax ... v=$qq}
-Hoss
Re: Dismax question
Hello!
> : I've got a problem and maybe someone had a similar one ;) I want to
> : 'force' dismax to make a query like the following one:
> : +title:foo^100 desc:foo
> :
> : The name and desc fields are only an exmaple, there can be a multiple
> : fields that lay under the name of 'title' and 'desc'.
> :
> : What I try to achieve is get the documents that have a match in both
> : the title and desc fields or only in the title and exclude the documents
> : that only have a match in the desc field.
> Deja-Vu...
> http://www.lucidimagination.com/search/document/3551f130b6772799/excluding_docs_from_results_based_on_matched_field
> -Hoss
Thanks for the answer, the problem is that the query like this:
q=foo&defType=dismax&qf=title&bq={!dismax qf='title desc' v=$q}
causes exception in Solr - "infinite recursion detected".
--
Regards,
Rafał Kuć
http://solr.pl
Re: Dismax question
: I've got a problem and maybe someone had a similar one ;) I want to : 'force' dismax to make a query like the following one: : +title:foo^100 desc:foo : : The name and desc fields are only an exmaple, there can be a multiple : fields that lay under the name of 'title' and 'desc'. : : What I try to achieve is get the documents that have a match in both : the title and desc fields or only in the title and exclude the documents : that only have a match in the desc field. Deja-Vu... http://www.lucidimagination.com/search/document/3551f130b6772799/excluding_docs_from_results_based_on_matched_field -Hoss
Dismax question
Hello! I've got a problem and maybe someone had a similar one ;) I want to 'force' dismax to make a query like the following one: +title:foo^100 desc:foo The name and desc fields are only an exmaple, there can be a multiple fields that lay under the name of 'title' and 'desc'. What I try to achieve is get the documents that have a match in both the title and desc fields or only in the title and exclude the documents that only have a match in the desc field. Is this possible with dismax ? -- Regards, Rafał Kuć http://solr.pl
Re: DisMax Question.
: but no results for: : qt=dismax&q= blabla : ^1.5&version=2.2&start=0&rows=1&indent=on&fl=*,score&debugQuery=true&sort=length_seconds+desc : : doesn't DisMax support term boosting? the dismax syntax is specifically very limited to match the common assumptions end users have about search input ... it understands + and - and "quotes" but none of hte other lucene query syntax ... boost info can be specified per field, or using the "bq" param. the DisMax wiki links tohte DisMax javadocs where all of hte request params are listed in detail ... there are also good examples of what types of values can be specifie for each param in the defaults and invarients for hte various DisMaxRequestHandler instances in the example solrconfig.xml -Hoss
Re: DisMax Question.
Thanks, that did the trick but now i have another problem (the documentation is very little). It fails when i try to boost terms in the query, i.e. I get results for : qt=dismax&q=blabla&version=2.2&start=0&rows=1&indent=on&fl=*,score&debugQuery=true&sort=length_seconds+desc but no results for: qt=dismax&q= blabla ^1.5&version=2.2&start=0&rows=1&indent=on&fl=*,score&debugQuery=true&sort=length_seconds+desc doesn't DisMax support term boosting? What is the alternative to DisMax then? something like this : q=field1:bla^0.1 fo OR field2:blabla^3;score+desc&version=2.2&start=0&rows=170&indent=on&fl=*,score&debugQuery=on ? On 3/14/07, Chris Hostetter <[EMAIL PROTECTED]> wrote: : When i run my query with qt=dismax i don't get results: : : /solr/select/?qt=dismax&q=blablabla%3BRelevanc : +desc&version=2.2&start=0&rows=1&indent=on&fl=*,score : : when i remove the qt=dismax i do get result back: : : /solr/select/?q=blablabla%3BRelevanc : +desc&version=2.2&start=0&rows=1&indent=on&fl=*,score dismax does not use hte ";" syntax for sorting .. this is the one usefull piece of documetnation i ever manged to put in the wiki about dismax... http://wiki.apache.org/solr/DisMaxRequestHandler ...if you add debugQuery=true to your URL it will give you a bunch of great debugging info thta would have pointed out you were actually geting a query for the terms "blablabla;Relevanc" and "desc" across all of those fields. ..also is "Relevanc" the name of a field you have, because if you are trying to sort by score that's not right for either handler ... you need "score desc", either after the ";" for standard, or in the "sort" param for dismax. -Hoss
Re: DisMax Question.
: When i run my query with qt=dismax i don't get results: : : /solr/select/?qt=dismax&q=blablabla%3BRelevanc : +desc&version=2.2&start=0&rows=1&indent=on&fl=*,score : : when i remove the qt=dismax i do get result back: : : /solr/select/?q=blablabla%3BRelevanc : +desc&version=2.2&start=0&rows=1&indent=on&fl=*,score dismax does not use hte ";" syntax for sorting .. this is the one usefull piece of documetnation i ever manged to put in the wiki about dismax... http://wiki.apache.org/solr/DisMaxRequestHandler ...if you add debugQuery=true to your URL it will give you a bunch of great debugging info thta would have pointed out you were actually geting a query for the terms "blablabla;Relevanc" and "desc" across all of those fields. ..also is "Relevanc" the name of a field you have, because if you are trying to sort by score that's not right for either handler ... you need "score desc", either after the ";" for standard, or in the "sort" param for dismax. -Hoss
DisMax Question.
Hi, I am trying to use DisMax handler in order to search multiple fields but i don't get results back. Assume the fields i want to search on are "abc", "def", "ghi" and "jkl" then i changed solrconfig.xml to this: explicit 0.01 abc^2 def ghi^0.1 jkl When i run my query with qt=dismax i don't get results: /solr/select/?qt=dismax&q=blablabla%3BRelevanc +desc&version=2.2&start=0&rows=1&indent=on&fl=*,score when i remove the qt=dismax i do get result back: /solr/select/?q=blablabla%3BRelevanc +desc&version=2.2&start=0&rows=1&indent=on&fl=*,score What am i doing wrong ? Thanks, S
