Relevant results with DisMaxRequestHandler
Hello, I did notice several strange behaviors on queries. I would like to share with you an example, so maybe you can explain to me what is going wrong. Using the following query : http://localhost:8983/solr/others/select/?debugQuery=true&q=anna%20lewis&rows=20&start=0&fl=*&qt=dismax I get back around 100 results. Follow the two first : Person:151 Victoria Davisson Person:37 Anna Lewis And the related debugs : 57.998047 = (MATCH) sum of: 0.048290744 = (MATCH) sum of: 0.024546575 = (MATCH) max plus 0.01 times others of: 0.024546575 = (MATCH) weight(text:anna^0.5 in 64288), product of: 0.027395602 = queryWeight(text:anna^0.5), product of: 0.5 = boost 5.734427 = idf(docFreq=564, numDocs=30400) 0.009554783 = queryNorm 0.8960042 = (MATCH) fieldWeight(text:anna in 64288), product of: 1.0 = tf(termFreq(text:anna)=1) 5.734427 = idf(docFreq=564, numDocs=30400) 0.15625 = fieldNorm(field=text, doc=64288) 0.02374417 = (MATCH) max plus 0.01 times others of: 0.02374417 = (MATCH) weight(text:lewi^0.5 in 64288), product of: 0.026944114 = queryWeight(text:lewi^0.5), product of: 0.5 = boost 5.6399217 = idf(docFreq=620, numDocs=30400) 0.009554783 = queryNorm 0.88123775 = (MATCH) fieldWeight(text:lewi in 64288), product of: 1.0 = tf(termFreq(text:lewi)=1) 5.6399217 = idf(docFreq=620, numDocs=30400) 0.15625 = fieldNorm(field=text, doc=64288) 57.949757 = (MATCH) FunctionQuery(ord(name_s)), product of: 1213.0 = ord(name_s)=1213 5.0 = boost 0.009554783 = queryNorm 5.006892 = (MATCH) sum of: 0.038405567 = (MATCH) sum of: 0.021955125 = (MATCH) max plus 0.01 times others of: 0.021955125 = (MATCH) weight(text:anna^0.5 in 62632), product of: 0.027395602 = queryWeight(text:anna^0.5), product of: 0.5 = boost 5.734427 = idf(docFreq=564, numDocs=30400) 0.009554783 = queryNorm 0.80141056 = (MATCH) fieldWeight(text:anna in 62632), product of: 2.236068 = tf(termFreq(text:anna)=5) 5.734427 = idf(docFreq=564, numDocs=30400) 0.0625 = fieldNorm(field=text, doc=62632) 0.016450444 = (MATCH) max plus 0.01 times others of: 0.016450444 = (MATCH) weight(text:lewi^0.5 in 62632), product of: 0.026944114 = queryWeight(text:lewi^0.5), product of: 0.5 = boost 5.6399217 = idf(docFreq=620, numDocs=30400) 0.009554783 = queryNorm 0.61053944 = (MATCH) fieldWeight(text:lewi in 62632), product of: 1.7320508 = tf(termFreq(text:lewi)=3) 5.6399217 = idf(docFreq=620, numDocs=30400) 0.0625 = fieldNorm(field=text, doc=62632) 4.968487 = (MATCH) FunctionQuery(ord(name_s)), product of: 104.0 = ord(name_s)=104 5.0 = boost 0.009554783 = queryNorm I'm using a simple boost function : dismax explicit 0.01 text^0.5 name_s^5.0 name_s^5.0 name_s^5.0 Can anyone explain to me why the first result is on top (the query is 'anna lewis') with a huge weight and nothing related (it seems the weight come from the name_s field...) ? A second general question... is it possible to boost a field if the query match exactly the content of a field? Thank you ! Vincent -- View this message in context: http://www.nabble.com/Relevant-results-with-DisMaxRequestHandler-tp24716870p24716870.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Relevant results with DisMaxRequestHandler
Hello, Thank you for your answer, I finally used only a 'qf' parameter in the dismax requesthandler and it seems that I have now better and more relevant results. I just don't understand why a result is mainly boosted by his last update by default ! Vincent -- View this message in context: http://www.nabble.com/Relevant-results-with-DisMaxRequestHandler-tp24716870p24903143.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Relevant results with DisMaxRequestHandler
I actually have an other question... The 'qf' parameter used in the dismax seems to work with a 'AND' separator. I have much more results without dixmax. Is there any way to keep the same amount of document and process the 'qf' ? My dismax : dismax explicit 0.01 text^0.5 title_ac^4.0 name_ac^4.0 authors_list_sm^4.0 -- View this message in context: http://www.nabble.com/Relevant-results-with-DisMaxRequestHandler-tp24716870p24903219.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Relevant results with DisMaxRequestHandler
Wow, it's like the 'mm' parameters is just appeared for the first time... Yes, I read the doc few times, but never understood that the documents who doesn't match any of the expressions will not be return... my apologize everything seems more clear now thanks to the min number parameter. Thank you, Vincent hossman wrote: > > > : The 'qf' parameter used in the dismax seems to work with a 'AND' > separator. > : I have much more results without dixmax. Is there any way to keep the > same > : amount of document and process the 'qf' ? > > did you read any of the docs on dismax? > > http://wiki.apache.org/solr/DisMaxRequestHandler > > did you look at the "mm" param? > > http://wiki.apache.org/solr/DisMaxRequestHandler#mm > > > -Hoss > > > -- View this message in context: http://www.nabble.com/Relevant-results-with-DisMaxRequestHandler-tp24716870p25041314.html Sent from the Solr - User mailing list archive at Nabble.com.
Need help to finalize my autocomplete
Hello, I'm using the following code for my autocomplete feature : The field type : The field : The query : ?q=*:*&fq=query_ac:harry*&wt=json&rows=15&start=0&fl=*&indent=on&fq=model:SearchQuery It gives me a list of results I can parse and use with jQuery autocomplete plugin and all that works very well. Example of results : > harry > harry potter > the last fighting harry > harry potter 5 > comic relief harry potter What I would like to do now is only to have results starting with the query, so it should be : > harry > harry potter > harry potter 5 Can anybody tell me if it is possible and so how to do it ? Thank you ! Vincent And -- View this message in context: http://www.nabble.com/Need-help-to-finalize-my-autocomplete-tp25468885p25468885.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Need help to finalize my autocomplete
Hello, I tried to replace the class as you suggested, but I still get the same result (and not results where the query start only with the giving query). -- View this message in context: http://www.nabble.com/Need-help-to-finalize-my-autocomplete-tp25468885p25469239.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Need help to finalize my autocomplete
After re-indexing it works very well ! Thanks a lot ! Vincent -- View this message in context: http://www.nabble.com/Need-help-to-finalize-my-autocomplete-tp25468885p25469931.html Sent from the Solr - User mailing list archive at Nabble.com.
DataImportHandler with multiple values
Hello, I'm trying to import a simple book table with the full-import command. The datas are stored in mysql. It worked well when I tried to import few fields from the 'book' table : title, author, publisher etc. Now I would like to create a facet (multi valued field) with the categories which belong to the book. There is my sql request to get the list of categories from a book (009959241X for example, return 7 categories) : SELECT abn.name AS cat, ab.isbn AS isbn_temp FROM (amazon_books AS ab LEFT JOIN amazon_book_browse_nodes AS abbn ON ab.isbn = abbn.amazon_book_id) LEFT JOIN amazon_browse_nodes AS abn ON abbn.amazon_browse_node_id = abn.id WHERE ab.isbn = '009959241X' I tried to integrate it on my dataconfig : And my solr schema : And below standart solr 1.4 dynamics fields... Ten fields are well created... but without the 'cat' multi value field. Terry Pratchett 47 0552124753 Corgi Books 2009-04-13T12:54:38.553Z The Colour of Magic (Discworld Novel) I guess I missed something, could you help me or redirect me to the right doc? Thank you ! Vincent -- View this message in context: http://www.nabble.com/DataImportHandler-with-multiple-values-tp23022195p23022195.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: DataImportHandler with multiple values
I changed the ISBN to lowercase (and the other fields as well) and it works ! Thanks very much ! -- View this message in context: http://www.nabble.com/DataImportHandler-with-multiple-values-tp23022195p23023374.html Sent from the Solr - User mailing list archive at Nabble.com.
How to update only few fields in a document
Hello, I did just find only post about updating document, maybe things evolved since that time. I need to update a field in few thousand documents in one time (or multiple request), but I wouldn't like to have to add a new document instead of the current one (I mean it's how it works if I well understand). Example : curl http://localhost:8982/solr/update --data-binary '1522' -H 'Content-type:text/xml; charset=utf-8' This request will replace the current documents by two new one. The documents contains big text parts and I wouldn't have to send them every time. Is there any feature which could allow me to do that? Thanks ! Vincent -- View this message in context: http://www.nabble.com/How-to-update-only-few-fields-in-a-document-tp23560169p23560169.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: How to update only few fields in a document
Hello, I went through the ticket but it seems that nothing is envisaged until version 1.5 Thanks for your answer !! Vincent Otis Gospodnetic wrote: > > > Vincent, > > Unfortunately things haven't changed yet. If all your fields are stored, > have a look at SOLR-139. > > > Otis -- > Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch > > > > ----- Original Message >> From: Vincent Pérès >> To: solr-user@lucene.apache.org >> Sent: Friday, May 15, 2009 9:45:06 AM >> Subject: How to update only few fields in a document >> >> >> Hello, >> >> I did just find only post about updating document, maybe things evolved >> since that time. >> I need to update a field in few thousand documents in one time (or >> multiple >> request), but I wouldn't like to have to add a new document instead of >> the >> current one (I mean it's how it works if I well understand). >> >> Example : >> curl http://localhost:8982/solr/update --data-binary ' >> name="id">15 >> name="id">22' -H >> 'Content-type:text/xml; charset=utf-8' >> >> This request will replace the current documents by two new one. The >> documents contains big text parts and I wouldn't have to send them every >> time. >> >> Is there any feature which could allow me to do that? >> >> Thanks ! >> Vincent >> -- >> View this message in context: >> http://www.nabble.com/How-to-update-only-few-fields-in-a-document-tp23560169p23560169.html >> Sent from the Solr - User mailing list archive at Nabble.com. > > > -- View this message in context: http://www.nabble.com/How-to-update-only-few-fields-in-a-document-tp23560169p23595879.html Sent from the Solr - User mailing list archive at Nabble.com.
How to change the weight of the fields ?
Hello, I'm stuck my the boost feature... I'm doing the following query : http://localhost:8983/solr/select/?q=novel&bf=title_s^5.0&fl=title_s+isbn_s&version=2.2&start=0&rows=5&indent=on&debugQuery=on Below a part of my debug (the results are correct) : novel novel text:novel text:novel − − 0.6041825 = (MATCH) fieldWeight(text:novel in 395), product of: 1.7320508 = tf(termFreq(text:novel)=3) 3.1892564 = idf(docFreq=55, numDocs=500) 0.109375 = fieldNorm(field=text, doc=395) − 0.51787066 = (MATCH) fieldWeight(text:novel in 443), product of: 1.7320508 = tf(termFreq(text:novel)=3) 3.1892564 = idf(docFreq=55, numDocs=500) 0.09375 = fieldNorm(field=text, doc=443) It seems Solr is only evaluating the 'text' field and skip the 'title_s' field. The line in the schema for dynamic string is : Do I need to enable something else to be able to change the weight of my fields? Thanks very much Vincent -- View this message in context: http://www.nabble.com/How-to-change-the-weight-of-the-fields---tp23619971p23619971.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: How to change the weight of the fields ?
Hello, I'm sorry I wrote a mistake, I mean : http://localhost:8983/solr/listings/select/?q=novel&qf=title_s^5.0&fl=title_s+isbn_s&version=2.2&start=0&rows=5&indent=on&debugQuery=on (using qf (Query Fields)) But it seems I need to add dismax as well and configure it by default in solr config? Thanks for your answer ! Vincent -- View this message in context: http://www.nabble.com/How-to-change-the-weight-of-the-fields---tp23619971p23631022.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: How to change the weight of the fields ?
I tried the following request after changed the dismax : http://localhost:8983/solr/listings/select/?q=novel&qt=dismax&qf=title_s^2.0&fl=title_s+isbn_s&version=2.2&start=0&rows=5&indent=on&debugQuery=on But I don't get any results : novel novel +DisjunctionMaxQuery((title_s:novel^2.0)~0.01) () +(title_s:novel^2.0)~0.01 () DismaxQParser − − 3.0 − 1.0 − 1.0 etc Do I need to add some specific attributes to my string fields? It looks now like this : and below : Thanks !! Vincent -- View this message in context: http://www.nabble.com/How-to-change-the-weight-of-the-fields---tp23619971p23631247.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: How to change the weight of the fields ?
It seems I can only search on the field 'text'. With the following url : http://localhost:8983/solr/select/?q=novel&qt=dismax&fl=title_s,id&version=2.2&start=0&rows=10&indent=on&debugQuery=on I get answers, but on the debug area, it seems it's only searching on the 'text' field (with or without 'qt' the results are displayed within the same order) : novel novel − +DisjunctionMaxQuery((text:novel^0.5 | title_s:novel^5.0 | id:novel^10.0)~0.01) () − +(text:novel^0.5 | title_s:novel^5.0 | id:novel^10.0)~0.01 () − − 0.014641666 = (MATCH) sum of: 0.014641666 = (MATCH) max plus 0.01 times others of: 0.014641666 = (MATCH) weight(text:novel^0.5 in 114927), product of: 0.01362607 = queryWeight(text:novel^0.5), product of: 0.5 = boost 3.4734163 = idf(docFreq=10634, numDocs=43213) 0.007845918 = queryNorm 1.0745333 = (MATCH) fieldWeight(text:novel in 114927), product of: 1.4142135 = tf(termFreq(text:novel)=2) 3.4734163 = idf(docFreq=10634, numDocs=43213) 0.21875 = fieldNorm(field=text, doc=114927) etc. I should have a debug below with a search of the term into 'title_s' and 'id' no? Thanks for your answers ! Vincent -- View this message in context: http://www.nabble.com/How-to-change-the-weight-of-the-fields---tp23619971p23649624.html Sent from the Solr - User mailing list archive at Nabble.com.
User search in Facebook like
Hello, I built a feature which allow users to search for other user thanks to a dynamic text box. Like facebook, when you search for your friends, the name is display in a javascript dropdown list with a small picture. But I'm not completely happy with the search... I'm using a standard search like "?q=vincent" and I get back the results list. If I type 'vinc' I will not get any results (But I would like to display all the users where the name start with 'vinc'). Maybe I need an extra param? I also tried the autosuggest, but I get a list of terms and not direct results... Could you suggest me some solr feature which could help me to get better results? Thanks a lot ! Vincent -- View this message in context: http://www.nabble.com/User-search-in-Facebook-like-tp23804854p23804854.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: User search in Facebook like
Thanks very much, that's solve my problem ! Now I see another question : how can I manage the lower/upper cases in my search? Thanks ! Dietrich Featherston-2 wrote: > > try searching for matches where the name starts with whatever the user has > entered so far with a wildcard > > ?q=vinc* > > Are you always going to be searching for names? If so you could see if > the > user has entered two terms and suffix each with a wildcard to get > potentially more relevant searches. > > For example, if a user enters "vince p", you might substitute that with > the > query "vince* p*" to get the following hits > Vincent Pérès > Vincent Price > Vince Price > Vince Pérès > etc... > > D > > > > 2009/5/31 Vincent Pérès > >> >> Hello, >> >> I built a feature which allow users to search for other user thanks to a >> dynamic text box. >> Like facebook, when you search for your friends, the name is display in a >> javascript dropdown list with a small picture. >> But I'm not completely happy with the search... I'm using a standard >> search >> like "?q=vincent" and I get back the results list. If I type 'vinc' I >> will >> not get any results (But I would like to display all the users where the >> name start with 'vinc'). Maybe I need an extra param? >> I also tried the autosuggest, but I get a list of terms and not direct >> results... >> >> Could you suggest me some solr feature which could help me to get better >> results? >> >> Thanks a lot ! >> Vincent >> -- >> View this message in context: >> http://www.nabble.com/User-search-in-Facebook-like-tp23804854p23804854.html >> Sent from the Solr - User mailing list archive at Nabble.com. >> >> > > -- View this message in context: http://www.nabble.com/User-search-in-Facebook-like-tp23804854p23805261.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: User search in Facebook like
Thanks a lot for your answer it fixed all my issues !!! It's really well working ! Cheers, Vincent -- View this message in context: http://www.nabble.com/User-search-in-Facebook-like-tp23804854p23818867.html Sent from the Solr - User mailing list archive at Nabble.com.
Solr update performance decrease after a while
Hello, We are indexing approximately 500 documents per day. My benchmark says an update is done in 0.7 sec just after Solr has been started. But it quickly decrease to 2.2 secs per update ! I have just been focused on the Schema until now, and didn't changed many stuffs in the solrconfig file. Maybe you have some tips which could help me to be more linear ? Thanks a lot Vincent -- View this message in context: http://www.nabble.com/Solr-update-performance-decrease-after-a-while-tp23945947p23945947.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr update performance decrease after a while
Thank for your answer, I will have a look and try to find a solution. Cheers -- View this message in context: http://www.nabble.com/Solr-update-performance-decrease-after-a-while-tp23945947p23980681.html Sent from the Solr - User mailing list archive at Nabble.com.
Stats for all documents and not current search
Hello, I need to retrieve the stats of my index (using StatsComponent). It's not a problem when my query is empty, but the stats are update according the current search... and I need the stats of the whole index everytime. I'm currently doing two request (one with empty keyword to get the stats, one to get the results). Any idea which could save me one request? Thanks ! Vincent -- View this message in context: http://www.nabble.com/Stats-for-all-documents-and-not-current-search-tp24001883p24001883.html Sent from the Solr - User mailing list archive at Nabble.com.
Create custom facets after building index
Hello, I would like to create a custom facet for a 'rating_facets'. The rates are like that : 1, 1.2, 5, 5.78 etc. Is it possible to tell solr to create a 'custom' facet : [0 to 0.99] is 0 [ 1 to 1.99] is 1 etc. (and get them back into xml results with number of results by value) Or I have to specify this way on my schema.xml? Thank you ! Vincent -- View this message in context: http://www.nabble.com/Create-custom-facets-after-building-index-tp20086166p20086166.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Create custom facets after building index
Hello !
Thank you it is working. I've done a query, and my facet query is :
"facet_queries":{
"published_year_facet:[1999 TO 2005]":95,
"rating_facet:[3 TO 3.99]":25,
"rating_facet:[1 TO 1.99]":1},
Is it possible to 'group' kind of queries (published together, rating
together etc.) ? Or I have to match every query with regex ?
Thanks !
--
View this message in context:
http://www.nabble.com/Create-custom-facets-after-building-index-tp20086166p20091818.html
Sent from the Solr - User mailing list archive at Nabble.com.
How to get the min and max values from facets?
Hello, I'm using Solr 1.3. I would like to get only minimum and maximum values from a facet. In fact I'm using a range to get the results : [value TO value], and I don't need to get the facets list in my XML results (which could be more than hundred thousands)... so, I have to display the range (minimum and maximum values) from a facet. Is there any way to do that? I found the new statistics components, follow the link : http://wiki.apache.org/solr/StatsComponent But it's for solr 1.4. Does anyone have any idea? Thank you ! Vincent -- View this message in context: http://www.nabble.com/How-to-get-the-min-and-max-values-from-facets--tp20243462p20243462.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: How to get the min and max values from facets?
Hello, Yes I understand, like : myfacet, ASC, limit 1 myfacet, DESC, limit 1 So I can get the first value and the last one. Do you think I will get more performance with this way than using stats? Thanks ! Vincent -- View this message in context: http://www.nabble.com/How-to-get-the-min-and-max-values-from-facets--tp20243462p20252126.html Sent from the Solr - User mailing list archive at Nabble.com.
What are the way to update / delete solr datas?
Hello, I'm trying to find the best way to update / delete datas according to my project (developed with javascript and rails). I would like to do something like that : http://localhost:8983/solr/update/?q=id:1&rate=4 and http://localhost:8983/solr/delete/?q=id:1 Is it possible ? But I found only these two ways : http://localhost:8983/solr/update/csv?commit=true&separator=%09&escape=\&stream.file=/tmp/result.text or using an xml file with that : load_id:20070424150841 The last possibility is to use the solr-ruby library. Is there any way I forgot ? Thanks, Vincent -- View this message in context: http://www.nabble.com/What-are-the-way-to-update---delete-solr-datas--tp20268507p20268507.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: What are the way to update / delete solr datas?
Thanks for your quick answer. I'm using only HTTP to display my results, that's why I would like to continue with this way. If I can use HTTP instead of solr, it will be better for me. Erik Hatcher wrote: > > > On Oct 31, 2008, at 11:40 AM, Vincent Pérès wrote: >> The last possibility is to use the solr-ruby library. > > If you're using Ruby, that's what I'd use. Were your other proposals > to still do those calls from Ruby, but with the HTTP library directly? > > Erik > > > -- View this message in context: http://www.nabble.com/What-are-the-way-to-update---delete-solr-datas--tp20268507p20268773.html Sent from the Solr - User mailing list archive at Nabble.com.
