Re: Norms - scoring issue
It seems that fieldNorm difference is coming from the field named 'text'. And you didn't include the definition of text field. Did you omit norms for that field too? By the way I see that you have store=true in some places but it should be store*d*=true. --- On Wed, 9/14/11, Adolfo Castro Menna adolfo.castrome...@gmail.com wrote: From: Adolfo Castro Menna adolfo.castrome...@gmail.com Subject: Norms - scoring issue To: solr-user@lucene.apache.org Date: Wednesday, September 14, 2011, 11:13 PM Hi All, I hope someone could shed some light on the issue I'm facing with solr 3.1.0. It looks like it's computing diferrent fieldNorm values despite my configuration that aims to ignore it. field name=item_name type=textgen indexed=true store=true omitNorms=true omitTermFrequencyAndPositions=true / field name=item_description type=textTight indexed=true store=true omitNorms=true omitTermFrequencyAndPositions=true / field name=item_tags type=text indexed=true stored=true multiValued=true omitNorms=true omitTermFrequencyAndPositions=true / I also have a custom class that extends DefaultSimilarity to override the idf method. Query: str name=qitem_name:octopus seafood OR item_description:octopus seafood OR item_tags:octopus seafood/str str name=sortscore desc,item_ranking desc/str The first 2 results are: doc float name=score0.5217492/float str name=item_nameGrilled Octopus/str arr name=item_tagsstrSeafood, tapas/str/arr /doc doc float name=score0.49379835/float str name=item_nameoctopus marisco/str arr name=item_tagsstrAppetizer, Mexican, Seafood, food/str/arr /doc Does anyone know why they get a different score? I'm expecting them to have the same scoring because both matched the two search terms. I checked the debug information and it seems that the difference involves the fieldNorm values. 1) Grilled Octopus 0.52174926 = (MATCH) product of: 0.7826238 = (MATCH) sum of: 0.4472136 = (MATCH) weight(item_name:octopus in 69), product of: 0.4472136 = queryWeight(item_name:octopus), product of: 1.0 = idf(docFreq=2, maxDocs=449) 0.4472136 = queryNorm 1.0 = (MATCH) fieldWeight(item_name:octopus in 69), product of: 1.0 = tf(termFreq(item_name:octopus)=1) 1.0 = idf(docFreq=2, maxDocs=449) 1.0 = fieldNorm(field=item_name, doc=69) 0.1118034 = (MATCH) weight(text:seafood in 69), product of: 0.4472136 = queryWeight(text:seafood), product of: 1.0 = idf(docFreq=8, maxDocs=449) 0.4472136 = queryNorm 0.25 = (MATCH) fieldWeight(text:seafood in 69), product of: 1.0 = tf(termFreq(text:seafood)=1) 1.0 = idf(docFreq=8, maxDocs=449) 0.25 = fieldNorm(field=text, doc=69) 0.1118034 = (MATCH) weight(text:seafood in 69), product of: 0.4472136 = queryWeight(text:seafood), product of: 1.0 = idf(docFreq=8, maxDocs=449) 0.4472136 = queryNorm 0.25 = (MATCH) fieldWeight(text:seafood in 69), product of: 1.0 = tf(termFreq(text:seafood)=1) 1.0 = idf(docFreq=8, maxDocs=449) 0.25 = fieldNorm(field=text, doc=69) 0.1118034 = (MATCH) weight(text:seafood in 69), product of: 0.4472136 = queryWeight(text:seafood), product of: 1.0 = idf(docFreq=8, maxDocs=449) 0.4472136 = queryNorm 0.25 = (MATCH) fieldWeight(text:seafood in 69), product of: 1.0 = tf(termFreq(text:seafood)=1) 1.0 = idf(docFreq=8, maxDocs=449) 0.25 = fieldNorm(field=text, doc=69) 0.667 = coord(4/6) 2) octopus marisco 0.49379835 = (MATCH) product of: 0.7406975 = (MATCH) sum of: 0.4472136 = (MATCH) weight(item_name:octopus in 81), product of: 0.4472136 = queryWeight(item_name:octopus), product of: 1.0 = idf(docFreq=2, maxDocs=449) 0.4472136 = queryNorm 1.0 = (MATCH) fieldWeight(item_name:octopus in 81), product of: 1.0 = tf(termFreq(item_name:octopus)=1) 1.0 = idf(docFreq=2, maxDocs=449) 1.0 = fieldNorm(field=item_name, doc=81) 0.09782797 = (MATCH) weight(text:seafood in 81), product of: 0.4472136 = queryWeight(text:seafood), product of: 1.0 = idf(docFreq=8, maxDocs=449) 0.4472136 = queryNorm 0.21875 = (MATCH) fieldWeight(text:seafood in 81), product of: 1.0 = tf(termFreq(text:seafood)=1) 1.0 = idf(docFreq=8, maxDocs=449) 0.21875 = fieldNorm(field=text, doc=81) 0.09782797 = (MATCH) weight(text:seafood in 81), product of: 0.4472136 = queryWeight(text:seafood), product of: 1.0 = idf(docFreq=8, maxDocs=449) 0.4472136 = queryNorm 0.21875 = (MATCH) fieldWeight(text:seafood in 81), product of: 1.0 = tf(termFreq(text:seafood)=1) 1.0 = idf(docFreq=8, maxDocs=449) 0.21875 = fieldNorm(field=text, doc=81
Re: Norms - scoring issue
Hi Ashmet, You're right. It was related to the text field which is the defaultSearch field. I also added omitNorms=true in the fieldtype definition and it's now working as expected Thanks, Adolfo.
Norms - scoring issue
Hi All, I hope someone could shed some light on the issue I'm facing with solr 3.1.0. It looks like it's computing diferrent fieldNorm values despite my configuration that aims to ignore it. field name=item_name type=textgen indexed=true store=true omitNorms=true omitTermFrequencyAndPositions=true / field name=item_description type=textTight indexed=true store=true omitNorms=true omitTermFrequencyAndPositions=true / field name=item_tags type=text indexed=true stored=true multiValued=true omitNorms=true omitTermFrequencyAndPositions=true / I also have a custom class that extends DefaultSimilarity to override the idf method. Query: str name=qitem_name:octopus seafood OR item_description:octopus seafood OR item_tags:octopus seafood/str str name=sortscore desc,item_ranking desc/str The first 2 results are: doc float name=score0.5217492/float str name=item_nameGrilled Octopus/str arr name=item_tagsstrSeafood, tapas/str/arr /doc doc float name=score0.49379835/float str name=item_nameoctopus marisco/str arr name=item_tagsstrAppetizer, Mexican, Seafood, food/str/arr /doc Does anyone know why they get a different score? I'm expecting them to have the same scoring because both matched the two search terms. I checked the debug information and it seems that the difference involves the fieldNorm values. 1) Grilled Octopus 0.52174926 = (MATCH) product of: 0.7826238 = (MATCH) sum of: 0.4472136 = (MATCH) weight(item_name:octopus in 69), product of: 0.4472136 = queryWeight(item_name:octopus), product of: 1.0 = idf(docFreq=2, maxDocs=449) 0.4472136 = queryNorm 1.0 = (MATCH) fieldWeight(item_name:octopus in 69), product of: 1.0 = tf(termFreq(item_name:octopus)=1) 1.0 = idf(docFreq=2, maxDocs=449) 1.0 = fieldNorm(field=item_name, doc=69) 0.1118034 = (MATCH) weight(text:seafood in 69), product of: 0.4472136 = queryWeight(text:seafood), product of: 1.0 = idf(docFreq=8, maxDocs=449) 0.4472136 = queryNorm 0.25 = (MATCH) fieldWeight(text:seafood in 69), product of: 1.0 = tf(termFreq(text:seafood)=1) 1.0 = idf(docFreq=8, maxDocs=449) 0.25 = fieldNorm(field=text, doc=69) 0.1118034 = (MATCH) weight(text:seafood in 69), product of: 0.4472136 = queryWeight(text:seafood), product of: 1.0 = idf(docFreq=8, maxDocs=449) 0.4472136 = queryNorm 0.25 = (MATCH) fieldWeight(text:seafood in 69), product of: 1.0 = tf(termFreq(text:seafood)=1) 1.0 = idf(docFreq=8, maxDocs=449) 0.25 = fieldNorm(field=text, doc=69) 0.1118034 = (MATCH) weight(text:seafood in 69), product of: 0.4472136 = queryWeight(text:seafood), product of: 1.0 = idf(docFreq=8, maxDocs=449) 0.4472136 = queryNorm 0.25 = (MATCH) fieldWeight(text:seafood in 69), product of: 1.0 = tf(termFreq(text:seafood)=1) 1.0 = idf(docFreq=8, maxDocs=449) 0.25 = fieldNorm(field=text, doc=69) 0.667 = coord(4/6) 2) octopus marisco 0.49379835 = (MATCH) product of: 0.7406975 = (MATCH) sum of: 0.4472136 = (MATCH) weight(item_name:octopus in 81), product of: 0.4472136 = queryWeight(item_name:octopus), product of: 1.0 = idf(docFreq=2, maxDocs=449) 0.4472136 = queryNorm 1.0 = (MATCH) fieldWeight(item_name:octopus in 81), product of: 1.0 = tf(termFreq(item_name:octopus)=1) 1.0 = idf(docFreq=2, maxDocs=449) 1.0 = fieldNorm(field=item_name, doc=81) 0.09782797 = (MATCH) weight(text:seafood in 81), product of: 0.4472136 = queryWeight(text:seafood), product of: 1.0 = idf(docFreq=8, maxDocs=449) 0.4472136 = queryNorm 0.21875 = (MATCH) fieldWeight(text:seafood in 81), product of: 1.0 = tf(termFreq(text:seafood)=1) 1.0 = idf(docFreq=8, maxDocs=449) 0.21875 = fieldNorm(field=text, doc=81) 0.09782797 = (MATCH) weight(text:seafood in 81), product of: 0.4472136 = queryWeight(text:seafood), product of: 1.0 = idf(docFreq=8, maxDocs=449) 0.4472136 = queryNorm 0.21875 = (MATCH) fieldWeight(text:seafood in 81), product of: 1.0 = tf(termFreq(text:seafood)=1) 1.0 = idf(docFreq=8, maxDocs=449) 0.21875 = fieldNorm(field=text, doc=81) 0.09782797 = (MATCH) weight(text:seafood in 81), product of: 0.4472136 = queryWeight(text:seafood), product of: 1.0 = idf(docFreq=8, maxDocs=449) 0.4472136 = queryNorm 0.21875 = (MATCH) fieldWeight(text:seafood in 81), product of: 1.0 = tf(termFreq(text:seafood)=1) 1.0 = idf(docFreq=8, maxDocs=449) 0.21875 = fieldNorm(field=text, doc=81) 0.667 = coord(4/6) Thanks in advance, Adolfo.
