Tim
Grant Ingersoll wrote:
When you do an explain on these results, what are all the factors
that contribute to the score?
Could you increase the coord() factor in a custom Similarity
implementation, to give a bigger boost to documents that have more
matching terms? The point of coord is to give a little bump to
those docs that have more terms from the query in a given
document. Sounds like you want a bigger bump once you have
multiple query terms in a document. Would this work for you?
Also, below...
On Jul 3, 2007, at 3:20 PM, Tim Sturge wrote:
That's true, but it's not clear that I want phrase matches.
Consider for example:
"Lucene Download" as a query. I want something that strongly
references "Lucene" (in the title) and strongly references
"Download" but "Download Lucene" or "Lucene Project Download" are
better than some page that happens to contain the exact phrase.
Not sure I follow you here. By strongly references, do you mean
there are multiple occurrences of Download? Why would those
alternatives be better than an exact phrase match?
Other examples are "camera review" or "Gonzales scandal"; there's
a whole class of "subject <modifier>" queries that are not really
phrase based, and my corpus isn't large enough to necessarily
contain the phrase anyway.
I agree that many two or three word queries are really best
matched by phrases, but not all. Is it common to use a phrase
query with high slop to overcome the unequal weighting problem?
Also, my interface does support "\"John Bush\"" (ie the user can
quote the phrase if they like) and I would prefer not to infer
automatically that they meant to do so.
Tim
Jason Pump wrote:
You're not using any type of phrase search. Try ->
( (title:"John Bush"^4.0) OR (body:"John Bush") ) AND
( (title:John^4.0 body:John) AND (title:Bush^4.0 body:Bush) )
or maybe
( (title:"John Bush"~4^4.0) OR (body:"John Bush"~4) ) AND
( (title:John^4.0 body:John) AND (title:Bush^4.0 body:Bush) )
Tim Sturge wrote:
I'm following myself up here to ask if anyone has experience or
code with a BooleanQuery that weights the terms it encounters
on a product basis rather than a sum basis.
This would effectively compute the geometric mean of the term
score (rather than the arithmetic mean) and would give me more
"middle bias". It also has the great advantage that it
automatically implements AND (as something without the term has
a score of 0.0 which causes the query to go to 0.0 as well.)
I'm curious though why this doesn't already exist. Is it a bad
idea in general (that I will discover once I implement it and
look at the results?) or does it make searching a lot slower?
Thanks,
Tim
Tim Sturge wrote:
I have an index with two different sources of information, one
small but of high quality (call it "title"), and one large,
but of lower quality (call it "body"). I give boosts to
certain documents related to their popularity (this is very
similar to what one would do indexing the web).
The problem I have is a query like "John Bush". I translate
that into " (title:John^4.0 body:John) AND (title:Bush^4.0
body:Bush) ". But the results I get are:
1. George Bush
...
4. John Kerry
...
10. John Bush
The reason is (looking at explain) that George Bush is scored:
169 = sum(
1 = <match in body with tiny norm for "John">
)
168 = sum(
160 = <title match for "Bush">
8 = <body match for "Bush">
)
)
and John Kerry is similar but reversed. Poor old "John Bush"
only scores:
72 = sum(
40 = (<title match for "John">+<body match>)
32 = (<title match for "Bush">+ <body match>)
)
because his initial boost was only 1/4 of George's.
The question I have is, how can tell the searcher to care
about "balance"? I really want the score over 2 terms to be
more like (sqrt(X)+sqrt(Y))^2 or maybe even exp(log(X)+log(Y))
rather than just X+Y. Is that supported in some obvious way,
or is there some other way to phrase my query to say "I want
both terms but they should both be important if possible?"
Thanks,
Tim
-----------------------------------------------------------------
-- --
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
------------------------------------------------------------------
-- -
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
-------------------------------------------------------------------
--
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
--------------------------------------------------------------------
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
------------------------------------------------------
Grant Ingersoll
http://www.grantingersoll.com/
http://lucene.grantingersoll.com
http://www.paperoftheweek.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
[
{
"explain" : "169.71423 = (MATCH) sum of:
0.033628028 = (MATCH) product of:
0.23539619 = (MATCH) sum of:
0.23539619 = (MATCH) weight(body:john in 1673743), product of:
0.12947647 = queryWeight(body:john), product of:
5.0118046 = idf(docFreq=185620)
0.025834301 = queryNorm
1.8180615 = (MATCH) fieldWeight(body:john in 1673743),
product of:
3.3166249 = tf(termFreq(body:john)=11)
5.0118046 = idf(docFreq=185620)
0.109375 = fieldNorm(field=wikipedia, doc=1673743)
0.14285715 = coord(1\/7)
169.6806 = (MATCH) product of:
197.9607 = (MATCH) sum of:
51.99727 = (MATCH) weight(name:bush in 1673743), product of:
0.25916338 = queryWeight(name:bush), product of:
10.0317545 = idf(docFreq=1225)
0.025834301 = queryNorm
200.63509 = (MATCH) fieldWeight(name:bush in 1673743),
product of:
1.0 = tf(termFreq(name:bush)=1)
10.0317545 = idf(docFreq=1225)
20.0 = fieldNorm(field=name, doc=1673743)
51.013706 = (MATCH) weight(alias:bush in 1673743), product of:
0.3630294 = queryWeight(alias:bush), product of:
14.052224 = idf(docFreq=21)
0.025834301 = queryNorm
140.52225 = (MATCH) fieldWeight(alias:bush in 1673743),
product of:
1.0 = tf(termFreq(alias:bush)=1)
14.052224 = idf(docFreq=21)
10.0 = fieldNorm(field=alias, doc=1673743)
33.15815 = (MATCH) weight(title:bush in 1673743), product of:
0.29268032 = queryWeight(title:bush), product of:
11.329136 = idf(docFreq=334)
0.025834301 = queryNorm
113.29136 = (MATCH) fieldWeight(title:bush in 1673743),
product of:
1.0 = tf(termFreq(title:bush)=1)
11.329136 = idf(docFreq=334)
10.0 = fieldNorm(field=title, doc=1673743)
41.10021 = (MATCH) weight(anchor:bush in 1673743), product of:
0.30437908 = queryWeight(anchor:bush), product of:
11.781975 = idf(docFreq=212)
0.025834301 = queryNorm
135.02968 = (MATCH) fieldWeight(anchor:bush in 1673743),
product of:
36.67424 = tf(termFreq(anchor:bush)=1345)
11.781975 = idf(docFreq=212)
0.3125 = fieldNorm(field=anchor, doc=1673743)
18.274998 = (MATCH) weight(text:bush in 1673743), product of:
0.25839525 = queryWeight(text:bush), product of:
10.002022 = idf(docFreq=1262)
0.025834301 = queryNorm
70.724976 = (MATCH) fieldWeight(text:bush in 1673743),
product of:
1.4142135 = tf(termFreq(text:bush)=2)
10.002022 = idf(docFreq=1262)
5.0 = fieldNorm(field=text, doc=1673743)
2.4163725 = (MATCH) weight(body:bush in 1673743), product of:
0.19234328 = queryWeight(body:bush), product of:
7.445267 = idf(docFreq=16284)
0.025834301 = queryNorm
12.562812 = (MATCH) fieldWeight(body:bush in 1673743),
product of:
15.427249 = tf(termFreq(body:bush)=238)
7.445267 = idf(docFreq=16284)
0.109375 = fieldNorm(field=wikipedia, doc=1673743)
0.85714287 = coord(6\/7)
",
"name" : "George W. Bush",
},
{
"explain" : "154.83218 = (MATCH) sum of:
0.02267201 = (MATCH) product of:
0.15870407 = (MATCH) sum of:
0.15870407 = (MATCH) weight(body:john in 14947), product of:
0.12947647 = queryWeight(body:john), product of:
5.0118046 = idf(docFreq=185620)
0.025834301 = queryNorm
1.2257367 = (MATCH) fieldWeight(body:john in 14947),
product of:
2.236068 = tf(termFreq(body:john)=5)
5.0118046 = idf(docFreq=185620)
0.109375 = fieldNorm(field=wikipedia, doc=14947)
0.14285715 = coord(1\/7)
154.80951 = (MATCH) product of:
180.6111 = (MATCH) sum of:
41.597813 = (MATCH) weight(name:bush in 14947), product of:
0.25916338 = queryWeight(name:bush), product of:
10.0317545 = idf(docFreq=1225)
0.025834301 = queryNorm
160.50807 = (MATCH) fieldWeight(name:bush in 14947),
product of:
1.0 = tf(termFreq(name:bush)=1)
10.0317545 = idf(docFreq=1225)
16.0 = fieldNorm(field=name, doc=14947)
61.216446 = (MATCH) weight(alias:bush in 14947), product of:
0.3630294 = queryWeight(alias:bush), product of:
14.052224 = idf(docFreq=21)
0.025834301 = queryNorm
168.6267 = (MATCH) fieldWeight(alias:bush in 14947),
product of:
1.0 = tf(termFreq(alias:bush)=1)
14.052224 = idf(docFreq=21)
12.0 = fieldNorm(field=alias, doc=14947)
26.526522 = (MATCH) weight(title:bush in 14947), product of:
0.29268032 = queryWeight(title:bush), product of:
11.329136 = idf(docFreq=334)
0.025834301 = queryNorm
90.63309 = (MATCH) fieldWeight(title:bush in 14947),
product of:
1.0 = tf(termFreq(title:bush)=1)
11.329136 = idf(docFreq=334)
8.0 = fieldNorm(field=title, doc=14947)
30.758215 = (MATCH) weight(anchor:bush in 14947), product of:
0.30437908 = queryWeight(anchor:bush), product of:
11.781975 = idf(docFreq=212)
0.025834301 = queryNorm
101.05233 = (MATCH) fieldWeight(anchor:bush in 14947),
product of:
34.307434 = tf(termFreq(anchor:bush)=1177)
11.781975 = idf(docFreq=212)
0.25 = fieldNorm(field=anchor, doc=14947)
18.274998 = (MATCH) weight(text:bush in 14947), product of:
0.25839525 = queryWeight(text:bush), product of:
10.002022 = idf(docFreq=1262)
0.025834301 = queryNorm
70.724976 = (MATCH) fieldWeight(text:bush in 14947),
product of:
1.4142135 = tf(termFreq(text:bush)=2)
10.002022 = idf(docFreq=1262)
5.0 = fieldNorm(field=text, doc=14947)
2.237126 = (MATCH) weight(body:bush in 14947), product of:
0.19234328 = queryWeight(body:bush), product of:
7.445267 = idf(docFreq=16284)
0.025834301 = queryNorm
11.630903 = (MATCH) fieldWeight(body:bush in 14947),
product of:
14.282857 = tf(termFreq(body:bush)=204)
7.445267 = idf(docFreq=16284)
0.109375 = fieldNorm(field=wikipedia, doc=14947)
0.85714287 = coord(6\/7)
",
"name" : "George H. W. Bush",
},
{
"explain" : "92.35373 = (MATCH) sum of:
92.255936 = (MATCH) product of:
107.63193 = (MATCH) sum of:
29.974728 = (MATCH) weight(name:john in 2198385), product of:
0.17962648 = queryWeight(name:john), product of:
6.9530225 = idf(docFreq=26641)
0.025834301 = queryNorm
166.87254 = (MATCH) fieldWeight(name:john in 2198385),
product of:
1.0 = tf(termFreq(name:john)=1)
6.9530225 = idf(docFreq=26641)
24.0 = fieldNorm(field=name, doc=2198385)
34.876133 = (MATCH) weight(alias:john in 2198385), product of:
0.27401346 = queryWeight(alias:john), product of:
10.606575 = idf(docFreq=689)
0.025834301 = queryNorm
127.2789 = (MATCH) fieldWeight(alias:john in 2198385),
product of:
1.0 = tf(termFreq(alias:john)=1)
10.606575 = idf(docFreq=689)
12.0 = fieldNorm(field=alias, doc=2198385)
17.689255 = (MATCH) weight(title:john in 2198385), product of:
0.19514729 = queryWeight(title:john), product of:
7.5538054 = idf(docFreq=14609)
0.025834301 = queryNorm
90.64566 = (MATCH) fieldWeight(title:john in 2198385),
product of:
1.0 = tf(termFreq(title:john)=1)
7.5538054 = idf(docFreq=14609)
12.0 = fieldNorm(field=title, doc=2198385)
14.100239 = (MATCH) weight(anchor:john in 2198385), product of:
0.20842676 = queryWeight(anchor:john), product of:
8.06783 = idf(docFreq=8737)
0.025834301 = queryNorm
67.65081 = (MATCH) fieldWeight(anchor:john in 2198385),
product of:
13.416408 = tf(termFreq(anchor:john)=180)
8.06783 = idf(docFreq=8737)
0.625 = fieldNorm(field=anchor, doc=2198385)
10.557128 = (MATCH) weight(text:john in 2198385), product of:
0.1792826 = queryWeight(text:john), product of:
6.9397116 = idf(docFreq=26998)
0.025834301 = queryNorm
58.885403 = (MATCH) fieldWeight(text:john in 2198385),
product of:
1.4142135 = tf(termFreq(text:john)=2)
6.9397116 = idf(docFreq=26998)
6.0 = fieldNorm(field=text, doc=2198385)
0.43445155 = (MATCH) weight(body:john in 2198385), product of:
0.12947647 = queryWeight(body:john), product of:
5.0118046 = idf(docFreq=185620)
0.025834301 = queryNorm
3.3554478 = (MATCH) fieldWeight(body:john in 2198385),
product of:
7.1414285 = tf(termFreq(body:john)=51)
5.0118046 = idf(docFreq=185620)
0.09375 = fieldNorm(field=wikipedia, doc=2198385)
0.85714287 = coord(6\/7)
0.097795136 = (MATCH) product of:
0.6845659 = (MATCH) sum of:
0.6845659 = (MATCH) weight(body:bush in 2198385), product of:
0.19234328 = queryWeight(body:bush), product of:
7.445267 = idf(docFreq=16284)
0.025834301 = queryNorm
3.559084 = (MATCH) fieldWeight(body:bush in 2198385),
product of:
5.0990195 = tf(termFreq(body:bush)=26)
7.445267 = idf(docFreq=16284)
0.09375 = fieldNorm(field=wikipedia, doc=2198385)
0.14285715 = coord(1\/7)
",
"name" : "John Kerry",
},
{
"explain" : "81.16132 = (MATCH) sum of:
81.13575 = (MATCH) product of:
94.65837 = (MATCH) sum of:
24.978941 = (MATCH) weight(name:john in 66053), product of:
0.17962648 = queryWeight(name:john), product of:
6.9530225 = idf(docFreq=26641)
0.025834301 = queryNorm
139.06046 = (MATCH) fieldWeight(name:john in 66053),
product of:
1.0 = tf(termFreq(name:john)=1)
6.9530225 = idf(docFreq=26641)
20.0 = fieldNorm(field=name, doc=66053)
29.063442 = (MATCH) weight(alias:john in 66053), product of:
0.27401346 = queryWeight(alias:john), product of:
10.606575 = idf(docFreq=689)
0.025834301 = queryNorm
106.06575 = (MATCH) fieldWeight(alias:john in 66053),
product of:
1.0 = tf(termFreq(alias:john)=1)
10.606575 = idf(docFreq=689)
10.0 = fieldNorm(field=alias, doc=66053)
14.741047 = (MATCH) weight(title:john in 66053), product of:
0.19514729 = queryWeight(title:john), product of:
7.5538054 = idf(docFreq=14609)
0.025834301 = queryNorm
75.538055 = (MATCH) fieldWeight(title:john in 66053),
product of:
1.0 = tf(termFreq(title:john)=1)
7.5538054 = idf(docFreq=14609)
10.0 = fieldNorm(field=title, doc=66053)
16.475775 = (MATCH) weight(anchor:john in 66053), product of:
0.20842676 = queryWeight(anchor:john), product of:
8.06783 = idf(docFreq=8737)
0.025834301 = queryNorm
79.04827 = (MATCH) fieldWeight(anchor:john in 66053),
product of:
2.4494898 = tf(termFreq(anchor:john)=6)
8.06783 = idf(docFreq=8737)
4.0 = fieldNorm(field=anchor, doc=66053)
8.797606 = (MATCH) weight(text:john in 66053), product of:
0.1792826 = queryWeight(text:john), product of:
6.9397116 = idf(docFreq=26998)
0.025834301 = queryNorm
49.071167 = (MATCH) fieldWeight(text:john in 66053),
product of:
1.4142135 = tf(termFreq(text:john)=2)
6.9397116 = idf(docFreq=26998)
5.0 = fieldNorm(field=text, doc=66053)
0.60155636 = (MATCH) weight(body:john in 66053), product of:
0.12947647 = queryWeight(body:john), product of:
5.0118046 = idf(docFreq=185620)
0.025834301 = queryNorm
4.646067 = (MATCH) fieldWeight(body:john in 66053), product
of:
7.4161983 = tf(termFreq(body:john)=55)
5.0118046 = idf(docFreq=185620)
0.125 = fieldNorm(field=wikipedia, doc=66053)
0.85714287 = coord(6\/7)
0.025572272 = (MATCH) product of:
0.17900589 = (MATCH) sum of:
0.17900589 = (MATCH) weight(body:bush in 66053), product of:
0.19234328 = queryWeight(body:bush), product of:
7.445267 = idf(docFreq=16284)
0.025834301 = queryNorm
0.9306584 = (MATCH) fieldWeight(body:bush in 66053),
product of:
1.0 = tf(termFreq(body:bush)=1)
7.445267 = idf(docFreq=16284)
0.125 = fieldNorm(field=wikipedia, doc=66053)
0.14285715 = coord(1\/7)
",
"name" : "John Denver",
},
{
"explain" : "72.412 = (MATCH) sum of:
23.203518 = (MATCH) product of:
32.484924 = (MATCH) sum of:
12.4894705 = (MATCH) weight(name:john in 535045), product of:
0.17962648 = queryWeight(name:john), product of:
6.9530225 = idf(docFreq=26641)
0.025834301 = queryNorm
69.53023 = (MATCH) fieldWeight(name:john in 535045),
product of:
1.0 = tf(termFreq(name:john)=1)
6.9530225 = idf(docFreq=26641)
10.0 = fieldNorm(field=name, doc=535045)
5.8964186 = (MATCH) weight(title:john in 535045), product of:
0.19514729 = queryWeight(title:john), product of:
7.5538054 = idf(docFreq=14609)
0.025834301 = queryNorm
30.215221 = (MATCH) fieldWeight(title:john in 535045),
product of:
1.0 = tf(termFreq(title:john)=1)
7.5538054 = idf(docFreq=14609)
4.0 = fieldNorm(field=title, doc=535045)
8.737598 = (MATCH) weight(anchor:john in 535045), product of:
0.20842676 = queryWeight(anchor:john), product of:
8.06783 = idf(docFreq=8737)
0.025834301 = queryNorm
41.921673 = (MATCH) fieldWeight(anchor:john in 535045),
product of:
3.4641016 = tf(termFreq(anchor:john)=12)
8.06783 = idf(docFreq=8737)
1.5 = fieldNorm(field=anchor, doc=535045)
4.9766784 = (MATCH) weight(text:john in 535045), product of:
0.1792826 = queryWeight(text:john), product of:
6.9397116 = idf(docFreq=26998)
0.025834301 = queryNorm
27.758846 = (MATCH) fieldWeight(text:john in 535045),
product of:
1.0 = tf(termFreq(text:john)=1)
6.9397116 = idf(docFreq=26998)
4.0 = fieldNorm(field=text, doc=535045)
0.38475677 = (MATCH) weight(body:john in 535045), product of:
0.12947647 = queryWeight(body:john), product of:
5.0118046 = idf(docFreq=185620)
0.025834301 = queryNorm
2.9716346 = (MATCH) fieldWeight(body:john in 535045),
product of:
3.1622777 = tf(termFreq(body:john)=10)
5.0118046 = idf(docFreq=185620)
0.1875 = fieldNorm(field=wikipedia, doc=535045)
0.71428573 = coord(5\/7)
49.208485 = (MATCH) product of:
68.89188 = (MATCH) sum of:
25.998634 = (MATCH) weight(name:bush in 535045), product of:
0.25916338 = queryWeight(name:bush), product of:
10.0317545 = idf(docFreq=1225)
0.025834301 = queryNorm
100.31754 = (MATCH) fieldWeight(name:bush in 535045),
product of:
1.0 = tf(termFreq(name:bush)=1)
10.0317545 = idf(docFreq=1225)
10.0 = fieldNorm(field=name, doc=535045)
13.263261 = (MATCH) weight(title:bush in 535045), product of:
0.29268032 = queryWeight(title:bush), product of:
11.329136 = idf(docFreq=334)
0.025834301 = queryNorm
45.316544 = (MATCH) fieldWeight(title:bush in 535045),
product of:
1.0 = tf(termFreq(title:bush)=1)
11.329136 = idf(docFreq=334)
4.0 = fieldNorm(field=title, doc=535045)
18.634373 = (MATCH) weight(anchor:bush in 535045), product of:
0.30437908 = queryWeight(anchor:bush), product of:
11.781975 = idf(docFreq=212)
0.025834301 = queryNorm
61.220936 = (MATCH) fieldWeight(anchor:bush in 535045),
product of:
3.4641016 = tf(termFreq(anchor:bush)=12)
11.781975 = idf(docFreq=212)
1.5 = fieldNorm(field=anchor, doc=535045)
10.3379 = (MATCH) weight(text:bush in 535045), product of:
0.25839525 = queryWeight(text:bush), product of:
10.002022 = idf(docFreq=1262)
0.025834301 = queryNorm
40.008087 = (MATCH) fieldWeight(text:bush in 535045),
product of:
1.0 = tf(termFreq(text:bush)=1)
10.002022 = idf(docFreq=1262)
4.0 = fieldNorm(field=text, doc=535045)
0.65770966 = (MATCH) weight(body:bush in 535045), product of:
0.19234328 = queryWeight(body:bush), product of:
7.445267 = idf(docFreq=16284)
0.025834301 = queryNorm
3.4194574 = (MATCH) fieldWeight(body:bush in 535045),
product of:
2.4494898 = tf(termFreq(body:bush)=6)
7.445267 = idf(docFreq=16284)
0.1875 = fieldNorm(field=wikipedia, doc=535045)
0.71428573 = coord(5\/7)
",
"name" : "John Bush",
}
]
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
