subject:"Re\: payload similarity"

Re: payload similarity

2015-04-25 Thread Dmitry Kan

Erick,

That's a neat example. Thanks! Btw, noticed, that luke can decode the
payloads quite well. Extra goody under debugger's belt.

Regards,
Dmitry

On Fri, Apr 24, 2015 at 9:22 PM, Erick Erickson 
wrote:

> I put up a complete example not too long ago that may help, see:
>
> http://lucidworks.com/blog/end-to-end-payload-example-in-solr/
>
> Best,
> Erick
>
> On Fri, Apr 24, 2015 at 6:33 AM, Dmitry Kan  wrote:
> > Ahmet, exactly. As I have just illustrated with code, simultaneously with
> > your reply. Thanks!
> >
> > On Fri, Apr 24, 2015 at 4:30 PM, Ahmet Arslan  >
> > wrote:
> >
> >> Hi Dmitry,
> >>
> >> I think, it is activated by PayloadTermQuery.
> >>
> >> Ahmet
> >>
> >>
> >>
> >> On Friday, April 24, 2015 2:51 PM, Dmitry Kan 
> >> wrote:
> >> Hi,
> >>
> >>
> >> Using the approach here
> >> http://lucidworks.com/blog/getting-started-with-payloads/ I have
> >> implemented my own PayloadSimilarity class. When debugging the code I
> have
> >> noticed, that the scorePayload method is never called. What could be
> wrong?
> >>
> >>
> >> [code]
> >>
> >> class PayloadSimilarity extends DefaultSimilarity {
> >> @Override
> >> public float scorePayload(int doc, int start, int end, BytesRef
> >> payload) {
> >> float payloadValue = PayloadHelper.decodeFloat(payload.bytes);
> >> System.out.println("payloadValue = " + payloadValue);
> >> return payloadValue;
> >> }
> >> }
> >>
> >> [/code]
> >>
> >>
> >> Here is how the similarity is injected during indexing:
> >>
> >> [code]
> >>
> >> PayloadEncoder encoder = new FloatEncoder();
> >> IndexWriterConfig indexWriterConfig = new
> >> IndexWriterConfig(Version.LUCENE_4_10_4, new
> >> PayloadAnalyzer(encoder));
> >> payloadSimilarity = new PayloadSimilarity();
> >> indexWriterConfig.setSimilarity(payloadSimilarity);
> >> IndexWriter writer = new IndexWriter(dir, indexWriterConfig);
> >>
> >> [/code]
> >>
> >>
> >> and during searching:
> >>
> >> [code]
> >>
> >> IndexReader indexReader = DirectoryReader.open(dir);
> >> IndexSearcher searcher = new IndexSearcher(indexReader);
> >> searcher.setSimilarity(payloadSimilarity);
> >>
> >> TermQuery termQuery = new TermQuery(new Term("body", "dogs"));
> >> termQuery.setBoost(1.1f);
> >> TopDocs topDocs = searcher.search(termQuery, 10);
> >> printResults(searcher, termQuery, topDocs);
> >>
> >>
> >> [/code]
> >>
> >> --
> >> Dmitry Kan
> >> Luke Toolbox: http://github.com/DmitryKey/luke
> >> Blog: http://dmitrykan.blogspot.com
> >> Twitter: http://twitter.com/dmitrykan
> >> SemanticAnalyzer: www.semanticanalyzer.info
> >>
> >
> >
> >
> > --
> > Dmitry Kan
> > Luke Toolbox: http://github.com/DmitryKey/luke
> > Blog: http://dmitrykan.blogspot.com
> > Twitter: http://twitter.com/dmitrykan
> > SemanticAnalyzer: www.semanticanalyzer.info
>



-- 
Dmitry Kan
Luke Toolbox: http://github.com/DmitryKey/luke
Blog: http://dmitrykan.blogspot.com
Twitter: http://twitter.com/dmitrykan
SemanticAnalyzer: www.semanticanalyzer.info

Re: payload similarity

2015-04-24 Thread Erick Erickson

I put up a complete example not too long ago that may help, see:

http://lucidworks.com/blog/end-to-end-payload-example-in-solr/

Best,
Erick

On Fri, Apr 24, 2015 at 6:33 AM, Dmitry Kan  wrote:
> Ahmet, exactly. As I have just illustrated with code, simultaneously with
> your reply. Thanks!
>
> On Fri, Apr 24, 2015 at 4:30 PM, Ahmet Arslan 
> wrote:
>
>> Hi Dmitry,
>>
>> I think, it is activated by PayloadTermQuery.
>>
>> Ahmet
>>
>>
>>
>> On Friday, April 24, 2015 2:51 PM, Dmitry Kan 
>> wrote:
>> Hi,
>>
>>
>> Using the approach here
>> http://lucidworks.com/blog/getting-started-with-payloads/ I have
>> implemented my own PayloadSimilarity class. When debugging the code I have
>> noticed, that the scorePayload method is never called. What could be wrong?
>>
>>
>> [code]
>>
>> class PayloadSimilarity extends DefaultSimilarity {
>> @Override
>> public float scorePayload(int doc, int start, int end, BytesRef
>> payload) {
>> float payloadValue = PayloadHelper.decodeFloat(payload.bytes);
>> System.out.println("payloadValue = " + payloadValue);
>> return payloadValue;
>> }
>> }
>>
>> [/code]
>>
>>
>> Here is how the similarity is injected during indexing:
>>
>> [code]
>>
>> PayloadEncoder encoder = new FloatEncoder();
>> IndexWriterConfig indexWriterConfig = new
>> IndexWriterConfig(Version.LUCENE_4_10_4, new
>> PayloadAnalyzer(encoder));
>> payloadSimilarity = new PayloadSimilarity();
>> indexWriterConfig.setSimilarity(payloadSimilarity);
>> IndexWriter writer = new IndexWriter(dir, indexWriterConfig);
>>
>> [/code]
>>
>>
>> and during searching:
>>
>> [code]
>>
>> IndexReader indexReader = DirectoryReader.open(dir);
>> IndexSearcher searcher = new IndexSearcher(indexReader);
>> searcher.setSimilarity(payloadSimilarity);
>>
>> TermQuery termQuery = new TermQuery(new Term("body", "dogs"));
>> termQuery.setBoost(1.1f);
>> TopDocs topDocs = searcher.search(termQuery, 10);
>> printResults(searcher, termQuery, topDocs);
>>
>>
>> [/code]
>>
>> --
>> Dmitry Kan
>> Luke Toolbox: http://github.com/DmitryKey/luke
>> Blog: http://dmitrykan.blogspot.com
>> Twitter: http://twitter.com/dmitrykan
>> SemanticAnalyzer: www.semanticanalyzer.info
>>
>
>
>
> --
> Dmitry Kan
> Luke Toolbox: http://github.com/DmitryKey/luke
> Blog: http://dmitrykan.blogspot.com
> Twitter: http://twitter.com/dmitrykan
> SemanticAnalyzer: www.semanticanalyzer.info

Re: payload similarity

2015-04-24 Thread Dmitry Kan

Ahmet, exactly. As I have just illustrated with code, simultaneously with
your reply. Thanks!

On Fri, Apr 24, 2015 at 4:30 PM, Ahmet Arslan 
wrote:

> Hi Dmitry,
>
> I think, it is activated by PayloadTermQuery.
>
> Ahmet
>
>
>
> On Friday, April 24, 2015 2:51 PM, Dmitry Kan 
> wrote:
> Hi,
>
>
> Using the approach here
> http://lucidworks.com/blog/getting-started-with-payloads/ I have
> implemented my own PayloadSimilarity class. When debugging the code I have
> noticed, that the scorePayload method is never called. What could be wrong?
>
>
> [code]
>
> class PayloadSimilarity extends DefaultSimilarity {
> @Override
> public float scorePayload(int doc, int start, int end, BytesRef
> payload) {
> float payloadValue = PayloadHelper.decodeFloat(payload.bytes);
> System.out.println("payloadValue = " + payloadValue);
> return payloadValue;
> }
> }
>
> [/code]
>
>
> Here is how the similarity is injected during indexing:
>
> [code]
>
> PayloadEncoder encoder = new FloatEncoder();
> IndexWriterConfig indexWriterConfig = new
> IndexWriterConfig(Version.LUCENE_4_10_4, new
> PayloadAnalyzer(encoder));
> payloadSimilarity = new PayloadSimilarity();
> indexWriterConfig.setSimilarity(payloadSimilarity);
> IndexWriter writer = new IndexWriter(dir, indexWriterConfig);
>
> [/code]
>
>
> and during searching:
>
> [code]
>
> IndexReader indexReader = DirectoryReader.open(dir);
> IndexSearcher searcher = new IndexSearcher(indexReader);
> searcher.setSimilarity(payloadSimilarity);
>
> TermQuery termQuery = new TermQuery(new Term("body", "dogs"));
> termQuery.setBoost(1.1f);
> TopDocs topDocs = searcher.search(termQuery, 10);
> printResults(searcher, termQuery, topDocs);
>
>
> [/code]
>
> --
> Dmitry Kan
> Luke Toolbox: http://github.com/DmitryKey/luke
> Blog: http://dmitrykan.blogspot.com
> Twitter: http://twitter.com/dmitrykan
> SemanticAnalyzer: www.semanticanalyzer.info
>



-- 
Dmitry Kan
Luke Toolbox: http://github.com/DmitryKey/luke
Blog: http://dmitrykan.blogspot.com
Twitter: http://twitter.com/dmitrykan
SemanticAnalyzer: www.semanticanalyzer.info

Re: payload similarity

2015-04-24 Thread Dmitry Kan

Answering my own question:

in order to account for payloads, PayloadTermQuery should be used instead
of TermQuery:

PayloadTermQuery payloadTermQuery = new PayloadTermQuery(new Term("body",
"dogs"), new MaxPayloadFunction());

Then in the query explanation we get:

---
Results for body:dogs of type:
org.apache.lucene.search.payloads.PayloadTermQuery
Doc: doc=0 score=3.125 shardIndex=-1
payloadValue = 10.0
Explain: 3.125 = (MATCH) btq, product of:
  0.3125 = weight(body:dogs in 0) [PayloadSimilarity], result of:
0.3125 = fieldWeight in 0, product of:
  1.0 = tf(freq=1.0), with freq of:
1.0 = phraseFreq=1.0
  1.0 = idf(docFreq=3, maxDocs=10)
  0.3125 = fieldNorm(doc=0)
  10.0 = MaxPayloadFunction.docScore()

Doc: doc=9 score=3.125 shardIndex=-1
payloadValue = 10.0
Explain: 3.125 = (MATCH) btq, product of:
  0.3125 = weight(body:dogs in 9) [PayloadSimilarity], result of:
0.3125 = fieldWeight in 9, product of:
  1.0 = tf(freq=1.0), with freq of:
1.0 = phraseFreq=1.0
  1.0 = idf(docFreq=3, maxDocs=10)
  0.3125 = fieldNorm(doc=9)
  10.0 = MaxPayloadFunction.docScore()

Doc: doc=1 score=0.3125 shardIndex=-1
Explain: 0.3125 = (MATCH) btq, product of:
  0.3125 = weight(body:dogs in 1) [PayloadSimilarity], result of:
0.3125 = fieldWeight in 1, product of:
  1.0 = tf(freq=1.0), with freq of:
1.0 = phraseFreq=1.0
  1.0 = idf(docFreq=3, maxDocs=10)
  0.3125 = fieldNorm(doc=1)
  1.0 = MaxPayloadFunction.docScore()

On Fri, Apr 24, 2015 at 2:50 PM, Dmitry Kan  wrote:

> Hi,
>
>
> Using the approach here
> http://lucidworks.com/blog/getting-started-with-payloads/ I have
> implemented my own PayloadSimilarity class. When debugging the code I have
> noticed, that the scorePayload method is never called. What could be wrong?
>
>
> [code]
>
> class PayloadSimilarity extends DefaultSimilarity {
> @Override
> public float scorePayload(int doc, int start, int end, BytesRef payload) {
> float payloadValue = PayloadHelper.decodeFloat(payload.bytes);
> System.out.println("payloadValue = " + payloadValue);
> return payloadValue;
> }
> }
>
> [/code]
>
>
> Here is how the similarity is injected during indexing:
>
> [code]
>
> PayloadEncoder encoder = new FloatEncoder();
> IndexWriterConfig indexWriterConfig = new 
> IndexWriterConfig(Version.LUCENE_4_10_4, new PayloadAnalyzer(encoder));
> payloadSimilarity = new PayloadSimilarity();
> indexWriterConfig.setSimilarity(payloadSimilarity);
> IndexWriter writer = new IndexWriter(dir, indexWriterConfig);
>
> [/code]
>
>
> and during searching:
>
> [code]
>
> IndexReader indexReader = DirectoryReader.open(dir);
> IndexSearcher searcher = new IndexSearcher(indexReader);
> searcher.setSimilarity(payloadSimilarity);
>
> TermQuery termQuery = new TermQuery(new Term("body", "dogs"));
> termQuery.setBoost(1.1f);
> TopDocs topDocs = searcher.search(termQuery, 10);
> printResults(searcher, termQuery, topDocs);
>
>
> [/code]
>
> --
> Dmitry Kan
> Luke Toolbox: http://github.com/DmitryKey/luke
> Blog: http://dmitrykan.blogspot.com
> Twitter: http://twitter.com/dmitrykan
> SemanticAnalyzer: www.semanticanalyzer.info
>
>


-- 
Dmitry Kan
Luke Toolbox: http://github.com/DmitryKey/luke
Blog: http://dmitrykan.blogspot.com
Twitter: http://twitter.com/dmitrykan
SemanticAnalyzer: www.semanticanalyzer.info

Re: payload similarity

2015-04-24 Thread Ahmet Arslan

Hi Dmitry,

I think, it is activated by PayloadTermQuery.

Ahmet



On Friday, April 24, 2015 2:51 PM, Dmitry Kan  wrote:
Hi,


Using the approach here
http://lucidworks.com/blog/getting-started-with-payloads/ I have
implemented my own PayloadSimilarity class. When debugging the code I have
noticed, that the scorePayload method is never called. What could be wrong?


[code]

class PayloadSimilarity extends DefaultSimilarity {
@Override
public float scorePayload(int doc, int start, int end, BytesRef payload) {
float payloadValue = PayloadHelper.decodeFloat(payload.bytes);
System.out.println("payloadValue = " + payloadValue);
return payloadValue;
}
}

[/code]


Here is how the similarity is injected during indexing:

[code]

PayloadEncoder encoder = new FloatEncoder();
IndexWriterConfig indexWriterConfig = new
IndexWriterConfig(Version.LUCENE_4_10_4, new
PayloadAnalyzer(encoder));
payloadSimilarity = new PayloadSimilarity();
indexWriterConfig.setSimilarity(payloadSimilarity);
IndexWriter writer = new IndexWriter(dir, indexWriterConfig);

[/code]


and during searching:

[code]

IndexReader indexReader = DirectoryReader.open(dir);
IndexSearcher searcher = new IndexSearcher(indexReader);
searcher.setSimilarity(payloadSimilarity);

TermQuery termQuery = new TermQuery(new Term("body", "dogs"));
termQuery.setBoost(1.1f);
TopDocs topDocs = searcher.search(termQuery, 10);
printResults(searcher, termQuery, topDocs);


[/code]

-- 
Dmitry Kan
Luke Toolbox: http://github.com/DmitryKey/luke
Blog: http://dmitrykan.blogspot.com
Twitter: http://twitter.com/dmitrykan
SemanticAnalyzer: www.semanticanalyzer.info

Re: payload similarity

Re: payload similarity

Re: payload similarity

Re: payload similarity

Re: payload similarity

5 matches

Site Navigation

Mail list logo

Footer information