subject:"Solr SearchComponent\-like functionality\?"

Re: Solr SearchComponent-like functionality?

2014-04-23 Thread Srinivasan Ramaswamy

Thanks for your answer ! I have some followup questions

1. I wrote a custom script to return a constant score of 2.5. It works for
the following query

{
query : {
function_score: {
query:{
match_all:{}
},
script_score: {
lang:native,
script: constant-score
}
}
}
}

in the result i see the value of _score is 2.5 for all docs. But when i
include a query string it doesnt work

{
query : {
function_score: {
query:{
query_string:{
query: dog
}
},
script_score: {
lang:native,
script: constant-score
}
}
}
}

2. I would like to access the search keyword (if possible analyzed version)
to some custom processing depending on the keyword. is that possible ?

3. You mentioned I can consider rescore api, I am interested in trying that
out. I have some external data stored outside the index (in-memory) , how
do i make the rescore function pick up data from that external data
structure instead of index ?

Thanks in advance for any help !

Thanks
Srini

On Friday, April 18, 2014 11:48:25 AM UTC-7, Matt Weber wrote:

Well, the scripts runs against all matching documents of the query so you
can do a match_all query [1] to have the logic applied to all your
documents. This is going to be expensive though, so try to filter out as
many documents as possible before applying the custom scoring. Maybe even
perform a rescore [2] on the top X docs. It really all depends on your
requirements though. Run some tests and tune based on those results.

When I said to be careful. I mean don't do a lot of blocking IO or long
running calculations as the script is ran against each matching document.
Cache results and make the script return as quick as possible.

[1]
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-match-all-query.html
[2]
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-rescore.html

Thanks,
Matt Weber

On Fri, Apr 18, 2014 at 9:46 AM, Srinivasan Ramaswamy
ursv...@gmail.comjavascript:
wrote:

Thats great, thanks for your reply. This looks like a good solution for
my requirement ! Is this script applied in each shard ? I want to apply
this function to all the documents so that the Top N picked from each shard
is picked by my custom score.

Also, can you elaborate a little bit on be careful you can significantly
impact your query performance if you are not careful. I would like to
understand the best practices there.

On Friday, April 18, 2014 8:14:54 AM UTC-7, Matt Weber wrote:

Yes, you can use the Function Score Query [1] in combination with a
native script written in java [2]. With the native script you can
basically do whatever you want, but be careful you can significantly impact
your query performance if you are not careful.

[1] http://www.elasticsearch.org/guide/en/elasticsearch/
reference/current/query-dsl-function-score-query.html
[2] http://www.elasticsearch.org/guide/en/elasticsearch/
reference/current/modules-scripting.html#_native_java_scripts

Thanks,
Matt Weber

On Thu, Apr 17, 2014 at 11:54 PM, Srinivasan Ramaswamy
ursv...@gmail.com wrote:

I would like to influence the ranking with few fields that are not
stored in the index (eg click data for keyword-documents). I have used
custom SearchComponent in Solr to implement similar functionality in the
past. I am wondering how can i achieve the same in ElasticSearch.

I know this thread is a very old thread, but i didnt find much
information on how to do custom scoring (in elasticsearch) with data thats
not stored in the index. This thread looked very relevant to my
requirement, so trying to see whether you guys have solved similar
requirements with elasticsearch.

Thanks
Srini

On Wednesday, September 7, 2011 12:18:09 PM UTC-7, Lukáš Vlček wrote:

Hi Otis,

So if I understand it correctly (providing my knowledge is quite
limited here) you are asking if
1) it is possible to hook into query processing flow and inject or
extend custom handlers for individual flow phases and
2) if we can find in ES the same functionality which is currently
provided by components listed here: http://wiki.apache.org/s
olr/SearchComponent (or here: http://lucene.apache.org/solr/
api/org/apache/solr/handler/component/SearchComponent.html).

As for #1, frankly, I do not know. I have been playing with plugins a
bit but did not have a chance to explore full potential of it yet. I
remember that Shay mentioned that not every aspect of ES is pluggable now
but that is all I know about it (personally, I did not hit the limits by
myself yet, may be I would if I wanted to employ Carrot2 clustering or
something like that)

Re: Solr SearchComponent-like functionality?

2014-04-18 Thread Srinivasan Ramaswamy

I would like to influence the ranking with few fields that are not stored
in the index (eg click data for keyword-documents). I have used custom
SearchComponent in Solr to implement similar functionality in the past. I
am wondering how can i achieve the same in ElasticSearch.

I know this thread is a very old thread, but i didnt find much information
on how to do custom scoring (in elasticsearch) with data thats not stored
in the index. This thread looked very relevant to my requirement, so trying
to see whether you guys have solved similar requirements with elasticsearch.

Thanks
Srini

On Wednesday, September 7, 2011 12:18:09 PM UTC-7, Lukáš Vlček wrote:

Hi Otis,

So if I understand it correctly (providing my knowledge is quite limited
here) you are asking if
1) it is possible to hook into query processing flow and inject or extend
custom handlers for individual flow phases and
2) if we can find in ES the same functionality which is currently provided
by components listed here: http://wiki.apache.org/solr/SearchComponent (or
here:
http://lucene.apache.org/solr/api/org/apache/solr/handler/component/SearchComponent.html
).

As for #1, frankly, I do not know. I have been playing with plugins a bit
but did not have a chance to explore full potential of it yet. I remember
that Shay mentioned that not every aspect of ES is pluggable now but that
is all I know about it (personally, I did not hit the limits by myself yet,
may be I would if I wanted to employ Carrot2 clustering or something like
that)

As for #2, if you are after one-to-one comparison of Solr SearchComponents
and ES then I think we would find some matches and also some misses. Still
it could be an interesting exercise to do (although we should be careful to
include only those features that do work well in distributed environment).
We could probably end up identifying new feature requests, so this can be
useful.

Regards,
Lukas

On Wed, Sep 7, 2011 at 6:17 PM, Otis Gospodnetic
otis.gos...@gmail.comjavascript:
wrote:

Hi Lukas,

Yes, SearchComponents are about extensibility, but specifically about
extending how queries are handled within Solr once Solr gets them. I
know ES has other types of plugins, and you've listed several of them,
but I'm wondering about which of them is SearchComponent-like.
I've looked at
http://www.elasticsearch.org/guide/reference/modules/plugins.html
, but couldn't find the answer to my Q there. Maybe I'm looking at
the wrong place?

Thanks,
Otis
--
Sematext is hiring Search Engineers --
http://sematext.com/about/jobs.html

On Sep 6, 2:57 pm, Lukáš Vlček lukas.vl...@gmail.com wrote:
Hi,

I am not Solr expert but to me it seems that SearchComponents in Solr
are
about extensibility of out of the box functionality. If that is the case
then I would say that we can talk about plugins in ES world. Although
there
is no official doc about how to implement custom plugins yet it is
really
not difficult. Apart from that there are several plugins that are part
of
distribution (river plugins, attachments mapper, ICU analysis, scripting
languages ... to name a few) and they can be used as an inspiration if
a new
plugin implementation is needed.

My 2 cents.

Lukas

On Tue, Sep 6, 2011 at 5:35 PM, Otis Gospodnetic
otis.gospodne...@gmail.com

wrote:
Hello,

A long time Solr user posted a good question about ES over on Sematext
Blog, about an equivalent of Solr's SearchComponents in ES:

http://blog.sematext.com/2010/05/03/elastic-search-distributed-lucene.
..

I'm curious, too. Thanks.

Otis
--
Sematext is hiring Search Engineers --
http://sematext.com/about/jobs.html

--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/16add2bc-c629-4613-934f-004c8cc749df%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Solr SearchComponent-like functionality?

2014-04-18 Thread Matt Weber

Yes, you can use the Function Score Query [1] in combination with a native
script written in java [2]. With the native script you can basically do
whatever you want, but be careful you can significantly impact your query
performance if you are not careful.

[1]
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-function-score-query.html
[2]
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-scripting.html#_native_java_scripts

Thanks,
Matt Weber

On Thu, Apr 17, 2014 at 11:54 PM, Srinivasan Ramaswamy
ursva...@gmail.comwrote:

I would like to influence the ranking with few fields that are not stored
in the index (eg click data for keyword-documents). I have used custom
SearchComponent in Solr to implement similar functionality in the past. I
am wondering how can i achieve the same in ElasticSearch.

I know this thread is a very old thread, but i didnt find much information
on how to do custom scoring (in elasticsearch) with data thats not stored
in the index. This thread looked very relevant to my requirement, so trying
to see whether you guys have solved similar requirements with elasticsearch.

Thanks
Srini

On Wednesday, September 7, 2011 12:18:09 PM UTC-7, Lukáš Vlček wrote:

Hi Otis,

So if I understand it correctly (providing my knowledge is quite limited
here) you are asking if
1) it is possible to hook into query processing flow and inject or extend
custom handlers for individual flow phases and
2) if we can find in ES the same functionality which is currently
provided by components listed here: http://wiki.apache.org/
solr/SearchComponent (or here: http://lucene.apache.org/solr/
api/org/apache/solr/handler/component/SearchComponent.html).

As for #1, frankly, I do not know. I have been playing with plugins a bit
but did not have a chance to explore full potential of it yet. I remember
that Shay mentioned that not every aspect of ES is pluggable now but that
is all I know about it (personally, I did not hit the limits by myself yet,
may be I would if I wanted to employ Carrot2 clustering or something like
that)

As for #2, if you are after one-to-one comparison of Solr
SearchComponents and ES then I think we would find some matches and also
some misses. Still it could be an interesting exercise to do (although we
should be careful to include only those features that do work well in
distributed environment). We could probably end up identifying new feature
requests, so this can be useful.

Regards,
Lukas

On Wed, Sep 7, 2011 at 6:17 PM, Otis Gospodnetic
otis.gos...@gmail.comwrote:

Hi Lukas,

Yes, SearchComponents are about extensibility, but specifically about
extending how queries are handled within Solr once Solr gets them. I
know ES has other types of plugins, and you've listed several of them,
but I'm wondering about which of them is SearchComponent-like.
I've looked at http://www.elasticsearch.org/guide/reference/modules/
plugins.html
, but couldn't find the answer to my Q there. Maybe I'm looking at
the wrong place?

Thanks,
Otis
--
Sematext is hiring Search Engineers -- http://sematext.com/about/
jobs.html

On Sep 6, 2:57 pm, Lukáš Vlček lukas.vl...@gmail.com wrote:
Hi,

I am not Solr expert but to me it seems that SearchComponents in Solr
are
about extensibility of out of the box functionality. If that is the
case
then I would say that we can talk about plugins in ES world. Although
there
is no official doc about how to implement custom plugins yet it is
really
not difficult. Apart from that there are several plugins that are part
of
distribution (river plugins, attachments mapper, ICU analysis,
scripting
languages ... to name a few) and they can be used as an inspiration if
a new
plugin implementation is needed.

My 2 cents.

Lukas

On Tue, Sep 6, 2011 at 5:35 PM, Otis Gospodnetic
otis.gospodne...@gmail.com

wrote:
Hello,

A long time Solr user posted a good question about ES over on
Sematext
Blog, about an equivalent of Solr's SearchComponents in ES:

http://blog.sematext.com/2010/05/03/elastic-search-
distributed-lucene...

I'm curious, too. Thanks.

Otis
--
Sematext is hiring Search Engineers --http://sematext.com/about/
jobs.html

--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/16add2bc-c629-4613-934f-004c8cc749df%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/16add2bc-c629-4613-934f-004c8cc749df%40googlegroups.com?utm_medium=emailutm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
elasticsearch

Re: Solr SearchComponent-like functionality?

2014-04-18 Thread Srinivasan Ramaswamy

Thats great, thanks for your reply. This looks like a good solution for my
requirement ! Is this script applied in each shard ? I want to apply this
function to all the documents so that the Top N picked from each shard is
picked by my custom score.

Also, can you elaborate a little bit on be careful you can significantly
impact your query performance if you are not careful. I would like to
understand the best practices there.

On Friday, April 18, 2014 8:14:54 AM UTC-7, Matt Weber wrote:

Yes, you can use the Function Score Query [1] in combination with a native
script written in java [2]. With the native script you can basically do
whatever you want, but be careful you can significantly impact your query
performance if you are not careful.

[1]
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-function-score-query.html
[2]
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-scripting.html#_native_java_scripts

Thanks,
Matt Weber

On Thu, Apr 17, 2014 at 11:54 PM, Srinivasan Ramaswamy
ursv...@gmail.comjavascript:
wrote:

I would like to influence the ranking with few fields that are not stored
in the index (eg click data for keyword-documents). I have used custom
SearchComponent in Solr to implement similar functionality in the past. I
am wondering how can i achieve the same in ElasticSearch.

Thanks
Srini

On Wednesday, September 7, 2011 12:18:09 PM UTC-7, Lukáš Vlček wrote:

Hi Otis,

So if I understand it correctly (providing my knowledge is quite limited
here) you are asking if
1) it is possible to hook into query processing flow and inject or
extend custom handlers for individual flow phases and
2) if we can find in ES the same functionality which is currently
provided by components listed here: http://wiki.apache.org/
solr/SearchComponent (or here: http://lucene.apache.org/solr/
api/org/apache/solr/handler/component/SearchComponent.html).

As for #2, if you are after one-to-one comparison of Solr
SearchComponents and ES then I think we would find some matches and also
some misses. Still it could be an interesting exercise to do (although we
should be careful to include only those features that do work well in
distributed environment). We could probably end up identifying new feature
requests, so this can be useful.

Regards,
Lukas

On Wed, Sep 7, 2011 at 6:17 PM, Otis Gospodnetic
otis.gos...@gmail.comwrote:

Hi Lukas,

Yes, SearchComponents are about extensibility, but specifically about
extending how queries are handled within Solr once Solr gets them. I
know ES has other types of plugins, and you've listed several of them,
but I'm wondering about which of them is SearchComponent-like.
I've looked at http://www.elasticsearch.org/guide/reference/modules/
plugins.html
, but couldn't find the answer to my Q there. Maybe I'm looking at
the wrong place?

Thanks,
Otis
--
Sematext is hiring Search Engineers -- http://sematext.com/about/
jobs.html

On Sep 6, 2:57 pm, Lukáš Vlček lukas.vl...@gmail.com wrote:
Hi,

I am not Solr expert but to me it seems that SearchComponents in Solr
are
about extensibility of out of the box functionality. If that is the
case
then I would say that we can talk about plugins in ES world. Although
there
is no official doc about how to implement custom plugins yet it is
really
not difficult. Apart from that there are several plugins that are
part of
distribution (river plugins, attachments mapper, ICU analysis,
scripting
languages ... to name a few) and they can be used as an inspiration
if a new
plugin implementation is needed.

My 2 cents.

Lukas

On Tue, Sep 6, 2011 at 5:35 PM, Otis Gospodnetic
otis.gospodne...@gmail.com

wrote:
Hello,

A long time Solr user posted a good question about ES over on
Sematext
Blog, about an equivalent of Solr's SearchComponents in ES:

http://blog.sematext.com/2010/05/03/elastic-search-
distributed-lucene...

I'm curious, too. Thanks.

Otis
--
Sematext is hiring Search Engineers --http://sematext.com/about/
jobs.html

--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To

Re: Solr SearchComponent-like functionality?

2014-04-18 Thread Matt Weber

Thanks,
Matt Weber

On Fri, Apr 18, 2014 at 9:46 AM, Srinivasan Ramaswamy ursva...@gmail.comwrote:

Thats great, thanks for your reply. This looks like a good solution for my
requirement ! Is this script applied in each shard ? I want to apply this
function to all the documents so that the Top N picked from each shard is
picked by my custom score.

Also, can you elaborate a little bit on be careful you can significantly
impact your query performance if you are not careful. I would like to
understand the best practices there.

On Friday, April 18, 2014 8:14:54 AM UTC-7, Matt Weber wrote:

Thanks,
Matt Weber

On Thu, Apr 17, 2014 at 11:54 PM, Srinivasan Ramaswamy ursv...@gmail.com
wrote:

Thanks
Srini

On Wednesday, September 7, 2011 12:18:09 PM UTC-7, Lukáš Vlček wrote:

Hi Otis,

As for #2, if you are after one-to-one comparison of Solr
SearchComponents and ES then I think we would find some matches and also
some misses. Still it could be an interesting exercise to do (although we
should be careful to include only those features that do work well in
distributed environment). We could probably end up identifying new feature
requests, so this can be useful.

Regards,
Lukas

On Wed, Sep 7, 2011 at 6:17 PM, Otis Gospodnetic otis.gos...@gmail.com
wrote:

Hi Lukas,

Yes, SearchComponents are about extensibility, but specifically about
extending how queries are handled within Solr once Solr gets them. I
know ES has other types of plugins, and you've listed several of them,
but I'm wondering about which of them is SearchComponent-like.
I've looked at http://www.elasticsearch.org/guide/reference/modules/
plugins.html
, but couldn't find the answer to my Q there. Maybe I'm looking at
the wrong place?

Thanks,
Otis
--
Sematext is hiring Search Engineers -- http://sematext.com/about/jobs
.html

On Sep 6, 2:57 pm, Lukáš Vlček lukas.vl...@gmail.com wrote:
Hi,

Re: Solr SearchComponent-like functionality?

Re: Solr SearchComponent-like functionality?

Re: Solr SearchComponent-like functionality?

Re: Solr SearchComponent-like functionality?

Re: Solr SearchComponent-like functionality?

5 matches

Site Navigation

Mail list logo

Footer information