ElasticsSearch 'Script Filter'
I have a ES search question and I think it can be solved using ES
scripting, but I was not able to solve it and there may be a better way.
The index has 3 document types, 'province', 'city' and 'neighborhood'
Here is how the index is created:
curl -s -XPUT 'localhost:9200/test/province/1' -d '{ province: Ontario
}'
curl -s -XPUT 'localhost:9200/test/city/2' -d '{ province: Ontario,
city: Toronto }'
curl -s -XPUT 'localhost:9200/test/city/3' -d '{ province: Ontario,
city : Ontario City }'
curl -s -XPUT 'localhost:9200/test/neighborhood/4' -d '{ province:
Ontario, city : Ontario City, neighborhood : Waterfront }'
curl -s -XPUT 'localhost:9200/test/neighborhood/5' -d '{ province:
Ontario, city : Ontario City, neighborhood : Midtown Ontario }'
The incoming search is for Ontario. If the document type in the index is
'province',I want to be able to search the 'province' field, if the
document type in the index is 'city' I want to search the 'city' field, and
if the document type in the index is 'neighborhood', I want to search the
'neighborhood' field. So for the search of 'Ontario', the desired results
would be to return document:
1,3,and 4 (1 is desired because the docuument type is 'province' and
'Ontario' matched to the 'province' field, 3 is desired because 'Ontario'
matched to the 'city', and 4 is desired because 'Ontario' matched to the
district field.
Here is a simple search that does not produce the desired results:
curl -s -XGET 'localhost:9200/test/_search?pretty=true' -d '{
query:{
bool:{
should : [ {
multi_match : {
query : Ontario,
fields : [ province, city, neighborhood ]
}
}]
}
}
}'
The problem is that the search for 'Ontario' get a match on all the docs.
Is it possible to search a specific field based on the document type?
Based on some example, I it seems like this might be a good usage for
scripting using a script filter?
Thanks for your help.
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/55fe24a5-86e9-42cb--29dcee3557e0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: Corruption error after upgrade to 1.0
Did you ever get this resolved and if so, how was it resolved? I am experiencing the same issue... On Monday, February 17, 2014 4:25:00 PM UTC-5, Mo wrote: After upgrading to 1.0 I am unable to index any documents. I get the following error. Could somebody help? [Aardwolf] Message not fully read (response) for [0] handler future(org.elasticsearch.client.transport.TransportClientNodesService$SimpleNodeSampler$1@5c6e3b4c), error [true], resetting [Aardwolf] failed to get node info for [#transport#-1][inet[/10.80.140.59:9300]], disconnecting... org.elasticsearch.transport.RemoteTransportException: Failed to deserialize exception response from stream Caused by: org.elasticsearch.transport.TransportSerializationException: Failed to deserialize exception response from stream at org.elasticsearch.transport.netty.MessageChannelHandler.handlerResponseError(MessageChannelHandler.java:168) at org.elasticsearch.transport.netty.MessageChannelHandler.messageReceived(MessageChannelHandler.java:122) at org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) at org.elasticsearch.common.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:296) at org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462) at org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443) at org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303) at org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559) at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:268) at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:255) at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88) at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:109) at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312) at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:90) at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178) at org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) at org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) Caused by: java.io.StreamCorruptedException: unexpected end of block data at java.io.ObjectInputStream.readObject0(Unknown Source) at java.io.ObjectInputStream.defaultReadFields(Unknown Source) at java.io.ObjectInputStream.defaultReadObject(Unknown Source) at java.lang.Throwable.readObject(Throwable.java:913) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at java.io.ObjectStreamClass.invokeReadObject(Unknown Source) at java.io.ObjectInputStream.readSerialData(Unknown Source) at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source) at java.io.ObjectInputStream.readObject0(Unknown Source) at java.io.ObjectInputStream.defaultReadFields(Unknown Source) -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/4af31baf-d27b-4f90-8f9d-fc6e72f70ead%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Exact phrase match - city names example
Thanks for the reply Prashy - I tried performing a term query like you
suggested; I get the same results (all documents containing main are
returned...E Main St, W Main St...) Do you only get one document returned
using the example I provided above (doc id 9/Main)??
On Thursday, February 27, 2014 2:25:09 AM UTC-5, Prashy wrote:
Try using the term query as term query is not analyzed so it might search
the
exact term only.
{
query : {
term : { street : xxx }
}
}
--
View this message in context:
http://elasticsearch-users.115913.n3.nabble.com/Exact-phrase-match-city-names-example-tp4019310p4050604.html
Sent from the ElasticSearch Users mailing list archive at Nabble.com.
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/6c5de1e2-b65d-4824-81d5-2e0e9636094d%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Re: Exact phrase match - city names example
I get the same results as you using your example Thanks for posting it. I am not sure why my original example does not work, but that is for me to figure out! Thanks again. On Thursday, June 14, 2012 2:02:28 PM UTC-4, Greg Silin wrote: Hi, One of our fields in the index stores city names, and we need to ensure that the term is matched exactly. So if we have san francisco indexed, we need to ensure that *only* the term san francisco matches; san or francisco or south san francisco should all be misses. In particular, I don't have a solution on how to make sure san francisco does not match against south san francisco Thanks -greg -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/5ef2e54a-b7b3-4dc6-a4d9-32d2eabb2010%40googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Exact phrase match - city names example
I am having problem a similar problem too. Here is how I set it up the
test index:
Create the index:
curl -s -XPUT 'localhost:9200/test' -d '{
mappings: {
properties: {
name: {
street: {
type: string,
index_analyzer: not_analyzed,
search_analyzer: not_analyzed,
index : not_analyzed
}
}
}
}
}'
Inert some data:
curl -s -XPUT 'localhost:9200/test/name/5' -d '{ street: [E Main St]}'
curl -s -XPUT 'localhost:9200/test/name/6' -d '{ street: [W Main St] }'
curl -s -XPUT 'localhost:9200/test/name/7' -d '{ street: [East Main Rd]
}'
curl -s -XPUT 'localhost:9200/test/name/8' -d '{ street: [West Main Rd]
}'
curl -s -XPUT 'localhost:9200/test/name/9' -d '{ street: [Main] }'
curl -s -XPUT 'localhost:9200/test/name/10' -d '{ street: [Main St] }'
--Now attempt to search for Main... Not Main St, Not East Main Rd...I
only want to return doc #9 - Main
curl -s -XGET 'localhost:9200/test/_search?pretty=true' -d '{
query:{
bool:{
must:[
{
match:{
street:{
query:main,
type:phrase,
analyzer : keyword
}
}
}
]
}
}
}';
The best document returned is Main, but I don't know how to filter out
the others that are not exact matches (although they contain matching
terms).
...
Here the results from my example above:
_score : 0.2876821, _source : { street: [Main] }
_score : 0.25316024, _source : { street: [East Main Rd] }
_score : 0.25316024, _source : { street: [W Main St] }
_score : 0.25316024, _source : { street: [E Main St]}
_score : 0.1805489, _source : { street: [Main St] }
_score : 0.14638957, _source : { street: [West Main Rd] }
On Thursday, June 14, 2012 3:38:31 PM UTC-4, Colin Dellow wrote:
Does index: not_analyzed not work for you (
http://www.elasticsearch.org/guide/reference/mapping/core-types.html) ?
On Thursday, 14 June 2012 14:02:28 UTC-4, Greg Silin wrote:
Hi,
One of our fields in the index stores city names, and we need to ensure
that the term is matched exactly.
So if we have san francisco indexed, we need to ensure that *only* the
term san francisco matches; san or francisco or south san francisco
should all be misses.
In particular, I don't have a solution on how to make sure san
francisco does not match against south san francisco
Thanks
-greg
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/42921778-0a92-4a57-ab6f-7f089ebe95ec%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Re: Best way to search/index the data - with and without whitespace
Hello Binh Ly - Thanks for the replay. I thought I had read that ngram searching should only be used at either index time or search time, but not both... Is that not the case? Thanks again. Thale On Wednesday, January 29, 2014 6:49:10 PM UTC-5, Binh Ly wrote: Thale, I would try edge ngrams (both index and search) and see how that works. I don't see why it wouldn't work for your 2 cases - just make your queries into match queries and use the AND operator. Good luck! -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/ef6a8b2f-e291-419f-8a8b-1eefa8657d2b%40googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Re: Best way to search/index the data - with and without whitespace
This is how I set up the mappings:
curl -s -XPUT 'localhost:9200/test' -d '{
mappings: {
properties: {
name: {
street: {
type: string,
index_analyzer: index_ngram,
search_analyzer: search_ngram
}
}
}
},
settings: {
analysis: {
filter: {
desc_ngram: {
type: edgeNGram,
min_gram: 3,
max_gram: 20
}
},
analyzer: {
index_ngram: {
type: custom,
tokenizer: keyword,
filter: [ desc_ngram, lowercase ]
},
search_ngram: {
type: custom,
tokenizer: keyword,
filter: lowercase
}
}
}
}
}'
This is how I built the index:
curl -s -XPUT 'localhost:9200/test/name/1' -d '{ street: Lakeshore Dr }'
curl -s -XPUT 'localhost:9200/test/name/2' -d '{ street: Sunnyshore Dr
}'
curl -s -XPUT 'localhost:9200/test/name/3' -d '{ street: Lake View Dr }'
curl -s -XPUT 'localhost:9200/test/name/4' -d '{ street: Shore Dr }'
If a user attempts to search for Lake Shore Dr, I want to only match to
document 1/Lakeshore Dr
If a user attempts to search for Lakeview Dr, I want to only match to
document 3/Lake View Dr
Here is an example of the query that is not working correctly:
curl -s -XGET 'localhost:9200/test/_search?pretty=true' -d '{
query:{
bool:{
must:[
{
match:{
street:{
query:lake shore dr,
type:boolean
}
}
}
]
}
}
}';
So is the issue with how I am setting up the mappings (tokenizer?, edgegram
vs ngrams?, size of ngrams?) or the query (I have tried things like setting
the minimum_should_match, and the analyzer to use), but I have not been
able to get the desired results.
Thanks all.
On Thursday, February 6, 2014 10:16:40 AM UTC-5, Binh Ly wrote:
Thale, you are correct - ngrams are usually used at index-time only, but
in your case and requirements, you might want to experiment both index and
seach time. I'd probably just increase the edge min ngram size to something
reasonable like maybe 4(?) and see if that works or not.
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/3b7a9d63-3a08-4cfc-96ce-4b22d44cd9db%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Best way to search/index the data - with and without whitespace
Hello - I am having a problem indexing and searching for words that may or
may not contain whitespace...Below is an example
Here is how the index is created:
curl -s -XPUT 'localhost:9200/test/name/1' -d '{ street: Lakeshore Dr }'
curl -s -XPUT 'localhost:9200/test/name/2' -d '{ street: Sunnyshore Dr
}'
curl -s -XPUT 'localhost:9200/test/name/3' -d '{ street: Lake View Dr }'
curl -s -XPUT 'localhost:9200/test/name/4' -d '{ street: Shore Dr }'
If I want to query for record 1/Lakeshore Dr, I can using the following
query:
curl -s -XGET 'localhost:9200/test/name/_search?pretty=true' -d '{
query:{
bool:{
must:[
{
match:{
street:{
query:lakeshore dr,
type:phrase
}
}
}
]
}
}
}';
This returns the desired result of document id 1. But if a user searches
for Lake Shore Dr (a space between Lake and Shore), it is still desired
to return document id 1.
And the inverse of this problem is if a user searches for Lakeview Dr
(but indexed as Lake View Dr):
curl -s -XGET 'localhost:9200/test/name/_search?pretty=true' -d '{
query:{
bool:{
must:[
{
match:{
street:{
query:lakeview dr,
type:phrase
}
}
}
]
}
}
}';
The search matches to no documents. If the search is changed to a
booleansearch instead of a phrase
,
many docs will match on dr, but doc #3, Lake Shore is not necessarily
returned as the top match.
NGrams at index time?? Ngrams at search time?? Remove whitespace at index
time/search time??
Any suggestions would be appreciated. Thanks.
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/06538a83-17d1-446c-9b27-cebf12c6fc47%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Re: 0.90.9 in elasticsearch - indexing with ngrams, search returns 0 results
Hi David - Thanks for the reply. I just tried it on 90.5, 90.9, 90.10, and
1.0.0.RC1. No results were returned in my searches. I forgot to include
that I have ES running on ubuntu if that makes a difference.
Thale
On Thursday, January 16, 2014 10:14:20 AM UTC-5, David Pilato wrote:
Hey!
Just tested it with es 1.0.0.RC1 and it's working fine.
See https://gist.github.com/dadoonet/8456535
--
*David Pilato* | *Technical Advocate* | *Elasticsearch.com*
@dadoonet https://twitter.com/dadoonet |
@elasticsearchfrhttps://twitter.com/elasticsearchfr
Le 16 janvier 2014 at 14:25:02, thale jacobs
(thale...@gmail.comjavascript:)
a écrit:
Hello - I am attempting to run a ngram test using elastics 0.90.9. I was
able to replicate the problem I am having in our production system by
following the example from here:
http://blog.rnf.me/2013/exact-substring-search-in-elasticsearch.html
The Mappings looks like this:
{
mappings: {
homes: {
properties: {
desc: {
type: string,
index_analyzer: index_ngram,
search_analyzer: search_ngram
}
}
}
},
settings: {
analysis: {
filter: {
desc_ngram: {
type: ngram,
min_gram: 3,
max_gram: 8
}
},
analyzer: {
index_ngram: {
type: custom,
tokenizer: keyword,
filter: [ desc_ngram, lowercase ]
},
search_ngram: {
type: custom,
tokenizer: keyword,
filter: lowercase
}
}
}
}
}
The mapping is set up as follows:
curl -s -XPUT 'localhost:9200/listings' -d @settings.json
Three records are inserted into the index as follows:
curl -s -XPUT 'localhost:9200/listings/homes/1' -d '{ desc: This is a
lovely home with a large yard. }'
curl -s -XPUT 'localhost:9200/listings/homes/2' -d '{ desc: This large
fixer-upper has a gravelly yard. }'
curl -s -XPUT 'localhost:9200/listings/homes/3' -d '{ desc: A colonial
mansion with a large yard and a pool. }'
The Search looks like this:
curl -s -XGET 'localhost:9200/listings/homes/_search?pretty=true' -d '{
query : { bool : { must : [ { match : { desc : { query : large
ya, type : phrase } } }, { match : { desc : { query : arge
yar, type : phrase } } }, { match : { desc : { query : rge
yard, type : phrase } } } ] } } }';
The problem is no results are returned. Any input that can be given for a
solution would be appreciated.
Thank you!
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com javascript:.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/4bee1be0-7ee4-4667-b4f7-b51371963928%40googlegroups.com
.
For more options, visit https://groups.google.com/groups/opt_out.
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/7bccf38a-eaa7-4e7b-9f82-a364a1d410df%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
