Re: [ANN] Elasticsearch Simple Action Plugin

[Luiz Guilherme Pais dos Santos]

Hi Jörg,

Thanks a lot! Some month ago I started to write one plugin and it was
really difficult. This skeleton is perfect for a cold start :)


On Tue, Jun 3, 2014 at 12:15 PM, joergpra...@gmail.com <
joergpra...@gmail.com> wrote:

> Hi,
>
> many of us want to start writing extensions for Elasticsearch.
>
> Except submitting pull requests to the core code, one great advantage of
> Elasticsearch is the plugin mechanism. Here, custom code can be hooked into
> Elasticsearch, without having to ask for inclusion into the core code.
> Nevertheless, plugin code can be published on Github and easily included
> into a running ES instance by using the ES plugin command line tool.
>
> Unfortunately, writing plugins is not so easy as it seems. There are many
> plugins, some of them are very advanced, and finding a starting point for a
> personal project could be quite hard.
>
> Hence, for educational purposes, I wrote a tiny plugin, as a starting
> point, to demonstrate how a plugin works.
>
> The simple plugin is indeed very simple. It makes reuse of the standard
> search action:
>
> - it defines a built-in query (a "match all" query)
>
> - it creates a custom action for it
>
> - the action is called from Java API
>
> - the result of the action (the search response of the "match all" query)
> is logged
>
> The plugin code comes with a junit test. It is available at
>
> https://github.com/jprante/elasticsearch-simple-action-plugin
>
> In the hope it is useful,
>
> Jörg
>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/CAKdsXoH-M6%2BZroAz8Reb3e2agW0vXKSavk%3D0hD_bq%2BBHtRYLhw%40mail.gmail.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>



-- 
Luiz Guilherme P. Santos

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAMdL%3DZEcM%3DzRmLk9%2BK7fiZ1hoT1zbcy_ScYT0F_8Kx0RzPkm%2BA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Analyzing URLs for regexp queries

[Luiz Guilherme Pais dos Santos]

Hi Matt,

If you mark your field as not_analyzed:
{
"mappings": {
"type1": {
"properties": {
"url": {
"type": "string",
"index": "not_analyzed"
}
}
}
}
}

You could use a regexp query:
POST _search
{
"query": {
"regexp": {
"url": "http://example\.com/\d{4}/\d{2}/\d{2}/([^/]+)/$"
}
}
}



On Tue, Apr 15, 2014 at 5:57 PM, matt burton  wrote:

> I have a field in my documents that consists of a URL.
> {...
> "url":"http://example.com/2014/04/15/foo-bar-baz/";
> ...}
>
> I would like to use a regexp query/filter to find documents in my index
> with urls matching a regex pattern.
> For example: "http://example\.com/\d{4}/\d{2}/\d{2}/([^/]+)/$"
>
> I'm a bit stumped about how to configure an analyzer in the document
> _mapping to enable a regexp search (like above) for the url field. I've
> tried the standard and keyword analyzer, but they didn't work.
>
> I'm not even sure if this is possible to do, if not I'll can do it outside
> of ES, but I thought I'd ask here to see if ya'll had any guidance.
>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/62e05ecc-500f-474e-a5e6-220a9eb86eb3%40googlegroups.com
> .
> For more options, visit https://groups.google.com/d/optout.
>



-- 
Luiz Guilherme P. Santos

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAMdL%3DZGwrZWON6tKoZDf4d0BOenDJDNyxaU0HfUOOV83%2Bh9KKA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Retrieve specific fields

[Luiz Guilherme Pais dos Santos]

curl -XGET "http://localhost:9200/_search?fields=field1";
curl -XGET "http://localhost:9200/_search?fields=field1,field2";


On Thu, Apr 10, 2014 at 7:46 PM, Mohit Anchlia wrote:

> I should be more explicit, is it possible when using "q=" type of query
> string? For eg: http://host/_search?q="field:hello";
>
> Is it possible to add a query string parameter to achieve that?
>
> On Thu, Apr 10, 2014 at 3:04 PM, Luiz Guilherme Pais dos Santos <
> luizgpsan...@gmail.com> wrote:
>
>> Hi Mohit,
>>
>> It is possible, as shown here:
>>
>> http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-fields.html#search-request-fields
>>
>>
>> On Thu, Apr 10, 2014 at 4:27 PM, Mohit Anchlia wrote:
>>
>>> Is there a way to search and retrieve only specific fields using simple
>>> query string?
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "elasticsearch" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to elasticsearch+unsubscr...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/elasticsearch/CAOT3TWoPZ0mdu8EJUtjOazQ34uVFhnmk7g9Yf6OGYM2az9mpBA%40mail.gmail.com<https://groups.google.com/d/msgid/elasticsearch/CAOT3TWoPZ0mdu8EJUtjOazQ34uVFhnmk7g9Yf6OGYM2az9mpBA%40mail.gmail.com?utm_medium=email&utm_source=footer>
>>> .
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>
>>
>> --
>> Luiz Guilherme P. Santos
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to elasticsearch+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/elasticsearch/CAMdL%3DZHUqCnNMKc5wN6sysUzy%2BY%2BB79v6FYd6GUcew-87k8kPQ%40mail.gmail.com<https://groups.google.com/d/msgid/elasticsearch/CAMdL%3DZHUqCnNMKc5wN6sysUzy%2BY%2BB79v6FYd6GUcew-87k8kPQ%40mail.gmail.com?utm_medium=email&utm_source=footer>
>> .
>>
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>  --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/CAOT3TWo2VWCMRaSsuQXAe4cQDu90vsGSg9%3D8x8d7CrWROH8k-A%40mail.gmail.com<https://groups.google.com/d/msgid/elasticsearch/CAOT3TWo2VWCMRaSsuQXAe4cQDu90vsGSg9%3D8x8d7CrWROH8k-A%40mail.gmail.com?utm_medium=email&utm_source=footer>
> .
>
> For more options, visit https://groups.google.com/d/optout.
>



-- 
Luiz Guilherme P. Santos

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAMdL%3DZH9D6H74%2BimMucnyP2PUYKh2MMY0LbcbFJyJ4qStoji9Q%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Retrieve specific fields

[Luiz Guilherme Pais dos Santos]

Hi Mohit,

It is possible, as shown here:
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-fields.html#search-request-fields


On Thu, Apr 10, 2014 at 4:27 PM, Mohit Anchlia wrote:

> Is there a way to search and retrieve only specific fields using simple
> query string?
>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/CAOT3TWoPZ0mdu8EJUtjOazQ34uVFhnmk7g9Yf6OGYM2az9mpBA%40mail.gmail.com
> .
> For more options, visit https://groups.google.com/d/optout.
>



-- 
Luiz Guilherme P. Santos

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAMdL%3DZHUqCnNMKc5wN6sysUzy%2BY%2BB79v6FYd6GUcew-87k8kPQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: synonyms in a query

[Luiz Guilherme Pais dos Santos]

Hi Alex,

First create your index with the synonym filter:

curl -XPUT 'localhost:9200/myindex/' -d '{
"settings": {
"index": {
"analysis": {
"filter": {
"synonym_filter": {
"type": "synonym",
"synonyms": [
"foo, foo bar => core"
]
}
},
"analyzer": {
"synonym": {
"tokenizer": "whitespace",
"filter": [
"synonym_filter"
],
"type": "custom"
}
}
}
}
}
}'

After index one document with the title equals do "core":

curl -XPOST 'localhost:9200/myindex/test/1' -d '{
"title": "core"
}'

Then search for "foo":

curl -XPOST localhost:1980/myindex/_search -d '{
"query": {
"multi_match": {
"query": "foo",
"fields": [
"title"
],
"type": "phrase_prefix",
"analyzer": "synonym"
}
}
}'

The response is the previous indexed document:
{
"took": 1,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 0.30685282,
"hits": [
{
"_index": "myindex",
"_type": "test",
"_id": "1",
"_score": 0.30685282,
"_source": {
"title": "core"
}
}
]
}
}



On Mon, Apr 7, 2014 at 11:03 AM, Alex K  wrote:

> Hello Luiz, thank you for your reply!
>
> As we use rivers, I was told to declare the analyzer there.
> It looks like this for me:
> {
>"index" : {
>   "analysis" : {
>  "filter" : {
> "synonym_filter" : {
> "type" : "synonym",
> "synonyms" : [
> "foo, foo bar => core"
> ]
> }
>  },
>  "analyzer" : {
> "synonym" : {
> "tokenizer" : "whitespace",
> "filter" : [
> "synonym_filter"
> ],
> "type" : "custom",
> }
>  }
>   }
>}
> }
> which acctually says, for testing-purpose, 'if someone searches for 'foo'
> or 'foo bar', search for 'core' '
>
> Now my query uses the analyzer:
> {
> "query": {
> "bool": {
> "should": [
> {
> "multi_match": {
> "query": "foo",
> "fields": [
> "TITLE",
> "SHORTDESC"
> ],
> "type": "phrase_prefix",
>
> "analyzer": "synonym"
>
> }
> },
> {
> "multi_match": {
> "query": "foo",
> "cutoff_frequency": null,
> "fields": [
> "TITLE",
> "SHORTDESC"
> ]
> }
> }
> ]
> }
> },
> "filter": {
> "term": {
> "ACTIVE": 1
> }
> },
> "sort": {
> "TITLE": {
> "order": "asc"
> }
> },
> "size": 7
> }
>
> But I get an error there:
> [...]nested: QueryParsingException[[test484] [multi_match] analyzer
> [synonym] not found];[...]
>
> What am I doing wrong here?
>
>
> Am Montag, 7. April 2014 09:29:17 UTC+2 schrieb Alex K:
>
>> Hello there,
>>
>> i have a query, example is this:
>> {
>> "query": {
>> "bool": {
>> "should": [
>> {
>> "multi_match": {
>> "query": "foo",
>> "fields": [
>> "TITLE",
>> "SHORTDESC"
>> ],
>> "type": "phrase_prefix"
>> }
>> },
>> {
>> "multi_match": {
>> "query": "foo",
>> "cutoff_frequency": null,
>> "fields": [
>> "TITLE",
>> "SHORTDESC"
>> ]
>> }
>> }
>> ]
>> }
>> },
>> "filter": {
>> "term": {
>> "ACTIVE": 1
>> }
>> },
>> "sort": {
>> "TITLE": {
>> "order": "asc"
>> }
>> },
>> "size": 7
>> }
>>
>> Now I have the question if I can use synonyms he

Re: synonyms in a query

[Luiz Guilherme Pais dos Santos]

Hi Alex,

If the fields you are searching ("TITLE","SHORTDESC") were indexed using an
analyzer that declares synonyms the "multi_match" query will use it. It
happens because the multi_match query use the analyzer explicit defined in
mapping or the default analyzer.[1]

You can explicit declare the analyzer in the query if you want:

POST test/_search
{
"query": {
"multi_match": {
   "query": "test",
   "fields": ["title"],
   "analyzer": "standard"
}
}
}

[1]
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-match-query.html#_boolean


On Mon, Apr 7, 2014 at 4:29 AM, Alex K  wrote:

> Hello there,
>
> i have a query, example is this:
> {
> "query": {
> "bool": {
> "should": [
> {
> "multi_match": {
> "query": "foo",
> "fields": [
> "TITLE",
> "SHORTDESC"
> ],
> "type": "phrase_prefix"
> }
> },
> {
> "multi_match": {
> "query": "foo",
> "cutoff_frequency": null,
> "fields": [
> "TITLE",
> "SHORTDESC"
> ]
> }
> }
> ]
> }
> },
> "filter": {
> "term": {
> "ACTIVE": 1
> }
> },
> "sort": {
> "TITLE": {
> "order": "asc"
> }
> },
> "size": 7
> }
>
> Now I have the question if I can use synonyms here?
>
> I already saw that you can use a synonym-token inside an analyzer.
> But I have a query here, not an analyzer.
> Do I have to put an analyzer inside the query?
>
> I don't know much about ES yet, so this may be a total stupid question.
> Thank you in advance :-)
>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/5fe2d157-b437-4bd8-8a18-8aa4f41f63fe%40googlegroups.com
> .
> For more options, visit https://groups.google.com/d/optout.
>



-- 
Luiz Guilherme P. Santos

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAMdL%3DZHOw14zbQuROWbam7%3Dd%3DYkSH3K5DJHEzsz73kqdJ0nsUg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: unable to get function_score to work

[Luiz Guilherme Pais dos Santos]

Hi Srinivasan,

It seems your intern "query" is malformed. Try this:

{
   "query": {
  "function_score": {
 "boost_mode": "replace",
 "query": {
"match_all": {}
 },
 "script_score": {
"params": {
   "param1": 2,
   "param2": 3.1
},
"script": "_score * doc['design.CLICKS'].value / pow(param1,
param2)"
 }
  }
   }
}

Cheers,
Luiz Guilherme


On Thu, Apr 3, 2014 at 5:03 PM, Srinivasan Ramaswamy wrote:

> Hi
>
> I just started playing around with elastic search and I am trying out the
> examples in the website with my data. I am trying to get custom scoring
> working. I am following the example from
> http://www.elasticsearch.org/guide/en/elasticsearch/reference/1.x/query-dsl-function-score-query.html
>
> But unable to get the example working
>
> {
> "query" : {
> "function_score": {
> "boost_mode": "replace",
> "query": {"design.PRODUCT_DESC":"cool"},
> "script_score": {
> "params": {
> "param1": 2,
> "param2": 3.1
> },
> "script": "_score * doc['design.CLICKS'].value / pow(param1,
> param2)"
> }
> }
> }
> }
>
> I tried this query and i get the error
>
> Caused by: org.elasticsearch.index.query.QueryParsingException: [designs]
> [_na] query malformed, no field after start_object
>
> at
> org.elasticsearch.index.query.QueryParseContext.parseInnerQuery(QueryParseContext.java:216)
>
> at
> org.elasticsearch.index.query.functionscore.FunctionScoreQueryParser.parse(FunctionScoreQueryParser.java:89)
>
> at
> org.elasticsearch.index.query.QueryParseContext.parseInnerQuery(QueryParseContext.java:223)
>
> at
> org.elasticsearch.index.query.IndexQueryParserService.parse(IndexQueryParserService.java:330)
>
> at
> org.elasticsearch.index.query.IndexQueryParserService.parse(IndexQueryParserService.java:260)
>
> at
> org.elasticsearch.search.query.QueryParseElement.parse(QueryParseElement.java:33)
>
>  at
> org.elasticsearch.search.SearchService.parseSource(SearchService.java:622)
>
>
> Any help would be appreciated.
>
> Thanks
> Srini
>
>  --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/16a38015-eaed-45af-9ad3-c474521b81a8%40googlegroups.com
> .
> For more options, visit https://groups.google.com/d/optout.
>



-- 
Luiz Guilherme P. Santos

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAMdL%3DZGr12cwOJ_3DXUfPFefn_jmWhbj95wXw%3D2UZVQiZkTE1Q%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Error in query string

[Luiz Guilherme Pais dos Santos]

What happened when you executed the command, an exception? Which? No
results? Is there any documents indexed in elasticsearch? Please be more
clear about the problem.


On Thu, Apr 3, 2014 at 3:37 PM, Mohit Anchlia wrote:

> That doesn't work either. I am trying to execute this command from here:
>
>
> http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-query-string-query.html
>
> On Thu, Apr 3, 2014 at 11:33 AM, Luiz Guilherme Pais dos Santos <
> luizgpsan...@gmail.com> wrote:
>
>> Hi Mohit,
>>
>> In the exception raised it says: No parser for element [query_string]
>> which means elasticsearch don't recognize "query_string" as the root of
>> JSON. In this case you must start with "query". Look:
>>
>> curl -XPOST "http://localhost:9200/users/_search"; -d'
>> {
>> "query": {
>> "query_string": {
>>"default_field": "title",
>>"query": "this AND that OR thus"
>> }
>> }
>> }'
>>
>>  Cheers,
>> Luiz Guilherme
>>
>>
>>
>> On Thu, Apr 3, 2014 at 3:26 PM, Mohit Anchlia wrote:
>>
>>> I am just trying to execute a simple query string and I am getting this
>>> error:
>>>
>>> [ec2-user@ip-10-80-140-13 ~]$ curl -XGET "
>>> http://10.80.140.13:9200/users/_search"; -d'
>>> > {
>>> > "query_string" : {
>>> > "query" : "this AND that OR thus"
>>> > }
>>> > }'
>>> {"error":"SearchPhaseExecutionException[Failed to execute phase [query],
>>> all shards failed; shardFailures {[5ay80g1HTAinyRZBzEZHTg][users][2]:
>>> SearchParseException[[users][2]: from[-1],size[-1]: Parse Failure [Failed
>>> to parse source [\n{\n\"query_string\" : {\n\"query\" : \"this
>>> AND that OR thus\"\n}\n}]]]; nested: SearchParseException[[users][2]:
>>> from[-1],size[-1]: Parse Failure [No parser for element [query_string]]];
>>> }{[Eybinjt4Sl21p1AujiwiaA][users][3]:
>>> RemoteTransportException[[10.80.140.58][inet[/10.80.140.58:9300]][search/phase/query]];
>>> nested: SearchParseException[[users][3]: from[-1],size[-1]: Parse Failure
>>> [Failed to parse source [\n{\n\"query_string\" : {\n\"query\" :
>>> \"this AND that OR thus\"\n}\n}]]]; nested:
>>> SearchParseException[[users][3]: from[-1],size[-1]: Parse Failure [No
>>> parser for element [query_string]]]; }{[Eybinjt4Sl21p1AujiwiaA][users][0]:
>>> RemoteTransportException[[10.80.140.58][inet[/10.80.140.58:9300]][search/phase/query]];
>>> nested: SearchParseException[[users][0]: from[-1],size[-1]: Parse Failure
>>> [Failed to parse source [\n{\n\"query_string\" : {\n\"query\" :
>>> \"this AND that OR thus\"\n}\n}]]]; nested:
>>> SearchParseException[[users][0]: from[-1],size[-1]: Parse Failure [No
>>> parser for element [query_string]]]; }{[LrIIW8-MTr6-qoWXk-Xl6A][users][1]:
>>> RemoteTransportException[[10.80.140.14][inet[/10.80.140.14:9300]][search/phase/query]];
>>> nested: SearchParseException[[users][1]: from[-1],size[-1]: Parse Failure
>>> [Failed to parse source [\n{\n\"query_string\" : {\n\"query\" :
>>> \"this AND that OR thus\"\n}\n}]]]; nested:
>>> SearchParseException[[users][1]: from[-1],size[-1]: Parse Failure [No
>>> parser for element [query_string]]]; }{[LrIIW8-MTr6-qoWXk-Xl6A][users][4]:
>>> RemoteTransportException[[10.80.140.14][inet[/10.80.140.14:9300]][search/phase/query]];
>>> nested: SearchParseException[[users][4]: from[-1],size[-1]: Parse Failure
>>> [Failed to parse source [\n{\n\"query_string\" : {\n\"query\" :
>>> \"this AND that OR thus\"\n}\n}]]]; nested:
>>> SearchParseException[[users][4]: from[-1],size[-1]: Parse Failure [No
>>> parser for element [query_string]]];
>>> }]","status":400}[ec2-user@ip-10-80-140-13 ~]$
>>> [ec2-user@ip-10-80-140-13 ~]$
>>> [ec2-user@ip-10-80-140-13 ~]$
>>>
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "elasticsearch" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to elasticsearch+unsubscr...@google

Re: Error in query string

[Luiz Guilherme Pais dos Santos]

Hi Mohit,

In the exception raised it says: No parser for element [query_string] which
means elasticsearch don't recognize "query_string" as the root of JSON. In
this case you must start with "query". Look:

curl -XPOST "http://localhost:9200/users/_search"; -d'
{
"query": {
"query_string": {
   "default_field": "title",
   "query": "this AND that OR thus"
}
}
}'

Cheers,
Luiz Guilherme



On Thu, Apr 3, 2014 at 3:26 PM, Mohit Anchlia wrote:

> I am just trying to execute a simple query string and I am getting this
> error:
>
> [ec2-user@ip-10-80-140-13 ~]$ curl -XGET "
> http://10.80.140.13:9200/users/_search"; -d'
> > {
> > "query_string" : {
> > "query" : "this AND that OR thus"
> > }
> > }'
> {"error":"SearchPhaseExecutionException[Failed to execute phase [query],
> all shards failed; shardFailures {[5ay80g1HTAinyRZBzEZHTg][users][2]:
> SearchParseException[[users][2]: from[-1],size[-1]: Parse Failure [Failed
> to parse source [\n{\n\"query_string\" : {\n\"query\" : \"this
> AND that OR thus\"\n}\n}]]]; nested: SearchParseException[[users][2]:
> from[-1],size[-1]: Parse Failure [No parser for element [query_string]]];
> }{[Eybinjt4Sl21p1AujiwiaA][users][3]:
> RemoteTransportException[[10.80.140.58][inet[/10.80.140.58:9300]][search/phase/query]];
> nested: SearchParseException[[users][3]: from[-1],size[-1]: Parse Failure
> [Failed to parse source [\n{\n\"query_string\" : {\n\"query\" :
> \"this AND that OR thus\"\n}\n}]]]; nested:
> SearchParseException[[users][3]: from[-1],size[-1]: Parse Failure [No
> parser for element [query_string]]]; }{[Eybinjt4Sl21p1AujiwiaA][users][0]:
> RemoteTransportException[[10.80.140.58][inet[/10.80.140.58:9300]][search/phase/query]];
> nested: SearchParseException[[users][0]: from[-1],size[-1]: Parse Failure
> [Failed to parse source [\n{\n\"query_string\" : {\n\"query\" :
> \"this AND that OR thus\"\n}\n}]]]; nested:
> SearchParseException[[users][0]: from[-1],size[-1]: Parse Failure [No
> parser for element [query_string]]]; }{[LrIIW8-MTr6-qoWXk-Xl6A][users][1]:
> RemoteTransportException[[10.80.140.14][inet[/10.80.140.14:9300]][search/phase/query]];
> nested: SearchParseException[[users][1]: from[-1],size[-1]: Parse Failure
> [Failed to parse source [\n{\n\"query_string\" : {\n\"query\" :
> \"this AND that OR thus\"\n}\n}]]]; nested:
> SearchParseException[[users][1]: from[-1],size[-1]: Parse Failure [No
> parser for element [query_string]]]; }{[LrIIW8-MTr6-qoWXk-Xl6A][users][4]:
> RemoteTransportException[[10.80.140.14][inet[/10.80.140.14:9300]][search/phase/query]];
> nested: SearchParseException[[users][4]: from[-1],size[-1]: Parse Failure
> [Failed to parse source [\n{\n\"query_string\" : {\n\"query\" :
> \"this AND that OR thus\"\n}\n}]]]; nested:
> SearchParseException[[users][4]: from[-1],size[-1]: Parse Failure [No
> parser for element [query_string]]];
> }]","status":400}[ec2-user@ip-10-80-140-13 ~]$
> [ec2-user@ip-10-80-140-13 ~]$
> [ec2-user@ip-10-80-140-13 ~]$
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/CAOT3TWqrnc3hb9bJZqhNJBTknvX1%3D64uKwW54sSdvT1HAMykoA%40mail.gmail.com
> .
> For more options, visit https://groups.google.com/d/optout.
>



-- 
Luiz Guilherme P. Santos

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAMdL%3DZGnvYGrhyjyKjAroM57j7pmuUVFgviARE713xSVF_HezQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Plugin development - Get Action problem

[Luiz Guilherme Pais dos Santos]

I'm trying to migrate from 0.20 to 0.90.

The changes I made to get it compile are in my forked repo:
https://github.com/luizgpsantos/elasticsearch-view-plugin/blob/master/src/main/java/org/elasticsearch/action/view/TransportViewAction.java#L56

The specific commit is:
https://github.com/luizgpsantos/elasticsearch-view-plugin/commit/fc041baa6a039a71a87c1e33f36b887b0b803479




On Sat, Mar 8, 2014 at 2:56 PM, Ivan Brusic  wrote:

> When I saw the last commit of the plugin was March 3rd, I assumed it was
> this year and you were trying to backport the plugin to 0.90. Now I
> understand that you are trying to upgrade from 0.20, which indeed involve
> an upgrade in Lucene.
>
> Perhaps if you shared the TransportViewAction code, we can help. The
> change might be minor.
>
> Cheers,
>
> Ivan
>
>
> On Sat, Mar 8, 2014 at 7:34 AM, Luiz Guilherme Pais dos Santos <
> luizgpsan...@gmail.com> wrote:
>
>> Hi Jörg,
>>
>> After your e-mail I tried to complete rewrite my "TransportViewAction"
>> based in the TransportGetAction. But it still don't work LOL
>>
>> There might have something trick that I couldn't guess.
>>
>>
>>
>> On Fri, Mar 7, 2014 at 10:47 PM, joergpra...@gmail.com <
>> joergpra...@gmail.com> wrote:
>>
>>> Lucene 4 has changed in many ways. Renaming does not help.
>>>
>>> Look at TransportGetAction to implement a correct GetResult:
>>>
>>>
>>> https://github.com/elasticsearch/elasticsearch/blob/master/src/main/java/org/elasticsearch/action/get/TransportGetAction.java#L101
>>>
>>> Jörg
>>>
>>>
>>> On Fri, Mar 7, 2014 at 10:15 PM, Luiz Guilherme Pais dos Santos <
>>> luizgpsan...@gmail.com> wrote:
>>>
>>>>
>>>>
>>>> It might be some break change in ShardGetService but I still couldn't
>>>> find it :(
>>>>
>>>>
>>>>  --
>>> You received this message because you are subscribed to the Google
>>> Groups "elasticsearch" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to elasticsearch+unsubscr...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGfw%2BbkDsU01XiSwcrW5%3Duy5MC8CDKf8jx3Aq%3DhA-TGyQ%40mail.gmail.com<https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGfw%2BbkDsU01XiSwcrW5%3Duy5MC8CDKf8jx3Aq%3DhA-TGyQ%40mail.gmail.com?utm_medium=email&utm_source=footer>
>>> .
>>>
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>
>>
>> --
>> Luiz Guilherme P. Santos
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to elasticsearch+unsubscr...@googlegroups.com.
>>  To view this discussion on the web visit
>> https://groups.google.com/d/msgid/elasticsearch/CAMdL%3DZGKqC%3DU6-xq_T7ntzVVOEfYzOiBX5wit0MwPYWcbbKqjw%40mail.gmail.com<https://groups.google.com/d/msgid/elasticsearch/CAMdL%3DZGKqC%3DU6-xq_T7ntzVVOEfYzOiBX5wit0MwPYWcbbKqjw%40mail.gmail.com?utm_medium=email&utm_source=footer>
>> .
>>
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>  --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQCDfusua8im2CinkTQ7%3D7jqvc5_3_nk9HBJeRetmiWK7A%40mail.gmail.com<https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQCDfusua8im2CinkTQ7%3D7jqvc5_3_nk9HBJeRetmiWK7A%40mail.gmail.com?utm_medium=email&utm_source=footer>
> .
>
> For more options, visit https://groups.google.com/d/optout.
>



-- 
Luiz Guilherme P. Santos

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAMdL%3DZEi8YY0LnMW5S6veke24CLe29bvECQjFXoAdumDRJ_XCg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Plugin development - Get Action problem

[Luiz Guilherme Pais dos Santos]

Hi Jörg,

After your e-mail I tried to complete rewrite my "TransportViewAction"
based in the TransportGetAction. But it still don't work LOL

There might have something trick that I couldn't guess.



On Fri, Mar 7, 2014 at 10:47 PM, joergpra...@gmail.com <
joergpra...@gmail.com> wrote:

> Lucene 4 has changed in many ways. Renaming does not help.
>
> Look at TransportGetAction to implement a correct GetResult:
>
>
> https://github.com/elasticsearch/elasticsearch/blob/master/src/main/java/org/elasticsearch/action/get/TransportGetAction.java#L101
>
> Jörg
>
>
> On Fri, Mar 7, 2014 at 10:15 PM, Luiz Guilherme Pais dos Santos <
> luizgpsan...@gmail.com> wrote:
>
>>
>>
>> It might be some break change in ShardGetService but I still couldn't
>> find it :(
>>
>>
>>  --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGfw%2BbkDsU01XiSwcrW5%3Duy5MC8CDKf8jx3Aq%3DhA-TGyQ%40mail.gmail.com<https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGfw%2BbkDsU01XiSwcrW5%3Duy5MC8CDKf8jx3Aq%3DhA-TGyQ%40mail.gmail.com?utm_medium=email&utm_source=footer>
> .
>
> For more options, visit https://groups.google.com/d/optout.
>



-- 
Luiz Guilherme P. Santos

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAMdL%3DZGKqC%3DU6-xq_T7ntzVVOEfYzOiBX5wit0MwPYWcbbKqjw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Plugin development - Get Action problem

[Luiz Guilherme Pais dos Santos]

Hi Ivan,

My first attempt was with indices created with older versions of Lucene but
after your email I create a new index to test but the problem still occurs.

In TransportViewAction I just had to change the name of the methods, for
example,

instead of:
viewContext.index(getResult.index()).type(getResult.type()).id(getResult.id()).version(getResult.version())

I used:
viewContext.index(getResult.getIndex()).type(getResult.getType()).id(getResult.getId()).version(getResult.getVersion())

It might be some break change in ShardGetService but I still couldn't find
it :(


On Fri, Mar 7, 2014 at 2:13 PM, Ivan Brusic  wrote:

> Are you trying to read indices written with older versions of Lucene? What
> changes did you need to make to TransportViewAction? I have never used the
> plugin, but perhaps it is a question of a field being stored or not.
>
> Looking briefly at the code, perhaps there was some breaking change
> in ShardGetService.
>
>
>
>
>
>
>  On Thu, Mar 6, 2014 at 5:48 PM, Luiz Guilherme Pais dos Santos <
> luizgpsan...@gmail.com> wrote:
>
>>  Hi,
>>
>> I'm trying to make elasticsearch-view-plugin (
>> https://github.com/tlrx/elasticsearch-view-plugin) to work with the
>> 0.90.* version of elasticsearch (
>> https://github.com/tlrx/elasticsearch-view-plugin/pull/1) and after few
>> changes to make it compile it started to work on localhost.
>>
>> Unfortunately, after deploying in a cluster with four nodes the requests
>> are not working when the load balance deliver the request to the nodes
>> without the shards the the document is stored.
>>
>> The exception and the documents I used to test are listed in this gist:
>> https://gist.github.com/luizgpsantos/ccb4ff9a4ebb99153dc9
>>
>> It seems that for any reason the type or the id of the document is null
>> when the method org.elasticsearch.index.mapper.Uid.createUidAsByte is
>> executed.
>>
>> Is there anyone who knows the internals of elasticsearch that could help
>> me with this problem?
>>
>> Thank you!
>>
>> --
>> Luiz Guilherme P. Santos
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to elasticsearch+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/elasticsearch/CAMdL%3DZGrnu47vNqkx-OdWT2BwfVoPrCsmvM1crKmXDJ-PM_waA%40mail.gmail.com<https://groups.google.com/d/msgid/elasticsearch/CAMdL%3DZGrnu47vNqkx-OdWT2BwfVoPrCsmvM1crKmXDJ-PM_waA%40mail.gmail.com?utm_medium=email&utm_source=footer>
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>  --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQDXQTg8YeV4h7aHfVcL0OrHoDwM%2B410%3DTOxa-ZiKg8tHA%40mail.gmail.com<https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQDXQTg8YeV4h7aHfVcL0OrHoDwM%2B410%3DTOxa-ZiKg8tHA%40mail.gmail.com?utm_medium=email&utm_source=footer>
> .
> For more options, visit https://groups.google.com/d/optout.
>



-- 
Luiz Guilherme P. Santos

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAMdL%3DZGqaZspoJmGXOG0MVA4YwMW8Xzen8h_OH%3DuCcC4rWtKxg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Plugin development - Get Action problem

[Luiz Guilherme Pais dos Santos]

Hi,

I'm trying to make elasticsearch-view-plugin (
https://github.com/tlrx/elasticsearch-view-plugin) to work with the 0.90.*
version of elasticsearch (
https://github.com/tlrx/elasticsearch-view-plugin/pull/1) and after few
changes to make it compile it started to work on localhost.

Unfortunately, after deploying in a cluster with four nodes the requests
are not working when the load balance deliver the request to the nodes
without the shards the the document is stored.

The exception and the documents I used to test are listed in this gist:
https://gist.github.com/luizgpsantos/ccb4ff9a4ebb99153dc9

It seems that for any reason the type or the id of the document is null
when the method org.elasticsearch.index.mapper.Uid.createUidAsByte is
executed.

Is there anyone who knows the internals of elasticsearch that could help me
with this problem?

Thank you!

-- 
Luiz Guilherme P. Santos

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAMdL%3DZGrnu47vNqkx-OdWT2BwfVoPrCsmvM1crKmXDJ-PM_waA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Compute TF/IDF across indexes

[Luiz Guilherme Pais dos Santos]

Hi Ivan,

The DFS query then fetch worked very well!

Thank you!

Cheers,
Luiz Guilherme


On Tue, Feb 25, 2014 at 5:15 PM, Ivan Brusic  wrote:

> I have never tried or looked at the code, but off the top of my head
> perhaps the DFS query type would work:
> http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-search-type.html#dfs-query-then-fetch
>
> Since the DFS query type calculates the TF/IDF values based on the values
> in each individual shard, perhaps it ignores which index the shard belongs
> to. Easy to test.
>
> If not, the solution might be tricky. You can eliminate term length
> normalization, but your issue is with the IDF. You can create your own
> Similarity, but the best you can do is ignore the IDF, which probably would
> not be ideal.
>
> Ultimately, you can try script based scoring. The TF/IDF values are
> exposed to the scripts, so you can try to apply some type of normalization
> yourself. Kludgy and it would impact performance.
>
>
> http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-advanced-scripting.html
>
> Hopefully DFS queries would work or someone else has a better idea!
>
> Cheers,
>
> Ivan
>
>
>  On Tue, Feb 25, 2014 at 12:00 PM, Luiz Guilherme Pais dos Santos <
> luizgpsan...@gmail.com> wrote:
>
>>  Hi,
>>
>> I'm trying to search across multiple indexes and I couldn't understand
>> the result of the TF/TDF function. I didn't expect for the indexes where
>> the term is more frequent to get penalized.
>>
>> Here follows an example:
>> https://gist.github.com/luizgpsantos/9216108
>>
>> When searching for the term "alice" the document {"_index": "index2",
>> "_type": "type", "_id": "1"} got a score 0.8784157 while {"_index":
>> "index1", "_type": "type", "_id": "1"} got a score 0.4451987.
>>
>> In my use case I got one index about sports and another about celebrities
>> and when I search for a celebrity documents across sports and celebrities
>> indexes, results from sports index tend to appear in first place due to the
>> explanation above (we have few celebrities documents in sports index). But
>> the point is that when searching for a celebrity I would expect results
>> from the celebrity index.
>>
>> Is there any way to calculate the score not penalizing indexes where the
>> frequency of a term is higher?
>>
>> Cheers,
>>
>> --
>> Luiz Guilherme P. Santos
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to elasticsearch+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/elasticsearch/CAMdL%3DZGe4ywgNX0JaBjQQ0HAc9_CQ-iz0trZ7vbqT4CVvizmpQ%40mail.gmail.com
>> .
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>
>  --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQDgREX6svvcso%2Bf6VqW2Y6-DvBnWUtO5tVod8GAX2b0Bw%40mail.gmail.com
> .
> For more options, visit https://groups.google.com/groups/opt_out.
>



-- 
Luiz Guilherme P. Santos

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAMdL%3DZGLPTbZgwyoBARjwcg9v0sUsjuxw4m_6X1iFQqO6zTHaQ%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Compute TF/IDF across indexes

[Luiz Guilherme Pais dos Santos]

Hi,

I'm trying to search across multiple indexes and I couldn't understand the
result of the TF/TDF function. I didn't expect for the indexes where the
term is more frequent to get penalized.

Here follows an example:
https://gist.github.com/luizgpsantos/9216108

When searching for the term "alice" the document {"_index": "index2",
"_type": "type", "_id": "1"} got a score 0.8784157 while {"_index": "index1",
"_type": "type", "_id": "1"} got a score 0.4451987.

In my use case I got one index about sports and another about celebrities
and when I search for a celebrity documents across sports and celebrities
indexes, results from sports index tend to appear in first place due to the
explanation above (we have few celebrities documents in sports index). But
the point is that when searching for a celebrity I would expect results
from the celebrity index.

Is there any way to calculate the score not penalizing indexes where the
frequency of a term is higher?

Cheers,

-- 
Luiz Guilherme P. Santos

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAMdL%3DZGe4ywgNX0JaBjQQ0HAc9_CQ-iz0trZ7vbqT4CVvizmpQ%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.