Re: elasticsearch how to display the csv data

[David Pilato]

Try sense https://github.com/bleskes/sense

--
David ;-)
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs


Le 29 janv. 2014 à 07:57, Navneet Mathpal  a écrit :

it was a csv file and i have imported it.there is field called first_name.

it showing the correct result in cygwin:- $  curl -XGET 
'http://localhost:9200/my_csv_data/csv_row/_search?q=first_name:Aaron' but not 
in head-master

> On Wednesday, 29 January 2014 12:09:30 UTC+5:30, David Pilato wrote:
> How does your document look like?
> How did you index it?
> 
> --
> David ;-)
> Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs
> 
> 
> Le 29 janv. 2014 à 07:29, Navneet Mathpal  a écrit :
> 
> hi,
> 
> I an doing query like this 
> 
> my_csv_data/csv_row/_search
> 
> {
>   "field": {
> "first_name": "Aaron"
>   }
> }
> 
> 
> 
> the name Aaron are not getting diplayed?? why?
> -- 
> You received this message because you are subscribed to the Google Groups 
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to elasticsearc...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/elasticsearch/8ec59e3a-85fd-4487-8b7b-3eaf2d9528da%40googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/f4999a9d-1f64-4fad-a946-982b700716ae%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/402F3229-6BAE-45FE-8A36-0FC0C28FE1DD%40pilato.fr.
For more options, visit https://groups.google.com/groups/opt_out.

Re: elasticsearch how to display the csv data

[Navneet Mathpal]

it was a csv file and i have imported it.there is field called first_name.

it showing the correct result in cygwin:- $  curl -XGET 
'http://localhost:9200/my_csv_data/csv_row/_search?q=first_name:Aaron' but 
not in head-master

On Wednesday, 29 January 2014 12:09:30 UTC+5:30, David Pilato wrote:
>
> How does your document look like?
> How did you index it?
>
> --
> David ;-)
> Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs
>
>
> Le 29 janv. 2014 à 07:29, Navneet Mathpal 
> > 
> a écrit :
>
> hi,
>
> I an doing query like this 
>
> my_csv_data/csv_row/_search
>
> {
>   "field": {
> "first_name": "Aaron"
>   }
> }
>
>
>
> the name Aaron are not getting diplayed?? why?
>
> -- 
> You received this message because you are subscribed to the Google Groups 
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to elasticsearc...@googlegroups.com .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/elasticsearch/8ec59e3a-85fd-4487-8b7b-3eaf2d9528da%40googlegroups.com
> .
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/f4999a9d-1f64-4fad-a946-982b700716ae%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Query Time Analysis: Are field value also analyzed?

[Jayesh Bhoyar]

Hi Karan,

Can you please tell us what mapping you have applied?

If you are applying EdgeNGram in Query time Analyzer then your search query 
"harry g" will get tokenized as per your custom analyser.

Regards,
Jayesh Bhoyar

On Wednesday, January 29, 2014 9:51:08 AM UTC+5:30, Karan Verma wrote:
>
> Hi 
>
> Lets say I have indexed a field person_name as a string, with a custom 
> analyzer. person_name is stored int the index in one of the documents as: 
> "Harry Greenberg" 
>
> I make a *match query* on the field : "harry g"
>
> I have a custom edgengram tokenizer which breaks the query down as 
> follows: 
>
> {
> "tokens": [
> {
> "token": "h",
> "start_offset": 0,
> "end_offset": 1,
> "type": "word",
> "position": 1
> },
> {
> "token": "ha",
> "start_offset": 0,
> "end_offset": 2,
> "type": "word",
> "position": 2
> },
> {
> "token": "har",
> "start_offset": 0,
> "end_offset": 3,
> "type": "word",
> "position": 3
> },
> {
> "token": "harr",
> "start_offset": 0,
> "end_offset": 4,
> "type": "word",
> "position": 4
> },
> {
> "token": "harry",
> "start_offset": 0,
> "end_offset": 5,
> "type": "word",
> "position": 5
> },
> {
> "token": "g",
> "start_offset": 6,
> "end_offset": 7,
> "type": "word",
> "position": 6
> }
> ]
> }
>
>
> Will all of these tokens be matched agains "Harry Greenberg" or 
> person_name will also be broken down as defined by my custom analyzer? 
>
>
> If not, how can I make it so that it will also be broken down? Will it 
> make the search significantly slower? 
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/7301bfe9-ae7c-48ca-af38-ed369e7cc78d%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: elasticsearch how to display the csv data

[David Pilato]

How does your document look like?
How did you index it?

--
David ;-)
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs


Le 29 janv. 2014 à 07:29, Navneet Mathpal  a écrit :

hi,

I an doing query like this 

my_csv_data/csv_row/_search

{
  "field": {
"first_name": "Aaron"
  }
}



the name Aaron are not getting diplayed?? why?
-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/8ec59e3a-85fd-4487-8b7b-3eaf2d9528da%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/48B6648E-2502-4537-803A-028F9093DEDE%40pilato.fr.
For more options, visit https://groups.google.com/groups/opt_out.

elasticsearch how to display the csv data

[Navneet Mathpal]

hi,

I an doing query like this 

my_csv_data/csv_row/_search

{
  "field": {
"first_name": "Aaron"
  }
}



the name Aaron are not getting diplayed?? why?

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/8ec59e3a-85fd-4487-8b7b-3eaf2d9528da%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [RUBY] : elasticsearch-ruby : Special characters not escaped by the library

[Srirang Doddihal]

Sorry for the delayed response.


On Fri, Jan 10, 2014 at 3:43 PM, Jason Wee  wrote:

>
> http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-query-string-query.html#_reserved_characters
>
>
imho, || could means OR or literary as it is. So if the library escape for
> you, when you mean || to OR, that's wrong.
>
>
Makes sense. However a character like " (double quote - when used like :
40" tv) will make elasticsearch fail outright with the message saying that
it was unable to parse the query.

Wrong search happening is somewhat ok and can be considered as user not
knowing how to use those operators, but an explicit error is a much worse
user behaviour, isn't it?

Could the library add some such query string sanitize feature?


> hth
>
> Jason
>
>
> On Fri, Jan 10, 2014 at 8:34 AM, Srirang Doddihal 
> wrote:
>
>> Hi,
>>
>> I tried out the elasticsearch Ruby gem today and found that it does not
>> escape the reserve characters when searching with the query_string query.
>>
>> As a library providing easy to use search API, wouldn't it be better if
>> the library escaped the reserve characters in this case?
>> The API can support a flag,  with a sensible default value, to enable or
>> disable this escaping behavior.
>>
>> Or is it an explicit design decision that the users themselves have to
>> escape the reserve characters before sending it to this library?
>>
>> I am using v0.4.5.
>>
>> Regards,
>> Brahmana
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to elasticsearch+unsubscr...@googlegroups.com.
>>
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/elasticsearch/0bc324f1-dc81-4640-aea0-1a3b08663f20%40googlegroups.com
>> .
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>
>  --
> You received this message because you are subscribed to a topic in the
> Google Groups "elasticsearch" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/elasticsearch/QsVgeOywOkk/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/CAHO4itzMkB6Jv0EkfiXSB7528EPVN4Zwtp96bs5Of93mk%2BRnFg%40mail.gmail.com
> .
>
> For more options, visit https://groups.google.com/groups/opt_out.
>



-- 
Regards,
Srirang G Doddihal
Brahmana.

The LIGHT shows the way.
The WISE see it.
The BRAVE walk it.
The PERSISTENT endure and complete it.

I want to do it all ALONE.

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CACBGNLt6OkUdEFN%2BFffc-AxfrrbV-2%3Difq5uWMQ1c0h3KsEgbw%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

NullPointerException in Elasticsearch 0.90.10 when searching with a nested filter in the parent document

[seallison]

I'm getting a NullPointerException in Elasticsearch 0.90.10 when searching 
using a nested filter on the parent document.

The exception is:

[2014-01-28 23:25:37,483][DEBUG][action.search.type   ] [Gorilla-Man] 
[3] Failed to execute fetch phase
java.lang.NullPointerException
at 
org.elasticsearch.index.search.child.ParentConstantScoreQuery$ChildrenWeight$ChildrenDocIdIterator.match(ParentConstantScoreQuery.java:176)
at 
org.apache.lucene.search.FilteredDocIdSetIterator.advance(FilteredDocIdSetIterator.java:71)
at 
org.elasticsearch.index.search.child.ConstantScorer.advance(ConstantScorer.java:70)
at 
org.apache.lucene.search.ConstantScoreQuery$ConstantScorer.advance(ConstantScoreQuery.java:217)
at 
org.apache.lucene.search.ConstantScoreQuery$ConstantWeight.explain(ConstantScoreQuery.java:166)
at org.apache.lucene.search.FilteredQuery$1.explain(FilteredQuery.java:102)
at org.apache.lucene.search.IndexSearcher.explain(IndexSearcher.java:672)
at org.apache.lucene.search.IndexSearcher.explain(IndexSearcher.java:652)
at 
org.elasticsearch.search.internal.ContextIndexSearcher.explain(ContextIndexSearcher.java:192)
at 
org.elasticsearch.search.fetch.explain.ExplainFetchSubPhase.hitExecute(ExplainFetchSubPhase.java:72)
at org.elasticsearch.search.fetch.FetchPhase.execute(FetchPhase.java:197)
at 
org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.java:434)
at 
org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteFetch(SearchServiceTransportAction.java:406)
at 
org.elasticsearch.action.search.type.TransportSearchDfsQueryThenFetchAction$AsyncAction.executeFetch(TransportSearchDfsQueryThenFetchAction.java:249)
at 
org.elasticsearch.action.search.type.TransportSearchDfsQueryThenFetchAction$AsyncAction$5.run(TransportSearchDfsQueryThenFetchAction.java:233)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)

I've created a gist with a sample mapping and how I am building the nested 
filter: https://gist.github.com/seallison/8681846

I know this worked in a previous version of Elasticsearch (maybe 0.90.5?) I 
can't remember.

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/f2768bb8-20e9-44cc-af56-e69e802fb0c0%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: For those wanting to try Marvel

[Roy Russo]

Although I agree with the code transparency in principle, it isn't a sound 
business idea. Scaling an OSS company on support/services revenue alone is 
very linear and that's not something investors want. You also risk becoming 
a consulting company, rather than a software distributor. 

In the end, all successful OSS companies have a commercial component. This 
is ES's first stab at it and I think it's a good one. 



On Tuesday, January 28, 2014 11:02:50 PM UTC-5, Lukáš Vlček wrote:
>
> If my memory serves me well one of the issues with plugins is missing 
> classloading isolation. If you have two plugins installed on a single node 
> they can clash (we have been hit by this with our own plugins and it took 
> us some time to figure it out because the symptom was not deterministic). 
> If Marvel contains Java code that is installed as a plugin it would be imo 
> useful if the code were a bit more transparent, especially for plugin 
> developers.
>
> Regards,
> Lukáš
>
>
> On Wed, Jan 29, 2014 at 2:42 AM, Ivan Brusic 
> > wrote:
>
>> Marvel doesn't different much from elasticsearch plugins in that the code 
>> now runs in the same JVM instead of a separate process. The event data is 
>> pushed rather than pulled. It is great not having to re-invent the wheel, 
>> but having monitoring outside of elasticsearch is not an issue. Great 
>> observation about the tribe node BTW.
>>
>> Given that efficient JSON parsing libraries exist in most languages, I 
>> rather go that route over the cat API. Time to re-visit monitoring.
>>
>> Ivan
>>  
>>
>> On Tue, Jan 28, 2014 at 5:33 PM, Mark Walkom 
>> 
>> > wrote:
>>
>>> Or perhaps something could come from the new cat API if they don't want 
>>> to go that route.
>>>
>>> Regards,
>>> Mark Walkom
>>>
>>> Infrastructure Engineer
>>> Campaign Monitor
>>> email: ma...@campaignmonitor.com 
>>> web: www.campaignmonitor.com
>>>  
>>>
>>> On 29 January 2014 12:21, joerg...@gmail.com  <
>>> joerg...@gmail.com > wrote:
>>>
 Marvel comes with hidden Java plugin code for an event pusher. Node 
 events, route events, and shard events of ES can be indexed into ES. Very 
 useful for historic analysis and post mortem views.

 It seems this was also a motivation for the tribe node mode: grow two 
 separate clusters, one for the data, another one for the metrics.

 It would be nice if also the event pushing source code could be opened, 
 so other monitoring tools are able to build on this facility too. Or at 
 least documenting the event pushing API, for re-implementing it from 
 scratch.

 Jörg

  -- 
 You received this message because you are subscribed to the Google 
 Groups "elasticsearch" group.
 To unsubscribe from this group and stop receiving emails from it, send 
 an email to elasticsearc...@googlegroups.com .
 To view this discussion on the web visit 
 https://groups.google.com/d/msgid/elasticsearch/CAKdsXoG-JaWZU4qFQsUaM_UHCwkzMdYnLruUHNmcnkiZ-wX6jA%40mail.gmail.com
 .
 For more options, visit https://groups.google.com/groups/opt_out.

>>>
>>>  -- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "elasticsearch" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to elasticsearc...@googlegroups.com .
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/elasticsearch/CAEM624Yn5y1bxGYd%3DjeG%3DiUVzowhOsuzKi_8erKrPEmTNbxb%3DQ%40mail.gmail.com
>>> .
>>>
>>> For more options, visit https://groups.google.com/groups/opt_out.
>>>
>>
>>  -- 
>> You received this message because you are subscribed to the Google Groups 
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to elasticsearc...@googlegroups.com .
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQD%3Dh804DJBr2J8V3gDvkSK0ddXHvg0xDBf9XYzUza_n5w%40mail.gmail.com
>> .
>>
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/6de0c52f-f0f8-45d5-b19d-32c149bc1fe2%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Query Time Analysis: Are field value also analyzed?

[Karan Verma]

Hi 

Lets say I have indexed a field person_name as a string, with a custom 
analyzer. person_name is stored int the index in one of the documents as: 
"Harry Greenberg" 

I make a *match query* on the field : "harry g"

I have a custom edgengram tokenizer which breaks the query down as follows: 

{
"tokens": [
{
"token": "h",
"start_offset": 0,
"end_offset": 1,
"type": "word",
"position": 1
},
{
"token": "ha",
"start_offset": 0,
"end_offset": 2,
"type": "word",
"position": 2
},
{
"token": "har",
"start_offset": 0,
"end_offset": 3,
"type": "word",
"position": 3
},
{
"token": "harr",
"start_offset": 0,
"end_offset": 4,
"type": "word",
"position": 4
},
{
"token": "harry",
"start_offset": 0,
"end_offset": 5,
"type": "word",
"position": 5
},
{
"token": "g",
"start_offset": 6,
"end_offset": 7,
"type": "word",
"position": 6
}
]
}


Will all of these tokens be matched agains "Harry Greenberg" or person_name 
will also be broken down as defined by my custom analyzer? 


If not, how can I make it so that it will also be broken down? Will it make 
the search significantly slower? 

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/af5354e7-5f7b-4b6e-96e6-f5e81df825db%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: For those wanting to try Marvel

[Lukáš Vlček]

If my memory serves me well one of the issues with plugins is missing
classloading isolation. If you have two plugins installed on a single node
they can clash (we have been hit by this with our own plugins and it took
us some time to figure it out because the symptom was not deterministic).
If Marvel contains Java code that is installed as a plugin it would be imo
useful if the code were a bit more transparent, especially for plugin
developers.

Regards,
Lukáš


On Wed, Jan 29, 2014 at 2:42 AM, Ivan Brusic  wrote:

> Marvel doesn't different much from elasticsearch plugins in that the code
> now runs in the same JVM instead of a separate process. The event data is
> pushed rather than pulled. It is great not having to re-invent the wheel,
> but having monitoring outside of elasticsearch is not an issue. Great
> observation about the tribe node BTW.
>
> Given that efficient JSON parsing libraries exist in most languages, I
> rather go that route over the cat API. Time to re-visit monitoring.
>
> Ivan
>
>
> On Tue, Jan 28, 2014 at 5:33 PM, Mark Walkom wrote:
>
>> Or perhaps something could come from the new cat API if they don't want
>> to go that route.
>>
>> Regards,
>> Mark Walkom
>>
>> Infrastructure Engineer
>> Campaign Monitor
>> email: ma...@campaignmonitor.com
>> web: www.campaignmonitor.com
>>
>>
>> On 29 January 2014 12:21, joergpra...@gmail.com wrote:
>>
>>> Marvel comes with hidden Java plugin code for an event pusher. Node
>>> events, route events, and shard events of ES can be indexed into ES. Very
>>> useful for historic analysis and post mortem views.
>>>
>>> It seems this was also a motivation for the tribe node mode: grow two
>>> separate clusters, one for the data, another one for the metrics.
>>>
>>> It would be nice if also the event pushing source code could be opened,
>>> so other monitoring tools are able to build on this facility too. Or at
>>> least documenting the event pushing API, for re-implementing it from
>>> scratch.
>>>
>>> Jörg
>>>
>>>  --
>>> You received this message because you are subscribed to the Google
>>> Groups "elasticsearch" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to elasticsearch+unsubscr...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/elasticsearch/CAKdsXoG-JaWZU4qFQsUaM_UHCwkzMdYnLruUHNmcnkiZ-wX6jA%40mail.gmail.com
>>> .
>>> For more options, visit https://groups.google.com/groups/opt_out.
>>>
>>
>>  --
>> You received this message because you are subscribed to the Google Groups
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to elasticsearch+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/elasticsearch/CAEM624Yn5y1bxGYd%3DjeG%3DiUVzowhOsuzKi_8erKrPEmTNbxb%3DQ%40mail.gmail.com
>> .
>>
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>
>  --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQD%3Dh804DJBr2J8V3gDvkSK0ddXHvg0xDBf9XYzUza_n5w%40mail.gmail.com
> .
>
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAO9cvUbtaLcYfteU3CO8EWL4oP36OR86cE4Rj-JFEs3SLvjWGA%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Some clarity about Update index

[Arjit Gupta]

Hi , 

Lets say I have a document {name :"Jhon"} and index is eventually 
consistent. I update the document to {name :"Tom"}. 
The lucene index is not updated. 
But In search I am getting 1 result for name : "Jhon" . 
1. What will field value for name Jhon or Tom  ? 


Thanks,
Arjit

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/4a3c2b14-ba5b-4def-8aa4-d0a97256c249%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Persisting IndicesRoutingTable

[Anantha Govindarajan]

Hi ,

In case of planned shutdown (curl -XPOST 'http://localhost:9200/_shutdown') 
, is it possible to persist the RoutingTable in master node. (In case , we 
are using separate master and data nodes.) So that when we start the 
cluster node by node (starting from master) along with 
gateway.expectednodes setting , then master node can re-use the persisted 
RoutingTable instead of constructing new one (New one causing unnecessary 
shard relocation, that too we have shards in big sizes).  Once the expected 
nodes are available in cluster reusing the same avoids unnecessary movement 
of shrads with-in cluster.


Anantha Govindarajan.




-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/47ce730a-8521-463e-82e8-8eb271548664%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

elasticsearch transport-ware version 1.8.0 -- does not create logstash indices during startup

[Mahesh Venkat]

Hi,
I am attempting to run elasticsearch 0.9.10 as a servlet 
using elasticsearch transport-ware version 1.8.0.
In this mode,  elasticsearch does not create logstash time-based indices.
When I attempt to view the log through Kibana 3, the following calls fails. 
I get a 404 error.
http://localhost:9292/logstash-2014.01.28,logstash-2014.01.29,logstash-2014.01.29/_aliases?ignore_missing=true

The same call works when I run elasticsearch as a standlone java app.
As a standalone app I see the following elasticsearch debug messages, which 
isn't generated when I run it as a servlet:
[2014-01-28 17:28:35,023][DEBUG][indices ] [allegro] creating Index 
[logstash-2014.01.29], shards [5]/[1] [2014-01-28 
17:28:35,206][DEBUG][index.mapper ] [allegro] [logstash-2014.01.29] using 
dynamic[true], default mapping: default_mapping_location[null], 
loaded_from[jar:file:/C:/elasticsearch/elasticsearch-0.90.10/lib/elasticsearch-0.90.10.jar!/org/elasticsearch/index/mapper/default-mapping.json]
 
[2014-01-28 17:28:35,208][DEBUG][index.cache.query.parser.resident] 
[allegro] [logstash-2014.01.29] using [resident] query cache with max_size 
[100], expire [null] [2014-01-28 17:28:35,214][DEBUG][index.store.fs ] 
[allegro] [logstash-2014.01.29] using index.store.throttle.type [node], 
with index.store.throttle.max_bytes_per_sec [0b] [2014-01-28 
17:28:35,266][INFO ][cluster.metadata ] [allegro] [logstash-2014.01.29] 
creating index, cause [auto(bulk api)], shards [5]/[1], mappings [] 
[2014-01-28 17:28:35,280][DEBUG][index.cache.filter.weighted] [allegro] 
[logstash-2014.01.29] full cache clear, reason [close]

any clues ? Appreciate your help

thanks
--Mahesh

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/7621e148-97c5-4882-a665-ad7fa278a5d8%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Elasticsearch java client on EC2

[Maxime Nay]

Nope, I just made a typo by translating my elasticsearch properties file 
here. :)
I use spring to inject all my properties into my Client. All the properties 
for the client are actually like this :

node.name=Elastic Search Realtime Client
node.client=true
node.data=false
cluster.name=elasticsearchrealtime

 Index 


index.number_of_shards=16
index.number_of_replicas=1

### Memory 


# ElasticSearch performs poorly when JVM starts swapping=you should ensure 
that
# it _never_ swaps.
#
# Set this property to true to lock the memory:
#
bootstrap.mlockall=true

## Discovery 
##

discovery.type=ec2
discovery.ec2.groups=elasticsearchrealtime,storm
discovery.ec2.ping_timeout=60s
discovery.ec2.any_group=true

### AWS Settings
cloud.aws.region=us-east-1
cloud.aws.access_key=
cloud.aws.secret_key=

Thanks though!

On Tuesday, January 28, 2014 5:38:01 PM UTC-8, Jörg Prante wrote:
>
> Is this a typo?
>
> discovery.type=ec2
>
> Jörg
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/f3c995a0-52c7-4984-b80f-dc0143a4abdc%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: For those wanting to try Marvel

[Ivan Brusic]

Marvel doesn't different much from elasticsearch plugins in that the code
now runs in the same JVM instead of a separate process. The event data is
pushed rather than pulled. It is great not having to re-invent the wheel,
but having monitoring outside of elasticsearch is not an issue. Great
observation about the tribe node BTW.

Given that efficient JSON parsing libraries exist in most languages, I
rather go that route over the cat API. Time to re-visit monitoring.

Ivan


On Tue, Jan 28, 2014 at 5:33 PM, Mark Walkom wrote:

> Or perhaps something could come from the new cat API if they don't want to
> go that route.
>
> Regards,
> Mark Walkom
>
> Infrastructure Engineer
> Campaign Monitor
> email: ma...@campaignmonitor.com
> web: www.campaignmonitor.com
>
>
> On 29 January 2014 12:21, joergpra...@gmail.com wrote:
>
>> Marvel comes with hidden Java plugin code for an event pusher. Node
>> events, route events, and shard events of ES can be indexed into ES. Very
>> useful for historic analysis and post mortem views.
>>
>> It seems this was also a motivation for the tribe node mode: grow two
>> separate clusters, one for the data, another one for the metrics.
>>
>> It would be nice if also the event pushing source code could be opened,
>> so other monitoring tools are able to build on this facility too. Or at
>> least documenting the event pushing API, for re-implementing it from
>> scratch.
>>
>> Jörg
>>
>>  --
>> You received this message because you are subscribed to the Google Groups
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to elasticsearch+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/elasticsearch/CAKdsXoG-JaWZU4qFQsUaM_UHCwkzMdYnLruUHNmcnkiZ-wX6jA%40mail.gmail.com
>> .
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>
>  --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/CAEM624Yn5y1bxGYd%3DjeG%3DiUVzowhOsuzKi_8erKrPEmTNbxb%3DQ%40mail.gmail.com
> .
>
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQD%3Dh804DJBr2J8V3gDvkSK0ddXHvg0xDBf9XYzUza_n5w%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Elasticsearch java client on EC2

[joergpra...@gmail.com]

Is this a typo?

discovery.type=ec2

Jörg

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoF8mo64zwmiyYwQ6FQQPLX9iuPT8-G2O50-mSdnAUFVnQ%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: For those wanting to try Marvel

[Mark Walkom]

Or perhaps something could come from the new cat API if they don't want to
go that route.

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: ma...@campaignmonitor.com
web: www.campaignmonitor.com


On 29 January 2014 12:21, joergpra...@gmail.com wrote:

> Marvel comes with hidden Java plugin code for an event pusher. Node
> events, route events, and shard events of ES can be indexed into ES. Very
> useful for historic analysis and post mortem views.
>
> It seems this was also a motivation for the tribe node mode: grow two
> separate clusters, one for the data, another one for the metrics.
>
> It would be nice if also the event pushing source code could be opened, so
> other monitoring tools are able to build on this facility too. Or at least
> documenting the event pushing API, for re-implementing it from scratch.
>
> Jörg
>
>  --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/CAKdsXoG-JaWZU4qFQsUaM_UHCwkzMdYnLruUHNmcnkiZ-wX6jA%40mail.gmail.com
> .
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAEM624Yn5y1bxGYd%3DjeG%3DiUVzowhOsuzKi_8erKrPEmTNbxb%3DQ%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: For those wanting to try Marvel

[joergpra...@gmail.com]

Marvel comes with hidden Java plugin code for an event pusher. Node events,
route events, and shard events of ES can be indexed into ES. Very useful
for historic analysis and post mortem views.

It seems this was also a motivation for the tribe node mode: grow two
separate clusters, one for the data, another one for the metrics.

It would be nice if also the event pushing source code could be opened, so
other monitoring tools are able to build on this facility too. Or at least
documenting the event pushing API, for re-implementing it from scratch.

Jörg

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoG-JaWZU4qFQsUaM_UHCwkzMdYnLruUHNmcnkiZ-wX6jA%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Elasticsearch java client on EC2

[Maxime Nay]

Hi,

For some reason I can't get the java client working on EC2.

My elasticsearch cluster and the cluster running my java application are in 
two different security groups ('elasticsearchrealtime' and 'storm'), in the 
same aws region.
I am using elasticsearch version 0.90.2 and using the cloud aws plugin 
version 1.12.0. (in my es cluster and in my java application) 

I tried lot of combinations for the properties, and none of them worked. 
I always get the following warning : 

[Elastic Search Realtime Client] waited for 30s and no initial state was 
set by the discovery

And after that, each time the java application is trying to interact with 
the client the following exception is thrown :
org.elasticsearch.discovery.MasterNotDiscoveredException: waited for [30s]
at 
org.elasticsearch.action.support.master.TransportMasterNodeOperationAction$3.onTimeout(TransportMasterNodeOperationAction.java:169)
at 
org.elasticsearch.cluster.service.InternalClusterService$NotifyTimeout.run(InternalClusterService.java:377)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)


The elasticsearchrealtime security group authorizes storm security group on 
every tcp,udp and icmp ports.
The storm security group authorizes elasticsearchrealtime security group on 
every tcp,udp and icmp ports.


Here are my current properties for the client :

### Cluster 
###

cluster.name: elasticsearchrealtime

 Index 


index.number_of_shards: 16
index.number_of_replicas: 1

### Memory 


bootstrap.mlockall: true

## Discovery 
##

discovery.zen.ping.timeout: 5m
discovery.type=ec2
discovery.ec2.groups: elasticsearchrealtime,storm
discovery.ec2.ping_timeout: 60s
discovery.ec2.any_group: true

# AWS Settings 
###

cloud.aws.region: us-east-1
cloud.aws.access_key: 
cloud.aws.secret_key: 
cloud.aws.s3.endpoint: s3.amazonaws.com
cloud.aws.ec2.endpoint: ec2.us-east-1.amazonaws.com
gateway.s3.region: us-east-1

### Client 
###

node.name: Elastic Search Realtime Client
node.client: true
node.data: false




And here is the elasticsearch.yml file for one of my elasticsearch node :

### Cluster 
###

cluster.name: elasticsearchrealtime

 Index 


index.number_of_shards: 16
index.number_of_replicas: 1

### Memory 


bootstrap.mlockall: true

## Discovery 
##

discovery.zen.ping.timeout: 5m
discovery.type: ec2
discovery.ec2.groups: elasticsearchrealtime,storm
discovery.ec2.ping_timeout: 60s
discovery.ec2.any_group: true

### AWS Settings
cloud.aws.region: us-east-1
cloud.aws.access_key: x
cloud.aws.secret_key: xx
cloud.aws.s3.endpoint: s3.amazonaws.com
cloud.aws.ec2.endpoint: ec2.us-east-1.amazonaws.com
gateway.s3.region: us-east-1

node.name: Elastic Search Realtime 1




Any help would be greatly appreciated !

Maxime

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/58880718-a1cd-4a8f-8bbd-84dbc71d4805%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Logstash RPM on centos5.x

[Kevin Pearson]

Hello,

I am having no problems using the elasticsearch logstash yum repo on 
centos6.x, but when I try on centos5.x I am not able to get an rpm to 
install.

This is the error I get when I use the 1.3 repo:

http://packages.elasticsearch.org/logstash/1.3/centos/repodata/primary.sqlite.bz2:
 
[Errno -3] Error performing checksum
Trying other mirror.
primary.sqlite.bz2   | 2.6 kB 00:00 

http://packages.elasticsearch.org/logstash/1.3/centos/repodata/primary.sqlite.bz2:
 
[Errno -3] Error performing checksum
Trying other mirror.
Error: failure: repodata/primary.sqlite.bz2 from logstash-1.3: [Errno 256] 
No more mirrors to try.


Here is my repo file:

[logstash-1.3]

name=logstash repository for 1.3.x packages
baseurl

baseurl=http://packages.elasticsearch.org/logstash/1.3/centos

gpgcheck=1

gpgkey=http://packages.elasticsearch.org/GPG-KEY-elasticsearch

enabled=1 


I have tried turning off gpgcheck as well and I get the same error.
I also tried just downloading the rpm directly and then installing it 
without yum and I get this:

[root@host ~]# rpm -q -p logstash-1.3.2-1_centos.noarch.rpm 
error: logstash-1.3.2-1_centos.noarch.rpm: Header V4 RSA/SHA1 signature: 
BAD, key ID d88e42b4 


I believe that is caused by signing an rpm with V4, which centos5 is not 
compatible with.

Here is my uname:
[root@host ~]# uname -a
Linux host 2.6.18-128.el5 #1 SMP Wed Dec 17 11:41:38 EST 2008 x86_64 x86_64 
x86_64 GNU/Linux


Is there a different repository I should be using for centos/redhat 5.x?

Thank you,
Kevin

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/6596039b-5be9-4d8e-aef6-eff9f190b8ae%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: For those wanting to try Marvel

[Ivan Brusic]

There is also
https://github.com/elasticsearch/elasticsearch-metrics-reporter-java, but
it does not seem to be as advanced graphically. I am currently pushing data
to graphite. Would love to try out other tools, but then I realize I am not
an ops guy. :)

Ivan


On Tue, Jan 28, 2014 at 4:15 PM, Mark Walkom wrote:

> I think something like this
> https://github.com/abronner/elasticsearch-monitoring could be extended to
> include a lot of what is in Marvel.
>
> I've made a few changes to the way the the indexes are handled with this,
> which I've been meaning to properly fork and push back up. It'd be great if
> anyone else could look at what can be done with it if they are interested.
>
> Regards,
> Mark Walkom
>
> Infrastructure Engineer
> Campaign Monitor
> email: ma...@campaignmonitor.com
> web: www.campaignmonitor.com
>
>
> On 29 January 2014 11:08, Ivan Brusic  wrote:
>
>> My current code is not compatible with 0.90.10, so I will try again in
>> the future. Still, great job. Couldn't convince the higher ups to use SPM,
>> but perhaps self-hosted Marvel might be easier (although I wish the pricing
>> was more transparent).
>>
>> --
>> Ivan
>>
>>
>> On Tue, Jan 28, 2014 at 3:09 PM, Ivan Brusic  wrote:
>>
>>> I am sure there will be more than a few bugs to work out. :) Not bashing
>>> their great work, that is how software works.
>>>
>>> Creating a new 0.90.10 instance right now. I have been pushing to
>>> upgrade for a while (no rolling restarts makes it harder to schedule
>>> upgrades), perhaps this would be the time.
>>>
>>> Ivan
>>>
>>>
>>> On Tue, Jan 28, 2014 at 2:52 PM, Mark Walkom 
>>> wrote:
>>>
 Ahh, someone pointed out other discrepancies with the docs in the IRC
 channel as well, so I guess there are still a few bugs to work out!

 Regards,
 Mark Walkom

 Infrastructure Engineer
 Campaign Monitor
 email: ma...@campaignmonitor.com
 web: www.campaignmonitor.com


 On 29 January 2014 09:51, Ivan Brusic  wrote:

> From my log:
>
> [2014-01-28 14:31:01,915][WARN ][marvel.agent ]
> Elasticsearch version [0.90.2] is too old. Marvel is disabled (requires
> version 0.90.8 or higher)
> [2014-01-28 14:31:01,921][INFO ][plugins  ] [xxx]
> loaded [marvel, refresh-token-filters], sites [bigdesk, marvel, browser,
> head]
>
>
>
> On Tue, Jan 28, 2014 at 2:47 PM, Mark Walkom <
> ma...@campaignmonitor.com> wrote:
>
>> The docs mention 0.90.9 and higher -
>> http://www.elasticsearch.org/guide/en/marvel/current/#_system_requirements
>>
>> Regards,
>> Mark Walkom
>>
>> Infrastructure Engineer
>> Campaign Monitor
>> email: ma...@campaignmonitor.com
>> web: www.campaignmonitor.com
>>
>>
>> On 29 January 2014 09:37, Ivan Brusic  wrote:
>>
>>>  It is only supported with Elasticsearch 0.90.8 and higher.
>>>
>>> Yet another push for me to upgrade!
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "elasticsearch" group.
>>> To unsubscribe from this group and stop receiving emails from it,
>>> send an email to elasticsearch+unsubscr...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQCjpsnrgo_1jenzVJxamZEFNKWHztdAF142QeV9BDqMew%40mail.gmail.com
>>> .
>>> For more options, visit https://groups.google.com/groups/opt_out.
>>>
>>
>>  --
>> You received this message because you are subscribed to the Google
>> Groups "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it,
>> send an email to elasticsearch+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/elasticsearch/CAEM624aVcB7FdaVoWq5z0i61Omi8r7MsJcL26w6%2B7cnLn678cA%40mail.gmail.com
>> .
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>
>  --
> You received this message because you are subscribed to the Google
> Groups "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQAiQ%3D30YcMCwH7kgdtZc8FKwY8JZ3UqU5qEGfrNQk4G9g%40mail.gmail.com
> .
>
> For more options, visit https://groups.google.com/groups/opt_out.
>

  --
 You received this message because you are subscribed to the Google
 Groups "elasticsearch" group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to elasticsearch+unsubscr...@googlegroups.com.
 To view this discussion on the web visit
 https://groups.google.com/d/msgid/elasticsearch/CAEM624avhH_RtD0%3DazXMLkF%3DsLb6uAmvayTvUU%

Re: For those wanting to try Marvel

[Mark Walkom]

I think something like this
https://github.com/abronner/elasticsearch-monitoring could be extended to
include a lot of what is in Marvel.

I've made a few changes to the way the the indexes are handled with this,
which I've been meaning to properly fork and push back up. It'd be great if
anyone else could look at what can be done with it if they are interested.

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: ma...@campaignmonitor.com
web: www.campaignmonitor.com


On 29 January 2014 11:08, Ivan Brusic  wrote:

> My current code is not compatible with 0.90.10, so I will try again in the
> future. Still, great job. Couldn't convince the higher ups to use SPM, but
> perhaps self-hosted Marvel might be easier (although I wish the pricing was
> more transparent).
>
> --
> Ivan
>
>
> On Tue, Jan 28, 2014 at 3:09 PM, Ivan Brusic  wrote:
>
>> I am sure there will be more than a few bugs to work out. :) Not bashing
>> their great work, that is how software works.
>>
>> Creating a new 0.90.10 instance right now. I have been pushing to upgrade
>> for a while (no rolling restarts makes it harder to schedule upgrades),
>> perhaps this would be the time.
>>
>> Ivan
>>
>>
>> On Tue, Jan 28, 2014 at 2:52 PM, Mark Walkom 
>> wrote:
>>
>>> Ahh, someone pointed out other discrepancies with the docs in the IRC
>>> channel as well, so I guess there are still a few bugs to work out!
>>>
>>> Regards,
>>> Mark Walkom
>>>
>>> Infrastructure Engineer
>>> Campaign Monitor
>>> email: ma...@campaignmonitor.com
>>> web: www.campaignmonitor.com
>>>
>>>
>>> On 29 January 2014 09:51, Ivan Brusic  wrote:
>>>
 From my log:

 [2014-01-28 14:31:01,915][WARN ][marvel.agent ]
 Elasticsearch version [0.90.2] is too old. Marvel is disabled (requires
 version 0.90.8 or higher)
 [2014-01-28 14:31:01,921][INFO ][plugins  ] [xxx]
 loaded [marvel, refresh-token-filters], sites [bigdesk, marvel, browser,
 head]



 On Tue, Jan 28, 2014 at 2:47 PM, Mark Walkom >>> > wrote:

> The docs mention 0.90.9 and higher -
> http://www.elasticsearch.org/guide/en/marvel/current/#_system_requirements
>
> Regards,
> Mark Walkom
>
> Infrastructure Engineer
> Campaign Monitor
> email: ma...@campaignmonitor.com
> web: www.campaignmonitor.com
>
>
> On 29 January 2014 09:37, Ivan Brusic  wrote:
>
>>  It is only supported with Elasticsearch 0.90.8 and higher.
>>
>> Yet another push for me to upgrade!
>>
>> --
>> You received this message because you are subscribed to the Google
>> Groups "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it,
>> send an email to elasticsearch+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQCjpsnrgo_1jenzVJxamZEFNKWHztdAF142QeV9BDqMew%40mail.gmail.com
>> .
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>
>  --
> You received this message because you are subscribed to the Google
> Groups "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/CAEM624aVcB7FdaVoWq5z0i61Omi8r7MsJcL26w6%2B7cnLn678cA%40mail.gmail.com
> .
> For more options, visit https://groups.google.com/groups/opt_out.
>

  --
 You received this message because you are subscribed to the Google
 Groups "elasticsearch" group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to elasticsearch+unsubscr...@googlegroups.com.
 To view this discussion on the web visit
 https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQAiQ%3D30YcMCwH7kgdtZc8FKwY8JZ3UqU5qEGfrNQk4G9g%40mail.gmail.com
 .

 For more options, visit https://groups.google.com/groups/opt_out.

>>>
>>>  --
>>> You received this message because you are subscribed to the Google
>>> Groups "elasticsearch" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to elasticsearch+unsubscr...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/elasticsearch/CAEM624avhH_RtD0%3DazXMLkF%3DsLb6uAmvayTvUU%2BBz6f%2BCvCkoA%40mail.gmail.com
>>> .
>>>
>>> For more options, visit https://groups.google.com/groups/opt_out.
>>>
>>
>>
>  --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQCBzWjAdC6TG0qykLVykGHLnD

Re: For those wanting to try Marvel

[Ivan Brusic]

My current code is not compatible with 0.90.10, so I will try again in the
future. Still, great job. Couldn't convince the higher ups to use SPM, but
perhaps self-hosted Marvel might be easier (although I wish the pricing was
more transparent).

-- 
Ivan


On Tue, Jan 28, 2014 at 3:09 PM, Ivan Brusic  wrote:

> I am sure there will be more than a few bugs to work out. :) Not bashing
> their great work, that is how software works.
>
> Creating a new 0.90.10 instance right now. I have been pushing to upgrade
> for a while (no rolling restarts makes it harder to schedule upgrades),
> perhaps this would be the time.
>
> Ivan
>
>
> On Tue, Jan 28, 2014 at 2:52 PM, Mark Walkom wrote:
>
>> Ahh, someone pointed out other discrepancies with the docs in the IRC
>> channel as well, so I guess there are still a few bugs to work out!
>>
>> Regards,
>> Mark Walkom
>>
>> Infrastructure Engineer
>> Campaign Monitor
>> email: ma...@campaignmonitor.com
>> web: www.campaignmonitor.com
>>
>>
>> On 29 January 2014 09:51, Ivan Brusic  wrote:
>>
>>> From my log:
>>>
>>> [2014-01-28 14:31:01,915][WARN ][marvel.agent ]
>>> Elasticsearch version [0.90.2] is too old. Marvel is disabled (requires
>>> version 0.90.8 or higher)
>>> [2014-01-28 14:31:01,921][INFO ][plugins  ] [xxx] loaded
>>> [marvel, refresh-token-filters], sites [bigdesk, marvel, browser, head]
>>>
>>>
>>>
>>> On Tue, Jan 28, 2014 at 2:47 PM, Mark Walkom 
>>> wrote:
>>>
 The docs mention 0.90.9 and higher -
 http://www.elasticsearch.org/guide/en/marvel/current/#_system_requirements

 Regards,
 Mark Walkom

 Infrastructure Engineer
 Campaign Monitor
 email: ma...@campaignmonitor.com
 web: www.campaignmonitor.com


 On 29 January 2014 09:37, Ivan Brusic  wrote:

>  It is only supported with Elasticsearch 0.90.8 and higher.
>
> Yet another push for me to upgrade!
>
> --
> You received this message because you are subscribed to the Google
> Groups "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQCjpsnrgo_1jenzVJxamZEFNKWHztdAF142QeV9BDqMew%40mail.gmail.com
> .
> For more options, visit https://groups.google.com/groups/opt_out.
>

  --
 You received this message because you are subscribed to the Google
 Groups "elasticsearch" group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to elasticsearch+unsubscr...@googlegroups.com.
 To view this discussion on the web visit
 https://groups.google.com/d/msgid/elasticsearch/CAEM624aVcB7FdaVoWq5z0i61Omi8r7MsJcL26w6%2B7cnLn678cA%40mail.gmail.com
 .
 For more options, visit https://groups.google.com/groups/opt_out.

>>>
>>>  --
>>> You received this message because you are subscribed to the Google
>>> Groups "elasticsearch" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to elasticsearch+unsubscr...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQAiQ%3D30YcMCwH7kgdtZc8FKwY8JZ3UqU5qEGfrNQk4G9g%40mail.gmail.com
>>> .
>>>
>>> For more options, visit https://groups.google.com/groups/opt_out.
>>>
>>
>>  --
>> You received this message because you are subscribed to the Google Groups
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to elasticsearch+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/elasticsearch/CAEM624avhH_RtD0%3DazXMLkF%3DsLb6uAmvayTvUU%2BBz6f%2BCvCkoA%40mail.gmail.com
>> .
>>
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQCBzWjAdC6TG0qykLVykGHLnDr%3DsEvpZVa7uPvvhE0O0Q%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Elastic search 1.0.0 RC1 and Logstash 1.3.3?

[Binh Ly]

Jilles,

I'd give it a try and report back. FYI, I have been using LS 1.3.3 and ES 
1.0 RC1 with the elasticsearch_http output and so far, it seems to be 
working fine.

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/33be999b-98a6-4ec9-93a4-547ba6d02cc1%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: For those wanting to try Marvel

[Ivan Brusic]

I am sure there will be more than a few bugs to work out. :) Not bashing
their great work, that is how software works.

Creating a new 0.90.10 instance right now. I have been pushing to upgrade
for a while (no rolling restarts makes it harder to schedule upgrades),
perhaps this would be the time.

Ivan


On Tue, Jan 28, 2014 at 2:52 PM, Mark Walkom wrote:

> Ahh, someone pointed out other discrepancies with the docs in the IRC
> channel as well, so I guess there are still a few bugs to work out!
>
> Regards,
> Mark Walkom
>
> Infrastructure Engineer
> Campaign Monitor
> email: ma...@campaignmonitor.com
> web: www.campaignmonitor.com
>
>
> On 29 January 2014 09:51, Ivan Brusic  wrote:
>
>> From my log:
>>
>> [2014-01-28 14:31:01,915][WARN ][marvel.agent ] Elasticsearch
>> version [0.90.2] is too old. Marvel is disabled (requires version 0.90.8 or
>> higher)
>> [2014-01-28 14:31:01,921][INFO ][plugins  ] [xxx] loaded
>> [marvel, refresh-token-filters], sites [bigdesk, marvel, browser, head]
>>
>>
>>
>> On Tue, Jan 28, 2014 at 2:47 PM, Mark Walkom 
>> wrote:
>>
>>> The docs mention 0.90.9 and higher -
>>> http://www.elasticsearch.org/guide/en/marvel/current/#_system_requirements
>>>
>>> Regards,
>>> Mark Walkom
>>>
>>> Infrastructure Engineer
>>> Campaign Monitor
>>> email: ma...@campaignmonitor.com
>>> web: www.campaignmonitor.com
>>>
>>>
>>> On 29 January 2014 09:37, Ivan Brusic  wrote:
>>>
  It is only supported with Elasticsearch 0.90.8 and higher.

 Yet another push for me to upgrade!

 --
 You received this message because you are subscribed to the Google
 Groups "elasticsearch" group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to elasticsearch+unsubscr...@googlegroups.com.
 To view this discussion on the web visit
 https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQCjpsnrgo_1jenzVJxamZEFNKWHztdAF142QeV9BDqMew%40mail.gmail.com
 .
 For more options, visit https://groups.google.com/groups/opt_out.

>>>
>>>  --
>>> You received this message because you are subscribed to the Google
>>> Groups "elasticsearch" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to elasticsearch+unsubscr...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/elasticsearch/CAEM624aVcB7FdaVoWq5z0i61Omi8r7MsJcL26w6%2B7cnLn678cA%40mail.gmail.com
>>> .
>>> For more options, visit https://groups.google.com/groups/opt_out.
>>>
>>
>>  --
>> You received this message because you are subscribed to the Google Groups
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to elasticsearch+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQAiQ%3D30YcMCwH7kgdtZc8FKwY8JZ3UqU5qEGfrNQk4G9g%40mail.gmail.com
>> .
>>
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>
>  --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/CAEM624avhH_RtD0%3DazXMLkF%3DsLb6uAmvayTvUU%2BBz6f%2BCvCkoA%40mail.gmail.com
> .
>
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQD1qTNocpSWE1n9Gu4h%3Dc0OV6w%2Btys%3DAcQwVuP9CS-HbA%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: script filter slow query execution

[Binh Ly]

Vinoth,

I'd try to eliminate that "_source._boost" part from your script and see if 
that makes any difference. If it does, store your doc boost in a field and 
access it like:

doc["myboostvalue"].value

and see if that helps. 

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/5fe43c96-d3e8-4cfa-8004-a7e7c0b665f8%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: For those wanting to try Marvel

[Mark Walkom]

Ahh, someone pointed out other discrepancies with the docs in the IRC
channel as well, so I guess there are still a few bugs to work out!

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: ma...@campaignmonitor.com
web: www.campaignmonitor.com


On 29 January 2014 09:51, Ivan Brusic  wrote:

> From my log:
>
> [2014-01-28 14:31:01,915][WARN ][marvel.agent ] Elasticsearch
> version [0.90.2] is too old. Marvel is disabled (requires version 0.90.8 or
> higher)
> [2014-01-28 14:31:01,921][INFO ][plugins  ] [xxx] loaded
> [marvel, refresh-token-filters], sites [bigdesk, marvel, browser, head]
>
>
>
> On Tue, Jan 28, 2014 at 2:47 PM, Mark Walkom wrote:
>
>> The docs mention 0.90.9 and higher -
>> http://www.elasticsearch.org/guide/en/marvel/current/#_system_requirements
>>
>> Regards,
>> Mark Walkom
>>
>> Infrastructure Engineer
>> Campaign Monitor
>> email: ma...@campaignmonitor.com
>> web: www.campaignmonitor.com
>>
>>
>> On 29 January 2014 09:37, Ivan Brusic  wrote:
>>
>>>  It is only supported with Elasticsearch 0.90.8 and higher.
>>>
>>> Yet another push for me to upgrade!
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "elasticsearch" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to elasticsearch+unsubscr...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQCjpsnrgo_1jenzVJxamZEFNKWHztdAF142QeV9BDqMew%40mail.gmail.com
>>> .
>>> For more options, visit https://groups.google.com/groups/opt_out.
>>>
>>
>>  --
>> You received this message because you are subscribed to the Google Groups
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to elasticsearch+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/elasticsearch/CAEM624aVcB7FdaVoWq5z0i61Omi8r7MsJcL26w6%2B7cnLn678cA%40mail.gmail.com
>> .
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>
>  --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQAiQ%3D30YcMCwH7kgdtZc8FKwY8JZ3UqU5qEGfrNQk4G9g%40mail.gmail.com
> .
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAEM624avhH_RtD0%3DazXMLkF%3DsLb6uAmvayTvUU%2BBz6f%2BCvCkoA%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: For those wanting to try Marvel

[Ivan Brusic]

>From my log:

[2014-01-28 14:31:01,915][WARN ][marvel.agent ] Elasticsearch
version [0.90.2] is too old. Marvel is disabled (requires version 0.90.8 or
higher)
[2014-01-28 14:31:01,921][INFO ][plugins  ] [xxx] loaded
[marvel, refresh-token-filters], sites [bigdesk, marvel, browser, head]



On Tue, Jan 28, 2014 at 2:47 PM, Mark Walkom wrote:

> The docs mention 0.90.9 and higher -
> http://www.elasticsearch.org/guide/en/marvel/current/#_system_requirements
>
> Regards,
> Mark Walkom
>
> Infrastructure Engineer
> Campaign Monitor
> email: ma...@campaignmonitor.com
> web: www.campaignmonitor.com
>
>
> On 29 January 2014 09:37, Ivan Brusic  wrote:
>
>> It is only supported with Elasticsearch 0.90.8 and higher.
>>
>> Yet another push for me to upgrade!
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to elasticsearch+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQCjpsnrgo_1jenzVJxamZEFNKWHztdAF142QeV9BDqMew%40mail.gmail.com
>> .
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>
>  --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/CAEM624aVcB7FdaVoWq5z0i61Omi8r7MsJcL26w6%2B7cnLn678cA%40mail.gmail.com
> .
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQAiQ%3D30YcMCwH7kgdtZc8FKwY8JZ3UqU5qEGfrNQk4G9g%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: For those wanting to try Marvel

[Mark Walkom]

The docs mention 0.90.9 and higher -
http://www.elasticsearch.org/guide/en/marvel/current/#_system_requirements

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: ma...@campaignmonitor.com
web: www.campaignmonitor.com


On 29 January 2014 09:37, Ivan Brusic  wrote:

> It is only supported with Elasticsearch 0.90.8 and higher.
>
> Yet another push for me to upgrade!
>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQCjpsnrgo_1jenzVJxamZEFNKWHztdAF142QeV9BDqMew%40mail.gmail.com
> .
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAEM624aVcB7FdaVoWq5z0i61Omi8r7MsJcL26w6%2B7cnLn678cA%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

For those wanting to try Marvel

[Ivan Brusic]

It is only supported with Elasticsearch 0.90.8 and higher.

Yet another push for me to upgrade!

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQCjpsnrgo_1jenzVJxamZEFNKWHztdAF142QeV9BDqMew%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

ProcessClusterEventTimeoutException in Elasticsearch. Is this timeout value configurable? If yes how?

[Ahaduzzaman Munna]

Getting timeout exceptions like below for different index operations. 
 Below one shows for "index-aliases" operation. I would like to know how 
can I configure this timeout value in elasticsearch.yml so that I configure 
it once and I do not have to worry about this timeout exception. 

org.elasticsearch.cluster.metadata.ProcessClusterEventTimeoutException: 
failed to process cluster event (index-aliases) within 30s
at 
org.elasticsearch.cluster.service.InternalClusterService$2$1.run(InternalClusterService.java:237)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)


I saw in elasticsearch code that 

public abstract class MasterNodeOperationRequest extends ActionRequest {

public static TimeValue DEFAULT_MASTER_NODE_TIMEOUT = 
TimeValue.timeValueSeconds(30);

protected TimeValue masterNodeTimeout = DEFAULT_MASTER_NODE_TIMEOUT;


Thanks,
Munna

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/8ee9ef0f-e1f2-4230-a7cf-1dfec4eee9b7%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Elastic search 1.0.0 RC1 and Logstash 1.3.3?

[Jilles van Gurp]

I'm considering to start using elastic search 1.0.0RC1 in a new project. 
However, I also need to deploy logstash (and probably also Kibana). Given 
the API changes in the new release candidate, can I expect Logstash to play 
nice with elastic search, should I wait, or should I use development 
versions of logstash and Kibana? I'd appreciate any feedback people who 
have tried this already might have on this before trying to find out the 
hard way and possibly wasting a few hours.

My main concern is that I don't want to code against APIs that are going to 
be deprecated or change a few weeks into my project when 1.0.0 is released. 
Also, there are some features in there that I definitely want to use and 
I'd prefer to code against the new APIs from day one.

JIlles

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/9aa4248f-4b0f-4786-9174-1816a11d2463%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Scan/scroll facet data?

[Josh Harrison]

I've got fields that have a few hundred thousand+ unique values that I'd 
like to be able to facet on. Is there some way of essentially streaming the 
exhaustive list of facet results, like I can search hits? 

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/3b3581eb-a852-444d-ae79-739ec0b56dc8%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: How to monitor for filter cache churn?

[Tikitu de Jager]

Zach, thanks very much for that. Of course "it depends" but you've given me 
lots of good tips to keep in mind. (Especially the warning not to take 
eviction metrics too seriously probably saved me *hours* of sweat and 
worry.)

My stats calls behave a bit differently (0.90.9) so I list them here in 
case anyone else runs into the same confusion:

Per-index:
curl -XGET "http://localhost:9200//_stats?filter_cache"
(filter_cache entry not shown by default; param ?all shows it as well)

Per-node:
curl -XGET "http://localhost:9200/_nodes/stats?all";
curl -XGET "http://localhost:9200/_nodes/stats?indices";
(without ?all or ?indices the indices.filter_cache entry isn't included; I 
didn't find a way to specify only that entry)

Per-cluster:
curl -XGET "http://localhost:9200/_cluster/stats";
(included by default)

Thanks again for your help!
Tikitu

On Wednesday, 29 January 2014 00:06:05 UTC+13, Zachary Tong wrote:
>
> Oh, final note that I forgot:  if you cache the boolean combination of 
> filters, the individual leaf node filters will still be cached by default. 
>  You'll have to explicitly disable caching for leaf nodes if you don't want 
> that behavior.
>
> -Z
>
>
> On Tuesday, January 28, 2014 5:58:21 AM UTC-5, Zachary Tong wrote:
>>
>> You can monitor filter cache from three different levels - index, node 
>> and cluster.  The output is similar for all three outputs, you'll see a 
>> size in bytes and an eviction count.
>>
>>- Per-index: 
>>curl -XGET "http://localhost:9200//_stats"
>>
>>- Per-node: 
>>curl -XGET "http://localhost:9200/_nodes/stats";
>>
>>- Entire Cluster (this is actually pretty new, introduced in 0.90.8): 
>>curl -XGET "http://localhost:9200/_cluster/stats";
>>
>>
>> Regarding your question about bitset combination speed and caching 
>> intermediate booleans...it depends (heh).  The question is less about the 
>> speed of combining say 50 individual bitsets compared to 1 combined bitset. 
>>  The single bitset will obviously be faster, but the speed difference is 
>> pretty negligible compared to other operations.
>>
>> What you should be thinking about is the effect of a "cache miss".  Let's 
>> operate under the assumption that filter cache size is limited, and 
>> evictions will occur in some fashion (otherwise we'd just keep everything 
>> in memory and be happy).   
>>
>> If you have a boolean combination of 50 filters that is cached, you only 
>> need to keep that single filter "hot" in the cache.  If your usage pattern 
>> keeps it cached, you will have very little churn.  But if it happens that 
>> there is a lull and the combo-filter falls out of cache, the next time it 
>> is executed you'll need to re-evaluate all 50 "interior" filters to derive 
>> the final bitset.  Those interior filters could potentially touch a large 
>> number of documents (and the associated disk access).  A cache miss could 
>> be relatively expensive (still fast, but relative to simple bitset lookups)
>>
>> When filters are specified independently, there is a greater chance that 
>> individual filters may be missing from the cache.  Each execution of the 
>> set of filters may require a few of the "interior" filters to be evaluated, 
>> but since the filters are cached separately there is a good chance the 
>> majority of them remain cached.  So the computational cost 
>> is amortized over time instead of being lumpy.  There is also a better 
>> chance that filters stay cached since they are reused in other parts of 
>> your query, which keeps them "live" even if the total combination rarely 
>> re-occurs.
>>
>> It also gets complicated because filter caching is technically 
>> per-segment.  It is possible for the *very same* filter to be cached in 
>> one segment but evicted on another segment.  The LRU cache tries to evict 
>> old (unused) filters, but weights towards smaller segments since they are 
>> cheaper to recalculate.  This also means that data indexing has an affect 
>> on filter caching, since a constant ingestion of new documents equals 
>> segment merges, which clears the caches for those newly created segment.
>>
>> Some other assorted thoughts:
>>
>>- Remember that boolean caching will cache the *result* of the bool, 
>>not the filter's themselves.  E.g. if you have a bool of 10 Terms, the 
>>final bitset is the set of documents that matches the filters, not the 10 
>>Term filters themselves.  Probably obvious, but wanted to make it clear
>>
>>- Eviction metrics are...meh at best.  Like I mentioned above, 
>>evictions are per-segment, and weighted towards small segments.  You can 
>>see high eviction rates without it actually equalling much churn (e.g. 
>> lots 
>>of newly created, small segments are evicting when they merge, but 95% of 
>>your data is remaining safely cached).  You can even see a lot of churn 
>> but 
>>still get good performance, since big segments tend to keep their caches 
>>

Re: There were no results because no indices were found that match your selected time span

[David Pilato]

Should work from 0.90.9. 

-- 
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet | @elasticsearchfr


Le 28 janvier 2014 at 20:51:14, Thomas Ardal (thomasar...@gmail.com) a écrit:

I know and that's the plan. But with 1.0.0 right around the corner and a lot of 
data to migrate, I'll probably wait for that one.

Does Marvel only support the most recent versions of ES?

On Tuesday, January 28, 2014 8:43:26 PM UTC+1, David Pilato wrote:
0.90.1?
You should update to 0.90.10.

--
David ;-)
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 28 janv. 2014 à 20:11, Thomas Ardal  a écrit :

As bonus info I'm running Elasticsearch 0.90.1 on windows server 2012. I'm 
using the Jetty plugin to force https and basic authentication, but are 
accessing Marvel from localhost through http. My browser asks me for 
credentials when opening the Marvel url, so it could be caused by the basic 
authentication setup. Or?

On Tuesday, January 28, 2014 8:01:21 PM UTC+1, Thomas Ardal wrote:
When trying out Marvel on my Elasticsearch installation, I get the error "There 
were no results because no indices were found that match your selected time 
span" in the top of the page.

If I understand the documentation, Marvel automatically collects statistics 
from all indexes on the node. What am I doing wrong?
--
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearc...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/7255ee52-5101-4942-8abd-b29642035237%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
--
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/8bee9ba2-d0bf-42c3-b8ac-2c45707b9f96%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/etPan.52e80ab7.79e2a9e3.ec%40MacBook-Air-de-David.local.
For more options, visit https://groups.google.com/groups/opt_out.

Re: There were no results because no indices were found that match your selected time span

[Thomas Ardal]

I know and that's the plan. But with 1.0.0 right around the corner and a 
lot of data to migrate, I'll probably wait for that one.

Does Marvel only support the most recent versions of ES?

On Tuesday, January 28, 2014 8:43:26 PM UTC+1, David Pilato wrote:
>
> 0.90.1?
> You should update to 0.90.10.
>
> --
> David ;-)
> Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs
>
> Le 28 janv. 2014 à 20:11, Thomas Ardal > 
> a écrit :
>
> As bonus info I'm running Elasticsearch 0.90.1 on windows server 2012. I'm 
> using the Jetty plugin to force https and basic authentication, but are 
> accessing Marvel from localhost through http. My browser asks me for 
> credentials when opening the Marvel url, so it could be caused by the basic 
> authentication setup. Or?
>
> On Tuesday, January 28, 2014 8:01:21 PM UTC+1, Thomas Ardal wrote:
>>
>> When trying out Marvel on my Elasticsearch installation, I get the error 
>> "There were no results because no indices were found that match your 
>> selected time span" in the top of the page.
>>
>> If I understand the documentation, Marvel automatically collects 
>> statistics from all indexes on the node. What am I doing wrong?
>>
>  -- 
> You received this message because you are subscribed to the Google Groups 
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to elasticsearc...@googlegroups.com .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/elasticsearch/7255ee52-5101-4942-8abd-b29642035237%40googlegroups.com
> .
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/8bee9ba2-d0bf-42c3-b8ac-2c45707b9f96%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: There were no results because no indices were found that match your selected time span

[David Pilato]

0.90.1?
You should update to 0.90.10.

--
David ;-)
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 28 janv. 2014 à 20:11, Thomas Ardal  a écrit :

> As bonus info I'm running Elasticsearch 0.90.1 on windows server 2012. I'm 
> using the Jetty plugin to force https and basic authentication, but are 
> accessing Marvel from localhost through http. My browser asks me for 
> credentials when opening the Marvel url, so it could be caused by the basic 
> authentication setup. Or?
> 
> On Tuesday, January 28, 2014 8:01:21 PM UTC+1, Thomas Ardal wrote:
>> 
>> When trying out Marvel on my Elasticsearch installation, I get the error 
>> "There were no results because no indices were found that match your 
>> selected time span" in the top of the page.
>> 
>> If I understand the documentation, Marvel automatically collects statistics 
>> from all indexes on the node. What am I doing wrong?
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/elasticsearch/7255ee52-5101-4942-8abd-b29642035237%40googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/BB35B6A2-973F-46E4-B553-00C89D938AEF%40pilato.fr.
For more options, visit https://groups.google.com/groups/opt_out.

script filter slow query execution

[vinamar]

I'm trying to use script filter with filtered query as given below. The query
execution takes around 20 to 30 secs for 70K matching facet results. How to
speed up the query execution ?

{
  "timeout": 3,
  "highlight": {
"pre_tags": [
  ""
],
"post_tags": [
  ""
],
"fields": {
  "srId": {
"number_of_fragments": 0
  },
  "emails.emailBody": {
"number_of_fragments": 0
  },
  "chatTextArray.text": {
"number_of_fragments": 0
  }
}
  },
  "query": {
"custom_filters_score": {
  "query": {
"filtered": {
  "filter": {
"and": [
  {
"term": {
  "domains.L1.domain.id": "1"
}
  },
  {
"range": {
  "closeDt": {
"from": 132540480,
"to": 139098240
  }
}
  }
]
  },
  "query": {
"query_string": {
  "query": "payment button",
  "default_operator": "AND",
  "fields": [
[
  "emails.emailBody",
  "srId",
  "chatTextArray.text"
]
  ]
}
  }
}
  },
  "filters": [
{
  "filter": {
"exists": {
  "field": "domains.L1.domain.id"
}
  },
  "script": "int
docscore=doc['domains.L1.domain.1.prob'].value*1.5;int
expscore=pow(docscore,7);_score=_score*_source._boost*expscore"
}
  ]
}
  },
  "size": 10
}



--
View this message in context: 
http://elasticsearch-users.115913.n3.nabble.com/script-filter-slow-query-execution-tp4048301.html
Sent from the ElasticSearch Users mailing list archive at Nabble.com.

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/1390937270853-4048301.post%40n3.nabble.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Loading JSON to ElasticSearch

[ZenMaster80]

Thanks David, I will certainly look into logstash. Do you think it is a 
good idea to separate data analysis and indexing into 2 different machines 
since both require lots of cpu time. 
If I use logstash to send files over to ES, will I be able to use native 
Java API or http, and is there any preference to the API? I have noticed 
there are somethings that aren't very easy and may be don't even work in 
the native API? 
Thanks again

On Tuesday, January 28, 2014 1:05:32 PM UTC-5, David Pilato wrote:
>
> Did you try https://github.com/dadoonet/fsriver?
> Never tested it with so many docs but may be it could help you here?
>
> If you have already generated json files on a server, then I would 
> recommend trying logstash to send them into elasticsearch. 
>
> My 2 cents
>
> -- 
> *David Pilato* | *Technical Advocate* | *Elasticsearch.com*
> @dadoonet  | 
> @elasticsearchfr
>
>
> Le 28 janvier 2014 at 16:46:06, ZenMaster80 (sabda...@gmail.com) 
> a écrit:
>
> I would like to get your perspective on how to load json to index server 
> in my scenario. 
> We have about 15 million documents in html/pdf/... on Server 1
> I would like to process the data and convert to json on server 2
> I would like the indexer to index json n a separate machine/server server 3
>
> Ideally I thought on Server 2, as I prepare json and have it ready in 
> memory, I can feed it to indexer. But since data processing is cpu 
> intensive, I want indexing to be done on a separate machines/server.
> How do you guys deal with this since I can no longer feed in-memory json 
> to the indexer on separate machine? Do I just grab files from server 2 and 
> index them then?
>  --
> You received this message because you are subscribed to the Google Groups 
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to elasticsearc...@googlegroups.com .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/elasticsearch/05b977ac-00d0-45c0-9e58-8df523e6978c%40googlegroups.com
> .
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/f536d58c-89ab-4609-b5ca-cef44e2b879a%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: There were no results because no indices were found that match your selected time span

[Thomas Ardal]

As bonus info I'm running Elasticsearch 0.90.1 on windows server 2012. I'm 
using the Jetty plugin to force https and basic authentication, but are 
accessing Marvel from localhost through http. My browser asks me for 
credentials when opening the Marvel url, so it could be caused by the basic 
authentication setup. Or?

On Tuesday, January 28, 2014 8:01:21 PM UTC+1, Thomas Ardal wrote:
>
> When trying out Marvel on my Elasticsearch installation, I get the error 
> "There were no results because no indices were found that match your 
> selected time span" in the top of the page.
>
> If I understand the documentation, Marvel automatically collects 
> statistics from all indexes on the node. What am I doing wrong?
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/7255ee52-5101-4942-8abd-b29642035237%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

There were no results because no indices were found that match your selected time span

[Thomas Ardal]

When trying out Marvel on my Elasticsearch installation, I get the error 
"There were no results because no indices were found that match your 
selected time span" in the top of the page.

If I understand the documentation, Marvel automatically collects statistics 
from all indexes on the node. What am I doing wrong?

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/454e6e01-de1a-4a23-b270-16bf90273c47%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Loading JSON to ElasticSearch

[ZenMaster80]

Thanks David, I will certainly look into hashtag. Do you think it is a good 
idea to separate data analysis and indexing into 2 different machines since 
both require lots of cpu time. 
If I use hashtag to send files over to ES, will I be able to use native 
Java API or http, and is there any preference to the API? I have noticed 
there are somethings that aren't very easy and may be don't even work in 
the native API? 
Thanks again.

On Tuesday, January 28, 2014 1:05:32 PM UTC-5, David Pilato wrote:
>
> Did you try https://github.com/dadoonet/fsriver?
> Never tested it with so many docs but may be it could help you here?
>
> If you have already generated json files on a server, then I would 
> recommend trying logstash to send them into elasticsearch. 
>
> My 2 cents
>
> -- 
> *David Pilato* | *Technical Advocate* | *Elasticsearch.com*
> @dadoonet  | 
> @elasticsearchfr
>
>
> Le 28 janvier 2014 at 16:46:06, ZenMaster80 (sabda...@gmail.com) 
> a écrit:
>
> I would like to get your perspective on how to load json to index server 
> in my scenario. 
> We have about 15 million documents in html/pdf/... on Server 1
> I would like to process the data and convert to json on server 2
> I would like the indexer to index json n a separate machine/server server 3
>
> Ideally I thought on Server 2, as I prepare json and have it ready in 
> memory, I can feed it to indexer. But since data processing is cpu 
> intensive, I want indexing to be done on a separate machines/server.
> How do you guys deal with this since I can no longer feed in-memory json 
> to the indexer on separate machine? Do I just grab files from server 2 and 
> index them then?
>  --
> You received this message because you are subscribed to the Google Groups 
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to elasticsearc...@googlegroups.com .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/elasticsearch/05b977ac-00d0-45c0-9e58-8df523e6978c%40googlegroups.com
> .
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/a02427ec-a3d8-484f-9cfb-2ba7628192b1%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Add a custom lucene search filter to elasticsearch

[Ivan Brusic]

Jörg, what is the purpose of tweaking the IndexQueryParserService? I have a
few custom filters, and all I do is add them via
tokenFiltersBindings.processTokenFilter()
or analysisModule.addTokenFilter().

-- 
Ivan


On Tue, Jan 28, 2014 at 9:40 AM, joergpra...@gmail.com <
joergpra...@gmail.com> wrote:

> Adding a Lucene filter via a plugin is possible, but a bit more demanding,
> you have to tweak the IndexQueryParserService like it is demonstrated in
> org.elasticsearch.index.query.guice.IndexQueryParserModuleTests
>
> Jörg
>
>  --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGA9d4t2h5RSbKbXuY4oJnwVQkW_TYFpvc%3DT3Cgo0Cf%2Bw%40mail.gmail.com
> .
>
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQCqPcUeJU9y3axf8SC-baAdWVu6bBUqT-%2B%2Bu%3DrbwtjSEg%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Complex or impossible query

[Ivan Brusic]

What you are probably looking for is field collapsing, which is not yet
supported in elasticsearch (it is planned). You can use a term facet to
retrieve the count for all terms and then do a separate query for each
unique term. In addition to the slowness of having to do multiple queries,
you would also face the issue of the facets not returning all the values,
especially on fields with many values. Not sure if the new aggregations
framework will help with this last part (I really need to try it out).

-- 
Ivan


On Tue, Jan 28, 2014 at 7:00 AM, Ernesto Reig  wrote:

> Hello, I have a requirement we need to implement and I really don´t know
> how to do it or even if it is possible (maybe the solution is easy but I
> just don´t know).
> We have Person documents like this:
> {
>   "id": 1,
>   "name": "Ernesto",
>   "NID": "AAA"
> }
>
> {
>   "id": 2,
>   "name": "Enrique",
>   "NID": "AAA"
> }
>
> {
>   "id": 3,
>   "name": "Antonio",
>   "NID": "BBB"
> }
>
> {
>   "id": 4,
>   "name": "Karlos",
>   "NID": "CCC"
> }
>
> Imagine we search for Persons with ["NID": "AAA"], we would get 2
> documents (id 1 and id 2).
> If we search for Persons with ["NID": "BBB"], we would get 1 document (id
> 3).
> And if we search for Persons with ["NID": "CCC"], we would get 1 document
> (id 3).
> Ok, so what we need is get all the documents in which the NID is unique. I
> mean, the _count would be only 1 if we search for the NID of those
> documents. The results from that query/filter would be documents 3 and 4,
> since the NID value is unique for those documents (there are no more
> documents in the index with that NID).
>
> Thank you very much,
>
> Ernesto
>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/977fa0d9-1b88-4147-8073-dd4d29aff1a6%40googlegroups.com
> .
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBL%2B4T44RTucL4cHVDt4Laqpc2fsC94yox9jRxQT1R%3D5g%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Loading JSON to ElasticSearch

[David Pilato]

Did you try https://github.com/dadoonet/fsriver?
Never tested it with so many docs but may be it could help you here?

If you have already generated json files on a server, then I would recommend 
trying logstash to send them into elasticsearch. 

My 2 cents

-- 
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet | @elasticsearchfr


Le 28 janvier 2014 at 16:46:06, ZenMaster80 (sabdall...@gmail.com) a écrit:

I would like to get your perspective on how to load json to index server in my 
scenario.
We have about 15 million documents in html/pdf/... on Server 1
I would like to process the data and convert to json on server 2
I would like the indexer to index json n a separate machine/server server 3

Ideally I thought on Server 2, as I prepare json and have it ready in memory, I 
can feed it to indexer. But since data processing is cpu intensive, I want 
indexing to be done on a separate machines/server.
How do you guys deal with this since I can no longer feed in-memory json to the 
indexer on separate machine? Do I just grab files from server 2 and index them 
then?
--
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/05b977ac-00d0-45c0-9e58-8df523e6978c%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/etPan.52e7f16c.74b0dc51.ec%40MacBook-Air-de-David.local.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Add a custom lucene search filter to elasticsearch

[joergpra...@gmail.com]

Adding a Lucene filter via a plugin is possible, but a bit more demanding,
you have to tweak the IndexQueryParserService like it is demonstrated in
org.elasticsearch.index.query.guice.IndexQueryParserModuleTests

Jörg

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGA9d4t2h5RSbKbXuY4oJnwVQkW_TYFpvc%3DT3Cgo0Cf%2Bw%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Add a custom lucene search filter to elasticsearch

[Ivan Brusic]

Correct, create a plugin. Take a look at some of the existing analysis
plugins as a template:
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-plugins.html#analysis-plugins

-- 
Ivan


On Tue, Jan 28, 2014 at 8:48 AM, Guilhem Legal
wrote:

> Hi!
>
> i'm trying to migrate my java application from Lucene to Elasticsearch.
>
> I have a custom java filter (extending org.apache.lucene.search.Filter)
> that i like to include in elasticSearch.
>
> How can i do that? with a plugin ?
>
> Guilhem Legal
>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/2243c8a6-ba15-43b5-9651-297ade39d73f%40googlegroups.com
> .
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQDFyy0sacOh8OmuBh9b%2BDAueRrfsL_9mb%2Bx71LfZxHEKQ%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Marvel behind Nginx and https

[Ivan Brusic]

Understood. :) Can't quite parse twitterspeak early in the morning. It was
interesting seeing a post about Marvel and then immediately someone on the
mailing list asking a question. Wow, that was fast.

For those that missed the news:
http://www.elasticsearch.com/blog/introducing-elasticsearch-marvel-native-monitoring-deployments/


On Tue, Jan 28, 2014 at 8:22 AM, J. Schulz  wrote:

> You wrote: "I am assuming that the elasticsearch team has Marvel as a
> private repo, which means you can't post issues."
>
> I just wanted to show you that I already searched for a place where to
> report "bugs" for Marvel and ask Shay on Twitter.
>
> Is it clear now? Have a nice day :-)
>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/1d3b1c85-4cf9-43fd-bd40-72e728b3ddaa%40googlegroups.com
> .
>
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQCRARt0KPAqu0HkMOiMOEv5hp4pbnf9VXb-pKM2UVJeZw%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Add a custom lucene search filter to elasticsearch

[Guilhem Legal]

Hi!

i'm trying to migrate my java application from Lucene to Elasticsearch.

I have a custom java filter (extending org.apache.lucene.search.Filter) 
that i like to include in elasticSearch.

How can i do that? with a plugin ?

Guilhem Legal

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/2243c8a6-ba15-43b5-9651-297ade39d73f%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Marvel behind Nginx and https

[J. Schulz]

You wrote: "I am assuming that the elasticsearch team has Marvel as a 
private repo, which means you can't post issues."

I just wanted to show you that I already searched for a place where to 
report "bugs" for Marvel and ask Shay on Twitter.

Is it clear now? Have a nice day :-)

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/1d3b1c85-4cf9-43fd-bd40-72e728b3ddaa%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Optional asciifolding

[Itamar Syn-Hershko]

Ok so the idea is you store each term twice - once stemmed (+ ascii folded
+ whatever) and once just lowercased, and add a character (we used $) to
mark that term as the "original".

You can see it in action here:
https://github.com/synhershko/elasticsearch-analysis-hebrew/blob/master/src/main/java/com/code972/elasticsearch/analysis/HebrewIndexingAnalyzer.java#L20
(warning: plugin still under work, and is using some non-traditional
methods to do stuff)

There's some details to take into account - like how to search for the
original etc, but if you'll look at the code there you'll get an idea of
how its done

We did that also for non-Hebrew and non-English texts. It works quite
nicely, but it doubles the amount of terms in your index.

--

Itamar Syn-Hershko
http://code972.com | @synhershko 
Freelance Developer & Consultant
Author of RavenDB in Action 


On Tue, Jan 28, 2014 at 6:09 PM, Nikolas Everett  wrote:

> I'd prefer multiple terms in the same position if I can get away with it.
> That way it'd all be configured by the analyzer so it wouldn't add any
> extra complexity to other languages.  It'd take up much less space that way
> as well.
>
>
> On Tue, Jan 28, 2014 at 11:04 AM, Itamar Syn-Hershko 
> wrote:
>
>> You will have to use 2 fields, or multiple terms on the same position. In
>> a recent project we found a nice way of dealing with that on the same
>> field, I hope to have a blog post about that soon..
>>
>> --
>>
>> Itamar Syn-Hershko
>> http://code972.com | @synhershko 
>> Freelance Developer & Consultant
>> Author of RavenDB in Action 
>>
>>
>> On Tue, Jan 28, 2014 at 6:00 PM, Nikolas Everett wrote:
>>
>>>  I'm looking to make asciifolding optional in my (English) index. If
>>> the user searches without any high ascii characters then I want to match
>>> against the folded tokens. If the user searches with high ascii characters
>>> then I only want to match the unfolded tokens.  Is this possible with
>>> Elasticsearch right now?
>>>
>>> Nik
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "elasticsearch" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to elasticsearch+unsubscr...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/elasticsearch/CAPmjWd3w%2BNHJZQkcCRnEKuowAuObkBTVbHEhnCFpkLH7y0Pa0Q%40mail.gmail.com
>>> .
>>> For more options, visit https://groups.google.com/groups/opt_out.
>>>
>>
>>  --
>> You received this message because you are subscribed to the Google Groups
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to elasticsearch+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/elasticsearch/CAHTr4ZuXY6wTDZJNEmwrXN8dRESSYJrKSkcHvSC6KkzYp4TLtg%40mail.gmail.com
>> .
>>
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>
>  --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/CAPmjWd2imi0WzibxZC_KmeK0J139fR6zjB5H0ij1fdLoxvzJzQ%40mail.gmail.com
> .
>
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAHTr4Zt2LWsXrQ-fXL49u-azWpGvCqvUh-%2BdN13nYT1SqrOFEQ%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Modifying scoring algorithm during search operations

[Ivan Brusic]

Norms are applied at the field level, not at the index level. You would
need to omit norms for every field it is meant to apply to. Another
alternative would be to use index templates:
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/indices-templates.html

-- 
Ivan



On Mon, Jan 27, 2014 at 10:32 PM, Hiro Gangwani wrote:

> Hi Ivan,
> Thanks for the reply. We tried using norms.enabled property and it is
> working fine. But what we have observed is this attribute works only on
> string types. In our application we are indexing the word (.doc,.docx) and
> pdf documents and performing test based search from document content. When
> we define the norm.enabled for attachments types, normalization is not
> working and size of document is being considered while calculating the
> score.
>
> Please suggest how do resolve this issue for attachment types.
>
> Code to create the index for attachment types
> ---
> XContentBuilder map = XContentFactory.jsonBuilder().startObject()
> .startObject(idxType)
>  .startObject("properties")
>  .startObject("file")
>  .field("type", "attachement")
>  .field("norms.enabled", false)
>  .startObject("fields")
>  .startObject("refid")
>  .field("store", "yes")
>  .endObject()
>  .startObject("name")
>  .field("store", "yes")
>  .endObject()
>  .startObject("itexp")
>  .field("store", "yes")
>  .endObject()
>  .startObject("totalexp")
>  .field("store", "yes")
>  .endObject()
>  .endObject()
>  .endObject()
> .endObject()
> .endObject();
> ---
>
>
>
> Hiro
>
>
> On Monday, 27 January 2014 23:50:41 UTC+5:30, Ivan Brusic wrote:
>
>> For the third rule, you can omit index norms for a field which will
>> prevent length normalization. See [1]. The option is either
>> called omit_norms or norms.enabled depending on your version.
>>
>> For the second rule, it is slightly more complicated. You can define your
>> own custom similarity [2] that dictates how the TF, IDF and norms are used.
>> You simply extends Lucene's DefaultSimilarity (of TDIDFSimilarity) and at
>> it to elasticsearch's classpath.
>>
>> [1] http://www.elasticsearch.org/guide/en/elasticsearch/
>> reference/current/mapping-core-types.html#string
>> [2] http://www.elasticsearch.org/guide/en/elasticsearch/
>> reference/current/index-modules-similarity.html
>>
>> --
>> Ivan
>>
>>
>> On Sun, Jan 26, 2014 at 11:12 PM, Hiro Gangwani wrote:
>>
>>> Dear Team,
>>>
>>> I have been looking at search algorithm being used in elastic search and
>>> found following set of rules which are applied while calculating the score
>>> (Boolean Model)
>>>
>>>
>>>- more occurrences in the document are preferred
>>>- terms rarer in the corpus are preferred
>>>- shorter documents are more heavily weighted
>>>- other functions used to adjust score, boosts, etc.
>>>
>>> In my application we are doing text based search across set of word
>>> documents. We would like to assign the higher scroe to documents having
>>> more occurances and show at the top irrespective of size of document.
>>> Primarily our application is recruitment system where is search is based
>>> upon skill sets. So our business team wants to show the resumes having more
>>> occurrences of search key words at top irrespective of size and rare terms.
>>> Is there any mechanism to ignore second and third rules as listed below
>>> and calculate the score based upon More occurrences condition only. We are
>>> executing search operations using Java API. Please let me know is it
>>> possible to achieve the same and if yes how?
>>>
>>> Thanks in advance for suggesting solution.
>>>
>>> Hiro
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "elasticsearch" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to elasticsearc...@googlegroups.com.
>>>
>>> To view this discussion on the web visit https://groups.google.com/d/
>>> msgid/elasticsearch/f6936b6f-ef7c-4497-b186-bdba28176d89%
>>> 40googlegroups.com.
>>> For more options, visit https://groups.google.com/groups/opt_out.
>>>
>>
>>  --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/f80933eb-1b68-4c6f-b073-39b78e3f45e9%40googlegroups.com
> .
>
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elastics

Re: Optional asciifolding

[Itamar Syn-Hershko]

You will have to use 2 fields, or multiple terms on the same position. In a
recent project we found a nice way of dealing with that on the same field,
I hope to have a blog post about that soon..

--

Itamar Syn-Hershko
http://code972.com | @synhershko 
Freelance Developer & Consultant
Author of RavenDB in Action 


On Tue, Jan 28, 2014 at 6:00 PM, Nikolas Everett  wrote:

> I'm looking to make asciifolding optional in my (English) index. If the
> user searches without any high ascii characters then I want to match
> against the folded tokens. If the user searches with high ascii characters
> then I only want to match the unfolded tokens.  Is this possible with
> Elasticsearch right now?
>
> Nik
>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/CAPmjWd3w%2BNHJZQkcCRnEKuowAuObkBTVbHEhnCFpkLH7y0Pa0Q%40mail.gmail.com
> .
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAHTr4ZuXY6wTDZJNEmwrXN8dRESSYJrKSkcHvSC6KkzYp4TLtg%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Marvel behind Nginx and https

[Ivan Brusic]

Not sure what you are trying to accomplish by applying Twitter conventions
on a mailing list. :)

Shay has not been on the mailing list in a long time. Eventually someone
from elasticsearch will read your email.

-- 
Ivan


On Tue, Jan 28, 2014 at 7:54 AM, J. Schulz  wrote:

> I know ;-)
>
> @kimchy where to report bugs? On
> http://github.com/elasticsearch/elasticsearch ...?
> @bloonix for now, best place is the google group:
> https://groups.google.com/forum/#!forum/elasticsearch ...
>
>
> Am Dienstag, 28. Januar 2014 16:51:39 UTC+1 schrieb Ivan Brusic:
>>
>> Marvel was just announced today (to me at least), and there already is a
>> question/issue? :) Let us know how it is working out.
>>
>> I am assuming that the elasticsearch team has Marvel as a private repo,
>> which means you can't post issues.
>>
>> --
>> Ivan
>>
>>
>>
>>
>> On Tue, Jan 28, 2014 at 7:45 AM, J. Schulz  wrote:
>>
>>> Hi,
>>>
>>> I have Nginx configured as reverse proxy to access elasticsearch over
>>> https + auth basic. As example
>>>
>>> Unfortunately Marvel tries to connect to http://hostname/.
>>>
>>> The affected code line is in /usr/share/elasticsearch/plugi
>>> ns/marvel/_site/kibana/config.js
>>>
>>> elasticsearch: "http://"+window.location.hostname+(window.location.port
>>> !== '' ? ':'+window.location.port : ''),
>>>
>>> should be
>>>
>>> elasticsearch: window.location.protocol+"//"+window.location.hostname+(
>>> window.location.port !== '' ? ':'+window.location.port : '')
>>>
>>> Cheers,
>>> Jonny
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "elasticsearch" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to elasticsearc...@googlegroups.com.
>>> To view this discussion on the web visit https://groups.google.com/d/
>>> msgid/elasticsearch/c536426a-36a6-4a24-888c-95ff0a4d053f%
>>> 40googlegroups.com.
>>>
>>> For more options, visit https://groups.google.com/groups/opt_out.
>>>
>>
>>  --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/897ac725-e037-4f77-9541-ec56a90b00bf%40googlegroups.com
> .
>
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQDvY_ETGisEGiOyKNLirn%2Bo8PiD8xhe%2BQij0PQB4Jbz_A%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Optional asciifolding

[Nikolas Everett]

I'd prefer multiple terms in the same position if I can get away with it.
That way it'd all be configured by the analyzer so it wouldn't add any
extra complexity to other languages.  It'd take up much less space that way
as well.


On Tue, Jan 28, 2014 at 11:04 AM, Itamar Syn-Hershko wrote:

> You will have to use 2 fields, or multiple terms on the same position. In
> a recent project we found a nice way of dealing with that on the same
> field, I hope to have a blog post about that soon..
>
> --
>
> Itamar Syn-Hershko
> http://code972.com | @synhershko 
> Freelance Developer & Consultant
> Author of RavenDB in Action 
>
>
> On Tue, Jan 28, 2014 at 6:00 PM, Nikolas Everett wrote:
>
>>  I'm looking to make asciifolding optional in my (English) index. If the
>> user searches without any high ascii characters then I want to match
>> against the folded tokens. If the user searches with high ascii characters
>> then I only want to match the unfolded tokens.  Is this possible with
>> Elasticsearch right now?
>>
>> Nik
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to elasticsearch+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/elasticsearch/CAPmjWd3w%2BNHJZQkcCRnEKuowAuObkBTVbHEhnCFpkLH7y0Pa0Q%40mail.gmail.com
>> .
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>
>  --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/CAHTr4ZuXY6wTDZJNEmwrXN8dRESSYJrKSkcHvSC6KkzYp4TLtg%40mail.gmail.com
> .
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAPmjWd2imi0WzibxZC_KmeK0J139fR6zjB5H0ij1fdLoxvzJzQ%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Marvel behind Nginx and https

[J. Schulz]

I know ;-)

@kimchy where to report bugs? On 
http://github.com/elasticsearch/elasticsearch …?
@bloonix for now, best place is the google group: 
https://groups.google.com/forum/#!forum/elasticsearch …


Am Dienstag, 28. Januar 2014 16:51:39 UTC+1 schrieb Ivan Brusic:
>
> Marvel was just announced today (to me at least), and there already is a 
> question/issue? :) Let us know how it is working out.
>
> I am assuming that the elasticsearch team has Marvel as a private repo, 
> which means you can't post issues. 
>
> -- 
> Ivan
>
>
>
>
> On Tue, Jan 28, 2014 at 7:45 AM, J. Schulz 
> > wrote:
>
>> Hi,
>>
>> I have Nginx configured as reverse proxy to access elasticsearch over 
>> https + auth basic. As example
>>
>> Unfortunately Marvel tries to connect to http://hostname/.
>>
>> The affected code line is in /usr/share/elasticsearch/
>> plugins/marvel/_site/kibana/config.js
>>
>> elasticsearch: "http://"+window.location.hostname+(window.location.port 
>> !== '' ? ':'+window.location.port : ''),
>>
>> should be
>>
>> elasticsearch: 
>> window.location.protocol+"//"+window.location.hostname+(window.location.port 
>> !== '' ? ':'+window.location.port : '')
>>
>> Cheers,
>> Jonny
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to elasticsearc...@googlegroups.com .
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/elasticsearch/c536426a-36a6-4a24-888c-95ff0a4d053f%40googlegroups.com
>> .
>>
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/897ac725-e037-4f77-9541-ec56a90b00bf%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Optional asciifolding

[Nikolas Everett]

I'm looking to make asciifolding optional in my (English) index. If the
user searches without any high ascii characters then I want to match
against the folded tokens. If the user searches with high ascii characters
then I only want to match the unfolded tokens.  Is this possible with
Elasticsearch right now?

Nik

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAPmjWd3w%2BNHJZQkcCRnEKuowAuObkBTVbHEhnCFpkLH7y0Pa0Q%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Marvel behind Nginx and https

[Ivan Brusic]

Marvel was just announced today (to me at least), and there already is a
question/issue? :) Let us know how it is working out.

I am assuming that the elasticsearch team has Marvel as a private repo,
which means you can't post issues.

-- 
Ivan




On Tue, Jan 28, 2014 at 7:45 AM, J. Schulz  wrote:

> Hi,
>
> I have Nginx configured as reverse proxy to access elasticsearch over
> https + auth basic. As example
>
> Unfortunately Marvel tries to connect to http://hostname/.
>
> The affected code line is in /usr/share/elasticsearch/
> plugins/marvel/_site/kibana/config.js
>
> elasticsearch: "http://"+window.location.hostname+(window.location.port
> !== '' ? ':'+window.location.port : ''),
>
> should be
>
> elasticsearch: 
> window.location.protocol+"//"+window.location.hostname+(window.location.port
> !== '' ? ':'+window.location.port : '')
>
> Cheers,
> Jonny
>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/c536426a-36a6-4a24-888c-95ff0a4d053f%40googlegroups.com
> .
>
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQCmZM31%3DN6qa3XiLRCCm34M%2Bgq0Hn-qHth0jXPsEUg%3DSw%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Marvel behind Nginx and https

[J. Schulz]

Hi,

I have Nginx configured as reverse proxy to access elasticsearch over https 
+ auth basic. As example

Unfortunately Marvel tries to connect to http://hostname/.

The affected code line is in 
/usr/share/elasticsearch/plugins/marvel/_site/kibana/config.js

elasticsearch: "http://"+window.location.hostname+(window.location.port !== 
'' ? ':'+window.location.port : ''),

should be

elasticsearch: 
window.location.protocol+"//"+window.location.hostname+(window.location.port 
!== '' ? ':'+window.location.port : '')

Cheers,
Jonny

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/c536426a-36a6-4a24-888c-95ff0a4d053f%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Loading JSON to ElasticSearch

[ZenMaster80]

I would like to get your perspective on how to load json to index server in 
my scenario.
We have about 15 million documents in html/pdf/... on Server 1
I would like to process the data and convert to json on server 2
I would like the indexer to index json n a separate machine/server server 3

Ideally I thought on Server 2, as I prepare json and have it ready in 
memory, I can feed it to indexer. But since data processing is cpu 
intensive, I want indexing to be done on a separate machines/server.
How do you guys deal with this since I can no longer feed in-memory json to 
the indexer on separate machine? Do I just grab files from server 2 and 
index them then?

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/05b977ac-00d0-45c0-9e58-8df523e6978c%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Marvel behind Nginx and https

[J. Schulz]

Hi,

I have Nginx configured as reverse proxy to access elasticsearch over https 
+ auth basic. As example

Unfortunately Marvel tries to connect to http://hostname/.

The affected code line is in 
/usr/share/elasticsearch/plugins/marvel/_site/kibana/config.js

elasticsearch: "https://"+window.location.hostname+(window.location.port 
!== '' ? ':'+window.location.port : ''),

should be

elasticsearch: 
window.location.protocol+"//"+window.location.hostname+(window.location.port 
!== '' ? ':'+window.location.port : '')

Cheers,
Jonny

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/b8f2d612-871c-43bc-aaf6-4a27bbd43d41%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Printing all children recursively while modeling parent-child mapping

[jayanth saimani]

I am just getting started with elastic search , and one of our use cases we 
are trying to model our data which is very hirerchical in nature as a 
parent child.
Was wondering if i have a document, could print all its children or the 
parent hirerchy recursively . 
Is this even possible?

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/ec70a28e-fec6-4fcd-892b-888416fad39c%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Complex or impossible query

[Ernesto Reig]

Hello, I have a requirement we need to implement and I really don´t know 
how to do it or even if it is possible (maybe the solution is easy but I 
just don´t know).
We have Person documents like this:
{
  "id": 1,
  "name": "Ernesto",
  "NID": "AAA"
}

{
  "id": 2,
  "name": "Enrique",
  "NID": "AAA"
}

{
  "id": 3,
  "name": "Antonio",
  "NID": "BBB"
}

{
  "id": 4,
  "name": "Karlos",
  "NID": "CCC"
}

Imagine we search for Persons with ["NID": "AAA"], we would get 2 documents 
(id 1 and id 2).
If we search for Persons with ["NID": "BBB"], we would get 1 document (id 
3).
And if we search for Persons with ["NID": "CCC"], we would get 1 document 
(id 3).
Ok, so what we need is get all the documents in which the NID is unique. I 
mean, the _count would be only 1 if we search for the NID of those 
documents. The results from that query/filter would be documents 3 and 4, 
since the NID value is unique for those documents (there are no more 
documents in the index with that NID).

Thank you very much,

Ernesto

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/977fa0d9-1b88-4147-8073-dd4d29aff1a6%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: String and Float for a multi_field?

[Binh Ly]

Arthur,

1) That section is only to tell ES how to order documents when the field is 
missing. So if you specify for example: 

"sort" : [
{ "price" : {"missing" : "_last"} },
]


If the price field is missing, the document will be pushed to the bottom of 
the results. But this still does not help when you define multi-fields with 
numeric type and want to store a non-numeric string value.

2) It sounds fine to me. You don't even need a multi-field if you don't 
care about the string value. So for example, you can do something like this:

"foo": {
  "type": "float",
  "ignore_malformed": true
}

And then just sort by "foo" numerically.

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/7293f0c9-fafe-49b2-96e3-b0f54396b84c%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: How to get mappings using the Java API with ElasticSearch 0.90.10 and 1.0.0 RC1

[Luca Cavanna]

True, although this relates to the REST api but not to the Java API, for
which the GetMappingsRequest stays the same. As I said, we introduced the
get field mapping api and related GetFieldMappingsRequest.


On Tue, Jan 28, 2014 at 3:13 PM, Roy Russo  wrote:

> Just a side note that the API response format changed in 1.0.0RC1.
> http://www.elasticsearch.org/guide/en/elasticsearch/reference/master/_indices_apis.html
>
>
> On Tuesday, January 28, 2014 3:15:57 AM UTC-5, Thierry Templier wrote:
>>
>> Hello,
>>
>> I wonder what is the best way to get the mappings for an index using the
>> ElasticSearch Java API with
>> version 0.90.10. I saw that there is only a GetFieldMappingsRequest class
>> for this version whereas
>> a class GetMappingsRequest is now present in 1.0RC1.
>>
>> Thanks very much for your answer and your help!
>> Thierry
>>
>  --
> You received this message because you are subscribed to a topic in the
> Google Groups "elasticsearch" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/elasticsearch/7TOQO2piFV4/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/568456a2-c76b-43ef-ac0a-76b1bd4faffd%40googlegroups.com
> .
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CADdZ9MUzJntjtYO--_ZqsjXwkOzFG3kT2WV0YJcZp3jHiV%2BTmQ%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: How to get mappings using the Java API with ElasticSearch 0.90.10 and 1.0.0 RC1

[Roy Russo]

Just a side note that the API response format changed in 
1.0.0RC1. 
http://www.elasticsearch.org/guide/en/elasticsearch/reference/master/_indices_apis.html


On Tuesday, January 28, 2014 3:15:57 AM UTC-5, Thierry Templier wrote:
>
> Hello,
>
> I wonder what is the best way to get the mappings for an index using the 
> ElasticSearch Java API with
> version 0.90.10. I saw that there is only a GetFieldMappingsRequest class 
> for this version whereas
> a class GetMappingsRequest is now present in 1.0RC1.
>
> Thanks very much for your answer and your help!
> Thierry 
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/568456a2-c76b-43ef-ac0a-76b1bd4faffd%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

More like multiple documents

[James Massey]

I'm looking for a way to suggest documents for a user based on documents 
they have previously saved in our system. I have a list of document ids in 
Elasticsearch and I'd like to suggest documents that are related to that 
corpus of saved documents. In Solr this can be done through Carrot 
clustering on those document ids, but the Elasicsearch Carrot plugin 
doesn't fetch additional documents related to the clusters like Solr does. 
Does anyone have any suggestions for an approach/solution to this problem?

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/d9bddba8-54a1-4993-9c4c-fdc924692b3a%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: order of the elements does matter?

[Nikolay Chankov]

Thanks got clarification Zachary,

I was expected an exception too. Anyway, I need to change my query.

Thanks for your help guys!

On Tuesday, January 28, 2014 12:28:20 PM UTC, Zachary Tong wrote:
>
> So the root cause is that your query is structured incorrectly.  The 
> "match_all" should be inside of a "query" element, inside the "filtered" 
> query:
>
> curl -XGET "http://localhost:9200/test_search/_search?pretty=true"; -d'
> {
> "query": {
> "filtered": {
> "query" : {"match_all": {}},
> "filter": {
> "term": {
> "object": "User"
> }
> }
> }
> },
> "size" : 2
> }'
>
>
> Although this is technically a syntax error, it is very unfriendly of 
> Elasticsearch to not throw an exception and let you know.  There is a PR to 
> fix this problem and it'll probably be merged soon:  
> https://github.com/elasticsearch/elasticsearch/pull/4913
>
> In the future Elasticsearch will throw an exception instead of silently 
> eating the error and giving strange results.
>
> -Zach
>
>
>
> On Tuesday, January 28, 2014 3:48:26 AM UTC-5, Nikolay Chankov wrote:
>>
>> Hi David,
>>
>> Here is full gist:
>>
>> curl -XDELETE 'http://localhost:9200/test_search'
>> curl -XPUT 'http://localhost:9200/test_search/' -d '
>> {
>> "mappings" : {
>> "record" : {
>> "properties" : {
>> "object" : { 
>> "type" : "string",
>> "index" : "not_analyzed"
>> },
>> "name" : { 
>> "type" : "string"
>> }
>> }
>> }
>> }
>> }
>> '
>> curl -XPUT 'http://localhost:9200/test_search/record/1' -d '{
>> "object" : "User",
>> "name" : "John Doe"
>> }'
>> curl -XPUT 'http://localhost:9200/test_search/record/2' -d '{
>> "object" : "User",
>> "name" : "Jane Doe"
>> }'
>> curl -XPUT 'http://localhost:9200/test_search/record/3' -d '{
>> "object" : "User",
>> "name" : "Joseph Doe"
>> }'
>> curl -XPUT 'http://localhost:9200/test_search/record/4' -d '{
>> "object" : "User",
>> "name" : "Anna Doe"
>> }'
>> curl -XPUT 'http://localhost:9200/test_search/record/5' -d '{
>> "object" : "Venue",
>> "name" : "Bar Luna"
>> }'
>>
>> curl -XGET 'http://localhost:9200/test_search/_search?pretty=true' -d '{
>> "query": {
>> "match_all": {},
>> "filtered": {
>> "filter": {
>> "term": {
>> "object": "User"
>> }
>> }
>> }
>> },
>> "size" : 2
>> }'
>>
>> I've noticed that the problem exist only if under the top "query" node 
>> there are 2 elements. If I remove "match_all" or "filtered" section the 
>> size does take effect.
>> I've combined the examples in "And Filter" + "Term Filter" to create the 
>> query, but probably this is the wrong way?
>>
>> Thanks
>>
>> On Monday, January 27, 2014 7:42:15 PM UTC, David Pilato wrote:
>>>
>>> Yes please. If you can gist a full curl recreation, that will help a lot!
>>>
>>> --
>>> David ;-)
>>> Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs
>>>
>>>
>>> Le 27 janv. 2014 à 19:36, Nikolay Chankov  a écrit :
>>>
>>> I've noticed, that the "problem" came when in the request there is a 
>>> "filtered" node. Here is the full request:
>>>
>>> curl-XGET 'http://localhost/search/_search' -d'{
>>>   "query": {
>>> "match_all": {},
>>> "filtered": {
>>>   "filter": {
>>> "term": {
>>>   "object": "User"
>>> }
>>>   }
>>> }
>>>   },
>>>   "size": 3,
>>>   "sort": [
>>> {
>>>   "name.untouched": "asc"
>>> }
>>>   ]
>>> }'
>>>
>>> So, if it's called this way the sort and size are ignored, while if they 
>>> are placed above the query, they take effect, and I can see 3 records.
>>> if it's not correct, I would expect to get an error, rather than 
>>> ignoring the params...
>>>
>>> name is a multi_field with name.untouched is index not analyzed, object 
>>> is string, not analyzed. If it's still required I will try to create a full 
>>> gist tomorrow.
>>>
>>>
>>> On Monday, January 27, 2014 5:54:48 PM UTC, David Pilato wrote:

  Can you reproduce it with a full curl recreation and gist it?
 In which version?

 If confirmed, could you open an issue?

 -- 
 *David Pilato* | *Technical Advocate* | *Elasticsearch.com 
 *
 @dadoonet  | 
 @elasticsearchfr


 Le 27 janvier 2014 at 18:50:32, Nikolay Chankov (ncha...@gmail.com) a 
 écrit:

 Hi guys, 

 today I've noticed that order of the elements in the request does 
 matter for example:

  curl -XGET 'http://localhost:9200/search/_search'-d '
 {
"sort" : {...},
"size" : 100,
"query" : {...}
 }'
  
 is w

Re: order of the elements does matter?

[Zachary Tong]

So the root cause is that your query is structured incorrectly.  The 
"match_all" should be inside of a "query" element, inside the "filtered" 
query:

curl -XGET "http://localhost:9200/test_search/_search?pretty=true"; -d'
{
"query": {
"filtered": {
"query" : {"match_all": {}},
"filter": {
"term": {
"object": "User"
}
}
}
},
"size" : 2
}'


Although this is technically a syntax error, it is very unfriendly of 
Elasticsearch to not throw an exception and let you know.  There is a PR to 
fix this problem and it'll probably be merged soon: 
 https://github.com/elasticsearch/elasticsearch/pull/4913

In the future Elasticsearch will throw an exception instead of silently 
eating the error and giving strange results.

-Zach



On Tuesday, January 28, 2014 3:48:26 AM UTC-5, Nikolay Chankov wrote:
>
> Hi David,
>
> Here is full gist:
>
> curl -XDELETE 'http://localhost:9200/test_search'
> curl -XPUT 'http://localhost:9200/test_search/' -d '
> {
> "mappings" : {
> "record" : {
> "properties" : {
> "object" : { 
> "type" : "string",
> "index" : "not_analyzed"
> },
> "name" : { 
> "type" : "string"
> }
> }
> }
> }
> }
> '
> curl -XPUT 'http://localhost:9200/test_search/record/1' -d '{
> "object" : "User",
> "name" : "John Doe"
> }'
> curl -XPUT 'http://localhost:9200/test_search/record/2' -d '{
> "object" : "User",
> "name" : "Jane Doe"
> }'
> curl -XPUT 'http://localhost:9200/test_search/record/3' -d '{
> "object" : "User",
> "name" : "Joseph Doe"
> }'
> curl -XPUT 'http://localhost:9200/test_search/record/4' -d '{
> "object" : "User",
> "name" : "Anna Doe"
> }'
> curl -XPUT 'http://localhost:9200/test_search/record/5' -d '{
> "object" : "Venue",
> "name" : "Bar Luna"
> }'
>
> curl -XGET 'http://localhost:9200/test_search/_search?pretty=true' -d '{
> "query": {
> "match_all": {},
> "filtered": {
> "filter": {
> "term": {
> "object": "User"
> }
> }
> }
> },
> "size" : 2
> }'
>
> I've noticed that the problem exist only if under the top "query" node 
> there are 2 elements. If I remove "match_all" or "filtered" section the 
> size does take effect.
> I've combined the examples in "And Filter" + "Term Filter" to create the 
> query, but probably this is the wrong way?
>
> Thanks
>
> On Monday, January 27, 2014 7:42:15 PM UTC, David Pilato wrote:
>>
>> Yes please. If you can gist a full curl recreation, that will help a lot!
>>
>> --
>> David ;-)
>> Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs
>>
>>
>> Le 27 janv. 2014 à 19:36, Nikolay Chankov  a écrit :
>>
>> I've noticed, that the "problem" came when in the request there is a 
>> "filtered" node. Here is the full request:
>>
>> curl-XGET 'http://localhost/search/_search' -d'{
>>   "query": {
>> "match_all": {},
>> "filtered": {
>>   "filter": {
>> "term": {
>>   "object": "User"
>> }
>>   }
>> }
>>   },
>>   "size": 3,
>>   "sort": [
>> {
>>   "name.untouched": "asc"
>> }
>>   ]
>> }'
>>
>> So, if it's called this way the sort and size are ignored, while if they 
>> are placed above the query, they take effect, and I can see 3 records.
>> if it's not correct, I would expect to get an error, rather than ignoring 
>> the params...
>>
>> name is a multi_field with name.untouched is index not analyzed, object 
>> is string, not analyzed. If it's still required I will try to create a full 
>> gist tomorrow.
>>
>>
>> On Monday, January 27, 2014 5:54:48 PM UTC, David Pilato wrote:
>>>
>>>  Can you reproduce it with a full curl recreation and gist it?
>>> In which version?
>>>
>>> If confirmed, could you open an issue?
>>>
>>> -- 
>>> *David Pilato* | *Technical Advocate* | *Elasticsearch.com 
>>> *
>>> @dadoonet  | 
>>> @elasticsearchfr
>>>
>>>
>>> Le 27 janvier 2014 at 18:50:32, Nikolay Chankov (ncha...@gmail.com) a 
>>> écrit:
>>>
>>> Hi guys, 
>>>
>>> today I've noticed that order of the elements in the request does matter 
>>> for example:
>>>
>>>  curl -XGET 'http://localhost:9200/search/_search'-d '
>>> {
>>>"sort" : {...},
>>>"size" : 100,
>>>"query" : {...}
>>> }'
>>>  
>>> is working, while
>>>
>>>  curl -XGET 'http://localhost:9200/search/_search'-d '
>>> {
>>>"query" : {...},
>>>"sort" : {...},
>>>"size" : 100
>>> }'
>>>  
>>> Doesn't take effect of size as well as on sort. 
>>>
>>> I think the order shouldn't matter, and ES should reorder the elements 
>>> internally. Am I get it wrong, or there is special reason for this?
>>>
>>> Thanks in advance. 
>

Re: Any possibility of permalinks to documentation for each version?

[Zachary Tong]

The documentation will follow the Elasticsearch branching structure, which 
will include minor branches too (1.x, etc).  Minor versions aren't 
available right now since the versioned doc support was just recently added 
to 0.90.x.

"Current" refers to the currently recommended, stable version.  For 
example, the most recent version is 1.0.RC1, but the "current" version is 
still 0.90.10 since the RC isn't a GA release yet.

Hope that helps!



On Tuesday, January 28, 2014 4:42:06 AM UTC-5, Tim S wrote:
>
> "It doesn't seem like they support minor versions." - this is exactly what 
> I'm asking "them" to do...
>
> It doesn't have to be linked to from the main page, but a stable link that 
> I know will still be there a year or two in the future will be very useful.
>
> On Monday, January 27, 2014 7:54:47 PM UTC, Ivan Brusic wrote:
>>
>> Here are the current branches: 
>> http://www.elasticsearch.org/guide/en/elasticsearch/reference/index.html
>>
>>
>> On Mon, Jan 27, 2014 at 11:53 AM, Ivan Brusic  wrote:
>>
>>> The documentation is now versioned. For example (random page)
>>>
>>> 0.90:  
>>> http://www.elasticsearch.org/guide/en/elasticsearch/reference/0.90/search-search.html
>>> master: 
>>> http://www.elasticsearch.org/guide/en/elasticsearch/reference/master/search-search.html
>>> current: 
>>> http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-search.html
>>>
>>> I am guessing a 1.0 version will be built once it is released. It 
>>> doesn't seem like they support minor versions. Not sure what "current" 
>>> signifies, but you can probably figure it out by reading the documentation 
>>> building script source: https://github.com/elasticsearch/docs
>>>  
>>> -- 
>>> Ivan
>>>
>>>
>>> On Mon, Jan 27, 2014 at 1:27 AM, Tim S  wrote:
>>>
 I notice the docs for version 1.0.0 are in a "1.x" branch. When 1.1 is 
 released, I'm assuming the 1.x branch docs will be updated to 1.1?

 Is there any possibility of creating permanent links to each version? 
 (like lucene do, e.g http://lucene.apache.org/core/3_6_1/). The reason 
 is simply that if I build a solution for a customer that requires a 
 specific version of elasticsearch, then my documentation will probably end 
 up referencing various bits of the elasticsearch documentation. Which is 
 tricky if the documentation I'm referencing is changing.

 So, I was wondering if it would be possible to create links for each 
 release version, so instead of referencing 
 http://www.elasticsearch.org/guide/en/elasticsearch/reference/1.x/search.htmlI
  can reference 
 http://www.elasticsearch.org/guide/en/elasticsearch/reference/1.0.0.Release/search.htmlor
  similar, and then I can be happy that the documentation I'm linking to 
 is the exact documentation for the version that my customer uses.

 Thanks,

 Tim.

 -- 
 You received this message because you are subscribed to the Google 
 Groups "elasticsearch" group.
 To unsubscribe from this group and stop receiving emails from it, send 
 an email to elasticsearc...@googlegroups.com.
 To view this discussion on the web visit 
 https://groups.google.com/d/msgid/elasticsearch/dd467b38-da9b-4f4e-9ebe-f39adb403f2f%40googlegroups.com
 .
 For more options, visit https://groups.google.com/groups/opt_out.

>>>
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/815cb53c-867d-4061-bd71-dc6cf016fa2f%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: How to monitor for filter cache churn?

[Zachary Tong]

Oh, final note that I forgot:  if you cache the boolean combination of 
filters, the individual leaf node filters will still be cached by default. 
 You'll have to explicitly disable caching for leaf nodes if you don't want 
that behavior.

-Z


On Tuesday, January 28, 2014 5:58:21 AM UTC-5, Zachary Tong wrote:
>
> You can monitor filter cache from three different levels - index, node and 
> cluster.  The output is similar for all three outputs, you'll see a size in 
> bytes and an eviction count.
>
>- Per-index: 
>curl -XGET "http://localhost:9200//_stats"
>
>- Per-node: 
>curl -XGET "http://localhost:9200/_nodes/stats";
>
>- Entire Cluster (this is actually pretty new, introduced in 0.90.8): 
>curl -XGET "http://localhost:9200/_cluster/stats";
>
>
> Regarding your question about bitset combination speed and caching 
> intermediate booleans...it depends (heh).  The question is less about the 
> speed of combining say 50 individual bitsets compared to 1 combined bitset. 
>  The single bitset will obviously be faster, but the speed difference is 
> pretty negligible compared to other operations.
>
> What you should be thinking about is the effect of a "cache miss".  Let's 
> operate under the assumption that filter cache size is limited, and 
> evictions will occur in some fashion (otherwise we'd just keep everything 
> in memory and be happy).   
>
> If you have a boolean combination of 50 filters that is cached, you only 
> need to keep that single filter "hot" in the cache.  If your usage pattern 
> keeps it cached, you will have very little churn.  But if it happens that 
> there is a lull and the combo-filter falls out of cache, the next time it 
> is executed you'll need to re-evaluate all 50 "interior" filters to derive 
> the final bitset.  Those interior filters could potentially touch a large 
> number of documents (and the associated disk access).  A cache miss could 
> be relatively expensive (still fast, but relative to simple bitset lookups)
>
> When filters are specified independently, there is a greater chance that 
> individual filters may be missing from the cache.  Each execution of the 
> set of filters may require a few of the "interior" filters to be evaluated, 
> but since the filters are cached separately there is a good chance the 
> majority of them remain cached.  So the computational cost 
> is amortized over time instead of being lumpy.  There is also a better 
> chance that filters stay cached since they are reused in other parts of 
> your query, which keeps them "live" even if the total combination rarely 
> re-occurs.
>
> It also gets complicated because filter caching is technically 
> per-segment.  It is possible for the *very same* filter to be cached in 
> one segment but evicted on another segment.  The LRU cache tries to evict 
> old (unused) filters, but weights towards smaller segments since they are 
> cheaper to recalculate.  This also means that data indexing has an affect 
> on filter caching, since a constant ingestion of new documents equals 
> segment merges, which clears the caches for those newly created segment.
>
> Some other assorted thoughts:
>
>- Remember that boolean caching will cache the *result* of the bool, 
>not the filter's themselves.  E.g. if you have a bool of 10 Terms, the 
>final bitset is the set of documents that matches the filters, not the 10 
>Term filters themselves.  Probably obvious, but wanted to make it clear
>
>- Eviction metrics are...meh at best.  Like I mentioned above, 
>evictions are per-segment, and weighted towards small segments.  You can 
>see high eviction rates without it actually equalling much churn (e.g. 
> lots 
>of newly created, small segments are evicting when they merge, but 95% of 
>your data is remaining safely cached).  You can even see a lot of churn 
> but 
>still get good performance, since big segments tend to keep their caches 
>around and they account for most of your data.
>
>- I'd tune by setting a cache size, timing query latency across a wide 
>variety of queries and sorta watching eviction metrics.  Bump the filter 
>cache size and repeat, see if latency gets better.  Similar idea for 
> caching
>
>- In general, I only reach for caching boolean combinations when I 
>know it will get hit very often. 
>
>
> This turned into a really long message!  Let me know if you have any 
> questions :)
> -Zach
>
>
> On Monday, January 27, 2014 5:03:47 PM UTC-5, Tikitu de Jager wrote:
>>
>> Hi folks,
>>
>> I'm optimising our queries based on the advice in Zachary Tong's 
>> presentation: 
>>   https://speakerdeck.com/polyfractal/elasticsearch-query-optimization
>> So far just switching all our query elements to filters has given a 6x 
>> speedup on a monster query (65Kchars of compact json), which is very 
>> encouraging :-)
>>
>> All our queries are auto-generated from our own query syntax, though,

Re: How to monitor for filter cache churn?

[Zachary Tong]

You can monitor filter cache from three different levels - index, node and 
cluster.  The output is similar for all three outputs, you'll see a size in 
bytes and an eviction count.

   - Per-index: 
   curl -XGET "http://localhost:9200//_stats"
   
   - Per-node: 
   curl -XGET "http://localhost:9200/_nodes/stats";
   
   - Entire Cluster (this is actually pretty new, introduced in 0.90.8): 
   curl -XGET "http://localhost:9200/_cluster/stats";
   
   
Regarding your question about bitset combination speed and caching 
intermediate booleans...it depends (heh).  The question is less about the 
speed of combining say 50 individual bitsets compared to 1 combined bitset. 
 The single bitset will obviously be faster, but the speed difference is 
pretty negligible compared to other operations.

What you should be thinking about is the effect of a "cache miss".  Let's 
operate under the assumption that filter cache size is limited, and 
evictions will occur in some fashion (otherwise we'd just keep everything 
in memory and be happy).   

If you have a boolean combination of 50 filters that is cached, you only 
need to keep that single filter "hot" in the cache.  If your usage pattern 
keeps it cached, you will have very little churn.  But if it happens that 
there is a lull and the combo-filter falls out of cache, the next time it 
is executed you'll need to re-evaluate all 50 "interior" filters to derive 
the final bitset.  Those interior filters could potentially touch a large 
number of documents (and the associated disk access).  A cache miss could 
be relatively expensive (still fast, but relative to simple bitset lookups)

When filters are specified independently, there is a greater chance that 
individual filters may be missing from the cache.  Each execution of the 
set of filters may require a few of the "interior" filters to be evaluated, 
but since the filters are cached separately there is a good chance the 
majority of them remain cached.  So the computational cost 
is amortized over time instead of being lumpy.  There is also a better 
chance that filters stay cached since they are reused in other parts of 
your query, which keeps them "live" even if the total combination rarely 
re-occurs.

It also gets complicated because filter caching is technically per-segment. 
 It is possible for the *very same* filter to be cached in one segment but 
evicted on another segment.  The LRU cache tries to evict old (unused) 
filters, but weights towards smaller segments since they are cheaper to 
recalculate.  This also means that data indexing has an affect on filter 
caching, since a constant ingestion of new documents equals segment merges, 
which clears the caches for those newly created segment.

Some other assorted thoughts:

   - Remember that boolean caching will cache the *result* of the bool, not 
   the filter's themselves.  E.g. if you have a bool of 10 Terms, the final 
   bitset is the set of documents that matches the filters, not the 10 Term 
   filters themselves.  Probably obvious, but wanted to make it clear
   
   - Eviction metrics are...meh at best.  Like I mentioned above, evictions 
   are per-segment, and weighted towards small segments.  You can see high 
   eviction rates without it actually equalling much churn (e.g. lots of newly 
   created, small segments are evicting when they merge, but 95% of your data 
   is remaining safely cached).  You can even see a lot of churn but still get 
   good performance, since big segments tend to keep their caches around and 
   they account for most of your data.
   
   - I'd tune by setting a cache size, timing query latency across a wide 
   variety of queries and sorta watching eviction metrics.  Bump the filter 
   cache size and repeat, see if latency gets better.  Similar idea for caching
   
   - In general, I only reach for caching boolean combinations when I know 
   it will get hit very often. 


This turned into a really long message!  Let me know if you have any 
questions :)
-Zach


On Monday, January 27, 2014 5:03:47 PM UTC-5, Tikitu de Jager wrote:
>
> Hi folks,
>
> I'm optimising our queries based on the advice in Zachary Tong's 
> presentation: 
>   https://speakerdeck.com/polyfractal/elasticsearch-query-optimization
> So far just switching all our query elements to filters has given a 6x 
> speedup on a monster query (65Kchars of compact json), which is very 
> encouraging :-)
>
> All our queries are auto-generated from our own query syntax, though, so 
> if we switch to filters it's gonna have to be pretty much across the board 
> (all terminals in the query AST, or all boolean nodes, or some similarly 
> blunt instrument). Which makes me worry about cache churn.
>
> Actually I have two questions:
>
> 1. Can I monitor the *filter* cache size and eviction rate somehow? (REST 
> for preference, but jmx would be fine too.) I only seem to see 
> documentation for the field data cache.
>
> 2. Any advice for caching/not caching th

Re: How to get mappings using the Java API with ElasticSearch 0.90.10 and 1.0.0 RC1

[Luca Cavanna]

Hi Thierry,
the get field mapping 
apiwas
 recently added to retrieve mappings for specific fields. If you need to 
read the whole mapping I would still use the get mapping api, while I would 
use the get field mapping if you only need to get the mapping for a few 
fields. 

On Tuesday, January 28, 2014 9:15:57 AM UTC+1, Thierry Templier wrote:
>
> Hello,
>
> I wonder what is the best way to get the mappings for an index using the 
> ElasticSearch Java API with
> version 0.90.10. I saw that there is only a GetFieldMappingsRequest class 
> for this version whereas
> a class GetMappingsRequest is now present in 1.0RC1.
>
> Thanks very much for your answer and your help!
> Thierry 
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/2a209b00-76ac-4db3-92b8-ba3df0443753%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Completion - updated inputs are not reflected

[barak]

The very odd thing is that I just tried running these use cases with the 
hotels example and it works perfect! So apparently there is no bug on ES 
side, so I guess I did things wrong. With that in mind, are there some logs 
I can activate to find out why those docs returned although not matching 
the inputs?Also tried refresh, optimize, etc. I guess deleting all data and 
starting all over again will solve it, I just want to find out what I did 
wrong so I'll try to avoid it in the future... :-)

Thanks.

On Tuesday, January 28, 2014 10:43:54 AM UTC+2, Alexander Reelsen wrote:
>
> Hey,
>
> can you post a gist with a full reproduction? See 
> http://www.elasticsearch.org/help
>
> I did some tests and the only thing I could come up with, is that if you 
> delete a document, but do not run optimize, you will get old data back. If 
> you index the new document, but you do not refresh after indexing it, you 
> will get back the old result until the next automatic refresh happened.
>
>
> --Alex
>
>
> --Alex
>
>
> On Mon, Jan 27, 2014 at 8:44 PM, barak  >wrote:
>
>> So not a clue? I've opened an issue for that on github, but I guess it 
>> might takes some time... Any workaround I can use?
>>
>>
>> On Monday, January 27, 2014 4:13:58 PM UTC+2, barak wrote:
>>>
>>> Did that, still returns the incorrect results. Just to be sure, I even 
>>> changed the output field, and again - although seen when I examine the doc 
>>> (head plugin), the api ( REST and JAVA concreteOption.getText().toString() 
>>> ) returns the old name. Is it possible to delete the data structure 
>>> directly from the fs?
>>>
>>> On Monday, January 27, 2014 3:58:08 PM UTC+2, Alexander Reelsen wrote:

 Hey,

 can you try running an optimize with expunge deletes set?

 http://www.elasticsearch.org/guide/en/elasticsearch/
 reference/current/indices-optimize.html

 curl -XPOST 'http://localhost:9200/twitter/_optimize?only_
 expunge_deletes=true'


 --Alex


 On Mon, Jan 27, 2014 at 2:22 PM, barak  wrote:

> Hi,
>
> I'm using the completion feature on 0.90.9. I've a document which I 
> index with multiple inputs and suggest based search works ok. Then I 
> deleted that doc and re-create it with different inputs, and it still 
> being 
> returned with the first search I did, although I examined the doc (head 
> plugin) and the input field is indeed updated, which does not match that 
> search. Any flush need to issued after updates? Does the suggest API 
> explains the results?
>
> Thanks.
>
> -- 
> You received this message because you are subscribed to the Google 
> Groups "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send 
> an email to elasticsearc...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/elasticsearch/741ebbf0-2a9d-464c-8946-ca1d13da5ff3%
> 40googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>

  -- 
>> You received this message because you are subscribed to the Google Groups 
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to elasticsearc...@googlegroups.com .
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/elasticsearch/13ab2569-038a-4b68-a252-d8ade2c9ca2d%40googlegroups.com
>> .
>>
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/e107281e-809e-4b9b-bbc2-7c211991d514%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Any possibility of permalinks to documentation for each version?

[Tim S]

"It doesn't seem like they support minor versions." - this is exactly what 
I'm asking "them" to do...

It doesn't have to be linked to from the main page, but a stable link that 
I know will still be there a year or two in the future will be very useful.

On Monday, January 27, 2014 7:54:47 PM UTC, Ivan Brusic wrote:
>
> Here are the current branches: 
> http://www.elasticsearch.org/guide/en/elasticsearch/reference/index.html
>
>
> On Mon, Jan 27, 2014 at 11:53 AM, Ivan Brusic 
> > wrote:
>
>> The documentation is now versioned. For example (random page)
>>
>> 0.90:  
>> http://www.elasticsearch.org/guide/en/elasticsearch/reference/0.90/search-search.html
>> master: 
>> http://www.elasticsearch.org/guide/en/elasticsearch/reference/master/search-search.html
>> current: 
>> http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-search.html
>>
>> I am guessing a 1.0 version will be built once it is released. It doesn't 
>> seem like they support minor versions. Not sure what "current" signifies, 
>> but you can probably figure it out by reading the documentation building 
>> script source: https://github.com/elasticsearch/docs
>>  
>> -- 
>> Ivan
>>
>>
>> On Mon, Jan 27, 2014 at 1:27 AM, Tim S > >wrote:
>>
>>> I notice the docs for version 1.0.0 are in a "1.x" branch. When 1.1 is 
>>> released, I'm assuming the 1.x branch docs will be updated to 1.1?
>>>
>>> Is there any possibility of creating permanent links to each version? 
>>> (like lucene do, e.g http://lucene.apache.org/core/3_6_1/). The reason 
>>> is simply that if I build a solution for a customer that requires a 
>>> specific version of elasticsearch, then my documentation will probably end 
>>> up referencing various bits of the elasticsearch documentation. Which is 
>>> tricky if the documentation I'm referencing is changing.
>>>
>>> So, I was wondering if it would be possible to create links for each 
>>> release version, so instead of referencing 
>>> http://www.elasticsearch.org/guide/en/elasticsearch/reference/1.x/search.htmlI
>>>  can reference 
>>> http://www.elasticsearch.org/guide/en/elasticsearch/reference/1.0.0.Release/search.htmlor
>>>  similar, and then I can be happy that the documentation I'm linking to 
>>> is the exact documentation for the version that my customer uses.
>>>
>>> Thanks,
>>>
>>> Tim.
>>>
>>> -- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "elasticsearch" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to elasticsearc...@googlegroups.com .
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/elasticsearch/dd467b38-da9b-4f4e-9ebe-f39adb403f2f%40googlegroups.com
>>> .
>>> For more options, visit https://groups.google.com/groups/opt_out.
>>>
>>
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/88e08dda-b9c5-4bdd-b6e3-73bba41ca503%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Bulk indexing via ES HTTP

[joergpra...@gmail.com]

You should consider if it is possible to install the latest ES (current
1.0.0.RC1) and the latest JVM.

If you use 4 nodes, you should consider 4 shards by default, for balanced
resources on every index.

You did not set anything special for bulk indexing thread pool if you mean
that, the settings are in threadpool.bulk, not threadpool.index (I don't
know if your Logstash is using bulk or index)

indices.memory.index_buffer_size is adjusted automatically, no need to cap
it to 50%.

Also index.translog.flush_threshold_ops, I wonder why you adjust that value.

By moving the search pool away from the number of CPU cores, you reduce the
automatic scale of search in your cluster which is bad. Using 20 instead of
18 (3*6 is default) makes not much difference per se. But reducing the
queue size from 1000 to 100 will make your search load bail out early and
often.

Your heap size is very large (30g) and you should be prepared that you have
to take additional efforts to tackle GC challenges.

You should also think about dedicated master nodes if you want to drive
large heaps with expected high GC on data nodes.

The indexing load is automatically distributed, no need to care for that in
Logstash. But you should consider to set up Logstash so that it can index
to more than one node, just for more resiliency.

Jörg



On Tue, Jan 28, 2014 at 9:51 AM, Luca Belluccini
wrote:

> Hello,
> I am putting in place an ES cluster with 4 nodes (6 Cores + 48GB RAM).
> The aim is to use Kibana as a data analysis tool.
> I set up Logstash to properly feed ES and use the following:
>
>- https://gist.github.com/lucabelluccini/7563998 for index templates
>- Some tweaks to elasticsearch.yml:
>   - indices.memory.index_buffer_size: 50%
>   - index.translog.flush_threshold_ops: 5
>   - index.number_of_shards: 3
>   - threadpool.search.type: fixed
>   - threadpool.search.size: 20
>   - threadpool.search.queue_size: 100
>   - threadpool.index.type: fixed
>   - threadpool.index.size: 60
>   - threadpool.index.queue_size: 200
>   - node.master: true
>   - node.data: true
>   - ES_HEAP_SIZE=30g
>
> Logstash is sending to one of the hosts and I wanted to ask if the
> indexing is automatically distributed over all the nodes or you have to set
> up something to exploit all the processing power of all the 4 nodes.
>
> Thanks in advance,
> Luca B.
>
>  --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/15f9d547-0d78-48bb-bb33-c18d88e78687%40googlegroups.com
> .
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoE8tvmcty8HMxyJogOWW-L5wL%3D3sQjtuR-A3r8o1r%2BwCg%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Bulk indexing via ES HTTP

[Alexander Reelsen]

Hey,

might make sense to use only half of your memory to the ES process, the
rest is used by the filesystem cache to speed up index operations. You may
want to enable bootstrap.mlockall and ensure it is working. I guess you are
using indices per day/week so that it does not matter too much that you are
using less shards than you have nodes per index. Three shards per index
means that those three primary shards are distributed across your cluster,
which implies that your indexing is also distributed.

Wondering about your threadpool changes and the translog stuff. Any
particular reason? Did you run into something while testing?


--Alex



On Tue, Jan 28, 2014 at 9:51 AM, Luca Belluccini
wrote:

> Hello,
> I am putting in place an ES cluster with 4 nodes (6 Cores + 48GB RAM).
> The aim is to use Kibana as a data analysis tool.
> I set up Logstash to properly feed ES and use the following:
>
>- https://gist.github.com/lucabelluccini/7563998 for index templates
>- Some tweaks to elasticsearch.yml:
>   - indices.memory.index_buffer_size: 50%
>   - index.translog.flush_threshold_ops: 5
>   - index.number_of_shards: 3
>   - threadpool.search.type: fixed
>   - threadpool.search.size: 20
>   - threadpool.search.queue_size: 100
>   - threadpool.index.type: fixed
>   - threadpool.index.size: 60
>   - threadpool.index.queue_size: 200
>   - node.master: true
>   - node.data: true
>   - ES_HEAP_SIZE=30g
>
> Logstash is sending to one of the hosts and I wanted to ask if the
> indexing is automatically distributed over all the nodes or you have to set
> up something to exploit all the processing power of all the 4 nodes.
>
> Thanks in advance,
> Luca B.
>
>  --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/15f9d547-0d78-48bb-bb33-c18d88e78687%40googlegroups.com
> .
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAGCwEM_u9W_Xh4kuX3Z3oExydm%2BH%2B%3Db4%3DtYEi4CN1BkRJ7iBrQ%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Document size and score during search

[Alexander Reelsen]

Hey,

iirc you cannot set omit_norms in the attachment type itself (as that type
consist of several sub fields). You need to set it for the "file", like
this (untested, on the top of my head)

XContentBuilder map = XContentFactory.jsonBuilder().startObject()
.startObject(idxType)
 .startObject("properties")
 .startObject("file")
 .field("type", "attachment")
 .startObject("fields")
 .startObject("file")
.field("omit_norms",true)
 .endObject()

Also, note that it omit_norms and not omit_norm. In addition I dont know,
what the refid, itexp and totalexp fields are used for inside of the
attachment, see the documentation for supported fields
https://github.com/elasticsearch/elasticsearch-mapper-attachments
Maybe it is something internal I am not aware of.


--Alex


On Mon, Jan 27, 2014 at 12:02 PM, Hiro Gangwani wrote:

> Dear Sir,
>
> We are indexing the document content and executing text based search from
> document content. Is there way to disable lucene's preference to score
> shorter documents higher?
>
> Consider following example
>
> 1. we have 2 documents A and B
> 2. We execute search based upon text Java
> 3. A document has 10 matching words while document B has 7 words. But size
> of document B is smaller than document A.
> 4. During search operation document B is assigned higher score because its
> size is smaller than document A( Though A has higher count of words Java)
>
> In one of forums we learnt that using omit_norms we can ignore the size of
> document while indexing. We tried this approach and still not getting the
> desired results.
>
> we are using Java API to create the indexes using XContentBuilder. please
> find the code below
>
>
> -- Creating index
> XContentBuilder map = XContentFactory.jsonBuilder().startObject()
> .startObject(idxType)
>  .startObject("properties")
>  .startObject("file")
>  .field("type", "attachment")
>  .*field("omit_norm","true")*
>  .startObject("fields")
>  .startObject("refid")
>  .field("store", "yes")
>  .endObject()
>  .startObject("name")
>  .field("store", "yes")
>  .endObject()
>  .startObject("itexp")
>  .field("store", "yes")
>  .endObject()
>  .startObject("totalexp")
>  .field("store", "yes")
>  .endObject()
>  .endObject()
>  .endObject()
> .endObject()
> .endObject();
>  CreateIndexResponse lResponse =
> client.admin().indices().prepareCreate(idxName)
> .addMapping("attachment", map).execute().actionGet();
>
> --- Indexing document
>
> XContentBuilder source = XContentFactory.jsonBuilder().startObject()
> .field("file", data64)
> .field("refid", "2")
> .field("name", "Maya")
> .field("totalexp",11.0);
>
> Please let me know if above code is correct.We are not getting desired
> results even after applying the omit_norms parameter.
>
> Thanks in advance
>
> Hiro Gangwani
>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/b707663b-7b7b-48cd-ab96-4490a0171ec1%40googlegroups.com
> .
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAGCwEM9O96SW1Xxac-dhYaZcD9dW%3DD8GCT2rSqVFANkr236%3DYg%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: strange sort behaviour, values replaced

[DH]

Sorry, I couldn't find a way to edit my first post.
I made a typo. Of course in the query, the sort is : 
 
 
> "sort" : [{
>"fieldToSortOn1" : {
> "asc"
>}
>   }, {
>"fieldToSortOn2" : {
> "asc"
>}
>   }
>  ]
>
 
So that is not the problem. 

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/2f36d2c8-b4cc-49b9-ba9a-e36272743f0d%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Bulk indexing via ES HTTP

[Luca Belluccini]

Hello,
I am putting in place an ES cluster with 4 nodes (6 Cores + 48GB RAM).
The aim is to use Kibana as a data analysis tool.
I set up Logstash to properly feed ES and use the following:

   - https://gist.github.com/lucabelluccini/7563998 for index templates
   - Some tweaks to elasticsearch.yml:
  - indices.memory.index_buffer_size: 50%
  - index.translog.flush_threshold_ops: 5
  - index.number_of_shards: 3
  - threadpool.search.type: fixed
  - threadpool.search.size: 20
  - threadpool.search.queue_size: 100
  - threadpool.index.type: fixed
  - threadpool.index.size: 60
  - threadpool.index.queue_size: 200
  - node.master: true
  - node.data: true
  - ES_HEAP_SIZE=30g
  
Logstash is sending to one of the hosts and I wanted to ask if the indexing 
is automatically distributed over all the nodes or you have to set up 
something to exploit all the processing power of all the 4 nodes.

Thanks in advance,
Luca B.

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/15f9d547-0d78-48bb-bb33-c18d88e78687%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: order of the elements does matter?

[Nikolay Chankov]

Hi David,

Here is full gist:

curl -XDELETE 'http://localhost:9200/test_search'
curl -XPUT 'http://localhost:9200/test_search/' -d '
{
"mappings" : {
"record" : {
"properties" : {
"object" : { 
"type" : "string",
"index" : "not_analyzed"
},
"name" : { 
"type" : "string"
}
}
}
}
}
'
curl -XPUT 'http://localhost:9200/test_search/record/1' -d '{
"object" : "User",
"name" : "John Doe"
}'
curl -XPUT 'http://localhost:9200/test_search/record/2' -d '{
"object" : "User",
"name" : "Jane Doe"
}'
curl -XPUT 'http://localhost:9200/test_search/record/3' -d '{
"object" : "User",
"name" : "Joseph Doe"
}'
curl -XPUT 'http://localhost:9200/test_search/record/4' -d '{
"object" : "User",
"name" : "Anna Doe"
}'
curl -XPUT 'http://localhost:9200/test_search/record/5' -d '{
"object" : "Venue",
"name" : "Bar Luna"
}'

curl -XGET 'http://localhost:9200/test_search/_search?pretty=true' -d '{
"query": {
"match_all": {},
"filtered": {
"filter": {
"term": {
"object": "User"
}
}
}
},
"size" : 2
}'

I've noticed that the problem exist only if under the top "query" node 
there are 2 elements. If I remove "match_all" or "filtered" section the 
size does take effect.
I've combined the examples in "And Filter" + "Term Filter" to create the 
query, but probably this is the wrong way?

Thanks

On Monday, January 27, 2014 7:42:15 PM UTC, David Pilato wrote:
>
> Yes please. If you can gist a full curl recreation, that will help a lot!
>
> --
> David ;-)
> Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs
>
>
> Le 27 janv. 2014 à 19:36, Nikolay Chankov > 
> a écrit :
>
> I've noticed, that the "problem" came when in the request there is a 
> "filtered" node. Here is the full request:
>
> curl-XGET 'http://localhost/search/_search' -d'{
>   "query": {
> "match_all": {},
> "filtered": {
>   "filter": {
> "term": {
>   "object": "User"
> }
>   }
> }
>   },
>   "size": 3,
>   "sort": [
> {
>   "name.untouched": "asc"
> }
>   ]
> }'
>
> So, if it's called this way the sort and size are ignored, while if they 
> are placed above the query, they take effect, and I can see 3 records.
> if it's not correct, I would expect to get an error, rather than ignoring 
> the params...
>
> name is a multi_field with name.untouched is index not analyzed, object is 
> string, not analyzed. If it's still required I will try to create a full 
> gist tomorrow.
>
>
> On Monday, January 27, 2014 5:54:48 PM UTC, David Pilato wrote:
>>
>>  Can you reproduce it with a full curl recreation and gist it?
>> In which version?
>>
>> If confirmed, could you open an issue?
>>
>> -- 
>> *David Pilato* | *Technical Advocate* | *Elasticsearch.com 
>> *
>> @dadoonet  | 
>> @elasticsearchfr
>>
>>
>> Le 27 janvier 2014 at 18:50:32, Nikolay Chankov (ncha...@gmail.com) a 
>> écrit:
>>
>> Hi guys, 
>>
>> today I've noticed that order of the elements in the request does matter 
>> for example:
>>
>>  curl -XGET 'http://localhost:9200/search/_search'-d '
>> {
>>"sort" : {...},
>>"size" : 100,
>>"query" : {...}
>> }'
>>  
>> is working, while
>>
>>  curl -XGET 'http://localhost:9200/search/_search'-d '
>> {
>>"query" : {...},
>>"sort" : {...},
>>"size" : 100
>> }'
>>  
>> Doesn't take effect of size as well as on sort. 
>>
>> I think the order shouldn't matter, and ES should reorder the elements 
>> internally. Am I get it wrong, or there is special reason for this?
>>
>> Thanks in advance. 
>>
>>
>>  --
>> You received this message because you are subscribed to the Google Groups 
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to elasticsearc...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/elasticsearch/c0d7791a-9c8a-40e9-855d-b6a88f2f2c87%40googlegroups.com
>> .
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>>  -- 
> You received this message because you are subscribed to the Google Groups 
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to elasticsearc...@googlegroups.com .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/elasticsearch/efb884a7-2e0c-4194-82b3-c4b91f5f7751%40googlegroups.com
> .
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+u

Re: Completion - updated inputs are not reflected

[Alexander Reelsen]

Hey,

can you post a gist with a full reproduction? See
http://www.elasticsearch.org/help

I did some tests and the only thing I could come up with, is that if you
delete a document, but do not run optimize, you will get old data back. If
you index the new document, but you do not refresh after indexing it, you
will get back the old result until the next automatic refresh happened.


--Alex


--Alex


On Mon, Jan 27, 2014 at 8:44 PM, barak  wrote:

> So not a clue? I've opened an issue for that on github, but I guess it
> might takes some time... Any workaround I can use?
>
>
> On Monday, January 27, 2014 4:13:58 PM UTC+2, barak wrote:
>>
>> Did that, still returns the incorrect results. Just to be sure, I even
>> changed the output field, and again - although seen when I examine the doc
>> (head plugin), the api ( REST and JAVA concreteOption.getText().toString()
>> ) returns the old name. Is it possible to delete the data structure
>> directly from the fs?
>>
>> On Monday, January 27, 2014 3:58:08 PM UTC+2, Alexander Reelsen wrote:
>>>
>>> Hey,
>>>
>>> can you try running an optimize with expunge deletes set?
>>>
>>> http://www.elasticsearch.org/guide/en/elasticsearch/
>>> reference/current/indices-optimize.html
>>>
>>> curl -XPOST 'http://localhost:9200/twitter/_optimize?only_
>>> expunge_deletes=true'
>>>
>>>
>>> --Alex
>>>
>>>
>>> On Mon, Jan 27, 2014 at 2:22 PM, barak  wrote:
>>>
 Hi,

 I'm using the completion feature on 0.90.9. I've a document which I
 index with multiple inputs and suggest based search works ok. Then I
 deleted that doc and re-create it with different inputs, and it still being
 returned with the first search I did, although I examined the doc (head
 plugin) and the input field is indeed updated, which does not match that
 search. Any flush need to issued after updates? Does the suggest API
 explains the results?

 Thanks.

 --
 You received this message because you are subscribed to the Google
 Groups "elasticsearch" group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to elasticsearc...@googlegroups.com.
 To view this discussion on the web visit https://groups.google.com/d/
 msgid/elasticsearch/741ebbf0-2a9d-464c-8946-ca1d13da5ff3%
 40googlegroups.com.
 For more options, visit https://groups.google.com/groups/opt_out.

>>>
>>>  --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/13ab2569-038a-4b68-a252-d8ade2c9ca2d%40googlegroups.com
> .
>
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAGCwEM9KRgWbbvSWp54obYEz%3DgqAcKYzAAqbRu8DUui3FZ58Fw%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

strange sort behaviour, values replaced

[DH]

Hello everyone.
 
Yesterday, I noticed a strange behaviour in the sorting of the data, in one 
of my clusters. 
I have a request like this : 
 
 
> {
>  "size" : 25,
>  "query" : {
>   "filtered" : {
>"query" : {
> "match_all" : {}
>},
>"filter" : {
> "and" : {
>  "filters" : [{
>"and" : {
> "filters" : [{
>   "range" : {
>"myDateField" : {
> "from" : "2013-09-26",
> "to" : "2014-01-27",
> "include_lower" : true,
> "include_upper" : true
>}
>   }
>  }
> ]
>}
>   }
>  ]
> }
>}
>   }
>  },
>  "fields" : ["fieldToSortOn1",
>   "fieldToSortOn2",
>  ],
>  "sort" : [{
>"fieldToSortOn1" : {
> "asc"
>}
>   }, {
>"fieldToSortOn1" : {
> "asc"
>}
>   }
>  ]
> }
>
 
So, as you see, I ask for two fields, and I request the results to be 
sorted by those fields.
 
The problem is that I get results like these : 
 
>  
> {
> _index: myIndex
> _type: myType
> _id: 5454245454146
> _score: null
> fields: {
> fieldToSortOn1: 13207580
> fieldToSortOn2: 1
> }
> sort: [
> 13207580
> 9223372036854776000
> ]
> }
>
 
The "fields" part is ok, I indeed get the values present in my index. But 
es didn't use the value of the second field to sort, it used that big 
number instead, and so, the result is threw at the end and the sort is all 
wrong. It doesn't happen to all the documents, but it happens enougn to be 
a serious issue (I'd say a sixth of the docs seems to be affected). 
By luck, this is happening on the development environmentI have the exact 
same index on my production environment, and there, I do not have any 
problems.
 
Prior to this, we had a problem, on this cluster. A human error caused that 
cluster to lose a bunch of shards. We deleted the indices with missing 
shards and kept those who were complete. That indice was, of course, kept, 
as it was still complete. in the end, we got back a green state and could 
issue queries.
 
Can that strange sort behavior be linked to that problem, as I suspect?  If 
so, thats a real problem. Indeed, the cluster says that everything is fine 
(green state) but the sort is broken. If that were to happen in production, 
we would be in a world of pain and misery. 
Did that happens to anyone else? 
Any help would be welcome.
Thanks,
Deny
 

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/55b3ed1e-d451-4f91-b91c-a3c12bc9%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

How to get mappings using the Java API with ElasticSearch 0.90.10 and 1.0.0 RC1

[Thierry Templier]

Hello,

I wonder what is the best way to get the mappings for an index using the 
ElasticSearch Java API with
version 0.90.10. I saw that there is only a GetFieldMappingsRequest class 
for this version whereas
a class GetMappingsRequest is now present in 1.0RC1.

Thanks very much for your answer and your help!
Thierry 

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/3d0fb56f-52c6-4b02-96b5-d0728612101d%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.