Re: Unable to delete child documents by query that uses "has parent" (ES 1.4.1)

[Ron Sher]

Posted there on August 29 -
https://discuss.elastic.co/t/unable-to-delete-child-documents-by-query-that-uses-has-parent-es-1-4-1/28276

Didn't get any reply :-(

On Thu, Aug 27, 2015 at 4:25 PM, Nikolas Everett  wrote:

> You should try posting this on https://discuss.elastic.co/ . This email
> list has been deprecated in favor of using that. There are settings that
> make it function almost the same way the mailing list functioned.
>
> On Thu, Aug 27, 2015 at 3:39 AM, Ron Sher  wrote:
>
>> Hi,
>>
>> I'm using the query below in my search to find all users without parents.
>> The search works as expected, but delete by query fails with has_parent]
>> query and filter unsupported in delete_by_query api
>>
>> I've searched and it seems that it should be supported.
>>
>> Anything that I'm missing?
>>
>> {
>>   "query": {
>> "bool": {
>>   "must": [
>> {
>>   "term": {
>> "user.service_id": "12345"
>>   }
>> },
>> {
>>   "filtered": {
>> "filter": {
>>   "not": {
>> "filter": {
>>   "has_parent": {
>> "query": {
>>   "match_all": {}
>> },
>> "parent_type": "account"
>>   }
>> }
>>   }
>> }
>>   }
>> }
>>   ]
>> }
>>   }
>> }
>>
>> --
>> Please update your bookmarks! We have moved to
>> https://discuss.elastic.co/
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to elasticsearch+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/elasticsearch/d57b6443-1221-4109-a654-1a36c3b05d70%40googlegroups.com
>> <https://groups.google.com/d/msgid/elasticsearch/d57b6443-1221-4109-a654-1a36c3b05d70%40googlegroups.com?utm_medium=email&utm_source=footer>
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
> --
> Please update your bookmarks! We have moved to https://discuss.elastic.co/
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "elasticsearch" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/elasticsearch/f3FhMBZ1U1Q/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/CAPmjWd19UzYzgsXYhccTcZiDKje5aRRwq6vDpiB_DXSGe71zNQ%40mail.gmail.com
> <https://groups.google.com/d/msgid/elasticsearch/CAPmjWd19UzYzgsXYhccTcZiDKje5aRRwq6vDpiB_DXSGe71zNQ%40mail.gmail.com?utm_medium=email&utm_source=footer>
> .
>
> For more options, visit https://groups.google.com/d/optout.
>

-- 
Please update your bookmarks! We have moved to https://discuss.elastic.co/
--- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAKHuyJqNnbus7orqLaJNZ5eQb7aeeppX95OodFGdsZeN-ofTdw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Unable to delete child documents by query that uses "has parent" (ES 1.4.1)

[Ron Sher]

Hi,

I'm using the query below in my search to find all users without parents.
The search works as expected, but delete by query fails with has_parent] 
query and filter unsupported in delete_by_query api

I've searched and it seems that it should be supported.

Anything that I'm missing?

{
  "query": {
"bool": {
  "must": [
{
  "term": {
"user.service_id": "12345"
  }
},
{
  "filtered": {
"filter": {
  "not": {
"filter": {
  "has_parent": {
"query": {
  "match_all": {}
},
"parent_type": "account"
  }
}
  }
}
  }
}
  ]
}
  }
}

-- 
Please update your bookmarks! We have moved to https://discuss.elastic.co/
--- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/d57b6443-1221-4109-a654-1a36c3b05d70%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: How do you do rolling upgrade with Chef?

[Ron Sher]

Nope. Didn't find anything
בתאריך 8 במאי 2015 12:06 לפנה״צ,‏ "Richard Eyre"  כתב:

> Hi Ron,
>
> Did you ever make any progress on rolling upgrades?
>
> Thanks,
> Richard
>
> On Sunday, July 27, 2014 at 11:04:31 PM UTC-6, Ron Sher wrote:
>>
>> Hi,
>>
>> I know there's a guide on installing elasticsearch with Chef
>> <http://www.elasticsearch.org/tutorials/deploying-elasticsearch-with-chef-solo/>
>> <http://www.elasticsearch.org/tutorials/deploying-elasticsearch-with-chef-solo/>
>> <http://www.elasticsearch.org/tutorials/deploying-elasticsearch-with-chef-solo/>
>> here
>> <http://www.elasticsearch.org/tutorials/deploying-elasticsearch-with-chef-solo/>
>>  but it looks like it's suitable for first installation or a complete
>> cluster shutdown.
>> I'm looking for an automated rolling upgrade of elasticsearch.
>> Any suggestion?
>>
>> Thanks,
>> Ron
>>
>  --
> Please update your bookmarks! We moved to https://discuss.elastic.co/
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "elasticsearch" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/elasticsearch/fiJ8shkhwvg/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/2b39515c-e26e-4e32-9f72-3ef1e2b4c287%40googlegroups.com
> <https://groups.google.com/d/msgid/elasticsearch/2b39515c-e26e-4e32-9f72-3ef1e2b4c287%40googlegroups.com?utm_medium=email&utm_source=footer>
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
Please update your bookmarks! We moved to https://discuss.elastic.co/
--- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAKHuyJq%3DzWqDsdhwW%3D71WvqkiPOR6PJ1oNdOb%2BsTkTGq%3Dq_ybA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

finding child documents without parents

[Ron Sher]

We have child documents and we suspect that some of them don't have their 
parents. Can someone suggest a query I can use to find child documents without 
parents? 
Thanks,
Ron

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/76a913db-8ed9-48e9-bd0f-29495c1b6e0f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Looking for a best practice to get all data according to some filters

[Ron Sher]

Again, why not use a very large count size? What are the implications of 
using a very large count?
Regarding performance - it seems doing 1 request with a very large count 
performs better than using scan scroll (with count of 100 using 32 shards)

On Wednesday, December 10, 2014 10:53:50 PM UTC+2, David Pilato wrote:
>
> No I did not say that. Or I did not mean that. Sorry if it was unclear.
> I said: don’t use large sizes:
>
> Never use size:1000 or from:1000. 
>>
>
> You should read this: 
> http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-scroll.html#scroll-scan
>
> -- 
> *David Pilato* | *Technical Advocate* | *Elasticsearch.com 
> <http://Elasticsearch.com>*
> @dadoonet <https://twitter.com/dadoonet> | @elasticsearchfr 
> <https://twitter.com/elasticsearchfr> | @scrutmydocs 
> <https://twitter.com/scrutmydocs>
>
>
>  
> Le 10 déc. 2014 à 21:16, Ron Sher > a 
> écrit :
>
> So you're saying there's no impact on elasticsearch if I issue a large 
> size? 
> If that's the case then why shouldn't I just call size of 1M if I want to 
> make sure I get everything?
>
> On Wednesday, December 10, 2014 8:22:47 PM UTC+2, David Pilato wrote:
>>
>> Scan/scroll is the best option to extract a huge amount of data.
>> Never use size:1000 or from:1000. 
>>
>> It's not realtime because you basically scroll over a given set of 
>> segments and all new changes that will come in new segments won't be taken 
>> into account during the scroll.
>> Which is good because you won't get inconsistent results.
>>
>> About size, I'd would try and test. It depends on your docs size I 
>> believe.
>> Try with 1 and see how it goes when you increase it. You will may be 
>> discover that getting 10*1 docs is the same as 1*10. :)
>>
>> Best
>>
>> David
>>
>> Le 10 déc. 2014 à 19:09, Ron Sher  a écrit :
>>
>> Hi,
>>
>> I was wondering about best practices to to get all data according to some 
>> filters.
>> The options as I see them are:
>>
>>- Use a very big size that will return all accounts, i.e. use some 
>>value like 1m to make sure I get everything back (even if I need just a 
>> few 
>>hundreds or tens of documents). This is the quickest way, development 
>> wise.
>>- Use paging - using size and from. This requires looping over the 
>>result and the performance gets worse as we advance to later pages. Also, 
>>we need to use preference if we want to get consistent results over the 
>>pages. Also, it's not clear what's the recommended size for each page.
>>- Use scan/scroll - this gives consistent paging but also has several 
>>drawbacks: If I use search_type=scan then it can't be sorted; using 
>>scan/scroll is (maybe) less performant than paging (the documentation 
>> says 
>>it's not for realtime use); again not clear which size is recommended.
>>
>> So you see - many options and not clear which path to take.
>>
>> What do you think?
>>
>> Thanks,
>> Ron
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to elasticsearc...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/elasticsearch/764a37c5-1fec-48c4-9c66-7835d8141713%40googlegroups.com
>>  
>> <https://groups.google.com/d/msgid/elasticsearch/764a37c5-1fec-48c4-9c66-7835d8141713%40googlegroups.com?utm_medium=email&utm_source=footer>
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>>
> -- 
> You received this message because you are subscribed to the Google Groups 
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to elasticsearc...@googlegroups.com .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/elasticsearch/838020dc-d2ea-423d-9606-778d807b1a0d%40googlegroups.com
>  
> <https://groups.google.com/d/msgid/elasticsearch/838020dc-d2ea-423d-9606-778d807b1a0d%40googlegroups.com?utm_medium=email&utm_source=footer>
> .
> For more options, visit https://groups.google.com/d/optout.
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/ac0841ac-4150-435c-a3da-afbf2a4b06a6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Looking for a best practice to get all data according to some filters

[Ron Sher]

Just tested this.
When I used a large number to get all of my documents according to some 
criteria (4926 in the result) I got:
13.951s when using a size of 1M
43.6s when using scan/scroll (with a size of 100)

Looks like I should be using the not recommended paging.
Can I make the scroll better?

Thanks,
Ron

On Wednesday, December 10, 2014 10:53:50 PM UTC+2, David Pilato wrote:
>
> No I did not say that. Or I did not mean that. Sorry if it was unclear.
> I said: don’t use large sizes:
>
> Never use size:1000 or from:1000. 
>>
>
> You should read this: 
> http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-scroll.html#scroll-scan
>
> -- 
> *David Pilato* | *Technical Advocate* | *Elasticsearch.com 
> <http://Elasticsearch.com>*
> @dadoonet <https://twitter.com/dadoonet> | @elasticsearchfr 
> <https://twitter.com/elasticsearchfr> | @scrutmydocs 
> <https://twitter.com/scrutmydocs>
>
>
>  
> Le 10 déc. 2014 à 21:16, Ron Sher > a 
> écrit :
>
> So you're saying there's no impact on elasticsearch if I issue a large 
> size? 
> If that's the case then why shouldn't I just call size of 1M if I want to 
> make sure I get everything?
>
> On Wednesday, December 10, 2014 8:22:47 PM UTC+2, David Pilato wrote:
>>
>> Scan/scroll is the best option to extract a huge amount of data.
>> Never use size:1000 or from:1000. 
>>
>> It's not realtime because you basically scroll over a given set of 
>> segments and all new changes that will come in new segments won't be taken 
>> into account during the scroll.
>> Which is good because you won't get inconsistent results.
>>
>> About size, I'd would try and test. It depends on your docs size I 
>> believe.
>> Try with 1 and see how it goes when you increase it. You will may be 
>> discover that getting 10*1 docs is the same as 1*10. :)
>>
>> Best
>>
>> David
>>
>> Le 10 déc. 2014 à 19:09, Ron Sher  a écrit :
>>
>> Hi,
>>
>> I was wondering about best practices to to get all data according to some 
>> filters.
>> The options as I see them are:
>>
>>- Use a very big size that will return all accounts, i.e. use some 
>>value like 1m to make sure I get everything back (even if I need just a 
>> few 
>>hundreds or tens of documents). This is the quickest way, development 
>> wise.
>>- Use paging - using size and from. This requires looping over the 
>>result and the performance gets worse as we advance to later pages. Also, 
>>we need to use preference if we want to get consistent results over the 
>>pages. Also, it's not clear what's the recommended size for each page.
>>- Use scan/scroll - this gives consistent paging but also has several 
>>drawbacks: If I use search_type=scan then it can't be sorted; using 
>>scan/scroll is (maybe) less performant than paging (the documentation 
>> says 
>>it's not for realtime use); again not clear which size is recommended.
>>
>> So you see - many options and not clear which path to take.
>>
>> What do you think?
>>
>> Thanks,
>> Ron
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to elasticsearc...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/elasticsearch/764a37c5-1fec-48c4-9c66-7835d8141713%40googlegroups.com
>>  
>> <https://groups.google.com/d/msgid/elasticsearch/764a37c5-1fec-48c4-9c66-7835d8141713%40googlegroups.com?utm_medium=email&utm_source=footer>
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>>
> -- 
> You received this message because you are subscribed to the Google Groups 
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to elasticsearc...@googlegroups.com .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/elasticsearch/838020dc-d2ea-423d-9606-778d807b1a0d%40googlegroups.com
>  
> <https://groups.google.com/d/msgid/elasticsearch/838020dc-d2ea-423d-9606-778d807b1a0d%40googlegroups.com?utm_medium=email&utm_source=footer>
> .
> For more options, visit https://groups.google.com/d/optout.
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/d41729a8-8dfc-48eb-ae7b-1ac16cd05787%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Looking for a best practice to get all data according to some filters

[Ron Sher]

So you're saying there's no impact on elasticsearch if I issue a large 
size? 
If that's the case then why shouldn't I just call size of 1M if I want to 
make sure I get everything?

On Wednesday, December 10, 2014 8:22:47 PM UTC+2, David Pilato wrote:
>
> Scan/scroll is the best option to extract a huge amount of data.
> Never use size:1000 or from:1000. 
>
> It's not realtime because you basically scroll over a given set of 
> segments and all new changes that will come in new segments won't be taken 
> into account during the scroll.
> Which is good because you won't get inconsistent results.
>
> About size, I'd would try and test. It depends on your docs size I believe.
> Try with 1 and see how it goes when you increase it. You will may be 
> discover that getting 10*1 docs is the same as 1*10. :)
>
> Best
>
> David
>
> Le 10 déc. 2014 à 19:09, Ron Sher > a 
> écrit :
>
> Hi,
>
> I was wondering about best practices to to get all data according to some 
> filters.
> The options as I see them are:
>
>- Use a very big size that will return all accounts, i.e. use some 
>value like 1m to make sure I get everything back (even if I need just a 
> few 
>hundreds or tens of documents). This is the quickest way, development wise.
>- Use paging - using size and from. This requires looping over the 
>result and the performance gets worse as we advance to later pages. Also, 
>we need to use preference if we want to get consistent results over the 
>pages. Also, it's not clear what's the recommended size for each page.
>- Use scan/scroll - this gives consistent paging but also has several 
>drawbacks: If I use search_type=scan then it can't be sorted; using 
>scan/scroll is (maybe) less performant than paging (the documentation says 
>it's not for realtime use); again not clear which size is recommended.
>
> So you see - many options and not clear which path to take.
>
> What do you think?
>
> Thanks,
> Ron
>
> -- 
> You received this message because you are subscribed to the Google Groups 
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to elasticsearc...@googlegroups.com .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/elasticsearch/764a37c5-1fec-48c4-9c66-7835d8141713%40googlegroups.com
>  
> <https://groups.google.com/d/msgid/elasticsearch/764a37c5-1fec-48c4-9c66-7835d8141713%40googlegroups.com?utm_medium=email&utm_source=footer>
> .
> For more options, visit https://groups.google.com/d/optout.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/838020dc-d2ea-423d-9606-778d807b1a0d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Peformance issues with has_parent filters

[Ron Sher]

We had poor experience with has_parent queries and we ended up implementing 
it ourselves using 2 steps:

   1. Filter the parent documents to get a list of IDs.
   2. Filter the child documents and look only for IDs in the list of IDs 
   from 1


On Wednesday, December 10, 2014 7:48:55 PM UTC+2, Xiaolin Xie wrote:
>
> Hi David
>
> Both queries return only 3 records. If I change the size to 10, the time 
> that the two queries take does not change . The first query still takes 
> about 100 milliseconds, and the second one still takes about 2000 
> milliseconds. Thanks a lot!
>
> Xiaolin.
>
>
> On Tuesday, December 9, 2014 10:53:51 PM UTC-8, David Pilato wrote:
>>
>> What happen if you change size to 10?
>>
>> David
>>
>> Le 10 déc. 2014 à 03:53, Xiaolin Xie  a écrit :
>>
>> Hi Elastic Search developers
>>
>> I am new to ES. We had some performance issues with our Elastic Search 
>> system, and we would like to get some ideas/thoughts about this issue from 
>> your guys.
>>
>> Here is our use case: we have three types of documents in one index: 
>> “campaign_group”, “campaign”, and “ad”. “campaign_group” is the parent of 
>> “campaign”, and “campaign” is the parent of “ad”.  Each document type has 
>> about 10 simple properties, such as string, long, short. The three kinds of 
>> documents all have a property “user”(long) and a property 
>> “run_status”(short). Documents are hashed by “user”, documents with the 
>> same “user” are mapped into the same shard. 
>>
>> We have about 1.4 billion documents in total. We have 200 shards, 3 
>> master node, and 21 data nodes, and each shard has too replica.  The 
>> total data size is 1.5TB. We are running elasticsearch 1.21.
>>
>> Queries are made against specific shard by routing. The flowing query(1) 
>> checks the run_status of “ads”(run_status is a short type), and it takes 
>> about 100 milliseconds. The query(2) checks both the run_status of “ad”, 
>> and the run_status of its parent, and it takes about 2000 milliseconds.  It 
>> looks like there are some performance issues with the has_parent filter.
>>
>> Do your guys have any thoughts about this problem? Is it expected(because 
>> ES cannot support has_parent well)? Or something else cloud result this 
>> problem? Or we should upgrade our Elastic Search version?
>>
>> Please let me know if you need any other information about our uses cases.
>>  
>>
>> Any thoughts/ideas will be highly appreciated.
>>
>> Query(1) 
>>
>> {
>>
>>   "filter":{
>>
>> "and":[
>>
>>   {
>>
>> "term":{
>>
>>   "user":1436594776581528
>>
>> }
>>
>>   },
>>
>>   {
>>
>> "terms":{
>>
>>   "run_status":[
>>
>> 1
>>
>>   ]
>>
>> }
>>
>>   }
>>
>> ]
>>
>>   },
>>
>>   "sort":{
>>
>> "_uid":"desc"
>>
>>   },
>>
>>   "size":100,
>>
>>   "from":0
>>
>> }
>>
>>  
>>
>> ===Query(2)
>>
>> {
>>
>>   "filter":{
>>
>> "and":[
>>
>>   {
>>
>> "term":{
>>
>>   "user":1436594776581528
>>
>> }
>>
>>   },
>>
>>   {
>>
>> "terms":{
>>
>>   "run_status":[
>>
>> 1
>>
>>   ]
>>
>> }
>>
>>   },
>>
>>   {
>>
>>   "has_parent" : {
>>
>>   "parent_type": "campaign",
>>
>>   "filter" : {
>>
>>   "terms" : {
>>
>>   "run_status" : [1]
>>
>>   }
>>
>>   }
>>
>>   }
>>
>>   }
>>
>> ]
>>
>>   },
>>
>>   "sort":{
>>
>> "_uid":"desc"
>>
>>   },
>>
>>   "size":100,
>>
>>   "from":0
>>
>> }
>>
>>  
>>  
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to elasticsearc...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/elasticsearch/220b1d9a-da80-416c-8b8d-d7cc3efc8b5a%40googlegroups.com
>>  
>> 
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/c534701d-cd81-4d3c-8150-e3b797cf941a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Looking for a best practice to get all data according to some filters

[Ron Sher]

Hi,

I was wondering about best practices to to get all data according to some 
filters.
The options as I see them are:

   - Use a very big size that will return all accounts, i.e. use some value 
   like 1m to make sure I get everything back (even if I need just a few 
   hundreds or tens of documents). This is the quickest way, development wise.
   - Use paging - using size and from. This requires looping over the 
   result and the performance gets worse as we advance to later pages. Also, 
   we need to use preference if we want to get consistent results over the 
   pages. Also, it's not clear what's the recommended size for each page.
   - Use scan/scroll - this gives consistent paging but also has several 
   drawbacks: If I use search_type=scan then it can't be sorted; using 
   scan/scroll is (maybe) less performant than paging (the documentation says 
   it's not for realtime use); again not clear which size is recommended.

So you see - many options and not clear which path to take.

What do you think?

Thanks,
Ron

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/764a37c5-1fec-48c4-9c66-7835d8141713%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Looking for a suggestion to better organize our indices for performance

[Ron Sher]

BTW, we use c3.4xlarge and not as I said before

On Tuesday, December 9, 2014 5:52:55 PM UTC+2, Jilles van Gurp wrote:
>
> Indeed increase your shard count. Also, you may want to consider using a 
> routing parameter based on e.g. a tenant_id to ensure all queries related 
> to a tenant only hit shards that actually have data for that tenant. Those 
> two measures would reduce the size of each shard and the number of shards 
> involved for each tenant. To increase query capacity, you could consider 
> increasing the number of replicas as well this ways, you have more nodes 
> that can handle query traffic for the same data.
>
> Jilles
>
> On Tuesday, December 9, 2014 3:56:06 PM UTC+1, Mark Walkom wrote:
>>
>> Currently you have shards upwards of over 100GB, which is massive and 
>> probably causing you some issues. Ideally you should be aiming for a max 
>> shard size of 40-50GB, so increasing your shard count to 24 brings you 
>> under this level and also gives you room for growth on an index level.
>>
>> Having a higher shard count also spreads the query load, and reduces the 
>> amount of thrashing (ie data transfer) if/when a node goes down.
>>
>> On 9 December 2014 at 15:50, Ron Sher  wrote:
>>
>>> we have 24 data nodes, 3 master nodes and 3 client nodes.
>>> We use  m3.4xlarge for the data nodes 
>>>
>>> On Tue, Dec 9, 2014 at 4:37 PM, Mark Walkom  wrote:
>>>
>>>> How many servers are in this cluster?
>>>>
>>>> On 9 December 2014 at 14:36, Ron Sher  wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> We have a multi tenant SAAS application in which we keep data for all 
>>>>> accounts of our clients (300 of them which we call services).
>>>>> We keep data in monthly indices that grew to be about 700GB with 4.6 
>>>>> billion documents each month.
>>>>> Each day we index a new account per day for each service.
>>>>>
>>>>> Each index is built from 6 shards and we use 1 replica.
>>>>>
>>>>> We're starting to have second thoughts of the structure of our indices 
>>>>> and about proper use of routing.
>>>>> Few things we contemplate:
>>>>>
>>>>>- Use routing according to service so that we will probably 
>>>>>benefit from caching better.
>>>>>- Change the indices according to service + month so that we will 
>>>>>query much less data, but will add many indices (now instead of 12 
>>>>> indices 
>>>>>a year we will have 300x12 and growing when the number of clients 
>>>>> grow).
>>>>>
>>>>> Any thoughts/suggestions?
>>>>>
>>>>> Thanks,
>>>>> Ron
>>>>>
>>>>> -- 
>>>>> You received this message because you are subscribed to the Google 
>>>>> Groups "elasticsearch" group.
>>>>> To unsubscribe from this group and stop receiving emails from it, send 
>>>>> an email to elasticsearc...@googlegroups.com.
>>>>> To view this discussion on the web visit 
>>>>> https://groups.google.com/d/msgid/elasticsearch/096f4d7b-2702-46d7-96d6-f746f2389623%40googlegroups.com
>>>>>  
>>>>> <https://groups.google.com/d/msgid/elasticsearch/096f4d7b-2702-46d7-96d6-f746f2389623%40googlegroups.com?utm_medium=email&utm_source=footer>
>>>>> .
>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>
>>>>
>>>>  -- 
>>>> You received this message because you are subscribed to a topic in the 
>>>> Google Groups "elasticsearch" group.
>>>> To unsubscribe from this topic, visit 
>>>> https://groups.google.com/d/topic/elasticsearch/32uvCMR1kl4/unsubscribe
>>>> .
>>>> To unsubscribe from this group and all its topics, send an email to 
>>>> elasticsearc...@googlegroups.com.
>>>> To view this discussion on the web visit 
>>>> https://groups.google.com/d/msgid/elasticsearch/CAEYi1X_TaWiZvpEpZ10_bMHhuOwSxpmBv%2BBhTNjPUHnKTA-Bgg%40mail.gmail.com
>>>>  
>>>> <https://groups.google.com/d/msgid/elasticsearch/CAEYi1X_TaWiZvpEpZ10_bMHhuOwSxpmBv%2BBhTNjPUHnKTA-Bgg%40mail.gmail.com?utm_medium=email&utm_source=footer>
>>>> .
>>>>
>>>> For more options, visit https://groups.google.com/d/optout.
>>>>
>>>
>>>  --

Re: Looking for a suggestion to better organize our indices for performance

[Ron Sher]

we have 24 data nodes, 3 master nodes and 3 client nodes.
We use  m3.4xlarge for the data nodes

On Tue, Dec 9, 2014 at 4:37 PM, Mark Walkom  wrote:

> How many servers are in this cluster?
>
> On 9 December 2014 at 14:36, Ron Sher  wrote:
>
>> Hi,
>>
>> We have a multi tenant SAAS application in which we keep data for all
>> accounts of our clients (300 of them which we call services).
>> We keep data in monthly indices that grew to be about 700GB with 4.6
>> billion documents each month.
>> Each day we index a new account per day for each service.
>>
>> Each index is built from 6 shards and we use 1 replica.
>>
>> We're starting to have second thoughts of the structure of our indices
>> and about proper use of routing.
>> Few things we contemplate:
>>
>>- Use routing according to service so that we will probably benefit
>>from caching better.
>>- Change the indices according to service + month so that we will
>>query much less data, but will add many indices (now instead of 12 indices
>>a year we will have 300x12 and growing when the number of clients grow).
>>
>> Any thoughts/suggestions?
>>
>> Thanks,
>> Ron
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to elasticsearch+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/elasticsearch/096f4d7b-2702-46d7-96d6-f746f2389623%40googlegroups.com
>> <https://groups.google.com/d/msgid/elasticsearch/096f4d7b-2702-46d7-96d6-f746f2389623%40googlegroups.com?utm_medium=email&utm_source=footer>
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>  --
> You received this message because you are subscribed to a topic in the
> Google Groups "elasticsearch" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/elasticsearch/32uvCMR1kl4/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/CAEYi1X_TaWiZvpEpZ10_bMHhuOwSxpmBv%2BBhTNjPUHnKTA-Bgg%40mail.gmail.com
> <https://groups.google.com/d/msgid/elasticsearch/CAEYi1X_TaWiZvpEpZ10_bMHhuOwSxpmBv%2BBhTNjPUHnKTA-Bgg%40mail.gmail.com?utm_medium=email&utm_source=footer>
> .
>
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAKHuyJo5nyDgUfur-%3DNw6O4Jg98duO1GQVr5zRf_tcqqg-kJ-w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Looking for a suggestion to better organize our indices for performance

[Ron Sher]

Hi,

We have a multi tenant SAAS application in which we keep data for all 
accounts of our clients (300 of them which we call services).
We keep data in monthly indices that grew to be about 700GB with 4.6 
billion documents each month.
Each day we index a new account per day for each service.

Each index is built from 6 shards and we use 1 replica.

We're starting to have second thoughts of the structure of our indices and 
about proper use of routing.
Few things we contemplate:

   - Use routing according to service so that we will probably benefit from 
   caching better.
   - Change the indices according to service + month so that we will query 
   much less data, but will add many indices (now instead of 12 indices a year 
   we will have 300x12 and growing when the number of clients grow).

Any thoughts/suggestions?

Thanks,
Ron

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/096f4d7b-2702-46d7-96d6-f746f2389623%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

what would be the effect of using an arbitrary large count

[Ron Sher]

Hi
We have a multi tenant SaaS solution and we expose our use of elasticsearch 
through an API. We noticed that clients who use the API directly, and not 
through our front-end, don't bother with paging and just use a count of 500k or 
1m even if the actual count is much lower. I was wondering about the effect of 
not limiting the count since we're thinking of making a breaking change and 
limit this to 100.

Thanks for your help
Ron

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/9507d1c8-5f32-41bf-a537-3cd4e0c2ba9c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Java client - setTimeout vs actionGet(timeout)

[Ron Sher]

Thanks for the info.

Do you know what are the defaults?

On Sunday, November 30, 2014 5:53:49 PM UTC+2, Nikolas Everett wrote:
>
> Timeouts are server side and best effort. I believe action get(timeout) is 
> client side. 
>
> I use the http client but use both and set the server side timeout to 
> lower than the client side timeout. 
>
> The server side timeout should return partial results if possible. 
> On Nov 30, 2014 10:41 AM, "Ron Sher" > 
> wrote:
>
>> Hi all,
>>
>> I want to make sure the search query doesn't exceed some limit.
>>
>> I've seen the option to use a setTimeout vs actionGet(timeout).
>>
>> Can someone please explain the difference?
>>
>> Also, I've read somewhere that there's a default connection timeout. Can 
>> that be used instead. If so, how?
>>
>> Thanks for your help,
>>
>> Ron
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to elasticsearc...@googlegroups.com .
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/elasticsearch/0f39e669-bd98-4fc3-afbc-df9e3d9b1f52%40googlegroups.com
>>  
>> <https://groups.google.com/d/msgid/elasticsearch/0f39e669-bd98-4fc3-afbc-df9e3d9b1f52%40googlegroups.com?utm_medium=email&utm_source=footer>
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/660f86bb-0f96-431c-acb5-aa4f5971578a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Java client - setTimeout vs actionGet(timeout)

[Ron Sher]

Hi all,

I want to make sure the search query doesn't exceed some limit.

I've seen the option to use a setTimeout vs actionGet(timeout).

Can someone please explain the difference?

Also, I've read somewhere that there's a default connection timeout. Can 
that be used instead. If so, how?

Thanks for your help,

Ron

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/0f39e669-bd98-4fc3-afbc-df9e3d9b1f52%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

performance of aggregations as opposed to facets

[Ron Sher]

Hi,

I know that facets are deprecated and going to be removed eventually.

But, we currently use facets and we're fine with its current behavior.

My question is do aggregations offer any improvement in performance? Or 
maybe a degradation due to added functionality?

Any experience?

We're starting to run queries in a loop that may suggest that facets are 
actually (a bit) faster.

Thanks for your feedback,

Ron

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/3d272687-cd03-45d4-972c-96570396b3e1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: doc_count on aggregations doesn't match total hits

[Ron Sher]

So the doc count is for the nested documents?
what if I want to get the doc count of the "parent" docs? I guess I can use 
the total hits, right?

On Saturday, September 6, 2014 8:52:30 AM UTC+3, David Pilato wrote:
>
> It sounds like you are using nested documents. I guess it comes from here.
>
> --
> David ;-)
> Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs
>
>
> Le 6 sept. 2014 à 07:49, Ron Sher > a 
> écrit :
>
> Hi,
>
> I've started to use aggregations and it works very fast and cool. But it 
> seems to get the wrong doc_count as opposed to the total hits.
>
> For example - here's a typical search I use (I got for it total hits - 111 
> and doc_count 1348) :
> {
>   "query": {
> "filtered": {
>   "query": {
> "match_all": {}
>   },
>   "filter": {
> "bool": {
>   "must": [
> {
>   "term": {
> "service_id": "880"
>   }
> },
> {
>   "bool": {
> "must": {
>   "term": {
> "status_group": "paying"
>   }
> }
>   }
> }
>   ]
> }
>   }
> }
>   },
>   "aggregations": {
> "atts": {
>   "nested": {
> "path": "string_attributes"
>   },
>   "aggregations": {
> "filter_atts": {
>   "filter": {
> "term": {
>   "string_attributes.key": "Sales Manager"
> }
>   },
>   "aggregations": {
> "values": {
>   "terms": {
> "field": "string_attributes.value",
> "size": 0,
> "order": {
>   "_term": "asc"
> }
>   },
>   "aggregations": {
> "reversed": {
>   "reverse_nested": {},
>   "aggregations": {
> "health": {
>   "terms": {
> "field": "health"
>   }
> },
> "missing_health": {
>   "missing": {
> "field": "health"
>   }
> },
> "avg_contract_value": {
>   "avg": {
> "field": "contract_value"
>   }
> },
> "sum_contract_value": {
>   "sum": {
> "field": "contract_value"
>   }
> }
>   }
> }
>   }
> }
>   }
> }
>   }
> },
> "avg_contract_value": {
>   "avg": {
> "field": "contract_value"
>   }
> },
> "sum_contract_value": {
>   "sum": {
> "field": "contract_value"
>   }
> }
>   }
> }
>
> -- 
> You received this message because you are subscribed to the Google Groups "
> elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to elasticsearc...@googlegroups.com .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/elasticsearch/CAKHuyJpNLN8Zw7cQjFjTvQTpfrPLDkrhzBmEyFXApWYti1r1tQ%40mail.gmail.com
>  
> <https://groups.google.com/d/msgid/elasticsearch/CAKHuyJpNLN8Zw7cQjFjTvQTpfrPLDkrhzBmEyFXApWYti1r1tQ%40mail.gmail.com?utm_medium=email&utm_source=footer>
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/774c45ee-6ded-4577-ab8b-e517844b34d0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

doc_count on aggregations doesn't match total hits

[Ron Sher]

Hi,

I've started to use aggregations and it works very fast and cool. But it
seems to get the wrong doc_count as opposed to the total hits.

For example - here's a typical search I use (I got for it total hits - 111
and doc_count 1348) :
{
  "query": {
"filtered": {
  "query": {
"match_all": {}
  },
  "filter": {
"bool": {
  "must": [
{
  "term": {
"service_id": "880"
  }
},
{
  "bool": {
"must": {
  "term": {
"status_group": "paying"
  }
}
  }
}
  ]
}
  }
}
  },
  "aggregations": {
"atts": {
  "nested": {
"path": "string_attributes"
  },
  "aggregations": {
"filter_atts": {
  "filter": {
"term": {
  "string_attributes.key": "Sales Manager"
}
  },
  "aggregations": {
"values": {
  "terms": {
"field": "string_attributes.value",
"size": 0,
"order": {
  "_term": "asc"
}
  },
  "aggregations": {
"reversed": {
  "reverse_nested": {},
  "aggregations": {
"health": {
  "terms": {
"field": "health"
  }
},
"missing_health": {
  "missing": {
"field": "health"
  }
},
"avg_contract_value": {
  "avg": {
"field": "contract_value"
  }
},
"sum_contract_value": {
  "sum": {
"field": "contract_value"
  }
}
  }
}
  }
}
  }
}
  }
},
"avg_contract_value": {
  "avg": {
"field": "contract_value"
  }
},
"sum_contract_value": {
  "sum": {
"field": "contract_value"
  }
}
  }
}

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAKHuyJpNLN8Zw7cQjFjTvQTpfrPLDkrhzBmEyFXApWYti1r1tQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: inconsistent paging

[Ron Sher]

Thanks for the answer and sorry for the duplicate (posted from a different 
source by mistake)

On Monday, August 18, 2014 11:02:47 AM UTC+3, Adrien Grand wrote:
>
> Hi Ron,
>
> The cause of this issue is that Elasticsearch uses Lucene's internal doc 
> IDs as tie-breakers. Internal doc IDs might be completely different 
> across replicas of the same data, so this explains why documents that have 
> the same sort values are not consistently ordered.
>
> There are 2 potential ways to fix that problem:
>  1. Use scroll as David mentionned. It will create a context around your 
> request and will make sure that the same shards will be used for all pages. 
> However, it also gives another warranty, which is that the same 
> point-in-time view on the index will be used for each page, and this is 
> expensive to maintain.
>  2. Use a custom string value as a preference in order to always hit the 
> same shards for a given session[1]. This will help with always hitting 
> the same shards likely to 1. but without adding the additional cost of a 
> scroll.
>
> [1] 
> http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-preference.html
>
>
>
> On Mon, Aug 18, 2014 at 8:02 AM, Ron Sher 
> > wrote:
>
>> Hi,
>>
>> We've noticed a strange behavior in elasticsearch during paging. 
>>
>> In one case we use a paging size of 60 and we have 63 documents. So the 
>> first page is using size 60 and offset 0. The second page is using size 60 
>> and offset 60. What we see is that the result is inconsistent. Meaning, 
>> on the 2nd page, we sometimes get results that were before in the 1st page. 
>>
>> The query we use has an order by some numberic field that has many 
>> documents with the same value (0). 
>> It looks like the ordering between documents according to the same value, 
>> which is 0, isn't consistent. 
>>
>> Did anyone encounter such behavior? Any suggestions on resolving this? 
>>
>> We're using version 1.3.1. 
>>
>> Thanks, 
>> Ron
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to elasticsearc...@googlegroups.com .
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/elasticsearch/CAKHuyJpcYKepYzh%2BBU2MSD2RQ19zjHYiXgf3anWBL9esq9fkGQ%40mail.gmail.com
>>  
>> <https://groups.google.com/d/msgid/elasticsearch/CAKHuyJpcYKepYzh%2BBU2MSD2RQ19zjHYiXgf3anWBL9esq9fkGQ%40mail.gmail.com?utm_medium=email&utm_source=footer>
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
>
> -- 
> Adrien Grand
>  

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/277ec3ee-f7bf-4862-a816-efe2937a9609%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

inconsistent paging

[Ron Sher]

Hi,

We've noticed a strange behavior in elasticsearch during paging.

In one case we use a paging size of 60 and we have 63 documents. So the
first page is using size 60 and offset 0. The second page is using size 60
and offset 60. What we see is that the result is inconsistent. Meaning, on
the 2nd page, we sometimes get results that were before in the 1st page.

The query we use has an order by some numberic field that has many
documents with the same value (0).
It looks like the ordering between documents according to the same value,
which is 0, isn't consistent.

Did anyone encounter such behavior? Any suggestions on resolving this?

We're using version 1.3.1.

Thanks,
Ron

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAKHuyJpcYKepYzh%2BBU2MSD2RQ19zjHYiXgf3anWBL9esq9fkGQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

How do you do rolling upgrade with Chef?

[Ron Sher]

Hi,

I know there's a guide on installing elasticsearch with Chef 

 

 

here 

 but it looks like it's suitable for first installation or a complete 
cluster shutdown.
I'm looking for an automated rolling upgrade of elasticsearch.
Any suggestion?

Thanks,
Ron

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/bd2d123b-0a61-43f0-ba54-230198a89f43%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: marvel shows empty dashboard

[Ron Sher]

Getting somewhere...

Changed the hosts and now I get the following:
type[cluster_stats] missing: trying to auto create mapping, but dynamic
mapping is disabled


On Sun, Feb 2, 2014 at 5:26 PM, Boaz Leskes  wrote:

> It looks like marvel can't store the data on localhost:9200. The most
> common reason is running on another port. Another option is that ES is not
> bound on localhost or that http access is turned off (in which case marvel
> won't work).
>
>  Can you verify you can connect ES via the terminal while on the machine
> itself? If needed you can change the host & port marvel where sends data
> to. See
> http://www.elasticsearch.org/guide/en/marvel/current/index.html#stats-export
>
>
> On Sun, Feb 2, 2014 at 3:23 PM, Ron Sher  wrote:
>
>> The beginning of the log looks like:
>> [2014-02-02 15:38:56,003][INFO ][node ] [hades3]
>> version[0.90.10], pid[20219], build[0a5781f/2014-01-10T10:18:37Z]
>> [2014-02-02 15:38:56,004][INFO ][node ] [hades3]
>> initializing ...
>> [2014-02-02 15:38:56,027][INFO ][plugins  ] [hades3]
>> loaded [marvel], sites [marvel, head]
>> [2014-02-02 15:39:01,302][INFO ][node ] [hades3]
>> initialized
>> [2014-02-02 15:39:01,302][INFO ][node ] [hades3]
>> starting ...
>> [2014-02-02 15:39:01,410][INFO ][transport] [hades3]
>> bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address {inet[/
>> 192.168.10.148:9300]}
>> [2014-02-02 15:39:06,408][ERROR][marvel.agent.exporter] error
>> sending data
>>  java.io.FileNotFoundException: http://localhost:9200/_bulk
>>  
>> atsun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1623)
>> at org.elasticsearch.marvel.agent.exporter.ESExporter.
>> sendCloseExportingConnection(ESExporter.java:232)
>> at org.elasticsearch.marvel.agent.exporter.ESExporter.
>> exportXContent(ESExporter.java:252)
>> at org.elasticsearch.marvel.agent.exporter.ESExporter.
>> exportNodeStats(ESExporter.java:134)
>> at org.elasticsearch.marvel.agent.AgentService$ExportingWorker.
>> exportNodeStats(AgentService.java:274)
>> at org.elasticsearch.marvel.agent.AgentService$ExportingWorker.
>> run(AgentService.java:174)
>> at java.lang.Thread.run(Thread.java:724)
>>
>>
>>
>> On Sun, Feb 2, 2014 at 3:43 PM, Ron Sher  wrote:
>>
>>> I've noticed that I didn't have a logging.yml (was called logging.xml
>>> instead).
>>> Changed that and then I see:
>>>
>>>  [2014-02-02 15:42:19,824][ERROR][marvel.agent.exporter] error
>>> sending data
>>>  java.io.FileNotFoundException: http://localhost:9200/_bulk
>>> 
>>> atsun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1623)
>>> at org.elasticsearch.marvel.agent.exporter.ESExporter.
>>> sendCloseExportingConnection(ESExporter.java:232)
>>> at org.elasticsearch.marvel.agent.exporter.ESExporter.
>>> exportXContent(ESExporter.java:252)
>>> at org.elasticsearch.marvel.agent.exporter.ESExporter.
>>> exportClusterStats(ESExporter.java:172)
>>> at org.elasticsearch.marvel.agent.AgentService$ExportingWorker.
>>> exportClusterStats(AgentService.java:214)
>>> at org.elasticsearch.marvel.agent.AgentService$ExportingWorker.
>>> run(AgentService.java:180)
>>> at java.lang.Thread.run(Thread.java:724)
>>>
>>>
>>> What now?
>>>
>>>
>>> On Sun, Feb 2, 2014 at 11:29 AM, Ron Sher  wrote:
>>>
>>>> This is what I see in the log after the restart:
>>>> STATUS | wrapper  | 2014/02/02 11:26:54 |   Copyright (C) 1999-2011
>>>> Tanuki Software, Ltd. All Rights Reserved.
>>>> STATUS | wrapper  | 2014/02/02 11:26:54 |
>>>> http://wrapper.tanukisoftware.com
>>>> STATUS | wrapper  | 2014/02/02 11:26:54 |
>>>> WARN   | wrapper  | 2014/02/02 11:26:54 | Encountered an invalid
>>>> numerical value for configuration property 
>>>> wrapper.java.initmemory=%ES_HEAP_SIZE%.
>>>>  Resolving to 0.
>>>> WARN   | wrapper  | 2014/02/02 11:26:54 | Encountered an invalid
>>>> numerical value for configuration property 
>>>> wrapper.java.maxmemory=%ES_HEAP_SIZE%.
>>>>  Resolving to 0.
>>>> STATUS | wrapper  | 2014/02/02 11:26:54 | Launching a JVM...
>>>> INFO   | jvm 1| 2014/02/02 11:26:55

Re: marvel shows empty dashboard

[Ron Sher]

The beginning of the log looks like:
[2014-02-02 15:38:56,003][INFO ][node ] [hades3] version
[0.90.10], pid[20219], build[0a5781f/2014-01-10T10:18:37Z]
[2014-02-02 15:38:56,004][INFO ][node ] [hades3]
initializing ...
[2014-02-02 15:38:56,027][INFO ][plugins  ] [hades3] loaded
[marvel], sites [marvel, head]
[2014-02-02 15:39:01,302][INFO ][node ] [hades3]
initialized
[2014-02-02 15:39:01,302][INFO ][node ] [hades3]
starting ...
[2014-02-02 15:39:01,410][INFO ][transport] [hades3]
bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address {inet[/
192.168.10.148:9300]}
[2014-02-02 15:39:06,408][ERROR][marvel.agent.exporter] error sending
data
java.io.FileNotFoundException: http://localhost:9200/_bulk

atsun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1623)
at org.elasticsearch.marvel.agent.exporter.ESExporter.
sendCloseExportingConnection(ESExporter.java:232)
at org.elasticsearch.marvel.agent.exporter.ESExporter.exportXContent
(ESExporter.java:252)
at org.elasticsearch.marvel.agent.exporter.ESExporter.
exportNodeStats(ESExporter.java:134)
at org.elasticsearch.marvel.agent.AgentService$ExportingWorker.
exportNodeStats(AgentService.java:274)
at org.elasticsearch.marvel.agent.AgentService$ExportingWorker.run(
AgentService.java:174)
at java.lang.Thread.run(Thread.java:724)



On Sun, Feb 2, 2014 at 3:43 PM, Ron Sher  wrote:

> I've noticed that I didn't have a logging.yml (was called logging.xml
> instead).
> Changed that and then I see:
>
> [2014-02-02 15:42:19,824][ERROR][marvel.agent.exporter] error sending
> data
> java.io.FileNotFoundException: http://localhost:9200/_bulk
> 
> atsun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1623)
> at org.elasticsearch.marvel.agent.exporter.ESExporter.
> sendCloseExportingConnection(ESExporter.java:232)
> at org.elasticsearch.marvel.agent.exporter.ESExporter.
> exportXContent(ESExporter.java:252)
> at org.elasticsearch.marvel.agent.exporter.ESExporter.
> exportClusterStats(ESExporter.java:172)
> at org.elasticsearch.marvel.agent.AgentService$ExportingWorker.
> exportClusterStats(AgentService.java:214)
> at org.elasticsearch.marvel.agent.AgentService$ExportingWorker.run
> (AgentService.java:180)
> at java.lang.Thread.run(Thread.java:724)
>
>
> What now?
>
>
> On Sun, Feb 2, 2014 at 11:29 AM, Ron Sher  wrote:
>
>> This is what I see in the log after the restart:
>> STATUS | wrapper  | 2014/02/02 11:26:54 |   Copyright (C) 1999-2011
>> Tanuki Software, Ltd. All Rights Reserved.
>> STATUS | wrapper  | 2014/02/02 11:26:54 |
>> http://wrapper.tanukisoftware.com
>> STATUS | wrapper  | 2014/02/02 11:26:54 |
>> WARN   | wrapper  | 2014/02/02 11:26:54 | Encountered an invalid
>> numerical value for configuration property 
>> wrapper.java.initmemory=%ES_HEAP_SIZE%.
>>  Resolving to 0.
>> WARN   | wrapper  | 2014/02/02 11:26:54 | Encountered an invalid
>> numerical value for configuration property 
>> wrapper.java.maxmemory=%ES_HEAP_SIZE%.
>>  Resolving to 0.
>> STATUS | wrapper  | 2014/02/02 11:26:54 | Launching a JVM...
>> INFO   | jvm 1| 2014/02/02 11:26:55 | WrapperManager: Initializing...
>> INFO   | jvm 1| 2014/02/02 11:27:01 | log4j:WARN No appenders could
>> be found for logger (node).
>> INFO   | jvm 1| 2014/02/02 11:27:01 | log4j:WARN Please initialize
>> the log4j system properly.
>> INFO   | jvm 1| 2014/02/02 11:27:01 | log4j:WARN See
>> http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
>>
>>
>>
>> On Sun, Feb 2, 2014 at 10:05 AM, Boaz Leskes  wrote:
>>
>>> OK, interesting.
>>>
>>> Can you check the logs on one of the nodes to if there are any errors?
>>> also please double check that when the node started it logged a line
>>> similar to the following (containing marvel in the list after loaded):
>>>
>>> [2014-02-02 09:03:48,043][INFO ][plugins  ] 
>>> [Stonewall]loaded
>>> [marvel], sites [marvel]
>>>
>>>
>>>
>>> start a node (one is enough)
>>>
>>>
>>> On Sunday, February 2, 2014 8:50:47 AM UTC+1, Ron Sher wrote:
>>>
>>>> again, using 0.90.10.
>>>> This is what I did:
>>>> bin/plugin -i elasticsearch/marvel/latest
>>>>  /etc/init.d/elasticsearch restart
>>>>
>>>> Did the same on a different cluster - still doesn't work.
>>>>
>>

Re: marvel shows empty dashboard

[Ron Sher]

I've noticed that I didn't have a logging.yml (was called logging.xml
instead).
Changed that and then I see:

[2014-02-02 15:42:19,824][ERROR][marvel.agent.exporter] error sending
data
java.io.FileNotFoundException: http://localhost:9200/_bulk

atsun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1623)
at org.elasticsearch.marvel.agent.exporter.ESExporter.
sendCloseExportingConnection(ESExporter.java:232)
at org.elasticsearch.marvel.agent.exporter.ESExporter.exportXContent
(ESExporter.java:252)
at org.elasticsearch.marvel.agent.exporter.ESExporter.
exportClusterStats(ESExporter.java:172)
at org.elasticsearch.marvel.agent.AgentService$ExportingWorker.
exportClusterStats(AgentService.java:214)
at org.elasticsearch.marvel.agent.AgentService$ExportingWorker.run(
AgentService.java:180)
at java.lang.Thread.run(Thread.java:724)


What now?


On Sun, Feb 2, 2014 at 11:29 AM, Ron Sher  wrote:

> This is what I see in the log after the restart:
> STATUS | wrapper  | 2014/02/02 11:26:54 |   Copyright (C) 1999-2011 Tanuki
> Software, Ltd. All Rights Reserved.
> STATUS | wrapper  | 2014/02/02 11:26:54 |
> http://wrapper.tanukisoftware.com
> STATUS | wrapper  | 2014/02/02 11:26:54 |
> WARN   | wrapper  | 2014/02/02 11:26:54 | Encountered an invalid numerical
> value for configuration property wrapper.java.initmemory=%ES_HEAP_SIZE%.
>  Resolving to 0.
> WARN   | wrapper  | 2014/02/02 11:26:54 | Encountered an invalid numerical
> value for configuration property wrapper.java.maxmemory=%ES_HEAP_SIZE%.
>  Resolving to 0.
> STATUS | wrapper  | 2014/02/02 11:26:54 | Launching a JVM...
> INFO   | jvm 1| 2014/02/02 11:26:55 | WrapperManager: Initializing...
> INFO   | jvm 1| 2014/02/02 11:27:01 | log4j:WARN No appenders could
> be found for logger (node).
> INFO   | jvm 1| 2014/02/02 11:27:01 | log4j:WARN Please initialize
> the log4j system properly.
> INFO   | jvm 1| 2014/02/02 11:27:01 | log4j:WARN See
> http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
>
>
>
> On Sun, Feb 2, 2014 at 10:05 AM, Boaz Leskes  wrote:
>
>> OK, interesting.
>>
>> Can you check the logs on one of the nodes to if there are any errors?
>> also please double check that when the node started it logged a line
>> similar to the following (containing marvel in the list after loaded):
>>
>> [2014-02-02 09:03:48,043][INFO ][plugins  ] [Stonewall]loaded
>> [marvel], sites [marvel]
>>
>>
>>
>> start a node (one is enough)
>>
>>
>> On Sunday, February 2, 2014 8:50:47 AM UTC+1, Ron Sher wrote:
>>
>>> again, using 0.90.10.
>>> This is what I did:
>>> bin/plugin -i elasticsearch/marvel/latest
>>>  /etc/init.d/elasticsearch restart
>>>
>>> Did the same on a different cluster - still doesn't work.
>>>
>>>
>>> On Sun, Feb 2, 2014 at 9:45 AM, Boaz Leskes  wrote:
>>>
>>>> Hi Ron,
>>>>
>>>> It looks like no data is sent. What version of ES are you running? Did
>>>> you restart tES after installing the plugin?
>>>>
>>>> Regards,
>>>> Boaz
>>>>
>>>>
>>>> On Sun, Feb 2, 2014 at 8:27 AM, Ron Sher  wrote:
>>>>
>>>>> Tony,
>>>>> What do you mean by ?
>>>>>
>>>>>  Indeed I've added marvel to an already running cluster and no data
>>>>> is shown in the dashboard and I don't see any marvel index
>>>>>
>>>>> Ron
>>>>>
>>>>>
>>>>> On Fri, Jan 31, 2014 at 12:47 AM, Tony Su  wrote:
>>>>>
>>>>>>  Just a FYI
>>>>>> I just installed Marvel and I noticed you need  for Marvel
>>>>>> to start collecting and displaying data, even about itself.
>>>>>> If your machines are all previously setup and data already in place,
>>>>>> I don't know if you'd read anything.
>>>>>> Other thing I noticed is that  ES nodes have to be the same
>>>>>> version.
>>>>>> So, for example if Marvel is pointing to an odd-ball ES node, there
>>>>>> would not be any activity with any other ES node so would be blank.
>>>>>>
>>>>>> And, I assume that after changing ES versions you restarted the ES
>>>>>> services on every node? Otherwise you're still stuck on the previous
>>>>>> version.
>>>>>>
>>>>>> HTH,
>&g

Re: marvel shows empty dashboard

[Ron Sher]

This is what I see in the log after the restart:
STATUS | wrapper  | 2014/02/02 11:26:54 |   Copyright (C) 1999-2011 Tanuki
Software, Ltd. All Rights Reserved.
STATUS | wrapper  | 2014/02/02 11:26:54 |
http://wrapper.tanukisoftware.com
STATUS | wrapper  | 2014/02/02 11:26:54 |
WARN   | wrapper  | 2014/02/02 11:26:54 | Encountered an invalid numerical
value for configuration property wrapper.java.initmemory=%ES_HEAP_SIZE%.
 Resolving to 0.
WARN   | wrapper  | 2014/02/02 11:26:54 | Encountered an invalid numerical
value for configuration property wrapper.java.maxmemory=%ES_HEAP_SIZE%.
 Resolving to 0.
STATUS | wrapper  | 2014/02/02 11:26:54 | Launching a JVM...
INFO   | jvm 1| 2014/02/02 11:26:55 | WrapperManager: Initializing...
INFO   | jvm 1| 2014/02/02 11:27:01 | log4j:WARN No appenders could be
found for logger (node).
INFO   | jvm 1| 2014/02/02 11:27:01 | log4j:WARN Please initialize the
log4j system properly.
INFO   | jvm 1| 2014/02/02 11:27:01 | log4j:WARN See
http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.



On Sun, Feb 2, 2014 at 10:05 AM, Boaz Leskes  wrote:

> OK, interesting.
>
> Can you check the logs on one of the nodes to if there are any errors?
> also please double check that when the node started it logged a line
> similar to the following (containing marvel in the list after loaded):
>
> [2014-02-02 09:03:48,043][INFO ][plugins  ] [Stonewall]loaded
> [marvel], sites [marvel]
>
>
>
> start a node (one is enough)
>
>
> On Sunday, February 2, 2014 8:50:47 AM UTC+1, Ron Sher wrote:
>
>> again, using 0.90.10.
>> This is what I did:
>> bin/plugin -i elasticsearch/marvel/latest
>>  /etc/init.d/elasticsearch restart
>>
>> Did the same on a different cluster - still doesn't work.
>>
>>
>> On Sun, Feb 2, 2014 at 9:45 AM, Boaz Leskes  wrote:
>>
>>> Hi Ron,
>>>
>>> It looks like no data is sent. What version of ES are you running? Did
>>> you restart tES after installing the plugin?
>>>
>>> Regards,
>>> Boaz
>>>
>>>
>>> On Sun, Feb 2, 2014 at 8:27 AM, Ron Sher  wrote:
>>>
>>>> Tony,
>>>> What do you mean by ?
>>>>
>>>>  Indeed I've added marvel to an already running cluster and no data is
>>>> shown in the dashboard and I don't see any marvel index
>>>>
>>>> Ron
>>>>
>>>>
>>>> On Fri, Jan 31, 2014 at 12:47 AM, Tony Su  wrote:
>>>>
>>>>>  Just a FYI
>>>>> I just installed Marvel and I noticed you need  for Marvel
>>>>> to start collecting and displaying data, even about itself.
>>>>> If your machines are all previously setup and data already in place, I
>>>>> don't know if you'd read anything.
>>>>> Other thing I noticed is that  ES nodes have to be the same
>>>>> version.
>>>>> So, for example if Marvel is pointing to an odd-ball ES node, there
>>>>> would not be any activity with any other ES node so would be blank.
>>>>>
>>>>> And, I assume that after changing ES versions you restarted the ES
>>>>> services on every node? Otherwise you're still stuck on the previous
>>>>> version.
>>>>>
>>>>> HTH,
>>>>> Tony
>>>>>
>>>>>
>>>>>
>>>>> On Thursday, January 30, 2014 11:24:11 AM UTC-8, Brad Jordan wrote:
>>>>>
>>>>>> I keep getting this alert: *No results* There were no results
>>>>>> because no indices were found that match your selected time span.
>>>>>>
>>>>>> -Brad
>>>>>>
>>>>>> On Thursday, January 30, 2014 12:23:04 PM UTC-7, Brad Jordan wrote:
>>>>>>>
>>>>>>> Upgraded my cluster to 0.9.10 and still get a blank Marvel
>>>>>>> dashboard...
>>>>>>>
>>>>>>> I have about 30 indexes which all have data starting on about
>>>>>>> 1/20/2014 going back to 5/01/2013. Bigdesk and the Head plugin have no
>>>>>>> problems seeing that nodes, shards, indexes. Is there an additional step
>>>>>>> I'm missing?
>>>>>>>
>>>>>>> -Brad
>>>>>>>
>>>>>>> On Thursday, January 30, 2014 10:48:04 AM UTC-7, Brad Jordan wrote:
>>>>>>>>
>>>>>>>> Thanks!
>>>>>>>>
>&g

Re: marvel shows empty dashboard

[Ron Sher]

again, using 0.90.10.
This is what I did:
bin/plugin -i elasticsearch/marvel/latest
 /etc/init.d/elasticsearch restart

Did the same on a different cluster - still doesn't work.


On Sun, Feb 2, 2014 at 9:45 AM, Boaz Leskes  wrote:

> Hi Ron,
>
> It looks like no data is sent. What version of ES are you running? Did you
> restart tES after installing the plugin?
>
> Regards,
> Boaz
>
>
> On Sun, Feb 2, 2014 at 8:27 AM, Ron Sher  wrote:
>
>> Tony,
>> What do you mean by ?
>>
>>  Indeed I've added marvel to an already running cluster and no data is
>> shown in the dashboard and I don't see any marvel index
>>
>> Ron
>>
>>
>> On Fri, Jan 31, 2014 at 12:47 AM, Tony Su  wrote:
>>
>>>  Just a FYI
>>> I just installed Marvel and I noticed you need  for Marvel to
>>> start collecting and displaying data, even about itself.
>>> If your machines are all previously setup and data already in place, I
>>> don't know if you'd read anything.
>>> Other thing I noticed is that  ES nodes have to be the same version.
>>> So, for example if Marvel is pointing to an odd-ball ES node, there
>>> would not be any activity with any other ES node so would be blank.
>>>
>>> And, I assume that after changing ES versions you restarted the ES
>>> services on every node? Otherwise you're still stuck on the previous
>>> version.
>>>
>>> HTH,
>>> Tony
>>>
>>>
>>>
>>> On Thursday, January 30, 2014 11:24:11 AM UTC-8, Brad Jordan wrote:
>>>
>>>> I keep getting this alert: *No results* There were no results because
>>>> no indices were found that match your selected time span.
>>>>
>>>> -Brad
>>>>
>>>> On Thursday, January 30, 2014 12:23:04 PM UTC-7, Brad Jordan wrote:
>>>>>
>>>>> Upgraded my cluster to 0.9.10 and still get a blank Marvel
>>>>> dashboard...
>>>>>
>>>>> I have about 30 indexes which all have data starting on about
>>>>> 1/20/2014 going back to 5/01/2013. Bigdesk and the Head plugin have no
>>>>> problems seeing that nodes, shards, indexes. Is there an additional step
>>>>> I'm missing?
>>>>>
>>>>> -Brad
>>>>>
>>>>> On Thursday, January 30, 2014 10:48:04 AM UTC-7, Brad Jordan wrote:
>>>>>>
>>>>>> Thanks!
>>>>>>
>>>>>>
>>>>>> On Thursday, January 30, 2014 10:43:35 AM UTC-7, Boaz Leskes wrote:
>>>>>>>
>>>>>>> Hi brad,
>>>>>>>
>>>>>>> You should upgrade to 0.90.10. 0.90.7 is not supported by marvel.
>>>>>>>
>>>>>>> Cheers,
>>>>>>> Boaz
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Jan 30, 2014 at 6:39 PM, Brad Jordan wrote:
>>>>>>>
>>>>>>>> I also have the same issue. I'm running elasticsearch-0.90.7. I
>>>>>>>> installed the plugin, bounced the cluster and Marvel is just blank. Is
>>>>>>>> there a config I'm missing?
>>>>>>>>
>>>>>>>> -Brad
>>>>>>>>
>>>>>>>> On Thursday, January 30, 2014 5:22:03 AM UTC-7, Ron Sher wrote:
>>>>>>>>>
>>>>>>>>> Hey,
>>>>>>>>>
>>>>>>>>> Just tried installing the plugin.
>>>>>>>>> Installation was easy (bin/plugin -i elasticsearch/marvel/latest 
>>>>>>>>> followed
>>>>>>>>> by a restart) by then the plugin shows an empty dashboard.
>>>>>>>>>
>>>>>>>>> Am I missing something?\
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Ron
>>>>>>>>>
>>>>>>>>>--
>>>>>>>> You received this message because you are subscribed to a topic in
>>>>>>>> the Google Groups "elasticsearch" group.
>>>>>>>> To unsubscribe from this topic, visit https://groups.google.com/d/
>>>>>>>> topic/elasticsearch/UPBMKvxwwM8/unsubscribe.
>>>>>>>> To unsubscribe from this group and all its topics, send an email to
&g

Re: marvel shows empty dashboard

[Ron Sher]

Tony,
What do you mean by ?

Indeed I've added marvel to an already running cluster and no data is shown
in the dashboard and I don't see any marvel index

Ron


On Fri, Jan 31, 2014 at 12:47 AM, Tony Su  wrote:

> Just a FYI
> I just installed Marvel and I noticed you need  for Marvel to
> start collecting and displaying data, even about itself.
> If your machines are all previously setup and data already in place, I
> don't know if you'd read anything.
> Other thing I noticed is that  ES nodes have to be the same version.
> So, for example if Marvel is pointing to an odd-ball ES node, there would
> not be any activity with any other ES node so would be blank.
>
> And, I assume that after changing ES versions you restarted the ES
> services on every node? Otherwise you're still stuck on the previous
> version.
>
> HTH,
> Tony
>
>
>
> On Thursday, January 30, 2014 11:24:11 AM UTC-8, Brad Jordan wrote:
>
>> I keep getting this alert: *No results* There were no results because no
>> indices were found that match your selected time span.
>>
>> -Brad
>>
>> On Thursday, January 30, 2014 12:23:04 PM UTC-7, Brad Jordan wrote:
>>>
>>> Upgraded my cluster to 0.9.10 and still get a blank Marvel dashboard...
>>>
>>> I have about 30 indexes which all have data starting on about 1/20/2014
>>> going back to 5/01/2013. Bigdesk and the Head plugin have no problems
>>> seeing that nodes, shards, indexes. Is there an additional step I'm missing?
>>>
>>> -Brad
>>>
>>> On Thursday, January 30, 2014 10:48:04 AM UTC-7, Brad Jordan wrote:
>>>>
>>>> Thanks!
>>>>
>>>>
>>>> On Thursday, January 30, 2014 10:43:35 AM UTC-7, Boaz Leskes wrote:
>>>>>
>>>>> Hi brad,
>>>>>
>>>>> You should upgrade to 0.90.10. 0.90.7 is not supported by marvel.
>>>>>
>>>>> Cheers,
>>>>> Boaz
>>>>>
>>>>>
>>>>> On Thu, Jan 30, 2014 at 6:39 PM, Brad Jordan wrote:
>>>>>
>>>>>> I also have the same issue. I'm running elasticsearch-0.90.7. I
>>>>>> installed the plugin, bounced the cluster and Marvel is just blank. Is
>>>>>> there a config I'm missing?
>>>>>>
>>>>>> -Brad
>>>>>>
>>>>>> On Thursday, January 30, 2014 5:22:03 AM UTC-7, Ron Sher wrote:
>>>>>>>
>>>>>>> Hey,
>>>>>>>
>>>>>>> Just tried installing the plugin.
>>>>>>> Installation was easy (bin/plugin -i elasticsearch/marvel/latest 
>>>>>>> followed
>>>>>>> by a restart) by then the plugin shows an empty dashboard.
>>>>>>>
>>>>>>> Am I missing something?\
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Ron
>>>>>>>
>>>>>>>   --
>>>>>> You received this message because you are subscribed to a topic in
>>>>>> the Google Groups "elasticsearch" group.
>>>>>> To unsubscribe from this topic, visit https://groups.google.com/d/
>>>>>> topic/elasticsearch/UPBMKvxwwM8/unsubscribe.
>>>>>> To unsubscribe from this group and all its topics, send an email to
>>>>>> elasticsearc...@googlegroups.com.
>>>>>> To view this discussion on the web visit https://groups.google.com/d/
>>>>>> msgid/elasticsearch/4dcf998b-89fe-4f07-bb2d-a036cd567c66%
>>>>>> 40googlegroups.com.
>>>>>> For more options, visit https://groups.google.com/groups/opt_out.
>>>>>>
>>>>>
>>>>>  --
> You received this message because you are subscribed to a topic in the
> Google Groups "elasticsearch" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/elasticsearch/UPBMKvxwwM8/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/43083c5a-bc16-4f01-8450-15402e82d798%40googlegroups.com
> .
>
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAKHuyJr7%3Deh-s85HrwTjmKJ7wTcezU4aiSQG6P8vC2zFd2OD-w%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: marvel shows empty dashboard

[Ron Sher]

I don't see any indices of marve:

ron@ron-VirtualBox:~$ curl -XGET "http://hades3:1/.marvel*/_stats?
clear&store"
curl: (7) couldn't connect to host
ron@ron-VirtualBox:~$ curl -XGET "http://hades3:9200/.marvel*/_stats?
clear&store"
{"error":"IndexMissingException[[.marvel*] missing]","status":404}


On Fri, Jan 31, 2014 at 10:31 AM, Boaz Leskes  wrote:

> Hi Tony,
>
> Marvel starts collecting stats 5 seconds after the node starts. If that's
> not happening for you, something else is wrong (an let's please open
> another thread on it, this one is already all over the place ;) ).
>
> As to ES version - with the exception of a rolling upgrade you are highly
> recommended to keep the versions of all nodes the same. This goes for both
> ES versions and the Java version you use. This is necessary to guaranty the
> correct behavior of the cluster.
>
> Cheers,
> Boaz
>
>
> On Thursday, January 30, 2014 11:47:36 PM UTC+1, Tony Su wrote:
>>
>> Just a FYI
>> I just installed Marvel and I noticed you need  for Marvel to
>> start collecting and displaying data, even about itself.
>> If your machines are all previously setup and data already in place, I
>> don't know if you'd read anything.
>> Other thing I noticed is that  ES nodes have to be the same version.
>> So, for example if Marvel is pointing to an odd-ball ES node, there would
>> not be any activity with any other ES node so would be blank.
>>
>> And, I assume that after changing ES versions you restarted the ES
>> services on every node? Otherwise you're still stuck on the previous
>> version.
>>
>> HTH,
>> Tony
>>
>>
>>
>> On Thursday, January 30, 2014 11:24:11 AM UTC-8, Brad Jordan wrote:
>>
>>> I keep getting this alert: *No results* There were no results because
>>> no indices were found that match your selected time span.
>>>
>>> -Brad
>>>
>>> On Thursday, January 30, 2014 12:23:04 PM UTC-7, Brad Jordan wrote:
>>>>
>>>> Upgraded my cluster to 0.9.10 and still get a blank Marvel dashboard...
>>>>
>>>> I have about 30 indexes which all have data starting on about 1/20/2014
>>>> going back to 5/01/2013. Bigdesk and the Head plugin have no problems
>>>> seeing that nodes, shards, indexes. Is there an additional step I'm 
>>>> missing?
>>>>
>>>> -Brad
>>>>
>>>> On Thursday, January 30, 2014 10:48:04 AM UTC-7, Brad Jordan wrote:
>>>>>
>>>>> Thanks!
>>>>>
>>>>>
>>>>> On Thursday, January 30, 2014 10:43:35 AM UTC-7, Boaz Leskes wrote:
>>>>>>
>>>>>> Hi brad,
>>>>>>
>>>>>> You should upgrade to 0.90.10. 0.90.7 is not supported by marvel.
>>>>>>
>>>>>> Cheers,
>>>>>> Boaz
>>>>>>
>>>>>>
>>>>>> On Thu, Jan 30, 2014 at 6:39 PM, Brad Jordan wrote:
>>>>>>
>>>>>>> I also have the same issue. I'm running elasticsearch-0.90.7. I
>>>>>>> installed the plugin, bounced the cluster and Marvel is just blank. Is
>>>>>>> there a config I'm missing?
>>>>>>>
>>>>>>> -Brad
>>>>>>>
>>>>>>> On Thursday, January 30, 2014 5:22:03 AM UTC-7, Ron Sher wrote:
>>>>>>>>
>>>>>>>> Hey,
>>>>>>>>
>>>>>>>> Just tried installing the plugin.
>>>>>>>> Installation was easy (bin/plugin -i elasticsearch/marvel/latest 
>>>>>>>> followed
>>>>>>>> by a restart) by then the plugin shows an empty dashboard.
>>>>>>>>
>>>>>>>> Am I missing something?\
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Ron
>>>>>>>>
>>>>>>>>   --
>>>>>>> You received this message because you are subscribed to a topic in
>>>>>>> the Google Groups "elasticsearch" group.
>>>>>>> To unsubscribe from this topic, visit https://groups.google.com/d/
>>>>>>> topic/elasticsearch/UPBMKvxwwM8/unsubscribe.
>>>>>>> To unsubscribe from this group and all its topics, send an email to
>>>>>>> elasticsearc...@googlegroups.com.
>>>>>>> To view this discussion on the web visit
>>>>>>> https://groups.google.com/d/msgid/elasticsearch/4dcf998b-
>>>>>>> 89fe-4f07-bb2d-a036cd567c66%40googlegroups.com.
>>>>>>> For more options, visit https://groups.google.com/groups/opt_out.
>>>>>>>
>>>>>>
>>>>>>  --
> You received this message because you are subscribed to a topic in the
> Google Groups "elasticsearch" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/elasticsearch/UPBMKvxwwM8/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/d55bddb3-f7d4-4247-ab26-6d7007486eac%40googlegroups.com
> .
>
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAKHuyJpBsioPKLDXUcfiu_o6yHVa0_4cWVTU65g5WB6gXrD7PQ%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: marvel shows empty dashboard

[Ron Sher]

We have 0.90.10


On Thu, Jan 30, 2014 at 2:41 PM, DH  wrote:

> Hello,
> I had the same problem.
> For me, It was because I didn't pay attention enough.
> What is your version of ES? You need to have a 90.9 or higher. On my
> former 0.90.5, the pugin was detected but showed nothing.
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "elasticsearch" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/elasticsearch/UPBMKvxwwM8/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/d852c6a9-2bd4-428f-a36a-7f3458537e81%40googlegroups.com
> .
>
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAKHuyJphzG4%3D4%2B8NYedR6%2BLCXwuYPioyLKMXsGfLw5majuG%2BqA%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

marvel shows empty dashboard

[Ron Sher]

Hey,

Just tried installing the plugin.
Installation was easy (bin/plugin -i elasticsearch/marvel/latest followed 
by a restart) by then the plugin shows an empty dashboard.

Am I missing something?\

Thanks,
Ron

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/4d0f4f2e-f824-408f-bff1-d4d79aec470c%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.