Re: Possible to see how 'busy' an index is vis-a-vis updates?

You can do a count and then monitor that, but there is not a specific API endpoint for this sort of thing (unless I forgot one). On 14 February 2015 at 06:32, MrLarryQ wrote: > Hi everyone, > > We use elasticsearch at work and it's terrific. One thing we do use is > run various curl commands ag

Re: Logstash Elasticsearch Reindexing Question

Have you tried exporting and importing the mapping to the new cluster, then reindex using LS? LS makes a number of assumptions when it creates mappings for a new index, which is probably what is happening. On 16 February 2015 at 07:54, Sumit Arora wrote: > I'm new to logstash and trying to make

Re: Spikes in usage and Logstash connection issues

Can you link us to your LS config? On 14 February 2015 at 08:56, rhea ghosh wrote: > Hi, I have a 5 node cluster that I'm using as part of an ELK system. Most > of the time it works great but today we saw a spike in writes for one of > the nodes and around the same time we saw indexing on that n

Re: Adding timestamp property

You have the _timestamp field as you have defined, and you should be able to search on that field in KB. Are you not seeing this? On 15 February 2015 at 00:30, Roy Zanbel wrote: > Hi, > > New to elasticsearch and have a simple question had a hard time finding > online. > I wish to add a timesta

Re: Best practice - architecture feedback/opinion needed

With one node you can never allocate replicas, which is why it's in a yellow state. I'd go with setup #1 personally, but you probably want more RAM, say 4GB heap. Then set all to be master eligible so that you get some level of protection against node loss. Make sure you also use curator for mana

Re: Cassandra monitoring by ELK

Thanks for sharing this! On 15 February 2015 at 09:22, kamal bisht wrote: > Hi > > I developed and contributing in "Cassandra cluster monitoring by using > ELK", If anyone wants to use that or wants to contributing then this is the > Github link https://github.com/kmluvce/cassandra-monitoring-b

Re: how to see individual node stats on marvel?

totally agree. In my cluster, each box have two es nodes. it is sort of useful for me to see combined metrics on individual box. Jinyuan (Jack) Zhou On Sun, Feb 15, 2015 at 10:22 PM, Boaz Leskes wrote: > That may indeed work but is much trickier to interpret the results, so > use with care.

Re: how to see individual node stats on marvel?

That may indeed  work but is much trickier to interpret the results, so use with care. On Mon, Feb 16, 2015 at 7:07 AM, Jinyuan Zhou wrote: > Another one is the query box. just type full or partial name of node(s). > one search box corresponds to display of the combined metrics of nodes > whos

Re: how to see individual node stats on marvel?

Another one is the query box. just type full or partial name of node(s). one search box corresponds to display of the combined metrics of nodes whose name is/matches the typed in text. Jinyuan (Jack) Zhou On Sun, Feb 15, 2015 at 2:30 PM, Boaz Leskes wrote: > Two more little tricks - you can cl

Re: Scrolling using the Scala API to ES

I've added an issue to clarify the usage of the scroll syntax in elastic4s. https://github.com/sksamuel/elastic4s/issues/273 On Wednesday, November 12, 2014 at 3:01:10 PM UTC, Ramdev Wudali wrote: > > > ok, So I figured this bit out with a few provisos ( there wasn't > a test that covere

Re: Using scala elastic4s module for search an ES cluster

Interesting question. I can't find anything in the scroll docs that says it would be single node only, so I guess the next question is - does this work in the Java client for you? If so, raise a bug issue on github.com/sksamuel/elastic4s if you can. On Thursday, February 5, 2015 at 1:22:58 PM U

Re: disabling dynamic date time formatting in python api or globally

Set the mapping for this field to be a string. http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/mapping-core-types.html#string It won’t be converted as a date. -- David P

Re: how to see individual node stats on marvel?

Two more little tricks - you can click on any metric in the node table in the Overview dashboard and it will take you to the Node Stats dashboard for that node including opening the right chart for that metric. Also, you can select the checkbox next to the node names in the same table and then c

Re: Upgrade to 1.4.2 from 1.1.1 causes spike in Disk IO

1.4.2 comes with a couple optimization which might explain the drop in field data cache and lucene memory (most notably here is the disabling of bloom filters by default - due to improvements in lucene they are not as needed as before and we can save on the memory). The IO spike might come fro

Re: Marvel being updated along with Kibana?

Hi Scott, Yeah, there are plans to upgrade Marvel to the awesomeness of K4. One of the design corner stones of K4 was to allow extending it and building apps such as Marvel on top of it. As you can imagine this is a big project and there is a lot of work to be done, both on the Kibana side and

Logstash Elasticsearch Reindexing Question

I'm new to logstash and trying to make a proof of concept to reindex an already created Index (This index is not logs but standard files like .doc/pdf/emails etc). It will be greatly appreciated if you answer my question below and share your thoughts. 1) Can we reindex a non-log index by using

Shard query cache without specifying search_type parameter

Hi all, I need to calculate aggregations for every search request since my UI requires it. For performance considerations I ended up splitting every search request to 2, the first to do the information retrieval itself and another to aggregate the results while using the query cache by specifyin

Re: How to delete older logs in ELK to give each application a certain disk quota?

The feature you described does not exist yet in elasticsearch. There is an open issue for such IIRC. For now, I'd use curator. My 2 cents. -- David ;-) Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > Le 15 févr. 2015 à 19:34, Gabriel Corrêa de Oliveira a > écrit : > > Dear All, > >

How to delete older logs in ELK to give each application a certain disk quota?

Dear All, I am trying to use the ELK stack in the following scenario: I have about ten applications that send their logs, through Logstash, to a single Elasticsearch cluster. Some of these applications naturally generate more logs than others, and, sometimes, one of them can go 'crazy', becaus

Re: kibana version 4 no way to import Dashboard

One other part is I am using the http://localhost:5601 URL On Sunday, February 15, 2015 at 9:45:59 AM UTC-5, Sam Degan wrote: > > Ok, have done a ton of searches to try and find this and nothing seems to > be there. > > Everything seems to point to the older versions of kibana not version 4. > >

kibana version 4 no way to import Dashboard

Ok, have done a ton of searches to try and find this and nothing seems to be there. Everything seems to point to the older versions of kibana not version 4. I want to import a dashboard - using elasticsearch, kibana - If I go to settings I do not see anything that allows me to import a json f

Re: disabling dynamic date time formatting in python api or globally

i know i just want to disable automatic date conversion as my date strings are not in valid format(they can be empty as well) On Sunday, February 15, 2015 at 7:29:59 PM UTC+5:30, David Pilato wrote: > > Sounds like "--" is not a valid date. > > -- > David ;-) > Twitter : @dadoonet / @elasticsearc

Re: disabling dynamic date time formatting in python api or globally

Sounds like "--" is not a valid date. -- David ;-) Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > Le 15 févr. 2015 à 14:44, Shahid Qadri a écrit : > > guys getting this error > > raise HTTP_EXCEPTIONS.get(status_code, TransportError)(status_code, > error_message, additional_info)

disabling dynamic date time formatting in python api or globally

guys getting this error raise HTTP_EXCEPTIONS.get(status_code, TransportError)(status_code, error_message, additional_info) RequestError: TransportError(400, u'MapperParsingException[failed to parse [SOURCES.DATE_COMP]]; nested: MapperParsingException[failed to parse date field [--], tried

can someone explain term_vectors vs index_options v2

I am confused by when to use term_vector or index_options in the mapping definition. Google led me to: https://groups.google.com/forum/#!searchin/elasticsearch/term_vector$20vs$20index_options/elasticsearch/3OsHn5w3H-E/bgFRLcdgqGQJ But unfortunately - I don't understand those words when they are

Re: Elasticsearch + attachment plugin + Kibana + couchbase

thanks, sent you a pm On Thursday, February 12, 2015 at 5:24:33 PM UTC+2, Itamar Syn-Hershko wrote: > > Yes, that too :) > > Also if its a time based data, you will not be able to use kibana's date > filtering etc - because it lacks the @timestamp field. Basically, the XDCR > elasticsearch plu

doc_value overhead in disc space

Hi, I'm using elasticsearch 1.4.1, with extensive use of doc-values. Checking the stats, I have a feeling that there is a large overhead in disc space - compared to the raw indexed docs size. My questions are: 1. is there a way to tell how much disc space is used by the doc-values? I didn't

Re: eshadoop : why lowercasing field names ?

Hi, Actually the documentation is incorrect. The problem lies mainly with Hive who treat field names as case insensitive. In Pig the behavior was inconsistent, depending on the version and the script used. Hence why using the mapping feature is recommended. However, the doc needs to be correct

Is it possible to have a reusable type in elasticsearch?

I added the question to stackoverflow before I thought to post here: http://stackoverflow.com/questions/28524850/add-reusable-field-type-to-elasticsearch I'm looking for something similar to index templates, except I don't want to use dynamic fields. This possible? -- You received this m