Installation Guide

I'm newbie to Elasticsearch, Am little bit confused on the Workflow of ELK stack. Installed ELK stack. LOG-FORWARDER not sending data to Elasticsearch server. Kibana not loading. I think, Once log-forwarder workers kibana will show the graphs. Kindly any one help me with the complete guide.

Re: Kibana: Mark warnings as "solved"

I know how to use a programming language and I could do start a own project. But I would like to avoid it, since it leads to "plubming". I guess other people have same use case, and I would like to use (and improve) an existing project. But I have not found any up to now. How do other ELK users

Re: Monitor incoming data rates

On Thursday, April 09, 2015 at 05:49 CEST, Bernie Carolan wrote: > I have alerts configured on my Logstash - Elasticsearch setup which > perform regular queries to see what state the cluster is in etc. > Recently I had a situation where Logstash was running OK and ES > cluster was in Green s

Re: Monitor incoming data rates

Marvel monitors these sorts of things out of the box. But maybe _cluster/stats and keep an eye on docs.count? What is monitoring things now? On 9 April 2015 at 13:49, Bernie Carolan wrote: > I have alerts configured on my Logstash - Elasticsearch setup which > perform regular queries to see wha

Monitor incoming data rates

I have alerts configured on my Logstash - Elasticsearch setup which perform regular queries to see what state the cluster is in etc. Recently I had a situation where Logstash was running OK and ES cluster was in Green state, but there was no data going into ES. Is there a way to monitor this, e.

elastic search query syntax vs lucene query syntax

Can some one clarify /confirm this . End user uses lucene query syntax .' For proximity searches lucene has ~ operator where elastic search expects provide DSL in span queries Example : lucene query "United States"~ 30(which is the user query) will be converted to span query by the elastic se

Re: Query Syntax for proximity searches

Actually I wrote the below queries , let me know if they are correct 1) "United States" ~ 3 - to get documents with United States with in 30 words 2)"United States" NOT "United States" ~ 30 - to get documents which has united states not with in 30 words 3)war AND peace ~ (novel OR book) -

Reverting a node exclusion setting?

Hello, I followed the instructions at the bottom of http://www.elastic.co/guide/en/elasticsearch/reference/1.5/modules-cluster.html to exclude an instance in our test ES cluster: curl -v -XPUT http://dev:9200/_cluster/settings -H "Content-Type: application/json" -d '{ "transient":{ "clu

Kibana - Error including scripted fields in aggregation

Hello, I am trying to add a manually created scripted field using the dropdown box withing Aggregation: Any pointers on how to resolve this error would be very helpful. Scriped field: Name: @hourOfDay Script: Integer.parseInt(new Date(doc["@timestamp"}.value).format("H")) Below is the error:

Kibana: scripts in json input not working

Hello, I am getting error while adding scripts in json input while doing aggregation(date histogram). Am I missing any configuration options which I need to do before using this? Any help would be appreciated! Below is the script: { "script":"log(_value)" } Request to Elasticsearch failed: {"e

Re: use filter specified in json through java API

Or you could say that I need convert request specified in json format through java API. On Wednesday, April 8, 2015 at 3:16:37 PM UTC-7, Prateek Asthana wrote: > > > I am having requirement where in I need to use filter specified in json > through elasticsearch Java API. > > for example below e

use filter specified in json through java API

I am having requirement where in I need to use filter specified in json through elasticsearch Java API. for example below elasticsearch json filter needs to be used through Java API: "filter" : { "bool" : { "should" : [ { "term" : {"productID" : "KDKE-

Re: Fixed schema

Take a look at http://www.elastic.co/guide/en/elasticsearch/reference/current/indices-put-mapping.html, or do a search for mapping in the docs, there is lots of info available. On 9 April 2015 at 05:30, bvnrwork wrote: > Can some one throw some ideas /explain me about fixed schema ? I am using >

Re: Elastic search goes down periodically

Do you have dynamic scripting enabled? Is your instance accessible to the internet? It looks like someone is running scripts on your system, whether it's something your users are doing or someone else, you need to determine this. On 9 April 2015 at 06:19, Florian Wilk wrote: > Hi, > > i have th

Re: How to get aggregations working in Elasticsearch Spark adapter ?

Facets are deprecated and will be removed and as such, there is no support or plans to add support for them in the near future. As for when aggregations will land in 2.1, the near future - I don't want to give estimates (only to miss them) but let's just say it's very high priority. Cheers,

Re: elasticsearch-hadoop - getting specified fields from elasticsearch as an input to a mapreduce job.

I'm having an issue very similar to this; I'm not sure exactly what you did to get the array contents. I've made a new post here: https://groups.google.com/forum/#!topic/elasticsearch/MpOqKthgqtA -- Paul Chua Data Scientist 317-979-5643 [image: cid:02526A0B-9444-47C7-A3EC-12B05A02CB50] *We help

Re: How to get aggregations working in Elasticsearch Spark adapter ?

Anyone having an answer for this ? Thanks in advance. Il giorno mercoledì 1 aprile 2015 17:58:19 UTC+2, michele crudele ha scritto: > > Thanks, > > when is the 2.1 release coming? > > Another question, which I think is related to this one btw... I was able > to run this piece of code using facet

Getting contents of org.elasticsearch.hadoop.mr.WritableArrayWritable in scala

I'm having an issue very similar to this one, but I'm not sure exactly what they did to get the array contents. I can't apply the toStrings() function to the arraywritable because it's in a writable. http://grokbase.com/t/gg/elasticsearch/14c2sb14rk/hadoop-getting-specified-fields-from-elasticse

Re: Elastic search goes down periodically

Hi, i have the same problem. My ES worked fine for a long time and since a few weeks i periodically have to restart my ES. The logs are full of messages like this: [2015-04-08 20:16:47,281][DEBUG][action.search.type ] [Norman Osborn] [22485] Failed to execute fetch phase org.elasticsearc

Re: Create mapping for nested json

No bug in ES, it is a missing feature in Kibana. https://github.com/elastic/kibana/issues/1084 I am using ES 1.5.0-1 (rpm for RHEL). On Tue, Apr 7, 2015 at 12:32 AM, Krishna Raj wrote: > What version of ES are you trying on ? I faced this issue due to a bug in > lower versions. But I am suc

Completion suggester - Finite Strings Error

I get the following error when I try to insert a doc *TransportError(500, u'IllegalArgumentException[TokenStream expanded to 44800 finite strings. Only <= 256 finite strings are supported]')* *Index: This gets created successfully* index_body = { "settings": {

Fixed schema

Can some one throw some ideas /explain me about fixed schema ? I am using default ES behavior that is dynamic schema -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an ema

Re: URI search not returning results when &fields used

Running into an issue with URI searches not returning expected results after upgrading from 0.90.9 to 1.4.1 The issue is that subfields are not returned when specified with "&fields". I have confirmed that the data is there by doing the same search without the &fields specifications and get th

Re: URI search not returning results when &fields used

Interesting thing to note. ... This issue is only for fields of Geo Point Type (geo_point) http://www.elastic.co/guide/en/elasticsearch/reference/1.4/mapping-geo-point-type.html Tests of other fields work. On Wednesday, April 8, 2015 at 9:56:54 AM UTC-6, Tony Nesavich wrote: > > Running into an

Filter only returns less results results than same filter combined with a must match query

I am struggling to understand why a filter only returns older results than that same filter combined with a must match query. More specifically, a prefix filter on a field which contains a link will return items which are two days old. However, that same filter added with a must match query wil

URI search not returning results when &fields used

Running into an issue with URI searches not returning expected results after upgrading from 0.90.9 to 1.4.1 The issue is that subfields are not returned when specified with "&fields". I have confirmed that the data is there by doing the same search without the &fields specifications and get th

Re: SynonymTokenFilterFactory class passes indexSettings to TokenizerFactory

Hi Yosuke, Thanks for reporting. I think that is bug, it should work 1.5. I will open Issue and create PR, soon. 2015-03-31 14:55 GMT+09:00 : > Hi. > I have a question about behavior of SynonymTokenFilterFactory class. > > I expect that "settings" parameter of TokenizerFactory is tokenizer's > s

snapshot api: Failed to derive xcontent from (offset=0, length=0)

Hi, unfortunately during the easter holidays the backup space of our ES cluster was running out of diskspace. We added some diskspace to the backup location, but we are getting this error when ES is trying to access the Repo: curl -XGET 'localhost:9200/_snapshot/backup/_all?pretty' { "erro

Re: java api question

Elasticsearch uses HPPC. Please read about it at http://labs.carrotsearch.com/hppc.html In order to minimize footprint on memory (autoboxing, primitive types) and to increase performance with less garbage collection, HPPC collections do not implement Java Collection API, but their own HPPC API.

java api question

Hi, Just started doing some development w/ the Java API and one thing I immediately noticed are things like this http://javadoc.kyubu.de/elasticsearch/v1.4.2/org/elasticsearch/common/collect/ImmutableOpenMap.html Why does something like ImmutableOpenMap not implement "Map"? -- You received th

ElasticSearch and Kibana: compare value with aggregated value

I'm wondering how to accomplish this requirement. I have to compare value data with the average over the selected period or over another period. I've collected millions of records in an index. These records contains the sellout amount day by day for different vendors, products, sectors and pr

Re: river jdbc plugin install for windows-not working

Please note, JDBC plugin is not only a river any more, it can also be used as a standalone tool like Logstash. Jörg On Wed, Apr 8, 2015 at 10:58 AM, James Green wrote: > As discussed elsewhere please avoid Rivers as they are deprecated for > removal. > > On 6 April 2015 at 13:36, Sanu Vimal wr

Big free space dis-balance

Hi, I have 9 node cluster. I notice that the free space is greatly dis-balanced. On node1 i have only 90GB left, while on other nodes I still have around 180GB free. I am pretty sure that no new shards will be allocated as the node is above the watermark. I think this started when I upgraded

Scripted metric aggregation over nested docs

I'm trying to do scripted metric aggregation using Groovy with Elasticsearch 1.4.4 I've a document model where each parent document can have 0-n nested documents and I need

Re: Query Syntax for proximity searches

Thank you , can you clarify this , do we need to use not operator for getting "united and states not with in 30 words " I tested it works to get united and states with in 30 words but not for" united and states not with in 30 words" united AND states NOT WITHIN 30 WORDS On Wednesday, 8 April

Re: Master keeps forgeting nodes

Both _cat/indices and _cat/shards appear to be working during the cluster failure. Em terça-feira, 7 de abril de 2015 14:05:02 UTC+1, João Costa escreveu: > > All machines are on the same region, the AZ is different though. > > When you say "check the _cat outputs", you mean making a call to > _

Re: Nodes randomly disconnected from the ES cluster.

Link below seems like a good direction to solve the problem https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1317811 [image: photo] *Tomer Levy* CEO, Co-Founder, Logz.io p:+972-544235023 | e:to...@logz.io | w:on.logz.io/1C2UlMi | a: +1-617-314-3318

ElasticSearch analyzer for short strings

I am creating an ES application where it should be able to index and search file content (pdf, word, txt, etc) and file names where it is possible for any file to be indexed. For the content I use the compact language detector to detect the language of the content and assign it to its corresp

Re: Kibana: Mark warnings as "solved"

Couldn't you update the document with a flag on a field? On 8 April 2015 at 09:43, Thomas Güttler wrote: > We are evaluating if ELK is the right tool for our logs and event messages. > > We need a way to mark warnings as "done". All warnings of this type should > be invisible in the future. > >

Re: river jdbc plugin install for windows-not working

As discussed elsewhere please avoid Rivers as they are deprecated for removal. On 6 April 2015 at 13:36, Sanu Vimal wrote: > Hi Jorg, > > I have found what was the issue. I was just trying the POC on MySQL but > the real requirement is in oracle11g. I have installed through plugin.bat > after se

Kibana: Mark warnings as "solved"

We are evaluating if ELK is the right tool for our logs and event messages. We need a way to mark warnings as "done". All warnings of this type should be invisible in the future. Use case: There was a bug in our code and the dev team has created a fix. Continuous Integration is running, and so

Re: timestamp

Hello, I solved my problem, thanks. actually the rest command should be: curl -XPUT 'http://localhost:9200/ubilogs-mbr/_mapping/logs' -d '{ "logs" : { "_timestamp" : { "enabled" : true, "store" : true, "format": "-MM-dd HH:mm:ss.SSS" } } }' with "store" a

Re: [ANN] Deprecating Rivers

I agree James. That's a nice suggestion (warn when running deprecated code). BTW we have a lot of work in the code such as mark some Java classes as deprecated, modify docs... Things will come. IMO it's better to announce the deprecation sooner than later. That's the purpose of this blog post.

Re: [ANN] Deprecating Rivers

Also note that the current documentation lists a load of river plugins without stating that they are officially deprecated as an idea: http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-plugins.html Rather concerned that people are still adopting rivers due to the apparent con

Re: 2 Errors with Elasticsearch

On Wednesday, April 08, 2015 at 09:13 CEST, Magnus Bäck wrote: > On Monday, April 06, 2015 at 17:05 CEST, > kelnrluierhfeulne wrote: [...] > > - Upgrade Required Your version of Elasticsearch is too old. Kibana > > requires Elasticsearch 0.90.9 or above. > > - Error Could not reach h

Re: 2 Errors with Elasticsearch

On Monday, April 06, 2015 at 17:05 CEST, kelnrluierhfeulne wrote: > When I open Kibana by searching for my IP in my browser, I get the > following 2 errors. Would anyone happen to have any advice on how to > fix these errors? I already updated Elasticsearch to its latest version > (elasticse

Suggestions with special characters

If I trying to get suggestions for word with special characters using this query POST _search { "suggest":{ "my-suggest-1" : { "text" : "täta", "term" : { "field" : "fieldName" } } } } I get response for two words: original and with r

Elastic search double words in suggestions

If I trying to get suggestions for phrase using this query POST _search { "suggest":{ "my-suggest-1" : { "text" : "täta tack", "phrase" : { "field" : "fieldName" } } } } I get response with double words in suggestions "suggest": {