Re: Very regular disconnect and recover - every 2 hours

You can try winding out the timeouts, see http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-discovery-zen.html#fault-detection On 31 March 2015 at 16:57, Neil Andrassy neil.andra...@thefilter.com wrote: It's probably something like that, but it only seems to be a problem

Re: mlockall

Hi, here is the info Distributor ID: Ubuntu Description:Ubuntu 14.04.1 LTS Release:14.04 Codename: trusty I have tried to set mlockall = true on two server . i am able to successfully do it on one but not able to do it on other server On Tue, Mar 31, 2015 at 11:56 AM, Mark

Re: What is the best configuration to run on linux VPS server without crashing?

Hi Mark, Our server have 7.5GB RAM, 4vCPUs (in Amazon EC2), our data about 50 thousand records (about 50MB) I think ElasticSearch (ES) server handle 10 connections at the same time (per second) I have assign 2GB RAM for HEAP_SIZE, our server will run normally for a while about 5 minutes and

Re: mlockall

Which linux? On 31 March 2015 at 16:16, Cheten Dev cheten@carwale.com wrote: Hi, I am on linux server with elasticsearch 1.5.0 On Tue, Mar 31, 2015 at 8:07 AM, Mark Walkom markwal...@gmail.com wrote: A bit more info would be useful. What version of ES, what OS? On 30 March 2015 at

Re: Elastic Search - How to delete old records and improve performance

Hi Mark, thanks for taking the time to reply 1- yes I believe all the data is in one index .. I had attached the index_status output ... and a screenshot for the head_plugin output 2- I am considering the upgrade to latest version .. but would that impose a risk (if any)? thanks again

Re: I want to load dashboard developed by another developer in kibana4 in my system using separate elasticsearch.It is possible in kibana 3 by using json file.is it possible in kibana 4?

This is currently not possible, but it's being worked on. See https://github.com/elastic/kibana/issues/2310, it might also have something to help in the meantime. On 31 March 2015 at 16:09, Priya G g.shanmugapriy...@gmail.com wrote: I want to load dashboard developed by another developer in

Security Suggestion In Elasticsearch

Hi, I am trying to deploy my ES server in Digital ocean. But Digital ocean had some hacker attack as i didn't add any security to ES, So they blocked my droplet and gave me warning. So i tried to find out how can i secure my ES server. I have found out that Shield , using proxy or some other

Re: Timestamps stored as seconds since Unix epoch

Though support may be coming - https://github.com/elastic/elasticsearch/issues/10072 On 30 March 2015 at 09:16, Jean Marc Saffroy j...@scality.com wrote: I thought I might have missed something, and it surprises me that the Unix epoch isn't supported! Well, I guess I'll have to work around

[ANN] Elasticsearch Twitter River plugin 2.5.0 released

Heya, We are pleased to announce the release of the Elasticsearch Twitter River plugin, version 2.5.0. The Twitter river indexes the public twitter stream, aka the hose, and makes it searchable. https://github.com/elastic/elasticsearch-river-twitter/ Release Notes -

Re: Security Suggestion In Elasticsearch

The rule is not new. Do not expose Elasticsearch to the public internet, just like Postgresql and Gearman. Jörg On Tue, Mar 31, 2015 at 8:45 AM, Shohedul Hasan sha...@qianalysis.com wrote: Hi, I am trying to deploy my ES server in Digital ocean. But Digital ocean had some hacker attack as i

Re: Security Suggestion In Elasticsearch

Shield would be the best. But you can also use iptables and nginx to secure and proxy things. On 31 March 2015 at 17:45, Shohedul Hasan sha...@qianalysis.com wrote: Hi, I am trying to deploy my ES server in Digital ocean. But Digital ocean had some hacker attack as i didn't add any security

[ANN] Elasticsearch Wikipedia River plugin 2.5.0 released

Heya, We are pleased to announce the release of the Elasticsearch Wikipedia River plugin, version 2.5.0. The Wikipedia River plugin allows index wikipedia. https://github.com/elastic/elasticsearch-river-wikipedia/ Release Notes - elasticsearch-river-wikipedia - Version 2.5.0 Update: *

Re: A cautionary tale about index autocreation

The problem with action.auto_create_index: false is that it disables index templates... So it is basically useless if you are using one -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails

Re: Very regular disconnect and recover - every 2 hours

Both clusters have the following settings so, if that's related, I think there must be another contributing factor... discovery.zen.fd.ping_interval : 1s, discovery.zen.fd.ping_timeout : 60s, discovery.zen.fd.ping_retries : 3, On 31 March 2015 at 07:32, Mark Walkom markwal...@gmail.com wrote:

Default template mapping at type level

Hallo, I have 6 types that I would like to keep in the same time-series index. 5 of these types are very similar and so I have defined the common fields in the _default_ mapping. The 6th is very different and shares very few fields with the other types: but since the _default_ mapping applies

How to merge elasticsearch 1.4.2 data with already running elasticsearch 1.4.4?

I am having some data in elasticsearch 1.4.2.i am also having some data in elasticsearch 1.4.4. Now i want to take data stored in elasticsearch 1.4.2 and i have to store that data in elasticsearch1.4.4. Can anyone tell me how to do? -- You received this message because you are subscribed to

Re: Scoring and explain

FYI ExplainableSearchScript interface is back in 1.4 branch. Le mercredi 19 novembre 2014 04:04:05 UTC+1, Nick Tarleton a écrit : On Thursday, November 6, 2014 3:30:26 AM UTC-8, Guillaume H wrote: Hi guys, I'm developing a scoring plugin and I need to display some information about

IFrame options

Hi, I'm displaying Kibana graphs in another system using the generated iframe, but I need to get rid of both the Kibana loading screen as well as error messages when no data is found. I only need to display graphs. Are there any options for doing that? Thanks, Ole Hedegaard -- You received

[ANN] Elasticsearch RabbitMQ River plugin 2.5.0 released

Heya, We are pleased to announce the release of the Elasticsearch RabbitMQ River plugin, version 2.5.0. The RabbitMQ River plugin allows index bulk format messages into elasticsearch.. https://github.com/elastic/elasticsearch-river-rabbitmq/ Release Notes - elasticsearch-river-rabbitmq -

Problems upgrading to 1.5.0

We just encountered some mysterious problems when upgrading from 1.1.1 to 1.5.0. The cluster consists of three machines, two data nodes and one master-only node. It hosts 86 indices which each has one replica. I stopped writes, did a snapshot and stopped the entire cluster before I upgraded

Re: ES/Lucene eating up entire memory!

Thanks Uwe. As I mentioned earlier, I did guess that VIRT doesn't indicate RAM consumption. What I am concerned about is the 3rd row which shows memory and indicates that out of the total 50g, 43g is in use. Once this number crosses 45g, my other databases start behaving badly. Problem is, even

Script to configure Kibana 4 index pattern for logstash, import saved searches, visualizations dashboards

Hi, Is there a script available to configure Kibana 4. I'm starting Elasticsearch, Logstash Kibana 4 inside Docker. I'm looking for a way to automatically configure: - the Kibana 4 index pattern for logstash - import Kibana saved searches, visualizations dashboards -- You

Re: EC2 client node cluster discovery

Seems like not many other people have this problem... In my case this was only a problem using unicast (which cloud-aws does), so I changed the cloud plugin to work for my case. The file that needed to be changed was AwsEc2UnicastHostsProvider.java. Look for UnicastZenPing.LIMIT_PORTS_COUNT.

Re: How getting document match rate?

Hi guys, The Explain api returns coord in bool query. This could work. On Saturday, March 28, 2015 at 6:33:53 PM UTC+2, Terra Sacer wrote: Hello everyone, For example my data [ { id : 1, type : article, title : About the Java Technology }, { id : 2, type : article, title :

[ANN] Elasticsearch CouchDB River plugin 2.5.0 released

Heya, We are pleased to announce the release of the Elasticsearch CouchDB River plugin, version 2.5.0. The CouchDB River plugin allows to hook into couchdb _changes feed and automatically index it into elasticsearch.. https://github.com/elastic/elasticsearch-river-couchdb/ Release Notes -

Re: Health Status Red(Accidentally started another node)

Thank you for the solution ! A quick clarification though. So I am assuming that on restarting the second node, the cluster will automatically come back to green, right? Also, is it possible that instead of restarting the second node(which was accidental and not really needed), I can force

Re: Reindex from the existing index

If you Search on the web site for reindex, you will find this: http://www.elastic.co/guide/en/elasticsearch/guide/current/reindex.html David Le 1 avr. 2015 à 06:15, Vladi Feigin vla...@liveperson.com a écrit : Hi, Is there a way to build a new index from the existing index ? Thank you,

Re: Nested object under path [messages] is not of nested type

Hi, You put mapping of message type while you query on thread type. Since thread type isn't defined, you get the error. Masaru On April 1, 2015 at 01:06:51, Daniel Buckle (danielbuckl...@gmail.com) wrote: I am having a lot of issues with the nested type mapping in Elasticsearch, I have ran

Re: Health Status Red(Accidentally started another node)

Restart the other node, let it join and then bring the cluster back to green. Then either 1) add replicas so each node has the data and remove the old node and set replicas to 0, or 2) disable allocation and move the shards off the other node to the main one manually. On 1 April 2015 at 11:08,

Re: Elastic Search - How to delete old records and improve performance

You could do a few things, from my perspective. (Hope this hits some idea's you like) - upgrade, never hurts :) as long as you read the release notes to make sure nothing your depending on. - Add some filters to your existing queries to exclude unneeded data - you can create filter

How to achieve ELK High availability

Hi Exerts, Need your valuable suggestions here . I have ELK on a single windows instance and I want to make it high available . I mean if one machine goes down second will take up the whole load, like clustering. Can you suggest how I can achieve this. -- You received this message because

Reindex from the existing index

Hi, Is there a way to build a new index from the existing index ? Thank you, Vladi -- This message may contain confidential and/or privileged information. If you are not the addressee or authorized to receive this on behalf of the addressee you must not use, copy, disclose or take action based

Re: Health Status Red(Accidentally started another node)

It should, only way to know is to try. You can try to force allocation if that node has them there. On 1 April 2015 at 12:39, Shomo shoumitrados...@gmail.com wrote: Thank you for the solution ! A quick clarification though. So I am assuming that on restarting the second node, the cluster will

Re: ElasticSearch date format

I am speculating here that you should use the ISO 8601 format for dates since this is quite common and has a timezone which yours lacks (introducing ambiguity). The T is a common separator between the date and time parts. On 31 March 2015 at 14:26, phani.nadimi...@goktree.com wrote: Hi All,

Re: is _id of document affects on scoring?

Thanks for your response. it is completely confusing for me why elasticsearch was configured for large data by default instead of development and developers should find these tiny tips under thousands of documentations. On Tuesday, 31 March 2015 17:54:00 UTC+4:30, Adrien Grand wrote: The

Re: Try to make es-hadoop run

Debugging Hadoop is ... tricky. Most likely the parameter you are passing (es.version) is incorrect - there's no such Elasticsearch version. 2.1.0.Beta3 is the connector, es-hadoop version. es.version indicates the version of Elasticsearch itself. My advice is to first start with basic steps

Problems with combination of attributes in Elasticsearch

Hi all, When designing an application I would like to know whether ElasticSearch is a suitable tool for implementing it (and how to do it). Any advice will be much appreciated! My application needs to store (many) documents, each document being represented as a sequence of words. I want

Re: Elastic Search - How to delete old records and improve performance

mkBig ... thanks for the suggestion ... but how do I exclude the things I dont need ? cheers! -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to

Re: is _id of document affects on scoring?

On Tue, Mar 31, 2015 at 4:13 PM, Mohammad.R esmailzadeh esmailza...@gmail.com wrote: it is completely confusing for me why elasticsearch was configured for large data by default instead of development and developers should find these tiny tips under thousands of documentations. I'm sorry

parent-child document relation problem

Hi all, could you help me with little problem regarding parent-child documents relation? I use elasticsearch v1.4.4. Considering JSON, I have objects, each of them contains an array of sub-objects. Sub-objects contain some text fields. I need to maintain full-text-search on these objects and

A problem: ClusterBlockException : blocked by: [SERVICE_UNAVAILABLE/1/state not recovered / initialized];

This error occurred when i restart cluster. Debug log: [2015-04-01 11:25:09,743][DEBUG][node ] [es_node_4_1] using home [/home/nat/elasticsearch-1.1.2], config [/home/nat/elasticsearch-1.1.2/config], data [[/home/nat/esdata.d, /natlog1/nat/esdata.d, /natlog4/nat/esdata.d,

Re: How to achieve ELK High availability

Which part do you want to make HA? ES is pretty simple to do, the rest is dependent on a few things. On 1 April 2015 at 14:06, vikas gopal vikas.ha...@gmail.com wrote: Hi Exerts, Need your valuable suggestions here . I have ELK on a single windows instance and I want to make it high

Does english analyzer prevent fields from highlighting?

Hi all, could you help me with little problem regarding language-specific analyzers and highliting in elasticsearch? I need search documents by a query string and highlight matched strings. Here is my mapping: { usr: { properties: { text0: { type: string,

Debug Rest Plugin

Hi I created a plugin, I'm able to create the zip, install it in my cluster, etc. My problem comes if I want to debug from Eclipse having access to the current data in my cluster. To do that the easiest way should be make the debug node part of my cluster. When I run the java application in

Re: elasticsearch high cpu usage every hourly

From what I can see in your graphs I noticed two things. You seem to have a spike in search requests at that time, a spike in http traffic, and a cache eviction right at the beginning of it. Are you certain you don't have an external user with a cron job that runs at the top of the hour?

Re: Debug Rest Plugin

When I want to debug a plugin, I often run the org.elasticsearch.bootstrap.Bootstrap class. And I pass something like -Des.foreground=true -Des.http.cors.enabled=true -Dhttp.cors.allow-origin=/.*/ -Dhttp.cors.allow-credentials=true -Des.path.conf=/path/to/your/config/dir/

Nested object under path [messages] is not of nested type

I am having a lot of issues with the nested type mapping in Elasticsearch, I have ran this to create my index: curl -XPOST 'http://localhost:9200/thread_and_messages' -d '{mappings : { message: { properties: { messages: { type: nested,

[ANN] Elasticsearch Mapper Attachment plugin 2.5.0 released

Heya, We are pleased to announce the release of the Elasticsearch Mapper Attachment plugin, version 2.5.0. The mapper attachments plugin adds the attachment type to Elasticsearch using Apache Tika.. https://github.com/elastic/elasticsearch-mapper-attachments/ Release Notes -

Re: ES/Lucene eating up entire memory!

You need to read up a bit on how memory is allocated in Linux. In an ElasticSearch or Database server, this seems to be both, you want that free column to be 0. All available free memory should be used to cache files. In your snapshot you have 35GB of file cache listed under the cached

Re: ElasticSearch date format

http://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-date-format.html The T is a standard symbol, not exclusive to any one software project. ElasticSearch uses Joda to parse dates, and it looks like the default accepts ISO 8601 which I would expect. On 31 March 2015 at 15:32,

Re: Wildcard search on raw not_analyzed field

I need help determining how wildcard matching for non_analyzed fields works. I have a field `message` with this mapping defined: message : { type : string, norms : { enabled : false }, fields : { raw : { type : string, index : not_analyzed, doc_values : true, ignore_above : 256 } } } I've indexed

Is there any blog which shows new frameworks related to elasticsearch?

Hi all, Does elasticsearch itself have a blog which basically combines and shows extensions or tools made by developers from all over the world? Why I am asking is bacause we have also implemented an extension called Pes which helps in building inline query DSLs with less effort. Perhaps,

[ANN] JDBC plugin with feeder mode as an alternative to the deprecated Elasticsearch River API

Hi, if you use the JDBC river plugin and you are concerned about the deprecation of the river API, I wrote a step-by-step guide how to start the JDBC plugin in a feeder mode. The feeder mode is a standalone JVM which connects to an ES cluster using Java TransportClient under the hood. You can

Re: elastisticsearch_dsl python to create pivot tables

tested it. works as expected. Thanks again for your help Am Dienstag, 31. März 2015 00:04:39 UTC+2 schrieb Mike: Thanks Honza. You made my day (my night rather, it is midnight here in Brussels). I quickly tested the code and it gives the same results as the manually chained

Re: Wildcard search on raw not_analyzed field

Update, it seems that the problem is lowercase_expanded_terms defaults to true. Setting it to false in my query returned results for the first two queries. On Monday, January 26, 2015 at 5:04:44 PM UTC-6, BradVido wrote: I have a dynamic template that sets up not_analyzed raw mappings for all

Running Elasticsearch error

Hello, When I run ./bin/elasticsearch like it says in the http://logstash.net/docs/1.4.2/tutorials/getting-started-with-logstash tutorial I get the following error? failed to connect to master [[Annie Ghazikhanian] [127.0.0.1:9300]], retrying... Caused by: java.net.ConnectException:

Re: what are the research papers that ES relies on?

Thats what I was looking for (murmur3) I really wondered what they used and I was going to ask about murmur3 as weel. But as I see things, are going pretty awesome. Thanks 31 Mart 2015 Salı 00:42:45 UTC+3 tarihinde Aaron Mefford yazdı: I understand that if you do not have sufficient storage

Storing fields in _source and nested values in a strict mapping

Hi, I seem to be having an issue with getting the _source value to populate - I'm not sure why as I thought this happened on default - but it keeps coming back empty. I also got the following error while trying to save the second value in a nested array (end_call_info). Do I need an _index

Can I turn off kibana search highlight?

The search highlight is somehow very useful. But is there a setting or a configuration that allow me to turn the search highlight off? -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from

Re: what are the research papers that ES relies on?

Murmur3 appears to be coming in 2.0. Currently it looks like it is using DJB2. On Tue, Mar 31, 2015 at 11:53 AM, MrBu metin.aky...@gmail.com wrote: Thats what I was looking for (murmur3) I really wondered what they used and I was going to ask about murmur3 as weel. But as I see things, are

Re: Debug Rest Plugin

Thank you David! It worked :-) El martes, 31 de marzo de 2015, 12:13:23 (UTC-4), David Pilato escribió: When I want to debug a plugin, I often run the org.elasticsearch.bootstrap.Bootstrap class. And I pass something like -Des.foreground=true -Des.http.cors.enabled=true

Re: unable to get script_fields to work

I'm having the same problem. Were you able to resolve this? Thanks! On Thursday, February 26, 2015 at 10:38:26 AM UTC-8, Gerald DeConto wrote: fyi: my config files (elasticsearch.yml and logging.yml) are in /etc/elasticsearch. the script_fields documentation indicated that I put the

Re: Is there any blog which shows new frameworks related to elasticsearch?

We don't have a blog but we do include things in our weekly summary, so I will pass this onto our community team. Thanks for sharing it though, and good luck! On 1 April 2015 at 06:21, Hakan Özler ozler.ha...@gmail.com wrote: Hi all, Does elasticsearch itself have a blog which basically

Re: Groovy ElsticSearch script

I'm having a similar problem. Were you able to get this to work? Thanks! On Friday, February 13, 2015 at 4:18:06 AM UTC-8, Sergio Soto Núñez wrote: Hi guys, I'm making a simple script to add information to documents. My first aproximation to solve my problem is based in a local script. I

Re: ElasticSearch date format

HI James, Thanks for quick reply.. so my understanding is while inserting in to elasticsearch we need to use T as separator other wise it won't insert in to date field from java API because it expecting standard format of ISO 8601 am I right? Thanks phani On Tuesday, March 31, 2015 at

Wrong load distribution

Hi all, I meet a load distribution problem today and I browsed the Internet to find out someone having the same problem but unfortunately, no one seems to have. Here is my cluster configuration: - 4 machines with 16 Go Ram (9600 Mo allocated to JVM) - 8 core CPUs - Special cluster config:

Re: Is there any blog which shows new frameworks related to elasticsearch?

Thank you :) On Tuesday, March 31, 2015 at 11:40:47 PM UTC+3, Mark Walkom wrote: We don't have a blog but we do include things in our weekly summary, so I will pass this onto our community team. Thanks for sharing it though, and good luck! On 1 April 2015 at 06:21, Hakan Özler

is _id of document affects on scoring?

I add two same documents the only different thing is _id of documents (I restart scenario for each of them and I do not add them sequentially. to be sure my test is correct) one of them changes order of result of this query and one of them does not: GET index_for_test/business/_search {

Re: is _id of document affects on scoring?

The difference is probably due to the fact that your documents ended on different shards where doc frequencies are different. You should be able to make it a bit better by using distributed frequencies. https://www.elastic.co/blog/understanding-query-then-fetch-vs-dfs-query-then-fetch On Tue, Mar

Re: How to merge elasticsearch 1.4.2 data with already running elasticsearch 1.4.4?

One option would be to take snapshots of your data in your 1.4.2 cluster and restore it in your 1.4.4 cluster. http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-snapshots.html On Tue, Mar 31, 2015 at 1:33 PM, Priya G g.shanmugapriy...@gmail.com wrote: I am having some data

Re: Kibana showing wrong sales per month

Hi Schmorgs , Thanks for reply. using daily netsales also I tested, those also going wrong and I checked my query it is fine and per year I am checking my monthly netsales and one more point here the date field values we are inserting as CST time.is it be a problem when we mapped to

Re: Elastic Search - How to delete old records and improve performance

Can you create a new index and exclude what you dont need. we did this recently because of some other mapping reason. take a snapshot (with newwer version) before doing so. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from

Re: Question on upsert

NVM, I found the problem. On Tue, Mar 31, 2015 at 4:10 PM, Bae, Jae Hyeon metac...@gmail.com wrote: Hi I am using upsert bulk api with ES 1.4.1. The problem is, I get the following errors so frequently. Error messages is error - ElasticsearchIllegalArgumentException[failed to execute

Re: mlockall

Can you show us the relevant config sections from the nodes that don't have mlockall set? On 31 March 2015 at 17:40, Cheten Dev cheten@carwale.com wrote: Hi, here is the info Distributor ID: Ubuntu Description:Ubuntu 14.04.1 LTS Release:14.04 Codename: trusty I

Health Status Red(Accidentally started another node)

Today, I accidentally started another node on a different machine on the same LAN which resulted in my first ES instance adding this new node to the cluster. I killed this second node with SIGKILL and restarted my original ES instance. However, on checking the health status of the various

Re: Does english analyzer prevent fields from highlighting?

Using inline highlighters doesn't help highlighting. No. For the most part you should stay away from inline analyzers and use a mapping instead. On Tue, Mar 31, 2015 at 12:02 PM, Viacheslav Shalamov sslavian...@gmail.com wrote: Hi all, could you help me with little problem regarding

Re: Wrong load distribution

Do you have your shards equally distributed over the 4 nodes? Or do you use the default of 5 shards? Jörg On Tue, Mar 31, 2015 at 5:28 PM, Loïc Wenkin loic.wen...@gmail.com wrote: Hi all, I meet a load distribution problem today and I browsed the Internet to find out someone having the same

Question on upsert

Hi I am using upsert bulk api with ES 1.4.1. The problem is, I get the following errors so frequently. Error messages is error - ElasticsearchIllegalArgumentException[failed to execute script]; nested: GroovyScriptExecutionException[NullPointerException[Cannot execute null+null]]; and the