Re: ElasticsearchIllegalStateException: Maximum number of bytes allocated exceeded

Hey, do you have a full stack trace in one of your log files? Also, are you creating lots of new connections all the time and then stop them abruptly (just a wild guess here)? --Alex On Mon, Jun 16, 2014 at 11:18 PM, Sriram Kannan wrote: > I am getting this error constantly and the 5 nodes i

Re: Creating a browse interface from ES

joergpra...@gmail.com schreef op ma 16-06-2014 om 13:12 [+0200]: > This is how I implement "register search" This is interesting. It could work for me. Though, I'm not sure I totally understand it. To find, say "Smith", I'd search for it, get its index, and then use the from/size stuff to bring

Boost field does not work..

Hi Guys, I have two types in a index, one is used for indexing topics and another one is used for indexing comments. Here is the sample mapping: *Topic*: curl -XPUT http://localhost:9200/bbs/topic/_mapping -d' { "topic": { "_timestamp": { "enabled": true },

Boost field does not work..

Hi Guys, I have two types in a index, one is used for indexing topics and another one is used for indexing comments. Here is the sample mapping: *Topic*: curl -XPUT http://localhost:9200/bbs/topic/_mapping -d' { "topic": { "_timestamp": { "enabled": true },

Kibana chart data view understanding

I have a problem trying to visualise the data below in Kibana. Each document describes a test run audit entry with passing, failing and pending tests along with a timestamp, project identifier and host name. The curls below setup four documents and they are correctly returned if I do http://local

Re: Tribe node data compliance across regions

It's best explained in the docs http://www.elasticsearch.org/guide/en/elasticsearch/reference/1.x/modules-tribe.html Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: ma...@campaignmonitor.com web: www.campaignmonitor.com On 17 June 2014 05:24, Kyle House wrote: > We are st

Re: IllegalArgumentException[No type mapped for [43]], version 1.2.1

Thank you for the suggestions. A couple clarifications and additional information: - We do use bulk indexing, as you inferred, using the Java TransportClient directly from our Java application. - This index, however, does not use dynamic mapping. I included the mapping in my first post and we

Re: node failures

TTL does use a lot of resources as it constantly scans for expired docs. It'd be more efficient to switch to daily indexes and then drop them, though that might not fit your business requirements. You can try forcing an optimise on an index, http://www.elasticsearch.org/guide/en/elasticsearch/refe

ElasticsearchIllegalStateException: Maximum number of bytes allocated exceeded

I am getting this error constantly and the 5 nodes in the cluster goes down one by one. Is there a way to fix the issue? Could it be a corrupt data issue? [2014-06-16 13:13:34,422][WARN ][discovery.zen.ping.multicast] [nw-es-201] failed to receive confirmation on sent ping response to [[nw-e

Re: IllegalArgumentException[No type mapped for [43]], version 1.2.1

I guess you hit the following condition: - you insert data with bulk indexing - your index has dynamic mapping and already has huge field mappings - bulk requests span over many nodes / shards / replicas and introduce tons of new fields into the dynamic mapping - you do not wait for bulk respon

Re: Share a document across multiple indices

Thanks, I'll explore your suggestion. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the

Re: node failures

java version is 1.7.0_55. the servers have a 32GB heap, 96GB of memory, 12 logical cores, and 4 spinning disks. Currently we have about 450GB of data on each machine, average doc size is about 1.5KB. We create an index (4 shards, 1 replica) every N days. Right now we have 12 indices, meaning ab

Re: better places to store es.nodes and es.port in ES Hive integration?

Thanks for sharing - can you also give an example of the table initialization in init.hive vs myscript.hql? Cheers! On 6/16/14 11:19 PM, Jinyuan Zhou wrote: Just share a solution I learned hive side. hive cli has an -i option that takes a file of hive commands to initilize the session. so

Creating sub-objects and nested objects in Elasticsearch plugin

Hello. I'm creating an Elasticsearch plugin that creates a new field type (similarly to how the elastic-mapper-attachments plugin does here

Re: Index template requires settings object even if its value is empty

Alex, I am running ES version 1.2.1. It seemed to work (no errors in the logs), but I did it as an on-disk template and not via PUT. And without the settings, it behaved as if it wasn't there. The question is now moot, because I actually need the following setting: "settings" : { "index.map

Re: IllegalArgumentException[No type mapped for [43]], version 1.2.1

Thanks for the reply. We've checked the log files on all the nodes - no errors or warnings. Disks were practically empty - it was a fresh cluster, fresh index. We have noticed that the problem occurs less frequently the more data we send to the cluster. Our latest theory is that it "corrects it

Re: 0.90.2 _update or _bulk update causing NullPointerException in logs and I start losing shards

Hi Boaz, We are using 0.90.2 and run into this issue. As i understand, one option is to upgrade to 0.90.3. If we continue using 0.90.2 and use (increase) retry_on_conflict, we will not see the problem? Please clarify. Thanks, Rohit On Wednesday, August 7, 2013 9:39:56 AM UTC-7,

Re: better places to store es.nodes and es.port in ES Hive integration?

Just share a solution I learned hive side. hive cli has an -i option that takes a file of hive commands to initilize the session. so I can put a list of set comand as well as add jar ... command in one file, say inithive then run the cli as this: hive -i init.hive -f myscript.hql. Note table

Percolating a document with nested mapping

Hello, I have noticed that a document with fields of nested type is not percolated correctly (all search IDs are returned). Once I remove those fields everything works just fine. Any ideas? -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To

Correlation Aggregation function

Hi, We are trying to replace our home-grown relational Data Model based analytics engine with elastic search. In the existing implementation we used CORR (correlation function) in couple of charts, couldn't find this in elastic search. Is this something that's in the pipeline for future releas

Drop down box panel

Has anybody created a drop down aggregation panel for kibana? I.E. a list of customers. We have too many customers for the provided panels. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving ema

Tribe node data compliance across regions

We are staging a rollout of multiple ES clusters across multiple regions, and we have a requirement to keep data contained inside the predefined regions. I wanted to check on how a tribe node would impact this requirement. My understanding is this node would be attached to multiple clusters for

ElasticSearch coté Front

Bonjour à tous, Je débute avec *ElasticSearch *et j'ai une interrogation sur la mise en place coté front, je m'explique. J'ai une application Java JEE d'administration qui permet de consulter les Logs qui ont été enregistrés sous ES par le biais de l'API Java de *ElasticSearch*. Jusque là pa

Re: Losing data after Elasticsearch restart

Hi Alexander, Thanks for your reply. We plan to upgrade in the long run, however we need to fix the data loss problem on 0.90.2 in the immediate term. Here is the stack trace - 10:09:37.783 PM [22:09:37,783][WARN ][indices.cluster ] [Storm] [b7a76aa06cfd4048987d

Re: does minimum_master_nodes include ones "self"?

Hey, yes, it is the number of nodes in a cluster, which are able to become a master node. If node A and node B are configured that way, both are counted, possibly including themselves. Makes sense? --Alex On Mon, Jun 16, 2014 at 5:11 PM, wrote: > running 1.2.1 > > If a cluster has 3 master

Re: Detecting end of Scroll...

Just to confirm: Is it correct that in order to automatically clean up the resources associated with a scroll, you must execute it until zero results are retrieved? What about the case when all of the hits are returned in the first result (ie num hits = total hits) -- You received this messag

Re: Cannot Increase Write TPS in Elasticsearch by adding more nodes

We used Jmeter for this test. On Friday, June 13, 2014 10:13:02 AM UTC-4, Greg Murnane wrote: > > I haven't seen it asked yet; what is feeding data into your elasticsearch? > Depending on what you're doing to get it there, a large document size could > easily bottleneck some feeding mechanisms.

spark and elasticsearch-hadoop-2.0.0

Hi! I am trying to setup a spark + elasticsearch solution. I am trying the simple example done by Barnaby Gray here [1], trying to adapt it to the elasticsearch-hadoop 2.0.0. I get some connectivity error that I am not able to overcome. I get the following error: 14/06/16 17:11:13 INFO HttpM

Re: date_histogram not returning key_as_string

Thank you Alexander, but I got it figured out. I was storing sessionStartTime as a Unix timestamp but did not provide any mapping, so the field type was set to be a double. Once I explicitly mapped sessionStartTime to be a date my query results started including the key_as_string field. Tim

Re: Detecting end of Scroll...

Oh, I see. Can we make any assumption about the scrollId not changing when we reach the end of the results? On Monday, 16 June 2014 15:58:20 UTC+1, Ralph Meijer wrote: > > On Mon, Jun 16, 2014 at 07:45:00AM -0700, mooky wrote: > > [..] > > That said, I do notice that when the end of the resu

does minimum_master_nodes include ones "self"?

running 1.2.1 If a cluster has 3 master eligible nodes, and one node dies leaving nodeA and nodeB and minimum_master_nodes = 2 Does nodeA when up, include itself when evaluating minimum_master_nodes? -- You received this message because you are subscribed to the Google Groups "elasticsearch"

Re: Detecting end of Scroll...

On Mon, Jun 16, 2014 at 07:45:00AM -0700, mooky wrote: > [..] > That said, I do notice that when the end of the results are reached, the > scrollId stops changing - and its also a lot shorter - and also it seems to > always be the same value (c2NhbjswOzE7dG90YWxfaGl0czoxOw==) That scroll id is the

Re: Percollation limits

It is something we might look into but there's no concrete plan for now. On the other hand metadata filtering (or eventually routing) allows to reduce the number of queries that need to be run, only problem is that users need to do it themselves. On Mon, Jun 16, 2014 at 4:53 PM, Ralph Meijer wro

Re: Percollation limits

On Mon, Jun 16, 2014 at 06:19:23AM -0700, Luca Cavanna wrote: > Hi Maciej, > what you describe doesn't sound insane, just make sure you use proper > filtering > as much as you can to limit the number of queries you execute when percolating > each document. > Also, with the percolator available sin

Detecting end of Scroll...

According to the documentation, you know when you have reached the end of the scroll results when you receive zero hits in the response. However, if your SearchType is SCAN, then you receive zero hits in the *first* result, The question is how to avoid this ambiguity. My code that handles the s

Re: Accessing Search Templates via Rest

Hey, no, this is not yet possible, but this will be added sooner or later as the search template API should behave like any other API. --Alex On Fri, Jun 13, 2014 at 9:51 AM, Sebastian Gräser wrote: > so i guess its not possible? > > Am Dienstag, 10. Juni 2014 16:58:31 UTC+2 schrieb Sebastia

Re: Elasticsearch field mapping, dynamic_templates

Hey, using your curl call you only set the mapping for existing indices, but not for newly created ones. You may want to consider using index templates for this specific field. --Alex On Mon, Jun 9, 2014 at 4:45 PM, sirkubax wrote: > As I did mention here > https://groups.google.com/forum/#!

Re: Find all the geoshapes that insersects with a given latitude/longitude

Hey, this looks like a problem, because you are using an _id object, I filed an issue here: https://github.com/elasticsearch/elasticsearch/issues/6514 You can just specify ids manually or make sure, that you _id field is empty for now. --Alex On Sun, Jun 8, 2014 at 9:42 PM, Vidal Chriqui wro

I need to query parent which has not child attach to it

Hi The below query search for all entries in a index which has child with the given stepUUID. I am interested to search for all entries which don't have child item populated yet. I was looking at using 'must_not" but not sure how to construct the query. Your help is very much appreciated. Reg

Re: Percollation limits

Hi Maciej, what you describe doesn't sound insane, just make sure you use proper filtering as much as you can to limit the number of queries you execute when percolating each document. Also, with the percolator available since 1.0 you can scale out just by adding more nodes and have the percolat

Re: Losing data after Elasticsearch restart

Hey, without stack traces it is pretty hard to see the actual problem, do you have them around (on one node this exception has happened, so it should have been logged into the elasticsearch logfile as well). Also, you should really upgrade if possible, as releases after 0.90.2 have seen many many

Re: IllegalArgumentException[No type mapped for [43]], version 1.2.1

Hey, it seems, as if writing into the translog fails at some stage (from a complete birds eye view). Can you check your logfiles, if you ran into some weird exceptions before that happens? Also, you did not run out of disk space at any time when this has happened? --Alex On Fri, Jun 6, 2014 at

Re: Bug in context suggester?

Hey, the problem is, that the mapping of the context variable 'network_id' is not a string. Thinking if we either should support this or reject it directly and only support string fields in the suggester. Added github issue https://github.com/elasticsearch/elasticsearch/issues/6512 --Alex On

Re: Help needed: CentOS ES-Logstash-Kibana running, but 0 docs, 0 indices

After renaming my cluster from 'elastic search-demo' to the default 'elasticsearch' the connection between ES and Logstash automatically was fixed after a restart of ES. Next question: Where do I have to change the name of the cluster, other than /etc/elasticsearch/elasticsearch.yml ? Op zater

Re: date_histogram not returning key_as_string

Hey, can you create a full gist to reproduce this? Should work out of the box (dont see anything wrong here), also on 1.1.0 --Alex On Thu, Jun 5, 2014 at 8:06 PM, Tim Heikell wrote: > Sorry for the noob question, but is there some setting I am missing? It's > not clear to me why I'm not gett

Re: Logash 1.4.1, Windows Server 2008, Tomcat 7.0.30, stuck with localhost_access_log.2014-06-16.txt

Done! Damn slashes! It should be as this, then glob search works fine: "/" -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroup

Re: Aggregation average value is not coming correct

Hey, you are setting a post filter, which means, that the aggregations will work without the range filter applied. You may want to use a filtered query and move the filter inside the filter part of that particular query. --Alex On Thu, Jun 5, 2014 at 12:38 PM, Subhadip Bagui wrote: > Hi, > >

Re: Geo Distance Facet - ElasticsearchParseException

Oops! That one was very silly overlook mistake. Thank you for pointing it out. Yes, I moved to aggregations with few failed tries on facets. On Monday, 16 June 2014 17:57:34 UTC+5:30, Alexander Reelsen wrote: > > Hey, > > your ranges specification is broken, you need to use an array of objects.

Re: Object property vs array index

Hey, that doesnt work like you want it to work. I am not sure, if I understood the intention of your query. Do you want to count on all first elements of all documents or do you just want to filter for that term? If its the first, maybe a script aggregations can help you, that just returns the fir

Re: Geo Distance Facet - ElasticsearchParseException

Hey, your ranges specification is broken, you need to use an array of objects. Also, please try to use aggregations if possible, as they are the successor of facets. --Alex On Thu, Jun 5, 2014 at 10:37 AM, Munjal Dhamecha wrote: > Hello All, > > I've been facing problem with geo_distance fac

Re: elasticsearch filter query with time range using java api

Hi Alex, Yes I tried that and it's working. Thanks :) -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this

What would be the best data structure for efficient search on this data?

I am using elasticsearch 1.2.0. I have around 3 million documents which can be of 2 types - 'products' or 'categories'. Currently I am usingscript filters to check whether category_name1 exis

Re: MapperParsingException on data that should not be parsed but caught by my plugin

Hey, first you should really upgrade ES, 0.90.3 is not the freshest one and there have been many improvements. Second, logging this as DEBUG makes a lot of sense, as otherwise you logfiles might explode, just because you are indexing a lot of bogus documents. You source of truth should not be the

Re: elasticsearch filter query with time range using java api

Hey, did you try something along the lines FilterBuilders.rangeFilter("@timestamp").from("now").to("now - 5d") - did that pose problems? make sure you are reading this paragraph about date range filters: http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/filter-caching.html --Ale

Re: Logash 1.4.1, Windows Server 2008, Tomcat 7.0.30, stuck with localhost_access_log.2014-06-16.txt

Have this in logstash log itself: {:timestamp=>"2014-06-16T15:01:50.817000+0300", :message=>"_discover_file_glob: c:\\apache\\logs\\localhost_access_log2.*.txt: glob is: []", :level=>:debug, :file=>"/logstash/vendor/bundle/jruby/1.9/gems/filewatch-0.5.1/lib/filewatch/watch.rb", :line=>"117"}

Re: Prefix search on integer field

Hey, not sure here, whats your expectation of the data being returned? If you search for "1", should "1", "10", "111" be returned (otherwise a range query would make more sense)? If so, it might make more sense to have an additional multi field as a not_analyzed string and use that one for these k

Re: Index template requires settings object even if its value is empty

Hey, which ES version are you using? Seems to work with the latest version. You can also use the index template API, so you do not have to fiddle with local files (and copy them when adding new nodes). PUT _template/automap { "template": "*", "mappings": { "_default_": { "numeric_de

Re: ElasticSearch Node.Client Options

Hey, client nodes do execute a reduce phase (merging all the results from a search in order to find out the top-n results), and if you do deep pagination this can take some memory. There is no data stored on a client node (as node.data property is always set to false). Do you have a stack trace o

Re: Configure connections using TransportClient

Hey, those configuration parameters still exist (there are more for ping and recovery). Wondering why exactly you want to change those? Do you have any specific problems going on? --Alex On Tue, Jun 3, 2014 at 7:47 AM, Srinath C wrote: > Hi, > I wasn't able to figure out how to configure

Re: Term Suggester Usage

Hi Alex, thanks for the clarification about the term suggester. That leaves either the phrase suggester or autocomplete suggester. The field I want to find suggestions for usually has a lot of words. So I'd prefer to complete only the current word, not the whole content of the field. Is this poss

Re: How to debug aggregation problems?

Hey, if you think this is a bug, please try to create a reproducible example and file a github issue. See http://www.elasticsearch.org/help If you are using the terms aggregation, you might want to read this one as well before filing a bug: https://github.com/elasticsearch/elasticsearch/issues/130

Re: Term Suggester Usage

Hey, the term suggester takes a query like "search is awesome" and returns suggestions for each of the terms, so three different suggestions are returned here. You maybe want to take a look at the completion suggester, see http://www.elasticsearch.org/blog/you-complete-me/ http://www.elasticsearc

Re: Detective wanted: Green cluster but query timeout unless changing preference to local.

Hey, you should check your logfiles on each of the nodes. Maybe they disconnected from each other (and reconnect all the time), so that you get problems when you execute your requests round robin. Using curl from one node to another does not show any problems? They are reachable all the time (at l

Re: Could not lock IndexWriter isLocked [false]

Hey, what elasticsearch version are you using? Judging from the directory I dont think you are using NFS, right? Are you running multiple instances locally? Have you shutdown elasticsearch properly so that no other instance is lingering around (you can use jps or or to check) --Alex On Mon, Ju

Logash 1.4.1, Windows Server 2008, Tomcat 7.0.30, stuck with localhost_access_log.2014-06-16.txt

Hi! I have stuck with logfile initialization in Windows using Logstash. I want to make logstash read localhost_access_log, but seems that Logstash can't understand (or maybe I am somewhere wrong) my path that I try to pass in config-file. input section contains this: file { codec => multili

Query Performance

Hi All, I am trying to improve my ES query performance. The goal is to get response times for 3 related queries under a second!. In my test i have seen 90th percentile response time (*took time*) for combined 3 queries to be ~1.8 seconds. Here are the details: *Cluster*: - 5 Machines, 5 Sha

Re: Help needed: CentOS ES-Logstash-Kibana running, but 0 docs, 0 indices

Tnx. Maybe there is an connection to my other problem: No index created: IndexCreationException: failed to create index (For that post I might better go to the Logstash mailinglist too, but I think it might be an ES config probl

No index created: IndexCreationException: failed to create index

Logstash: 1.4 Elasticsearch: 1.2 When trying the examples of 'Logstash 10-minute Tutorial ' step 3 goes wrong. First, the problem is 'embedded' is not accepted, resulting in Java-errors. So I changed the output a little bit:

Re: Creating a browse interface from ES

What about this: - build author name index - page size is static (e.g. 20) - absolute position: you must index each author name with absolute position info (sort author names before indexing, use a counter and increment it while indexing) - sort asc/desc works on author's name keyword analyzed

Elasticsearch inserting issue

Hi, My application is reading data from a sqs and writtes the data in elasticsearch(alias ES) and mysql. We are writting in both for back-up(for the moment). This is done with 2 consumers with 5 workers. One worker is writing in mysql and ES. The app is written in java and supports inserting,

Re: Help needed: CentOS ES-Logstash-Kibana running, but 0 docs, 0 indices

You might want to take this over to the Logstash mailing list instead as that appears to be the problem. Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: ma...@campaignmonitor.com web: www.campaignmonitor.com On 16 June 2014 17:39, 'Robert AMS' via elasticsearch < elasticsea

Re: ElasticSearch not starting after switching network.

It could be a change of network interface which has caused problems with the firewall? Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: ma...@campaignmonitor.com web: www.campaignmonitor.com On 16 June 2014 19:06, Ashok Chidige wrote: > yes, but I've not changed anything i

Re: How to get rid of org.elasticsearch.plugins information logging

You can probably put org.elasticsearch.plugins module to ERROR level instead of INFO here. --  David Pilato | Technical Advocate | Elasticsearch.com @dadoonet | @elasticsearchfr Le 16 juin 2014 à 10:57:17, Georgi Ivanov (georgi.r.iva...@gmail.com) a écrit: Hello, How can i get rid of  Jun 16,

Re: ElasticSearch not starting after switching network.

yes, but I've not changed anything in the firewall. not sure if the my company pushed anything silently. i'll check. Regardless I cannot turn off the firewall. What firewall rules do I've to enable to get it working? On Sunday, June 15, 2014 6:59:14 PM UTC-4, Mark Walkom wrote: > > Are you on Wi

How to get rid of org.elasticsearch.plugins information logging

Hello, How can i get rid of Jun 16, 2014 10:38:13 AM org.elasticsearch.plugins Information: [Thinker] loaded [], sites [] every time my client connects to ES ? It is not a big problem, but this output is messing up with my shell scripts. I am using transport client if this matters. Is this s

Re: java client, do I put the elasticsearch.yml in my /resources?

You should define more than one node in the transport list but you can also set  client.transport.sniff to true to sniff other nodes. If you define in your cluster a load balancing only node (no data), you probably should define it (or them) in the transport list of nodes. --  David Pilato | Tec

Re: java client, do I put the elasticsearch.yml in my /resources?

For "2/" : let's suppose I have a cluster with next arhitecture : non-data node, primary node and data nodes and I want to insert documents with java api. Which is the node which should be set for transportClient in addTransportAddress? The primary, the data or the balancing(non-data) node? Wh

Re: Help needed: CentOS ES-Logstash-Kibana running, but 0 docs, 0 indices

This did not help. Ik paramedic it looks as if everything is running, except there are no docs (and indices) coming in ES. Seen screenshot. Op zondag 15 juni 2014 02:11:09 UTC+2 s