Re: Best practice architecture using ES

Thanks, Will do. On Monday, July 14, 2014 11:57:14 PM UTC+3, Nikolas Everett wrote: > > You could look here: > http://git.wikimedia.org/tree/mediawiki%2Fextensions%2FCirrusSearch.git/master/ > Its a long way from perfect but it is open source. Its PHP so the general > organization is a bit dif

Re: Best practice architecture using ES

Danny, > a bit of code running in node.js That piece of code is is my java mid layer. On Tuesday, July 15, 2014 8:37:03 AM UTC+3, Danny Lieberman wrote: > > Nik > > Excellent point. > > However - consider that sending search requests to a MVC back end service > doesn't mitigate client-side J

ElasticSearch+Hadoop+Spark

Hi guys, I writing a spark application where I want to use ES with Hadoop. I have a lot of document in ES now I want to aggregate but I can't. My document's have different fields which means some have "twitter" field, with values, some have "facebook" etc When I try to read the data from ES I g

Re: An Open Source implementation of Google Drive Realtime API

Currently supported features: - User presence - Undo/Redo - Collaborative live JSON Data Model which is synchronized automatically between client and server - Automatically data conflict resolution powered by operational transformation - Two-Way Data Binding between the live

Re: Best practice architecture using ES

Nik Excellent point. However - consider that sending search requests to a MVC back end service doesn't mitigate client-side Javascript denial of service threat scenario - it just means that the attacker is now attacking the back end and not the proxy. Idan's MVC application is by defini

Re: Shard allocation after upgrade from 1.1.2 to 1.2.2

That sounds like an idea :) My indexes get created dynamically around midnight, I'll check one more day and if it still looks this unbalanced, I'll put the settings in my template and see what happens. Thanks :) On Tuesday, 15 July 2014 12:35:14 UTC+9, smonasco wrote: > > > http://www.elastics

Re: tiering storage / Curator

Nope, you can use allocation awareness to have indexes on different machines - http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/index-modules-allocation.html Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: ma...@campaignmonitor.com web: www.campaignmonito

tiering storage / Curator

Hello, Curator makes is possible to migrate an index to another storage programmatically, and that's very nice to keep old indices on cheap storage. But if I understand correctly, a unique ES cluster cannot handle two different storages. Hence, having small but fast storage for recent files and

Re: Delete by time range with ES 1.1

Maybe you just discovered by yourself what Aaron described in a previous message (Message-Id: <1405173409302.ce95d5ef@Nodemailer> on logstash mailing list): > [deleting a type of documents from an index] is a horrible idea in the same > way that running DELETE FROM TABLE WHERE TYPE='cisco' AND

Re: Cassandra + Elasticsearch or Just Elasticsearch for Primary data store.

> > > I'm just confused if Cassandra can really make a difference here, since > looks to me ES can suffice here. > > > If you are not going to be using Cassandra for indexing then there is no reason to have it. If you want durability in case something goes wrong with ES you can just store your

Re: Upgrade 0.26.6 -> 1.2.2 any catches?

@Ivan, what do you mean the stores are throttled with a very low level? Do you mean index threads or merge factor or what? -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send

Shard allocation after upgrade from 1.1.2 to 1.2.2

http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/index-modules-allocation.html There is a total shards per node feature that will allow you to force 4 per node and it can be set on live indexes. Not sure how you are creating your indexes but you can add that to the index du

Re: Garbage collection pauses causing cluster to get unresponsive

Hi Mark, The total targetted data rate is around 17Mb per sec. The expected number of indices are around 50 with each index having 10 shards. Regards, Srinath. On Monday, 14 July 2014 09:00:32 UTC+5:30, Mark Walkom wrote: > > How much data does that all come out to be, GB and doc count? > H

Re: Garbage collection pauses causing cluster to get unresponsive

Each document is around 300 bytes on average so that bring up the data rate to around 17Mb per sec. This is running on ES version 1.1.1. I have been trying out different values for these configurations. queue_size was increased when I got EsRejectedException due to queue going full (default siz

percolator throughput is not stable

​ this is the statistic I'm seeing from marvel in my dev cluster. I run a new process around 11:00 and as you see throughput fluctuates and eventully stuck at less than 10k/s which is worse than the throughput in it's early phase. It seems very weired to me, I looked up resource usage(cpu, mem) n

Shard allocation after upgrade from 1.1.2 to 1.2.2

I have daily rotating indexes and everything usually balances out nicely. 8 shards plus 1 replica over 4 nodes goes nicely 4/4/4/4. All nodes are identical with the same ES settings. Now I checked the kopf plugin this morning to just check the status and I noticed that after the 1.2.2 upgrade t

Re: Elasticsearch-Indexing data in two different DataCenters

You'll have to send another request, but someone might be able to give you some hints around optimising that in your code. Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: ma...@campaignmonitor.com web: www.campaignmonitor.com On 15 July 2014 09:12, srikanth ramineni wrote:

Re: Elasticsearch-Indexing data in two different DataCenters

Hi Mark, Thanks for your response.. Now we are using bulk request to index the data Settings settings = ImmutableSettings.settingsBuilder().put("cluster.name", "DC1_TS3_ES_CLUSTER").build(); Client client = new TransportClient(settings).addTransportAddress(new InetSocketTransportAddress("one",

Re: elasticsearch dies every other day

Looks like your cluster is at capacity. Try closing/deleting some old indexes, adding more nodes or more RAM. Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: ma...@campaignmonitor.com web: www.campaignmonitor.com On 14 July 2014 23:22, smonasco wrote: > Do you have thread

Re: Elasticsearch-Indexing data in two different DataCenters

How are you expecting to index the data to one cluster? Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: ma...@campaignmonitor.com web: www.campaignmonitor.com On 15 July 2014 07:33, srikanth ramineni wrote: > Hi , > > i want to index same data in two different servers. Ca

Re: Exceptions related admin cluster stats

Looks like the endpoint that is being called is incorrect - cluster/stats/n Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: ma...@campaignmonitor.com web: www.campaignmonitor.com On 15 July 2014 07:39, Nikhil Mulley wrote: > In my elasticsearch cluster version 0.90, I hav

Re: Upgrade 0.26.6 -> 1.2.2 any catches?

First of all, there is no version 0.26. I am assuming you meant 0.20.6. Either way, any upgrade from prior of 1.0 to 1.x will require a full cluster restart. 1. No clue 2. Many settings like omit_norms were deprecated, but are still support. I think that omit_tf has been changed. 3. I would inst

Re: Upgrade 0.26.6 -> 1.2.2 any catches?

You probably want to go from 0.20.X to 0.90.12, then to 1.X to be safe. Ensure you do a optimise (merge) on each index so that they are updated to the newer lucene version number, then when they are all done move to the next major version. Regards, Mark Walkom Infrastructure Engineer Campaign Mon

Upgrade 0.26.6 -> 1.2.2 any catches?

Hey all, We are going to start our battle plans for upgrading from 0.26 to 1.2.2 and I have been reading through this doc: http://www.elasticsearch.org/guide/en/elasticsearch/reference/master/breaking-changes.html to help see what things we may need to worry about. Some specific questions 1 -

Re: Unable to delete indexes - "failed to process cluster event (acquire index lock) within 30s"

Hi, I am facing similar exceptions with elasticsearch 0.90. Wondering if there was any follow up on this in the community. thanks, Nikhil On Wednesday, April 16, 2014 9:58:39 AM UTC-7, nicktgr15 wrote: > > Similar error > > [2014-04-16 16:57:08,464][DEBUG][action.admin.indices.create] [Jerry >

Failed to delete index using curator

I saw these messages in the elasticsearch logs when the curator is failing to attempt the deletion of an old index org.elasticsearch.cluster.metadata.ProcessClusterEventTimeoutException: failed to process cluster event (delete-index [logstash-2014.01.02]) within 30s at org.elasticsearch

Re: Delete by time range with ES 1.1

Not sure what's happening, I restarted both elasticsearch instance for my 2 nodes-cluster, and now it's working. But I'm sure the cluster was in good state, I could PUT and GET. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe f

Delete by time range with ES 1.1

I'm trying to delete document from an index based on the timestamp. I'm using ES v1.1, so I have to use the query wrapper. This return the wanted result : curl -*XGET *'http://localhost:9200/test/_*search*?pretty=1' -d '{"query":{"range":{"@timestamp":{"from":"2014-07-10T00:00:00","to":"2014-07

Exceptions related admin cluster stats

In my elasticsearch cluster version 0.90, I have started seeing these exceptions lately. I am wondering any known reasons. Restart of the nodes in the cluster does not help either. [2014-07-14 17:35:43,897][DEBUG][action.admin.cluster.stats] [Abner Little] failed to execute on node [VVnZM0T7S-

Re: Best practice to design index/type for Elasticsearch

Thanks Jorg, I will try this out and get back. meanwhile if you have a sample to create the mapping at the 'item' level, please share it. Thanks, Ranjith On Monday, July 14, 2014 3:05:32 AM UTC-4, Jörg Prante wrote: > > Regarding the JDBC river, it depends how many data sources you have. If > y

Elasticsearch-Indexing data in two different DataCenters

Hi , i want to index same data in two different servers. Can you please suggest bestways to index in both DCs at same time. having two Data centers- means two different clusters, want to index data into two different data centers(two different clusters) with same index

Re: Disabling _all-Field but keep Netflow-Events searchable

This technically sounds like a Kibana question, so you might have better luck with the Logstash mailing list. Can't you simply prepend the field name in the query instead of relying on the default field? You can also change field names in Logstash. Another option is the copy-to-field. Similar to

Re: Best practice architecture using ES

You could look here: http://git.wikimedia.org/tree/mediawiki%2Fextensions%2FCirrusSearch.git/master/ Its a long way from perfect but it is open source. Its PHP so the general organization is a bit different then you'd get in Java. But it does work. Nik On Mon, Jul 14, 2014 at 4:49 PM, Idan Fri

Re: Incorrect offset/length values in phrase suggester result

I must be doing something silly. Looking at the documents returned within the suggestion result (responses \ hits \ hits), there's absolutely no correlation between those documents and the query. And for some reason it just returns documents that are all associated with domain names from South

Re: Best practice architecture using ES

Nik Good discussion - excellent point. Maybe I'm missing something but it seems to me this is why Elasticsearch has paging and why Ray is running Angular JS in the client so that he can enforce some business rules to throttle the amount of data requested and returned by a search and paginate

Re: Best practice architecture using ES

Nikolas, How did you structure your middle tier?(Java? Node? some details on the layers inside..) 2014-07-14 23:48 GMT+03:00 Nikolas Everett : > I'm looking at this from the perspective of a script kiddy trying to make > your life hard - if you open up Elasticsearch someone will notice that > qu

Re: Best practice architecture using ES

@Danny, Nope. Well maybe that could be a future reason. But I was looking for best practice implementation. thinking: 1. Should we have middle tier between ES and the client. 2.(Which is more important) how did you structure it(in case you did). That way ill be able to learn from you, experience

Re: Best practice architecture using ES

I'm looking at this from the perspective of a script kiddy trying to make your life hard - if you open up Elasticsearch someone will notice that queries are going through and then start hammering you with requests for pages of ten billion records each. Or something. I'm just saying that trying to

Re: Best practice architecture using ES

On Mon, Jul 14, 2014 at 4:25 PM, Danny Lieberman wrote: > There is no added value having a search box and sending a search request > to server layers and waiting for a response when the client can send the > query directly to the ES server and get a request directly (The ES server > sits behind a

Re: Best practice architecture using ES

Ray - Specifically relating to the search aspect not to business logic between the client and server. There is no added value having a search box and sending a search request to server layers and waiting for a response when the client can send the query directly to the ES server and get a re

Cassandra + Elasticsearch or Just Elasticsearch for Primary data store.

Hi, I'm struggling to chose between these two options: with having Elasticsearch as a primary data store or should I need Cassandra as the primary data store and then data being copied in ES for indexing? The goal is just to store documents worth of 144 KB and possibly increasing to 512KB. The

Re: What types of SSDs?

So I got my server with SAS It's an HP DL 380P G7 2 x6 (Hyperthreaded) 24 cores 72GB RAM and 5 Intel 530 SSDs (RAID 10) These are the stats while JMeter is pushing 3,500 indexing operations/sec Average documents size is 2,500 bytes. Indexing - Index:1.98msIndexing - Delete:0msSearch - Query:9.8

Re: Delete by time range with ES v1.0

> > [2014-07-14 19:49:16,154][DEBUG][action.search.type ] [Pisces] > [check][1], node[BF93qRWnRDGuT9GQozB0LQ], [P], s[STARTED]: Failed to > execute [org.elasticsearch.action.search.SearchRequest@8e75504] > org.elasticsearch.search.SearchParseException: [check][1]: > query[@timestamp:[1404

Re: Python version for curator

To continue, I installed curator on a Python 2.6.6 system thusly: pip install elasticsearch-curator And Elasticsearch 1.2.1 is installed on the same server. But when running curator --help, I see: *$ curator --help* Traceback (most recent call last): File "/usr/bin/curator", line 5, in f

Re: Best practice architecture using ES

@Itamar, > > "..like duplicating the model too many times " I am talking about best practice layers that in the future will help me to maintenance the code in case of changing.. (pretty similar to GenericDAO that we used to have with Hibernate..). take the most simple scenario when I am creat

Delete by time range with ES v1.0

I'm trying to delete document from an index based on the timestamp. I'm using ES v1.0, so I have to use the query wrapper. This return the wanted result : curl -*XGET *'http://localhost:9200/test/_*search*?pretty=1' -d '{"query": {"range":{"@timestamp":{"from":"2014-07-10T00:00:00","to":"2014-0

Python version for curator

A quick question: Is Python 2 acceptable for use with curator, or is Python 3 required? Thanks! Brian -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elastic

Re: What is bad of using pulsing postings format?

+1 on having more information on codecs/postings format. Including examples on where each makes sense and where they don't. The current documentation on the codec module page in reference doesn't make the trade offs very clear. It's quite vague to somebody who has not used it (not complaining

Re: How to change similarity settings runtime?

Jörg is correct. In general, it would be a bad idea to change the similarity during runtime, but there are cases were it would be acceptable and the system should allow for those cases: https://github.com/elasticsearch/elasticsearch/issues/4403 -- Ivan On Mon, Jul 14, 2014 at 12:00 AM, Jörg Pr

Child Injectors or Index scoped objects

I am learning the code and sounds like I am getting it. Would love to fix bugs and contributions back to such a great code base. What is the intention behind childInjectors? For example, ModulesBuilder modules = new ModulesBuilder(); modules.add(new IndexNameModule(index));

Re: Is this a valid backup strategy for 0.90.10?

For Step #3 you can use LVM Snapshots to make your copy of the data. It would be a fairly quick operation and you won't have to wait for the full data copy to happen, just for the snapshot to finish. Once the snapshot is complete, you can do step 4 and 5 and then mount the snapshot on a differ

Re: No node available

If I remember correctly this indicates a connection issue when you are using node client. Are there typos in elasticsearch client connection config? Is ES running at that host/port? Are firewalls blocking traffic on port 9300 on either client and server? > On Jul 14, 2014, at 11:34 AM, Meeraiah

ElasticSearch scaling issue

Hello, We recently had an issue adding a node to our Elastic Search cluster, and would be interested in anything I could do to troubleshoot the issue. We currently have a 2 node cluster that has been running for some time. I am looking to add a third node to this cluster. All nodes are runnin

Re: Elasticsearch 1.2 list of settings

I tried to catch all the component settings, and expanded them to full notation, yes. Still I'm far from perfect and some might have slipped through. Still thinking about moving this and my ES gists to a repo, it should give more than a settings list or an addendum to the existing ES guide and boo

Is this a valid backup strategy for 0.90.10?

We're running a large 0.90.10 cluster. Due to performance problems we're seeing with has_parent queries (unrelated issue), upgrading to 1.x is not an option for us at the current time. We're trying to figure out how to backup 0.90. The following link gives some ideas for doing backups: http:/

Doc values for field data

When you map fields to use doc values for field data, does that limit the functionality afforded to those fields to merely sorting and aggregations/faceting? The documentation mentions that filtering is not supported by numeric or string types when stored as doc values. Yikes, I thought that do

Re: Filter aggregation and nested documents

When fielddata is loaded, is it only that of which the aggregation job needs (items.question_label.raw in this case) or does it load the full _source of every match and extract the field? On Monday, April 28, 2014 9:04:09 PM UTC-4, Olivier B wrote: > > Thanks you. > OK, that's what I was fearin

Re: Elasticsearch 1.2 list of settings

There are a few settings where the full named is not specified in the code, but is relative to the module it is in. Does your grep code account for these settings? A repo with pull requests might be too much for the maintainer, but a wiki would work well. Great job, Ivan On Mon, Jul 14, 2014 a

Re: Optimizing a query that matches a large number of documents

Since the script is executed against lots of matched documents, perhaps converting it into a native Java script (not Javascript) would provide a performance boost. Note that using fields in scripts will force their values to be loaded into the cache. -- Ivan On Sun, Jul 13, 2014 at 8:54 AM, Da

Field norm calculation on ElasticSearch array fields

(Originally posted on Stack Overflow ) Here's the mapping for one of the fields in my index: "resourceId": { "type": "string", "index_analyzer": "partial_match", "search_analyzer": "l

Re: What do these metrics mean?

So it doesn't appear that the sum of currentqueries = K * active search threads . In other words they are not proportional. Currentqueries was flat, and search.active jumped 2 orders of magnitude. We have highly varied indexes and searches. Some indexes have only one shard with one replica a

Search(term and fuzzy) all fields of all documents of an index

I am new to elastic search and want to have a search field in my application using which I would want to search all the documents of elasticsearch whereever they appear. I also want to show the related search terms as dropdown (like google search box). It helps user choose the correct search ter

Re: Docker & Elasticsearch using Unicast

I got it to work without my iptables rules loaded by changing my discovery settings on both my host and container to: discovery.zen.ping.unicast.hosts: ["172.17.42.1:9300", "172.17.42.1:9301"] Thanks for all of your help! Knowing Docker needed UDP access helped quite a bit. On Monday, July 14

Re: Docker & Elasticsearch using Unicast

Ah, learned something new. I've updated my iptables to accept tcp and udp for 9200:9400. However, I'm still running into the same error. I'm still getting the earlier mentioned error message when using the container's IP (not ideal). When I use the following unicast list on the host: host: disc

Re: [Hadoop] : Parsing error in MR integration

Hi, Nothing jumps out from your configuration. The error indicates that the values passed to es-hadoop cannot be processed for some reason. Which is more surpsing considering your Mapper writes some constants to the output. I've pushed some improvements to the 2.x branch which explain better con

Re: No node available

can any one through any idea on this please... On Monday, July 14, 2014 10:02:38 AM UTC-5, Meeraiah Yampalla wrote: > > I am new to elasticsearch, can someone please help me on this exception. > > Exception in thread "main" > org.elasticsearch.client.transport.NoNodeAvailableException: No node

Re: Suspected bug with Panel.parameters.length when there # samples than length

Itamar, So I assume then that I need to customize the facet filter query using aggregations http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-aggregations.html Don't see any way to do this out of the box in Kibana GUI fwiw Thanks Danny On Monday, July 14, 2014

Re: Best practice architecture using ES

Except from the fact those search commands will still not be verified and could compromise your system (by executing PUT and DELETE verbs or scripts), or simply leak information In most scenarios, you should definitely not just proxy client requests but verify them or build them from scratch in yo

Re: Suspected bug with Panel.parameters.length when there # samples than length

Thanks - suspected bug in my thinking. D On Monday, July 14, 2014 6:01:04 PM UTC+3, Danny Lieberman wrote: > > I'm using ES and Kibana to collect security events and visualize them > using an Items panel. For the sake of illustration a security event has > a site and subject and other a

Re: Best practice architecture using ES

Itamar You are 100% right! Of course - one would not open your ES server to public but put it behind a firewall or proxy like nginx. That is a Very Bad Idea. I was (perish the thought!!) not suggesting that. I was suggesting that the approach of client code talking directly to a firewal

Re: Best practice architecture using ES

I would strongly suggest against that. Never expose ES to the public, always put it behind a server facade. To get a glimpse of what you are exposing yourself, see this recent blog post http://www.elasticsearch.org/blog/scripting-security/ -- Itamar Syn-Hershko http://code972.com | @synhershko

Re: Suspected bug with Panel.parameters.length when there # samples than length

You don't You either pull a small page of data to display (match_all query or any filtering query), or ask ES to aggregate the data for you and get back the metrics or buckets. You can also do both at the same time. -- Itamar Syn-Hershko http://code972.com | @synhershko

Re: Best practice architecture using ES

Ray Kind of a broad question.I assume you want to provide search to an existing Web app with MVC backend and Angular front-end. Just my 2c - but we are developing in Meteor and node js - and I would suggest shaking free of your MVC mind-set. I would have the data model push updates/ins

No node available

I am new to elasticsearch, can someone please help me on this exception. Exception in thread "main" org.elasticsearch.client.transport.NoNodeAvailableException: No node available at org.elasticsearch.client.transport.TransportClientNodesService.execute(TransportClientNodesService.java:202) at

Re: how to translate “and or” where clause from sql query to elasticsearch filter

I've just figured out the problem. It's no need in my case to remap the data. It's only needed to change the type:nested to type:object in my mapping of productOptionValues property. понедельник, 14 июля 2014 г., 13:03:23 UTC+3 пользователь Yarek T написал: > > Always build up your logic bit by

Suspected bug with Panel.parameters.length when there # samples than length

I'm using ES and Kibana to collect security events and visualize them using an Items panel. For the sake of illustration a security event has a site and subject and other attributes. My expectation was that if you specify panel.parameters.length = 10 (for example) and a panel.paramete

Re: Keep the number of segments to 5

Again, it highly depends on your data and your usage. My rule of thumb is to have enough memory to hold the entire active indexes. So if you have a 16GB server, that means Elasticsearch gets 8GB, that means you can deal with indexes of size 8GB in total. The OS will use the memory assigned to it t

Re: Best practice architecture using ES

I'm afraid that's still too broad a question The communication between your SPA and Controller is up to you to decide. Then you can structure a query manually quite easily using the QueryBuilder Java classes from the elasticsearch jar. Mappings will depends on your data structure and query types.

Re: Best practice architecture using ES

Hi, ill try to focus my question. Maybe that will gain any replies. I am looking for a simple java application which using ES. I want to learn best practices of the layers structured. DTO's mapping/services/json/dto mappings. I just wanna create(right) the logic layer between the client an

Re: ES cluster in Amazon auto scaling group

Is it possible that when 2 nodes were up, somehow some shards an their replica ended up on the same node, so when it was terminated, I lost data ? Isn't ES supposed to balance shard/replica in a way that this won't happen ? On Monday, July 14, 2014 9:52:12 AM UTC-4, Bastien Chong wrote: > > Hi,

ES cluster in Amazon auto scaling group

Hi, I have setup an autoscaling group in Amazon for my Elasticsearch Cluster. Everything was working fine with the discovery and all. I currently have 2 instances in the AutoScaling group. I started to work on a new Elastic Load Balancer health check, but since it's not done yet, the check onl

Re: Docker & Elasticsearch using Unicast

curl uses TCP, but for multicast/unicast, you need also UDP. Jörg On Mon, Jul 14, 2014 at 3:13 PM, Tony P. wrote: > I've tried a few different things to no avail. > > Current setup: > # vain effort to attempt to have the host find the node in the docker > container > host: discovery.zen.ping.u

Re: elasticsearch dies every other day

Do you have thread counts and do any of them correlate to the crash times? I'm guessing that we'll find index threads leap up. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, s

Re: elasticsearch dies every other day

Is the non data node a client node? Here we are counting master eligible nodes. Whether you havever 4 or 5 I would go with min master eligible nodes 3. I'm sure what your replica setup is, but only 2 nodes is probably not a healthy cluster. With this set to 3 you won't have a split brain. -

Re: Docker & Elasticsearch using Unicast

I've tried a few different things to no avail. Current setup: # vain effort to attempt to have the host find the node in the docker container host: discovery.zen.ping.unicast.hosts: ["127.0.0.1", "172.17.0.14"] container: discovery.zen.ping.unicast.hosts: ["127.0.0.1"] # Which won't be particular

Is it possible to control the colors of the bars in Kibana?

Something which I would have thought is trivial - so apologies in advance. I'm collecting data in elastic search. the mapping looks like this: "mappings" : { "dosing": { "properties" : { "sitename" : {"type" : "string", "index" : "not_analyzed"}, "subjectlabel" : {"type" : "string", "index"

Re: Keep the number of segments to 5

Is there an optimal ratio between index disk size and ram? On Monday, July 14, 2014 12:44:14 PM UTC+3, Michael McCandless wrote: > > Also, optimize is an incredibly costly (CPU, IO) operation. Really, it > should only be done when you know the index will no longer change, e.g. > when the da

Re: Keep the number of segments to 5

On Monday, July 14, 2014 12:44:14 PM UTC+3, Michael McCandless wrote: > > Also, optimize is an incredibly costly (CPU, IO) operation. Really, it > should only be done when you know the index will no longer change, e.g. > when the daily log index is done being written. > > Mike McCandless > >

Re: elasticsearch dies every other day

Here's the heap usage graph for the last week: http://blog.klavsen.info/ES-heapusage.png I've marked where it crashes. I hope this can help give an idea of what goes wrong and what I could try? -- You received this message because you are subscribed to the Google Groups "elasticsearch" group.

Re: elasticsearch dies every other day

Den søndag den 13. juli 2014 01.22.46 UTC+2 skrev Mark Walkom: > > What's your heap size how much data do you have in your indexes? > data used = approx. 4,5TB heapsize = 32700M (less than 32GB :) Heap memory used usually runs between 24 and 28GB - but when this happens, I see heap usage go to

Re: REST vs URL searching - what's the difference in this query?

You are using a query_string query in the first query, and a term query in the second query. http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-query-string-query.html#query-dsl-query-string-query http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/q

REST vs URL searching - what's the difference in this query?

This works in a browser and finds tens of thousands of records: http://localhost:9200/accountevents/_search?q=Medium:SMS This cmd finds nothing: curl -XGET 'http://localhost:9200/accountevents/_search' -d '{ "query": { "term": { "Medium": "SMS" } } }' We can't work out what

Update ES mapping from Groovy/Gradle script

Hi I'm trying to update mapping of one field to be *not_analyzed* First I send POST to create the river and the index: > > println "starting the river" > http.request(POST, JSON) { req -> > uri.path = "_river/ffxi/_meta" > body = [ > type: "mongodb", >

Re: elasticsearch dies every other day

Changing it to: threadpool.index.queue_size: 900 hope it helps :) -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. T

Re: Disabling _all-Field but keep Netflow-Events searchable

Anyone got an idea how to realize that? I think that there are a few uses which got Netflow AND other types of events inserting into Elasticsearch and for those a disabled _all Field would save much hard disk space -- You received this message because you are subscribed to the Google Groups "e

Re: Marvel does not display any contents with ES1.2.1.

Can you share a screen shot of what you see and also the logs of the node? I'm looking for the adress Es bound it self to and any error from marvel -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receivi

Indexing and Searching Multimedia Files

Hi, In elasticsearch the following datatypes of data can be indexed string, integer/long, float/double, boolean and null which is referred in the following link http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/mapping-core-types.html. Is there any method/approach can be

[Hadoop] : Parsing error in MR integration

Hi, I can't sort that ! I'm using hadoop CDH3u6, and trying to get ES index my data. I tried with raw json and MapWritable, I always get the same kind of errors : java.lang.Exception: org.elasticsearch.hadoop. EsHadoopIllegalArgumentException: [org.elasticsearch.hadoop.serialization. field.Ma

Re: how to translate “and or” where clause from sql query to elasticsearch filter

Always build up your logic bit by bit. Don't just bang it all in at once. Once you understand how individual components evaluate you will be able to build a working complex query. https://gist.github.com/yarekt/4bd8e0bccc8ca694c855 On Sunday, 13 July 2014 00:14:34 UTC+1, Artem Frolov wrote: >

Re: Garbage collection pauses causing cluster to get unresponsive

index.merge.policy.segments_per_tier: 100 and threadpool.bulk.queue_size: 500 are extreme settings that should be avoided as they allocate much resources. What you see by UnavailbleShardException / NoNodes is congestion because of such extreme values. What ES version is this? Why don't you use the

  1   2   >