Re: Any experience with ES and Data Compressing Filesystems?

Hi again, a quick report regarding compression: we are using a 3-TB btrfs-volume with 32k block size now which reduced the amount of data from 3,2 TB to 1,1TB without any segnificant performance losses ( we are using a 8 CPU, 20 GB Memory machine with an iSCSI.Link to the volume ). So for us

Re: boosting query howto?

Hi Jörg, thanks for the advise, it seams to be my solution. Are there any API javadocs for ES? It takes me 3 to 4 times longer writing something for ES than for Solr because of searching through the sources and no useful javadocs. Bernd Am Freitag, 1. August 2014 16:07:10 UTC+2 schrieb Jörg Pr

Re: Design HA ES for 16 TB logs data | Is SAN storage a good idea?

Heavy aggregations = lots of ram Storage, if you can use SSD. The only rule of thumb is get the best possible hardware that you can afford. Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: ma...@campaignmonitor.com web: www.campaignmonitor.com On 4 August 2014 13:09, John C

Re: Design HA ES for 16 TB logs data | Is SAN storage a good idea?

SAN question aside - what are guidelines on the balance of CPU/RAM/Storage so that no one thing is the obvious bottleneck. I know it depends on workload, so * For aggregation heavy workloads, about how much RAM : Storage? * For high volume, but smaller queries (individual log retrieval), what's

hi , about copying index data of elasticsearch ,i have a question. can you give me a suggestion?

i had build a elasticsearch cluster in one idc of city A, at the same time, indexing in city A. but in some time,i need have the same cluster in city B,including index data. can i copy the index data of city A to cluster of city B but index in city B again? if this idea if ok,what i should do

Re: action.admin.indices.create failed to create

Bit of a late reply as I never saw this, but like the other guys say, sounds like you're not specifying the index. What was the syntax you were using. On Monday, April 21, 2014 6:37:39 PM UTC+1, miki haiat wrote: > > HI , > > I using elastic4s for an api client , i cant index anything > im get

Re: rest api or java client?

It does support 2.11 of course. And about the Java client documentation - one more reason to use the Scala DSL in Elastic4s as you'll get code completion. For example you can do this `search in "places"->"cities" query "paris" start 5 limit 10` and each step of the way the DSL will let you kno

Re: Shard rebalancing

Shard size will depend entirely on how many shards you've set and how big the index is. Allocation of data to shards happens in a round-robin manner, so balancing isn't needed. What do you mean by shards changing in the background? Regards, Mark Walkom Infrastructure Engineer Campaign Monitor em

Re: Guice Creation Error

Do you create a new client for each request? If so, create only one client when your application start. -- David ;-) Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs Le 4 août 2014 à 00:42, Subacini B a écrit : Hi, We are using Spring Framework with elasticsearch. ES Version : 1.2.1 Co

Re: If I have ELK stack running on EC2. How can I make the ES as a cluster?

ES can take disk space into account, 1.3.X does this automatically - http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/index-modules-allocation.html#disk Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: ma...@campaignmonitor.com web: www.campaignmonitor.com

Guice Creation Error

Hi, We are using Spring Framework with elasticsearch. *ES Version* : 1.2.1 *Code* Client client = new TransportClient().addTransportAddress(new InetSocketTransportAddress(url, 9300)); SearchResponse response = client.prepareSearch(""escore") .addAggregation(A

Re: Design HA ES for 16 TB logs data | Is SAN storage a good idea?

A. There are many unknown factors regarding "SAN storage", e.g. how is the latency and the IOPS? Most of SAN are black boxes and do not scale over the number of connected hosts, so you should test it thoroughly to make an educated decision. There is no simple "yes" or "no". As a matter of fact, I w

Design HA ES for 16 TB logs data | Is SAN storage a good idea?

Hi, I'm testing/planning implementation for 16 TB data logs (1 month, daily indexes about 530GB/day). Indexes are deleted after 1 month (TTL is 1 month). The documents size vary from few bytes to 1MB (average of ~3 kb). We have 2 data center, and the requirement is to provide access to datas

Re: Elasticsearch field mapping, dynamic_templates

I dod migrate to ES 1.3.1 I did try to do the same trick, but it's fail to PUT oryginal, just dumped settings. Any ideas? curl -XGET localhost:9200/_template?pretty > template_all curl -XPUT localhost:9200/_template/*?pretty -d @template_all *{* * "error" : "ActionRequestValidationException[V

Re: Route query so that data for a shard is localized

Have you consulted the docs http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-terms-filter.html#_terms_lookup_mechanism about the optimizations of term lookup for TermFilter? There are caches in use, and for term lookup, you can also use routing to select a particul

Shard rebalancing

What is the behavior of ES when it comes to shard sizes? Does it do automatic shard rebalancing at any point of time? If so, is it also controlled through an API? How can I know if the shards are changing in the background? If I do not add any new node or change any cluster configuration once

Route query so that data for a shard is localized

Hi, I have fairly large data and a ES cluster. Can I use some shard knowledge to execute queries so that only data relevant to a particular shard is fetched for that shard/node? I want to make sure that if I have a filter, then the values in the TermFilter only hold records that are relevant to

Re: If I have ELK stack running on EC2. How can I make the ES as a cluster?

I don't know that ES has any intelligence to support varied node sizes so I would say yes they should be the same size. I've not looked into this so I may be wrong. Also I use multiple Ebs volumes in a software raid. to increase non provisioned iops. Not necessary if you use piops. Aaron Sent

How to find null_value in query_string like we can in missing filter

I'm using elasticsearch v0.90.5 With a missing filter, we can track missing fields: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-missing-filter.html and make sure that a null_value also counts as missing. How can we do the same in a query_string? http://www.elas

Re: SocketTimeoutException while using JEST to connect to ES

Hi Im facing the same issue. can you please elaborate about yout HTTP proxy issue? or any other options?? On Saturday, August 2, 2014 7:09:30 PM UTC+3, anuj maheshwari wrote: > > My client was not able to communicate with ES server. It was a HTTP proxy > issue. Try looking around this for your c

Inexplicable wrong results in automated tests

Hi So after running a few rounds of local automated tests, I've noticed that sometimes I get the wrong results in my index. This seems to only be an issue with my automated tests and not when running the application manually (at least I haven't seen the wrong results after several executions).

Nest - range filter in a form of BaseFilter

Hi, I'm using Nest version 0.12.0. I need to get a range filter in a form of BaseFilter. However, this line of code creates a faulty BaseFilter which doesn't contain the actual condition: *agg.ElasticsearchFilter = Nest.Filter.Range(i => i.GreaterOrEquals(filteredUpdateDate));* Is this a bug, or

[Kibana] Plugin to add new char type

Hi , I would like to add a new chart type for 2 level term grouping ( treemap visualziation). Is there anyway i can add this as a plugin to existing Kibana ? Thanks Vineeth -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsu

Using existing field for mapping

I am new to elastic search, I have created an index "cmn" with a type "mention". I am trying to import data from my existing solr to elasticsearch, so I want to map an existing field to the _id field. I have created the following file under /config/mappings/cmn/, { "mappings": {

Kibana Search Question

Hi, i'm using ELK for stop my API logs My message contains a json with an API request, something like that {"initObj": {"mediaType": 0, "pageSize": 100, "pageIndex": 0, "exact": false, "orderBy": "NONE", "orderDir": "ASC", "orderMeta": ""} I'm looking to find all the massages where "pageSize" in

Re: elasticsearch jdbc-river do not import data from mysql

As the error message indicates, at line 10, you have a comma after "type" : "test", and before a closing "}", which is invalid JSON syntax. Jörg On Sun, Aug 3, 2014 at 3:14 AM, mithril wrote: > There is a error seems be the key. > > > >> [2014-08-03 09:10:57,791][DEBUG][action.index