ES on windows without admin permissions?

Hi, I want to setup a demo for my manager on ES with production data (from the mainframe). I don't have admin rights on my machine and production data is sensitive, so if I load it in a server elsewhere the whole department may be in trouble. Is there any way that I can mount a portable server,

Re: Aggregation query

Hello Ivan , This is expected. Only the top N(size mentioned in aggregation) results are taken from each shard before reducing the result. Due this , the accuracy is not guaranteed but the order is guaranteed. As a fix , you can use this to improve accuracy at the cost of memory - http://www.elast

Re: restore data on disk

Hi Chenlin, Thank you! I did not find any interested message in the elasticsearch.log. It is just filled with warning messages. I would like to join the QQ group, but I submitted join request to the QQ group and have not been approved. best, Yuheng On Friday, August 15, 2014 1:02:54 PM UTC

Optimization Questions

Hey Guys, We were doing some updates to our es(1.3.1) clusters recently and had some questions about _optimize. We optimized with max_num_segments 1 and we're still seeing ~25 segments per shard. The index that was optimized had no writes going to it during the time, it was actually freshly r

Re: A few questions about node types + usage

1 - Up to you. We use the http output and then just use a round robin A record to our 3 masters. 2 - They are routed but it makes more sense to specify. 3 - You're right, but most people only use 1 or 2 masters which is why they get recommended to have at least 3. 4 - That sounds like a lot. We use

elasticsearch 1.3.2 process logging.yml as a json foramt when logging.yml is deployed from subversion repo

I deploy these files from svn repository. since logging.yml is under subversion there is a .svn/text-base/logging.yml.text-base file. ES load this file ends with text-base and assumes its a json format. a org.elasticsearch.common.jackson.core.JsonParseException is raised. Is there a way to

Re: Quick Kibnana exclude terms question

No takers on this? On Wednesday, August 13, 2014 9:27:57 AM UTC-6, digit...@gmail.com wrote: > > He all... > > What's the format for the "exclude terms" in the Terms panel? I'm trying > to not show all IP's that have 172.16 in them, and I'm not having much > luck. I've tried: > > 172.16 > 172.

Re: Help with data recovery!

How are you running ES? Installed package? I would copy the data from /var/lib/elasticsearch to the new server. On Friday, August 15, 2014 10:24:31 AM UTC-6, Richard Wolford wrote: > > We had an Elasticsearch server that was reading IIS log files and storing > the data on a remote file share.

Re: Exclude specific bucket with integer key from term aggregation

I have this problem too - this was easily solved using the Terms Facet's exclude feature , but I haven't found a solution *within* Elasticsearch (aggregations) to this either.

Re: Elasticsearch cluster on AWS. Article.

Sort of. If you use private IPs on both side the rate for transfer between availability zones (not regions) is 0.00 per GB. Zero cents. Just another reason not to use public IPs on your ES instances. David On Friday, August 15, 2014 7:09:52 AM UTC-7, Andrej Rosenheinrich wrote: > > David, you a

Re: restore data on disk

check the elasticsearch.log for more informations. BTW: you can join QQ group: 315428175, many chinese elk users there. 2014-08-16 0:06 GMT+08:00 Yuheng Du : > typo: the second image shows the directory > at '$ES_Home/data/elasticsearch/nodes/1/indices/' > > -- > You received this message becau

Re: Node Client with bulk request indefinitely blocked thread when ClusterBlockException is being thrown

ES guys told they will fix this in the next release, probably in like two months. We wanted to catch this exception so we could do a retry if ES cluster is down(we have some sort of SLA which ensure the users that every document will be indexed). We found two ways to fix it: 1. Do a request befo

Re: Aggregation

What's your `deviceId` mapping type? Make sure it's a number as using in percentile aggregation. 2014-08-15 23:49 GMT+08:00 Yuheng Du : > > I am using: > > > > and I got the following errors: > > >

Help with data recovery!

We had an Elasticsearch server that was reading IIS log files and storing the data on a remote file share. Everything was working just fine. The Elasticsearch server then encountered a problem, so we spun up a new one and pointed it to the same remote file share. However, the new Elasticsear

Re: restore data on disk

typo: the second image shows the directory at '$ES_Home/data/elasticsearch/nodes/1/indices/' -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+un

restore data on disk

Hi guys, I am new to elastic search. I found that some indexes from the logstash were missing this morning. So I looked at the '$ES_Home/data/elasticsearch/nodes/0/indices/' directory:

Re: Aggregation

I am using: and I got the following errors: can anyone tell me what is going w

Re: slowlog not populating after deletion

When the logs weren't populating initially, I created new log files with `touch`. I deleted all those files and restarted my cluster, and ES created new log files and they seem to be populating again. On Friday, August 15, 2014 10:03:53 AM UTC-4, Tim Hopper wrote: > > We added another node to o

Aggregation query

When I run the following query on a 5 shard ES db I don't get accurate results. I have had to reduce the amount of shards on my ES server to 1 to get the accuracy I need? Has anyone had a similar issue? GET /incidents/_search?search_type=count { "query" : { "filtered" : { "filter" :

Re: Document corruption in index, id field is garbled text

Hi Rafal, Thanks for the helpful insights, and setting me in the right direction. All of those makes sense, now to investigate why Activerecord might be triggering these document to index without an id. - anurag On Friday, August 15, 2014 6:46:30 AM UTC-7, Rafał Kuć wrote: > > Hello! > > Your

Re: Can plugin be written for TCP transport?

Of course understood. I think subnetting/double firewall. DMZ will not have access to 9300 and internal network will not have access to 9300 only subnet for ES cluster functions. On Friday, 15 August 2014 05:04:35 UTC-4, Jörg Prante wrote: > > You can not protect from superuser access from wit

Re: Elasticsearch cluster on AWS. Article.

David, you are of course right with 2), but one thing to concider is that you pay for incoming and outgoing traffic between different availability zones. Am Freitag, 15. August 2014 14:32:54 UTC+2 schrieb David Severski: > > Thanks for collecting this information together! A couple points for >

Re: slowlog not populating after deletion

We added another node to our cluster yesterday, and its slowlog immediately started populating. Any idea how to get these populating again after they've been deleted? On Wednesday, August 13, 2014 3:45:45 PM UTC-4, Tim Hopper wrote: > > This morning, I enabled slowlogs on a bunch of indices in m

Re: Document corruption in index, id field is garbled text

Title: Re: Document corruption in index, id field is garbled text Hello! Your document is not corrupted - during indexation the _id field was set to null - this is what _source shows. The _id you are seeing, that contains a random characters was just generated by Elasticsearch, which is the defa

Document corruption in index, id field is garbled text

We are using ES 1.2.2 server with a rails application as the client (ActiveRecord document model) and it seems as though some of the documents in the index might have been corrupted because the *id *field of the document is some garbled text like "JorMcjefSe2_VQkP_ntd8Q" when its supposed to be

Re: Elasticsearch cluster on AWS. Article.

@David Severski One more question > 3) EC2-Classic is deprecated. Demonstrating use of VPC would be helpful. What do you mean by that? On Fri, Aug 15, 2014 at 3:39 PM, Pavel P wrote: > @David Severski > > Thanks for the input. > > I've actually encountered the issue, when my security group w

Hooks for knowing when topology has changed

how to know when a shard has moved or a new node is added in elasticsearch. is there any plugin/hook in java to do so? -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an em

Re: Elasticsearch cluster on AWS. Article.

@David Severski Thanks for the input. I've actually encountered the issue, when my security group was closed for the world and the private IPs were not stated in the security group rules. I had no idea why the could-aws does not connect to the hosts, because the public IPs were there. I've stated

Re: Elasticsearch cluster on AWS. Article.

Thanks for collecting this information together! A couple points for tweaking: 1) Instead of hard coding the IAM credentials into the file, associate the instances with an IAM role. cloud-aws will use those automatically and AWS will handle key rotation for you. 2) You are launching all the ins

Re: Elasticsearch cluster on AWS. Article.

Thanks Andrej, Would have it in mind! On Fri, Aug 15, 2014 at 2:25 PM, Andrej Rosenheinrich < andrej.rosenheinr...@unister.de> wrote: > Nice collection, well presented, thanks! > > One note, you can probably restrict your permissions even further, instead > of ReadOnly I use just the following

Marvel not working

Hi , I have setup marvel on my single node elastic search (version 1.2.2) deployment. Initially after deployment it worked but now it has stopped working. Please see the attached screenshot of what is displayed on the browser. I see the following exception in the elastic search logfile. [2014

Re: Elasticsearch cluster on AWS. Article.

Nice collection, well presented, thanks! One note, you can probably restrict your permissions even further, instead of ReadOnly I use just the following 5 rules (may be even that is too much) : { "Statement": [ { "Effect": "Allow", "Action": [ "ec2:DescribeAvailability

Re: A few questions about node types + usage

Bump. Any help? Thanks On Wednesday, 13 August 2014 12:10:14 UTC+1, Alex wrote: > > Hello I would like some clarification about node types and their usage. > > We will have 3 client nodes and 6 data nodes. The 6 1TB data nodes can > also be masters (discovery.zen.minimum_master_nodes set to 4).

Help with the percentiles aggregation

Hi, Am trying to run a single command which calculates percentiles for multiple search queries. The data for this is an Apache log file, and I want to get the percentile response times for the gets, posts, heads (etc) in one go If I run this: curl -XPOST 'http://localhost:9200/_search?search_ty

Re: Can plugin be written for TCP transport?

You can not protect from superuser access from within an app, except when you are also a superuser, and can create obscure kernel capabilities to protect an app from another superuser. Usually this is not solvable by technical solutions, it is a matter of trust into the infrastructure (data center

Re: Reduce threads used by elasticsearch

"transport.netty.worker_count" is for "org.elasticsearch.transport.netty.NettyTransport", while "http_server_works" are started by "org.elasticsearch.http.netty.NettyHttpServerTransport". If a class that uses ES's componentSetting is in org.elasticsearch, the componentSetting takes the part be

Using Kibana's query functionality with Java API

Hello, I was wondering if it is possible to use Kibana's query functionality with the Java API. For instance in Kibana when I enter *message="test", Kibana would return a list of documents/entries that have column "message" matching "test". I want to be able to do the same using Java without havin

Re: top_children query: how to play with factor and incremental_factor knobs?

Hi, The "from" and "size" are parameters of the overall query, not of the TopChildren clause. You can find a clarification of TopChildren here .

Kibana missing fields not working in Terms Panel

the documents stored in Elastic Search have an embedded object called address with inner properties city_name,state_code etc.Some documents dont have address field at all.I am trying to get a count of all documents which dont have any address.But when I add a field called address the count includes