elasticsearch-js download link does not work

2015-02-10 Thread Robin Clarke
Trying to download latest elasticsearch-js, and download link does not work: http://www.elasticsearch.org/guide/en/elasticsearch/client/javascript-api/current/browser-builds.html#_download -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To uns

failed engine - MapperParsingException

2015-01-22 Thread Robin Clarke
So it seems that we have somehow got some invalid events into our index. The effect is that two shards are always in a state of initializing, never completing. In the Elasticsearch logs I see entries like this: [2015-01-22 12:09:01,124][WARN ][index.engine.internal] [es113-es1] > [logstas

Default shard allocation (where new shards are created)

2014-12-19 Thread Robin Clarke
My current setup is with 10 nodes with ample space on spinning disks, and 20 nodes with smaller SSD disks. I would like my workflow to be that all data is initially indexed on the SSD nodes, after 10 days is reallocated to the spinning disks, after a further 10 days the index is closed, and afte

Re: processors configuration (available_processors)

2014-12-16 Thread Robin Clarke
d", "min" : 16, "max" : 16, "queue_size" : "200" ... It seems this cannot be set using the config API Cheers, -Robin- On Tuesday, 16 December 2014 12:04:03 UTC+1, Robin Clarke wrote: > > I have a machine with 132GB of memor

processors configuration (available_processors)

2014-12-16 Thread Robin Clarke
I have a machine with 132GB of memory and 32 cores on which I am running two elasticsearch nodes. Each node should have only half the total number of CPU cores available so that both nodes can work at full capacity and not block each other. I believe the correct configuration option would be:

Re: network.publish_host for an address outside a docker container

2014-11-19 Thread Robin Clarke
complete with the switches you're using. > > Tony > > > > On Tuesday, November 18, 2014 5:38:48 AM UTC-8, Robin Clarke wrote: >> >> I am trying to do a setup like this: >> >> machine1 - 192.168.0.10 >> >> 9300 => 9300 Docker contai

network.publish_host for an address outside a docker container

2014-11-18 Thread Robin Clarke
I am trying to do a setup like this: machine1 - 192.168.0.10 9300 => 9300 Docker container elasticsearch1 internal vlan 172.17.0.68 Elasticsearch transport running on port 9300 network.publish_host 192.168.10:9300 9301 => 9300 Docker container elasticsearch2 internal vlan 172.17.0.69 Elasti

Docker configuration to allow multicast across machines

2014-11-17 Thread Robin Clarke
We are putting together a new cluster of 10 machines. Each machine has 132GB memory, so we want to put two Elasticsearch nodes (each with 30GB) on each machine, each node in its own docker container - that should result in 20 nodes on 10 machines. We have got docker to allow the two nodes on ea

Re: Many indices.fielddata.breaker errors in logs and cluster slow...

2014-10-15 Thread Robin Clarke
I'm still having this problem... has anybody got an idea what the cause / solution might be? Thank you! :) On Tuesday, 7 October 2014 14:29:22 UTC+2, Robin Clarke wrote: > > I'm getting a lot of these errors in my Elasticsearch logs, and am also > experiencing a lot of slow

Many indices.fielddata.breaker errors in logs and cluster slow...

2014-10-07 Thread Robin Clarke
I'm getting a lot of these errors in my Elasticsearch logs, and am also experiencing a lot of slowness on the cluster... New used memory 7670582710 [7.1gb] from field [machineName.raw] would be larger than configured breaker: 7666532352 [7.1gb], breaking ... New used memory 7674188379 [7.1gb] f

Re: Unstable cluster - "suspect illegal state: trying to move shard from primary mode to replica mode"

2014-07-28 Thread Robin Clarke
> > > On Saturday, May 24, 2014 5:05:55 AM UTC-4, Robin Clarke wrote: >> >> And found this error too in one of the nodes which left the cluster: >> >> java.lang.NullPointerException >> at org.elasticsearch.gateway.local.state.meta. >> Loca

Re: Getting OOME's (Out of Memory Exceptions) to stop

2014-06-26 Thread Robin Clarke
got similar issues and telltale sign of when things are about to go > south is when the old GC count starts to rise, and GC old duration > increases. > > On Wednesday, June 25, 2014 9:54:27 AM UTC-4, Robin Clarke wrote: >> >> I have a 10 machine cluster where freque

Getting OOME's (Out of Memory Exceptions) to stop

2014-06-25 Thread Robin Clarke
I have a 10 machine cluster where frequently (about once per day when indexing and querying is at its height) one elasticsearch node goes OOM... It usually recovers, but by this time the cluster is redistributing the lost shards, which causes more load, which often in turn causes an OOM on ano

Elasticsearch logs in JSON?

2014-06-23 Thread Robin Clarke
Is there any way to configure Elasticsearch to output its logs in JSON (custom log format, or configuration option)? This would make it much easier to import the logs via logstash... Cheers, -Robin- -- You received this message because you are subscribed to the Google Groups "elasticsearch"

Delete doesn't delete all indexes...

2014-06-10 Thread Robin Clarke
I am trying to delete a month of logstash indexes, and fire off e.g. this command: curl -XDELETE "http://localhost:9200/logstash-2014.05*?pretty"; which returns within a few seconds (less than a minute - the default timeout afaik) with: { "acknowledged" : true } But when I look at the indexe

Re: Unstable cluster - "suspect illegal state: trying to move shard from primary mode to replica mode"

2014-05-24 Thread Robin Clarke
And found this error too in one of the nodes which left the cluster: java.lang.NullPointerException at org.elasticsearch.gateway.local.state.meta.LocalGatewayMetaState.clusterChanged(LocalGatewayMetaState.java:185) at org.elasticsearch.gateway.local.LocalGateway.clusterChanged(Lo

Unstable cluster - "suspect illegal state: trying to move shard from primary mode to replica mode"

2014-05-24 Thread Robin Clarke
I have a 10 machine cluster (named es101->es110) with 32GB RAM per machine. I've allocated 12GB per machine to Elasticsearch. Memory usage on the machines looks ok, cpu and iowait is also not dramatic, nonetheless the cluster is frequently becoming instable and losing nodes... In the logs I am

Re: wait_for_completion doesn't seem to be working when making a snapshot

2014-04-02 Thread Robin Clarke
"ignore_unavailable": true, > "include_global_state": false > }' > > On Wednesday, April 2, 2014 9:43:14 AM UTC-4, Robin Clarke wrote: >> >> I am writing a small script to create a snapshot of my kibana-int index, >> and hit an odd rac

wait_for_completion doesn't seem to be working when making a snapshot

2014-04-02 Thread Robin Clarke
I am writing a small script to create a snapshot of my kibana-int index, and hit an odd race condition. I delete the old snapshot if it exists: curl -XDELETE 'http://localhost:9200/_snapshot/backup/snapshot_kibana?pretty' Then make the new snapshot curl -XPUT "http://localhost:9200/_snapshot/ba

Re: Benchmarks (again)

2014-03-26 Thread Robin Clarke
In that case, *7.4* -Robin- On 26 March 2014 12:43, Steinar Bang wrote: > >>>>> Robin Clarke : > > > I just pasted output from uname there - that's kernel 3.2.54-2 you're > > reading there. > > I think the Debian release is Wheezy (7.0)

Re: Benchmarks (again)

2014-03-26 Thread Robin Clarke
I just pasted output from uname there - that's kernel 3.2.54-2 you're reading there. I think the Debian release is Wheezy (7.0) -Robin- On 26 March 2014 12:17, Steinar Bang wrote: > >>>>> Robin Clarke : > > > Thanks for the tip with the number of masters!

Re: Benchmarks (again)

2014-03-25 Thread Robin Clarke
e: > Ouch, why such an old version of java? (And is is Open or Oracle?) > v7 is the minimum for ES and LS. > > Regards, > Mark Walkom > > Infrastructure Engineer > Campaign Monitor > email: ma...@campaignmonitor.com > web: www.campaignmonitor.com > > > On 25 M

Re: Benchmarks (again)

2014-03-25 Thread Robin Clarke
e you get a majority quorum. > > Regards, > Mark Walkom > > Infrastructure Engineer > Campaign Monitor > email: ma...@campaignmonitor.com > web: www.campaignmonitor.com > > > On 25 March 2014 17:35, Robin Clarke wrote: > >> Each node had 8 cores (2.4GHz Xeon),

Re: Benchmarks (again)

2014-03-24 Thread Robin Clarke
> email: ma...@campaignmonitor.com > web: www.campaignmonitor.com > > > On 25 March 2014 16:11, Robin Clarke wrote: > >> I did some intensive tests last week on a 20-node cluster and had the >> following insights - I'd be interested if anyone has similar/dissimila

Benchmarks (again)

2014-03-24 Thread Robin Clarke
I did some intensive tests last week on a 20-node cluster and had the following insights - I'd be interested if anyone has similar/dissimilar experience. The had 20 nodes had 8 cores each, and 32GB memory each. I set up Elasticsearch to have 15GB of that memory. The sample events I was using we

What happens (or should happen) if your elasticsearch configurations don't match?

2014-03-07 Thread Robin Clarke
I'm wondering if anyone knows (can't easily test right now) what happens if your elasticsearch configurations on different nodes don't match: e.g. if you have index.number_of_replicas: 0 on one node, and 1 on another - will replicas be made, or not? Is there any reasonable use of having different