Re: How big can/should you scale Elasticsearch

2014-08-29 Thread Nikolas Everett
On Fri, Aug 29, 2014 at 1:27 PM, Rob Blackin wrote: > We are trying to implement a 5 TB, 10 Billion item Elasticsearch cluster. > The key is an integer and the item data is fairly small. > We're running around 5.5TB right now without a problem. The biggest annoyance is that rolling restarts tak

Re: How big can/should you scale Elasticsearch

2014-08-29 Thread Arie
When you look to the guys @ found (https://www.found.no/pricing/) then the data on one ES server is 8 times memory, if it should run smooth, but do not know how valuable that is. If you have a lot of ES nodes, then consider one master node without data, it's a best practice I have read somewhere.

How big can/should you scale Elasticsearch

2014-08-29 Thread Rob Blackin
We are trying to implement a 5 TB, 10 Billion item Elasticsearch cluster. The key is an integer and the item data is fairly small. We seem to run into issues around loading. Seems to slow down as the index gets bigger. We are doing this on EC2 i2.xlarge nodes. How many documents/TB do you thin