Adding shards to reduce average size

Hello I have been struggling off-and-on for months now to run an apparently highly updated ES cluster on as little hardware as possible. There's more history here , but the overall

Re: Sustainable way to regularly purge deleted docs

I've been able to back down to only 3 nodes again. So, I've probably wasted dozens of hours a hundreds of dollars of server time resolving what was ultimately a self-inflicted problem that should have been fixed easily months ago. So it goes. On Thursday, December 4, 2014 11:54:07

Re: Looking for a best practice to get all data according to some filters

Just to reword what others have said, ES will allocate memory for [size] scores as I understand it (per shard?) regardless of the final result count. If you're getting back 4986 results from a query, it'd be faster to use "size": 4986 than "size": 100. What I've done in similar situations

Re: Sustainable way to regularly purge deleted docs

pled with > the fact that you have a lot of reindexing going on (that too with child > documents, since any change in any one of the child documents results in > all other child documents and the parent document being marked as deleted), > things have gotten particularly out of hand

Re: Sustainable way to regularly purge deleted docs

he merge policy will still > shy away from merging them. > I raised reclaim_deletes_weight slightly (2.5 or 3 or so) and lowered > segments_per_tier and max_merge_at_once to get slightly better search > performance. These were likely less important. > > I hope that helps some!

Re: Sustainable way to regularly purge deleted docs

; Jonathan, > Did you find a solution to this? I've been facing pretty much the same > issue since I've added nested documents to my index - delete percentage > goes really high and an explicit optimize leads to an OOM. > Thanks. > > On Saturday, August 23, 2014 8:08:

Re: Native script unable to get values, perhaps because it's a child doc? ES v1.1.1

Thursday, 20 November 2014 11:38:30 UTC-8, Jonathan Foy wrote: >> >> Hello >> >> I have a native script that I'm using to score/sort queries and it is not >> working properly for one of my three types. >> >> All three types have the same nested field, a

Native script unable to get values, perhaps because it's a child doc? ES v1.1.1

Hello I have a native script that I'm using to score/sort queries and it is not working properly for one of my three types. All three types have the same nested field, and I'm using the script to check values and score/sort by an externally defined order. However, for one of the three types t

Re: Update merge settings pre-1.4 without downtime

ching merge policies and then switching back. I never > tried that but I think it might work. > > Nik > On Oct 10, 2014 12:33 AM, "Jonathan Foy" > > wrote: > >> Hello >> >> Is there any way of changing the merge

Re: Sorting by nested fields

I suspect that because you have multiple levels of nesting you need to use the nested path (it's probably choosing marketplaces.prices, and you would want just market

Update merge settings pre-1.4 without downtime

Hello Is there any way of changing the merge settings of a live index without downtime in ES versions prior to 1.4 (I'm on 1.1.1)? After toying with some of the settings for quite a while (such as index.merge.policy.expunge_deletes_allowed and index.merge.policy.max_merge_at_once_explicit) I h

Re: Elastic HQ not getting back vendor info from Elasticsearch.

Development seems to have stalled on ElasticHQ for whatever reason, but look at the various forks, several people have fixed bugs, including perhaps the one you're running into. On Wednesday, August 27, 2014 10:27:41 AM UTC-4, John Smith wrote: > > I know, but i think it's 2 issues... > > 1- The

Re: Optimizing queries for a 5 node cluster with 250 M documents (causes OutOfMemory exceptions and GC pauses)

I ran into the same issue when using Integer.MAX_VALUE as the size parameter (migrating from a DB-based search). Perhaps someone can come up with a proper reference, I cannot, but according to a comment in this SO

Re: Sustainable way to regularly purge deleted docs

> your index, do you know what causes this memory pressure? In case it is due > to field data maybe you could consider storing field data on disk? (what we > call "doc values") > > > > On Fri, Aug 22, 2014 at 5:27 AM, Jonathan Foy > wrote: > >> Hello

Sustainable way to regularly purge deleted docs

Hello I'm in the process of putting a two-node Elasticsearch cluster (1.1.2) into production, but I'm having a bit of trouble keeping it stable enough for comfort. Specifically, I'm trying to figure out the best way to keep the number of deleted documents under control. Both nodes are r3.xlar

Re: Every other query slow

Note also that the slow instances of the query do not appear to show up in the slow query log. Also, I'm pulling the referenced times out of the response's "took" field. On Tuesday, July 8, 2014 11:09:21 PM UTC-4, Jonathan Foy wrote: > > Hello > > I'm

Every other query slow

Hello I'm trying to get a new ES cluster tuned properly to actually put into production, and I'm running into some performance issues. While testing, I noticed that when running the same query multiple times, I had alternating fast (~50 ms), and slow (2-3 s) results. It's the exact same quer

Warmer queries - many small or one large

Hello The subject pretty much says it all...is there an advantage one way or the other to having several (or many) small (single term) warmer queries rather than a single large query that searches all desired fields? -- You received this message because you are subscribed to the Google Groups

Re: Sorting by a nested multi-field only works for one of the fields

On Thursday, April 24, 2014 8:12:26 AM UTC-4, Jonathan Foy wrote: > > Hello > > In my mapping I have some String values that I need to use for text > matching as well as for sorting. To support this, I use a multi-field with > two different versions of the field using two d

Sorting by a nested multi-field only works for one of the fields

Hello In my mapping I have some String values that I need to use for text matching as well as for sorting. To support this, I use a multi-field with two different versions of the field using two different analyzers. To sort, I have a keyword-lowercasing analyzer. For the text search I'm usin