OK. Optimize finally returned, so I suppose something was happening in the background, but I'm still seeing over 6500 segments. Even after setting max_num_segments=5. Does this seem right? Queries are a little faster (350-400ms) but still not great. Bigdesk is still showing a fair amount of file IO.
On Thursday, April 3, 2014 8:47:32 AM UTC-4, Elliott Bradshaw wrote: > > Hi All, > > I've recently upgraded to Elasticsearch 1.1.0. I've got a 4 node cluster, > each with 64G of ram, with 24G allocated to Elasticsearch on each. I've > batch loaded approximately 86 million documents into a single index (4 > shards) and have started benchmarking cross_field/multi_match queries on > them. The index has one replica and takes up a total of 111G. I've run > several batches of warming queries, but queries are not as fast as I had > hoped, approximately 400-500ms each. Given that *top *(on Centos) shows > 5-8 GB of free memory on each server, I would assume that the entire index > has been paged into memory (I had worried about disk performance > previously, as we are working in a virtualized environment). > > A stats query on the index in questions shows that the index is composed > of > 7000 segments. This seemed high to me, but maybe it's appropriate. > Regardless, I dispatched an optimize command, but I am not seeing any > progress and the command has not returned. Current merges remains at zero, > and the segment count is not changing. Checking out hot threads in > ElasticHQ, I initially saw an optimize call in the stack that was blocked > on a waitForMerge call. This however has disappeared, and I'm seeing no > evidence that the optimize is occuring. > > Does any of this seem out of the norm or unusual? Has anyone else had > similar issues. This is the second time I have tried to optimize an index > since upgrading. I've gotten the same result both time. > > Thanks in advance for any help/tips! > > - Elliott > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/01ad85fc-3ad0-4c90-9f6e-1ed275bd7312%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
