Re: Strange system load increase
At first I was using elaticsearch paramedic ( https://github.com/karmi/elasticsearch-paramedic) recently I used marvel. Marvel was reporting a 2000 searches/s mark while the cluster was acting up. After the restart, it now reports 600 searches/s. Looking at nginx logs I see no change in rate before or after the restart. Maybe something other than elasticsearch is acting up, but I have no clue what else could it be. I do need to upgrade, but the breaking changes are making it hard for me to keep moving :( On Sat, Aug 9, 2014 at 7:03 PM, Mark Walkom wrote: > How are you measuring the searches/s metric? ES doesn't run searches > within itself, they have to be initiated externally somehow. > > > Also, you should really upgrade :) > > Regards, > Mark Walkom > > Infrastructure Engineer > Campaign Monitor > email: ma...@campaignmonitor.com > web: www.campaignmonitor.com > > > On 10 August 2014 02:24, Rafael Almeida wrote: > >> Hello, >> >> Last year paramedic reported thousands o searches per second (whereas our >> regular load are in the hundreds range) this eventually led to an excessive >> cpu load across the cluster (4 machines). Not more than a month later the >> same thing happened. We updated to ES 0.90.12 at the time, thinking it >> could have something to do with a "forever looping query" bug that was >> fixed. >> >> Since then (around 6 months) everything was fine. Yesterday the same >> thing happened again (we're still in the same elastic search version). One >> interesting thing we noted is that the increase in searches over time, >> which we thought was due to more adoption of the cluster in the company, >> was actually a product of that weird behavior. We were at 1000 searches per >> second. Yesterday it suddenly spiked to 2000 and it required a cluster >> restart. After the restart it dropped to 600 and stayed like that. >> >> Is there some recommendation for restarting machines in the cluster from >> time to time? Has anyone seen anything like this? >> >> []'s >> Rafael >> >> -- >> You received this message because you are subscribed to the Google Groups >> "elasticsearch" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to elasticsearch+unsubscr...@googlegroups.com. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/elasticsearch/CAKCnWVk2Ji0TNEx_Jdgzw1G5cMTiZZuk%2B1LzsSWVDbkzqiwJ1A%40mail.gmail.com >> <https://groups.google.com/d/msgid/elasticsearch/CAKCnWVk2Ji0TNEx_Jdgzw1G5cMTiZZuk%2B1LzsSWVDbkzqiwJ1A%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> For more options, visit https://groups.google.com/d/optout. >> > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to elasticsearch+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elasticsearch/CAEM624bqfuF4Kucx2gWSy40quQQHaFUf0DZwZ3GiaaupHROB-g%40mail.gmail.com > <https://groups.google.com/d/msgid/elasticsearch/CAEM624bqfuF4Kucx2gWSy40quQQHaFUf0DZwZ3GiaaupHROB-g%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKCnWVkYGwdAEPmQ0EK%3Dqw5LbSM8fV1S%2BjE5cm-Aqii-9dQkJg%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
Strange system load increase
Hello, Last year paramedic reported thousands o searches per second (whereas our regular load are in the hundreds range) this eventually led to an excessive cpu load across the cluster (4 machines). Not more than a month later the same thing happened. We updated to ES 0.90.12 at the time, thinking it could have something to do with a "forever looping query" bug that was fixed. Since then (around 6 months) everything was fine. Yesterday the same thing happened again (we're still in the same elastic search version). One interesting thing we noted is that the increase in searches over time, which we thought was due to more adoption of the cluster in the company, was actually a product of that weird behavior. We were at 1000 searches per second. Yesterday it suddenly spiked to 2000 and it required a cluster restart. After the restart it dropped to 600 and stayed like that. Is there some recommendation for restarting machines in the cluster from time to time? Has anyone seen anything like this? []'s Rafael -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKCnWVk2Ji0TNEx_Jdgzw1G5cMTiZZuk%2B1LzsSWVDbkzqiwJ1A%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
Re: Changing analyzer filters
Right. Let me elaborate my question, then.
This is my analyzer at first:
test:
type: custom
tokenizer: standard
filter: [standard]
$ curl -XPUT 'http://localhost:1980/t';echo
{"ok":true,"acknowledged":true}
$ curl -XPUT 'http://localhost:1980/t/t/_mapping' -d'{"t": {"properties":
{"title": {"type":"string", "analyzer": "test"';echo
{"ok":true,"acknowledged":true}
$ curl -XPOST 'http://localhost:1980/t/t/1' -d'{"title": "ATLETICO"}';echo
{"ok":true,"_index":"t","_type":"t","_id":"1","_version":1}
$ curl -XPOST 'http://localhost:1980/t/_search?pretty' -d'{"query":
{"query_string": {"fields": ["title"], "query": "ATLETICO",
"analyzer":"test"}}}';echo
{
"took" : 44,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"failed" : 0
},
"hits" : {
"total" : 1,
"max_score" : 0.30685282,
"hits" : [ {
"_index" : "t",
"_type" : "t",
"_id" : "1",
"_score" : 0.30685282, "_source" : {"title": "ATLETICO"}
} ]
}
}
$ curl -XPOST 'http://localhost:1980/t/_search?pretty' -d'{"query":
{"query_string": {"fields": ["title"], "query": "atletico",
"analyzer":"test"}}}';echo
{
"took" : 2,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"failed" : 0
},
"hits" : {
"total" : 0,
"max_score" : null,
"hits" : [ ]
}
}
So far so good. Now I changed my analyzer to this (and restarted
elasticsearch):
test:
type: custom
tokenizer: standard
filter: [lowercase, standard]
Now that last query returns a result:
$ curl -XPOST 'http://localhost:1980/t/_search?pretty' -d'{"query":
{"query_string": {"fields": ["title"], "query": "atletico",
"analyzer":"test"}}}';echo
{
"took" : 45,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"failed" : 0
},
"hits" : {
"total" : 1,
"max_score" : 0.30685282,
"hits" : [ {
"_index" : "t",
"_type" : "t",
"_id" : "1",
"_score" : 0.30685282, "_source" : {"title": "ATLETICO"}
} ]
}
}
What's happening?
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKCnWVn%3DhD30h8nEzw4fWxbqGcqD%3DAX%2BiJhg%3D%3D%2BSroXb%2BzRcag%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Changing analyzer filters
Hello, I had the following analyzer on my elasticsearch configuration: teste: type: custom tokenizer: standard filter: [standard] A field named title uses that analyzer. I indexed ATLÉTICO using it. Searching for ATLÉTICO works. Searching for atletico gives me no result. Afterwards, I stoped elasticsearch and changed the analyzer to teste: type: custom tokenizer: standard filter: [lowercase_pt, asciifolding, standard] in the configuration file. Now, without reindexing anything, I searched for "atletico" and the document was returned. That looks odd to me. Was the document reindexed automatically? How did the query work? I used query string to search for it and I specified that analyzer on it. []'s Rafael -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKCnWVn9gG8LiakK5%2BE454RZX26aAJvxeQW7biBzjPGMbop4hw%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
Differences between an index and an alias which points to a single index
Hello, Say I have an index called duckling and an alias called duck which points to duckling and no other index. Except for creating an alias pointing to duck, all the other operations seem to route directly to duckling. Is that true? is there something besides creating an alias that I can do with an index, but not with its alias? []'s Rafael -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKCnWVmmw%3Dtfn3%2B%3DH9V6THQ9Q%2BFq_v2W44RmCz5VSEZ3Zyh_xA%40mail.gmail.com. For more options, visit https://groups.google.com/groups/opt_out.
Optimize
Hello, I want to know when and if I should manually call optimize on elasticsearch. This blog seems to say it's a bad idea: http://gibrown.wordpress.com/2013/01/24/elasticsearch-five-things-i-was-doing-wrong/ However, there must be a reason for optimize to be exposed in the rest api. []'s Rafael -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKCnWVm9GTpo%2B79uCCfgfr9Tej98M5UyiB02BDTJ-8EJyMv%3Djg%40mail.gmail.com. For more options, visit https://groups.google.com/groups/opt_out.
