I listed below instance and his heap size details.
Medium instance 3.75 RAM 1 cores Storage :4 GB SSD 64-bit Network
Java heap size: 2gb
R3 Large 15.25 RAM 2 cores Storage :32 GB SSD
Java heap size: 7gb
R3 High-Memory Extra Large r3.xlarge 30.5 RAM 4 cores
Java heap size: 15gb
Thanks
Devaraj
On Friday, February 20, 2015 at 4:15:12 AM UTC+5:30, Mark Walkom wrote:
>
> Don't change cache and buffer sizes unless you know what is happening, the
> defaults are going to be fine.
> How much heap did you give ES?
>
> I'm not sure you can do much about the date filter though, maybe someone
> else has pointers.
>
> On 19 February 2015 at 21:12, Deva Raj <devara...@gmail.com <javascript:>>
> wrote:
>
>> Hi Mark Walkom,
>>
>> I have given below logstash conf file
>>
>>
>> Logstash conf
>>
>> input {
>> file {
>>
>> }
>>
>> }
>>
>> filter {
>> mutate
>> {
>> gsub => ["message", "\n", " "]
>> }
>> mutate
>> {
>> gsub => ["message", "\t", " "]
>> }
>> multiline
>> {
>> pattern => "^ "
>> what => "previous"
>> }
>>
>> grok { match => [ "message",
>> "%{TIME:log_time}\|%{WORD:Message_type}\|%{GREEDYDATA:Component}\|%{NUMBER:line_number}\|
>> %{GREEDYDATA:log_message}"]
>> match => [ "path" ,
>> "%{GREEDYDATA}/%{GREEDYDATA:loccode}/%{GREEDYDATA:_machine}\:%{DATE:logdate}.log"]
>>
>> break_on_match => false
>> }
>>
>>
>> #To check location is S or L
>> if [loccode] == "S" or [loccode] == "L" {
>> ruby {
>> code => " temp = event['_machine'].split('_')
>> if !temp.nil? || !temp.empty?
>> event['_machine'] = temp[0]
>> end"
>> }
>> }
>> mutate {
>>
>> add_field => ["event_timestamp", "%{@timestamp}" ]
>> replace => [ "log_time", "%{logdate} %{log_time}" ]
>> # Remove the 'logdate' field since we don't need it anymore.
>> lowercase=>["loccode"]
>> remove => "logdate"
>>
>> }
>> # to get all site details (site name, city and co-ordinates)
>> sitelocator{sitename => "loccode"
>> datafile=>"vendor/sitelocator/SiteDetails.csv"}
>> date { locale=>"en"
>> match => [ "log_time", "yyyy-MM-dd HH:mm:ss", "MM-dd-yyyy
>> HH:mm:ss.SSS","ISO8601" ] }
>>
>> }
>>
>> output {
>> elasticsearch{
>> }
>>
>> }
>>
>>
>>
>> I have checked step by step to find bottleneck filter. Below filter which
>> took much time. Can you guide me How can I tune it to get faster.
>>
>> date { locale=>"en" match => [ "log_time", "yyyy-MM-dd HH:mm:ss",
>> "MM-dd-yyyy HH:mm:ss.SSS","ISO8601" ] } }
>> <http://serverfault.com/questions/669534/elasticsearch-performance-tuning#comment818613_669558>
>>
>>
>> Thanks
>> Devaraj
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to elasticsearc...@googlegroups.com <javascript:>.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/elasticsearch/7eedf369-b10d-442e-b30d-5e7969bf1c59%40googlegroups.com
>>
>> <https://groups.google.com/d/msgid/elasticsearch/7eedf369-b10d-442e-b30d-5e7969bf1c59%40googlegroups.com?utm_medium=email&utm_source=footer>
>> .
>>
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/5335d517-d7d6-482f-a4b4-6ab06eb13e02%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
