Hello everyone, I am new to Elastic Search world and I am evaluating it for storing and retrieval of my application logs, which is currently being stored in postgres database. For performance and disk space reasons, I plan to switch logs storage from posgres to elastic search. I plan to store log information for various applications as different indices in ES.
But when I implemented this tool, I noticed lots of cpu spike on my system when it received the request for inserting logs data into ES. I am using Linux and allocated ES a 2GB RAM. I am using REST service calls to push logs data into ES. There are about 5000 records being inserted into one index for every request into the application (all in JSON format). Refresh interval is kept to 30 seconds in elasticsearch.yml file Number of shards configured to 1 and replicas to 0. There are about 20 index created for different applications, which might eventually grow upto 50 (as it is per application) Also, advise, if it should be separate index per application, or single index storing logs of all of the applications. I expect about 3k to 5k log messages per any request in any application Single node is being used for storing logs of multiple applications running on same node under separate JVM JDK: 1.7 ES_HEAP_SIZE=2g Also configured, mlockall to true in elasticsearch.yml file. I am clueless in identifying root cause of cpu spike. Can anyone please suggest a way out in further troubleshooting and determining the root cause. What should be the next steps to troubleshoot the same. ES version: 1.4.2 Please let me know, if you need any further information. Appreciate your inputs and help! Regards, Sagar Shah -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/aa4b625b-4e5d-4f6a-b92f-9b9c23cb3e4f%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
