Re: Need some advice to build a log central.

2014-08-26 Thread vineeth mohan
Hello Sang , Can i know why you are using Hive. I feel you can do the analysis in Elasticsearch itself. Rest seems good to me. Thanks Vineeth On Tue, Aug 26, 2014 at 8:03 AM, Sang Dang zkid...@gmail.com wrote: Hello All, I have selected #2 as my solution. I write data to ES, and

Re: Need some advice to build a log central.

2014-08-25 Thread Sang Dang
Hi Vineeth Mohan, My log central will contain 2 type of log, one is for log to debug/monitor, other is for stats. I have 2 ways to achieve it: #1 , I use only ES, it's ok to log for debug/monitor (using kibana). To do stats, I will build some extra api (base on filter/facet/agregration...)

Re: Need some advice to build a log central.

2014-08-25 Thread Sang Dang
Hello All, I have selected #2 as my solution. I write data to ES, and use kibana+ to realtime monitor. For stats, I use Hive. Each project, I will create a index, for each type of log I will put in a ES Type, ex: ProjectXlog_debug log_error Stats_API

Re: Need some advice to build a log central.

2014-08-24 Thread vineeth mohan
Hello Sang , As this is a question answer forum , we highly recommend you to take a shot yourself and post questions if you have hit a dead end. Thanks Vineeth On Mon, Aug 25, 2014 at 7:56 AM, Sang Dang zkid...@gmail.com wrote: Hi All, I am going to build a log central using