Thanks Antonio for sharing this.
I believe this could be one of the interesting case studies for Solr In
Action, if you are interested in sharing a bit more - I am sure the
authors would be more interested for upcoming revisions.
--
K K.
On 02/12/2010 06:02 PM, Antonio Lobato wrote:
Hey everyone, I don't actually have a question, but I just thought I'd
share something really cool that I did with Solr for our company.
We run a good amount of servers, well into the several hundreds, and
naturally we need a way to centralize all of the system logs. For a
while we used a commercial solution to centralize and search our logs,
but they wanted to charge us tens of thousands of dollars for just one
gigabyte/day more of indexed data. So I said forget it, I'll write my
own solution!
We already use Solr for some of our other backend searching systems,
so I came up with an idea to index all of our logs to Solr. I wrote a
daemon in perl that listens on the syslog port, and pointed every
single system's syslog to forward to this single server. From there,
this daemon will write to a Solr indexing server after parsing them
into fields, such as date/time, host, program, pid, text, etc. I then
wrote a cool javascript/ajax web front end for Solr searching, and
bam. Real time searching of all of our syslogs from a web interface,
for no cost!
Just thought this would be a neat story to share with you all. I've
really grown to love Solr, it's something else!
Thanks,
-Antonio
