On Fri, 11 Jul 2014, Tom Perrine wrote:
We run a pretty good sized log infra, log analysis always been one of
my "things".
I cannot agree enough with the "let the logging system handle
timestamps and hostnames" -AND- absolutely use a key=value format for
your information.
If you use Splunk, or any of the other logging tools, they love to
parse key=value to get information from the raw data. I believe that
lots of reporting and visualization tools (like Tableau) will like it
as well, or you can write your own parser and feed into anything.
Having a well defined format is critical, but key=value ends up eating a lot of
space and is rather expensive to parse, so I'm not in agreement that it's always
the best way to format the logs.
David Lang
Also, that means that if you write your own tools, or migrate, or your
logging items mutate or evolve over the years, you'll ALWAYS be able
to parse the logs, and know what each data item is and translate it
into any other format that you or your successors may need.
(If it is still going, SDSC.EDU should have a 20 year syslog baseline
at this point. At one point we did an analysis of the first 10 years
for a few projects. We have a 10 year baseline at $currentjob at this
point and have had to go back up to 5 years for some issues. Planning
for long term use of your logs is not a bad idea. You never know what
you'll want to go back to look at.)
_______________________________________________
Discuss mailing list
[email protected]
https://lists.lopsa.org/cgi-bin/mailman/listinfo/discuss
This list provided by the League of Professional System Administrators
http://lopsa.org/
_______________________________________________
Discuss mailing list
[email protected]
https://lists.lopsa.org/cgi-bin/mailman/listinfo/discuss
This list provided by the League of Professional System Administrators
http://lopsa.org/
