Hi
I have a rotating log file that I am trying to get into heka, using Logstreamer
and PayloadJsonDecoder and have a few questions about it.
1. The rotation naming scheme:
/somelogdir/
(oldest)
mylogfile.log_19700101..something.old
mylogfile.log_2040201..something.old
mylogfile.log_2040202..something.old
....
mylogfile.log
(newest)
I set this up as
[mylog_Streamer]
Type = "LogstreamerInput"
log_directory = "/somelogdir"
file_match =
'mylogfile\.log(_(?P<Year>\d{4})(?P<Month>\d\d)(?P<Day>\d\d)\.(?P<stuff>.*)\.(?P<Ftype>old))?'
priority = ["^Ftype","Year","Month","Day"]
[mylog_Streamer.translation.Ftype]
old = 1
Is there a better way to make the files sort the "right" way? (Ideally, I'd
provide the default values for the missing captures, or at least specify +-
infinity for the sort order)
2. Decoding.
Each line in the file is formatted as:
<version-stamp> <json_data>
e.g.
2 { ...some-json...}
Short of adding a regex extract capability to the PayloadJsonDecoder, is there
an existing way of parsing the JSON into the message fields?
3. Timestamp.
In the above JSON the timestamp is an integer, which represents the number of
milliseconds since the epoch, UTC. I do not believe that it can be handled by a
go time format. Any suggestions on how to best get it converted in?_______________________________________________
Heka mailing list
[email protected]
https://mail.mozilla.org/listinfo/heka
