Hello,
I've only been using heka for a couple months, and this is my first time
posting to this mailing list. I've been running into the following issue
with logstreamer input, and was hoping someone on here might point me in
the right direction.
Every so often, when logrotate rotates logs, the logstreamer input appears
to start tailing the oldest log, which gets deleted during the rotate, and
then it will just get stuck there and not make any progress.
Here is my logstreamer config:
[MongoTailer]
type = "LogstreamerInput"
log_directory = "/data/log/mongo"
journal_directory = "/data/appdata/heka"
# expect files named mongod.<rset>.log[-YYYYMMDD]
file_match =
'mongod\.(?P<Rset>\S+)\.log-?((?P<Year>\d{4})(?P<Month>\d{2})(?P<Day>\d{2}))?'
differentiator = ["all.", "Rset"]
priority = ["Year", "Month", "Day"]
# make the log ending in .log the most recent
[MongoTailer.translation.Year]
missing = 9999
Running heka-logstreamer on the config shows what I expect:
Logstream name: [all.venues0]
Files: 8 (printing oldest to newest)
/data/log/mongo/mongod.venues0.log-20160121
/data/log/mongo/mongod.venues0.log-20160122
/data/log/mongo/mongod.venues0.log-20160123
/data/log/mongo/mongod.venues0.log-20160124
/data/log/mongo/mongod.venues0.log-20160125
/data/log/mongo/mongod.venues0.log-20160126
/data/log/mongo/mongod.venues0.log-20160127
/data/log/mongo/mongod.venues0.log
During normal operation, looking at the journal file also shows what I
expect:
{"seek":79749937,"file_name":"/data/log/mongo/mongod.venues0.log","last_hash":"be40779a9aaa7371ee8a53fc448ba10d7c084fe3"}
But, sometimes when logrotate does rotation, the journal file will then
show:
{"seek":55436986,"file_name":"/data/log/mongo/mongod.venues0.log-20160120","last_hash":"ad99fe64c5f6cba6a2a197dfa14cc10c583cf712"}
The file it appears to tail no longer exists, and it will just sit there
until I bounce hekad. I'm not sure why it would go from tailing the newest
file to tailing the oldest file. After bouncing it will go back through all
the old logs again.
Any ideas what could be happening?
_______________________________________________
Heka mailing list
Heka@mozilla.org
https://mail.mozilla.org/listinfo/heka
