Hi, all. So, I finally got a custom Lua decoder to parse events coming in via TcpInput. On the other end, it's just nxlog watching a bunch of log files and then sending the new events to Heka via TCP. The log files are all named SystemOut.log and are located throughout a directory tree, one file per server.
Once the data gets to Heka, Heka sends it to Elasticsearch and I am viewing the results in Kibana. I started examining the data and found that I had no way of determining which SystemOut.log file a given event came from. :-) I have been playing with nxlog, rewriting the message on that end and then letting Heka parse out the SystemOut.log path that I'm tacking onto the end of the message. That seemed really wrong, though. I thought, if I'm doing this -- modifying the data at all -- I should really be doing it in a structured way, like using nxlog's JSON output and then adding an attribute for "filepath" or something. Which begs the question, what is the right way to handle this on the Heka side?? As I said, I already wrote the Lua decoder that parses out the payload and the funny timestamp so I'm thinking I'll still have to use it. But should I use that to parse out the JSON? It doesn't look like the Lua standard library has JSON parsing. Should I use one decoder to parse the JSON and the custom Lua decoder to parse out the timestamp? Seems like this would be simple to do in a Go plug-in. TIA, Ali
_______________________________________________ Heka mailing list [email protected] https://mail.mozilla.org/listinfo/heka
