Hi Rob, Thanks for the response. Comments are inline:
Not sure, really. Either would work. It's not immediately clear, but it > looks like they're both using HTTP POST requests to push their data, which > would work w/ Heka's HttpListenInput. On Linux, *some* of the system data > you're looking to gather can be processed using Heka itself, w/ the > FilePollingInput reading data directly out of /proc. Heka ships w/ some > decoders that know how to parse the contents of these files ( > http://is.gd/r2Fk8j, http://is.gd/vvbFDy, http://is.gd/wMR0W3, > http://is.gd/XQXzxZ), and some filters that know how to process the > output from those decoders (http://is.gd/P7tBqc, http://is.gd/d9zKPB, > http://is.gd/DQGOYU, http://is.gd/A0ymX8). These probably don't cover > everything you want to know, however, so you might need one of the external > tools anyway, in which case you might choose to use the other tool for > consistency. > > For now, I'm using heka to capture log file updates on the client side and push to the heka server. Since they are both heka, I'm guessing they have more efficient ways (other than http) to communicate. Is gob supported? What is the fastest way to connect if I expect to have a large number of heka agents connecting to the heka aggregation server. Is it possible to push the work on parsing the log files to the heka agents so that there is less load on the heka server? In that sense, I really like the use of LUA scripts, since you could push out updated parsers and add support for new log file types wihout have to ship out a new binary for the agent. > > - https://github.com/davidbirdsong/heka-prometheus > - https://github.com/docker-infra/heka-prometheus > > For now, I'm using scollector to push directly to Prometheus. What would be the advantage of pushing from scollector to heka and then to proetheus? As a final note, it should be mentioned that Heka is not a pure Go project. > While most of it is in Go, a lot of what makes Heka powerful is the way it > makes use of the Lua sandbox. The Lua sandbox itself is written in C ( > https://github.com/mozilla-services/lua_sandbox), and the use of said > sandbox, which is the recommended strategy for tackling many of the > problems for which Heka is intended, of course involves using Lua. The > sandbox is the core of the greater Heka ecosystem, and there are other > wrappers around the sandbox, such as Hindsight ( > https://github.com/trink/hindsight), which is written in C. > This is actually what got me attracted to Heka in the first place. I like the sand box capability and the ability to dynamically update and push new scripts without having to restart the agent or reinstall new binaries. Regards, WU
_______________________________________________ Heka mailing list [email protected] https://mail.mozilla.org/listinfo/heka
