I'm guessing that you're parsing historical log data. The http_status
filter's anomaly detection is currently set up to be tracking near real
time logs. Take a look at the following code:
https://github.com/mozilla-services/heka/blob/dev/sandbox/lua/filters/http_status.lua#L93
You'll notice that the `detect` call is passed in the `ns` value that
Heka passes in to the timer_event function. This causes the cbuf to be
updated to the current time, which pushes older data out of the sliding
window.
If you want to test the setup using historical data, you can try making
a copy of this filter and changing it slightly. Try replacing line 93,
linked above, with the following:
local t = status:current_time()
local msg, annos = anomaly_detect(t, title, status, anomaly_config)
This will tell the anomaly detection to base itself from the most recent
row in the cbuf rather than the current system's clock time.
-r
On 02/19/2016 02:48 PM, Justin Clark wrote:
Hi all,
I'm interested in added a few more anomaly detection methods to the
pipeline. Towards that end, I've tried for a while now to just get the
example
(http://hekad.readthedocs.org/en/v0.10.0/getting_started.html#anomaly-detection)
running.
I've followed the Getting Started guide fairly closely and was able to
get the dashboard example running against our own Apache logs pretty
quickly. But as soon as I add an anomaly_config row to my config file, I
lose all data in the dashboard. Remove the row, flush caches, start back
up and the data is back. I've tried with all sorts of detection params
for each of the different algos to no avail.
Looking at the JSON that backs the dashboard, it's all nan's when the
anomaly_config is included.
I've seen this behavior in v0.10.0 and the latest dev.
I don't understand much of the internals of Heka yet, and I don't know a
good way of debugging these Lua modules, but if I comment out this line:
https://github.com/mozilla-services/heka/blob/dev/sandbox/lua/modules/anomaly.lua#L420
I get data back (albeit without annotations). Maybe a pointer in the
right direction?
Thought I'd ask here before I file an issue in case I'm just thick. Any
insight would be appreciated.
- Justin
My config:
[hekad]
maxprocs = 2
share_dir = "/home/jclark/heka/heka_repo/build/heka/share/heka"
base_dir = "/tmp/cache"
[apache_log_input]
type = "LogstreamerInput"
splitter = "TokenSplitter"
decoder = "apache_log_decoder"
log_directory = "/home/jclark/test_data/"
file_match = 'access\.log\.?(?P<Index>\d+)?(.gz)?'
priority = ["^Index"]
[apache_log_decoder]
type = "SandboxDecoder"
filename = "lua_decoders/apache_access.lua"
[apache_log_decoder.config]
log_format = '%v:%p %h %l %u %t \"%r\" %>s %O \"%{Referer}i\"
\"%{User-Agent}i\"'
type = "apache.access"
user_agent_transform = true
user_agent_conditional = true
[http_status_short]
type = "SandboxFilter"
filename = "lua_filters/http_status.lua"
message_matcher = "Type == 'apache.access'"
ticker_interval = 1
preserve_data = true
[http_status_short.config]
sec_per_row = 60
rows = 1800
preservation_version = 1
[http_status_long]
type = "SandboxFilter"
filename = "lua_filters/http_status.lua"
message_matcher = "Type == 'apache.access'"
ticker_interval = 1
preserve_data = true
[http_status_long.config]
sec_per_row = 900
rows = 1800
preservation_version = 0
anomaly_config = 'roc("HTTP Status", 2, 15, 0, 1.5, true, false)
mww_nonparametric("HTTP Status", 5, 15, 10, 0.8)'
[DashboardOutput]
ticker_interval = 1
_______________________________________________
Heka mailing list
Heka@mozilla.org
https://mail.mozilla.org/listinfo/heka
_______________________________________________
Heka mailing list
Heka@mozilla.org
https://mail.mozilla.org/listinfo/heka
