GitHub user nickwallen opened a pull request:
https://github.com/apache/metron/pull/716
METRON-1132 Enhance Profiler Debug Functions
Enhanced the `PROFILER_APPLY` function so that multiple messages can be
applied at once. This makes it much simpler to test with larger, likely more
diverse, data sets.
This also enhanced the `StandAloneProfiler` to output the number of
profiles defined, the number of messages applied, and details on how those
messages were applied. This provides a greater understanding of the Profiler's
internals while debugging profiles and also improves the unit tests.
### Testing
Along with the additional unit tests that were added, this was tested
manually in the REPL. To test, follow along.
Create a Profiler configuration.
```
[Stellar]>>> conf := SHELL_EDIT()
[Stellar]>>> conf
{
"profiles": [
{
"profile": "hello-world",
"onlyif": "exists(ip_src_addr)",
"foreach": "ip_src_addr",
"init": { "count": "0" },
"update": { "count": "count + 1" },
"result": "count"
},
{
"profile": "hello-world",
"onlyif": "exists(ip_src_addr)",
"foreach": "ip_src_addr",
"init": { "count": "0" },
"update": { "count": "count + 1" },
"result": "count"
}
]
}
```
Create some sample message data.
```
[Stellar]>>> msgs := SHELL_EDIT()
[Stellar]>>> msgs
[
{
"ip_src_addr": "10.0.0.1",
"protocol": "HTTPS",
"length": "10",
"bytes_in": "234"
},
{
"ip_src_addr": "10.0.0.1",
"protocol": "HTTPS",
"length": "10",
"bytes_in": "234"
},
{
"ip_src_addr": "10.0.0.1",
"protocol": "HTTPS",
"length": "10",
"bytes_in": "234"
},
{
"ip_src_addr": "10.0.0.1",
"protocol": "HTTPS",
"length": "10",
"bytes_in": "234"
}
]
```
Initialize the Profiler.
```
[Stellar]>>> p := PROFILER_INIT(conf)
[Stellar]>>> p
Profiler{1 profile(s), 0 messages(s), 0 route(s)}
```
Apply 3 messages at a time. Notice how the number of messages and routes
increase each time.
```
[Stellar]>>> PROFILER_APPLY(msgs, p)
Profiler{2 profile(s), 8 messages(s), 16 route(s)}
[Stellar]>>> PROFILER_APPLY(msgs, p)
Profiler{2 profile(s), 12 messages(s), 24 route(s)}
[Stellar]>>> PROFILER_APPLY(msgs, p)
Profiler{2 profile(s), 16 messages(s), 32 route(s)}
[Stellar]>>> PROFILER_APPLY(msgs, p)
Profiler{2 profile(s), 20 messages(s), 40 route(s)}
[Stellar]>>> PROFILER_APPLY(msgs, p)
Profiler{2 profile(s), 24 messages(s), 48 route(s)}
[Stellar]>>> PROFILER_APPLY(msgs, p)
Profiler{2 profile(s), 28 messages(s), 56 route(s)}
[Stellar]>>> PROFILER_APPLY(msgs, p)
Profiler{2 profile(s), 32 messages(s), 64 route(s)}
[Stellar]>>> PROFILER_APPLY(msgs, p)
Profiler{2 profile(s), 36 messages(s), 72 route(s)}
[Stellar]>>> PROFILER_APPLY(msgs, p)
Profiler{2 profile(s), 40 messages(s), 80 route(s)}
```
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/nickwallen/metron METRON-1132
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/metron/pull/716.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #716
----
commit cedf8ec802413fb797cb7d98fe7eb8679cc9b841
Author: Nick Allen <[email protected]>
Date: 2017-08-24T21:38:42Z
METRON-1132 Enhance Profiler Debug Functions
commit a3b715b07a0d6b1deab16c5499c219ef42afc14c
Author: Nick Allen <[email protected]>
Date: 2017-08-24T21:48:46Z
Forgot to initialize a counter
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
