Github user cestella commented on the issue:
https://github.com/apache/incubator-metron/pull/449
So, I think this is an interesting approach. My issues with it are:
* You seem to be sending every profile into kafka, not just the configured
ones.
* You seem to be assuming that one value only is being sent into the
telemetry and it's the value that you store in HBase
I'd recommend, rather, that you make the `result` field more complex by
making it a map where the key is the source (e.g. "hbase" or "kafka"). This
allows you to separate the storage structure by storage medium. You may, for
instance, want to *STORE* a stats object in Hbase, but only send along the mean
and standard deviation. Also, I'd recommend allowing `result`to be either a
string (which would presume only hbase is supported) or a Map, which would
explicitly specify the structure for just the sources you want to write to.
Here's a worked example config for maximum clarity (!):
```
{
"profiles": [
{
"profile": "test",
"foreach": "'global'",
"onlyif": "source.type == 'squid'",
"init": { "stats": "STATS_INIT()" },
"update": { "stats": "STATS_ADD(stats, LENGTH(url))" },
"result": {
"hbase" : "stats",
"kafka" : "{ 'mean' : STATS_MEAN(stats), 'stddev' :
STATS_SD(stats) }"
}
}
]
}
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---