Hello Metron Guru, I would like to know if there's way to have a profiler that will flush the result to hbase not based on a time period but on some conditions.
For example, if we are tracking a specific sequence over the time for an ip/user like (event A, event b and event C). If this entire sequence happens inside the profil period duration its fine, but if this sequence happens over 2 or more profiler periods then it will not be detected. Moreover if the sequence occured in 1 sec but the profiler period is 15 minutes, then it will wait a long time before being flush to hbase. Another use case will be, if you are looking at the average of something and a threshold is reach to directly flush it and then generate an alert asap. Like number of ssh connection over the last x minutes bigger than N, then flush it. otherwise continue to profile the user. My first question is, is it currently possible to do that, because I have not found how. Secondly if it not feasible for the moment, do you think that might be an useful feature? I was also thinking that the result might be flush to a specific kafka topic and not to hbase. For example, if the profiler detect an anomaly on the behavior like number of ssh connection, or sequence of event, it flush the result to a kafka topic with all the "real-time" alert. Maybe this already exist and profiler is not the good place to do that, to be honest I'm not sure. But I suppose that Im not the first that imagine that, so any comment on how to realize this is welcome :) Best regard, Michel
