I have been thinking about an enhancement to the Profiler for quite some
time. Actually, my first pass at defining this was called "Replay
Telemetry through Profiler" back in METRON-594 [1].
I'd like to first discuss the use case to make sure we start out on the
right foot. Here is how I would define the use cases for this
functionality.
*> Scenario 1: Model Development*
As a Security Data Scientist, I want to understand the historical behaviors
and trends of a profile that I have created so that I can understand if it
is valuable for model building.
There are two possible negative outcomes that the Security Data Scientist
must be aware of when creating profiles.
- The profile might have been defined incorrectly resulting in a feature
set that does not match reality (a bug in the profile definition).
- The profile might have been defined correctly, but the feature set
itself has no predictive value.
Analyzing the profile over archived, historical telemetry allows the
Security Data Scientist to better to mitigate both of these negative
outcomes.
*> Scenario 2: Model Deployment*
As a Security Platform Engineer, I want to generate a profile using
archived telemetry when I deploy a new model to production so that models
depending on that profile can begin to function on day 1.
(Q) Do these make sense? Am I missing anything? Too broad or too narrow?
Once we nail down the use case(s), I'll delete the old JIRA and create a
new JIRA with the use cases. That would give us a place to start on the
technical details of the implementation.
[1] https://issues.apache.org/jira/browse/METRON-594
