Github user mattf-horton commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/343#discussion_r86648252 --- Diff: metron-analytics/metron-profiler/README.md --- @@ -1,16 +1,74 @@ # Metron Profiler -The Profiler is a feature extraction mechanism that can generate a profile describing the behavior of an entity on a network. An entity might be a server, user, subnet or application. Once a profile has been generated defining what normal behavior looks-like, models can be built that identify anomalous behavior. +The Profiler is a feature extraction mechanism that can generate a profile describing the behavior of an entity. An entity might be a server, user, subnet or application. Once a profile has been generated defining what normal behavior looks-like, models can be built that identify anomalous behavior. This is achieved by summarizing the streaming telemetry data consumed by Metron over sliding windows. A summary statistic is applied to the data received within a given window. Collecting this summary across many windows results in a time series that is useful for analysis. -## Usage +Any field contained within a message can be used to generate a profile. A profile can even be produced by combining fields that originate in different data sources. A user has considerable power to transform the data used in a profile by leveraging the Stellar language. A user only need configure the desired profiles and ensure that the Profiler topology is running. -Any field contained within a message can be used to generate a profile. A profile can even be produced from combining fields that originate in different data sources. A user has considerable power to transform the data used in a profile by leveraging the Stellar language. A user only need configure the desired profiles in Zookeeper and ensure that the Profiler topology is running. +* [Getting Started](#getting-started) +* [Creating Profiles](#creating-profiles) +* [Configuring the Profiler](#configuring-the-profiler) +* [Examples](#examples) +* [Implementation](#implementation) -### Configuration +## Getting Started + +This section will describe the steps required to get your first profile running. + +1. Stand-up a Metron environment. For this example, we will use the 'Quick Dev' environment. Follow the instructions included with [Quick Dev](../../metron-deployment/vagrant/quick-dev-platform) or build your own. + +1. Create a table within HBase that will store the profile data. The table name and column family must match the [Profiler's configuration](#configuring-the-profiler). + ``` + $ /usr/hdp/current/hbase-client/bin/hbase shell + hbase(main):001:0> create 'profiler', 'P' + ``` + +1. Define the profile in a file located at `$METRON_HOME/config/zookeeper/profiler.json`. The following JSON will create a profile that simply counts the number of messages. --- End diff -- Suggest edits: following JSON -> following example JSON number of messages -> number of messages per ip_src_addr, during each sampling interval.
--- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---