Github user mattf-horton commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/343#discussion_r86649179 --- Diff: metron-analytics/metron-profiler/README.md --- @@ -81,21 +139,46 @@ One or more expressions executed when a message is applied to the profile. A ma } ``` -#### `result` +### `result` *Required* A Stellar expression that is executed when the window period expires. The expression is expected to summarize the messages that were applied to the profile over the window period. The expression must result in a numeric value such as a Double, Long, Float, Short, or Integer. -#### `expires` +### `expires` *Optional* A numeric value that defines how many days the profile data is retained. After this time, the data expires and is no longer accessible. If no value is defined, the data does not expire. -### Examples +## Configuring the Profiler -Examples of the types of profiles that can be built include the following. Each shows the configuration that would be required to produce the profile. These examples assume a fictitious input messages that looks something like the following. +The Profiler runs as an independent Storm topology. The configuration for the Profiler topology is stored in Zookeeper at `/metron/topology/profiler`. These properties also exist in the the default installation of Metron at `$METRON_HOME/config/zookeeper/profiler.json`. The values can be changed on disk and then uploaded to Zookeeper using `$METRON_HOME/bin/zk_load_configs.sh`. + +| Setting | Description | +|--- |--- | +| profiler.workers | The number of worker processes to create for the topology. | +| profiler.executors | The number of executors to spawn per component. | +| profiler.input.topic | The name of the Kafka topic from which to consume data. | +| profiler.period.duration | The duration of each profile period. This value should be defined along with `profiler.period.duration.units`. | +| profiler.period.duration.units | The units used to specify the profile period duration. This value should be defined along with `profiler.period.duration`. | +| profiler.hbase.salt.divisor | A salt is prepended to the row key to help prevent hotspotting. This constant is used to generate the salt. Ideally, this constant should be roughly equal to the number of nodes in the Hbase cluster. | +| profiler.hbase.table | The name of the HBase table that profiles are written to. | +| profiler.hbase.column.family | The column family used to store profiles. | +| profiler.hbase.batch | The number of puts that are written in a single batch. | +| profiler.hbase.flush.interval.seconds | The maximum number of seconds between batch writes to HBase. | + +After altering the configuration, start the Profiler. + +``` +$ /usr/metron/0.2.1BETA/start_profiler_topology.sh +``` + +## Examples + +The following examples are intended to highlight the functionality provided by the Profiler. Each shows the configuration that would be required to generate the profile. + +These examples assume a fictitious input messages that looks something like the following. --- End diff -- a fictitious input messages -> a fictitious input message stream
--- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---