Github user MohanDV commented on a diff in the pull request: https://github.com/apache/metron/pull/653#discussion_r127390120 --- Diff: metron-analytics/metron-profiler/README.md --- @@ -6,36 +6,94 @@ This is achieved by summarizing the streaming telemetry data consumed by Metron Any field contained within a message can be used to generate a profile. A profile can even be produced by combining fields that originate in different data sources. A user has considerable power to transform the data used in a profile by leveraging the Stellar language. A user only need configure the desired profiles and ensure that the Profiler topology is running. +* [Installation](#installation) * [Getting Started](#getting-started) * [Creating Profiles](#creating-profiles) * [Configuring the Profiler](#configuring-the-profiler) * [Examples](#examples) * [Implementation](#implementation) -## Getting Started +## Installation + +Follow these instructions to install the Profiler. This assumes that core Metron has already been installed and validated. + +1. Build the Metron RPMs by [following these instructions](../../metron-deployment#rpm). -This section will describe the steps required to get your first profile running. + You may have already built the Metron RPMs when core Metron was installed. + + ``` + $ find metron-deployment/ -name "metron-profiler*.rpm" + metron-deployment//packaging/docker/rpm-docker/RPMS/noarch/metron-profiler-0.4.1-201707131420.noarch.rpm + ``` + +1. Copy the Profiler RPM to the installation host. + + The installation host must be the same host on which core Metron was installed. Depending on how you installed Metron, the Profiler RPM might have already been copied to this host with the other Metron RPMs. + + ``` + [root@node1 ~]# find /localrepo/ -name "metron-profiler*.rpm" + /localrepo/metron-profiler-0.4.0-201707112313.noarch.rpm + ``` -1. Stand-up a Metron environment. For this example, we will use the 'Quick Dev' environment. Follow the instructions included with [Quick Dev](../../metron-deployment/vagrant/quick-dev-platform) or build your own. +1. Install the RPM. + + ``` + [root@node1 ~]# rpm -ivh metron-profiler-*.noarch.rpm + Preparing... ########################################### [100%] + 1:metron-profiler ########################################### [100%] + ``` + + ``` + [root@node1 ~]# rpm -ql metron-profiler + /usr/metron + /usr/metron/0.4.1 + /usr/metron/0.4.1/bin + /usr/metron/0.4.1/bin/start_profiler_topology.sh + /usr/metron/0.4.1/config + /usr/metron/0.4.1/config/profiler.properties + /usr/metron/0.4.1/flux + /usr/metron/0.4.1/flux/profiler + /usr/metron/0.4.1/flux/profiler/remote.yaml + /usr/metron/0.4.1/lib + /usr/metron/0.4.1/lib/metron-profiler-0.4.0-uber.jar + ``` + +1. Create a table within HBase that will store the profile data. The table name and column family must match the [Profiler's configuration](#configuring-the-profiler). By default, the table is named `profiler` with a column family `P`. -1. Create a table within HBase that will store the profile data. The table name and column family must match the [Profiler's configuration](#configuring-the-profiler). ``` $ /usr/hdp/current/hbase-client/bin/hbase shell hbase(main):001:0> create 'profiler', 'P' ``` -1. Edit the configuration file located at `$METRON_HOME/config/profiler.properties`. Change the kafka.zk and kafka.broker values from "node1" to the appropriate host name. Keep the same port numbers: +1. Edit the configuration file located at `$METRON_HOME/config/profiler.properties`. ``` kafka.zk=node1:2181 kafka.broker=node1:6667 ``` + Change `kafka.zk` to refer to Zookeeper in your environment. + Change `kafka.broker` to refer to a Kafka Broker in your environment. + +1. Start the Profiler topology. + ``` + $ cd $METRON_HOME + $ bin/start_profiler_topology.sh + ``` + +At this point the Profiler is running and consuming telemetry messages. We have not defined any profiles yet, so it is not doing anything very useful. The next section walks you through the steps to create your very first "Hello, World!" profile. + +## Getting Started -1. Define the profile in a file located at `$METRON_HOME/config/zookeeper/profiler.json`. The following example JSON will create a profile that simply counts the number of messages per `ip_src_addr`, during each sampling interval. +This section will describe the steps required to get your first "Hello, World!"" profile running. This assumes that you have successfully [installed the Profiler](#installation) and have it running. + +1. Create the profile definition in a file located at `$METRON_HOME/config/zookeeper/profiler.json`. --- End diff -- $METRON_HOME/config/zookeeper/profiler.json needs to be created for the first time , it will not exist by default .
--- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---