Github user anandsubbu commented on the issue:
https://github.com/apache/metron/pull/864
Hi @nickwallen , I tried this on a 12-node cluster. I validated that
`clusterHostInfo` is populated properly for the alerts_ui, management_ui and
rest_ui hosts.
However, in my case it failed on the parser service check since the 'Metron
Check' step landed on a host without Kafka broker installed.
Here's the error excerpt:
```
<snip>
2017-12-20 18:42:54,285 - Performing Parser service check
2017-12-20 18:42:54,285 - Checking for grok patterns in HDFS for Parsers
2017-12-20 18:42:54,285 - Checking HDFS; directory=/apps/metron/patterns
user=metron
2017-12-20 18:42:54,285 - Execute['/usr/hdp/2.5.3.0-37/hadoop/bin/hdfs dfs
-test -d /apps/metron/patterns'] {'logoutput': True, 'path':
['/usr/sbin:/sbin:/usr/local/bin:/bin:/usr/bin'], 'tries': 3, 'user': 'metron',
'try_sleep': 5}
2017-12-20 18:42:56,822 - Checking Kafka topics for Parsers
2017-12-20 18:42:56,822 - Checking existence of Kafka topic 'bro'
2017-12-20 18:42:56,823 -
Execute['/usr/hdp/current/kafka-broker/bin/kafka-topics.sh --zookeeper
metronc-1.openstacklocal:2181,metronc-11.openstacklocal:2181,metronc-10.openstacklocal:2181
--list | awk 'BEGIN {cnt=0;} /bro/ {cnt++} END {if (cnt > 0) {exit
0} else {exit 1}}''] {'logoutput': True, 'path':
['/usr/sbin:/sbin:/usr/local/bin:/bin:/usr/bin'], 'tries': 3, 'user': 'kafka',
'try_sleep': 5}
-bash: /usr/hdp/current/kafka-broker/bin/kafka-topics.sh: No such file or
directory
2017-12-20 18:42:56,900 - Retrying after 5 seconds. Reason: Execution of
'/usr/hdp/current/kafka-broker/bin/kafka-topics.sh --zookeeper
metronc-1.openstacklocal:2181,metronc-11.openstacklocal:2181,metronc-10.openstacklocal:2181
--list | awk 'BEGIN {cnt=0;} /bro/ {cnt++} END {if (cnt > 0) {exit
0} else {exit 1}}'' returned 1. -bash:
/usr/hdp/current/kafka-broker/bin/kafka-topics.sh: No such file or directory
-bash: /usr/hdp/current/kafka-broker/bin/kafka-topics.sh: No such file or
directory
2017-12-20 18:43:01,987 - Retrying after 5 seconds. Reason: Execution of
'/usr/hdp/current/kafka-broker/bin/kafka-topics.sh --zookeeper
metronc-1.openstacklocal:2181,metronc-11.openstacklocal:2181,metronc-10.openstacklocal:2181
--list | awk 'BEGIN {cnt=0;} /bro/ {cnt++} END {if (cnt > 0) {exit
0} else {exit 1}}'' returned 1. -bash:
/usr/hdp/current/kafka-broker/bin/kafka-topics.sh: No such file or directory
-bash: /usr/hdp/current/kafka-broker/bin/kafka-topics.sh: No such file or
directory
Command failed after 1 tries
<snip>
```
I noticed that the `clusterHostInfo` indeed has a list of the
`kafka_broker_hosts` (see attached
[clusterHostInfo-12-node.txt](https://github.com/apache/metron/files/1576752/clusterHostInfo-12-node.txt).
Would it be possible to either a) force Ambari to run metron service check on
one of the kafka broker hosts; or b) run
[check_kafka_topics](https://github.com/apache/metron/blob/master/metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/scripts/metron_service.py#L259)
on a `kafka_broker_host`
I am perfectly fine if you think the kafka_broker fix should be a different
PR than this.
---
