Hey guys,
I wanted to bring attention to a tool I created for gathering cluster
details for debugging purposes. There are a number of locations that
properties get materialized, e.g. from Ambari -> properties file -> flux ->
Storm, which means a lot of hunting to guarantee that the changes you've
made are percolating correctly. Furthermore, it's generally useful to get a
sense of how your cluster is configured by gathering all of that info in
one place. I created a Python tool that does just that, and bundles up the
results in a tarball. Here is an overview of the artifacts I'm gathering -
you can see what commands are being used by looking at the script.
Ambari
full cluster config detail
Storm
cluster summary
cluster configuration
topology summary (enrichments and indexing)
topology status summary (enrichments and indexing)
Kafka
broker info
topics list
topic details (enrichments and indexing)
Metron
local file system configuration files
zookeeper configuration
flux files
lib directory file listing
rpm listing
Hadoop
version info
*** Are there any features/details you'd like to see added to this? Any
concerns or suggestions? ***
I am also planning to add log file support along with md5sum of the jar
files deployed in Metron's lib directory.
https://github.com/apache/metron/pull/988/files#diff-0eddfa8f1dd67247e0803e405497b6e2
Cheers,
Mike Miklavcic
