Ethanlm opened a new pull request #3318:
URL: https://github.com/apache/storm/pull/3318
## What is the purpose of the change
The configuration system on metrics reporters is messy. This PR cleans up
configs and separate them as topology worker metrics vs daemon metrics.
## How was the change tested
1. Add unit tests
2. Tested daemon metric reporters on nimbus with `ConsolePreparableReporter`
and `CsvPreparableReporter` to make sure daemon metric reporters still work
properly
`CsvPreparableReporter`:
```
-bash-4.2$ ls -ltrh /home/y/var/storm/csvmetrics/ |tail
-rw-r--r-- 1 xx yy 494 Aug 4 02:01 nimbus:num-uploadChunk-calls.csv
-rw-r--r-- 1 xx yy 494 Aug 4 02:01
nimbus:num-submitTopologyWithOpts-calls.csv
...
```
`ConsolePreparableReporter`:
```
8/4/20 1:49:01 AM
==============================================================
-- Gauges
----------------------------------------------------------------------
MetricsCleaner:purgeTimestamp
value = 0
nimbus:available-cpu-non-negative
value = 400.0
```
3. Tested worker metric reporters, for example, with this configuration
```
topology.metrics.reporters:
- class: "org.apache.storm.metrics2.reporters.ConsoleStormReporter"
report.period: 10
report.period.units: "SECONDS"
rate.unit: "SECONDS"
locale: "en-US"
- class: "org.apache.storm.metrics2.reporters.CsvStormReporter"
report.period: 10
report.period.units: "SECONDS"
rate.unit: "SECONDS"
locale: "en-US"
csv.log.dir: "./"
```
`ConsoleStormReporter`:
```
2020-08-04 02:33:17.738 c.c.m.ConsoleReporter
metrics-console-reporter-1-thread-1 [INFO] -- Counters
--------------------------------------------------------------------
2020-08-04 02:33:17.738 c.c.m.ConsoleReporter
metrics-console-reporter-1-thread-1 [INFO]
storm.worker.wc13-4-1596508362.hostname.count.default.2.6701-acked
2020-08-04 02:33:17.738 c.c.m.ConsoleReporter
metrics-console-reporter-1-thread-1 [INFO] count = 840
```
`CsvStormReporter`:
```
-bash-4.2$ sudo ls -l
/home/y/var/storm/workers/4d6259dc-ea7c-4048-a44d-be69adceecdd |tail -f
-rw-r----- 1 xx yy 143 Aug 4 02:34
storm.worker.wc13-4-1596508362.hostname.__system.3.6701-receive-queue-sojourn_time_ms.csv
-rw-r----- 1 xx yy 175 Aug 4 02:34
storm.worker.wc13-4-1596508362.hostname.__system.4.6701-receive-queue-arrival_rate_secs.csv
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
