Github user srdo commented on a diff in the pull request:
https://github.com/apache/storm/pull/2200#discussion_r126521238
--- Diff: docs/Metrics.md ---
@@ -125,3 +126,193 @@ The [builtin
metrics]({{page.git-blob-base}}/storm-client/src/jvm/org/apache/sto
[BuiltinMetricsUtil.java]({{page.git-blob-base}}/storm-client/src/jvm/org/apache/storm/daemon/metrics/BuiltinMetricsUtil.java)
sets up data structures for the built-in metrics, and facade methods that the
other framework components can use to update them. The metrics themselves are
calculated in the calling code -- see for example
[`ackSpoutMsg`]({{page.git-blob-base}}/storm-client/src/jvm/org/apache/storm/executor/Executor.java).
+#### Reporting Rate
+
+The rate at which built in metrics are reported is configurable through
the `topology.builtin.metrics.bucket.size.secs` metric. If you set this too
low it can overload the consumers
+and some metrics consumers expect metrics to show up at a fixed rate or
the numbers could be off, so please use caution when modifying this.
+
+
+#### Tuple Counting Metrics
+
+There are several different metrics related to counting what a bolt or
spout does to a tuple. These include things like emitting, transferring,
acking, and failing of tuples.
+
+In general all of these tuple count metrics are randomly sub-sampled
unless otherwise state. This means that the counts you see both on the UI and
from the built in metrics are not necessarily exact. In fact by default we
sample only 5% of the events and estimate the total number of events from that.
The sampling percentage is configurable per topology through the
`topology.stats.sample.rate` config. Setting it to 1.0 will make the counts
exact, but be aware that the more events we sample the slower your topology
will run (as the metrics are counted on the critical path). This is why we
have a 5% sample rate as the default.
+
+The tuple counting metrics are generally reported as maps unless
explicitly stated otherwise. They break down each count for finer grained
reporting.
+The keys to these maps fall into two categories `"${stream_name}"` or
`"${upstream_component}:${stream_name}"`. The former is used for all spout
metrics and for outgoing bolt metrics (`__emit-count` and `__transfer-count`).
The later is used for bolt metrics that deal with incoming tuples.
+
+So for a word count topology the count bolt might show something like the
following for an `__ack-count` metrics
+
+```
+{
+ "split:default": 80080
+}
+```
+
+But the spout would show something more like for the same metric.
+
+```
+{
+ "default": 12500
+}
+```
+
+
+##### `__ack-count`
+
+For bolts it is the number of incoming tuples that had the `ack` method
called on them. For spouts it is the number of tuples that were fully acked.
If acking is disabled this metric is still reported, but it is not really
meaningful.
+
+##### `__fail-count`
+
+For bolts this is the number of incoming tuples that had the `fail` method
called on them. For spouts this is the number of tuples that failed. It could
be because of a tuple timing out or it could be because a bolt called fail on
it. The two are not separated out.
+
+##### `__emit-count`
+
+This is the total number of times the `emit` method was called to send a
tuple. This is the same for both bolts and spouts.
+
+##### `__transfer-count`
+
+This is the total number of tuples transferred to a downstream bolt/spout
for processing. This number will not always match `__emit_count`. If nothing
is registered to receive a tuple down stream the number will be 0 even if
tuples were emitted. Similarly if there are multiple down stream consumers it
may be a multiple of the number emitted. The grouping also can play a role if
it sends the tuple to multiple instances of a single bolt down stream.
+
+##### `__execute-count`
+
+This count metrics is bolt specific. It counts the number of times that a
bolt's `execute` method on a bolt was called.
+
+#### Tuple Latency Metrics
+
+Similar to the tuple counting metrics storm also collects average latency
metrics for bolts and spouts. These follow the same structure as the
bolt/spout maps and are sub-sampled in the same way as well. In all cases the
latency is measured in milliseconds.
+
+##### `__complete-latency`
+
+The complete latency is just for spouts. It is the average amount of time
it took for `ack` or `fail` to be called for a tuple after it was emitted. If
acking is disabled this metric is likely to be blank or 0 for all values, but
should be ignored.
--- End diff --
but -> and
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
