[
https://issues.apache.org/jira/browse/NUTCH-3162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18072810#comment-18072810
]
ASF GitHub Bot commented on NUTCH-3162:
---------------------------------------
lewismc commented on PR #906:
URL: https://github.com/apache/nutch/pull/906#issuecomment-4228963368
The temp Indexer output isn't ideal. Thank you for reviewing. This "Indexer
Latency Merge" is one solution. I will follow up with alternative(s) soon.
Additionally, even documentation the metrics evolution is something I need
to get around to. As of right now I added a notice to the [wiki metrics
documentation](https://cwiki.apache.org/confluence/display/NUTCH/Metrics) to
state it is superseded.
The best current documentation is our Javadoc
*
[NutchMetrics](https://ci-builds.apache.org/job/Nutch/job/Nutch-trunk/javadoc/org/apache/nutch/metrics/NutchMetrics.html)
*
[ErrorTracker](https://ci-builds.apache.org/job/Nutch/job/Nutch-trunk/javadoc/org/apache/nutch/metrics/ErrorTracker.html)
*
[LatencyTracker](https://ci-builds.apache.org/job/Nutch/job/Nutch-trunk/javadoc/org/apache/nutch/metrics/LatencyTracker.html)
I think we can do better than Javadoc and I am thinking about how we package
better documentation.
> Latency metrics to properly merge data from all threads and tasks
> -----------------------------------------------------------------
>
> Key: NUTCH-3162
> URL: https://issues.apache.org/jira/browse/NUTCH-3162
> Project: Nutch
> Issue Type: Bug
> Components: fetcher, indexer, parser
> Affects Versions: 1.22
> Reporter: Sebastian Nagel
> Assignee: Lewis John McGibbney
> Priority: Major
> Fix For: 1.23
>
>
> The latency metrics (NUTCH-3134) have to issues:
> 1. Only the data from one thread is used, in case, a tool is multi-threaded.
> That's definitely the case for Fetcher. The "emitCounters" methods needs to
> increment the counter values, instead of calling "setValue". However, this is
> not the correct approach for the percentiles, see also next point.
> 2. If running full cluster mode with multiple parallel tasks, the task
> counters are summed up to the job counter value. However, the values of the
> latency percentiles then turn out to be too high.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
