[
https://issues.apache.org/jira/browse/NUTCH-3162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18098026#comment-18098026
]
Hudson commented on NUTCH-3162:
-------------------------------
SUCCESS: Integrated in Jenkins build Nutch ยป Nutch-trunk #254 (See
[https://ci-builds.apache.org/job/Nutch/job/Nutch-trunk/254/])
NUTCH-3162 Fetcher and Parser latency metrics to properly merge data from all
threads and tasks (#906) (github:
[https://github.com/apache/nutch/commit/101328833f02e0ec59d1fb3772065a4c5fcf5ed8])
* (edit) src/test/org/apache/nutch/util/ReducerContextWrapper.java
* (edit) src/java/org/apache/nutch/parse/ParseSegment.java
* (add) src/test/org/apache/nutch/metrics/TestLatencyTracker.java
* (add) src/test/org/apache/nutch/metrics/LatencyTestUtil.java
* (add) src/test/org/apache/nutch/fetcher/TestFetcherReducer.java
* (edit) src/java/org/apache/nutch/fetcher/FetcherThread.java
* (edit) src/java/org/apache/nutch/metrics/ErrorTracker.java
* (edit) src/java/org/apache/nutch/metrics/LatencyTracker.java
* (edit) src/java/org/apache/nutch/fetcher/Fetcher.java
* (edit) src/java/org/apache/nutch/metrics/NutchMetrics.java
* (edit) src/test/org/apache/nutch/parse/TestParseSegment.java
> Fetcher and Parser latency metrics to properly merge data from all threads
> and tasks
> ------------------------------------------------------------------------------------
>
> Key: NUTCH-3162
> URL: https://issues.apache.org/jira/browse/NUTCH-3162
> Project: Nutch
> Issue Type: Bug
> Components: fetcher, indexer, parser
> Affects Versions: 1.22
> Reporter: Sebastian Nagel
> Assignee: Lewis John McGibbney
> Priority: Major
> Fix For: 1.23
>
>
> The latency metrics (NUTCH-3134) have to issues:
> 1. Only the data from one thread is used, in case, a tool is multi-threaded.
> That's definitely the case for Fetcher. The "emitCounters" methods needs to
> increment the counter values, instead of calling "setValue". However, this is
> not the correct approach for the percentiles, see also next point.
> 2. If running full cluster mode with multiple parallel tasks, the task
> counters are summed up to the job counter value. However, the values of the
> latency percentiles then turn out to be too high.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
