[
https://issues.apache.org/jira/browse/NUTCH-207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12653404#action_12653404
]
Dennis Kubes commented on NUTCH-207:
------------------------------------
I think this would be an interesting addition. It would also need to be ported
to fetcher2 as well as fetcher. It you want to take on the task of porting it
that would be great. If you have any questions feel free to ask.
> Bandwidth target for fetcher rather than a thread count
> -------------------------------------------------------
>
> Key: NUTCH-207
> URL: https://issues.apache.org/jira/browse/NUTCH-207
> Project: Nutch
> Issue Type: New Feature
> Components: fetcher
> Affects Versions: 0.8
> Reporter: Rod Taylor
> Attachments: ratelimit.patch
>
>
> Increases or decreases the number of threads from the starting value
> (fetcher.threads.fetch) up to a maximum (fetcher.threads.maximum) to achieve
> a target bandwidth (fetcher.threads.bandwidth).
> It seems to be able to keep within 10% of the target bandwidth even when
> large numbers of errors are found or when a number of large pages is run
> across.
> To achieve more accurate tracking Nutch should keep track of protocol
> overhead as well as the volume of pages downloaded.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
