[ https://issues.apache.org/jira/browse/NUTCH-207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12653412#action_12653412 ]
Todd Lipcon commented on NUTCH-207: ----------------------------------- Are both fetcher and fetcher2 supposed to be supported for the forseeable future? Or could I simply implement this for one of them and not have it integrated until the other is removed in the future? > Bandwidth target for fetcher rather than a thread count > ------------------------------------------------------- > > Key: NUTCH-207 > URL: https://issues.apache.org/jira/browse/NUTCH-207 > Project: Nutch > Issue Type: New Feature > Components: fetcher > Affects Versions: 0.8 > Reporter: Rod Taylor > Attachments: ratelimit.patch > > > Increases or decreases the number of threads from the starting value > (fetcher.threads.fetch) up to a maximum (fetcher.threads.maximum) to achieve > a target bandwidth (fetcher.threads.bandwidth). > It seems to be able to keep within 10% of the target bandwidth even when > large numbers of errors are found or when a number of large pages is run > across. > To achieve more accurate tracking Nutch should keep track of protocol > overhead as well as the volume of pages downloaded. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.