[
https://issues.apache.org/jira/browse/NUTCH-1278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13221568#comment-13221568
]
behnam nikbakht commented on NUTCH-1278:
----------------------------------------
i edit this patch to make this changes:
use a class named HostsUtil that manage a xml file, named hosts_conf.xml for
maintaining hosts permanent and temporal informations in a multi thread
environment. this class maintain some of variables like timeout for fetch and
hostcount for generate , ...
this variables can used in fetch and generate and other parts of Nutch.
for adaptive http.timeout in fetch, we simply change some parts of Fetcher.java
and some changes in Protocol.java and it's implementations without disturb them.
> Fetch Improvement in threads per host
> -------------------------------------
>
> Key: NUTCH-1278
> URL: https://issues.apache.org/jira/browse/NUTCH-1278
> Project: Nutch
> Issue Type: New Feature
> Components: fetcher
> Affects Versions: 1.4
> Reporter: behnam nikbakht
> Attachments: NUTCH-1278-v.2.zip, NUTCH-1278.zip
>
>
> the value of maxThreads is equal to fetcher.threads.per.host and is constant
> for every host
> there is a possibility with using of dynamic values for every host that
> influeced with number of blocked requests.
> this means that if number of blocked requests for one host increased, then we
> most decrease this value and increase http.timeout
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
