[
https://issues.apache.org/jira/browse/NUTCH-1347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13265804#comment-13265804
]
Julien Nioche commented on NUTCH-1347:
--------------------------------------
bq. i can not recognize your solution
that's probably because you haven't really explained what the problem is. Are
you seeing URLs from the same host being put in different queues?
bq. also i found a bug in selector reduce task in generate phase, that result
from less of coordination between tasks.
then please open a separate issue for this and include a clear description so
that others can reproduce the problem or at least understand it
> fetcher politeness related to map-reduce
> ----------------------------------------
>
> Key: NUTCH-1347
> URL: https://issues.apache.org/jira/browse/NUTCH-1347
> Project: Nutch
> Issue Type: Improvement
> Components: fetcher
> Affects Versions: 1.4
> Reporter: behnam nikbakht
> Labels: fetch
>
> when Nutch is running on Hadoop , based on map-reduce concept, each map task
> do some thing on it's owned data, so, each fetcher map-task work with it's
> Queues and do not know any thing about other Queus. so, enforce delay between
> successive requests and maximum concurrent requests policies on it's Queues.
> but with a simple test we found that it's not good piliteness mechanism when
> we have multiple map tasks.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
