Refactoring of Generator, run all urls through checks
-----------------------------------------------------
Key: NUTCH-606
URL: https://issues.apache.org/jira/browse/NUTCH-606
Project: Nutch
Issue Type: Bug
Components: generator
Environment: all
Reporter: Dennis Kubes
Priority: Minor
Fix For: 1.0.0
Refactor the generator to make sure all host run through checks such as host
and protocol checks, ip checks if necessary. Currently the generator only does
this for urls if generate.max.per.host > 0 which by default is -1. So by
default all urls will get collected without checks.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
