Markus Jelsma created NUTCH-2214: ------------------------------------ Summary: Index clean to be flexible on what it deletes Key: NUTCH-2214 URL: https://issues.apache.org/jira/browse/NUTCH-2214 Project: Nutch Issue Type: Improvement Affects Versions: 1.11 Reporter: Markus Jelsma Assignee: Markus Jelsma Fix For: 1.13
Nutch clean removes all useless records, but if Nutch is configured correctly (-deleteGone etc), the index should only contain duplicates, if existing. On a large index, this could result in Nutch sending millions of getById's to Solr, for records that don't exist in the first place. This issue will make it configurable on what to delete, e.g. useless records (404, 30x) or duplicates. -- This message was sent by Atlassian JIRA (v6.3.4#6332)