Michael Joyce created NUTCH-2155: ------------------------------------ Summary: Create a "crawl completeness" utility Key: NUTCH-2155 URL: https://issues.apache.org/jira/browse/NUTCH-2155 Project: Nutch Issue Type: Improvement Components: util Affects Versions: 1.10 Reporter: Michael Joyce Fix For: 1.12
I've found it useful to have a tool for dumping some "completeness" information from a crawl similar to how domainstats does but including fetched and unfetched counts per domain/host. This is especially nice when doing vertical crawls over a few domains or just to see how much of a host/domain you've covered with your crawl so far. -- This message was sent by Atlassian JIRA (v6.3.4#6332)