Benchmark and a testbed proxy server ------------------------------------ Key: NUTCH-863 URL: https://issues.apache.org/jira/browse/NUTCH-863 Project: Nutch Issue Type: New Feature Affects Versions: 2.0 Reporter: Andrzej Bialecki Assignee: Andrzej Bialecki
This issue adds two components: * a testbed proxy server that can serve various content: pre-fetched Nutch segments, forward requests to original URLs, or create a lot of unique but predictable fake content (with outlinks) on the fly. * a simple Benchmark class to measure the time taken to complete several crawl cycles using fake content. * 'ant proxy' and 'ant benchmark' targets to execute a benchmark run. Together these tools should provide a more or less objective method to measure the end-to-end crawl performance. This initial version can be further instrumented to collect statistics about various stages of data processing. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.