Benchmark and a testbed proxy server
------------------------------------
Key: NUTCH-863
URL: https://issues.apache.org/jira/browse/NUTCH-863
Project: Nutch
Issue Type: New Feature
Affects Versions: 2.0
Reporter: Andrzej Bialecki
Assignee: Andrzej Bialecki
This issue adds two components:
* a testbed proxy server that can serve various content: pre-fetched Nutch
segments, forward requests to original URLs, or create a lot of unique but
predictable fake content (with outlinks) on the fly.
* a simple Benchmark class to measure the time taken to complete several crawl
cycles using fake content.
* 'ant proxy' and 'ant benchmark' targets to execute a benchmark run.
Together these tools should provide a more or less objective method to measure
the end-to-end crawl performance. This initial version can be further
instrumented to collect statistics about various stages of data processing.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
