Create a Fetcher for crawler to replace/wrap the use of Protocol+Worker
-----------------------------------------------------------------------
Key: DROIDS-57
URL: https://issues.apache.org/jira/browse/DROIDS-57
Project: Droids
Issue Type: New Feature
Affects Versions: 0.01
Reporter: Mingfai Ma
- the main argument is, "fetcher" is a common terminology in crawling
- a fetcher seems to be only required by droids-crawler but not in core.
- this task is more about the creation of an interface and a concept. fetching
function exists already in HttpProtocol.
the fetcher shall have a "fetch()" method. I propose the following 2 main
interface. let it be evolved over time
public Fetcher<T> fetch(T link) throws FetcherException;
public Fetcher<T> fetch(T link, Map params) throws FetcherException;
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
