assuming I need to accept urls under http://www.dmoz.org/Arts/ and not go
anywhere else, in DroidFactory I would do this:
public static URLFiltersFactory createDefaultURLFiltersFactory() {
URLFiltersFactory filtersFactory = new URLFiltersFactory();
URLFilter defaultURLFilter = new URLFilter() {
final private String prefix = "http://www.dmoz.org/Arts/";
public String filter(String urlString) {
if (urlString.startsWith(prefix))
return urlString;
return null;
}
};
filtersFactory.getMap().put("default", defaultURLFilter);
return filtersFactory;
}
Then would add it to the droid in the unit testing:
private final CrawlingDroid createDroid(final Queue<Link> queue) {
final CrawlingDroid droid = new SysoutCrawlingDroid(queue, null);
final ProtocolFactory protocolFactory = DroidsFactory
.createDefaultProtocolFactory();
droid.setProtocolFactory(protocolFactory);
URLFiltersFactory filtersFactory = DroidsFactory
.createDefaultURLFiltersFactory();
droid.setFiltersFactory(filtersFactory);
final ParserFactory parserFactory = parserSetup();
droid.setParserFactory(parserFactory);
return droid;
}
@Test
public void execute_linkIsParsed() throws DroidsException, IOException,
URISyntaxException {
final Link link = new LinkTask(null, new URI(searchUrl), 1);
this.instance.execute(link);
Mockito.verify(htmlParser).parse(Matchers.any(ContentEntity.class),
Matchers.any(Link.class));
}
However, iterating through the code, doesn't show it's being invoked. Is
there anything else I need to do to make sure this is being invoked
properly ??