[
https://issues.apache.org/jira/browse/DROIDS-56?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13038101#comment-13038101
]
Eugen Paraschiv commented on DROIDS-56:
---------------------------------------
Thanks for committing the sequential task master back. Related to my previous
comments about validation, unfortunately this is still a rather thorny problem
- potentially the filters could server this purpose, but cannot do so within
the current implementation. I will try to give a brief explanation as to why.
The signature of URLFilter only accepts a String (the URL of the task) - the
task itself is not passed to the filter; this means that the filter can only
make a decision based on the URL, unlike a validator which would receive the
entire task. One type of vital information that the task has is the depth,
which was indeed used by an existing validator - MaxDepthTaskValidator. Without
the depth, there is no way to crawl based on depth. This basically means that
the crawling process is using an infinite depth, which is obviously not very
useful at all.
Now for how to fix this - we can either add the validators back into the
process, or modify the signature of the filter (or some other solution I'm not
seeing right now).
Any feedback on this is appreciated.
Thanks.
Eugen.
> Change the TaskQueue interface to java.util.Queue
> -------------------------------------------------
>
> Key: DROIDS-56
> URL: https://issues.apache.org/jira/browse/DROIDS-56
> Project: Droids
> Issue Type: Improvement
> Components: core
> Affects Versions: 0.0.1
> Reporter: Mingfai Ma
> Assignee: Bertil Chapuis
> Fix For: 0.0.1, 0.0.2
>
> Attachments: DROIDS-56-v3.diff, Queue.diff, droids-56-v2.diff,
> droids-56.diff, taskQueue-to-queue.diff
>
>
> 1. any code that depends on the queue should depend on java.util.Queue<T>
> rather than TaskQueue.
> 2. either remove the TaskQueue or make it extends java.util.Queue
> - it means SimpleTaskQueue should extends a Queue implementation (or
> implement all method of a Queue)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
