[
https://issues.apache.org/jira/browse/DROIDS-120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12996027#comment-12996027
]
Eugen Paraschiv commented on DROIDS-120:
----------------------------------------
The way I see it, the merge(Collection) method is called in CrawlingWorker:
droid.getQueue().merge( outlinks );
Now, if an exception is thrown when merging one of these Links, and if the
exception is allowed to bubble up the stack, then the remaining links in that
collection will no longer be processed. I think it's perfectly OK to lose the
Link that had the problem and generated the exception, but not OK to loose the
entire collection because of a bad link. That is why I proposed to handle the
exception for each Link individually - because of this distinct possibility of
loosing the entire collection (which does happen in production).
Thanks for the feedback.
Eugen.
> Better exception handling in SimpleTaskQueue - merge
> ----------------------------------------------------
>
> Key: DROIDS-120
> URL: https://issues.apache.org/jira/browse/DROIDS-120
> Project: Droids
> Issue Type: Improvement
> Components: core
> Affects Versions: 0.0.1
> Reporter: Eugen Paraschiv
> Fix For: 0.0.1
>
> Attachments: DROIDS-120_v1.patch
>
>
> In SimpleTaskQueue - merge, any exception in any of the individual tasks
> would end up in closing down the entire process and loosing all the results
> in that particular collection.
> This should not be the case - when iterating over a (possibly large)
> collection, where each result is relevant, an exception should be logged and
> the iteration should continue on to the next element.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
