[
https://issues.apache.org/jira/browse/JENA-44?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12994852#comment-12994852
]
Paolo Castagna commented on JENA-44:
------------------------------------
Indeed, it would be beneficial to be able to cancel a large sort (even when it
is an external sort).
JENA-29 adds cancel, but if cancel is called while ARQ is performing a large
in-memory sort, the cancel will not work until the sort has finished (or the
machine has crashed or become unresponsive doing GC). We have experience these
sort of problems.
Therefore, JENA-29 and JENA-44 are complementary and they could benefit from
each others.
We are already seeing the benefit from JENA-44, since it put a cap, in
practice, on the amount of memory used by ARQ to do large sorts.
Moreover, JENA-44 could use same mechanism to serialize/deserialize bindings
from JENA-45, if that is going to go intro trunk before JENA-44.
> Support external sorting of bindings in ARQ
> -------------------------------------------
>
> Key: JENA-44
> URL: https://issues.apache.org/jira/browse/JENA-44
> Project: Jena
> Issue Type: New Feature
> Components: ARQ
> Reporter: Sam Tunnicliffe
> Priority: Minor
> Attachments: JENA-44-0.patch
>
>
> In QueryIterSort, the sorting of the contents of an Iterator<Binding> is done
> in memory, using Arrays.sort. This can be problematic where the set to be
> sorted is large. A possible solution could be to use an external, disk-backed
> algorithm. A hybrid approach may be better, whereby we attempt the in-memory
> sort, but when the number of bindings encountered goes over a certain number,
> resort to the disk-backed variant.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
