[
https://issues.apache.org/jira/browse/CALCITE-4193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17184985#comment-17184985
]
Ruben Q L commented on CALCITE-4193:
------------------------------------
{quote}The Enumerable convention was designed for in-memory data, on the JVM
heap. It wasn’t intended for external algorithms.
In particular, we have no mechanism to serialize and deserialize values, and no
on-disk format, and no scheduler that can handle asynchronous i/o.
{quote}
You are right [~julianhyde], I had not looked at the implementation details. I
am going to adapt the ticket's description accordingly.
> Implement new sort operator: EnumerableExternalSort
> ---------------------------------------------------
>
> Key: CALCITE-4193
> URL: https://issues.apache.org/jira/browse/CALCITE-4193
> Project: Calcite
> Issue Type: Improvement
> Components: core
> Reporter: Ruben Q L
> Priority: Major
>
> Sometimes we need to sort a big volume of data which does not fit into
> memory. In this situation EnumerableSort will cause an OutOfMemoryError.
> The solution for such a scenario will be using a different sorting algorithm:
> [External Sort|https://en.wikipedia.org/wiki/External_sorting].
> The goal of the current ticket is to implement a new operator
> (EnumerableExternalSort) to provide this feature.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
