[ https://issues.apache.org/jira/browse/CALCITE-4193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17184952#comment-17184952 ]
Julian Hyde commented on CALCITE-4193: -------------------------------------- The Enumerable convention was designed for in-memory data, on the JVM heap. It wasn’t intended for external algorithms. In particular, we have no mechanism to serialize and deserialize values, and no on-disk format, and no scheduler that can handle asynchronous i/o. We ought to be putting our effort into building a convention with a byte-oriented data format. > Implement new sort operator: EnumerableExternalSort > --------------------------------------------------- > > Key: CALCITE-4193 > URL: https://issues.apache.org/jira/browse/CALCITE-4193 > Project: Calcite > Issue Type: Improvement > Components: core > Reporter: Ruben Q L > Priority: Major > > Sometimes we need to sort a big volume of data which does not fit into > memory. In this situation EnumerableSort will cause an OutOfMemoryError. > The solution for such a scenario will be using a different sorting algorithm: > [External Sort|https://en.wikipedia.org/wiki/External_sorting]. > The goal of the current ticket is to implement a new operator > (EnumerableExternalSort) to provide this feature. -- This message was sent by Atlassian Jira (v8.3.4#803005)