[
https://issues.apache.org/jira/browse/CALCITE-7608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18090397#comment-18090397
]
Mihai Budiu commented on CALCITE-7608:
--------------------------------------
This is exactly the point of this operator: instead of needing a Correlate and
an Unnest you need only one of these.
This is mostly useful for the decorrelator: today the decorrelator is
all-or-nothing, if it cannot remove all Correlate operators, it gives up
removing any of them. And the Correlates paired with Unnest cannot be removed,
because there is no other way to represent them.
The LINQ SelectMap is really flatMap from functional languages, but this
operator is a specialized version, as I pointed out in the comments in the PR:
it does not take an arbitrary function with an iterator result, but rather a
(set) of real collections.
Our compiler backend which uses Calcite as a front-end has a native flatmap
operator, which is used to implement this combination.
One important quality of this operator is that it's a unary operator, whereas
Correlate + Unnest is a binary operator combination.
> Introduce a SelectMany operator
> -------------------------------
>
> Key: CALCITE-7608
> URL: https://issues.apache.org/jira/browse/CALCITE-7608
> Project: Calcite
> Issue Type: Improvement
> Components: core
> Affects Versions: 1.42.0
> Reporter: Mihai Budiu
> Assignee: Mihai Budiu
> Priority: Minor
> Labels: pull-request-available
>
> Today UNNEST is implemented using the Uncollect operator. We propose adding
> an alternative LogicalSelectMany operator, which generalizes Uncollect.
> (Notice that Enumerable API already has a SelectMany.) The main difference
> between Uncollect and SelectMany is that Uncollect unnests all the fields of
> its input relation, whereas LogicalSelectMany would only unnest SOME of the
> fields of the input collection, preserving the other ones in each output row.
> This distinction is very important, because:
> * LogicalSelectMany can be directly and efficiently implemented using the
> Enumerable SelectMany
> * UNNEST used in a cross-join is implemented using an Uncollect and a
> LogicalCorrelate. However, the same UNNEST can be represented using just one
> LogicalSelectMany node
> * Neither the old nor the new decorrelator can actually eliminate
> LogicalCorrelate nodes that are paired with Uncollect. Using
> LogicalSelectMany we can decorrelate many more plans.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
