[
https://issues.apache.org/jira/browse/CALCITE-7608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18090443#comment-18090443
]
Julian Hyde edited comment on CALCITE-7608 at 6/21/26 5:45 PM:
---------------------------------------------------------------
If SelectMany is introduced, every rewrite rule that was expecting a Join now
has to account for the fact that it may now instead see a SelectMany. Most of
those deficiencies won't be noticed, so SelectMany will forever feel like a
second-class citizen. (The same could be said for Window currently.)
Logical rels are an algebra. An algebra needs to be minimal. We need to fight,
fight to keep it minimal.
In other words, we tend to treat the rule support as an externality. The true
cost of adding SelectMany is adding support for SelectMany to every single rule
that might encounter it. That would be dozens of rules to be added today and
maintained for years.
The alternative is to make the decorrelator smarter, to handle the cases that
it struggles with today. Yes, it's complicated. But at least the complexity is
in one place.
was (Author: julianhyde):
If SelectMany is introduced, every rewrite rule that was expecting a Join now
has to account for the fact that it may now instead see a SelectMany. Most of
those deficiencies won't be noticed, so SelectMany will forever feel like a
second-class citizen. (The same could be said for Window currently.)
Logical rels are an algebra. An algebra needs to be minimal. We need to fight,
fight to keep it minimal.
In other words, we tend to treat the rule support as an externality. The true
cost of adding SelectMany is adding support for SelectMany to every single rule
that might encounter it. That would be dozens of rules to be added today and
maintained for years.
> Introduce a SelectMany operator
> -------------------------------
>
> Key: CALCITE-7608
> URL: https://issues.apache.org/jira/browse/CALCITE-7608
> Project: Calcite
> Issue Type: Improvement
> Components: core
> Affects Versions: 1.42.0
> Reporter: Mihai Budiu
> Assignee: Mihai Budiu
> Priority: Minor
> Labels: pull-request-available
>
> Today UNNEST is implemented using the Uncollect operator. We propose adding
> an alternative LogicalSelectMany operator, which generalizes Uncollect.
> (Notice that Enumerable API already has a SelectMany.) The main difference
> between Uncollect and SelectMany is that Uncollect unnests all the fields of
> its input relation, whereas LogicalSelectMany would only unnest SOME of the
> fields of the input collection, preserving the other ones in each output row.
> This distinction is very important, because:
> * LogicalSelectMany can be directly and efficiently implemented using the
> Enumerable SelectMany
> * UNNEST used in a cross-join is implemented using an Uncollect and a
> LogicalCorrelate. However, the same UNNEST can be represented using just one
> LogicalSelectMany node
> * Neither the old nor the new decorrelator can actually eliminate
> LogicalCorrelate nodes that are paired with Uncollect. Using
> LogicalSelectMany we can decorrelate many more plans.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
