This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch branch-4.2
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-4.2 by this push:
new 982d64dd246 branch-4.2: [fix](fe) Preserve column identity for
forwarded USING keys (branch-4.2) (#69389)
982d64dd246 is described below
commit 982d64dd246c641712fa3be8f7b6fd3f18f93684
Author: 924060929 <[email protected]>
AuthorDate: Thu Oct 22 14:57:58 2026 +0800
branch-4.2: [fix](fe) Preserve column identity for forwarded USING keys
(branch-4.2) (#69389)
Backport of #69388 to branch-4.2. The production repair is unchanged;
the backported tests use Java 8-compatible APIs.
### What problem does this PR solve?
Issue Number: N/A
Related PR: #69388
A qualified reference to a preserved-side USING key can be rejected even
when the same key appears unqualified in GROUP BY. For example:
```sql
SELECT t1.a, COUNT(*)
FROM t1 LEFT JOIN t2 USING(a)
GROUP BY a;
```
The USING-key projection creates a fresh Alias for a column forwarded
directly from the preserved side. SELECT t1.a references the original
ExprId while GROUP BY a references the new one. The same mismatch
affects HAVING and ORDER BY.
Keep the input ExprId for forwarded USING keys while clearing the
qualifier of the unqualified view. Preserve qualified outputs and the
explicit asterisk output list. FULL OUTER JOIN still builds a new
COALESCE expression with its own ExprId, and aggregate validation
remains unchanged.
Sharing the forwarded key identity can repeat a producer ExprId in CTE
output, for example:
```sql
WITH q(x,y,z) AS (
SELECT a,t1.b,t1.a FROM t1 JOIN t2 USING(a)
)
SELECT * FROM q;
```
Keep CTE consumer columns in their original positions using the ordered
consumer-to-producer map. Flattening the reverse multimap groups equal
producer slots and incorrectly changes [x,y,z] into [x,z,y]. Preserve
the ordered map when updating nullability, replacing producer ExprIds
and pruning variant subpaths, so those rewrites cannot reintroduce the
ordering defect.
### Release note
Fix spurious GROUP BY errors when qualified and unqualified references
to a forwarded USING join key are mixed in SELECT, GROUP BY, HAVING, or
ORDER BY. Preserve CTE output column order when the same source column
appears more than once.
### Check List (For Author)
- Test: Unit Test
- Official run-fe-ut.sh: 73 tests, 0 failures/errors across
BindUsingJoinTest, LogicalCTEConsumerTest, AnalyzeCTETest and
EliminateGroupByKeyTest.
- Reproduced the CTE column-order failure before the CTE repair:
expected [x,y,z], actual [x,z,y].
- USING tests cover complete planning across retained sides, mixed
grouping references, HAVING/ORDER BY, expressions, multiple keys,
derived inputs and invalid nullable-side/FULL references.
- Added CTE column-order cases for inner/left/right USING joins, an
actual FE-computed result check for (1,10,1), and repeated-producer
tests covering consumer copies, pruning, nullability changes and
producer-ID replacement.
- Added an EXPLAIN regression case to test_using_join_merge_key; the
same SQL is covered by the FE full-planner test. The regression suite
and BE data execution have not been run locally.
- FE server build through DISABLE_BUILD_UI=ON ./build.sh --fe -j4,
integrated Checkstyle, and git diff --check. Unmodified Web UI excluded.
- Behavior changed: Yes, valid grouping queries plan successfully and
CTE column positions remain intact.
- Does this need documentation: No.
---
.../nereids/rules/analysis/BindExpression.java | 8 +-
.../nereids/rules/rewrite/AdjustNullable.java | 16 ++-
.../nereids/rules/rewrite/EliminateGroupByKey.java | 10 +-
.../rules/rewrite/VariantSubPathPruning.java | 2 +-
.../trees/plans/logical/LogicalCTEConsumer.java | 3 +-
.../nereids/rules/analysis/BindUsingJoinTest.java | 115 +++++++++++++++++----
.../plans/logical/LogicalCTEConsumerTest.java | 104 +++++++++++++++++++
.../query_p0/join/test_using_join_merge_key.groovy | 15 +++
8 files changed, 234 insertions(+), 39 deletions(-)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/analysis/BindExpression.java
b/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/analysis/BindExpression.java
index 04cc43953ee..b86fabbd9c1 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/analysis/BindExpression.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/analysis/BindExpression.java
@@ -1057,7 +1057,9 @@ public class BindExpression implements
AnalysisRuleFactory {
ImmutableList.Builder<NamedExpression> projectExprs =
ImmutableList.builder();
ImmutableList.Builder<NamedExpression> asteriskOutputs =
ImmutableList.builder();
- // 1. Add merged key columns (without qualifier)
+ // 1. Add merged key columns (without qualifier). A forwarded key
keeps the preserved side's
+ // ExprId so qualified and unqualified references denote the same
column in GROUP BY/HAVING/ORDER BY.
+ // Only FULL OUTER JOIN computes a new key and therefore needs a new
Alias.
for (int i = 0; i < leftConjunctsSlots.size(); i++) {
Slot leftSlot = leftConjunctsSlots.get(i);
Slot rightSlot = rightConjunctsSlots.get(i);
@@ -1068,10 +1070,10 @@ public class BindExpression implements
AnalysisRuleFactory {
mergeExpr = new Alias(new Coalesce(leftSlot, rightSlot),
colName);
} else if (joinType.isRightJoin() ||
joinType.isRightSemiOrAntiJoin()) {
// RIGHT OUTER / RIGHT SEMI / RIGHT ANTI: use right side
(preserved side)
- mergeExpr = new Alias(rightSlot, colName);
+ mergeExpr = rightSlot.withQualifier(ImmutableList.of());
} else {
// LEFT OUTER / INNER / LEFT SEMI / LEFT ANTI / CROSS: use
left side
- mergeExpr = new Alias(leftSlot, colName);
+ mergeExpr = leftSlot.withQualifier(ImmutableList.of());
}
projectExprs.add(mergeExpr);
asteriskOutputs.add(mergeExpr);
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/AdjustNullable.java
b/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/AdjustNullable.java
index c8016a493c6..734779de37a 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/AdjustNullable.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/AdjustNullable.java
@@ -426,15 +426,13 @@ public class AdjustNullable extends
DefaultPlanRewriter<Map<ExprId, Slot>> imple
public Plan visitLogicalCTEConsumer(LogicalCTEConsumer cteConsumer,
Map<ExprId, Slot> replaceMap) {
Map<Slot, Slot> consumerToProducerOutputMap = new LinkedHashMap<>();
Multimap<Slot, Slot> producerToConsumerOutputMap =
LinkedHashMultimap.create();
- for (Slot producerOutputSlot :
cteConsumer.getConsumerToProducerOutputMap().values()) {
- Optional<Slot> newProducerOutputSlot =
updateExpression(producerOutputSlot, replaceMap, true);
- for (Slot consumerOutputSlot :
cteConsumer.getProducerToConsumerOutputMap().get(producerOutputSlot)) {
- Slot slot = newProducerOutputSlot.orElse(producerOutputSlot);
- Slot newConsumerOutputSlot =
consumerOutputSlot.withNullable(slot.nullable());
- producerToConsumerOutputMap.put(slot, newConsumerOutputSlot);
- consumerToProducerOutputMap.put(newConsumerOutputSlot, slot);
- replaceMap.put(newConsumerOutputSlot.getExprId(),
newConsumerOutputSlot);
- }
+ for (Map.Entry<Slot, Slot> entry :
cteConsumer.getConsumerToProducerOutputMap().entrySet()) {
+ Slot producerOutputSlot = entry.getValue();
+ Slot slot = updateExpression(producerOutputSlot, replaceMap,
true).orElse(producerOutputSlot);
+ Slot newConsumerOutputSlot =
entry.getKey().withNullable(slot.nullable());
+ producerToConsumerOutputMap.put(slot, newConsumerOutputSlot);
+ consumerToProducerOutputMap.put(newConsumerOutputSlot, slot);
+ replaceMap.put(newConsumerOutputSlot.getExprId(),
newConsumerOutputSlot);
}
return cteConsumer.withTwoMaps(consumerToProducerOutputMap,
producerToConsumerOutputMap);
}
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/EliminateGroupByKey.java
b/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/EliminateGroupByKey.java
index cff93b15b2b..7b9d3b7dd32 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/EliminateGroupByKey.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/EliminateGroupByKey.java
@@ -143,15 +143,15 @@ public class EliminateGroupByKey extends
DefaultPlanRewriter<Map<ExprId, ExprId>
// still references the old ExprId, so we must rebuild both maps with
the new ExprIds.
Map<Slot, Slot> newConsumerToProducer = new LinkedHashMap<>();
Multimap<Slot, Slot> newProducerToConsumer =
LinkedHashMultimap.create();
- for (Slot producerSlot :
cteConsumer.getConsumerToProducerOutputMap().values()) {
+ for (Map.Entry<Slot, Slot> entry :
cteConsumer.getConsumerToProducerOutputMap().entrySet()) {
+ Slot producerSlot = entry.getValue();
ExprId newExprId = resolveExprIdChain(producerSlot.getExprId(),
replaceMap);
Slot effectiveProducerSlot = newExprId != null
? (Slot) producerSlot.withExprId(newExprId)
: producerSlot;
- for (Slot consumerSlot :
cteConsumer.getProducerToConsumerOutputMap().get(producerSlot)) {
- newProducerToConsumer.put(effectiveProducerSlot, consumerSlot);
- newConsumerToProducer.put(consumerSlot, effectiveProducerSlot);
- }
+ Slot consumerSlot = entry.getKey();
+ newProducerToConsumer.put(effectiveProducerSlot, consumerSlot);
+ newConsumerToProducer.put(consumerSlot, effectiveProducerSlot);
}
return cteConsumer.withTwoMaps(newConsumerToProducer,
newProducerToConsumer);
}
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/VariantSubPathPruning.java
b/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/VariantSubPathPruning.java
index 3036a2ba4d9..6ce648f95d8 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/VariantSubPathPruning.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/VariantSubPathPruning.java
@@ -422,7 +422,7 @@ public class VariantSubPathPruning implements
CustomRewriter {
.map(ExpressionTrait::getDataType).noneMatch(VariantType.class::isInstance)) {
return cteConsumer;
}
- Map<Slot, Slot> consumerToProducerOutputMap = Maps.newHashMap();
+ Map<Slot, Slot> consumerToProducerOutputMap =
Maps.newLinkedHashMap();
Multimap<Slot, Slot> producerToConsumerOutputMap =
LinkedHashMultimap.create();
Map<Slot, Map<List<String>, SlotReference>> oriSlotToSubPathToSlot
= Maps.newHashMap();
for (Map.Entry<Slot, Slot> consumerToProducer :
cteConsumer.getConsumerToProducerOutputMap().entrySet()) {
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/logical/LogicalCTEConsumer.java
b/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/logical/LogicalCTEConsumer.java
index be2c41d2605..c631b4fdffa 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/logical/LogicalCTEConsumer.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/logical/LogicalCTEConsumer.java
@@ -53,6 +53,7 @@ public class LogicalCTEConsumer extends LogicalRelation
implements BlockFuncDeps
private final String name;
private final CTEId cteId;
+ // Insertion order is the consumer column order, including repeated
producer slots.
private final Map<Slot, Slot> consumerToProducerOutputMap;
private final Multimap<Slot, Slot> producerToConsumerOutputMap;
@@ -155,7 +156,7 @@ public class LogicalCTEConsumer extends LogicalRelation
implements BlockFuncDeps
@Override
public List<Slot> computeOutput() {
- return ImmutableList.copyOf(producerToConsumerOutputMap.values());
+ return ImmutableList.copyOf(consumerToProducerOutputMap.keySet());
}
@Override
diff --git
a/fe/fe-core/src/test/java/org/apache/doris/nereids/rules/analysis/BindUsingJoinTest.java
b/fe/fe-core/src/test/java/org/apache/doris/nereids/rules/analysis/BindUsingJoinTest.java
index 4fcc69953de..8f447a9bdba 100644
---
a/fe/fe-core/src/test/java/org/apache/doris/nereids/rules/analysis/BindUsingJoinTest.java
+++
b/fe/fe-core/src/test/java/org/apache/doris/nereids/rules/analysis/BindUsingJoinTest.java
@@ -17,6 +17,7 @@
package org.apache.doris.nereids.rules.analysis;
+import org.apache.doris.nereids.exceptions.AnalysisException;
import org.apache.doris.nereids.pattern.GeneratedPlanPatterns;
import org.apache.doris.nereids.rules.RulePromise;
import org.apache.doris.nereids.trees.expressions.Alias;
@@ -24,9 +25,12 @@ import org.apache.doris.nereids.trees.expressions.Expression;
import org.apache.doris.nereids.trees.expressions.NamedExpression;
import org.apache.doris.nereids.trees.expressions.Slot;
import org.apache.doris.nereids.trees.expressions.functions.scalar.Coalesce;
+import org.apache.doris.nereids.trees.plans.ComputeResultSet;
+import org.apache.doris.nereids.trees.plans.physical.PhysicalPlan;
import org.apache.doris.nereids.util.PlanChecker;
import org.apache.doris.utframe.TestWithFeService;
+import com.google.common.collect.ImmutableList;
import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.Test;
@@ -120,13 +124,12 @@ class BindUsingJoinTest extends TestWithFeService
implements GeneratedPlanPatter
// ---- 3. Merge key expression checks ----
@Test
- void testInnerJoinUsingMergeKeyIsLeftAlias() {
- String sql = "SELECT * FROM t1 JOIN t2 USING(a)";
- PlanChecker.from(connectContext)
- .analyze(sql)
- .matches(logicalProject().when(project ->
project.getProjects().stream()
- .filter(p -> p instanceof Alias &&
p.getName().equals("a"))
- .anyMatch(alias -> alias.child(0) instanceof Slot)));
+ void testInnerJoinUsingMergeKeyKeepsLeftExprId() {
+ List<Slot> output = PlanChecker.from(connectContext)
+ .analyze("SELECT a, t1.a FROM t1 JOIN t2
USING(a)").getPlan().getOutput();
+ Assertions.assertEquals(output.get(1).getExprId(),
output.get(0).getExprId());
+ Assertions.assertTrue(output.get(0).getQualifier().isEmpty());
+ Assertions.assertTrue(output.get(1).getQualifier().contains("t1"));
}
@Test
@@ -188,19 +191,11 @@ class BindUsingJoinTest extends TestWithFeService
implements GeneratedPlanPatter
@Test
void testRightJoinMergeKeyFromRightSide() {
- String sql = "SELECT * FROM t1 RIGHT JOIN t2 USING(a)";
- PlanChecker.from(connectContext)
- .analyze(sql)
- .matches(logicalProject().when(project ->
project.getProjects().stream()
- .filter(p -> p instanceof Alias &&
p.getName().equals("a"))
- .map(alias -> alias.child(0))
- .filter(Slot.class::isInstance)
- .map(Slot.class::cast)
- .anyMatch(slot -> {
- // Merge key should come from right table (t2)
- return slot.getQualifier().stream()
- .anyMatch(q -> q.contains("t2"));
- })));
+ List<Slot> output = PlanChecker.from(connectContext)
+ .analyze("SELECT a, t2.a FROM t1 RIGHT JOIN t2
USING(a)").getPlan().getOutput();
+ Assertions.assertEquals(output.get(1).getExprId(),
output.get(0).getExprId());
+ Assertions.assertTrue(output.get(0).getQualifier().isEmpty());
+ Assertions.assertTrue(output.get(1).getQualifier().contains("t2"));
}
// ---- 6. RIGHT SEMI JOIN merge key is visible ----
@@ -405,4 +400,84 @@ class BindUsingJoinTest extends TestWithFeService
implements GeneratedPlanPatter
return outputNames.equals(Arrays.asList("ak", "pk", "lv",
"rv", "nv"));
}));
}
+
+ @Test
+ void testUsingJoinGroupByPreservedKey() {
+ String[] joinTypes = {
+ "JOIN", "LEFT JOIN", "LEFT SEMI JOIN", "LEFT ANTI JOIN",
+ "RIGHT JOIN", "RIGHT SEMI JOIN", "RIGHT ANTI JOIN"
+ };
+ for (String joinType : joinTypes) {
+ String key = joinType.startsWith("RIGHT") ? "t2.a" : "t1.a";
+ String from = " FROM t1 " + joinType + " t2 USING(a)";
+ PlanChecker.from(connectContext).checkPlannerResult(
+ "SELECT " + key + ", COUNT(*)" + from + " GROUP BY a");
+ PlanChecker.from(connectContext).checkPlannerResult(
+ "SELECT a, COUNT(*)" + from + " GROUP BY " + key);
+ PlanChecker.from(connectContext).checkPlannerResult(
+ "SELECT COUNT(*)" + from + " GROUP BY a HAVING " + key + "
> 0 ORDER BY " + key);
+ PlanChecker.from(connectContext).checkPlannerResult(
+ "SELECT " + key + " + 1, COUNT(*)" + from + " GROUP BY a +
1");
+ }
+ }
+
+ @Test
+ void testUsingJoinGroupByMultipleKeys() {
+ PlanChecker.from(connectContext).checkPlannerResult(
+ "SELECT l.a, l.b, COUNT(*) FROM t1 l LEFT JOIN t1 r USING(a,b)
GROUP BY a,b");
+ PlanChecker.from(connectContext).checkPlannerResult(
+ "SELECT a, b, COUNT(*) FROM t1 l LEFT JOIN t1 r USING(a,b)
GROUP BY l.a,l.b");
+ }
+
+ @Test
+ void testUsingJoinGroupByDoesNotMergeNullableSide() {
+ String[] queries = {
+ "SELECT t2.a, COUNT(*) FROM t1 LEFT JOIN t2 USING(a) GROUP BY a",
+ "SELECT t1.a, COUNT(*) FROM t1 RIGHT JOIN t2 USING(a) GROUP BY a",
+ "SELECT t1.a, COUNT(*) FROM t1 FULL JOIN t2 USING(a) GROUP BY a",
+ "SELECT a, COUNT(*) FROM t1 FULL JOIN t2 USING(a) GROUP BY t1.a"
+ };
+ for (String sql : queries) {
+ AnalysisException error =
Assertions.assertThrows(AnalysisException.class,
+ () -> PlanChecker.from(connectContext).analyze(sql), sql);
+ Assertions.assertTrue(error.getMessage().contains("must appear in
the GROUP BY"), error.getMessage());
+ }
+ }
+
+ @Test
+ void testUsingJoinGroupByDerivedInput() {
+ PlanChecker.from(connectContext).checkPlannerResult(
+ "WITH l AS (SELECT 1 AS k, 10 AS v UNION ALL SELECT 2, 20), "
+ + "r AS (SELECT 1 AS k UNION ALL SELECT 3) "
+ + "SELECT l.k, SUM(l.v) FROM l LEFT JOIN r USING(k) "
+ + "GROUP BY k HAVING l.k > 0 ORDER BY l.k");
+ }
+
+ @Test
+ void testUsingJoinCteColumnOrder() {
+ String[] joins = {"JOIN", "LEFT JOIN", "RIGHT JOIN"};
+ for (String join : joins) {
+ String source = join.startsWith("RIGHT") ? "t2.a" : "t1.a";
+ String payload = join.startsWith("RIGHT") ? "t2.c" : "t1.b";
+ String sql = "WITH q(x,y,z) AS (SELECT a," + payload + "," + source
+ + " FROM t1 " + join + " t2 USING(a)) SELECT * FROM q";
+ List<Slot> output =
PlanChecker.from(connectContext).analyze(sql).getPlan().getOutput();
+ Assertions.assertEquals(ImmutableList.of("x", "y", "z"),
output.stream()
+ .map(Slot::getName).collect(Collectors.toList()), sql);
+ PlanChecker.from(connectContext).checkPlannerResult(sql);
+ }
+ }
+
+ @Test
+ void testCteRepeatedColumnResults() {
+ String sql = "WITH q(x,y,z) AS (SELECT a,b,a FROM (SELECT 1 AS a,10 AS
b) t) SELECT * FROM q";
+ PlanChecker.from(connectContext).checkPlannerResult(sql, planner -> {
+ PhysicalPlan plan = planner.getPhysicalPlan();
+ Assertions.assertTrue(plan instanceof ComputeResultSet);
+ Assertions.assertEquals(ImmutableList.of(ImmutableList.of("1",
"10", "1")),
+ ((ComputeResultSet)
plan).computeResultInFe(planner.getCascadesContext(), plan.getOutput())
+ .orElseThrow(() -> new AssertionError("Expected
FE-computed CTE result")).getResultRows());
+ });
+ }
+
}
diff --git
a/fe/fe-core/src/test/java/org/apache/doris/nereids/trees/plans/logical/LogicalCTEConsumerTest.java
b/fe/fe-core/src/test/java/org/apache/doris/nereids/trees/plans/logical/LogicalCTEConsumerTest.java
new file mode 100644
index 00000000000..7b9daaf7cd1
--- /dev/null
+++
b/fe/fe-core/src/test/java/org/apache/doris/nereids/trees/plans/logical/LogicalCTEConsumerTest.java
@@ -0,0 +1,104 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.nereids.trees.plans.logical;
+
+import org.apache.doris.nereids.rules.rewrite.AdjustNullable;
+import org.apache.doris.nereids.rules.rewrite.EliminateGroupByKey;
+import org.apache.doris.nereids.trees.expressions.CTEId;
+import org.apache.doris.nereids.trees.expressions.ExprId;
+import org.apache.doris.nereids.trees.expressions.Slot;
+import org.apache.doris.nereids.trees.expressions.SlotReference;
+import org.apache.doris.nereids.trees.plans.RelationId;
+import org.apache.doris.nereids.types.IntegerType;
+
+import com.google.common.collect.ImmutableList;
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.Test;
+
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+import java.util.stream.Collectors;
+
+class LogicalCTEConsumerTest {
+ private LogicalCTEConsumer newConsumer() {
+ Slot key = new SlotReference("x", IntegerType.INSTANCE, false);
+ Slot payload = new SlotReference("y", IntegerType.INSTANCE, false);
+ LogicalEmptyRelation producer = new LogicalEmptyRelation(new
RelationId(0),
+ ImmutableList.of(key, payload, key.withName("z")));
+ return new LogicalCTEConsumer(new RelationId(1), new CTEId(0), "q",
producer);
+ }
+
+ private void assertColumnOrder(LogicalCTEConsumer consumer) {
+ List<Slot> output = consumer.getOutput();
+ Assertions.assertEquals(ImmutableList.of("x", "y", "z"),
output.stream()
+ .map(Slot::getName).collect(Collectors.toList()));
+
Assertions.assertEquals(consumer.getProducerSlot(output.get(0)).getExprId(),
+ consumer.getProducerSlot(output.get(2)).getExprId());
+
Assertions.assertNotEquals(consumer.getProducerSlot(output.get(0)).getExprId(),
+ consumer.getProducerSlot(output.get(1)).getExprId());
+ }
+
+ @Test
+ void testRepeatedProducerSlotKeepsColumnOrder() {
+ LogicalCTEConsumer consumer = newConsumer();
+ assertColumnOrder(consumer);
+ assertColumnOrder((LogicalCTEConsumer)
consumer.withGroupExpression(Optional.empty()));
+ assertColumnOrder((LogicalCTEConsumer)
consumer.withGroupExprLogicalPropChildren(
+ Optional.empty(), Optional.empty(), ImmutableList.of()));
+ }
+
+ @Test
+ void testPruneOutputsKeepsColumnOrder() {
+ LogicalCTEConsumer consumer = newConsumer();
+ List<Slot> output =
ImmutableList.copyOf(consumer.getConsumerToProducerOutputMap().keySet());
+ LogicalCTEConsumer pruned = (LogicalCTEConsumer) consumer.pruneOutputs(
+ ImmutableList.of(output.get(1), output.get(2)));
+ Assertions.assertEquals(ImmutableList.of("y", "z"),
pruned.getOutput().stream()
+ .map(Slot::getName).collect(Collectors.toList()));
+ }
+
+ @Test
+ void testAdjustNullableKeepsColumnOrder() {
+ LogicalCTEConsumer consumer = newConsumer();
+ Slot producer =
consumer.getConsumerToProducerOutputMap().values().iterator().next();
+ Map<ExprId, Slot> replaceMap = new HashMap<>();
+ replaceMap.put(producer.getExprId(), producer.withNullable(true));
+ LogicalCTEConsumer updated = (LogicalCTEConsumer) new
AdjustNullable(false)
+ .visitLogicalCTEConsumer(consumer, replaceMap);
+ assertColumnOrder(updated);
+ Assertions.assertTrue(updated.getOutput().get(0).nullable());
+ Assertions.assertTrue(updated.getOutput().get(2).nullable());
+ Assertions.assertFalse(updated.getOutput().get(1).nullable());
+ }
+
+ @Test
+ void testReplaceProducerIdKeepsColumnOrder() {
+ LogicalCTEConsumer consumer = newConsumer();
+ Slot producer =
consumer.getConsumerToProducerOutputMap().values().iterator().next();
+ ExprId newId = new ExprId(999);
+ Map<ExprId, ExprId> replaceMap = new HashMap<>();
+ replaceMap.put(producer.getExprId(), newId);
+ LogicalCTEConsumer updated = (LogicalCTEConsumer) new
EliminateGroupByKey()
+ .visitLogicalCTEConsumer(consumer, replaceMap);
+ assertColumnOrder(updated);
+ Assertions.assertEquals(newId,
updated.getProducerSlot(updated.getOutput().get(0)).getExprId());
+ Assertions.assertEquals(newId,
updated.getProducerSlot(updated.getOutput().get(2)).getExprId());
+ }
+}
diff --git
a/regression-test/suites/query_p0/join/test_using_join_merge_key.groovy
b/regression-test/suites/query_p0/join/test_using_join_merge_key.groovy
index 1247afe8f79..7f61ac8e28e 100644
--- a/regression-test/suites/query_p0/join/test_using_join_merge_key.groovy
+++ b/regression-test/suites/query_p0/join/test_using_join_merge_key.groovy
@@ -136,4 +136,19 @@ suite("test_using_join_merge_key", "query_p0") {
FROM l FULL OUTER JOIN r USING(pk) FULL OUTER JOIN n USING(ak)
ORDER BY l.pk NULLS LAST, r.pk NULLS LAST, n.ak NULLS LAST;
"""
+
+ // Qualified and unqualified references to a forwarded USING key must bind
to the same slot.
+ sql """
+ EXPLAIN
+ WITH l AS (
+ SELECT 1 AS k, 10 AS v UNION ALL SELECT 2, 20
+ ), r AS (
+ SELECT 1 AS k UNION ALL SELECT 3
+ )
+ SELECT l.k, SUM(l.v)
+ FROM l LEFT JOIN r USING(k)
+ GROUP BY k
+ HAVING l.k > 0
+ ORDER BY l.k
+ """
}
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
