This is an automated email from the ASF dual-hosted git repository.
englefly pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new f2117db7497 [fix](fe) Collect ARRAY_MAP container metadata on the
input array (#69340)
f2117db7497 is described below
commit f2117db74978b2bf04f92c11cc2b349936ff9607
Author: minghong <[email protected]>
AuthorDate: Fri Oct 23 11:49:37 2026 +0800
[fix](fe) Collect ARRAY_MAP container metadata on the input array (#69340)
### What problem does this PR solve?
Issue Number: None
Related PR: None
Problem Summary:
With nested column pruning enabled, `CARDINALITY(ARRAY_MAP(x -> x, a))`
over an
`ARRAY<STRUCT<...>>` column fails at execution with
```
[E-3110][INTERNAL_ERROR]Invalid META access path for column 'item': OFFSET
metadata is not supported at this level
```
`EXPLAIN VERBOSE` shows the access path `a.*.OFFSET`: the OFFSET
metadata that `cardinality()` needs has been
pushed down to the struct element layer. Disabling the pruning makes the
same query succeed.
Root cause: `visitCardinality` creates a META/OFFSET context for the
ARRAY_MAP result, and `visitArrayMap`
forwarded that context unchanged into `collectArrayPathInLambda`. The
lambda body reaches the input array
through an array item, and `visitArrayItemSlot` adds the element-level
`*`, so the OFFSET suffix ends up on
the element (struct) layer. The BE only supports OFFSET metadata at
array/map/string level.
ARRAY_MAP keeps the container shape of its inputs: the result offsets
come from the first input array and the
result null map is the union of all inputs' null maps. A metadata
requirement on the ARRAY_MAP result is
therefore a requirement on the input arrays' containers, not on the
elements.
Fix: when ARRAY_MAP is collected with a root META context, route the
NULL/OFFSET requirement to the input
array containers (`a.OFFSET`, `s.arr.OFFSET`, `m.VALUES.OFFSET`,
`a.NULL`) and collect the lambda body with a
fresh DATA context, so the element paths the lambda actually reads are
still kept. Every input array gets the
container metadata, because the result offsets come from the first input
while the result null map is the
union of all inputs' null maps (equal lengths do not imply equal
nullness: `cardinality(array_map((x, y) ->
x + y, [1, 2], NULL))` is NULL).
An input array produced by the map (`map_values(m)` / `map_keys(m)`) is
normalized to the map column: the
produced array's container is the map itself (its length is the map's
entry count and its nullness is the
map's nullness), so `cardinality(array_map(x -> x.v + 1,
map_values(m)))` collects `m.OFFSET` instead of the
invalid `m.VALUES.OFFSET` that the VALUES sub-column of a struct-valued
map cannot serve.
Only an unprefixed `[OFFSET]` / `[NULL]` request describes the ARRAY_MAP
result's own container. A request
that addresses an element of the result, such as the `[*, OFFSET]` of
`cardinality(element_at(array_map(x ->
array(x.v), a), 1))`, keeps the previous lambda traversal instead: that
requirement is produced by the lambda
and cannot be mapped to the input array's element, so only the valid
`a.*.v` payload is read and no
element-level metadata path is emitted.
Before: the plan shows `[a.*.OFFSET]` and the query fails with E-3110.
After: the plan shows `[a.*, a.OFFSET]` and the query returns `2 / 0 /
NULL / 2` for a two-element array, an
empty array, a NULL array and an array holding a NULL element.
Nested variants are fixed the same way:
| expression | before | after |
| --- | --- | --- |
| `cardinality(array_map(x -> x, a))` | `a.*.OFFSET` (fails) | `a.*,
a.OFFSET` |
| struct field `s.arr` | `s.arr.*.OFFSET` (fails) | `s.arr.*,
s.arr.OFFSET` |
| map value `m['k']` | `m.*.*.OFFSET` (fails) | `m.KEYS, m.VALUES.*,
m.VALUES.OFFSET` |
| `array_map(x -> x, a) IS NULL` | element-level `a.*.NULL` | `a.*,
a.NULL` |
| result element: `cardinality(element_at(array_map(x -> array(x.v), a),
1))` | `a.*.v` | `a.*.v` (descendant metadata not routed) |
| multi-array `array_map((x, y) -> ..., a, b)` | `a.*.OFFSET,
b.*.OFFSET` (fails) | `a.*, a.OFFSET, b.*, b.OFFSET` |
### Release note
Fix `[E-3110] Invalid META access path ... OFFSET metadata is not
supported at this level` for
`CARDINALITY(ARRAY_MAP(...))` when nested column pruning is enabled on
array-of-struct columns.
### Check List (For Author)
- Test
- [x] Regression test: new suite
`nereids_rules_p0/column_pruning/array_map_container_metadata_pruning`
(10 result sections, 9 `explain` guards including `notContains
"a.*.OFFSET"`); the whole
`nereids_rules_p0/column_pruning` directory (16 suites) passes.
- [x] Unit Test: `PruneNestedColumnTest`, 7 new cases; 80/80 tests pass.
- [x] Manual test: the reproduction SQL fails with E-3110 before the fix
and returns the expected
`2 / 0 / NULL / 2` after it on a local FE + BE cluster.
- [ ] No need to test.
- Behavior changed:
- [ ] No.
- [x] Yes. `CARDINALITY(ARRAY_MAP(...))` and `ARRAY_MAP(...) IS NULL`
now collect the input array's
OFFSET/NULL metadata instead of an element-level metadata path, and keep
the element data that the
lambda reads.
- Does this need documentation?
- [x] No.
---
.../rewrite/AccessPathExpressionCollector.java | 61 +++++-
.../rules/rewrite/PruneNestedColumnTest.java | 100 ++++++++++
.../array_map_container_metadata_pruning.out | 57 ++++++
.../array_map_container_metadata_pruning.groovy | 220 +++++++++++++++++++++
4 files changed, 435 insertions(+), 3 deletions(-)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/AccessPathExpressionCollector.java
b/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/AccessPathExpressionCollector.java
index b3440c5a8bd..9e98a1c8819 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/AccessPathExpressionCollector.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/AccessPathExpressionCollector.java
@@ -292,14 +292,25 @@ public class AccessPathExpressionCollector extends
DefaultExpressionVisitor<Void
offsetContext.setType(ColumnAccessPathType.META);
// cardinality(map_keys(m)) == cardinality(m) ==
cardinality(map_values(m)):
// all three count map entries, so emit the same [map_col, OFFSET]
path.
- Expression effectiveArg = (arg instanceof MapKeys || arg
instanceof MapValues)
- ? arg.child(0) : arg;
+ Expression effectiveArg = normalizeMapArrayProducer(arg);
return effectiveArg.accept(this, offsetContext);
}
// fall through to default
return visit(cardinality, context);
}
+ /**
+ * map_keys / map_values produce an array whose container shape (length
and nullness) is the
+ * map's own: their metadata requirement belongs to the map column, not to
the KEYS / VALUES
+ * sub-column, which has no offsets of its own (the BE rejects e.g.
`m.VALUES.OFFSET` when the
+ * value is a struct). Returns the map expression for such an array
producer, the expression
+ * itself otherwise.
+ */
+ private static Expression normalizeMapArrayProducer(Expression expression)
{
+ return (expression instanceof MapKeys || expression instanceof
MapValues)
+ ? expression.child(0) : expression;
+ }
+
@Override
public Void visitArrayItemSlot(ArrayItemSlot arrayItemSlot,
CollectorContext context) {
if (exprIdToLambdaArguments.isEmpty()) {
@@ -576,11 +587,55 @@ public class AccessPathExpressionCollector extends
DefaultExpressionVisitor<Void
Expression argument = arrayMap.getArgument(0);
if ((argument instanceof Lambda)) {
- return collectArrayPathInLambda((Lambda) argument, context);
+ Lambda lambda = (Lambda) argument;
+ return collectArrayMapPathInLambda(lambda, context);
}
return visit(arrayMap, context);
}
+ private Void collectArrayMapPathInLambda(Lambda lambda, CollectorContext
context) {
+ if (isRootContainerMetadata(context)) {
+ // A metadata requirement on the ARRAY_MAP result itself (OFFSET
for cardinality(arr),
+ // NULL for arr IS NULL) describes the output ARRAY container, not
the lambda elements.
+ // The output container is shaped by the input arrays: array_map
copies the first
+ // input's offsets and its null map is the union of all inputs'
null maps. Pushing that
+ // context into the lambda body would attach the metadata to the
element layer instead,
+ // producing e.g. `arr.*.OFFSET`, which the BE rejects for a
struct element (OFFSET
+ // metadata is only supported at array/map/string level). Route
the container metadata
+ // to the input arrays, and collect the element paths the lambda
actually reads with a
+ // fresh data context so that the elements stay available to the
lambda body.
+ // A map_keys / map_values input is normalized to its map column:
the produced array's
+ // container is the map itself, so the metadata must not be
attached to the KEYS /
+ // VALUES sub-column (which has no offsets of its own, e.g. for a
struct value).
+ for (ArrayItemReference lambdaArgument :
lambda.getLambdaArguments()) {
+ continueCollectAccessPath(
+
normalizeMapArrayProducer(lambdaArgument.getArrayExpression()),
+ copyContext(context));
+ }
+ return collectArrayPathInLambda(lambda,
+ new CollectorContext(context.statementContext,
context.bottomFilter));
+ }
+ // Only a root request is routed to the input containers. A prefixed
request such as the
+ // `[*, OFFSET]` of cardinality(element_at(array_map(...), 1))
addresses an element of the
+ // ARRAY_MAP result, whose shape is produced by the lambda and is
unrelated to the input
+ // array's containers. Keep the lambda traversal for it: the
unrepresentable metadata is
+ // dropped and the element data the lambda reads is preserved.
+ return collectArrayPathInLambda(lambda, context);
+ }
+
+ /**
+ * Whether the pending metadata requirement addresses the current
expression's own container,
+ * e.g. the `[OFFSET]` of cardinality(arr) or the `[NULL]` of arr IS NULL,
rather than one of
+ * its descendants such as the `[*, OFFSET]` produced by
cardinality(element_at(<array>, 1)).
+ * Every metadata request is created together with exactly one OFFSET /
NULL suffix (see
+ * visitCardinality / visitLength / visitMapSize / visitIsNull), and the
visitors only prepend
+ * the components they descend through, so a single-component metadata
path is a root request.
+ */
+ private static boolean isRootContainerMetadata(CollectorContext context) {
+ return context.getType() == ColumnAccessPathType.META
+ && context.getAccessPathBuilder().getPathList().size() == 1;
+ }
+
@Override
public Void visitArraySort(ArraySort arraySort, CollectorContext context) {
// ARRAY_SORT(lambda, <arr>)
diff --git
a/fe/fe-core/src/test/java/org/apache/doris/nereids/rules/rewrite/PruneNestedColumnTest.java
b/fe/fe-core/src/test/java/org/apache/doris/nereids/rules/rewrite/PruneNestedColumnTest.java
index 18e0b48be1b..ce5e499ebdc 100644
---
a/fe/fe-core/src/test/java/org/apache/doris/nereids/rules/rewrite/PruneNestedColumnTest.java
+++
b/fe/fe-core/src/test/java/org/apache/doris/nereids/rules/rewrite/PruneNestedColumnTest.java
@@ -144,6 +144,16 @@ public class PruneNestedColumnTest extends
TestWithFeService implements MemoPatt
+ " map_arr_col map<string, array<int>>\n"
+ ") properties ('replication_num'='1')");
+ createTable("create table array_map_meta_tbl(\n"
+ + " id int,\n"
+ + " a array<struct<v: int>>\n"
+ + ") properties ('replication_num'='1')");
+
+ createTable("create table array_map_map_producer_tbl(\n"
+ + " id int,\n"
+ + " m map<string, struct<v: int>>\n"
+ + ") properties ('replication_num'='1')");
+
createTable("create table map_array_value_tbl(\n"
+ " id int,\n"
+ " s struct<\n"
@@ -250,6 +260,96 @@ public class PruneNestedColumnTest extends
TestWithFeService implements MemoPatt
ImmutableList.of(path("a", "*")));
}
+ @Test
+ public void testCardinalityArrayMapCollectsContainerOffsetPath() throws
Exception {
+ // ARRAY_MAP keeps the container shape of its inputs, so
cardinality(array_map(...)) must
+ // collect the input array's OFFSET metadata. Passing the OFFSET
requirement into the lambda
+ // body instead yields `a.*.OFFSET`, an OFFSET at the struct element
level that the BE
+ // rejects; the element data the lambda reads must stay collected as
well.
+ assertColumn("select cardinality(array_map(x -> x, a)) from
array_map_meta_tbl",
+ "array<struct<v:int>>",
+ ImmutableList.of(path("a", "*"), metaPath("a", "OFFSET")),
+ ImmutableList.of());
+ }
+
+ @Test
+ public void
testCardinalityArrayMapOnNestedArrayCollectsContainerOffsetPath() throws
Exception {
+ assertColumn("select cardinality(array_map(x -> x, element_at(s,
'arr'))) "
+ + "from nested_container_tbl",
+ "struct<arr:array<struct<str_field:text,int_field:int>>>",
+ ImmutableList.of(
+ path("s", "arr", "*"),
+ metaPath("s", "arr", "OFFSET")),
+ ImmutableList.of());
+ }
+
+ @Test
+ public void testCardinalityArrayMapOnMapValueCollectsValueOffsetPath()
throws Exception {
+ assertAllAccessPathsContain(
+ "select cardinality(array_map(x -> x, element_at(element_at(s,
'm'), 'k'))) "
+ + "from map_array_value_tbl",
+ ImmutableList.of(
+ path("s", "m", "KEYS"),
+ path("s", "m", "VALUES", "*"),
+ metaPath("s", "m", "VALUES", "OFFSET")),
+ ImmutableList.of(
+ metaPath("s", "m", "VALUES", "*", "OFFSET")));
+ }
+
+ @Test
+ public void testIsNullArrayMapCollectsContainerNullPath() throws Exception
{
+ assertAllAccessPathsContain(
+ "select array_map(x -> x, a) is null from array_map_meta_tbl",
+ ImmutableList.of(metaPath("a", "NULL")),
+ ImmutableList.of(metaPath("a", "*", "NULL")));
+ }
+
+ @Test
+ public void testCardinalityArrayMapKeepsElementFieldTheLambdaReads()
throws Exception {
+ assertColumn("select cardinality(array_map(x -> x.v + 1, a)) from
array_map_meta_tbl",
+ "array<struct<v:int>>",
+ ImmutableList.of(path("a", "*", "v"), metaPath("a", "OFFSET")),
+ ImmutableList.of());
+ }
+
+ @Test
+ public void testArrayMapResultElementMetadataKeepsLambdaPayloadOnly()
throws Exception {
+ // cardinality() of a selected ARRAY_MAP element addresses an element
of the ARRAY_MAP
+ // result, whose shape is produced by the lambda, not the input array
container. That
+ // metadata cannot be routed to the input array's element: doing so
emitted the invalid
+ // `a.*.OFFSET` on a struct element, so only the element data the
lambda reads is kept.
+ assertColumn("select cardinality(element_at(array_map(x -> array(x.v),
a), 1)) "
+ + "from array_map_meta_tbl",
+ "array<struct<v:int>>",
+ ImmutableList.of(path("a", "*", "v")),
+ ImmutableList.of());
+
+ assertColumn("select element_at(array_map(x -> array(x.v), a), 1) is
null "
+ + "from array_map_meta_tbl",
+ "array<struct<v:int>>",
+ ImmutableList.of(path("a", "*", "v")),
+ ImmutableList.of());
+ }
+
+ @Test
+ public void testArrayMapOnMapProducerUsesMapContainerMetadata() throws
Exception {
+ // map_values / map_keys produce an array whose container is the map
itself: its length is
+ // the map's entry count. The OFFSET requirement must be collected on
the map column, as
+ // cardinality(map_values(m)) does, instead of the VALUES / KEYS
sub-column, which has no
+ // offsets of its own when the value is a struct (`m.VALUES.OFFSET`
fails in the BE).
+ assertColumn("select cardinality(array_map(x -> x.v + 1,
map_values(m))) "
+ + "from array_map_map_producer_tbl",
+ "map<text,struct<v:int>>",
+ ImmutableList.of(path("m", "VALUES", "v"), metaPath("m",
"OFFSET")),
+ ImmutableList.of());
+
+ assertColumn("select cardinality(array_map(x -> 1, map_keys(m))) "
+ + "from array_map_map_producer_tbl",
+ "map<text,struct<v:int>>",
+ ImmutableList.of(path("m", "KEYS"), metaPath("m", "OFFSET")),
+ ImmutableList.of());
+ }
+
@Test
public void testComparatorArraySortKeepsPayloadPath() throws Exception {
assertColumn("select array_sort((x, y) -> if(cardinality(x) <
cardinality(y), -1, "
diff --git
a/regression-test/data/nereids_rules_p0/column_pruning/array_map_container_metadata_pruning.out
b/regression-test/data/nereids_rules_p0/column_pruning/array_map_container_metadata_pruning.out
new file mode 100644
index 00000000000..4402d6ad189
--- /dev/null
+++
b/regression-test/data/nereids_rules_p0/column_pruning/array_map_container_metadata_pruning.out
@@ -0,0 +1,57 @@
+-- This file is automatically generated. You should know what you did if you
want to edit this
+-- !root_array_map_cardinality --
+1 2 2
+2 0 0
+3 \N \N
+4 2 2
+
+-- !struct_field_array_map_cardinality --
+1 2 2
+2 0 0
+3 \N \N
+4 2 2
+
+-- !map_value_array_map_cardinality --
+1 2 2
+2 0 0
+3 \N \N
+4 2 2
+
+-- !multi_array_array_map_cardinality --
+1 2
+2 0
+3 \N
+4 2
+
+-- !array_map_is_null --
+1 false false
+2 false false
+3 true true
+4 false false
+
+-- !lambda_reads_element_field --
+1 2
+2 0
+3 \N
+4 2
+
+-- !array_map_result_element_cardinality --
+1 1
+2 \N
+3 \N
+4 1
+
+-- !array_map_result_element_is_null --
+1 false
+2 true
+3 true
+4 false
+
+-- !map_values_array_map --
+1 [{"v":1}, {"v":3}] [2, 4] 2
+2 [{"v":10}] [11] 1
+
+-- !map_keys_array_map --
+1 2
+2 1
+
diff --git
a/regression-test/suites/nereids_rules_p0/column_pruning/array_map_container_metadata_pruning.groovy
b/regression-test/suites/nereids_rules_p0/column_pruning/array_map_container_metadata_pruning.groovy
new file mode 100644
index 00000000000..c0b73b1de6f
--- /dev/null
+++
b/regression-test/suites/nereids_rules_p0/column_pruning/array_map_container_metadata_pruning.groovy
@@ -0,0 +1,220 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+suite("array_map_container_metadata_pruning") {
+ sql """ DROP TABLE IF EXISTS array_map_container_metadata_pruning_tbl """
+ sql """
+ CREATE TABLE array_map_container_metadata_pruning_tbl (
+ id INT,
+ a ARRAY<STRUCT<v: INT>>,
+ s STRUCT<arr: ARRAY<STRUCT<v: INT>>>,
+ m MAP<STRING, ARRAY<STRUCT<v: INT>>>
+ ) ENGINE = OLAP
+ DUPLICATE KEY(id)
+ DISTRIBUTED BY HASH(id) BUCKETS 1
+ PROPERTIES ("replication_allocation" = "tag.location.default: 1")
+ """
+ sql """
+ INSERT INTO array_map_container_metadata_pruning_tbl VALUES
+ (1,
+ array(named_struct('v', 1), named_struct('v', 2)),
+ named_struct('arr', array(named_struct('v', 1), named_struct('v',
2))),
+ map('k', array(named_struct('v', 1), named_struct('v', 2)))),
+ (2,
+ CAST([] AS ARRAY<STRUCT<v: INT>>),
+ named_struct('arr', CAST([] AS ARRAY<STRUCT<v: INT>>)),
+ map('k', CAST([] AS ARRAY<STRUCT<v: INT>>))),
+ (3,
+ CAST(NULL AS ARRAY<STRUCT<v: INT>>),
+ CAST(NULL AS STRUCT<arr: ARRAY<STRUCT<v: INT>>>),
+ CAST(NULL AS MAP<STRING, ARRAY<STRUCT<v: INT>>>)),
+ (4,
+ array(CAST(NULL AS STRUCT<v: INT>), named_struct('v', 3)),
+ named_struct('arr', array(CAST(NULL AS STRUCT<v: INT>),
named_struct('v', 3))),
+ map('k', array(CAST(NULL AS STRUCT<v: INT>), named_struct('v',
3))))
+ """
+
+ // ARRAY_MAP keeps the container shape of its inputs: the result offsets
come from the first
+ // input array and the result null map is the union of the inputs' null
maps. A metadata
+ // requirement (OFFSET from cardinality(), NULL from IS NULL) on the
ARRAY_MAP result therefore
+ // has to be collected on the input array container. Collecting it through
the lambda element
+ // instead yields paths such as a.*.OFFSET, which the BE rejects when the
element is a struct.
+ order_qt_root_array_map_cardinality """
+ SELECT id, cardinality(a) AS direct, cardinality(array_map(x -> x, a))
AS mapped
+ FROM array_map_container_metadata_pruning_tbl ORDER BY id
+ """
+
+ order_qt_struct_field_array_map_cardinality """
+ SELECT id,
+ cardinality(element_at(s, 'arr')) AS direct,
+ cardinality(array_map(x -> x, element_at(s, 'arr'))) AS mapped
+ FROM array_map_container_metadata_pruning_tbl ORDER BY id
+ """
+
+ order_qt_map_value_array_map_cardinality """
+ SELECT id,
+ cardinality(element_at(m, 'k')) AS direct,
+ cardinality(array_map(x -> x, element_at(m, 'k'))) AS mapped
+ FROM array_map_container_metadata_pruning_tbl ORDER BY id
+ """
+
+ order_qt_multi_array_array_map_cardinality """
+ SELECT id, cardinality(array_map((x, y) -> x.v + y.v, a, a)) AS mapped
+ FROM array_map_container_metadata_pruning_tbl ORDER BY id
+ """
+
+ order_qt_array_map_is_null """
+ SELECT id, a IS NULL AS direct, array_map(x -> x, a) IS NULL AS mapped
+ FROM array_map_container_metadata_pruning_tbl ORDER BY id
+ """
+
+ // The lambda body still reads element data: only the element field it
references is needed.
+ order_qt_lambda_reads_element_field """
+ SELECT id, cardinality(array_map(x -> x.v + 1, a)) AS mapped
+ FROM array_map_container_metadata_pruning_tbl ORDER BY id
+ """
+
+ // A metadata request that addresses an element of the ARRAY_MAP result
instead of its own
+ // container (cardinality of a selected result element) is produced by the
lambda. It must not
+ // be routed to the input array: that emitted `a.*.OFFSET` on the struct
element and failed
+ // with E-3110. Only the lambda payload is read and the query keeps
working.
+ order_qt_array_map_result_element_cardinality """
+ SELECT id, cardinality(element_at(array_map(x -> array(x.v), a), 1))
AS c
+ FROM array_map_container_metadata_pruning_tbl ORDER BY id
+ """
+
+ order_qt_array_map_result_element_is_null """
+ SELECT id, element_at(array_map(x -> array(x.v), a), 1) IS NULL AS n
+ FROM array_map_container_metadata_pruning_tbl ORDER BY id
+ """
+
+ explain {
+ sql """
+ SELECT cardinality(element_at(array_map(x -> array(x.v), a), 1))
+ FROM array_map_container_metadata_pruning_tbl
+ """
+ contains "a.*.v"
+ notContains "a.*.OFFSET"
+ }
+
+ explain {
+ sql """
+ SELECT element_at(array_map(x -> array(x.v), a), 1) IS NULL
+ FROM array_map_container_metadata_pruning_tbl
+ """
+ contains "a.*.v"
+ notContains "a.*.NULL"
+ }
+
+ // map_values / map_keys produce an array whose container is the map
itself: its length is the
+ // map's entry count. Cardinality over ARRAY_MAP of such an input must
collect the map column's
+ // OFFSET, not `m.VALUES.OFFSET` / `m.KEYS.OFFSET`: the KEYS / VALUES
sub-column has no offsets
+ // of its own when the value is a struct, and the BE rejects that metadata
path.
+ sql """ DROP TABLE IF EXISTS array_map_map_producer_pruning_tbl """
+ sql """
+ CREATE TABLE array_map_map_producer_pruning_tbl (
+ id INT,
+ m MAP<STRING, STRUCT<v: INT>>
+ ) ENGINE = OLAP
+ DUPLICATE KEY(id)
+ DISTRIBUTED BY HASH(id) BUCKETS 1
+ PROPERTIES ("replication_allocation" = "tag.location.default: 1")
+ """
+ sql """
+ INSERT INTO array_map_map_producer_pruning_tbl VALUES
+ (1, map('a', named_struct('v', 1), 'b', named_struct('v', 3))),
+ (2, map('c', named_struct('v', 10)))
+ """
+
+ order_qt_map_values_array_map """
+ SELECT id,
+ map_values(m) AS vals,
+ array_map(x -> x.v + 1, map_values(m)) AS mapped,
+ cardinality(array_map(x -> x.v + 1, map_values(m))) AS len
+ FROM array_map_map_producer_pruning_tbl ORDER BY id
+ """
+
+ order_qt_map_keys_array_map """
+ SELECT id, cardinality(array_map(x -> 1, map_keys(m))) AS len
+ FROM array_map_map_producer_pruning_tbl ORDER BY id
+ """
+
+ explain {
+ sql """
+ SELECT cardinality(array_map(x -> x.v + 1, map_values(m)))
+ FROM array_map_map_producer_pruning_tbl
+ """
+ contains "m.OFFSET"
+ notContains "m.VALUES.OFFSET"
+ }
+
+ explain {
+ sql """
+ SELECT cardinality(array_map(x -> 1, map_keys(m)))
+ FROM array_map_map_producer_pruning_tbl
+ """
+ contains "m.OFFSET"
+ notContains "m.KEYS.OFFSET"
+ }
+
+ explain {
+ sql """
+ SELECT cardinality(array_map(x -> x, a))
+ FROM array_map_container_metadata_pruning_tbl
+ """
+ contains "a.OFFSET"
+ contains "a.*"
+ notContains "a.*.OFFSET"
+ }
+
+ explain {
+ sql """
+ SELECT cardinality(array_map(x -> x, element_at(s, 'arr')))
+ FROM array_map_container_metadata_pruning_tbl
+ """
+ contains "s.arr.OFFSET"
+ notContains "s.arr.*.OFFSET"
+ }
+
+ explain {
+ sql """
+ SELECT cardinality(array_map(x -> x, element_at(m, 'k')))
+ FROM array_map_container_metadata_pruning_tbl
+ """
+ contains "m.VALUES.OFFSET"
+ notContains "m.VALUES.*.OFFSET"
+ }
+
+ explain {
+ sql """
+ SELECT array_map(x -> x, a) IS NULL
+ FROM array_map_container_metadata_pruning_tbl
+ """
+ contains "a.NULL"
+ notContains "a.*.NULL"
+ }
+
+ explain {
+ sql """
+ SELECT cardinality(array_map(x -> x.v + 1, a))
+ FROM array_map_container_metadata_pruning_tbl
+ """
+ contains "a.*.v"
+ contains "a.OFFSET"
+ notContains "a.*.OFFSET"
+ }
+}
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
