[
https://issues.apache.org/jira/browse/FLINK-34656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18142965#comment-18142965
]
Ramin Gharib commented on FLINK-34656:
--------------------------------------
The same root cause shows up without a NULL array, in plain SQL, and also for
MAP lookups and ELEMENT. Reproduced on master, identical in batch and streaming
mode:
{code:sql}
SELECT ARRAY[10,20,30][4], ARRAY[10,20,30][4] IS NULL;
SELECT x, ARRAY[10,20,30][x], COALESCE(ARRAY[10,20,30][x], 0) FROM (VALUES (3),
(4)) AS t(x);
SELECT x FROM (VALUES (2), (4)) AS t(x) WHERE ARRAY[10,20,30][x] IS NULL;
SELECT MAP['a', 1]['b'];
SELECT ELEMENT(ARRAY_REMOVE(ARRAY[1], 1));
{code}
||Expression||Expected||Actual||
|{{ARRAY\[10,20,30\]\[4\]}}|NULL|-1|
|{{ARRAY\[10,20,30\]\[4\] IS NULL}}|TRUE|FALSE|
|{{ARRAY\[10,20,30\]\[x\]}} for x = 4|NULL|-1|
|{{COALESCE(ARRAY\[10,20,30\]\[x\], 0)}} for x = 4|0|0|
|{{WHERE ARRAY\[10,20,30\]\[x\] IS NULL}} over x = 2, 4|row 4|no rows|
|{{MAP\['a', 1\]\['b'\]}}|NULL|-1|
|{{ELEMENT(ARRAY_REMOVE(ARRAY\[1\], 1))}}|NULL|-1|
|{{a\[1\]}} on a NULL {{ARRAY<INT NOT NULL>}} column|NULL|-1|
|{{ARRAY\[10,20,CAST(NULL AS INT)\]\[4\]}}|NULL|NULL|
{{TYPEOF(ARRAY\[10,20,30\]\[4\])}} is {{INT}}, so the planner types the result
as nullable. {{generateArrayElementAt}}, {{generateArrayElement}} and
{{generateMapGet}} in {{ScalarOperatorGens}} set the null term correctly but
return the element type, here {{INT NOT NULL}}. Consumers that trust the type
then drop the null term: {{CodeGenUtils.rowSetField}} skips {{setNullAt}} for a
NOT NULL field of a {{BoxedWrapperRowData}} or {{GenericRowData}}, which covers
the Calc output and constant folding, and {{generateIsNull}} folds to
{{false}}. Consumers that read the null term, such as COALESCE or {{=}}, see
NULL. That's why one row can show {{-1}} next to {{COALESCE(..., 0) = 0}}.
{{ArrayTypeTest#testReturnNullWhenArrayIndexOutOfBounds}} already asserts NULL
for {{f2\[4\]}} on an {{ARRAY<INT NOT NULL>}} and passes, because
{{ExpressionTestBase}} casts every expression to VARCHAR and writes a
{{BinaryRowData}}, and both read the null term.
I'm working on a fix that makes the three generators return the nullable call
type. I'll link the ticket and PR here.
> Generated code for `ITEM` operator should return null when getting element of
> a null map/array/row
> --------------------------------------------------------------------------------------------------
>
> Key: FLINK-34656
> URL: https://issues.apache.org/jira/browse/FLINK-34656
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Planner
> Affects Versions: 1.20.0
> Reporter: yisha zhou
> Assignee: Ramin Gharib
> Priority: Major
>
> In FieldAccessFromTableITCase we can find that the expected result of f0[1]
> is null when f0 is a null array.
> However, behavior in generated code for ITEM is not consistent with case
> above. The main code is:
>
> {code:java}
> val arrayAccessCode =
> s"""
> |${array.code}
> |${index.code}
> |boolean $nullTerm = ${array.nullTerm} || ${index.nullTerm} ||
> | $idxStr < 0 || $idxStr >= ${array.resultTerm}.size() || $arrayIsNull;
> |$resultTypeTerm $resultTerm = $nullTerm ? $defaultTerm : $arrayGet;
> |""".stripMargin {code}
> If `array.nullTerm` is true, a default value of element type will be
> returned, for example -1 for null bigint array.
> The reason why FieldAccessFromTableITCase can get expected result is that the
> ReduceExpressionsRule generated an expression code for that case like:
> {code:java}
> boolean isNull$0 = true || false ||
> ((int) 1) - 1 < 0 || ((int) 1) - 1 >=
> ((org.apache.flink.table.data.ArrayData) null).size() ||
> ((org.apache.flink.table.data.ArrayData) null).isNullAt(((int) 1) - 1);
> long result$0 = isNull$0 ? -1L : ((org.apache.flink.table.data.ArrayData)
> null).getLong(((int) 1) - 1);
> if (isNull$0) {
> out.setField(0, null);
> } else {
> out.setField(0, result$0);
> } {code}
> The reduced expr will be a null literal.
>
> I think the behaviors for getting element of a null value should be unified.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
