[
https://issues.apache.org/jira/browse/FLINK-40182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Nuyanzin updated FLINK-40182:
------------------------------------
Description:
{{ArrayToArrayCastRule}}, {{MapToMapAndMultisetToMultisetCastRule}} generate
code that trusted the target type's declared NOT NULL nullability instead of
checking the actual runtime value. When the source array/map element was
actually null (a cast doesn't enforce constraints, so a NOT NULL target type is
not a runtime guarantee), the generated code would either throw an NPE (e.g.,
unboxing a null) or, for primitive-backed arrays, silently write a default
value (e.g., 0) instead of preserving/rejecting the null — masking data
corruption.
repro
{code:sql}
SELECT CAST(
ARRAY_SLICE(
skills_array,
1,
LEAST( CARDINALITY(JSON_QUERY(SkillList__c, 'lax $[*]' RETURNING
ARRAY<STRING>)), 10 )
)
as ARRAY<ROW<`id` STRING NOT NULL> NOT NULL>)
FROM (
SELECT
*,
ARRAY[
ROW(JSON_VALUE(a.b, '$[0].id' RETURNING VARCHAR ERROR ON EMPTY)),
CASE WHEN JSON_EXISTS(a.b, '$[1]') THEN
ROW(JSON_VALUE(a.b, '$[1].id' RETURNING VARCHAR ERROR ON EMPTY))
END,
ROW(JSON_VALUE(a.b, '$[2].id' RETURNING VARCHAR ERROR ON EMPTY)),
CASE WHEN JSON_EXISTS(a.b, '$[3]') THEN
ROW(JSON_VALUE(a.b, '$[3].id' RETURNING VARCHAR ERROR ON EMPTY))
END
] as skills_array
FROM (VALUES (
'[{"id":"S1"},null,{"id":"S3"}]'
)) AS a(b)
) AS with_skills_array;{code}
was:
{{ArrayToArrayCastRule}}, {{MapToMapAndMultisetToMultisetCastRule}} generate
code that trusted the target type's declared NOT NULL nullability instead of
checking the actual runtime value. When the source array/map element was
actually null (a cast doesn't enforce constraints, so a NOT NULL target type is
not a runtime guarantee), the generated code would either throw an NPE (e.g.,
unboxing a null) or, for primitive-backed arrays, silently write a default
value (e.g., 0) instead of preserving/rejecting the null — masking data
corruption.
repro
{code:sql}
SELECT CAST(
ARRAY_SLICE(
skills_array,
1,
LEAST( CARDINALITY(JSON_QUERY(SkillList__c, 'lax $[*]' RETURNING
ARRAY<STRING>)), 10 )
)
as ARRAY<ROW<`id` STRING NOT NULL> NOT NULL>)
FROM (
SELECT
*,
ARRAY[
ROW(JSON_VALUE(data.SkillList__c, '$[0].id' RETURNING VARCHAR ERROR ON
EMPTY)),
CASE WHEN JSON_EXISTS(data.SkillList__c, '$[1]') THEN
ROW(JSON_VALUE(data.SkillList__c, '$[1].id' RETURNING VARCHAR ERROR ON
EMPTY))
END,
ROW(JSON_VALUE(data.SkillList__c, '$[2].id' RETURNING VARCHAR ERROR ON
EMPTY)),
CASE WHEN JSON_EXISTS(data.SkillList__c, '$[3]') THEN
ROW(JSON_VALUE(data.SkillList__c, '$[3].id' RETURNING VARCHAR ERROR ON
EMPTY))
END
] as skills_array
FROM (VALUES (
'[{"id":"S1"},null,{"id":"S3"}]'
)) AS data(SkillList__c)
) AS with_skills_array;{code}
> ArrayToArray and MapToMap cast rules generate code which might fail with NPE
> ----------------------------------------------------------------------------
>
> Key: FLINK-40182
> URL: https://issues.apache.org/jira/browse/FLINK-40182
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Planner
> Reporter: Sergey Nuyanzin
> Assignee: Sergey Nuyanzin
> Priority: Major
>
> {{ArrayToArrayCastRule}}, {{MapToMapAndMultisetToMultisetCastRule}} generate
> code that trusted the target type's declared NOT NULL nullability instead of
> checking the actual runtime value. When the source array/map element was
> actually null (a cast doesn't enforce constraints, so a NOT NULL target type
> is not a runtime guarantee), the generated code would either throw an NPE
> (e.g., unboxing a null) or, for primitive-backed arrays, silently write a
> default value (e.g., 0) instead of preserving/rejecting the null — masking
> data corruption.
> repro
> {code:sql}
> SELECT CAST(
> ARRAY_SLICE(
> skills_array,
> 1,
> LEAST( CARDINALITY(JSON_QUERY(SkillList__c, 'lax $[*]' RETURNING
> ARRAY<STRING>)), 10 )
> )
> as ARRAY<ROW<`id` STRING NOT NULL> NOT NULL>)
> FROM (
> SELECT
> *,
> ARRAY[
> ROW(JSON_VALUE(a.b, '$[0].id' RETURNING VARCHAR ERROR ON EMPTY)),
> CASE WHEN JSON_EXISTS(a.b, '$[1]') THEN
> ROW(JSON_VALUE(a.b, '$[1].id' RETURNING VARCHAR ERROR ON EMPTY))
> END,
> ROW(JSON_VALUE(a.b, '$[2].id' RETURNING VARCHAR ERROR ON EMPTY)),
> CASE WHEN JSON_EXISTS(a.b, '$[3]') THEN
> ROW(JSON_VALUE(a.b, '$[3].id' RETURNING VARCHAR ERROR ON EMPTY))
> END
> ] as skills_array
> FROM (VALUES (
> '[{"id":"S1"},null,{"id":"S3"}]'
> )) AS a(b)
> ) AS with_skills_array;{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
