[
https://issues.apache.org/jira/browse/FLINK-40346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18103291#comment-18103291
]
Dian Fu commented on FLINK-40346:
---------------------------------
[~sepurisaikrishna] Good catch! Looking forward to the PR.
> Triton array deserialization silently replaces JSON nulls with 0 / false /
> "null"
> ---------------------------------------------------------------------------------
>
> Key: FLINK-40346
> URL: https://issues.apache.org/jira/browse/FLINK-40346
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Runtime
> Affects Versions: 2.3.0
> Reporter: sepuri sai krishna
> Priority: Major
>
> h3. Problem
> {{TritonTypeMapper#deserializeArrayFromJson}} does not detect JSON {{null}}
> elements when converting a Triton inference response into {{ArrayData}}. A
> null is silently replaced by a value the model never produced:
> || Declared output type || Triton response || Deserialized result ||
> | {{ARRAY<STRING>}} | {{["a", null, "b"]}} | {{["a", "null", "b"]}} - element
> 1 is the literal 4-character string |
> | {{ARRAY<INT>}} | {{[1, null, 3]}} | {{[1, 0, 3]}} |
> | {{ARRAY<DOUBLE>}} | {{[1.5, null]}} | {{[1.5, 0.0]}} |
> | {{ARRAY<BOOLEAN>}} | {{[true, null]}} | {{[true, false]}} |
> In every case {{ArrayData#isNullAt}} returns {{false}}, so downstream
> operators cannot tell the substituted value apart from a genuine prediction.
> No exception is thrown and nothing is logged.
> The same substitution applies to the {{default-value}} connector option,
> which is parsed through the same method via
> {{TritonInferenceModelFunction#parseDefaultPayload}}: a configured fallback
> of {{[1, null, 3]}} for an {{ARRAY<INT>}} output silently becomes {{[1, 0,
> 3]}}.
> Note the asymmetry within the same class - the serialization side already
> handles this correctly ({{serializeArrayToJsonArray}} emits {{addNull()}} for
> null elements), and scalar deserialization handles it too
> ({{deserializeFromJson}} returns {{null}} for a null node). Only
> array-element deserialization is missing the check, so a null does not
> survive a serialize/deserialize round trip.
> h3. Impact
> Silent data corruption on the inference output path
> ({{TritonInferenceModelFunction#parseInferenceResponse}}). For
> {{ARRAY<STRING>}} the corrupted element is indistinguishable from a
> legitimate {{"null"}} string. For numeric arrays a missing prediction becomes
> a plausible {{0}} that flows into downstream aggregations unnoticed.
> h3. Affects
> 2.3.0 and master. The module was introduced by FLINK-38857 (fix version
> 2.3.0); the file is absent from release-2.2 and earlier.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
