[
https://issues.apache.org/jira/browse/FLINK-41087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yuepeng Pan updated FLINK-41087:
--------------------------------
Fix Version/s: 2.5.0
> TRY_PARSE_JSON builds the PARSE_JSON error for every row it rejects
> -------------------------------------------------------------------
>
> Key: FLINK-41087
> URL: https://issues.apache.org/jira/browse/FLINK-41087
> Project: Flink
> Issue Type: Improvement
> Components: Table SQL / Planner, Table SQL / Runtime
> Affects Versions: 2.4.0
> Reporter: Ramin Gharib
> Assignee: Ramin Gharib
> Priority: Major
> Labels: pull-request-available
> Fix For: 2.4.0, 2.5.0
>
>
> h3. Problem
> Since FLINK-40996, {{PARSE_JSON}} and {{TRY_PARSE_JSON}} share a generated
> method that calls {{SqlJsonUtils#parseJsonToVariant}}. When parsing fails,
> that method scans the whole input for invalid UTF-8, formats the input into a
> message and creates a {{TableRuntimeException}}. The generated code stores
> it. {{TRY_PARSE_JSON}} then drops it and returns NULL.
> Before FLINK-40996, {{TryParseJsonFunction}} returned NULL right after the
> parser failed. Each row that {{TRY_PARSE_JSON}} rejects now also pays for:
> * a second exception with its stack trace,
> * a scan of the input bytes,
> * a copy of the input into the message string.
> The results do not change. {{TRY_PARSE_JSON}} is meant for input that may not
> be valid JSON, so this can hit a large share of the rows.
> h3. Proposal
> Let the shared method keep the exception that the parser throws. Build the
> {{TableRuntimeException}} with the UTF-8 diagnostics only where
> {{PARSE_JSON}} rethrows, for example through a {{SqlJsonUtils}} method that
> takes the input and the cause. {{PARSE_JSON}} keeps its messages, such as
> {{Failed to parse JSON string: Invalid UTF-8 byte at index 1 of 3.}}. A
> {{TRY_PARSE_JSON}} that shares the parse with a {{PARSE_JSON}} keeps
> returning NULL.
> h3. Tests
> * The existing {{PARSE_JSON}} error tests in {{JsonFunctionsITCase}} pass
> unchanged.
> * {{JsonParseReuseTest}}: {{PARSE_JSON}} and {{TRY_PARSE_JSON}} on the same
> invalid input still parse once and still fail with {{Failed to parse JSON
> string}}.
> * {{JsonParseReuseTest}}: the generated code of a query with only
> {{TRY_PARSE_JSON}} does not build the error.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
