[
https://issues.apache.org/jira/browse/FLINK-39648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ramin Gharib updated FLINK-39648:
---------------------------------
Description:
Flink's regex-family built-ins (\{{REGEXP_EXTRACT}}, \{{REGEXP_REPLACE}},
\{{REGEXP}}) share two problems in their \{{SqlFunctionUtils}} runtime helpers:
# *No plan-time validation of literal patterns.* When the regex argument is a
string literal that fails \{{java.util.regex.Pattern.compile}}, the failure is
only discovered per record at runtime. The job plans successfully, runs, and
silently produces \{{NULL}} / \{{false}} for every row.
# *\{{LOG.error}} on the hot path.* The runtime helpers catch
\{{PatternSyntaxException}} and log at \{{ERROR}} level inside the per-record
code path. A high-throughput stream with a single bad literal regex emits one
stack trace per record processed.
Both problems are independent of the regex value: the pattern is known at
planning time when it is a literal, so the compile failure can and should
surface as a \{{ValidationException}} during type inference. For non-literal
regexes, the runtime should silently return the function's "no match" sentinel
(\{{NULL}} for extract/replace, \{{false}} for the predicate) without flooding
logs.
h2. Goal
Standardize the regex-family built-ins on:
* A plan-time \{{InputTypeStrategy}} that calls \{{Pattern.compile(...)}} on
literal regex arguments and surfaces failures via \{{callContext.fail(...)}}.
* Runtime helpers that use \{{SqlFunctionUtils.REGEXP_PATTERN_CACHE}} for
cached compilation and silently return the function's no-match sentinel on
\{{PatternSyntaxException}}, instead of logging at \{{ERROR}} on the hot path.
> Validate literal regex patterns at plan time and stop logging on the hot path
> for the regex function family
> -----------------------------------------------------------------------------------------------------------
>
> Key: FLINK-39648
> URL: https://issues.apache.org/jira/browse/FLINK-39648
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / API, Table SQL / Planner, Table SQL / Runtime
> Reporter: Ramin Gharib
> Priority: Major
>
> Flink's regex-family built-ins (\{{REGEXP_EXTRACT}}, \{{REGEXP_REPLACE}},
> \{{REGEXP}}) share two problems in their \{{SqlFunctionUtils}} runtime
> helpers:
> # *No plan-time validation of literal patterns.* When the regex argument is
> a string literal that fails \{{java.util.regex.Pattern.compile}}, the failure
> is only discovered per record at runtime. The job plans successfully, runs,
> and silently produces \{{NULL}} / \{{false}} for every row.
> # *\{{LOG.error}} on the hot path.* The runtime helpers catch
> \{{PatternSyntaxException}} and log at \{{ERROR}} level inside the per-record
> code path. A high-throughput stream with a single bad literal regex emits one
> stack trace per record processed.
> Both problems are independent of the regex value: the pattern is known at
> planning time when it is a literal, so the compile failure can and should
> surface as a \{{ValidationException}} during type inference. For non-literal
> regexes, the runtime should silently return the function's "no match"
> sentinel (\{{NULL}} for extract/replace, \{{false}} for the predicate)
> without flooding logs.
> h2. Goal
> Standardize the regex-family built-ins on:
> * A plan-time \{{InputTypeStrategy}} that calls \{{Pattern.compile(...)}} on
> literal regex arguments and surfaces failures via \{{callContext.fail(...)}}.
> * Runtime helpers that use \{{SqlFunctionUtils.REGEXP_PATTERN_CACHE}} for
> cached compilation and silently return the function's no-match sentinel on
> \{{PatternSyntaxException}}, instead of logging at \{{ERROR}} on the hot path.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
