lincoln lee created FLINK-27849:
-----------------------------------
Summary: Harden correctness for non-deterministic updates present
in the changelog pipeline
Key: FLINK-27849
URL: https://issues.apache.org/jira/browse/FLINK-27849
Project: Flink
Issue Type: Bug
Components: Table SQL / Runtime
Environment: There commonly exists updates(which means not only
RowKind.INSERT messages) in a streaming pipeline, then wrong results or error
may occurs when use some non-deterministic functions or operations.
It is a long lived issue since the first day that flink sql was available in
streaming, but it still not totally be eliminated though some efforts have been
taken.
We should detect all the non-deterministic operations in the changelog
pipelines, raise an error to tell users the risk and also add an mechanism that
can process such a issue if a user is willing to pay some cost(probably
introduce the state).
All non-deterministic operations include builtin temporal functions(now,
current_timestamp...), UUID, RAND...
or user defined non-deterministic functions (override isDeterministic return
false)
or a lookup join on a lookup source which data may change over time
Reporter: lincoln lee
Fix For: 1.16.0
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
