[
https://issues.apache.org/jira/browse/FLINK-23993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17407180#comment-17407180
]
Jingsong Lee commented on FLINK-23993:
--------------------------------------
I think we can not finish this in 1.14 release.
Actually, we can't solve all the disorder problems. Our sink can freely define
Primary key. There are many cases, the data seen by sink are disordered in a
sense, some are acceptable and some are unacceptable. This is a complex
mechanism (It may also involve the concept of dynamic table), and we need to
create a detailed explanation documentation. (Actually, FLINK-20374 just fixed
the more common cases, for upsert sink, sink just do what it should do)
> Describe eventually-consistency of materializing upserts
> --------------------------------------------------------
>
> Key: FLINK-23993
> URL: https://issues.apache.org/jira/browse/FLINK-23993
> Project: Flink
> Issue Type: Sub-task
> Components: Documentation, Table SQL / Ecosystem
> Affects Versions: 1.14.0
> Reporter: Nico Kruber
> Priority: Major
>
> FLINK-20374 added an upsert materialization operator which fixes the order of
> shuffled streams. The results of this operator are actually _eventually
> consistent_ (it collects the latest value it has seen and redacts older
> versions when these are not valid anymore). You could see a result stream
> like this, based on the order the materialization receives events:
> +I10, -I10, +I5, -I5, +I10, -I10, +I3, -I3, +I10
> Each time, the value stored in Kafka would change until the "final" result is
> in.
>
> It may be acceptable for upsert sinks, but should be documented (or
> changed/fixed) nonetheless.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
