When WindowAggregate works upon Changelog which contains update messages,
UPDATE BEFORE message may be dropped as a late message. [1]

In order to handle late UB message, user needs to set *all* the following 3
parameters:

(1) enable late fire by setting

table.exec.emit.late-fire.enabled : true

(2) set per record emit behavior for late records by setting

table.exec.emit.late-fire.delay : 0 s

(3) keep window state for extra time after window is fired by setting

table.exec.emit.allow-lateness : 1 h// 或者table.exec.state.ttl: 1h


The solution has two disadvantages:

(1) Users may not realize that UB messages may be dropped as a late event,
so they will not set related parameters.

(2) When users look for a solution to solve the dropped UB messages
problem, the current solution is a bit inconvenient for them because they
need to set all the 3 parameters. Besides, some configurations have overlap
ability.


Now there are two proposals to simplify the 3 parameters a little.

(1) Users only need set table.exec.emit.allow-lateness (just like the
behavior on Datastream, user only need set allow-lateness), framework could
atom set `table.exec.emit.late-fire.enabled` to true and set
`table.exec.emit.late-fire.delay` to 0s.

And in the later version, we deprecate `table.exec.emit.late-fire.delay`
and `table.exec.emit.late-fire.enabled`.


(2) Users need set `table.exec.emit.late-fire.enabled` to true and set
`table.exec.state.ttl`, framework  could atom set
`table.exec.emit.late-fire.delay` to 0s.

And in the later version, we deprecate `table.exec.emit.late-fire.delay`
and `table.exec.emit.allow-lateness `.


Please let me know what you think about the issue.

Thank you.

[1] https://issues.apache.org/jira/browse/FLINK-22781


Best regards,
JING ZHANG

Reply via email to