Thanks Jing for bringing up this topic, The emit strategy configs are annotated as Experiential and not public on documentations. However, I see this is a very useful feature which many users are looking for. I have posted these configs for many questions like "how to handle late events in SQL". Thus, I think it's time to make the configuration public and explicitly document it. In the long term, we would like to propose an EMIT syntax for SQL, but until then we can get more valuable feedback from users when they are using the configs.
Regarding the exposed configuration, I prefer proposal (2). But it would be better not to expose `table.exec.emit.late-fire.enabled` on docs and we can remove it in the next version. Best, Jark On Tue, 29 Jun 2021 at 11:09, JING ZHANG <beyond1...@gmail.com> wrote: > When WindowAggregate works upon Changelog which contains update messages, > UPDATE BEFORE message may be dropped as a late message. [1] > > In order to handle late UB message, user needs to set *all* the following > 3 parameters: > > (1) enable late fire by setting > > table.exec.emit.late-fire.enabled : true > > (2) set per record emit behavior for late records by setting > > table.exec.emit.late-fire.delay : 0 s > > (3) keep window state for extra time after window is fired by setting > > table.exec.emit.allow-lateness : 1 h// 或者table.exec.state.ttl: 1h > > > The solution has two disadvantages: > > (1) Users may not realize that UB messages may be dropped as a late event, > so they will not set related parameters. > > (2) When users look for a solution to solve the dropped UB messages > problem, the current solution is a bit inconvenient for them because they > need to set all the 3 parameters. Besides, some configurations have overlap > ability. > > > Now there are two proposals to simplify the 3 parameters a little. > > (1) Users only need set table.exec.emit.allow-lateness (just like the > behavior on Datastream, user only need set allow-lateness), framework could > atom set `table.exec.emit.late-fire.enabled` to true and set > `table.exec.emit.late-fire.delay` to 0s. > > And in the later version, we deprecate `table.exec.emit.late-fire.delay` > and `table.exec.emit.late-fire.enabled`. > > > (2) Users need set `table.exec.emit.late-fire.enabled` to true and set > `table.exec.state.ttl`, framework could atom set > `table.exec.emit.late-fire.delay` to 0s. > > And in the later version, we deprecate `table.exec.emit.late-fire.delay` > and `table.exec.emit.allow-lateness `. > > > Please let me know what you think about the issue. > > Thank you. > > [1] https://issues.apache.org/jira/browse/FLINK-22781 > > > Best regards, > JING ZHANG > > > >
