Hi Fabian,
thanks for proposing this FLIP. I agree that this join is super common,
after talking to many people at conferences, I could imagine it will be
one of the most used kinds of joins going forward.
Tightly coupling it with watermarks fits both from a semantical point of
view but also with other efforts such as FLIP-558 (Improvements to
SinkUpsertMaterializer and changelog disorder) [1]. In the near future,
we should work on more automated watermarking to power these
watermark-based operators, but this is an orthogonal effort.
Overall I'm strongly +1 on this. Also +1 on the syntax improvements for
lateral table functions by dropping the TABLE() wrapper.
Cheers,
Timo
[1]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-558%3A+Improvements+to+SinkUpsertMaterializer+and+changelog+disorder
On 18.05.26 11:47, Gustavo de Morais wrote:
Hi Fabian,
In general a strong +1 for the feature, without getting into the details of
the FLIP yet. This is a missing feature for years and I'm happy that we're
putting the time to address this - while also getting rid of some of the
hard restrictions we had. Thanks!
Kind regards,
Gustavo
On Fri, 15 May 2026 at 16:39, Fabian Hueske <[email protected]> wrote:
Hi everyone,
I'd like to start a discussion on FLIP-579: LATERAL SNAPSHOT Join [1].
Enriching a stream with data from a (slowly changing) dynamic table is a
super common use case.
Flink SQL features Temporal Joins [2] to address these use cases.
However, SQL users can only use the event-time variant which has many
limitations (heavy dependency on frequent WM updates on both inputs,
build-side table requires a PK, the join predicate must include the
build-side PK, etc).
The processing-time temporal join is disabled (due to build-side
initialization issues [3]) and temporal table function joins are
only available in Table API.
FLIP-579 proposes a new temporal join operator that operates in
processing-time and addresses the limitations of the existing
implementations:
* initialization of the build-side before joining
* no requirement of continuous, frequent build-side WMs (after the
initialization completed)
* no requirement for a PK on the build-side
* table function-based syntax [4] via a built-in SNAPSHOT function
(proposed in FLIP-517 [4])
Looking forward to your feedback.
Best,
Fabian
[1]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-579%3A+LATERAL+SNAPSHOT+Join
[2]
https://nightlies.apache.org/flink/flink-docs-stable/docs/dev/table/sql/queries/joins/#temporal-joins
[3] https://issues.apache.org/jira/browse/FLINK-19830
[4]
https://nightlies.apache.org/flink/flink-docs-stable/docs/dev/table/sql/queries/joins/#temporal-table-function-join
[5]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-517%3A+Better+Handling+of+Dynamic+Table+Primitives+with+PTFs#FLIP517:BetterHandlingofDynamicTablePrimitiveswithPTFs-SNAPSHOTfortemporaljoins
