You can also check the apache paimon project https://paimon.apache.org/
(previously known as Flink Table Store).
Might help in some scenarios
On Mon, Aug 28, 2023 at 5:05 AM liu ron wrote:
> Hi, Nirmal
>
> Flink SQL is standard ANSI SQL and extends upon it. Flink SQL provides
> rich Join and Aggregate syntax including Regular Streaming Join, Interval
> Join, Temporal Join, Lookup Join[2], Window Join[3], unbounded group
> aggregate[4] and window aggregate[5], and so on. Theoretically, it can
> support your complex query needs. You need to spend some time learning
> these syntaxes so that you can see which ones meet your needs from a
> business point of view!
>
> [1]
> https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/tableapi/
> [2]
> https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/sql/queries/joins/
> [3]
> https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/sql/queries/window-join/
> [4]
> https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/sql/queries/group-agg/
> [5]
> https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/sql/queries/window-agg/
>
> Best,
> Ron
>
> Nirmal Chhatrala 于2023年8月28日周一 04:36写道：
>
>> Hello!
>>
>> We have a use case requirement to implement complex joins and aggregation
>> on multiple sql tables. Today, it is happening at SQLServer level which is
>> degrading the performance of SQLServer Database.
>>
>> Is it a good idea to implement it through Apache Flink Table API for
>> real-time data joins?
>>
>> ie: there are 10 different sql tables with complex join queries with 10
>> different conditions. (Approx. - The accumulative 10 tables size is ~100GB
>> and could grow in future). We don't have experience with Flink Table API
>> and never measured the performance on such a huge dataset.
>>
>> Looking for your advice and best alternative if it is not suitable with
>> Apache Flink Table API.
>>
>> Thanks!
>>
>
