Hi Surendra,
where does your historical data reside? In a database? Files? Behind a REST
API?
Depending on the answer and which API you use, the AsyncIO [1] operator
(DataStream API) or a Lookup Table Join [2] might be good options. There
are other ways, too. A while back I did a webinar [3] on this. It is a bit
outdated, but still covers a few concepts and ideas around this.
Cheers,
Konstantin
[1]
https://nightlies.apache.org/flink/flink-docs-release-1.15/docs/dev/datastream/operators/asyncio/
[2]
https://nightlies.apache.org/flink/flink-docs-release-1.15/docs/dev/table/sql/queries/joins/#lookup-join
[3] https://www.youtube.com/watch?v=cJS18iKLUIY&t=2s
Am Mo., 6. Juni 2022 um 07:36 Uhr schrieb Surendra Lalwani <
surendra.lalw...@swiggy.in>:
> Hi Team,
>
> Since Flink does not support Stream-Batch Join, we need to perform a
> lookup with some historical data, what could be the best way in Flink to do
> that. I am aware of one such approach using Broadcasted stream, not sure if
> there are any other proper solutions. Any help in this will be helpful.
>
> Regards ,
> Surendra Lalwani
>
>
> --
> IMPORTANT NOTICE: This e-mail, including any attachments, may contain
> confidential information and is intended only for the addressee(s) named
> above. If you are not the intended recipient(s), you should not
> disseminate, distribute, or copy this e-mail. Please notify the sender by
> reply e-mail immediately if you have received this e-mail in error and
> permanently delete all copies of the original message from your system.
> E-mail transmission cannot be guaranteed to be secure as it could be
> intercepted, corrupted, lost, destroyed, arrive late or incomplete, or
> contain viruses. Company accepts no liability for any damage or loss of
> confidential information caused by this email or due to any virus
> transmitted by this email or otherwise.
--
https://twitter.com/snntrable
https://github.com/knaufk