Thanks Austin. Will take a look at the AsyncIO. Looks like a pretty cool
feature.
On Fri, Feb 10, 2023 at 1:31 PM Austin Cawley-Edwards <
austin.caw...@gmail.com> wrote:
> It's been a while, but I think I've done something similar before with
> Async I/O [1] and batching records with a window.
>
It's been a while, but I think I've done something similar before with
Async I/O [1] and batching records with a window.
This was years ago, so no idea if this was/is good practice, but
essentially it was:
-> Window by batch size (with a timeout trigger to maintain some SLA)
-> Process function
I wonder if windows will be the solution when it comes to datastream API.
On Fri, Feb 10, 2023 at 12:07 PM Leon Xu wrote:
> Hi Flink Users,
>
> We wanted to use Flink to run a decoration pipeline, where we would like
> to make calls to some external service to fetch data and alter the event in
Hi Flink Users,
We wanted to use Flink to run a decoration pipeline, where we would like to
make calls to some external service to fetch data and alter the event in
the Flink pipeline.
Since there's external service call involved so we want to do batch calls
so that it can reduce the load on the
