Hi Juho,
Thank you for bringing this up! Definitely +1 to this. We have had similar
requests for the AsyncSink as well.
As a side note, it would be useful to share the same implementation for both
somehow, to prevent duplicate code.
Happy to help with the implementation here.
For the AsyncSink
This is not supported by the current Async I/O API.
But I do think this is a very useful feature and we should support it.
Just as Jingsong said this allows changelog stream can also use Async
Lookup Join.
The rough idea is just like a mixture of the ordered and unordered modes of
async operator.
R
+1 for this.
Actually, this is a headache for Flink SQL too.
There is certainly a lot of updated data (CDC changelog) in real
stream processing, The semantics here is the need to ensure the order
between keys, and different keys can be handled in disorder.
I'm very happy that the community has a
Thank you very much! It seems like you have a quite similar goal. However,
could you clarify: do you maintain the stream order on key level, or do you
just limit the parallel requests per key to one without caring about the
order?
I'm not 100% sure how your implementation with futures is done. If
Hi Juho -- I'm doing something similar. In my case, I want to execute async
requests concurrently for inputs associated with different keys but issue
them sequentially for any given key. The way I do it is to create a keyed
stream and use it as an input to an async function. In this arrangement,
al
I need to make some slower external requests in parallel, so Async I/O
helps greatly with that. However, I also need to make the requests in a
certain order per key. Is that possible with Async I/O?
The documentation[1] talks about preserving the stream order of
results, but it doesn't discuss the
