Hi, There is no out-of-box Flink source/sink connector for this, but it isn't unheard of that users have implemented something to support what you outlined.
One way to possibly achieve this is: in terms of a Flink streaming job graph, what you would need to do is co-locate the source (which exposes the endpoint and maintains a pool of open client connections mapped by request ID), and the sink operators (which receives processed results with the original request IDs attached, and is in charge for replying to the original requests). The open client connections need to be process-wide accessible (e.g. via a static reference), so that when a co-located sink operator receives a result, it can directly fetch the corresponding client connection and return a response. The specifics are of course a bit more evolved; probably need some digging around previous Flink Forward conference talks to get a better picture. Hopefully this gives you a starting point to think about. Cheers, Gordon -- Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
