Hi Shannon, That's right. This FLIP aims to boost TPS of the task workers with async i/o operation.
As what Stephan has mentioned, by placing static attribute to shared resources(like event pool, connection), it is possible to share those resources among different slots in the same JVM. I will make a note in the FLIP about how to share resources ;D Thanks, David 2016-09-22 1:46 GMT+08:00 Stephan Ewen <se...@apache.org>: > @Shannon: One could have a "static" broker to share the same netty across > slots in the same JVM. Implicitly, Flink does the same with broadcast > variables. > > On Wed, Sep 21, 2016 at 5:04 PM, Shannon Carey <sca...@expedia.com> wrote: > > > David, > > > > I just wanted to say "thanks" for making this proposal! I'm also > > interested in performing nonblocking I/O (multiplexing threads/reactive > > programming) within Flink operators so that we can, for example, > > communicate with external web services with Netty/RxNetty without > blocking > > an entire Flink slot (aka a thread) while we wait for the operation to > > complete. It looks like your FLIP will enable that use case. > > > > I'm not sure whether it will be possible to share one Netty > EventLoopGroup > > (or the equivalent for any other non-blocking framework, connection pool, > > etc.) among multiple slots in a single JVM though. Flink supports > > open/close operation on a RichFunction, but that's on a per-slot basis. I > > don't know of a way to open/close objects on a per-job-JVM basis. But I > > suppose that's an issue that should be discussed and resolved separately. > > > > -Shannon > > > > >
