On Mon, Nov 24, 2014 at 10:20 AM, Stephan Ewen <se...@apache.org> wrote:
> Will the compression Codec will be inserted in the Netty loops, or before > that? > In the current master, I would say that it makes sense to do it in the Netty loops during shuffling. The compression would then be totally transparent to the writers/readers. With the new network stack we will have to think this through a little bit as we have more options and scenarios due to the fact that results will possibly be consumed multiple times. > In any case, would it make sense to prototype this on the current code and > forward port this to the new network stack later? I assume the code would > mostly be similar, especially all the JNI vs. Java considerations and > tricks. > Yes. It makes sense to have this in any case.
