Currently StreamExecutionEnvironment exposes Kryo thru the following: import com.esotericsoftware.kryo.Serializer;
I think shading is possible since we can declare the Serializer class to come from shaded Kryo namespace. On Mon, Jul 3, 2017 at 9:25 AM, Aljoscha Krettek <aljos...@apache.org> wrote: > Is it even possible to shade Kryo with it being in the public API? I don’t > think it is (without looking into this to deeply, though). > > > On 3. Jul 2017, at 17:23, Chesnay Schepler <ches...@apache.org> wrote: > > > > I would tackle Kryo later as it is exposed through the API > (StreamExecutionEnvironment#addDefaultKryoSerializer) > > which will make the shading process more difficult than for the modules > that are done so far. > > > > Technically this doesn't prevent us from adding a flink-shaded-kryo2/3 > module right now, but so far I went with the approach > > of creating the shaded module and actually integrating it into Flink > before submitting it to flink-shaded to make sure > > everything actually works. With kryo this approach may take a while so I > would suggest to do it later. > > > > On 03.07.2017 17:04, Stephan Ewen wrote: > >> Other non-Hadoop dependencies that I think are important are the Kryo > >> dependencies. > >> > >> Do we want them in the first release, or tackle them later? > >> > >> Specifically, I think we need > >> > >> - flink-shaded-kryo2 which includes kryo2 plus chill in the version we > >> currently use > >> - flink-shaded-kryo3 which includes kryo3 with the latest chill > version > >> > >> > >> > >> > >> On Mon, Jul 3, 2017 at 2:46 PM, Fabian Hueske <fhue...@gmail.com> > wrote: > >> > >>> Thanks for working on this Chesnay. > >>> I think releasing flink-shaded and integrating it in the build before > >>> touching the Hadoop dependencies it sounds like a good plan. > >>> > >>> +1 > >>> > >>> Cheers, Fabian > >>> > >>> 2017-07-03 14:32 GMT+02:00 Chesnay Schepler <ches...@apache.org>: > >>> > >>>> Hello, > >>>> > >>>> I would like to kick off the first release of flink-shaded. > >>>> > >>>> In the current state the release would include shaded dependencies for > >>>> asm, guava and netty. > >>>> > >>>> All changes required for the integration into Flink have been prepared > >>> and > >>>> tested locally and on yarn. > >>>> > >>>> In order for these changes to not lie around for too long i would > like to > >>>> the first release soon, as I expect > >>>> the additions of hadoop into flink-shaded to take some > time/discussions > >>> (I > >>>> also want to spend some time > >>>> to understand what we're shading for hadoop and why). > >>>> > >>>> Regards, > >>>> Chesnay > >>>> > >>>> > >>>> > > > >
