I would tackle Kryo later as it is exposed through the API
(StreamExecutionEnvironment#addDefaultKryoSerializer)
which will make the shading process more difficult than for the modules
that are done so far.
Technically this doesn't prevent us from adding a flink-shaded-kryo2/3
module right now, but so far I went with the approach
of creating the shaded module and actually integrating it into Flink
before submitting it to flink-shaded to make sure
everything actually works. With kryo this approach may take a while so I
would suggest to do it later.
On 03.07.2017 17:04, Stephan Ewen wrote:
Other non-Hadoop dependencies that I think are important are the Kryo
dependencies.
Do we want them in the first release, or tackle them later?
Specifically, I think we need
- flink-shaded-kryo2 which includes kryo2 plus chill in the version we
currently use
- flink-shaded-kryo3 which includes kryo3 with the latest chill version
On Mon, Jul 3, 2017 at 2:46 PM, Fabian Hueske <fhue...@gmail.com> wrote:
Thanks for working on this Chesnay.
I think releasing flink-shaded and integrating it in the build before
touching the Hadoop dependencies it sounds like a good plan.
+1
Cheers, Fabian
2017-07-03 14:32 GMT+02:00 Chesnay Schepler <ches...@apache.org>:
Hello,
I would like to kick off the first release of flink-shaded.
In the current state the release would include shaded dependencies for
asm, guava and netty.
All changes required for the integration into Flink have been prepared
and
tested locally and on yarn.
In order for these changes to not lie around for too long i would like to
the first release soon, as I expect
the additions of hadoop into flink-shaded to take some time/discussions
(I
also want to spend some time
to understand what we're shading for hadoop and why).
Regards,
Chesnay
