GitHub user tillrohrmann opened a pull request:
https://github.com/apache/incubator-flink/pull/74
[FLINK-610] Added KryoSerializer
I added the KryoSerializer and replaced the AvroSerializer as the standard
generic type serializer. Due to the way Flink's serialization works, we cannot
exploit Kryo's Inputs which buffer data for serialization. Instead, we have to
read byte after byte if we do not know the length of the underlying data. This
has consequences for variable length encodings. Kryo will fall back to a slow
deserialization of these types, since it will read only one byte at a time. I
do not know the implications of this on Kryo's performance. Therefore, I did
not remove Avro completely yet. Furthermore, Kryo should be tested on the
cluster and if possible its performance should be compared to Avro.
We could mitigate the problem if we don't use variable length encodings for
types such as ```int```s and ```long```s. This, however, comes at the price of
larger serialized objects. For UTF8 strings, the problem would persist.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/tillrohrmann/incubator-flink FLINK-610
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-flink/pull/74.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #74
----
commit 0c618c20bc7be64de82e0e7d3f7734e8c68593bb
Author: Till Rohrmann <[email protected]>
Date: 2014-07-17T15:21:59Z
Added KryoSerializer
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
