[
https://issues.apache.org/jira/browse/HIVE-1511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13741362#comment-13741362
]
Leo Romanoff commented on HIVE-1511:
------------------------------------
[~appodictic] Regarding the speed: I already mentioned these ideas on Kryo
mailing list, but may be it was overlooked. Kryo is very fast when used
properly.
Here are some possible improvements:
1) Do not create new Kryo instances every time you need to (de)serialize. It is
a pretty costly operation. Reuse Kryo instances instead. Use object pooling for
Kryo instances. But be careful with multi-threading. Kryo instances cannot be
used by different threads at _the same time_.
2) Preregister classes (use Kryo.register) to be serialized if possible. It
makes serialization faster and serialized representations shorter.
> Hive plan serialization is slow
> -------------------------------
>
> Key: HIVE-1511
> URL: https://issues.apache.org/jira/browse/HIVE-1511
> Project: Hive
> Issue Type: Improvement
> Affects Versions: 0.7.0
> Reporter: Ning Zhang
> Assignee: Mohammad Kamrul Islam
> Attachments: HIVE-1511.patch, HIVE-1511-wip2.patch,
> HIVE-1511-wip3.patch, HIVE-1511-wip.patch
>
>
> As reported by Edward Capriolo:
> For reference I did this as a test case....
> SELECT * FROM src where
> key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0
> OR key=0 OR key=0 OR key=0 OR
> key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0
> OR key=0 OR key=0 OR key=0 OR
> ...(100 more of these)
> No OOM but I gave up after the test case did not go anywhere for about
> 2 minutes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
