[ https://issues.apache.org/jira/browse/HIVE-1511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13759306#comment-13759306 ]
Leo Romanoff commented on HIVE-1511: ------------------------------------ Yes, we could consider this. Could you please file an issue at: https://code.google.com/p/kryo/issues/list? What I'm wondering about is the fact that Kryo is used in many multi-threaded apps, but nobody complained so far. I think the easiest workaround is probably to allocate ThreadLocal Kryo instances or using something like object pooling Kryo of instances and using it by a thread pool where each worker thread has all required classes on the classpath. Would such a workaround solve your problems? > Hive plan serialization is slow > ------------------------------- > > Key: HIVE-1511 > URL: https://issues.apache.org/jira/browse/HIVE-1511 > Project: Hive > Issue Type: Improvement > Affects Versions: 0.7.0, 0.11.0 > Reporter: Ning Zhang > Assignee: Mohammad Kamrul Islam > Attachments: failedPlan.xml, generated_plan.xml, HIVE-1511.10.patch, > HIVE-1511.11.patch, HIVE-1511.4.patch, HIVE-1511.5.patch, HIVE-1511.6.patch, > HIVE-1511.7.patch, HIVE-1511.8.patch, HIVE-1511.9.patch, HIVE-1511.patch, > HIVE-1511-wip2.patch, HIVE-1511-wip3.patch, HIVE-1511-wip4.patch, > HIVE-1511.wip.9.patch, HIVE-1511-wip.patch, KryoHiveTest.java, run.sh > > > As reported by Edward Capriolo: > For reference I did this as a test case.... > SELECT * FROM src where > key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 > OR key=0 OR key=0 OR key=0 OR > key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 OR key=0 > OR key=0 OR key=0 OR key=0 OR > ...(100 more of these) > No OOM but I gave up after the test case did not go anywhere for about > 2 minutes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira