I am seeing serialization issues on a single worker node, while the other nodes
are unaffected. I usually get the error shortly after the topology is loaded.
It happens on a single node any time the total worker nodes in the cluster is >
2, ie, the topology runs w/o error on 2 nodes (1 worker each), but when I
introduce a 3rd node, that node fails with the serialization error. Similarly
with 4 worker nodes, 3 run and the 4th fails. I have included the stack trace
below. FYI I am using the default serializer. I have tried cycling nimbus/zk
and each node many times, and it does not help (I have also done a complete
reinstall on the failing node(s)) Hopefully someone can help. Thanks
Bill
2014-10-06 11:17:30 b.s.util [ERROR] Async loop died!
java.lang.RuntimeException: com.esotericsoftware.kryo.KryoException:
Encountered unregistered class ID: 38
at
backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:128)
~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at
backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:99)
~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at
backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:80)
~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at
backtype.storm.daemon.executor$fn__5641$fn__5653$fn__5700.invoke(executor.clj:746)
~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at backtype.storm.util$async_loop$fn__457.invoke(util.clj:431)
~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na]
at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45]
Caused by: com.esotericsoftware.kryo.KryoException: Encountered unregistered
class ID: 38
at
com.esotericsoftware.kryo.util.DefaultClassResolver.readClass(DefaultClassResolver.java:119)
~[kryo-2.21.jar:na]
at com.esotericsoftware.kryo.Kryo.readClass(Kryo.java:610)
~[kryo-2.21.jar:na]
at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:721)
~[kryo-2.21.jar:na]
at
com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:109)
~[kryo-2.21.jar:na]
at
com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:18)
~[kryo-2.21.jar:na]
at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:629)
~[kryo-2.21.jar:na]
at
backtype.storm.serialization.KryoValuesDeserializer.deserializeFrom(KryoValuesDeserializer.java:38)
~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at
backtype.storm.serialization.KryoTupleDeserializer.deserialize(KryoTupleDeserializer.java:53)
~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at
backtype.storm.daemon.executor$mk_task_receiver$fn__5564.invoke(executor.clj:396)
~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at
backtype.storm.disruptor$clojure_handler$reify__745.onEvent(disruptor.clj:58)
~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at
backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:125)
~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
... 6 common frames omitted
----------------------------------------------------------------------
This message, and any attachments, is for the intended recipient(s) only, may
contain information that is privileged, confidential and/or proprietary and
subject to important terms and conditions available at
http://www.bankofamerica.com/emaildisclaimer. If you are not the intended
recipient, please delete this message.
