That is odd then. Kryo serializes classes in a more fully qualified manner the first time it sees the class for an object, and records and index of closes out has seen before. It serializes classes by index when it sees them subsequently. On deserialization it rebuilds the index as it encounters new class information. Your error means that it sees a class index value that it hasn't rebuilt yet. The only time I've run into this kind of error was when I was mucking with custom serialization.
What type of grouping are you using, and what type of data are you sending your collection? Also, how is your topology configured? Have you try writing a standalone program to serialize your data with Kryo outside of storm? On Oct 9, 2014 7:57 AM, "Brunner, Bill" <[email protected]> wrote: > 0.9.2-incubating, and all my external jar dependencies are on a network > share and symlinked to the /storm/lib folder (not that it would matter, but > this was also happening on a completely different environment when the jars > were not symlinked). > > > > *From:* Nathan Leung [mailto:[email protected]] > *Sent:* Thursday, October 09, 2014 7:42 AM > *To:* user > *Subject:* Re: Serialization issue? > > > > Can you double check that you're running the same version on all three > nodes? > > On Oct 9, 2014 7:35 AM, "Brunner, Bill" <[email protected]> wrote: > > I am seeing serialization issues on a single worker node, while the other > nodes are unaffected. I usually get the error shortly after the topology is > loaded. It happens on a single node any time the total worker nodes in the > cluster is > 2, ie, the topology runs w/o error on 2 nodes (1 worker each), > but when I introduce a 3rd node, that node fails with the serialization > error. Similarly with 4 worker nodes, 3 run and the 4th fails. I have > included the stack trace below. FYI I am using the default serializer. I > have tried cycling nimbus/zk and each node many times, and it does not help > (I have also done a complete reinstall on the failing node(s)) Hopefully > someone can help. Thanks > > > > Bill > > > > 2014-10-06 11:17:30 b.s.util [ERROR] Async loop died! > > java.lang.RuntimeException: com.esotericsoftware.kryo.KryoException: > Encountered unregistered class ID: 38 > > at > backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:128) > ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating] > > at > backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:99) > ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating] > > at > backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:80) > ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating] > > at > backtype.storm.daemon.executor$fn__5641$fn__5653$fn__5700.invoke(executor.clj:746) > ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating] > > at backtype.storm.util$async_loop$fn__457.invoke(util.clj:431) > ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating] > > at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na] > > at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45] > > Caused by: com.esotericsoftware.kryo.KryoException: Encountered > unregistered class ID: 38 > > at > com.esotericsoftware.kryo.util.DefaultClassResolver.readClass(DefaultClassResolver.java:119) > ~[kryo-2.21.jar:na] > > at com.esotericsoftware.kryo.Kryo.readClass(Kryo.java:610) > ~[kryo-2.21.jar:na] > > at > com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:721) > ~[kryo-2.21.jar:na] > > at > com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:109) > ~[kryo-2.21.jar:na] > > at > com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:18) > ~[kryo-2.21.jar:na] > > at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:629) > ~[kryo-2.21.jar:na] > > at > backtype.storm.serialization.KryoValuesDeserializer.deserializeFrom(KryoValuesDeserializer.java:38) > ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating] > > at > backtype.storm.serialization.KryoTupleDeserializer.deserialize(KryoTupleDeserializer.java:53) > ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating] > > at > backtype.storm.daemon.executor$mk_task_receiver$fn__5564.invoke(executor.clj:396) > ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating] > > at > backtype.storm.disruptor$clojure_handler$reify__745.onEvent(disruptor.clj:58) > ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating] > > at > backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:125) > ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating] > > ... 6 common frames omitted > ------------------------------ > > This message, and any attachments, is for the intended recipient(s) only, > may contain information that is privileged, confidential and/or proprietary > and subject to important terms and conditions available at > http://www.bankofamerica.com/emaildisclaimer. If you are not the intended > recipient, please delete this message. > ------------------------------ > This message, and any attachments, is for the intended recipient(s) only, > may contain information that is privileged, confidential and/or proprietary > and subject to important terms and conditions available at > http://www.bankofamerica.com/emaildisclaimer. If you are not the intended > recipient, please delete this message. >
