[
https://issues.apache.org/jira/browse/METRON-1757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596426#comment-16596426
]
ASF GitHub Bot commented on METRON-1757:
----------------------------------------
Github user nickwallen commented on the issue:
https://github.com/apache/metron/pull/1178
Here is one other detail that I forgot to add to the description. This may
be important if you are trying to validate this fix on a multi-node cluster.
* This serialization error will never show itself if you are running with a
single worker as there is no need for Storm to serialize.
* Even when running the Profiler in a cluster on multiple workers, it will
only show-up if Storm runs the `ProfileBuilderBolt` and the downstream bolts
(`HBaseBolt` or `KafkaBolt`) on a separate worker. This is why you may not
always see the problem when running on a cluster (and why it was confusing to
track down.)
> Storm Profiler Serialization Exception
> --------------------------------------
>
> Key: METRON-1757
> URL: https://issues.apache.org/jira/browse/METRON-1757
> Project: Metron
> Issue Type: Bug
> Reporter: Nick Allen
> Assignee: Nick Allen
> Priority: Major
>
> When running the Storm Profiler with multiple workers this serialization
> error can occur.
> Even in an environment with multiple workers It seems to not occur all the
> time. It may depend on how the individual bolts are executed across the
> workers in a cluster. This likely occurs only when the HBaseBolt is executed
> on a separate worker from the ProfileBuilderBolt.
> It will never occur when running with a single worker.
> {code}
> 2018-08-28 15:24:10.002 o.a.s.m.n.StormServerHandler
> Netty-server-localhost-6700-worker-1 [ERROR] server errors in handling the
> request
> com.esotericsoftware.kryo.KryoException: Class cannot be created (missing
> no-arg constructor):
> org.apache.metron.common.configuration.profiler.ProfileResult
> Serialization trace:
> result (org.apache.metron.common.configuration.profiler.ProfileConfig)
> definition (org.apache.metron.profiler.ProfileMeasurement)
> at
> com.esotericsoftware.kryo.Kryo$DefaultInstantiatorStrategy.newInstantiatorOf(Kryo.java:1272)
> ~[kryo-3.0.3.jar:?]
> at com.esotericsoftware.kryo.Kryo.newInstantiator(Kryo.java:1078)
> ~[kryo-3.0.3.jar:?]
> at com.esotericsoftware.kryo.Kryo.newInstance(Kryo.java:1087)
> ~[kryo-3.0.3.jar:?]
> at
> com.esotericsoftware.kryo.serializers.FieldSerializer.create(FieldSerializer.java:570)
> ~[kryo-3.0.3.jar:?]
> at
> com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:546)
> ~[kryo-3.0.3.jar:?]
> at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:711)
> ~[kryo-3.0.3.jar:?]
> at
> com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)
> ~[kryo-3.0.3.jar:?]
> at
> com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:551)
> ~[kryo-3.0.3.jar:?]
> at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:711)
> ~[kryo-3.0.3.jar:?]
> at
> com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)
> ~[kryo-3.0.3.jar:?]
> at
> com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:551)
> ~[kryo-3.0.3.jar:?]
> at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:793)
> ~[kryo-3.0.3.jar:?]
> at
> com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:134)
> ~[kryo-3.0.3.jar:?]
> at
> com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:40)
> ~[kryo-3.0.3.jar:?]
> at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:689)
> ~[kryo-3.0.3.jar:?]
> at
> org.apache.storm.serialization.KryoValuesDeserializer.deserializeFrom(KryoValuesDeserializer.java:37)
> ~[storm-core-1.1.0.2.6.5.0-292.jar:1.1.0.2.6.5.0-292]
> at
> org.apache.storm.serialization.KryoTupleDeserializer.deserialize(KryoTupleDeserializer.java:50)
> ~[storm-core-1.1.0.2.6.5.0-292.jar:1.1.0.2.6.5.0-292]
> at
> org.apache.storm.messaging.DeserializingConnectionCallback.recv(DeserializingConnectionCallback.java:56)
> ~[storm-core-1.1.0.2.6.5.0-292.jar:1.1.0.2.6.5.0-292]
> at org.apache.storm.messaging.netty.Server.enqueue(Server.java:133)
> ~[storm-core-1.1.0.2.6.5.0-292.jar:1.1.0.2.6.5.0-292]
> at org.apache.storm.messaging.netty.Server.received(Server.java:254)
> ~[storm-core-1.1.0.2.6.5.0-292.jar:1.1.0.2.6.5.0-292]
> at
> org.apache.storm.messaging.netty.StormServerHandler.messageReceived(StormServerHandler.java:61)
> ~[storm-core-1.1.0.2.6.5.0-292.jar:1.1.0.2.6.5.0-292]
> at
> org.apache.storm.shade.org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
> ~[storm-core-1.1.0.2.6.5.0-292.jar:1.1.0.2.6.5.0-292]
> at
> org.apache.storm.shade.org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
> [storm-core-1.1.0.2.6.5.0-292.jar:1.1.0.2.6.5.0-292]
> at
> org.apache.storm.shade.org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
> [storm-core-1.1.0.2.6.5.0-292.jar:1.1.0.2.6.5.0-292]
> at
> org.apache.storm.shade.org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
> [storm-core-1.1.0.2.6.5.0-292.jar:1.1.0.2.6.5.0-292]
> at
> org.apache.storm.shade.org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462)
> [storm-core-1.1.0.2.6.5.0-292.jar:1.1.0.2.6.5.0-292]
> at
> org.apache.storm.shade.org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443)
> [storm-core-1.1.0.2.6.5.0-292.jar:1.1.0.2.6.5.0-292]
> at
> org.apache.storm.shade.org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:310)
> [storm-core-1.1.0.2.6.5.0-292.jar:1.1.0.2.6.5.0-292]
> at
> org.apache.storm.shade.org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
> [storm-core-1.1.0.2.6.5.0-292.jar:1.1.0.2.6.5.0-292]
> at
> org.apache.storm.shade.org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
> [storm-core-1.1.0.2.6.5.0-292.jar:1.1.0.2.6.5.0-292]
> at
> org.apache.storm.shade.org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
> [storm-core-1.1.0.2.6.5.0-292.jar:1.1.0.2.6.5.0-292]
> at
> org.apache.storm.shade.org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
> [storm-core-1.1.0.2.6.5.0-292.jar:1.1.0.2.6.5.0-292]
> at
> org.apache.storm.shade.org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
> [storm-core-1.1.0.2.6.5.0-292.jar:1.1.0.2.6.5.0-292]
> at
> org.apache.storm.shade.org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
> [storm-core-1.1.0.2.6.5.0-292.jar:1.1.0.2.6.5.0-292]
> at
> org.apache.storm.shade.org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108)
> [storm-core-1.1.0.2.6.5.0-292.jar:1.1.0.2.6.5.0-292]
> at
> org.apache.storm.shade.org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:318)
> [storm-core-1.1.0.2.6.5.0-292.jar:1.1.0.2.6.5.0-292]
> at
> org.apache.storm.shade.org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
> [storm-core-1.1.0.2.6.5.0-292.jar:1.1.0.2.6.5.0-292]
> at
> org.apache.storm.shade.org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
> [storm-core-1.1.0.2.6.5.0-292.jar:1.1.0.2.6.5.0-292]
> at
> org.apache.storm.shade.org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
> [storm-core-1.1.0.2.6.5.0-292.jar:1.1.0.2.6.5.0-292]
> at
> org.apache.storm.shade.org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
> [storm-core-1.1.0.2.6.5.0-292.jar:1.1.0.2.6.5.0-292]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> [?:1.8.0_181]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> [?:1.8.0_181]
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
