Yes, right. Worker is a JVM instance which can contains one or more executors, each executor is spout, bolt, acker, etc.
2015년 5월 14일 목요일, Asif Ihsan<asifihsan.ih...@gmail.com>님이 작성한 메시지: > You are right about it. What does multiple workers means. Does it mean > that when I run topology with single worker than i will see one single > process handling all the bolts and when running 6 workers for 5 bolts and 1 > spout than i will see 6 separate processes running each bolt and spout? M i > right.If not then what does it mean? > > > On Thu, May 14, 2015 at 2:46 PM, 임정택 <kabh...@gmail.com > <javascript:_e(%7B%7D,'cvml','kabh...@gmail.com');>> wrote: > >> Hi. >> >> Storm serializes tuples when tuple should be sent to other (remote) >> worker. >> In other words, Storm doesn't serialize tuples when destination is local >> task. That's why you didn't meet error when testing with 1 worker. >> >> MapEventBean seems to be not serializable, so you need to convert to >> other data type. >> >> Hope this helps. >> >> Regards. >> Jungtaek Lim (HeartSaVioR) >> >> >> >> 2015-05-14 18:38 GMT+09:00 Asif Ihsan <asifihsan.ih...@gmail.com >> <javascript:_e(%7B%7D,'cvml','asifihsan.ih...@gmail.com');>>: >> >>> I am using Esper in one of the bolts of the Storm topology. Bolt emit >>> MapEventBean array (EventBean[]). With the single worker topology run >>> smoothly. With multiple workers it gives following error in emit function >>> call. >>> >>> May 14, 2015 2:37:07 PM clojure.tools.logging$eval1$fn__7 invoke >>> SEVERE: Async loop died! >>> java.lang.RuntimeException: java.lang.RuntimeException: >>> java.io.NotSerializableException: com.espertech.esper.event.map.MapEventBean >>> at >>> backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:87) >>> at >>> backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:58) >>> at >>> backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:62) >>> at >>> backtype.storm.disruptor$consume_loop_STAR_$fn__1619.invoke(disruptor.clj:73) >>> at backtype.storm.util$async_loop$fn__465.invoke(util.clj:377) >>> at clojure.lang.AFn.run(AFn.java:24) >>> at java.lang.Thread.run(Unknown Source) >>> Caused by: java.lang.RuntimeException: java.io.NotSerializableException: >>> com.espertech.esper.event.map.MapEventBean >>> at >>> backtype.storm.serialization.SerializableSerializer.write(SerializableSerializer.java:24) >>> at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:554) >>> at >>> com.esotericsoftware.kryo.serializers.MapSerializer.write(MapSerializer.java:87) >>> at >>> com.esotericsoftware.kryo.serializers.MapSerializer.write(MapSerializer.java:17) >>> at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:554) >>> at >>> com.esotericsoftware.kryo.serializers.CollectionSerializer.write(CollectionSerializer.java:77) >>> at >>> com.esotericsoftware.kryo.serializers.CollectionSerializer.write(CollectionSerializer.java:18) >>> at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:472) >>> at >>> backtype.storm.serialization.KryoValuesSerializer.serializeInto(KryoValuesSerializer.java:27) >>> at >>> backtype.storm.serialization.KryoTupleSerializer.serialize(KryoTupleSerializer.java:27) >>> at >>> backtype.storm.daemon.worker$mk_transfer_fn$fn__4152$fn__4156.invoke(worker.clj:99) >>> at backtype.storm.util$fast_list_map.invoke(util.clj:771) >>> at >>> backtype.storm.daemon.worker$mk_transfer_fn$fn__4152.invoke(worker.clj:99) >>> at >>> backtype.storm.daemon.executor$start_batch_transfer__GT_worker_handler_BANG_$fn__3914.invoke(executor.clj:238) >>> at >>> backtype.storm.disruptor$clojure_handler$reify__1606.onEvent(disruptor.clj:43) >>> at >>> backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:84) >>> ... 6 more >>> Caused by: java.io.NotSerializableException: >>> com.espertech.esper.event.map.MapEventBean >>> at java.io.ObjectOutputStream.writeObject0(Unknown Source) >>> at java.io.ObjectOutputStream.writeArray(Unknown Source) >>> at java.io.ObjectOutputStream.writeObject0(Unknown Source) >>> at java.io.ObjectOutputStream.writeObject(Unknown Source) >>> at >>> backtype.storm.serialization.SerializableSerializer.write(SerializableSerializer.java:21) >>> ... 21 more >>> >>> May 14, 2015 2:37:07 PM clojure.tools.logging$eval1$fn__7 invoke >>> SEVERE: >>> java.lang.RuntimeException: java.lang.RuntimeException: >>> java.io.NotSerializableException: com.espertech.esper.event.map.MapEventBean >>> at >>> backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:87) >>> at >>> backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:58) >>> at >>> backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:62) >>> at >>> backtype.storm.disruptor$consume_loop_STAR_$fn__1619.invoke(disruptor.clj:73) >>> at backtype.storm.util$async_loop$fn__465.invoke(util.clj:377) >>> at clojure.lang.AFn.run(AFn.java:24) >>> at java.lang.Thread.run(Unknown Source) >>> Caused by: java.lang.RuntimeException: java.io.NotSerializableException: >>> com.espertech.esper.event.map.MapEventBean >>> at >>> backtype.storm.serialization.SerializableSerializer.write(SerializableSerializer.java:24) >>> at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:554) >>> at >>> com.esotericsoftware.kryo.serializers.MapSerializer.write(MapSerializer.java:87) >>> at >>> com.esotericsoftware.kryo.serializers.MapSerializer.write(MapSerializer.java:17) >>> at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:554) >>> at >>> com.esotericsoftware.kryo.serializers.CollectionSerializer.write(CollectionSerializer.java:77) >>> at >>> com.esotericsoftware.kryo.serializers.CollectionSerializer.write(CollectionSerializer.java:18) >>> at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:472) >>> at >>> backtype.storm.serialization.KryoValuesSerializer.serializeInto(KryoValuesSerializer.java:27) >>> at >>> backtype.storm.serialization.KryoTupleSerializer.serialize(KryoTupleSerializer.java:27) >>> at >>> backtype.storm.daemon.worker$mk_transfer_fn$fn__4152$fn__4156.invoke(worker.clj:99) >>> at backtype.storm.util$fast_list_map.invoke(util.clj:771) >>> at >>> backtype.storm.daemon.worker$mk_transfer_fn$fn__4152.invoke(worker.clj:99) >>> at >>> backtype.storm.daemon.executor$start_batch_transfer__GT_worker_handler_BANG_$fn__3914.invoke(executor.clj:238) >>> at >>> backtype.storm.disruptor$clojure_handler$reify__1606.onEvent(disruptor.clj:43) >>> at >>> backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:84) >>> ... 6 more >>> Caused by: java.io.NotSerializableException: >>> com.espertech.esper.event.map.MapEventBean >>> at java.io.ObjectOutputStream.writeObject0(Unknown Source) >>> at java.io.ObjectOutputStream.writeArray(Unknown Source) >>> at java.io.ObjectOutputStream.writeObject0(Unknown Source) >>> at java.io.ObjectOutputStream.writeObject(Unknown Source) >>> at >>> backtype.storm.serialization.SerializableSerializer.write(SerializableSerializer.java:21) >>> ... 21 more >>> >>> >>> On Wed, May 13, 2015 at 6:11 PM, Jeffery Maass <maas...@gmail.com >>> <javascript:_e(%7B%7D,'cvml','maas...@gmail.com');>> wrote: >>> >>>> Asif: >>>> >>>> Would love to help you, however, we will need way more details. Why >>>> don't you start with telling us what the error is? How are you detecting >>>> the error? >>>> >>>> You will want to look over both the supervisor and worker logs. >>>> >>>> Thank you for your time! >>>> >>>> +++++++++++++++++++++ >>>> Jeff Maass <javascript:_e(%7B%7D,'cvml','maas...@gmail.com');> >>>> linkedin.com/in/jeffmaass >>>> stackoverflow.com/users/373418/maassql >>>> +++++++++++++++++++++ >>>> >>>> >>>> On Wed, May 13, 2015 at 2:37 AM, Asif Ihsan <asifihsan.ih...@gmail.com >>>> <javascript:_e(%7B%7D,'cvml','asifihsan.ih...@gmail.com');>> wrote: >>>> >>>>> Hi, >>>>> >>>>> I recently started using Storm. I am having topology. It works well >>>>> with numofworkers=1. However when I change it to 6 or more than 1. Its >>>>> gives error when emitting tuples. What is the issue and how to resolve it >>>>> with multiple workers. Thank you. >>>>> >>>>> -- >>>>> Regards >>>>> Muhammad Asif Ihsan >>>>> >>>> >>>> >>> >>> >>> -- >>> Regards >>> Muhammad Asif Ihsan >>> >> >> >> >> -- >> Name : 임 정택 >> Blog : http://www.heartsavior.net / http://dev.heartsavior.net >> Twitter : http://twitter.com/heartsavior >> LinkedIn : http://www.linkedin.com/in/heartsavior >> > > > > -- > Regards > Muhammad Asif Ihsan > -- Name : 임 정택 Blog : http://www.heartsavior.net / http://dev.heartsavior.net Twitter : http://twitter.com/heartsavior LinkedIn : http://www.linkedin.com/in/heartsavior
