en I submitted four typologies, I noted only three were running in
> parallel, while one of the typologies does not assigned to any worker. So I
> was wondering if someone help why slot 6703 does not appear to be active in
> the cluster..
> Thanks and Regards,
> --WA
>
>
>
>
>
>
>
>
--
Thanks & Regards
Rajasekhar
>>> I’m a new user of storm and I need help for an issue I have with storm.
>>>>
>>>> At a glance, I have a topology with three spouts and bolts ;
>>>>
>>>> - Spout -> get messages from Kafka
>>>> - Bolt -> transform the message from Kafka to a well formed son
>>>> - Bolt -> send the JSON to a kairosDB server
>>>>
>>>> All of these bolts/spouts are written in go by using the library
>>>> https://github.com/jsgilmore/gostorm
>>>>
>>>> Unfortunately after using the topology for a while, the CPU usage is
>>>> just growing and growing and never goes down. For instance after sending 1
>>>> million messages, the CPU stays at 200% of cpu usage.
>>>>
>>>> I tried to play with the configuration and get better result ; with the
>>>> following configuration :
>>>>
>>>> conf.put(Config.TOPOLOGY_SLEEP_SPOUT_WAIT_STRATEGY_TIME_MS,
>>>> 1000);
>>>> conf.put(Config.TOPOLOGY_DISRUPTOR_BATCH_SIZE, 10);
>>>> conf.put(Config.TOPOLOGY_DISRUPTOR_BATCH_TIMEOUT_MILLIS,
>>>> 500);
>>>> conf.put(Config.TOPOLOGY_DISRUPTOR_WAIT_TIMEOUT_MILLIS,
>>>> 500);
>>>> conf.setMaxSpoutPending(100);
>>>> conf.put(Config.TASK_HEARTBEAT_FREQUENCY_SECS , 600);
>>>>
>>>> My CPU usage after 1 million messages will be 50%, but it never goes
>>>> down...
>>>>
>>>> What could be the issue(s) and how can I debug that ? I’m pretty new to
>>>> the java/clojure world as well...
>>>>
>>>> Im running on storm 1.0.1 fyi
>>>>
>>>> Thanks in advance,
>>>> --
>>>> Alexandre Wilhelm
>>>>
>>>>
>>>
--
Thanks & Regards
Rajasekhar
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1891)
> ~[na:1.8.0_66]
> at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
> ~[na:1.8.0_66]
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
> ~[na:1.8.0_66]
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:371)
> ~[na:1.8.0_66]
> at org.apache.storm.utils.Utils.javaDeserialize(Utils.java:175)
> ~[storm-core-1.0.1.jar:1.0.1]
> ... 49 common frames omitted
>
>
> Kind Regards
> Rama
>
--
Thanks & Regards
Rajasekhar
ecause I
> don't see the implications to retain the tuples like that.
> More over the bulk tuple acking seems to be anormaly slow
>
> 18:20:51.826 [Thread-24-rawdata-bolt] INFO c.s.streamer.bolts.RawDataBolt
> - rawdatas indexing took 680 ms
> 18:21:23.029 [Thread-24-rawdata-bolt] INFO c.s.streamer.bolts.RawDataBolt
> - rawdatas indexing exit at 31883 ms
>
> Does exist another pattern to achieve what I'm trying ?
>
> Thanks
> Franck
>
>
>
>
--
Thanks & Regards
Rajasekhar
-core-0.10.0.jar:0.10.0]
>>at clojure.lang.RestFn.invoke(RestFn.java:423)
>> [clojure-1.6.0.jar:?]
>>at
>> backtype.storm.event$event_manager$fn__7258.invoke(event.clj:48)
>> [storm-core-0.10.0.jar:0.10.0]
>>at clojure.lang.AFn.run(AFn.java:22) [clojure-1.6.0.jar:?]
>>at java.lang.Thread.run(Thread.java:745) [?:1.8.0_60]
>> 2016-05-06 23:12:15.280 b.s.d.supervisor [INFO] Shutting down supervisor
>> 290f8e54-6f7c-4f07-b4f6-06a5144f0772
>> 2016-05-06 23:12:15.305 b.s.event [INFO] Event manager interrupted
>>
>
>
--
Thanks & Regards
Rajasekhar
41)
>> at
>> backtype.storm.daemon.executor$fn__6579$fn__6594$fn__6623.invoke(executor.clj:565)
>> at backtype.storm.util$async_loop$fn__459.invoke(util.clj:463)
>> at clojure.lang.AFn.run(AFn.java:24)
>> at java.lang.Thread.run(Thread.java:745)
>>
>> Need suggestions
>>
>>
>>
>>
>> *Best regards,*
>> *K.Sai Dilip Reddy.*
>>
>
>
--
Thanks & Regards
Rajasekhar
ce is it in the runtime of cluster?
>
> 3. Is it right to execute blocking call like REST call from a bolt?
>
> Any help would be much appreciated.
>
> --
> Thanks,
> Nageswara Rao.V
>
> *"The LORD reigns"*
>
--
Thanks & Regards
Rajasekhar
rresponding to
>>>> day. This is different that the existing functionality to rotate files
>>>> based on a set length of time. For unrelated reasons, we are probably not
>>>> going to pursue this further. However, I have some code changes that
>>>> implement most of this functionality for at least some partitioning use
>>>> cases. If there is interest from the user or developer community for this
>>>> feature, I could get in shape for a PR to get feedback about our
>>>> implementation approach.
>>>>
>>>> Any feedback on this idea is welcome. Thanks! -Aaron
>>>>
>>>
>>>
>>
>
--
Thanks & Regards
Rajasekhar
a connector to use new consumer api.
>
> Thanks,
> Harsha
>
>
> On Thu, Jan 28, 2016, at 02:04 PM, Rajasekhar wrote:
>
>
> Hi,
> *Backgroud* :
> I am working on kafka-Storm project. Storm reads messages from secured
> kafka cluster.
> SSL security was implemented
Hi,
*Backgroud* :
I am working on kafka-Storm project. Storm reads messages from secured
kafka cluster.
SSL security was implemented in Kafka broker. Consumers (In our case it is
Storm) has to provide the necessary information to get secured connection
to kafka broker.
Client has to provide below
10 matches
Mail list logo
