Hei
I have made a new test and discovered that in my environment a very
simple bolt will die too after around 2500 cycle.
Bolt's code:
1 package storm;
2
3 import backtype.storm.task.TopologyContext;
4 import backtype.storm.topology.BasicOutputCollector;
5 import
Some new information.
Set debug true and from active worker log I can see:
if worker is ok:
2014-06-03 11:04:55 b.s.d.task [INFO] Emitting: hbasewriter __ack_ack
[7197822474056634252 -608920652033678418]
2014-06-03 11:04:55 b.s.d.executor [INFO] Processing received message
source:
Thanks Alex and Ncleung for your inputs.
Both the options looks to be valid based on the data size. I am thinking now,
even RDBMS might scale well for this scenario if bolt is using only “Read”
operations on database. However, will update later on the approach taken.
Regards
Balakrishna
From:
Hi,
Right now, I have my storm cluster setting up (three machines respectively
for zookeeper, nimbus, supervisor). Before submitting the topology to the
storm cluster, I usually test storm program in local model with Eclipse.
And then I use Eclipse to package the storm project and send to nimbus
Hi,
I'm using python bolt which is in the resource directory but storm giving
me error
*7226 [Thread-19-exclaim1] INFO backtype.storm.daemon.executor - Preparing
bolt
I have a simple trident transactional topology that does something like the
following:
kafka transactional spout (~3000 rec/sec, 6 partitions thus paraHint=6) --
aggregate with reducerAggregator (paraHint=20) --
transactional state (I tried MemoryMapState, MemcachedMapState and
CassandraMapState)
Ok got more info.
Looks like the problem is related with spout.
I changed spout:
32 public void open(Map conf, TopologyContext
context,SpoutOutputCollector collector)
33 {
34 this.collector = collector;
35
36 Properties props = new
Thank you Derek for the explanation between :disallowed and :timed-out. That
was extremely helpful in understanding what decisions Storm is making. I
increased the timeouts for both messages to 5 minutes and returned the
zookeeper session timeouts to their default values. This made it plain
1) Is it appropriate to run Zookeeper in parallel on the same node with the
storm services?
I recommend separate, and even then to ZK storage to a path on its own disk
device if possible. ZK is a bottleneck for storm, and when it is too slow lots
of bad things can happen.
Some folks use
I'm getting this exception
2014-06-03 19:59:13 STDIO [ERROR][id:] Jun 03, 2014 7:59:13 PM
org.jboss.netty.channel.DefaultChannelPipeline
WARNING: An exception was thrown by a user handler while handling an
exception event ([id: 0xdcf3be42] EXCEPTION: java.net.ConnectException:
Connection refused)
How does PersistentAggregate distribute the database calls across all the
worked nodes ?
Does it do the global aggregation then choose a single host to do a
multiget/multiput to the external db ?
Thanks
--
Raphael Hsieh
Thanks for your quick reply nathan.
So I'm doing some debugging of my topology, and I've removed all the logic
from my MultiPut function, replacing it with a single System.out.println()
Then i am monitoring my logs to check when this gets printed out.
It looks like every single one of my hosts
Or do I need to worry about synchronization issue? Thanks in advance!
Sorry, changed the title to make it more accurate.
On Tue, Jun 3, 2014 at 11:12 PM, Xueming Li james.xueming...@gmail.com
wrote:
Or do I need to worry about synchronization issue? Thanks in advance!
14 matches
Mail list logo