It works. Thanks!
--
Best regards,
Ivy Tang
On 2014年2月21日, at 下午12:29, 刘健男 wrote:
> shuffle
Because you didn’t tell trident how to distribute tupes from 1 partition to 4
partitions.
I think the code below can work:
TridentState state = topology
.merge(merge_streams)
.parallelismHint(1)
Here is my code：
private static String[] stream_names = new String[] { "spout_0",
"spout_1" };
for (String stream_name : stream_names) {
Stream stream = topology.newStream(stream_name, new
WordSpout(stream_name, 1))
.parallel
・ The bug fix:”Fix race condition between supervisor and Nimbus
that could lead to stormconf.ser errors and infinite crashing of
supervisor” in storm 0.8.3 is very useful for me.
・ But storm 0.8.3 is unreleased for a long time. I compiled the
0.8.3-wip source code myself and instal
/brokers stands for the path in Kafka zookeeper;
Using StaticHosts u directly get information from these
StaticHosts(brokers), it is not extensive as brokers may change for some
reason.
Using ZkHosts u can dynamicly discover information about Kafka cluster(AKA
brokers) by reading ZkPath(most ofen
Is there a way to suspend a spout for a pre-determined period of time from
within that spout or an attached bolt?
As an example, assume that in your Topology you have a bolt retrieving data
from Twitter's api, but you get rate limited. To me the obvious solution is
to somehow suspend the spout for
Hi,
This is what I remember when I last worked on the IPartitionedSpout:
1) When new transactions are to be played, isReady() is called first. You
return true or false depending on the state of your source
2) If you return true, emitPartitionBatchNew() will be called for each
Emitter instance with
Hi,
I am a little confused about StaticHost, and ZkHosts, ZkPath, and
Partitions Per Host.
Our kafka uses its own zookeeper, and it has 10 partitions. Our storm use
other zookeepers
So If I am using ZkHosts
KafkaConfig.ZkHosts zkHosts = new KafkaConfig.ZkHosts(
"kafka-zooker1:2181,kafka-zooker
Hi, I have a Storm Topology composed of a AMPQ_Spout and a bolt capable of
writing to a MongoDB Collection
The problem is that once I run the Java Rabbitmq program that sends tuples
to Rabbitmq queue; the Storm topology pulls the Rabbitmq taking 100 tuples
and waiting for some minutes before it pu
I'm trying to implement an error-handling IPartitionedTridentSpout that limits
the number of retries of a batch. The problem I've got is the interaction
between the coordinator and emitter.
The spout reads from a kafka queue. If there's no messages been put on the
queue recently, then the coord
We generally instantiate a pool per JVM, where the maxActive is (# of bolts
using JDBC/num workers+1) (e.g. 64 bolts, 4 workers, 17 conns/JVM).
Pooling is our preferred strategy as the pool will shrink (and thus use
less memory on the corresponding SQL server) if it's not being utilized. In
either
Hi Pablo,
Your approach is fine. Alternative approach proposed by Brian can be used
if you can group the relevant bolt instance/es, which are communicating to
the database, into the appropriate worker process/es. However with the
alternative approach you shouldn't end up creating separate pools in
I keep one connection per bolt and for now it works just fine with many
bolts. I would also be interested in hearing from someone else and share
experiences.
For now, I open the connection in the method prepare (and close it in
cleanup), but to be completely honest, I'm not 100% sure if this is th
This leads me to conclude that storm saves in batches: either the entire batch
or nothing at all. So what if it saves half a batch and fails halfway. Does it
know to not send the first half again or does the db need to support idempotent
operations.
I do remember reading on one of the trident t
Hello!
In the script used for deploying storm to cloud(
https://github.com/nathanmarz/storm-deploy ) there is a dependency on
ZeroMQ. If I would like to use the latest version of Storm (
https://github.com/apache/incubator-storm) with Netty support, I think this
dependency should be removed.
Am
15 matches
Mail list logo
