When you send a tuple from the spout, it sets the message id in the acker.
When a message fans out, it did logical xor of new tuple ids with the
existing message id, and when tuples get acked their message id id xor'd
into the same value again. This fanning out tuples should be quite light
weight.
Hi Eran,
Have you checked storm UI metrics? Is there capacity overload?
Also please check log files for errors.
Best regards,
Dmytro Dragan
On May 14, 2015 08:45, "Eran Chinthaka Withana"
wrote:
> Hi Nathan
>
> No I still didn't try jstack
>
> But I'm just wondering whether this is something t
Hi Nathan
No I still didn't try jstack
But I'm just wondering whether this is something that could within storm
itself due to the amount of messages being emitted for one spout message.
Wondering whether the internal executor buffers or ackers are having
issues. Have you seen this before? Also, i
Did you try running jstack to see what your threads are doing?
On Wed, May 13, 2015 at 9:13 PM, Eran Chinthaka Withana <
eran.chinth...@gmail.com> wrote:
> Here is the complete config and topology builder code (written in scala):
> http://pastebin.com/xhJGDsXn
>
Here is the complete config and topology builder code (written in scala):
http://pastebin.com/xhJGDsXn
No they are not. I set a very high timeout. For testing it is,
conf.put(Config.TOPOLOGY_MESSAGE_TIMEOUT_SECS, Int.box(1 * 60 * 60))
Thanks,
Eran Chinthaka Withana
On Wed, May 13, 2015 at 6:03 PM, Nathan Leung wrote:
> Are your tuples from the spout timing out and being replayed? This could
> c
Are your tuples from the spout timing out and being replayed? This could
cause the topology to spin doing the same table/s over and over.
On May 13, 2015 8:12 PM, "Eran Chinthaka Withana"
wrote:
> Hi,
>
> Storm version: 0.9.2
>
> I'm running a topology where, based on an event, I try to synch one
Hi,
Storm version: 0.9.2
I'm running a topology where, based on an event, I try to synch one
database to the other. After a kafka spout (with db info in the message),
- the first bolt sends a tuple for each table in the db,
- the second bolt, reads from the given table and sends batches of rows f
