Turned out to not even be relevant to the problem we were having. It was a
bolt that hung periodically on processing a single tuple out of several
thousand tuples. When the bolt was removed, the processing flowed just fine.
On Wed, Dec 10, 2014 at 1:58 PM, P. Taylor Goetz wrote:
>
> Hi Mike,
>
>
Hi Mike,
Have you set a value for topology.max.spout.pending? If so what value are you
using?
For a core storm (spouts/bolts) topology that controls how many tuples can be
“in-flight.” For trident it controls how many batches can be outstanding.
-Taylor
On Dec 3, 2014, at 2:02 PM, Mike Thomse
What's your topology overall latency ? Looks like you have really slow bolts
On Wed, Dec 3, 2014 at 9:02 PM, Mike Thomsen wrote:
> I've found that our topology will fetch a large block of messages from
> kafka and wait about 14-15 minutes before going back to kafka for more
> messages. One sugge
I've found that our topology will fetch a large block of messages from
kafka and wait about 14-15 minutes before going back to kafka for more
messages. One suggestion I found on Stackoverflow suggested to increase
topology.spout.max.batch.size. Is this the right approach or is there
likely a differ
