No I just would like to say it's not strange.
What you need to know about is that there could be some additional latency
when your Spout spends some moments in nextTuple(), ack(), fail(). KafkaSpout
reads the data from Kafka so I assume there's execution time in nextTuple()
on KafkaSpout.
Apache S
We're using the out of the box kafka spout, are there any issues related to
that here?
> One thing you would like to check is, event handler on Spout is single
thread, which means that same thread calls nextTuple() and ack() and
fail().
I'm not following you here. What would I be checking here?
Oh right sorry it's introduced at Storm 1.0.0.
And max-spout-pending controls the call of nextTuple() so tuple latency
shouldn't be affected.
One thing you would like to check is, event handler on Spout is single
thread, which means that same thread calls nextTuple() and ack() and
fail().
http://
Yes it appears that the skipped-max-spout metric is only in 1.0.0 as part
of the automatic back pressure fixes (
https://issues.apache.org/jira/browse/STORM-886)
On Tue, Apr 19, 2016 at 9:21 PM, Kevin Conaway
wrote:
> We are already sending our metrics to graphite but I don't see
> __skipped-max
Thanks John. I had tried profiling locally with yourkit but my
recollection was that almost all of the time was spent in
backtype.storm.timer sleep which didn't smell right to me.
On Tue, Apr 19, 2016 at 8:43 PM, wrote:
> One thing you can do is profile a worker process with jvisualvm to see
>
We are already sending our metrics to graphite but I don't see
__skipped-max-spout being logged. Was that added after 0.10? I don't even
see a reference to it in the codebase.
We are capturing the queue metrics for each component (send_queue and
receive_queue) and the population for each is very
Hi Kevin,
You can attach metrics consumer to log additional informations for that
topology like disruptor queue metrics, __skipped-max-spout for spout, and
etc.
Please refer
https://github.com/apache/storm/blob/master/storm-core/src/jvm/org/apache/storm/metric/LoggingMetricsConsumer.java
to
how to
One thing you can do is profile a worker process with jvisualvm to see what
time is spent where for each executor thread as well as the netty and lmax
layers.
--John
Sent from my iPhone
> On Apr 19, 2016, at 8:41 PM, Kevin Conaway wrote:
>
> In Storm 0.10, is there a way to monitor the _maxS
In Storm 0.10, is there a way to monitor the _maxSpoutPending_ value? I
don' see it exposed in any of the metrics that storm publishes but I'd like
to be able to see how much time each tuple spends waiting and how big the
pending queue size is for each spout task. Is this possible?
Our topology
