[
https://issues.apache.org/jira/browse/SAMZA-503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14279233#comment-14279233
]
Yan Fang commented on SAMZA-503:
--------------------------------
{quote}
I guess it may have taken 10mins to fill up the BroxyProxy cache in my test
{quote}
Here, i am a little confusing as well. [~criccomini], how does the cache work
here? I see the threshold is set to 50000. Does that mean the BrokerProxy
fetches data as long as it has lower than 50000 messages? Or it only fetches
data when all the 50000 msgs are consumed? From the codebase, I think the
answer is the former one. But then it does not explain why it takes so long to
refresh the gauge. Maybe I miss something.
{quote}
Maybe it should be "prefetch-messages-behind-high-watermark" and the other lag
metric would be "commited-messages-behind-high-watermark".
{quote}
Yeah, if my understanding is correct, we may want to rename them to make it
more clear.
> Lag gauge very slow to update for slow jobs
> -------------------------------------------
>
> Key: SAMZA-503
> URL: https://issues.apache.org/jira/browse/SAMZA-503
> Project: Samza
> Issue Type: Bug
> Components: metrics
> Affects Versions: 0.8.0
> Environment: Mac OS X, Oracle Java 7, ProcessJobFactory
> Reporter: Roger Hoover
> Assignee: Yan Fang
> Fix For: 0.9.0
>
>
> For slow jobs, the
> KafkaSystemConsumerMetrics.%s-%s-messages-behind-high-watermark) gauge does
> not get updated very often.
> To reproduce:
> * Create a job that processes one message and sleeps for 5 seconds
> * Create it's input topic but do not populate it yet
> * Start the job
> * Load 1000s of messages to it's input topic. You can keep adding messages
> with a "wait -n 1 <kafka console producer command>"
> What happens:
> * Run jconsole to view the JMX metrics
> * The %s-%s-messages-behind-high-watermark gauge will stay at 0 for a LONG
> time (~10 minutes?) before finally updating.
> What should happen:
> * The gauge should get updated at a reasonable interval (a least every few
> seconds)
> I think what's happening is that the BrokerProxy only updates the high
> watermark when a consumer is ready for more messages. When the job is so
> slow, this rarely happens to the metric doesn't get updated.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
