Hello Sam, We have just back ported this fix into the 1.1 and 1.0 branch now: https://github.com/apache/kafka/pull/5277
The up coming 1.1.1 and 1.0.2 release should have this issue fixed. Guozhang On Thu, Jun 21, 2018 at 2:36 PM, Sam Lendle <slen...@pandora.com> wrote: > Yep that’s the change I saw. It does look like the issue should be fixed > since the implementation of update for Count always adds 1 regardless of > the value. > > Thanks for the followup. > > On 6/21/18, 1:15 PM, "John Roesler" <j...@confluent.io> wrote: > > Hi Sam, > > This sounds like a condition I fixed in > https://urldefense.proofpoint.com/v2/url?u=https-3A__github. > com_apache_kafka_commit_ed51b2cdf5bdac210a6904bead1a2ca6e8411406-23diff- > 2D8b364ed2d0abd8e8ae21f5d322db6564R221&d=DwIFaQ&c= > gFTBenQ7Vj71sUi1A4CkFnmPzqwDo07QsHw-JRepxyw&r= > BNCekDhngyXB6C2Ag7PIfHotiuqjAVwLOZLQHB7fyOM&m=q-lWaxuP-yHvVqew6oPVJiQPM_ > xal1kSHyqFZKeXFe4&s=fHXuRWDvMLDx75JEh_h4Y50zooJ9Z03d77TnX4tnxzE&e= > . I realized that the prior code creates a new Meter, which uses a > Total > metric instead of a Count. But that would total all the values of the > metric, when instead what we want is to "total" the number of > measurements > (aka count them). > > I just peeked at the 1.1 branch, and it seems this change made it in > after > the 1.1 branch cut, so it would only be fixed in 2.0. > > Thanks, > -John > > On Wed, Jun 20, 2018 at 8:44 PM Guozhang Wang <wangg...@gmail.com> > wrote: > > > Thanks for reporting this Sam, could you check and confirm if this > issue is > > fixed in trunk? If not, we should file a JIRA. > > > > > > Guozhang > > > > On Wed, Jun 20, 2018 at 6:41 PM, Sam Lendle <slen...@pandora.com> > wrote: > > > > > It looks like there is indeed a bug in kafka-streams 1.1.0. I > think what > > > was happening was the time spent processing each record in ns was > being > > > added to the total metric instead of incrementing by 1 for each > record. > > > Looks like the implementation has been changed in trunk. I don't > see any > > > commit messages mentioning this particular issue, but hopefully the > > change > > > fixes it. > > > ------------------------------ > > > *From:* Sam Lendle > > > *Sent:* Wednesday, June 20, 2018 6:10:03 PM > > > *To:* users@kafka.apache.org > > > *Subject:* Some Total and Rate metrics are not consistent > > > > > > > > > I’m trying to use the total metrics introduced in KIP-187 ( > > > https://urldefense.proofpoint.com/v2/url?u=https-3A__cwiki. > apache.org_confluence_display_KAFKA_KIP-2D&d=DwIFaQ&c= > gFTBenQ7Vj71sUi1A4CkFnmPzqwDo07QsHw-JRepxyw&r= > BNCekDhngyXB6C2Ag7PIfHotiuqjAVwLOZLQHB7fyOM&m=q-lWaxuP-yHvVqew6oPVJiQPM_ > xal1kSHyqFZKeXFe4&s=jYa-lA5HwT7eOoFDIhz215N6PbZerIuO9AQDqLpNYlQ&e= > > > 187+-+Add+cumulative+count+metric+for+all+Kafka+rate+metrics) > > > > > > > > > > > > For some metrics, the total and rates are not consistent. In > particular, > > > for stream-processor-node-metrics, I’m seeing about 500-800 > operations > > per > > > second in a particular streams thread/processor node as reported > by the > > > process-rate metric, but the process-total metric is increasing by > about > > > 100 million per second. See attached screenshot from VisualVM. > > > > > > > > > > > > Other metrics seem fine, for example forward-rate and forward-total > > > metrics under stream-processor-node-metrics are consistent. > > > > > > > > > > > > Am I misunderstanding the interpretation of the –total metrics? If > this > > is > > > a bug, can I do anything in addition to this email to report it? > File a > > > JIRA? > > > > > > > > > Best, > > > Sam > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > -- Guozhang > > > > > -- -- Guozhang
