[ 
https://issues.apache.org/jira/browse/STORM-971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Abhishek Agarwal updated STORM-971:
-----------------------------------
    Summary: Storm-Kafka: Emit metric for messages lost due to kafka retention  
(was: Storm-Kafka: Emit metric for failed messages which are not retried )

> Storm-Kafka: Emit metric for messages lost due to kafka retention
> -----------------------------------------------------------------
>
>                 Key: STORM-971
>                 URL: https://issues.apache.org/jira/browse/STORM-971
>             Project: Apache Storm
>          Issue Type: Improvement
>          Components: storm-kafka
>            Reporter: Abhishek Agarwal
>            Assignee: Abhishek Agarwal
>
> {noformat}
> if (offset < _emittedToOffset - _spoutConfig.maxOffsetBehind) {
>             LOG.info(
>                     "Skipping failed tuple at offset=" + offset +
>                             " because it's more than maxOffsetBehind=" + 
> _spoutConfig.maxOffsetBehind +
>                             " behind _emittedToOffset=" + _emittedToOffset
>             );
>         }
> {noformat}
> I believe, having a count metric for skipped messages will help configure the 
> correct maxOffsetBehind for the topology. Since the skipped messages could 
> potentially be lost, number is important for the user



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to