[
https://issues.apache.org/jira/browse/METRON-1238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16197007#comment-16197007
]
Simon Elliston Ball commented on METRON-1238:
---------------------------------------------
While this is potentially an issue in Kafka / Storm, I suspect the likely cause
is that you need to, for some reason reset kafka offsets when processing has
lagged to far.
Instead of reverting to EARLIEST, you can use my very hacky tool
https://github.com/simonellistonball/kafka-utils to manipulate offsets for the
relevant consumer group. Note this will be superseded by a much cleaner version
in Kafka 0.11.
See
https://cwiki.apache.org/confluence/display/KAFKA/KIP-122%3A+Add+Reset+Consumer+Group+Offsets+tooling
and https://issues.apache.org/jira/browse/KAFKA-4743
for details on the proper way.
> Enrichment topology start at EARLIEST again after de-activating topology
> ------------------------------------------------------------------------
>
> Key: METRON-1238
> URL: https://issues.apache.org/jira/browse/METRON-1238
> Project: Metron
> Issue Type: Bug
> Affects Versions: 0.4.1
> Environment: 0.4.1.3.0.0
> Reporter: Jasper Knulst
>
> When the enrichment topology is started with the firstPollStrategy EARLIEST
> and then de-activate the topology from the Storm UI, the kafkaspout begins
> consuming again from EARLIEST when the topology is activated again.
> To reproduce:
> -Start enrichment with firstPollOffsetStrategy set to EARLIEST
> -De-activate topology (Storm UI) after a while (check topology offset)
> -Wait
> -Activate topology again
> -Check topology offset
> I think the Kafkaspouts are reinstantiated when you do this and the spout
> config is hanging somewhere in its original state
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
