[
https://issues.apache.org/jira/browse/FLUME-2983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15453996#comment-15453996
]
Mike Percy commented on FLUME-2983:
-----------------------------------
+1. I am about to commit this.
> Handle offset migration in the new Kafka Source
> -----------------------------------------------
>
> Key: FLUME-2983
> URL: https://issues.apache.org/jira/browse/FLUME-2983
> Project: Flume
> Issue Type: Bug
> Affects Versions: v1.7.0
> Reporter: Grant Henke
> Assignee: Grant Henke
>
> Similar to FLUME-2972, Offsets tracking the position in Kafka consumers
> change from using zookeeper for offset storage to Kafka when moving from
> 0.8.x to 0.9.x.
> FLUME-2821 makes the client change in the Kafka Source but does not ensure
> existing offsets get migrated in order to continue consuming where it left
> off.
> Flume should have some automated logic on startup to check if Kafka offsets
> exist, if not and migration is enabled (by default) then copy the offsets
> from Zookeeper and commit them to Kafka.
> This change should also fix the backwards incompatibility caused by removing
> the zookeeperConnect property. The bootstrap can be looked up if
> zookeeperConnect is used.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
