Grant Henke created FLUME-2983:
----------------------------------
Summary: Handle offset migration in the new Kafka Source
Key: FLUME-2983
URL: https://issues.apache.org/jira/browse/FLUME-2983
Project: Flume
Issue Type: Bug
Affects Versions: v1.7.0
Reporter: Grant Henke
Assignee: Grant Henke
Similar to FLUME-2972, Offsets tracking the position in Kafka consumers change
from using zookeeper for offset storage to Kafka when moving from 0.8.x to
0.9.x.
FLUME-2821 makes the client change in the Kafka Source but does not ensure
existing offsets get migrated in order to continue consuming where it left off.
Flume should have some automated logic on startup to check if Kafka offsets
exist, if not and migration is enabled (by default) then copy the offsets from
Zookeeper and commit them to Kafka.
This change should also fix the backwards incompatibility caused by removing
the zookeeperConnect property. The bootstrap can be looked up if
zookeeperConnect is used.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
