[
https://issues.apache.org/jira/browse/FLUME-2983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15454127#comment-15454127
]
Hudson commented on FLUME-2983:
-------------------------------
FAILURE: Integrated in Jenkins build Flume-trunk-hbase-1 #202 (See
[https://builds.apache.org/job/Flume-trunk-hbase-1/202/])
FLUME-2983. Handle offset migration in the new Kafka Source (mpercy:
[http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=330e572879c9903f5b1084d84b237685aa9970c6])
* (edit)
flume-ng-channels/flume-kafka-channel/src/test/java/org/apache/flume/channel/kafka/TestKafkaChannel.java
* (edit)
flume-ng-sources/flume-kafka-source/src/main/java/org/apache/flume/source/kafka/KafkaSource.java
* (edit)
flume-ng-channels/flume-kafka-channel/src/main/java/org/apache/flume/channel/kafka/KafkaChannel.java
* (edit) flume-ng-sources/flume-kafka-source/pom.xml
* (edit) flume-ng-doc/sphinx/FlumeUserGuide.rst
* (edit)
flume-ng-sources/flume-kafka-source/src/main/java/org/apache/flume/source/kafka/KafkaSourceConstants.java
* (edit)
flume-ng-sources/flume-kafka-source/src/test/java/org/apache/flume/source/kafka/TestKafkaSource.java
* (edit)
flume-ng-sources/flume-kafka-source/src/test/java/org/apache/flume/source/kafka/KafkaSourceEmbeddedKafka.java
> Handle offset migration in the new Kafka Source
> -----------------------------------------------
>
> Key: FLUME-2983
> URL: https://issues.apache.org/jira/browse/FLUME-2983
> Project: Flume
> Issue Type: Bug
> Affects Versions: v1.7.0
> Reporter: Grant Henke
> Assignee: Grant Henke
> Fix For: v1.7.0
>
>
> Similar to FLUME-2972, Offsets tracking the position in Kafka consumers
> change from using zookeeper for offset storage to Kafka when moving from
> 0.8.x to 0.9.x.
> FLUME-2821 makes the client change in the Kafka Source but does not ensure
> existing offsets get migrated in order to continue consuming where it left
> off.
> Flume should have some automated logic on startup to check if Kafka offsets
> exist, if not and migration is enabled (by default) then copy the offsets
> from Zookeeper and commit them to Kafka.
> This change should also fix the backwards incompatibility caused by removing
> the zookeeperConnect property. The bootstrap can be looked up if
> zookeeperConnect is used.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
