[
https://issues.apache.org/jira/browse/APEXMALHAR-2459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15956619#comment-15956619
]
ASF GitHub Bot commented on APEXMALHAR-2459:
--------------------------------------------
GitHub user chaithu14 opened a pull request:
https://github.com/apache/apex-malhar/pull/600
APEXMALHAR-2459 1)Refactor the existing Kafka Input Operator. 2)Added the
support of KafkaInputOperator using 0.10 consumer API
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/chaithu14/incubator-apex-malhar
APEXMALHAR-2459-Kafkainput
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/apex-malhar/pull/600.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #600
----
----
> KafkaInputoperator using 0.10.* Kafka consumer API
> --------------------------------------------------
>
> Key: APEXMALHAR-2459
> URL: https://issues.apache.org/jira/browse/APEXMALHAR-2459
> Project: Apache Apex Malhar
> Issue Type: Task
> Reporter: Chaitanya
> Assignee: Chaitanya
>
> Changes in 0.10.0.0 version of Kafka
> -------------------------------------------------
> Message format in 0.10.0 has changed. Messages now include a time-stamp
> field and compressed messages include relative offsets.
>
> Please refer the below link about the changes in Kafka 0.10.0.0 version:
> https://kafka.apache.org/documentation/#upgrade_10_breaking.
> Design
> ---------
> I'm proposing the below design for new Kafka connectors by refactoring the
> existing Kafka connector(0.9.* version)
> 1) Split up the malhar-kafka to malhar-kafka-common, malhar-kafka_0.9,
> malhar-kafka_0.10. Directory structure would be as follows:
> malhar-kafka
> |--malhar-kafka-common
> |--malhar-kafka_0.9
> |--malhar-kafka_0.10
> If possible, will add the connectors for 0.8.x version.
> 2) Convert the KafkaConsumerWrapper to AbstractKafkaConsumerWrapper by adding
> the abstract methods over the consumer API's.
> 3) "malhar-kafka-common" package consists of the abstract classes and by
> default it builds over the 0.9.x version of Kafka.
> 4) "malhar-kafka-0.*" consists of concrete classes and its corresponding
> KafkaConsumerWrapper.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
