[
https://issues.apache.org/jira/browse/FLUME-2848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15221124#comment-15221124
]
Li Ye commented on FLUME-2848:
------------------------------
The optinal properties topicHeader, keyHeader, timestampHeader are also helpful
for performance of Kafka Sink.
> Kafka Source could not work with Kafka Sink
> -------------------------------------------
>
> Key: FLUME-2848
> URL: https://issues.apache.org/jira/browse/FLUME-2848
> Project: Flume
> Issue Type: Bug
> Components: Sinks+Sources
> Affects Versions: v1.6.0
> Environment: IMHO any
> Reporter: Arkadiusz Firus
> Labels: easyfix
> Attachments: FLUME-2789-1.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> KafkaSource stores information from which topic a record was read. This
> information is stored in an event header with the key "topic" (see
> org.apache.flume.source.kafka.KafkaSource line 108).
> KafkaSink can be instructed to store record in a different topic than the one
> in configuration (dynamic destination). To do that "topic" header have to be
> set in an event (see org.apache.flume.sink.kafka.KafkaSink line 107).
> When this two features are combined we have infinite loop - KafkaSink writes
> records to the topic which is read by KafkaSource.
> Two solve the problem fields:
> org.apache.flume.sink.kafka.KafkaSink.TOPIC_HDR
> org.apache.flume.source.kafka.KafkaSourceConstants.TOPIC
> need to have different values.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
