[
https://issues.apache.org/jira/browse/FLUME-2852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15068966#comment-15068966
]
Jeff Holoman commented on FLUME-2852:
-------------------------------------
Tristan,
This seems like a reasonable feature request to me. [~gherreros], Kafka doesn't
have the concept of headers, beyond a single key. What Tristan is proposing is
to extract the key if necessary, and encode the entire Event as an
AvroFlumeEvent. In this way, headers could be persisted across clusters.
I would suggest we hold off on the request, however, until we complete the
update for Kafka 0.9.
> Kafka Source/Sink should optionally read/write Avro Datums
> ----------------------------------------------------------
>
> Key: FLUME-2852
> URL: https://issues.apache.org/jira/browse/FLUME-2852
> Project: Flume
> Issue Type: Improvement
> Components: Sinks+Sources
> Affects Versions: v1.6.0
> Reporter: Tristan Stevens
> Assignee: Jeff Holoman
>
> Currently the Kafka Sink only writes the event body to Kafka rather than an
> Avro Datum. This works fine when being used with a Kafka Source, or when
> being used with Kafka Channel, however it does mean that any Flume headers
> are lost when transported via Kafka.
> Request is to implement an equivalent of the Kafka Channel's
> parseAsFlumeEvent parameter to the sink/source so that optionally Avro Datums
> can be read from and written to Kafka.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
