[
https://issues.apache.org/jira/browse/BEAM-4038?focusedWorklogId=92969&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-92969
]
ASF GitHub Bot logged work on BEAM-4038:
----------------------------------------
Author: ASF GitHub Bot
Created on: 20/Apr/18 00:34
Start Date: 20/Apr/18 00:34
Worklog Time Spent: 10m
Work Description: gkumar7 commented on a change in pull request #5111:
[BEAM-4038] Support Kafka Headers in KafkaIO
URL: https://github.com/apache/beam/pull/5111#discussion_r182916997
##########
File path:
sdks/java/io/kafka/src/main/java/org/apache/beam/sdk/io/kafka/KafkaRecordCoder.java
##########
@@ -66,9 +76,34 @@ public void encode(KafkaRecord<K, V> value, OutputStream
outStream) throws IOExc
longCoder.decode(inStream),
longCoder.decode(inStream),
KafkaTimestampType.forOrdinal(intCoder.decode(inStream)),
+ (Headers) toHeaders(headerCoder.decode(inStream)),
kvCoder.decode(inStream));
}
+ private Object toHeaders(Iterable<KV<String, byte[]>> records) {
+ if (!records.iterator().hasNext()){
+ return null;
+ }
+
+ // ConsumerRecord is used to simply create a list of headers
+ ConsumerRecord<String, String> consumerRecord = new ConsumerRecord<>(
Review comment:
```Headers headers = ... ```
raises a ```ClassNotFoundException``` when serializing
```KafkaRecordCoder```.
Updated to use empty strings instead of "key", "value"
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 92969)
Time Spent: 7h 40m (was: 7.5h)
> Support Kafka Headers in KafkaIO
> --------------------------------
>
> Key: BEAM-4038
> URL: https://issues.apache.org/jira/browse/BEAM-4038
> Project: Beam
> Issue Type: New Feature
> Components: io-java-kafka
> Reporter: Geet Kumar
> Assignee: Raghu Angadi
> Priority: Minor
> Time Spent: 7h 40m
> Remaining Estimate: 0h
>
> Headers have been added to Kafka Consumer/Producer records (KAFKA-4208). The
> purpose of this JIRA is to support this feature in KafkaIO.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
