Re: [Beginner] Kafka 0.11 header support in Spark Structured Streaming

I made a JIRA for it - https://issues.apache.org/jira/browse/SPARK-23539 Unfortunately it is blocked by Kafka version upgrade, which has a few nasty issues related to Kafka bugs - https://issues.apache.org/jira/browse/SPARK-18057 On Wed, Feb 28, 2018 at 3:17 PM, karthikus

Re: [Beginner] Kafka 0.11 header support in Spark Structured Streaming

TD, Thanks for your response. -- Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/ - To unsubscribe e-mail: user-unsubscr...@spark.apache.org

Re: [Beginner] Kafka 0.11 header support in Spark Structured Streaming

Unfortunately, exposing Kafka headers is not yet supported in Structured Streaming. The community is more than welcome to add support for it :) On Tue, Feb 27, 2018 at 2:51 PM, Karthik Jayaraman wrote: > Hi all, > > I am using Spark 2.2.1 Structured Streaming to read

[Beginner] Kafka 0.11 header support in Spark Structured Streaming

Hi all, I am using Spark 2.2.1 Structured Streaming to read messages from Kafka. I would like to know how to access the Kafka headers programmatically ? Since the Kafka message header support is introduced in Kafka 0.11 (https://issues.apache.org/jira/browse/KAFKA-4208