Re: [Beginner] Kafka 0.11 header support in Spark Structured Streaming

[Tathagata Das]

I made a JIRA for it - https://issues.apache.org/jira/browse/SPARK-23539
Unfortunately it is blocked by Kafka version upgrade, which has a few nasty
issues related to Kafka bugs -
https://issues.apache.org/jira/browse/SPARK-18057

On Wed, Feb 28, 2018 at 3:17 PM, karthikus  wrote:

> TD,
>
> Thanks for your response.
>
>
>
> --
> Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/
>
> -
> To unsubscribe e-mail: user-unsubscr...@spark.apache.org
>
>

Re: [Beginner] Kafka 0.11 header support in Spark Structured Streaming

[karthikus]

TD,

Thanks for your response. 



--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/

-
To unsubscribe e-mail: user-unsubscr...@spark.apache.org

Re: [Beginner] Kafka 0.11 header support in Spark Structured Streaming

[Tathagata Das]

Unfortunately, exposing Kafka headers is not yet supported in Structured
Streaming. The community is more than welcome to add support for it :)

On Tue, Feb 27, 2018 at 2:51 PM, Karthik Jayaraman 
wrote:

> Hi all,
>
> I am using Spark 2.2.1 Structured Streaming to read messages from Kafka. I
> would like to know how to access the Kafka headers programmatically ? Since
> the Kafka message header support is introduced in Kafka 0.11 (
> https://issues.apache.org/jira/browse/KAFKA-4208), is it supported in
> Spark. ? If yes, can anyone point me to an example ?
>
> - Karthik
>

[Beginner] Kafka 0.11 header support in Spark Structured Streaming

[Karthik Jayaraman]

Hi all,

I am using Spark 2.2.1 Structured Streaming to read messages from Kafka. I 
would like to know how to access the Kafka headers programmatically ? Since the 
Kafka message header support is introduced in Kafka 0.11 
(https://issues.apache.org/jira/browse/KAFKA-4208 
), is it supported in Spark. 
? If yes, can anyone point me to an example ?

- Karthik