[GitHub] [spark] HeartSaVioR edited a comment on issue #22282: [SPARK-23539][SS] Add support for Kafka headers in Structured Streaming

HeartSaVioR edited a comment on issue #22282: [SPARK-23539][SS] Add support for Kafka headers in Structured Streaming URL: https://github.com/apache/spark/pull/22282#issuecomment-523827720 The fix wouldn't work since KafkaOffsetReader will be shared across tasks in same JVM (since it's a s

[GitHub] [spark] HeartSaVioR edited a comment on issue #22282: [SPARK-23539][SS] Add support for Kafka headers in Structured Streaming

HeartSaVioR edited a comment on issue #22282: [SPARK-23539][SS] Add support for Kafka headers in Structured Streaming URL: https://github.com/apache/spark/pull/22282#issuecomment-523827720 The fix wouldn't work since KafkaOffsetReader will be shared across tasks in same JVM (since it's a s

[GitHub] [spark] HeartSaVioR edited a comment on issue #22282: [SPARK-23539][SS] Add support for Kafka headers in Structured Streaming

HeartSaVioR edited a comment on issue #22282: [SPARK-23539][SS] Add support for Kafka headers in Structured Streaming URL: https://github.com/apache/spark/pull/22282#issuecomment-523827720 The fix wouldn't work since KafkaOffsetReader will be shared across tasks (since it's a singleton `ob

[GitHub] [spark] HeartSaVioR edited a comment on issue #22282: [SPARK-23539][SS] Add support for Kafka headers in Structured Streaming

HeartSaVioR edited a comment on issue #22282: [SPARK-23539][SS] Add support for Kafka headers in Structured Streaming URL: https://github.com/apache/spark/pull/22282#issuecomment-523827720 The fix wouldn't work since KafkaOffsetReader will be shared across tasks (since it's a singleton `ob

[GitHub] [spark] HeartSaVioR edited a comment on issue #22282: [SPARK-23539][SS] Add support for Kafka headers in Structured Streaming

HeartSaVioR edited a comment on issue #22282: [SPARK-23539][SS] Add support for Kafka headers in Structured Streaming URL: https://github.com/apache/spark/pull/22282#issuecomment-514840592 Personally I would ask to rollback any irrelevant changes. Looks like the latest change contains some

[GitHub] [spark] HeartSaVioR edited a comment on issue #22282: [SPARK-23539][SS] Add support for Kafka headers in Structured Streaming

HeartSaVioR edited a comment on issue #22282: [SPARK-23539][SS] Add support for Kafka headers in Structured Streaming URL: https://github.com/apache/spark/pull/22282#issuecomment-514840592 Personally I would ask to rollback any irrelevant changes. Looks like the latest change contains some

[GitHub] [spark] HeartSaVioR edited a comment on issue #22282: [SPARK-23539][SS] Add support for Kafka headers in Structured Streaming

HeartSaVioR edited a comment on issue #22282: [SPARK-23539][SS] Add support for Kafka headers in Structured Streaming URL: https://github.com/apache/spark/pull/22282#issuecomment-514840592 Personally I would ask to rollback the latest commit and only address changing header from `Map` to `

[GitHub] [spark] HeartSaVioR edited a comment on issue #22282: [SPARK-23539][SS] Add support for Kafka headers in Structured Streaming

HeartSaVioR edited a comment on issue #22282: [SPARK-23539][SS] Add support for Kafka headers in Structured Streaming URL: https://github.com/apache/spark/pull/22282#issuecomment-513956104 Nice catch. Even for interface `Headers` it defines the behavior of iterable, and also defines the be