Re: Spark 2.3.0 Structured Streaming Kafka Timestamp

Hmm yeah that does look wrong. Would be great if someone opened a PR to correct the docs :) On Thu, May 10, 2018 at 5:13 PM Yuta Morisawa wrote: > The problem is solved. > The actual schema of Kafka message is different from documentation. > > >

Re: Spark 2.3.0 Structured Streaming Kafka Timestamp

The problem is solved. The actual schema of Kafka message is different from documentation. https://spark.apache.org/docs/latest/structured-streaming-kafka-integration.html The documentation says the format of "timestamp" column is Long type, but the actual format is timestamp. The

Spark 2.3.0 Structured Streaming Kafka Timestamp

Hi All I'm trying to extract Kafka-timestamp from Kafka topics. The timestamp does not contain milli-seconds information, but it should contain because ConsumerRecord class of Kafka 0.10 supports milli-second timestamp. How can I get milli-second timestamp from Kafka topics? These are