Yes, Akhil. We already have an origination timestamp in the body of the message when we send it. But we can't guarantee the network speed nor a precise enough synchronization of clocks across machines.
Pulling the timestamp from Kafka itself would be a step forward although the broker is most likely going to be not collocated with the consumers, so this would still be imprecise. > On Jun 23, 2015, at 3:46 AM, Akhil Das <ak...@sigmoidanalytics.com> wrote: > > May be while producing the messages, you can make it as a keyedMessage with > the timestamp as key and on the consumer end you can easily identify the key > (which will be the timestamp) from the message. If the network is fast > enough, then i think there could be a small millisecond lag. > > Thanks > Best Regards > >> On Tue, Jun 23, 2015 at 10:22 AM, dgoldenberg <dgoldenberg...@gmail.com> >> wrote: >> Is there any way to retrieve the time of each message's arrival into a Kafka >> topic, when streaming in Spark, whether with receiver-based or direct >> streaming? >> >> Thanks. >> >> >> >> -- >> View this message in context: >> http://apache-spark-user-list.1001560.n3.nabble.com/Any-way-to-retrieve-time-of-message-arrival-to-Kafka-topic-in-Spark-Streaming-tp23442.html >> Sent from the Apache Spark User List mailing list archive at Nabble.com. >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org >> For additional commands, e-mail: user-h...@spark.apache.org >
