On Mon, 30 Mar 2020 at 05:08, Lu Niu <qqib...@gmail.com> wrote: > $current_processing - $event_time works for event time. How about > processing time? Is there a good way to measure the latency? >
To measure latency you'll need some way to determine the time spent between the start and end of your pipeline. To measure latency when using processing time, you'll need to partially use ingestion time. That is, you'll need to add the 'current' processing time as soon as messages are ingested. With it, you can then use the $current_processing - $ingest_time solution that was already mentioned. Kind regards, Oscar -- Oscar Westra van Holthe - Kind
