subject:"\[Structured Streaming\] Query Metrics to MetricsSink"

Re: [Structured Streaming] Query Metrics to MetricsSink

2018-03-20 Thread lucas-vsco

It actually looks like I might have the answers via these following links: [Design] Metrics in Structured Streaming JIRA - Structured Streaming - Metrics

[Structured Streaming] Query Metrics to MetricsSink

2018-03-20 Thread lucas-vsco

I am looking to take the metrics exposed in the logs from MicroBatchExecution below and submit them as stats to implemented MetricsSinks. 2018-03-20 10:28:48 INFO MicroBatchExecution:54 - Streaming query made progress: { "id" : "42bb5c95-980d-480d-9dee-72e1baf6a5b3", "runId" :