L. C. Hsieh created SPARK-38593:
-----------------------------------
Summary: Incorporate numRowsDroppedByWatermark metric from
SessionWindowStateStoreRestoreExec into StateOperatorProgress
Key: SPARK-38593
URL: https://issues.apache.org/jira/browse/SPARK-38593
Project: Spark
Issue Type: Improvement
Components: Structured Streaming
Affects Versions: 3.3.0
Reporter: L. C. Hsieh
Although we added `numRowsDroppedByWatermark` to
`SessionWindowStateStoreRestoreExec`, but currently only `StateStoreWriter`
will be collected metrics for `StateOperatorProgress`. So if we need
`numRowsDroppedByWatermark` from `SessionWindowStateStoreRestoreExec` to be
used in streaming listener, we need to incorporate
`SessionWindowStateStoreRestoreExec` into `StateOperatorProgress`.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
