[jira] [Assigned] (SPARK-47733) Add operational metrics for TWS operators
[ https://issues.apache.org/jira/browse/SPARK-47733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-47733: Assignee: Anish Shrigondekar (was: Jing Zhan) > Add operational metrics for TWS operators > - > > Key: SPARK-47733 > URL: https://issues.apache.org/jira/browse/SPARK-47733 > Project: Spark > Issue Type: Task > Components: Structured Streaming >Affects Versions: 4.0.0 >Reporter: Jing Zhan >Assignee: Anish Shrigondekar >Priority: Major > Labels: pull-request-available > > Add metrics to improve observability for the newly added operator > TransformWithState and some changes we've made into RocksDB. > Proposed metrics to add: > * on the RocksDB StateStore metrics side, we will add the following: > ** num external col families > ** num internal col families > * on the operator side, we will add the following: > ** number of state vars > ** count of state vars by type > ** output mode > ** timeout mode > ** registered timers in batch > ** expired timers in batch > ** initial state enabled or not > ** number of state vars removed in batch -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-47733) Add operational metrics for TWS operators
[ https://issues.apache.org/jira/browse/SPARK-47733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-47733: Assignee: Jing Zhan > Add operational metrics for TWS operators > - > > Key: SPARK-47733 > URL: https://issues.apache.org/jira/browse/SPARK-47733 > Project: Spark > Issue Type: Task > Components: Structured Streaming >Affects Versions: 4.0.0 >Reporter: Jing Zhan >Assignee: Jing Zhan >Priority: Major > Labels: pull-request-available > > Add metrics to improve observability for the newly added operator > TransformWithState and some changes we've made into RocksDB. > Proposed metrics to add: > * on the RocksDB StateStore metrics side, we will add the following: > ** num external col families > ** num internal col families > * on the operator side, we will add the following: > ** number of state vars > ** count of state vars by type > ** output mode > ** timeout mode > ** registered timers in batch > ** expired timers in batch > ** initial state enabled or not > ** number of state vars removed in batch -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
