Github user arunmahadevan commented on a diff in the pull request: https://github.com/apache/spark/pull/21469#discussion_r194480087 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/HDFSBackedStateStoreProvider.scala --- @@ -247,6 +253,14 @@ private[state] class HDFSBackedStateStoreProvider extends StateStoreProvider wit private lazy val fm = CheckpointFileManager.create(baseDir, hadoopConf) private lazy val sparkConf = Option(SparkEnv.get).map(_.conf).getOrElse(new SparkConf) + private lazy val metricProviderLoaderMapSizeBytes: StateStoreCustomSizeMetric = + StateStoreCustomSizeMetric("providerLoadedMapSizeBytes", + "estimated size of states cache in provider") + + private lazy val metricProviderLoaderCountOfVersionsInMap: StateStoreCustomAverageMetric = --- End diff -- Why is "metricProviderLoaderCountOfVersionsInMap" an average metrics? The other metrics like "numTotalStateRows" and even "providerLoadedMapSizeBytes" is count metric. Shouldn't this be similar?
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org