Probably just oversight. Anyone is welcome to add it :) On Sun, Nov 25, 2018 at 8:55 AM Jacek Laskowski <ja...@japila.pl> wrote:
> Hi, > > Why is FlatMapGroupsWithStateExec not measuring the time taken on state > commit [1](like StreamingDeduplicateExec [2] and StreamingGlobalLimitExec > [3])? Is this on purpose? Why? Thanks. > > [1] > https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/FlatMapGroupsWithStateExec.scala#L135 > > [2] > https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/statefulOperators.scala#L482 > > [3] > https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamingGlobalLimitExec.scala#L87 > > Pozdrawiam, > Jacek Laskowski > ---- > https://about.me/JacekLaskowski > Mastering Spark SQL https://bit.ly/mastering-spark-sql > Spark Structured Streaming https://bit.ly/spark-structured-streaming > Mastering Kafka Streams https://bit.ly/mastering-kafka-streams > Follow me at https://twitter.com/jaceklaskowski >
