Looks like just a kind of missing spot. Just crafted the patch and now in progress of testing.
Once done with testing I'll file a new issue and submit a patch. Thanks, Jungtaek Lim (HeartSaVioR) 2018년 11월 26일 (월) 오전 4:49, Burak Yavuz <brk...@gmail.com>님이 작성: > Probably just oversight. Anyone is welcome to add it :) > > On Sun, Nov 25, 2018 at 8:55 AM Jacek Laskowski <ja...@japila.pl> wrote: > >> Hi, >> >> Why is FlatMapGroupsWithStateExec not measuring the time taken on state >> commit [1](like StreamingDeduplicateExec [2] and StreamingGlobalLimitExec >> [3])? Is this on purpose? Why? Thanks. >> >> [1] >> https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/FlatMapGroupsWithStateExec.scala#L135 >> >> [2] >> https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/statefulOperators.scala#L482 >> >> [3] >> https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamingGlobalLimitExec.scala#L87 >> >> Pozdrawiam, >> Jacek Laskowski >> ---- >> https://about.me/JacekLaskowski >> Mastering Spark SQL https://bit.ly/mastering-spark-sql >> Spark Structured Streaming https://bit.ly/spark-structured-streaming >> Mastering Kafka Streams https://bit.ly/mastering-kafka-streams >> Follow me at https://twitter.com/jaceklaskowski >> >
