Just filed a new JIRA issue [1] as well as PR [2]. - Jungtaek Lim (HeartSaVioR)
1. https://issues.apache.org/jira/browse/SPARK-26170 2. https://github.com/apache/spark/pull/23142 2018년 11월 26일 (월) 오후 3:26, Jungtaek Lim <kabh...@gmail.com>님이 작성: > Looks like just a kind of missing spot. Just crafted the patch and now in > progress of testing. > > Once done with testing I'll file a new issue and submit a patch. > > Thanks, > Jungtaek Lim (HeartSaVioR) > > 2018년 11월 26일 (월) 오전 4:49, Burak Yavuz <brk...@gmail.com>님이 작성: > >> Probably just oversight. Anyone is welcome to add it :) >> >> On Sun, Nov 25, 2018 at 8:55 AM Jacek Laskowski <ja...@japila.pl> wrote: >> >>> Hi, >>> >>> Why is FlatMapGroupsWithStateExec not measuring the time taken on state >>> commit [1](like StreamingDeduplicateExec [2] and StreamingGlobalLimitExec >>> [3])? Is this on purpose? Why? Thanks. >>> >>> [1] >>> https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/FlatMapGroupsWithStateExec.scala#L135 >>> >>> [2] >>> https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/statefulOperators.scala#L482 >>> >>> [3] >>> https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamingGlobalLimitExec.scala#L87 >>> >>> Pozdrawiam, >>> Jacek Laskowski >>> ---- >>> https://about.me/JacekLaskowski >>> Mastering Spark SQL https://bit.ly/mastering-spark-sql >>> Spark Structured Streaming https://bit.ly/spark-structured-streaming >>> Mastering Kafka Streams https://bit.ly/mastering-kafka-streams >>> Follow me at https://twitter.com/jaceklaskowski >>> >>
