[ https://issues.apache.org/jira/browse/SPARK-22278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tathagata Das resolved SPARK-22278. ----------------------------------- Resolution: Fixed Fix Version/s: (was: 2.2.0) 3.0.0 Issue resolved by pull request 19495 [https://github.com/apache/spark/pull/19495] > Expose current event time watermark and current processing time in GroupState > ----------------------------------------------------------------------------- > > Key: SPARK-22278 > URL: https://issues.apache.org/jira/browse/SPARK-22278 > Project: Spark > Issue Type: Sub-task > Components: Structured Streaming > Affects Versions: 2.2.0 > Reporter: Tathagata Das > Assignee: Tathagata Das > Fix For: 3.0.0 > > > Complex state-updating and/or timeout-handling logic in mapGroupsWithState > functions may require taking decisions based on the current event-time > watermark and/or processing-time. Currently, you can use the sql function > `current_timestamp` to get the current processing time, but it needs to > passed inserted in every row with a select, and then passed through the > encoder, which isnt efficient. Furthermore, there is no way to get the > current watermark. > This JIRA is to expose them through the GroupState API. -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org