[ https://issues.apache.org/jira/browse/SPARK-20717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tathagata Das updated SPARK-20717: ---------------------------------- Description: Timeout and state data are two independent entities and should be settable independently. Therefore, in the same call of the user-defined function, one should be able to set the timeout before initializing the state and also after removing the state. Whether timeouts can be set or not should not depend on the current state, and vice versa. However, a limitation of the current implementation is that state cannot be null while timeout is set. This is checked lazily after the function call has completed. was: Timeout and state data are two independent entities and should be settable independently. Therefore, - In the same call of the user-defined function, one should be able to set the timeout before initializing the state. - Removing the state should not reset timeouts. However, a limitation of the current implementation is that state cannot be null while timeout is set by the end of the function call is over. We should check this lazily. > Tweak MapGroupsWithState update function behavior > ------------------------------------------------- > > Key: SPARK-20717 > URL: https://issues.apache.org/jira/browse/SPARK-20717 > Project: Spark > Issue Type: Sub-task > Components: Structured Streaming > Affects Versions: 2.2.0 > Reporter: Tathagata Das > Assignee: Tathagata Das > Fix For: 2.2.0 > > > Timeout and state data are two independent entities and should be settable > independently. Therefore, in the same call of the user-defined function, one > should be able to set the timeout before initializing the state and also > after removing the state. Whether timeouts can be set or not should not > depend on the current state, and vice versa. > However, a limitation of the current implementation is that state cannot be > null while timeout is set. This is checked lazily after the function call has > completed. -- This message was sent by Atlassian JIRA (v6.3.15#6346) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org