Tathagata Das created SPARK-35800:
-------------------------------------
Summary: Improving testability of GroupState in streaming
flatMapGroupsWithState
Key: SPARK-35800
URL: https://issues.apache.org/jira/browse/SPARK-35800
Project: Spark
Issue Type: New Feature
Components: Structured Streaming
Affects Versions: 3.1.2
Reporter: Tathagata Das
GroupStateImpl is the internal implementation of the GroupState interface which
mean to be not exposed. Thus, it only has a private constructor. Such access
control does benefit encapsulation, however, this introduces difficulties for
unit tests and the users are calling the engine to construct such GroupState
instances in order to test their customized state transition functions.
The solution is to introduce new interfaces that allow users to create
instances of GroupState but also access internal values of what they have set
(for example, has to state been updated, or removed). This would allow them to
write unit tests of the state transition function with custom GroupState
objects and then verifying whether the state was updated in an expected way.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
