Hi All,

I am getting following error message while applying
*flatMapGroupsWithState.*

*Exception in thread "main" org.apache.spark.sql.AnalysisException:
flatMapGroupsWithState in update mode is not supported with aggregation on
a streaming DataFrame/Dataset;;*

Following is what I am trying to do.

- Read messages from Kafka & Parse It
- Group based on certain dimensions
- Ran UDAF for every group and calculated aggregation for each group. Agg
doesn't return KeyValueGroupDataSet. So applying groupByKey on previous
step output to group based on aggFunction column
- Merge this aggregates to previous state of stream using
*flatMapGroupsWithState.*

*Getting error message for the last step. *

*Does this error means I cannot apply **flatMapGroupsWithState after
applying agg(....) on dataset?*

*Regards*
*Sandeep*

Reply via email to