Xingcan Cui created SPARK-31838: ----------------------------------- Summary: The streaming output mode validator didn't treat union well Key: SPARK-31838 URL: https://issues.apache.org/jira/browse/SPARK-31838 Project: Spark Issue Type: Bug Components: Structured Streaming Affects Versions: 3.0.0 Reporter: Xingcan Cui
{{UnsupportedOperationChecker.checkForStreaming}} didn't treat the union operator well (it only assumes a linear query plan). To temporarily fix the broken semantics before we have a complete improvement plan for the output mode, the validator should do the following. 1. Allow multiple (two for now) aggregations from different branches of the last union operator. 2. For complete output mode, disallow a union on streams with and without aggregations. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org